Fork me on GitHub
#beginners
<
2019-11-18
>
ScArcher03:11:20

Hi all, I'm looking for a bit of help with this. I'm receiving a string from a socket that is basically name=value;name=value;name=value; format. I'd like to parse this out into a vector or sequence of name value pairs. I have to deal with some escape characters like "\=\\" ";=\;" \n=\\n, etc. but I'm unsure how to approach this in clojure. I tried using regex, but that didn't work out well for me. (probably because my regex kung-fu is weak), so then I started trying to iterate over the sequence of characters. The trouble is, I need to "look ahead" when I encounter certain characters. Does anyone have any advice on this? I implemented a function that takes the string and can return the appropriate character for that position, but I can't quite put it all together. Just looking for some general guidance. Thanks!

didibus03:11:34

(str/split
  "first=bla;second=foo;third=bar"
  #"[=;]+")

;; => ["first" "bla" "second" "foo" "third" "bar"]

didibus03:11:50

You'll need to require clojure.string as str ofcourse

andy.fingerhut03:11:56

That might not work if some of the names and/or values can be enclosed in double quotes, with characters like = and/or ; inside the double quotes, if I am understand the original question correctly.

ScArcher03:11:34

Right, that's the problem with a simple string split. The values can look like this "test\\=test=123;"

ScArcher03:11:04

which would result in ["test=test" "123"] after parsing.

ScArcher03:11:07

If done properly

andy.fingerhut03:11:12

It may be somewhat of a learning curve, perhaps, but you might want to take a look at a parsing library called .... thinking of the name

andy.fingerhut03:11:47

It might make it easier to express what you want than figuring out a regex for it.

ScArcher03:11:29

I know how to do this with loops, it's just a bit ugly and not clojure like.

ScArcher03:11:57

I can revert to that if I need to, but just thought I'd ask for some advice here in case someone had an idea or approach that may be better.

andy.fingerhut03:11:58

I am almost certain it is possible to write a regex for that, but the amount of escaping inside the regex, combined with the escaping of the strings you are trying to recognize, would probably get a little bit mind-bending.

andy.fingerhut03:11:08

It might with Instaparse, too, not sure.

andy.fingerhut03:11:15

Clojure's data.csv library sounds like it has code to recognize at least similar, if not identical, escaping rules -- you may want to see if you can figure out how it does it: https://github.com/clojure/data.csv

ScArcher03:11:30

For regex, I need it split on = where = is not preceded by \

ScArcher03:11:46

and only split on the first one.

andy.fingerhut03:11:35

For regex, I would think a fully correct solution should recognize double-quoted strings, and all of the escaped things inside of them, until it finds the first non-escaped double-quote character after the leading double-quote character. Anything else is just asking for wrong corner cases.

andy.fingerhut03:11:54

Same idea if you use Instaparse

ScArcher03:11:56

The CSV parser is helpful, they are using stringbuilders which is what I was going to use with loops.

ScArcher03:11:13

I may be able to make something work in a similar way. Thanks for the help!

andy.fingerhut03:11:45

Assuming that a string like name="some value containing = character" is one name name, whose value is a string containing a non-escaped = character, which depending on the rules of your data format, may not require escaping if it is inside of double quotes.

ScArcher03:11:33

Only = in the name portion are escaped, the value isn't as it's terminated by the ;

andy.fingerhut03:11:36

But I don't know if the string you are getting has any kind of spec for how it is formatted and ought to be parsed.

ScArcher03:11:06

There is no written spec, I'm looking at code written in c# that is supposed to do this correctly. Hopefully it does 🙂

andy.fingerhut03:11:25

What about name="some string value containing ; and ="; ?

ScArcher03:11:08

the quotes don't mean anything in this parser spec, so the ; inside the value would be escaped with a \ and only the final one would be used to indicate a new name value pair.

andy.fingerhut03:11:10

again, I don't know what all the rules are for your strings, but that is an example you may want to see whether it is considered legal to generate or not.

andy.fingerhut03:11:04

OK, then a warnings about looking at data.csv as an example: I am pretty sure that double quotes are special in CSV files.

andy.fingerhut03:11:27

So may not be as close to what you want as I originally thought it might be.

ScArcher03:11:49

It gives me a direction, this should be a simpler task than the csv parser.

Rex03:11:00

this isn't really a solution per se, but have you considered changing the protocol to something like JSON?

ScArcher03:11:25

I don't control the protocol. I'm just writing a lowly client 🙂

fastaction15:11:29

Does anyone here have healthy experience with middleware and reitit? I have been slamming my head against implementing authentication within reitit for weeks now. I feel like theres something small and stupid I'm missing. The origin of the problem is in basic-auth middleware within my user-login route. If anyone would take a peek and give their two cents, it may help immensely. https://github.com/mxjxn/eclecticlub/blob/master/src/clj/eclecticlub/routes/users.clj

henrik15:11:32

What’s the error?

fastaction16:11:51

It seems it's not actually using the basic-auth middleware. It goes straight to the auth middleware function, resulting in 401 unauthorized.

henrik16:11:52

Are they inserted in the right order? I believe Reitit uses the reverse order of Compojure.

ikitommi16:11:24

@U0Y53Q0R4 reitit applies the middleware in the order defined, so it looks to be correct. There is a request debugging option, prints out all the applied mw, and how they have changed the request/response. Here's how to enable it: https://github.com/metosin/reitit/blob/master/examples/ring-swagger/src/example/server.clj#L67

ikitommi16:11:37

hope this helps.

❤️ 4
fastaction16:11:07

I discovered my problem was that I was passing the email and password parameters in the body, not in a header as buddy auth expected. Thanks for your help.

tdantas21:11:52

is it possible to run on REPL the Cognitect test runner ?

tdantas21:11:53

awesome. reading now the repo ! thanks Alex

Alex Miller (Clojure team)21:11:12

(require '[cognitect.test-runner :as tr])
(tr/-main <whatever-options-you-need>)

tdantas21:11:56

yep, just saw the repo, straightforward ! thx

Alex Miller (Clojure team)21:11:01

I guess there's a little trickiness there - being a main function, it takes a String[]

👍 4
Alex Miller (Clojure team)21:11:49

(tr/-main (into-array String ["-d" "test"]))

Alex Miller (Clojure team)21:11:56

something like that, didn't actually run it

tdantas21:11:13

let you know ! gonna try in a few seconds

Alex Miller (Clojure team)21:11:39

looks like you can also just call the test function directly, that's probably better

tdantas21:11:42

yeah

(let [{:keys [fail error]} (test (:options args))]

tdantas21:11:58

gonna try the test, just setting up my environment, thx for your time

vaer-k23:11:09

So if you call in-ns at the repl, you have to require and refer clojure.repl again? Is there any easier way to switch namespaces?

Alex Miller (Clojure team)23:11:27

Yes, and no respectively

vaer-k23:11:41

I forgot about use, I guess that’s a little easier