Fork me on GitHub
#babashka
<
2020-10-13
>
jumar07:10:38

I'm trying to parse a json message from aws cli but got stuck pretty early - what am I doing wrong?

echo '{"DecodedMessage" : "{\"allowed\":false,\"explicitDeny\":false}"}' | bb -i "(do (require '[cheshire.core :as json]) (let [{:strs [DecodedMessage] :as msg} (json/parse-string *input*)] (println DecodedMessage) ))" | bb -i "(do (require '[cheshire.core :as json])
          (let [{:strs [DecodedMessage] :as msg} (json/parse-string *input*)] (println DecodedMessage) ))"
----- Error --------------------------------------------------------------------
Type:     java.lang.ClassCastException
Message:  clojure.lang.Cons cannot be cast to java.lang.String
Location: <expr>:2:50

----- Context ------------------------------------------------------------------
1: (do (require '[cheshire.core :as json])
2:           (let [{:strs [DecodedMessage] :as msg} (json/parse-string *input*)] (println DecodedMessage) ))
                                                    ^--- clojure.lang.Cons cannot be cast to java.lang.String

----- Stack trace --------------------------------------------------------------
cheshire.core/parse-string - <built-in>
user                       - <expr>:2:50
In the REPL, this works:
(def *input* "{\"DecodedMessage\" : \"{\\\"allowed\\\":false,\\\"explicitDeny\\\":false}\"}")
(do (require '[cheshire.core :as json])
          (let [{:strs [DecodedMessage] :as msg} (json/parse-string *input*)] (println DecodedMessage) ))
;;=> 
{"allowed":false,"explicitDeny":false}

jumar07:10:38

I'm trying to parse a json message from aws cli but got stuck pretty early - what am I doing wrong?

echo '{"DecodedMessage" : "{\"allowed\":false,\"explicitDeny\":false}"}' | bb -i "(do (require '[cheshire.core :as json]) (let [{:strs [DecodedMessage] :as msg} (json/parse-string *input*)] (println DecodedMessage) ))" | bb -i "(do (require '[cheshire.core :as json])
          (let [{:strs [DecodedMessage] :as msg} (json/parse-string *input*)] (println DecodedMessage) ))"
----- Error --------------------------------------------------------------------
Type:     java.lang.ClassCastException
Message:  clojure.lang.Cons cannot be cast to java.lang.String
Location: <expr>:2:50

----- Context ------------------------------------------------------------------
1: (do (require '[cheshire.core :as json])
2:           (let [{:strs [DecodedMessage] :as msg} (json/parse-string *input*)] (println DecodedMessage) ))
                                                    ^--- clojure.lang.Cons cannot be cast to java.lang.String

----- Stack trace --------------------------------------------------------------
cheshire.core/parse-string - <built-in>
user                       - <expr>:2:50
In the REPL, this works:
(def *input* "{\"DecodedMessage\" : \"{\\\"allowed\\\":false,\\\"explicitDeny\\\":false}\"}")
(do (require '[cheshire.core :as json])
          (let [{:strs [DecodedMessage] :as msg} (json/parse-string *input*)] (println DecodedMessage) ))
;;=> 
{"allowed":false,"explicitDeny":false}

jeroenvandijk07:10:12

I think

-i                  Bind *input* to a lazy seq of lines from stdi
@ Can you try replacing -i with -e?

jumar07:10:53

Ah right, now I'm getting a different error 🙂

----- Error --------------------------------------------------------------------
Type:     clojure.lang.EdnReader$ReaderException
Message:  java.lang.RuntimeException: Invalid token: :
Location: 1:99

----- Stack trace --------------------------------------------------------------
                           - <expr>:1:99
cheshire.core/parse-string - <built-in>
user                       - <expr>:1:80

jumar07:10:19

Btw. is this how would you write such a simple thing? Is that require even "required"?

borkdude07:10:11

You can just use cheshire for this with parse-stream and *in*

borkdude07:10:03

When using -i then input becomes a lazy seq of lines. You should be aware of this. Cheshire cannot parse a lazy seq of text, it just parses text.

jumar07:10:42

Well, I replaced -i with -e as per your suggestion. I'll try parse-stream

jumar08:10:03

That works!

jumar08:10:10

echo '{"DecodedMessage" : "{\"allowed\":false,\"explicitDeny\":false}"}' | bb -e "(do (require '[cheshire.core :as json]) (let [{:strs [DecodedMessage] :as msg} (json/parse-stream *in*)] (println DecodedMessage) ))"
{"allowed":false,"explicitDeny":false}

jeroenvandijk08:10:59

@ I think forgetting -i or using *input* instead of *in* are easy mistakes to make. Did you consider using something like delays to gives some user feedback here? E.g. @`*input*` could throw an error if -i is not used. Maybe it would also be helpful if there was one var for a particular input. Now *input* can have different shapes based on the flag. I understand something like this would be a breaking change, but I’m curious what you think

borkdude09:10:31

You don't need to use -i and *input* together, if you don't provide -i input contains EDN from stdin

borkdude09:10:21

Just read the docs, it's explained pretty elaborately.

borkdude09:10:54

And it's optional to use, you can also just use *in* and write more verbose code.

jeroenvandijk09:10:34

Yes I understand that reading the docs would help. I was thinking there might be a more foolproof way. But your position is clear :)

borkdude09:10:46

These flags were invented when babashka was more or less just for one-liners and it could not run scripts, namespaces, macros, etc, etc. I don't use these flags a lot myself. Maybe I should hide them a little bit in the README ;)

jeroenvandijk09:10:28

maybe hide them behind a flag --show-flags 😅

borkdude09:10:18

I was also thinking we could have another flag -j for JSON ;)

borkdude09:10:36

or maybe --input json

borkdude09:10:17

--input lines, --input edn, --input edn-stream, --input json, --input json-stream

jeroenvandijk09:10:49

It sounds useful. I haven’t done much with input yet. The potential error cases would grow I’m guessing, maybe like the example above

borkdude09:10:04

maybe --input lines and --input edn-stream is more clear than -i and -I

borkdude09:10:21

it defaults to --input edn (only one edn val from stdin)

borkdude09:10:14

also --output lines and --output edn-stream

jeroenvandijk09:10:45

Or an idea could be to not have flags, but leave it to the program? Maybe something like https://github.com/originrose/lazy-map that you would call like (:edn-stream *input*) . It would start parsing on the first call (i.e. lazy). If the value is incorrect it would throw. If there is no input it would throw. Would this make sense?

borkdude09:10:24

I think those would just be functions based on *in*

borkdude09:10:34

no special libs needed. can already write those

jeroenvandijk09:10:41

That’s true, only benefit would be that you don’t have to have a reference somewhere. But maybe it’s too magical for little benefit

borkdude09:10:25

The benefit of not having flags is that it works better for scripts.

borkdude09:10:04

http://babashka.io helper namespace with parse-edn-stream, etc could work

borkdude09:10:38

which then deprecates these flags over time

jeroenvandijk09:10:10

Makes sense. Would also make example scripts more explicit (no need to guess the flag settings)

borkdude09:10:10

(http://babashka.io/edn-stream) (assumes input is in) (http://babashka.io/edn-stream reader) (explicit reader) same for json, transit

borkdude09:10:41

and these functions could be part of the user namespace for one-liners

borkdude09:10:58

echo '{:a 1} {:a 2}' | bb '(edn-stream)'

borkdude09:10:02

echo '{:a 1} {:a 2}' | bb '(edn-stream *in*)' for more explicitness

borkdude09:10:14

(maybe better to be explicit anyway)

jeroenvandijk09:10:23

Nice 🙂 And this lib could potentially be used from the jvm as well (I guess). So would make it easier to reuse both directions

jeroenvandijk10:10:54

@ Thanks for listening 🙂