Fork me on GitHub
#babashka
<
2020-06-20
>
borkdude11:06:20

Custom linting using sci in clj-kondo:

Kevin12:06:14

Not really babashka related, but there's no Sci channel I'm using sci with the :bindings and :readers tags, but I also have an atom in my binding. E.g. this works:

(sci/eval-string "(inc @foo)"
                 {:bindings {'foo (atom 123)}})
;; => 124
But this fails
(sci/eval-string "#identity (inc @foo)"
                 {:bindings {'foo (atom 123)}
                  :readers {'identity identity}})
;; => [line 1, col 17] Invalid character: @ found while reading symbol.
I assume the reader is implemented is edamame / reader.edn. I don't know if this is fixable, or if you'd want this fixed @borkdude? Otherwise I'll just have to resort to functions, which is also fine.

Kevin12:06:23

(using deref does work)

Kevin12:06:29

Ah I actually do need the reader sinds I need the un-evaluated expression. But at least deref works

borkdude13:06:59

@kevin.van.rooijen This channel is also intended for sci. I haven't seen any examples with data-readers that accept something else than EDN. This example fails in Clojure:

user=> (set! *data-readers* {'identity inc})
{identity #object[clojure.core$inc 0x623e088f "clojure.core$inc@623e088f"]}
user=> #identity (inc @foo)
Syntax error reading source at (REPL:13:21).
clojure.lang.PersistentList cannot be cast to java.lang.Number

borkdude13:06:41

even #identity @foo fails

borkdude13:06:10

so if you can provide me with a good, practical example that works in Clojure, I'm willing to fix it

Kevin13:06:07

I think you made a typo?

(set! *data-readers* {'identity identity})
vs
(set! *data-readers* {'identity inc})

Kevin13:06:05

user=> (set! *data-readers* {'identity identity})
{identity #object[clojure.core$identity 0x67d32a54 "clojure.core$identity@67d32a54"]}
user=> (def foo (atom 123))
#'user/foo
user=> #identity (inc @foo)
124

borkdude13:06:15

I don't consider identity to be practical useful example of a data reader

borkdude13:06:05

usually data-readers don't act on sexprs that have yet to be evaluated, but EDN representations

borkdude13:06:34

that's why I provided inc as an example that fails although you might think it shouldn't

Kevin13:06:36

Right, and since @ isn't EDN, it's not going to work

borkdude13:06:04

There might still be a good use case for this, I just want to have the example

borkdude13:06:11

since I don't know any

Kevin13:06:23

Ok, I'll write up what I'm trying to do

borkdude13:06:48

This works:

user=> (set! *data-readers* {'do (fn [expr] `(do (prn 1) ~@expr))})
{do #object[user$eval16$fn__152 0x34a1d21f "user$eval16$fn__152@34a1d21f"]}
user=> #do @foo
1
#object[clojure.lang.Atom 0x1d7f7be7 {:status :ready, :val 1}]
but then you're basically using a data reader as a macro

Kevin13:06:32

Ok, I wrote an example: TLDR Evaluate an expression where we loop over the items of an atom. If an item gets "clicked", add that item to the end atom. We also want to save this expression syntax. This is because we want to evaluate it once, send it over the wire, and then evaluate it later again with an updated atom.

(def values (atom [1 2 3]))

(set! *data-readers*
      {'with-syntax (fn [syntax]
                      {:syntax (pr-str syntax)
                       :evaluated syntax})})

#with-syntax
(for [v @values]
  [:value {:on-click (fn [] (swap! values conj v))}
   v])

;;=> {:syntax "(for [v (clojure.core/deref values)] [:value {:on-click (fn [] (swap! values conj v))} v])"
;;=>  :evaluated ([:value {:on-click #object[...]} 1]
;;=>              [:value {:on-click #object[...]} 2]
;;=>              [:value {:on-click #object[...]} 3])}


;; Imitating "on-click" event:
(swap! values conj 2)

;; Then, evaluate the `:syntax` key again with sci:

;;=> {:syntax "(for [v (clojure.core/deref values)] [:value {:on-click (fn [] (swap! values conj 9))} v])"
;;=> :evaluated ([:value {:on-click #object[...]} 1]
;;=>             [:value {:on-click #object[...]} 2]
;;=>             [:value {:on-click #object[...]} 3]
;;=>             [:value {:on-click #object[...]} 2])}

Kevin14:06:37

So it already works, but using the deref function. And if I have to resort to just using that, it's not a big deal.

Kevin14:06:17

Worst case I can preprocess the template, replace any @\w with (deref \w), and it'll still work

borkdude14:06:36

so the goal of this exercise is to preserve the sexpr of the thing you put #with-syntax in front of? why?

borkdude14:06:45

sorry you already explained it

borkdude14:06:53

I was just looking at the code 😉

borkdude14:06:40

well, since it works in Clojure I think sci should also support it then

borkdude14:06:42

right now reader opts are just passed to tools.reader: :tools.reader/opts {:readers readers}, but we'll have to do something different here then

borkdude14:06:00

(and tools.reader is just parsing raw EDN)

Kevin14:06:51

Yeah I assumed it used tool.reader, and that doesn't actually support this

Kevin14:06:57

The @ symbol I mean

borkdude14:06:14

it uses tools.reader.edn to be more specific

borkdude14:06:38

but we'll have to re-implement this feature in edamame. it's probably not that difficult

Kevin14:06:09

Awesome, thanks

Kevin14:06:50

Maybe I'll dive into edamame if it's not too intimidating

borkdude14:06:39

What needs to be done: instead of letting tools.reader.edn read this value (which is done when none of the other conditions are met) we should now parse the symbol after # ourselves, look if that symbol is in the :readers map to find the reader function, then parse the next value and hand that value to the reader function. Done.

borkdude14:06:56

It's fun. Try it 😉

Kevin14:06:57

If the symbol isn't in the :readers map, won't the program crash?

borkdude14:06:21

Yes. But that's also the case in Clojure

borkdude14:06:57

If the symbol isn't in the map, we could just try the fallback which is again pass control to tools.reader.edn

borkdude14:06:02

This will catch stuff for uuid, etc

borkdude14:06:23

So: if the symbol is not in the map, do it like before. If it is in the map, we parse it

borkdude14:06:56

I have plenty of code that's going to test this parser

borkdude14:06:24

Hmm, but by then we've already read the symbol. So we'll probably have to do that call ourselves using default-data-readers

Kevin14:06:11

This one also checks the default-data-readers

Kevin18:06:51

Ah missed that for some reason. Going to check it out now 🙂

borkdude18:06:10

I also posted a couple of comments in the issue

Kevin18:06:10

Thanks 👍

Kevin19:06:43

Well, got it to work somehow

Kevin19:06:05

Added two lines of code but learned quite a lot lol

Kevin20:06:46

@borkdude What should we be using to parse the expression in the reader tag?

Kevin20:06:07

The only way of parsing I'm familiar with is instaparse but I'm sure that's not what you mean

borkdude20:06:42

edamame itself:

(let [sym (parse-next ctx reader)
                  data (parse-next ctx reader)
                  f (or (get (:readers ctx) sym)
                        #?(:clj (default-data-readers sym)))]
              (if f (f data)
                  (throw (new #?(:clj Exception :cljs js/Error)
                              (str "No reader function for tag " sym)))))
Already on it.

Kevin20:06:00

I see, guess I need to be a bit more familiar with edamame itself

borkdude20:06:59

edamame is already a clojure parser. you just need to parse the symbol foo in #foo, then parse the data part, and then call the appropriate function on the parsed data

borkdude20:06:18

the reason tools.reader.edn is used and not tools.reader is that tools.reader does too much, it causes problems with graalvm

borkdude21:06:10

@kevin.van.rooijen Fixed in 0.0.11-alpha.13 (building now in CI)

borkdude21:06:35

(now I'm going to test if it didn't break anything in sci/babashka)

borkdude21:06:44

sci is ok. now babashka

borkdude21:06:14

Pushing a new sci version with this: 0.1.1-alpha.1

Kevin21:06:06

Thanks, I'll also test it out for my usecase later tonight

borkdude21:06:26

@kevin.van.rooijen :

$ lein bb "(set! *data-readers* {'foo (fn [v] {:sexpr (list 'quote v) :val v})}) #foo (inc 1)"
{:sexpr (inc 1), :val 2}
thanks for filing the issue 🙂

Kevin21:06:02

Thanks for picking it up so fast!

Kevin22:06:04

Works like a charm!