Custom linting using sci in clj-kondo:

Not really babashka related, but there's no Sci channel I'm using sci with the :bindings and :readers tags, but I also have an atom in my binding. E.g. this works:

(sci/eval-string "(inc @foo)"
                 {:bindings {'foo (atom 123)}})
;; => 124

(sci/eval-string "#identity (inc @foo)"
                 {:bindings {'foo (atom 123)}
                  :readers {'identity identity}})
;; => [line 1, col 17] Invalid character: @ found while reading symbol.

(using deref does work)

Ah I actually do need the reader sinds I need the un-evaluated expression. But at least deref works

@kevin.van.rooijen This channel is also intended for sci. I haven't seen any examples with data-readers that accept something else than EDN. This example fails in Clojure:

user=> (set! *data-readers* {'identity inc})
{identity #object[clojure.core$inc 0x623e088f "clojure.core$inc@623e088f"]}
user=> #identity (inc @foo)
Syntax error reading source at (REPL:13:21).
clojure.lang.PersistentList cannot be cast to java.lang.Number

even #identity @foo fails

so if you can provide me with a good, practical example that works in Clojure, I'm willing to fix it

I think you made a typo?

(set! *data-readers* {'identity identity})

(set! *data-readers* {'identity inc})

user=> (set! *data-readers* {'identity identity})
{identity #object[clojure.core$identity 0x67d32a54 "clojure.core$identity@67d32a54"]}
user=> (def foo (atom 123))
#'user/foo
user=> #identity (inc @foo)
124

I don't consider identity to be practical useful example of a data reader

usually data-readers don't act on sexprs that have yet to be evaluated, but EDN representations

that's why I provided inc as an example that fails although you might think it shouldn't

Right, and since @ isn't EDN, it's not going to work

Ah right

There might still be a good use case for this, I just want to have the example

since I don't know any

Ok, I'll write up what I'm trying to do

This works:

user=> (set! *data-readers* {'do (fn [expr] `(do (prn 1) ~@expr))})
{do #object[user$eval16$fn__152 0x34a1d21f "user$eval16$fn__152@34a1d21f"]}
user=> #do @foo
1
#object[clojure.lang.Atom 0x1d7f7be7 {:status :ready, :val 1}]

Ok, I wrote an example: TLDR Evaluate an expression where we loop over the items of an atom. If an item gets "clicked", add that item to the end atom. We also want to save this expression syntax. This is because we want to evaluate it once, send it over the wire, and then evaluate it later again with an updated atom.

(def values (atom [1 2 3]))

(set! *data-readers*
      {'with-syntax (fn [syntax]
                      {:syntax (pr-str syntax)
                       :evaluated syntax})})

#with-syntax
(for [v @values]
  [:value {:on-click (fn [] (swap! values conj v))}
   v])

;;=> {:syntax "(for [v (clojure.core/deref values)] [:value {:on-click (fn [] (swap! values conj v))} v])"
;;=>  :evaluated ([:value {:on-click #object[...]} 1]
;;=>              [:value {:on-click #object[...]} 2]
;;=>              [:value {:on-click #object[...]} 3])}


;; Imitating "on-click" event:
(swap! values conj 2)

;; Then, evaluate the `:syntax` key again with sci:

;;=> {:syntax "(for [v (clojure.core/deref values)] [:value {:on-click (fn [] (swap! values conj 9))} v])"
;;=> :evaluated ([:value {:on-click #object[...]} 1]
;;=>             [:value {:on-click #object[...]} 2]
;;=>             [:value {:on-click #object[...]} 3]
;;=>             [:value {:on-click #object[...]} 2])}

So it already works, but using the deref function. And if I have to resort to just using that, it's not a big deal.

Worst case I can preprocess the template, replace any @\w with (deref \w), and it'll still work

so the goal of this exercise is to preserve the sexpr of the thing you put #with-syntax in front of? why?

sorry you already explained it

I was just looking at the code 😉

well, since it works in Clojure I think sci should also support it then

right now reader opts are just passed to tools.reader: :tools.reader/opts {:readers readers}, but we'll have to do something different here then

(and tools.reader is just parsing raw EDN)

Yeah I assumed it used tool.reader, and that doesn't actually support this

The @ symbol I mean

it uses tools.reader.edn to be more specific

but we'll have to re-implement this feature in edamame. it's probably not that difficult

@kevin.van.rooijen Made an issue here: https://github.com/borkdude/edamame/issues/50

Awesome, thanks

Maybe I'll dive into edamame if it's not too intimidating

What needs to be done: instead of letting tools.reader.edn read this value (which is done when none of the other conditions are met) we should now parse the symbol after # ourselves, look if that symbol is in the :readers map to find the reader function, then parse the next value and hand that value to the reader function. Done.

It's fun. Try it 😉

If the symbol isn't in the :readers map, won't the program crash?

Yes. But that's also the case in Clojure

The relevant section is here: https://github.com/borkdude/edamame/blob/master/src/edamame/impl/parser.cljc#L317

If the symbol isn't in the map, we could just try the fallback which is again pass control to tools.reader.edn

This will catch stuff for uuid, etc

So: if the symbol is not in the map, do it like before. If it is in the map, we parse it

I have plenty of code that's going to test this parser

Hmm, but by then we've already read the symbol. So we'll probably have to do that call ourselves using default-data-readers

so exactly like this: https://github.com/clojure/tools.reader/blob/master/src/main/clojure/clojure/tools/reader/edn.clj#L361-L364

Ah right

This one also checks the default-data-readers

So I think the change needs to be applied before this line? https://github.com/borkdude/edamame/blob/11e5bd589788895f1cfac2108a3a730b91b65aac/src/edamame/impl/parser.cljc#L28

@kevin.van.rooijen No, here: https://clojurians.slack.com/archives/CLX41ASCS/p1592663248281900

Ah missed that for some reason. Going to check it out now 🙂

I also posted a couple of comments in the issue

Thanks 👍

Well, got it to work somehow

Added two lines of code but learned quite a lot lol

@borkdude What should we be using to parse the expression in the reader tag?

The only way of parsing I'm familiar with is instaparse but I'm sure that's not what you mean

edamame itself:

(let [sym (parse-next ctx reader)
                  data (parse-next ctx reader)
                  f (or (get (:readers ctx) sym)
                        #?(:clj (default-data-readers sym)))]
              (if f (f data)
                  (throw (new #?(:clj Exception :cljs js/Error)
                              (str "No reader function for tag " sym)))))

I see, guess I need to be a bit more familiar with edamame itself

edamame is already a clojure parser. you just need to parse the symbol foo in #foo, then parse the data part, and then call the appropriate function on the parsed data

the reason tools.reader.edn is used and not tools.reader is that tools.reader does too much, it causes problems with graalvm

@kevin.van.rooijen Fixed in 0.0.11-alpha.13 (building now in CI)

(now I'm going to test if it didn't break anything in sci/babashka)

sci is ok. now babashka

Pushing a new sci version with this: 0.1.1-alpha.1

Thanks, I'll also test it out for my usecase later tonight

@kevin.van.rooijen :

$ lein bb "(set! *data-readers* {'foo (fn [v] {:sexpr (list 'quote v) :val v})}) #foo (inc 1)"
{:sexpr (inc 1), :val 2}

Thanks for picking it up so fast!

Works like a charm!