This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-11-05
Channels
- # announcements (5)
- # beginners (49)
- # boot (9)
- # calva (34)
- # cider (9)
- # clara (18)
- # clj-kondo (1)
- # cljsrn (7)
- # clojure (196)
- # clojure-dev (4)
- # clojure-europe (11)
- # clojure-france (3)
- # clojure-nl (13)
- # clojure-norway (2)
- # clojure-spec (8)
- # clojure-uk (85)
- # clojurescript (87)
- # clojurex (202)
- # core-logic (6)
- # data-science (6)
- # datomic (9)
- # dirac (1)
- # duct (93)
- # emacs (9)
- # events (1)
- # fulcro (22)
- # hoplon (30)
- # jackdaw (10)
- # keechma (1)
- # leiningen (5)
- # malli (17)
- # off-topic (9)
- # other-languages (3)
- # pedestal (7)
- # re-frame (3)
- # reagent (5)
- # reitit (9)
- # remote-jobs (6)
- # rewrite-clj (80)
- # ring (2)
- # shadow-cljs (191)
- # tools-deps (54)
- # vim (14)
- # xtdb (7)
Forgive me for stupid questions. I am quite unfamiliar with what is going on in rewrite-clj and only have the vaguest idea about zippers and stuff. Eventually I will fix that, but right now I am just a user by proxy and I wonder about how you guys think about parsing things that are not strictly EDN. My use cases are pretty printing, and syntax highlighting and such. Here’s an example:
(comment
(do
(def db-uri "datomic:")
(d/create-database db-uri)
(def conn (§d/connect db-uri))
@(d/transact conn [{:db/doc "Hello world"}])))
Evaluating the (do...
form here gives back something like this: {:db-before datomic.db.Db@d612a1f8, :db-after datomic.db.Db@298c26ad, :tx-data [#datom[13194139534316 50 #inst "2019-11-05T09:00:57.870-00:00" 13194139534316 true] #datom[17592186045421 62 "Hello world" 13194139534316 true]], :tempids {-9223301668109598141 17592186045421}}
Trying to pretty-print that with anything other than clojure.core/pprint
blows up, for somewhat different reasons, depending on which pretty printer is tried. But if rewrite.clj(s)
is involved it blows up on the db
values there, trying to parse the stuff after the @
as a number. On some level that makes sense, but from a Calva user perspective it most often doesn’t.So, my question. Is there a way today to get rewrite-clj to not totally croak on that input? And if it isn’t, do you think it would be something worth trying to add?
Another thing that confuses me is that when trying to pretty print the result above, nREPL server side, all pretty printers I have tried, except clojure.core/pprint
, start to evaluate the database values, so with the query above, zprint
, for instance, prints out the whole database. Twice. This isn’t happening in rewrite-clj
, right`?
This is what I'm seeing with rewrite-clj:
$ clj -Sdeps '{:deps {rewrite-clj {:mvn/version "RELEASE"}}}'
Clojure 1.10.1
user=> (require '[rewrite-clj.parser :as p])
nil
user=> (p/parse-string "{:db-before datomic.db.Db@d612a1f8, :db-after datomic.db.Db@298c26ad, :tx-data [#datom[13194139534316 50 #inst \"2019-11-05T09:00:57.870-00:00\" 13194139534316 true] #datom[17592186045421 62 \"Hello world\" 13194139534316 true]], :tempids {-9223301668109598141 17592186045421}}")
Execution error (ExceptionInfo) at clojure.tools.reader.impl.errors/throw-ex (errors.clj:34).
Invalid number: 298c26ad.
using clojure.pprint on this doesn't require you to parse the value from a string, because it can print the value directly from memory. that's why it works
Yeah. hence my question, if it would be something worth considering, to have a switch where it won’t croak when something doesn’t parse to a valid EDN type.
if it isn't valid code, it should probably croak. it should not try to be better than clojure.core/read-string
use the right tool for the right job, rewrite-clj isn't a pretty-printer, it's a rewriter from text -> text
Agree somewhat. I’m just trying to avoid having to write my own parser. Which I have already done, as it happens, so I could probably solve this that wat, but I would like to be able to use libraries built on top of rewrite-clj.
why are you first printing it to a string and then trying it to re-parse it, if the goal is to have pretty-printing?
it makes sense if you're trying to pretty-print text files (so code as you write it, re-formatting), but not for pretty-printing in memory values
So, there are reasons. Trying to do this server side, I get other, unacceptable side effects (as mentioned above), so I am trying to solve it client side and then what I have is a string, because that is what nrepl gives me. Maybe it is time to look at the EDN transport there…
nREPL should give you access to the value as is, which you can then pprint, no serialization in between
But pretty printing is not the only use case for me. I also do code formatting, using cljfmt
. And it creates a bad Ux when it refuses to format results like the above.
In a way it sounds like cljfmt, zprint and other formatters/pretty printers should not be using rewrite-clj. Is that correctly understood?
The above value cannot be parsed by Clojure itself so it should probably not occur in code. So I don't see a problem for cljfmt there?
Not completely. Using rewrite-clj for pprinting is fine as long as you start with valid textual code
Code files contain a lot of stuff. So what I often do is that I paste the result of some evaluation inside a (comment ...)
form to have as a reference. For that usage pretty printing is very helpful.
datomic.db.Db@298c26ad
is something that Java prints when an Object doesn't have an overriden toString method. The one who serializes it could chose to serialize it as "<<object>>"
so it will remain valid when parsed back. I think that's where it should/could be fixed.
> Code files contain a lot of stuff.
Yes, but if that code cannot be read by clojure.core/read-string
you're f*cked anyway.
Just try to insert the above db value in one of your project and then re-start your REPL.
Yeah, I’ve seen that often, trying to load a file with one of those (comment invalid-form)
in it. 😃
So, I know that read-string doesn’t like it. I have tried a lot of things when trying to fix this problem in Calva, believe me. haha
it's a common source of frustration that pr-str doesn't always output strings that can be read back with read-string
. Same with edn/read-string.
user=> (read-string (pr-str {:a (symbol "foo bar")}))
Execution error at user/eval9 (REPL:1).
Map literal must contain an even number of forms
Would be horrible if it did, because then I couldn’t use cljfmt for my formatting at all.
The parser doesn't complain but:
user=> (rewrite-clj.node/sexpr (p/parse-string "{:a foo bar}"))
Execution error (IllegalArgumentException) at rewrite-clj.node.seq/map-node$fn (seq.clj:110).
No value supplied for key: bar
@pez I would look into extending print-method
so it prints datomic.db.Db
values in a readable way
actually reading that back to a Db
value is a different story, but at least you'll be able to process it with rewrite-clj
, and you can define a dummy data reader so this syntax round trips, even if reading it doesn't yield an actual valid Db
instance
that works, but as an IDE you can't possibly foresee all the kinds of objects that people are going to print
maybe something like this could work:
(set! *warn-on-reflection* true)
(let [old-method (get-method print-method Object)]
(defmethod print-method Object [v ^java.io.Writer w]
(if old-method (old-method v w)
(.write w (format "<<%s>>" (.getName (.getClass ^Object v)))))))
(deftype XYZ [])
(XYZ.) ;;=> <<user.XYZ>>
(remove-method print-method Object)
maybe restoring the print-method after printing is best, so it doesn't interfere with other tooling
so maybe:
(let [old-method (get-method print-method Object)]
(defmethod print-method Object [v ^java.io.Writer w]
(.write w (format "<<%s>>" (.getName (.getClass ^Object v)))))
;; pprint here
;; restore method
(when old-method
(defmethod print-method Object [v w]
(old-method v w))))
This is likely going to interfere with other threads that are running in the same JVM
@pez Maybe you could also use print-dup
. That throws when there is no valid method defined:
user=> (let [sw (java.io.StringWriter.)] (print-dup (XYZ.) sw) (str sw))
Execution error (IllegalArgumentException) at user/eval146 (REPL:1).
No method in multimethod 'print-dup' for dispatch value: class
Thanks, both of you! I will experiment a bit and we’ll see if I can get this as user friendly as I’d like to.
@pez btw, regarding zippers, if you are interested, one of the most helpful videos i saw was plexus': https://lambdaisland.com/blog/2018-11-26-art-tree-shaping-clojure-zip -- i found the diagrams presented with the explanations to be particularly good.
yes, I agree with @sogaiu, that is an excellent video. @pez, I also found https://clojure.org/reference/other_libraries#_zippers_functional_tree_editing_clojure_zip useful
and also Tim Baldridge's videos are great (not entirely free, but also very inexpensive) https://tbaldridge.pivotshare.com/media/zippers-episode-1/11348/feature?t=0
oh yes, i have found that following along with tim baldridge's presentations at the repl to be particularly instructive. though it is effort, i have usually found it to be worth it :thumbsup:
on the note of zippers, not sure about how relevant it might be, but @martinklepsch noticed that in a java "port" of tree-sitter, there is a clojure api: https://github.com/JetBrains/jsitter/tree/master/clj -- it has some zipper stuff in it. apart from that though i wonder whether might be some ideas / synergy for rewrite-clj*...
I also enjoyed plexus' zipper video (small sidenote: my findings about performance were different than his conclusion at the end)
Can you elaborate @borkdude? I don't remember what I said exactly about performance, besides the recommendation to use fast-zip
Also yay glad to see people find that talk useful. The viz code is on github somewhere, although I never got around to releasing it properly.
@plexus https://www.youtube.com/watch?v=5Nm56YvTKZY&feature=youtu.be&t=2558 "one of the benefits of using zippers is the performance gain", although you state that's not the main reason why you would use them. I didn't find them to be more performant than manual editing (actually the opposite).
just so i understand, he was referring to the modifications being done all together vs each one separately, right?
I think so, but traversing with zippers also causes data structures to be modified, so I guess it depends on the amount of overhead vs. the amount of updates you're trying to accomplish. With typical things I was doing with rewrite-clj manual updates were much faster. I guess we need benchmarks.