Fork me on GitHub
#chlorine-clover
<
2020-04-21
>
mauricio.szabo01:04:06

I also tested edamame, parsing core.cljs to benchmark comparing with the current implementation. It's almost the same speed, so I think I'll keep the current approach

sogaiu04:04:06

out of curiosity, did you happen to try on clojure.core or clojurescript.core?

mauricio.szabo14:04:22

Yes, the file I tried was the core.cljs from the ClojureScript source 🙂. With Edamame, it parsed in 504ms. With rewrite-cljs, 610ms

sogaiu04:04:26

i recently looked at file sizes of real-world clojure code and found something like the following:

1k:  73042
  2k:  22604
  4k:  16474
  8k:   7887
 16k:   3027
 32k:   1014
 64k:    331
128k:    108
256k:     21
512k:      6
  1M:      2
  4M:      3
for the left column, 16k means roughly, not larger than 16k, but greater than 8k. for the right column, that's the total number of files that were in a range.

seancorfield04:04:23

@sogaiu What are those stats from? I'm curious what "real-world clojure code" is public...?

sogaiu04:04:33

yes, i fetched releases from clojars

seancorfield04:04:51

Ah, so this is libraries?

sogaiu04:04:03

whatever is on clojars

seancorfield04:04:28

That would not include applications, just open source libraries.

sogaiu04:04:43

i think there are some apps there too, but i didn't try to count

sogaiu04:04:57

i suspect they are mostly libraries though

seancorfield04:04:03

Just curious, since "most" real world Clojure is proprietary / behind closed doors, as far as I know...

4
sogaiu04:04:06

i do have a collection of things fetched from github as well loosely based on andy.fingerhut's haironfire work, but i haven't tried to count against those

sogaiu04:04:31

so libraries don't count as real world?

sogaiu04:04:36

just playing with you 🙂

sogaiu04:04:42

poor phrasing on my part

sogaiu04:04:50

i should have said clojars to begin with

seancorfield04:04:18

NP. I was just wondering where the stats came from and what you were implying by that...

seancorfield04:04:45

If it's "can we parse all this public code with tools.reader", yeah, that's a very real concern.

sogaiu04:04:10

i collected the code for testing of a tree-sitter grammar for clojure -- there is some weird stuff out there

4
Mattias07:04:26

Apropos namespaces, I have been wondering for awhile how when I execute something via Chlorine it knows the correct namespace.

Mattias07:04:35

Also curious if you can have multiple namespace declarations in a file, and how that works out (I’m assuming the Java rules of files/packages prevent that...).

mauricio.szabo14:04:41

@mattias504 It uses the current cursor position, and searches backwards. The first ns form it finds, is the one it uses 🙂

seancorfield15:04:37

Which explains why certain things don't work if the cursor is above the ns form (at the start of the file, for example).

mauricio.szabo16:04:15

Yes, probably... to be honest I never tested too much evaluating code before a ns form 🙂

seancorfield16:04:35

I mostly notice it when trying to run tests or my inspect namespace command.

Mattias18:04:24

Interesting, thanks!

mauricio.szabo19:04:35

Well, running tests on current namespace can be something I should look at... it's not rare to Clojure sources to have comments before the namespace.

seancorfield19:04:14

Yeah, I'd say that if you hit the start of the file with only comments/whitespace without finding a ns then you could look forward for one.

mauricio.szabo14:04:18

sql-korma uses multiple namespaces per file, that's why I implemented support for multiple ns forms per file 🙂

mauricio.szabo14:04:22

Yes, the file I tried was the core.cljs from the ClojureScript source 🙂. With Edamame, it parsed in 504ms. With rewrite-cljs, 610ms

borkdude15:04:38

Is that from JS?

mauricio.szabo15:04:12

@U04V15CAJ by the way... I wasn't able to use a default tag reader. The code below doesn't work:

(eda/parse-string "#some [:name]" {:reader {:default tagged-literal}})

borkdude15:04:05

@U3Y18N0UC try:

user=> (eda/parse-string "#some [:name]" {:tools.reader/opts {:default tagged-literal}})
#some [:name]

borkdude15:04:39

:readers (plural) is recognized as a convenience, but other options for tools reader you can pass like this

mauricio.szabo15:04:08

Yes, that solved it 🙂 Also, how do I use :auto-resolve?

mauricio.szabo15:04:40

When I tried to parse clojurescript core, I ended up replacing all :: to a single :

borkdude15:04:32

you can pass that a map of aliases to namespaces

borkdude15:04:39

or :current for the current namespace

mauricio.szabo16:04:48

Great, it works, thanks! 😄

mauricio.szabo15:04:39

Yes, I'm running both from ClojureScript, parsing the cljs.core namespace source file

👍 4
seancorfield15:04:37

Which explains why certain things don't work if the cursor is above the ns form (at the start of the file, for example).