@dominicm Agree that tree-sitter is super interesting! And also all the Lua stuff is super cool, Lua is a really fun (and weird). I think for the tree-sitter grammer stuff you wouldn’t need Lua though since it just takes some JSON as input for the parser? Could totally be missing the point here 😅

fennel looks fun too

Yeah, sooo much! I’ve used it to write a toy-ish CLI time tracker, getting back that REPL really unblocked my Lua learning at that point 😄

With all the Neovim Lua support it could be really cool, seems @olical is even playing around with Fennel/Lua in Conjure https://github.com/Olical/conjure/commits/master

cool! yeah, i have my fingers crossed that Olical's fennel / conjure efforts work out.

I'm just thinking about getting something now, rather than later

lpeg can parse clojure/core.clj in 20ms

just checking that in luajit

wow, that's pretty nice -- i test with clojure/core.clj as my "biggest" clojure file for a program that uses rewrite-clj, and i can definitely feel its weight there.

yeah, exactly

I'd prefer it to be closer to 5ms

i haven't found much that comes close to the size of clojure.core -- do you come across that much that's similar in size?

sometimes 🙂

i guess i'd better go searching for more data

luajit is 12ms

@sogaiu depends on your tool's purpose.

I was thinking of using this for highlighting, so it needs to be fast.

my current use case is invocation from an editor -- doing things like detecting the expression around or to the left of the cursor

if you have to do the whole buffer, i guess it would have to be fast

oh, same thing then 😄

lol

oh, maybe not.

what do you do with the detected expression?

the idea is to allow editors to not have to reimplement all of this type of functionality

provide pieces that can be built upon so behaviors can be consistent

what type of functionality 🙂

find an appropriate ns, detect expression, find close top-levels, etc.

these things vary across editors

so, to find the close top-level you need the whole buffer right?

almost

i have a hack that may make it a bit less -- though i have no proof that it works in general

ah actually, that's for detecting the expression -- not the top-level

you're right, you need the whole darn thing

so it does need to be fast

but typical clojure source files don't seem to be that large

at least for what i've tested on, it doesn't seem too bad

now I'm curious to find the largest file I have

didn't someone recently fetch some significant portion of clojars (or was it metadata of clojars)?

42415

I may have figured out how that is done, yeah

rsync usage is unmetered on clojars, so go crazy

that is, 42415 loc

oooh

http://github.com/clojure/clojurescript/src/main/cljs/cljs/core.cljs is the second biggest file though

clojure/core.clj is a measly 7892 loc

11720 in core.cljs

he he -- you are up-to-date 🙂

i had to pull

thing is, how typical is that?

I jump to source into these files

the 42k is pretty extreme tbh, I was surprised to find that

yes, i do jump-to-source in my deps too -- but i rarely eval expressions

woah, vis, an editor which uses lpeg for higlighting. Makes mince meat out of that 42k loc file

ha, but it's highlighting does not handle unicode properly

~/src                                                                 0.94s 01:09:11
❯ tokei **/*.clj{,s,c}               
-------------------------------------------------------------------------------
 Language            Files        Lines         Code     Comments       Blanks
-------------------------------------------------------------------------------
 Clojure              5882       743420       619406        37019        86995
 ClojureC             1505       517672       421894        33381        62397
 ClojureScript        4552      1487768      1233528        50189       204051
 Dockerfile              1            5            4            0            1
 Edn                     2           31           25            6            0
-------------------------------------------------------------------------------
 Total               11942      2748896      2274857       120595       353444
-------------------------------------------------------------------------------

i've given up for the moment on highlighting of large files -- try opening a clj-kondo analysis file for a project and its deps some time 🙂

I think this 42k loc file might be generated. It's a list of place names in the UK.

ahhhh

nice, didn't know about tokei

(that's an odd name -- the characters and pronunciation used at the repository readme are for "clock" (at least in japanese), but if you change the name slightly it would have been "statistics"...strange)

i don't have even a tenth -- though i don't think it's counting correctly:

$ tokei **/*.clj{,s,c}
-------------------------------------------------------------------------------
 Language            Files        Lines         Code     Comments       Blanks
-------------------------------------------------------------------------------
 Clojure               299        20633        16013         1744         2876
 ClojureC                8          772          532           50          190
 ClojureScript           1            2            2            0            0
-------------------------------------------------------------------------------
 Total                 308        21407        16547         1794         3066
-------------------------------------------------------------------------------

$ find | grep \\.cljs$ | wc -l
13183

I have a lot of code. Mine includes a lot of stuff in target folders.

oh, lpeg is included with neovim now

hurray for built-in goodies 🙂

$ find | grep \\.cljc$ | wc -l
2733

may be i'm not invoking tokei correctly?

$ find | grep \\.clj$ | wc -l
15712

might be to do with shell settings not expanding the same way mine does

it's possible I have something turned on especially

may be so -- will try the default invocation

-------------------------------------------------------------------------------
 Language            Files        Lines         Code     Comments       Blanks
-------------------------------------------------------------------------------
 Clojure              6835      1275323      1019262        92211       163850
 ClojureC             1692       308692       250172        17327        41193
 ClojureScript        8471      2162893      1864853        70179       227861
-------------------------------------------------------------------------------

totals are off -- because i just kept 3 rows

this includes duplicates though -- so unless tokei prunes, the numbers are inflated

yeah, same as mine, inflated 🙂

it's just fun really 🙂

indeed!

@martinklepsch https://neovim.io/doc/user/lua.html looks like treesitter uses .so files, so a C api?

the treesitter nvim api, is exposed via the built-in lua engine

https://github.com/neovim/neovim/pull/11113 I didn't notice the "and highlighting"

Yeah I guess it’s both in the end. What I was thinking is that you can write grammars in any language as the canonical format they are described in is JSON. How you then interact with tree sitter is probably a different thing, C or Lua as you said.

I'm not sure if it is json or not. The above seems to imply that a .so is produced to represent the parser. I'm guessing that atom is just using some API to create the parser from json.

https://tree-sitter.github.io/tree-sitter/creating-parsers maybe they're doing on the fly compilation? nodejs is a dependency for developing a new parser. tree-sitter converts javascript into a C parser, and then compiles that.

I'm finding vim-fireplace is constantly timing out and requiring me to restart the server and reconnect to cljs. I'm using it with Fulcro 3. Anyone else getting this behaviour?