This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-04-01
Channels
- # announcements (19)
- # asami (4)
- # babashka (34)
- # beginners (137)
- # calva (22)
- # cider (4)
- # clj-kondo (25)
- # cljs-dev (4)
- # clojure (67)
- # clojure-australia (1)
- # clojure-berlin (1)
- # clojure-europe (35)
- # clojure-germany (3)
- # clojure-nl (5)
- # clojure-serbia (3)
- # clojure-uk (8)
- # clojuredesign-podcast (2)
- # clojurescript (11)
- # conjure (56)
- # data-oriented-programming (1)
- # datascript (1)
- # datomic (6)
- # deps-new (11)
- # eastwood (1)
- # fulcro (11)
- # honeysql (48)
- # inf-clojure (1)
- # jobs (1)
- # joker (6)
- # lsp (26)
- # malli (2)
- # meander (3)
- # off-topic (48)
- # pathom (4)
- # polylith (4)
- # re-frame (19)
- # releases (2)
- # remote-jobs (1)
- # rewrite-clj (127)
- # shadow-cljs (6)
- # spacemacs (3)
- # tools-deps (43)
- # xtdb (16)
Hi! I’m interested in implementing a SLOC counter for clojure (if it is not too difficult)
it needs to count blank lines, comments (including (comment ) and #_
I will need the parser + node api right?
thanks for the pointers!
@dimitar.ouzounoff please let us know how it goes!
will do, I have to wrap one task first and I will get on this; my first impression is that most likely I will need to parse this line by line
@dimitar.ouzounoff No, you can just use parser/parse-string-all and this will give you a :forms node. Then you can go over its :children and skip all :comment and :uneval or empty lines, etc. And then just call str on the remaining nodes and count the number of lines from those.
well, I saw parse-string-all but it doesn’t return a seq; I guess I need to read on the nodes to understand them better
@dimitar.ouzounoff It returns one node, a :forms node
hmm this is what a comment looks like in cider-inspect:
Class: rewrite_clj.node.seq.SeqNode
Meta Information:
:row = 87
:col = 1
:end-row = 91
:end-col = 5
Contents:
:tag = :list
:format-string = "(%s)"
:wrap-length = 2
:seq-fn = rewrite_clj.node.seq$list_node$fn__7201@66935f35
:children = ( { :value comment, :string-value "comment", :map-qualifier nil } { :newlines "\n" } { :whitespace " " } { :tag :list, :format-string "(%s)", :wrap-length 2, :seq-fn rewrite_clj.node.seq$list_node$fn__7201@fab763f, :children ( { :value do,
I’m not sure how to access it, I can’t seem to use nth
in order to use a comment? predicate
@dimitar.ouzounoff you can achieve what your goal with the node API, but the zip API is a bit higher level.
something like:
(defn comment-node? [node]
(and (= :list (node/tag node)) (some-> node :children first :value (= 'comment))))Personally I don't see a need for the zipper API here, since it's pretty straightforward to only iterate over the top level nodes, there isn't a need to visit any deeper nodes and/or rewrite/remove them
No need, just an alternative that might offer easier nav through tree. Not sure exactly what @dimitar.ouzounoff wants to count here yet.
yes, I guess this is why something like rewrite-clj is necessary as it needs to exclude whole multiline comment forms from the line count
and you're also able to count whitespace lines, so rewrite-clj is a great tool for this problem to implement clocl (count line of clojure)
which will soon be available as a GraalVM binary? with proper command line interface please? :P
I’m still at a beginner level, but it sounds like fun 🙂
It is! Borkdude and I curate some tips over at https://github.com/lread/clj-graal-docs. I was more joking with borkdude about proper cmd lines, which is something he has been recently irked by.
well I’m not sure how to go over the nodes
rewrite-clj.zip/next might be something I can use
another thing that I’m not sure about is that if take this line: (set! *warn-on-reflection* true) ;; avoid reflexion so you can use graalvm
it is two children on the first depth
so it would count as a line of code + a line of column
user=> (require '[clojure.string :as str] '[rewrite-clj.parser :as p])
nil
user=> (:children (p/parse-string-all "(+ 1 2 3)\n;;hello\n(comment 1 2 3)"))
(<list: (+ 1 2 3)> <newline: "\n"> <comment: ";;hello\n"> <list: (comment 1 2 3)>)
user=> (map str (:children (p/parse-string-all "(+ 1 2 3)\n;;hello\n(comment 1 2 3)")))
("(+ 1 2 3)" "\n" ";;hello\n" "(comment 1 2 3)")
user=> (map (comp count str/split-lines str) (:children (p/parse-string-all "(+ 1 2 3)\n;;hello\n(comment 1 2 3)")))
(1 0 1 1)
user=> (apply + (map (comp count str/split-lines str) (:children (p/parse-string-all "(+ 1 2 3)\n;;hello\n(comment 1 2 3)"))))
3user=> (require '[rewrite-clj.node :as node])
nil
user=> (defn comment-node? [node]
(and (= :list (node/tag node)) (some-> node :children first :value (= 'comment))))
#'user/comment-node?
user=> (apply + (map (comp count str/split-lines str) (remove comment-node? (:children (p/parse-string-all "(+ 1 2 3)\n;;hello\n(comment 1 2 3)")))))
2@lee I'm pretty tempted to just add rewrite-clj to babashka so you can write little scripts like this ;)
ok, I think I get it now, thanks @borkdude!
(z/of-string "!/usr/bin/env bb\n\n(ns foo)")
=> Execution error (ExceptionInfo) at clojure.tools.reader.impl.errors/throw-ex (errors.clj:34).
Invalid symbol: !/usr/bin/env.Is just that if a user open a babashka file with that interpreter and is using clojure-lsp, it will throw a lot of those exceptions 😕
ouch!
I think rewrite-clj should be able to handle this, but you need to start it with a #
compare: https://github.com/clj-kondo/clj-kondo/blob/d90e2073fef67f4b0f6d9eabba9fa50c5c9f95dc/parser/clj_kondo/impl/rewrite_clj/parser/core.clj#L181 and https://github.com/clj-commons/rewrite-clj/blob/59668f5875b5933c84844f1a1e0e6a3c3b77ac59/src/rewrite_clj/parser/core.cljc#L126
user=> (p/parse-string ";; foo")
<comment: ";; foo">
user=> (p/parse-string "#! foo") ;;=> <shebang "#! foo">tis the way of things, I like shebang for a name, but I’ll see if it is called something specific in the reader
Yeah, exciting that rewrite-clj is finally catching up with clj-kondo's fork after 2 years :P
Here is the old issue for clj-kondo: https://github.com/clj-kondo/clj-kondo/issues/294
I didn't feel like I was arguing with someone, because nobody really replied, except the guy who was the "victim" of string quoting
I'm strangely excited about this Windows shebang though. It's always nice to support some archaic niche use case
@UKFSJSM38, https://github.com/clj-commons/rewrite-clj/commit/187497a5d017ee21e81c674938f2427707807ca4, lemme know if it works for ya.
Nice, thank you very much! Just don't giving the exception will certainly work 😄 I'll make the change on clojure-lsp on next rewrite-clj release
@UKFSJSM38 btw, you mentioned that the integration tests were broken with the newest clj-kondo. could you follow up on this?
yep, for some reason the order of the elements of the list were not in the expected order with the new version, it's not a issue and it seems now the order is "more" correct, following the asc pattern of the elements positions
I didn't investigate that much, but I'll keep an eye if next clj-kondo releases impact that again
the version from earlier in march had a bug which reported unresolved symbols in the wrong order. maybe you captured this in an integration test
3
3@borkdude, just reviewing https://github.com/clj-kondo/clj-kondo/commits/master/parser/clj_kondo/impl/rewrite_clj, I don’t think there is anything else immediately relevant that we are not already tracking for rewrite-clj v1, do you?
did you also catch this one? https://github.com/clj-kondo/clj-kondo/commit/e6f6bf097072c0ad0f6f5ffec13379d9c79db4e2#diff-72baeadefb655c4c65b933bbaef4c62cbf6397c2d460fff9010b09d824c9de15
@lee I already had some patches around namespaced maps in the first commit, unfortunately I didn't specify that explicitly
I guess we could drop alpha. I would have liked more feedback around sexpr work around namespaced elements from real usage, but if it does not come naturally, then… can’t force it.
fwiw tree-sitter-clojure recognizes #! too: https://github.com/sogaiu/tree-sitter-clojure/blob/master/grammar.js#L40
@sogaiu! Nice to hear from you! I finally took the time to introduce myself to tree-sitters the other day. Very interesting!
Yeah, I found the smart error detection pretty darn awesome. Not exactly sure how it works but the effect is nice.
oh, where did you try that out? atm i don't think tree-sitter gives a grammar author the ability to tune how the correction works. consequently, i haven't found it to be so flexible for coping with broken code. the work that you all are doing on rewrite-clj, clj-kondo, clojure-lsp, etc. is much better for editor users from this perspective.
Oh @sogaiu, I am a total tree-sitter noob and have only read/watched introductory stuff. I thought that https://github.com/tree-sitter/tree-sitter/issues/224, but again have not dug deep at all.
yeah what you mentioned has a nice explanation (which i confess i do not understand that well 🙂 ) here is a more recent discussion that may be relevant for lisp-likes: https://github.com/tree-sitter/tree-sitter/issues/923 i've been working on trying to spell out specifically what one can do in the case of unbalanced delimiters. afaict, in general, when there is a missing closing delimiter, there are mutiple possible places it might go. if one can trust existing indentation i think this can be narrowed down a bit, but still hammocking / researching. anyway, sorry to have drifted off topic.