This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-07-23
Channels
- # announcements (1)
- # aws (13)
- # babashka (31)
- # beginners (102)
- # calva (46)
- # cider (16)
- # clj-kondo (1)
- # cljs-dev (3)
- # clojars (1)
- # clojure (396)
- # clojure-argentina (1)
- # clojure-australia (4)
- # clojure-europe (64)
- # clojure-nl (2)
- # clojure-uk (8)
- # clojurescript (20)
- # conjure (5)
- # cursive (4)
- # datomic (15)
- # emacs (48)
- # graalvm (69)
- # graalvm-mobile (1)
- # jobs (4)
- # jobs-rus (1)
- # lsp (6)
- # malli (15)
- # meander (2)
- # observability (11)
- # off-topic (10)
- # pathom (2)
- # portal (4)
- # re-frame (19)
- # reitit (1)
- # remote-jobs (3)
- # sci (1)
- # shadow-cljs (51)
- # tools-deps (11)
- # vim (12)
- # xtdb (13)
Yeah, I remember an impassioned talk at Clojure/West about the (poor) state of web security in the Clojure world... by Aaron Bedra back in 2014 https://www.youtube.com/watch?v=CBL59w7fXw4
A comment about SQL injection?
I would have thought clojure.java.jdbc/`next.jdbc` were the answer to that security issue since that's where parameterized statements happen but, yeah, HoneySQL probably contributes to making that easier too...
Korma is... ORM-ish... so I have always recommended against it.
I don't remember when we first started using HoneySQL but a former colleague gave a talk about it at Clojure/West... I want to say in 2015?
I remember Aaron’s talk, that was a good one.
(def all-root-dirs
(for [path (java.io.File/listRoots)]
path))
(= (take 10 (map file-seq all-root-dirs))
(take 10 (file-seq (first all-root-dirs))))/, as I’m on MacOS.)
I get my entire filesystem when I run (take 10 (map file-seq all-root-dirs)) but I get only 10 liked I asked when I run (take 10 (file-seq (first all-root-dirs))).
Is this an edge case where take doesn’t work on lazy sequences?@c.westrom Since file-seq returns a sequence, map file-seq is going to return a sequence of sequences. That's why the results are different.
Try (mapcat file-seq all-root-dirs) and see if that gives you what you want.
Hey, I am using core.cache and was wondering why the creation of lru cache with large threshold and no intial data was taking so long.
(require '[clojure.core.cache.wrapped :as cw])
(time (cw/lu-cache-factory {} :threshold 10000000))cc @seancorfield. Ccing you as you are maintainer of the lib.
Can you write this up on http://ask.clojure.org please and tag it with core.cache?
It'll be a while before I get time to cycle around to that lib in my OSS time and I didn't write that part (@U050WRF8X did) so I'll have to figure it out in detail -- or maybe Fogus will see this and answer here...
Will write it up. Can also try to do a patch if you want, but maybe @U050WRF8X can chime in first to confirm the issue.
@UL638RXE2 There's a signed CA on file for you?
@seancorfield https://ask.clojure.org/index.php/10849/issues-with-lru-and-lu-cache-in-core-cache
I signed the CA this morning. I don't know if you have the authority to add me to JIRA so I can propose a patch.
I don't, I'm afraid, and I know Alex is not around much for the next week or two so I think you'll just have to be patient, until that happens. It'll take me a while to analyze the problem anyway before I can even look at a patch, and I may have to defer to Fogus since he wrote the LRU and LU cache implementations...
I'll look into the CA and Jira thing this morning. Alex will be back a couple of days this week so I'll make sure to get his eyes on it if I don't happen to have the keys to that kingdom. 🙂
@seancorfield Ah, that makes sense. Thank you. I finally have a use for mapcat now lol.
I find the diagnostic messages from clojure.test confusing. For example the following assertion fails and gives the message below:
(is (= (rte/canonicalize-pattern-once '(:cat ::x ::y))
'(:cat (:cat String (:* String)) (:cat Double (:* Double)))))expected: (:cat Long (:* Long) Double (:* Double))
actual: (:cat String (:* String) Double (:* Double)) (is (= think that the lhs is the expected value, and the rhs is the generated value. Isn't that really arbitrary since = is a symmetric relation?I always find it confusing. in fact I'd find it confusing either way. to me it'd be better to say rhs and lhs as it very well might be that both values are generated, and I just want to assert that they're the same, but not assert which one is the correct value
it says nothing about which one is the correct value, it just says that after seeing the lhs it expected to see it again, but sees rhs instead
ah ha. I read the message as. this is the "expected" value as opposed to this actual value.
I wonder if it was written by a native English speaker?
perhaps it is a silly argument, because I can always just put a message string of my own there, and everyone will be happy.
not convinced.
clearly whichever one is the constant is the expected value.
Uhm, class on any value will return you something. Because you already have that value.
Replace (range 10) in my code with the same (map identity (range 10)) - which is the constant now?
you missed the point of what I said. a clojure.lang.LongRange is a value per se, so range 10 can be considered a constant
in the context of the conversation, the thing that needs the least computation is the "constant". The actual terminology is expected/actual and can reasonably apply in 99% of cases
there will be cases where it doesn't, which presumably is your point, but those are rare
exactly, if neither is constant, the neither is the expected value. it is just expected that the two generated values are equal.
> the thing that needs the least computation With that definition, can you expect anyone to actually implement that, given how little value-add it has and how much ambiguity in brings? :)
I don't know. Does an informal definition given by @U45T93RA6 affect the validity of a practice that has been around for 20 years? Feel free to research
This conventions is very common. For example if you see a function like assert_equals or similar in any language you’d expect it to be read from left to right, where left is the “expected” value.
> in any language
Not in Python. :P
unittest doesn't declare anything as expected - it's just first and second
pytest treats the second argument in e.g. assert x == y as the expected value.
apparently assertions go back to Turing: https://en.wikipedia.org/wiki/Assertion_(software_development)
At a risk of contributing to the off-topic-ness, when I’m coding in Java (which is far too much of the time at the moment), I like to use an assertion library like Hamcrest or AssertJ because they make it really obvious which is the actual value, and which is the assertion. This discussion has reminded me of the bad old days when all we had was JUnit’s assertEquals() and I always felt that expected and actual were the wrong way round.
clojure often has a strong notion of left to right sequentiality as well. For example (or 1 2) => 1, (or (do (println 1) 1) (do (println 2) 2)) => 1 1
(= (do (println 1) 1) (do (println 1) 1) (do (println 2) 2) (do (println 2) 2)) => 1 1 2 2 false
I doubt there is a language where or doesn't have this property. I know djikstra was big on random choice out of lists and some languages leave argument order undefined. but Clojure guarantees argument evaluation in order (edit: i believe. i see ghadi typing so perhaps i will be corrected)
Pretty sure there are a few languages that allow you to choose which behavior you want. I seem to recall Ada supports both short circuit and non-short circuit boolean forms.
do you know if the non-short circuit forms are arbitrary element of the or statement and not left to right?
non-short circuit could mean evaluating all the arguments left to right even after you have found a true one. doesn't scramble the order
> In the absence of short-circuit forms, Ada does not provide a guarantee of the order of expression evaluation, nor does the language guarantee that evaluation of a relational expression is abandoned when it becomes clear that it evaluates to False (for and) or True (for or).
I think there are others where ordering isn’t defined but can’t think of them offhand. Probably there are other languages where OR/AND are implemented as functions rather than primitive language features, or macros as in clojure.
haskell they are functions and laziness deals with the expression. i imagine anyone can write a version that shuffles the values
I fully agree with this. This is very expected under the assumption that we encode control.
it gets more interesting with maps:
{:a (do (println "a") :foo)
:b (do (println "b") :bar)}(macroexpand '(or (do (println 1) 1) (do (println 2) 2)))
=>
(let*
[or__5533__auto__ (do (println 1) 1)]
(if or__5533__auto__ or__5533__auto__ (clojure.core/or (do (println 2) 2))))If you wanted to play around with macros, you could write your own version of = that has the behavior you were expecting. There’s nothing special about or other than it’s written as a macro instead of a function, so it can control when its arguments are evaluated.
not entirely sure if it’s right! and I have to go 😮
(defmacro short=
([_] true)
#_([x y]
(= x y))
([x & args]
`(if (not= ~x ~(first args))
false
(short= ~(first args) ~@(rest args)))))seems ok at first glance:
(macroexpand-all '(short= 1 1 2))
=>
(if (clojure.core/not= 1 1) false (if (clojure.core/not= 1 2) false true))Does anyone know whether the clojure compiler does some optimization with map which might interfere with dynamic binding?
I have a function that looks like the following:
(defn conversion-cat-99
[self]
(rte/create-cat (map canonicalize-pattern-once (operands self))))canonicalize-pattern-once is a dynamic variable.
When I rebind, the variable, sometimes (it seems) the old value gets used inside conversion-cat-99 . When I try to debug this, the problem is reproducible until I redefine this function to the same thing as it already is, and the problem goes away. it is as if the compiler has inserted the function object into the code rather than the variable, so that new bindings are ignored.
Perhaps the problem is elsewhere, probably a bug in my code, but I've been searching for days and my evidence points to this.voila
negative - it's the value inside the var #'canonicalize-pattern-once that is passed as the argument to map, so any rebinding you do after that doesn't apply
hmmm.
you could do (map #'canonicalize-pattern-once ....) and then rebindings will be seen, but then you're subject to the laziness problem that @borkdude mentions
ghadi, If you're right I could rewrite it as:
(defn conversion-cat-99
[self]
(rte/create-cat (map (fn [re] (canonicalize-pattern-once re)) (operands self))))it's often better to pass the value of the dynamic var explicitly as an additional arg or part of a map
but laziness happens implicitly all over the place.
but there is a conceptual issue going on: the value inside the var is passed to map, not the var itself
and does that value extraction happen at compile time or at evaluation time?
if it happens at compile time then rebinding will have no effect.
WAIT A MINUTE!!!! maybe I see the problem.
My code basically looks like the following, very roughly
(declare f)
(defn conversion-cat-99 [self]
(rte/create-cat (map f (operands self))))
(def :^dynamic f (fn ...)
(defn g []
(binding f something-new)
(conversion-cat-99 ...))I believe that the #' optimization must have occurred because I used declare thus clojure knows its a var, but doesn't know it's dynamic. then when I redefine the function later, clojure knows its dynamic.
it's a theory. I can test it to see if that is the case
it's probably best if you made a repro that other people can actually run instead of guessing what you are doing
well I'm glad to discuss it here because I've been banking my head for days. and discussing it here, if I'm correct, it is solved in minutes.
I'm using dynvars to avoid exhausting the java heap.
doesn't really work to turn f into the argument of conversion-cat-99.
I don't think the problem is laziness. I think it is forward declaration. Here is a test case:
(ns jimka-test
(:require [clojure.pprint :refer [cl-format]]))
(declare f)
(defn g []
(map f '(1 2 3)))
(def ^:dynamic f (fn [x] (* x x)))
(defn h []
(assert (= (g) '(1 4 9)))
(binding [f (fn [x] (+ x x))]
(assert (= (g) '(2 4 6))
(cl-format false "(g) returned ~A" (g)))))
(h)the second time g is called, in the second assertion, f has been rebound. However, g was compiled before the compiler knew f was a dynamic variable.
EXACTLY!!!!
obviously the compiled code looks very different if f is a dynamic variable or a lexical variable.
I didn't know you could do that. But I'll give it a try.
do I need ^:dynamic on the declare and also the def ?
but +1 on borkdude's comment - make the dynamic var an argument to conversion-cat-99 instead
oops my assertion was wrong. I need (2 4 6) not (1 2 6) blush
that seems to work, indeed.
@borkdude, hi Michiel .... that's a really interesting static check to make. a declared variable which is later decorated as dynamic.
no, it cannot be made an argument to conversion-cat-99. It only appears that way because I've reduced the test case for the example.
it's the first time in 10 years I've seen this problem. why are you forward declaring this dynamic var in the first place?
the values of dynamic variables are memoized versions of global functions. it is not a good idea to edit all the functions in all the possible call chains to add 20 extra parameters for all the memoized functions.
why is it dynamic? so that I can effectively do the following:
(binding [f (memoize f-implementation)]
...)binding form finishes, all the memoized information about f is GC'ed.this is great for running 1000s of test cases and avoiding filling up VM will all the memoized information.
the pre-binding and post-binding of f in my case have the same semantics. just binding forces re-memoizing the values.
sounds like it could be a map with a bunch of symbols->fns in? which could be lexically scoped and gc'd that way?
the 2 hardest problems in computer science are naming, caching, and off-by-1 errors.
not sure what you mean by symbols->fns. but the functions are defined in different packages, thanks to the limitation that clojure only allows one package for UNIX file.
The suggestion is instead binding in a global environment, create a local mapping of names to functions and pass it as an argument
refactor 100s of functions to pass that argument around?
no, I think dynamic variables are in the language because they are useful. Embrace the power of lisp.
just as I've learned clojure lets your declare dynamic variables AFTER it is already assumed they are not dynamic, and it doesn't warn you. presumably a bug in the compiler.
but one I know about now and I can guard against.
Useful, yes, but mixed with laziness it is like gasoline in a highly pressurized, hot, environment. Powerful and useful when contained. Explosive otherwise.
indeed. it is an argument non-lispers use against many of the features of lisp. Oh that's dangerous. I've heard the argument for 30 years.
It also breaks referential transparency. While they're useful, it seems like Clojure users have drifted towards general avoidance of dynamic environment. If you look at older libs they make pervasive use of them. New ones, not so much
Paul Graham writes about it in Hackers and Painters
Stylistic arguments aside: it is definitely the case that typical Clojure code uses dynamic vars infrequently. Far less than you might see in some Common Lisp or even Scheme codebases. It is also the case that you'll lose out on https://clojure.org/reference/compilation#directlinking, which is a pretty big help in production.
@dpsutton what is :declared true :dynamic true intended to do? I don't understand the intriguing suggestion.
do correct me if I'm wrong, but doesn't this approach look a lot like with-redefs?
But declare just emits (simplified) (def ^declared your-name). and you know that to mark something as dynamic you just add the appropriate :dynamic metadata. So combine the two
Yeah, with redefs is kinda the tool for testing since it does the right thing across all threads
does (declare ^:dynamic a b c) declare only a to be dynamic or a, b, and c?
@lassemaatta I believe with-redefs redefines in all threads, which I want to avoid. if the test cases are run in different threads, I don't one one test redefining a function being used by another thread
There's no reason to intentional make your code non-thread-safe unless there's a good reason to do so. right?
It's definitely more common / idiomatic to do this using with-redefs, if it's only done under test. The reason to do this is so the compiler can do a better job with your code, when not under test. But you're right to be cautious; it's a heavy hammer that can have unexpected side effects if you're not careful.
Great discussion guys (and ladies) I've been hitting my head against a wall for almost a week with this issue.
discussing it here solves the problem in minutes. bravo!
I added a thread here. https://clojureverse.org/t/string-idiosyncrasy-of-the-compiler/7909
On the topic of dynamic scope, this was written by Stuart Sierra in 2013 https://stuartsierra.com/2013/03/29/perils-of-dynamic-scope
@jimka.issy I didn't ask why it was dynamic, I asked why you needed declare + (def ^:dynamic x). Why the declare?
but even then, I think it's worth refactoring those 100 functions. I've done it myself in a project where a datomic db was referenced using a dynvar. This didn't work, functions referenced the wrong as-of due to laziness.
ah why the declare. without declare you have to define functions in an illogical order. It is better, in my opinion, to define similar functions together, or functions would work on the same problem or which treat the same object. without declare you have to define functions AFTER their dependencies have been defined.
also you have to declare (as I understand) if you want mutually recursive functions or functions which call each other even in a non recursive way.
One thing I do wonder is, what is supposed to tell me that I have unnecessary declarations. I.e., declarations which might have been necessary at some time but may no longer be?
@jimka.issy If you're using clojure-lsp (recommended! #lsp) it will show you the number of references next to the var.
3what is clojure-lsp ? is that an emacs mode?
https://clojure-lsp.github.io/clojure-lsp/ is such a server supporting Clojure
sounds complicated
how does it relate to cider?
it sounds complicated but it provides you features that you don't need a REPL for, such as navigation, renaming, reference count, etc. it is not related to CIDER at all
clojure-lsp uses clj-kondo for static analysis but provides additional tooling on top of this
@jimka.issy this may help you understand it: https://emacs-lsp.github.io/lsp-mode/tutorials/clojure-guide/ It also explains that you can use CIDER together if you want it
I'm tempted to try it. Question: how much of of the feature set of emacs-lsp (and of clojure-lsp in general) depend on adherence to idiomatic use of the clojure language, and how much really are based on correct language semantics? I quite often violate idiomatic usage, expecting the language to work as documented, and find myself fighting with the IDE because of it. For example, I tried out cursive/intelliJ at one point, but abandoned it because it seems cider understood the language better, and required less idiomatic programming. I admit I entered that short experiment already biased toward emacs, so my conclusions may be dubious.
I'm looking at the pages about getting started. and links lead me here https://clojure-lsp.github.io/clojure-lsp/building/
Do I really need to install GraalVM ?
I tried to install lsp-mode from emacs using M-x package-install lsp-mode, and I got the error: package-install-from-archive: http://melpa.org/packages/lv-20181110.1740.el: Not found
@jimka.issy You will have the same behavior as clj-kondo pretty much with clojure-lsp because it uses clj-kondo for static analysis
But that also means that if you configure clj-kondo correctly, clojure-lsp will also work better for you
You don't need to build clojure-lsp yourself, you can install or download a pre-compiled binary.
> package-install-from-archive: http://melpa.org/packages/lv-20181110.1740.el: Not found
This may mean you have to run package-refresh-contents first
that did the trick. now installing lsp-mode installed hundreds of things.
I don't really understand the installation page. https://emacs-lsp.github.io/lsp-mode/tutorials/clojure-guide/. It is not clear what I need to install myself, and what M-x package-install lsp-mode takes care of for me.
@jimka.issy This is my personal config:
https://github.com/borkdude/prelude/blob/2466381f2cc438f9c01fcb413fd73bd56903175e/personal/init.el#L357-L410
You have to require lsp-mode manually or using the tool you normally use, e.g. use-package or emacs prelude
yes. did that.
This is what I see in the *Messages* buffer.
LSP :: Download clojure-lsp started.
LSP :: Starting to download to /Users/jnewton/.emacs.d/.cache/lsp/clojure/clojure-lsp.zip...
Contacting host:
You can run the command 'lsp-install-server' with M-x l-i-se RET
Contacting host:
LSP :: There are language server((clojure-lsp)) installation in progress.
The server(s) will be started in the buffer when it has finished.
Mark set or was there an installation error?
depending on your project and computer, it can take a while before it's finished indexing
here is what I see on the window decoration:
oh that's interesting, it's installing clojure-lsp automatically. I've never seen that
miracle?
the emacs gods are kind to me
I wonder how to know if it finished, or failed ???
@jimka.issy FWIW I use this line: https://github.com/borkdude/prelude/blob/2466381f2cc438f9c01fcb413fd73bd56903175e/personal/init.el#L376 to point to a binary I downloaded myself
and you downloaded the executable from where?
The line lsp-diagnostics-provider :none is atypical: it disables all clj-kondo linting. I do this because I run clj-kondo myself, most users just use it with clojure-lsp.
depending on your OS there might be package managers that can install it for you as well
brew install clojure-lsp ?
brew remove clojure-lsp
brew install clojure-lsp/brew/clojure-lsp-native
seems to be working.
I see something intriguing:
why does it think there are tests for some functions but not others? How has it determined that I have a test or not?
Also I see it has problems understanding my code: 😞
it says one one line that gns/or? has 0 references, and just below where gns/or? is referenced it says unresolved var.
no it is the clojure def.
higher in the file I have
(alias 'gns 'clojure-rte.genus)no, def can take qualified or unqualified names. defn only takes unqualified names.
and obviously defmethod takes qualified or unqualified names, else it would be impossible for other applications to add methods to a multimethod
user=> (def clojure.core/dude 1)
Syntax error compiling def at (REPL:1:1).
Can't refer to qualified var that doesn't existI had a long discussion about this somewhere. maybe closure verse, maybe clojurians. I don't understand why defn explicitly disallows such names, when defn is just a macro expanding to def which DOES allow them.
it may allow it, but it's probably not intended to allow it, but relying on an implementation detail that the var already existed before you ran this
not sure why you get that error. works fine for me.
$ clj
Clojure 1.11.0-alpha1
user=> (def clojure.core/dude 1)
Syntax error compiling def at (REPL:1:1).
Can't refer to qualified var that doesn't existwhich is the default namespace of the repl?
try it with (def user/dude 1)
because I have other functions of the same name in other name spaces, and I never want to accidentally confuse them. even when I refactor and move code around
I guess we could teach clj-kondo about it, although in my opinion this is very much a niche use case
As I mentioned before, my code sometimes depends on the semantics of the language despite commonly used idioms. I know maintaining a tool is difficult. However, in my opinion a tool like clj-kondo should implement the language semantics as much as possible/practical, not try to create a better language.
I agree with that, but there is a backlog, you know, and this is not my paid full time job, although I would very much like it to be. You're welcome to post an issue about it and eventually it will be solved.
I see this https://clojureverse.org/t/feedback-wanted-on-new-clj-kondo-macroexpansion-feature/6043/7
is that the article I should be reading about how to handle macros? Or is there a better resource?
@jimka.issy I've got a workaround
The hook:
(ns def-hook
(:require [clj-kondo.hooks-api :as api]))
(defn transform-def [{:keys [:node]}]
(let [[name-node & arg-nodes] (rest (:children node))
name-sym (api/sexpr name-node)]
(when-not (simple-symbol? name-sym)
(let [new-node (with-meta
(api/list-node
(list*
(api/token-node 'def)
(api/token-node (symbol (name name-sym)))
arg-nodes))
(meta node))]
{:node new-node}))))so do I need to create a .clj-kondo directory somewhere?
done. now do I need to restart something?
if that doesn't work, remove .lsp/sqlite.db and .clj-kondo/.cache and run lsp-workspace-restart
hmm. seems to work at first glance!
yipeee
question: at the top of the buffer emacs is now displaying some sort of path to the cursor. Sometimes is is underlined in a squiggly green line but sometimes in a squiggly red line. There doesn't seem to be any hover text telling me what this means. Any idea what it's trying to tell me?
I think this means there are some warnings or errors in the code. Personally I've turned this off
BTW I was trying to create an issue for this problem. But the new-issue template asks me lots of questions which I don't know the answer to. For example. the kondo version. I don't know how to find this. when I type the suggested clj-kondo --version at the shell, the command is not found.
also I don't know which editor plug-in I am using.
the version is mostly for reminding people that they probably should upgrade if they have an old version
Thanks for the quick workaround.
sorry, but I didn't understand your response about the latest-and-greatest macros documentation.
was that the link https://github.com/clj-kondo/clj-kondo/blob/master/doc/hooks.md. ???
usually here is an easier way to configure macros using :lint-as but :hooks can be used for more advanced macros that have no counterpart syntax-wise
reading the section in that file. It is not 100% clear. If I have a macro named xyzzy it seems I need to create a file hooks/xyzzy.clj in the .clj-kondo directory. and in that file define a namespace hooks.xyzzy and use defn to create a function named xyzzy . is that correct?
and then register the hook. with {:hooks {:analyze-call {my-lib/xyzzy hooks.xyzzy/xyzzy}}} where exactly?
actually, the name of the file and the name of the function in the file doesn't have to correspond to the macro
and this code must be able to run WITHOUT loading my project. correct?
so {:analyze-call {foo.bar/baz my-hooks/hook1}} is a valid config, but you have to name the file accordingly to this config
you cannot use any project dependencies in these hooks, it must be pure clojure (for now)
so can I just copy the macro code there and return macro-expand blah blah blah ?
well it looks like the function must return some sort of wrapper {:node insert-expansion-here}
but the argument of :node, is it just plain old clojure expression? or is it somehow decorated ?
as long as you re-use most of the incoming nodes and re-arrange them syntactically in a way that makes sense
but isn't there already a kondo function which takes such an sexpression and returns such a data structure?
kind of, but clj-kondo really needs the location information as metadata on the nodes in order to produce useful diagnostics
ok, I admit that I don't yet see the final solution. but shouldn't there be an approximation function where I can just return the macro expansion? and let all the annotation information simply go onto the macro-name in the user code, which clj-kondo already knows the location of?
that question comes up more often. I have tried this when I implemented these hooks, but it breaks down rather quickly.
am I thinking to naively?
(defmacro foo [x] x)
(inc (foo 1))(inc (foo :foo)) one would like to have a type mismatch warning that you can't call inc with a keyword.I had another idea. try this out. The first time clj-kondo encounters a new use site for a given macro. it flags it as unknown macro usage. Then provide the user with a way to just do a macro expansion. Then clj-kondo could register that expansion statically with the sexpression being expanded. if it ever finds the exact same macro usage, it uses the same expansion. it would need to save the correspondance between in/out somewhere.
But when transforming the node that represents (foo :foo) to a s-expression, there isn't a way to hold on location information anymore for the keyword, as keywords don't take metadata. Thus, the transformation of nodes into s-exprs is lossy and doesn't fit well with how the static analysis works.
This is only a small example, but for macros that take a body representing some function, it becomes more problematic.
I don't doubt that there is a subtle and difficult problem. just it is hard for someone who doesn't understand the code to understand the problem.
Of course one could try to "repair" the transformed s-expression into a node and try to detect which location corresponds to an original node, but this is not trivial.
when trying to re-calculate the location, why not just assume it is exactly at the position of the macro name. foo in this case, even if the macro call is 100 lines long?
isn't that better than giving false errors?
The node representation is based on rewrite-clj. https://github.com/clj-commons/rewrite-clj Anyone who knows a bit about this library understands how to write hooks. I agree that direct macro-expansion is more ideal, but this wasn't possible without negative effects.
you mean if there are side effects in macro expansion?
Yes, one could attach all warnings to the original top level node, that could work, but is less precise. And yes, macro expansion would only work if it was pure Clojure, without any library code in the compilation phase.
that would work for 99% of my macros. maybe 100%
well, 99%. I have one macro which is a phd thesis.
wouldn't work for that one 😞
in summary: give the user a hook where you pass him the macro body from the call site, let him expand it and return the expansion, and you annotate everything as if it is at the open-paren of the macro-usage.
I'll be your test case.
I think you would have to walk the expansion to annotate it with the location of the outer expression
BTW lsp-mode has a automatic installation of clojure-lsp feature that downloads latest release binary :)
@jimka.issy Can you give me one of your macros + one example call?
I tried to go with the def one but that case doesn't work, since a call to clojure.core/def expands into cljoure.core/def which will loop forever. This is also a problem that hooks solve: returning no node, just means that the original node will be processed instead.
(defmacro defn-memoized
[[public-name internal-name] docstring & body]
(assert (string? docstring))
`(let []
(declare ~public-name) ;; so that the internal function can call the public function if necessary
(defn ~internal-name ~@body)
(def ~(with-meta public-name {:dynamic true}) ~docstring (gc-friendly-memoize ~internal-name))
))(defn-memoized [sort-method-keys sort-method-keys-impl]
"Given a multimethod object, return a list of method keys.
The :primary method comes first in the return list and the :default
method has been filtered away."
[f]
(cons :primary (remove #{:primary :default} (keys (methods f)))))
(defn-memoized [class-primary-flag class-primary-flag-impl]
"Takes a class-name and returns either :abstract, :interface, :public, or :final,
or throws an ex-info exception."
[t]
(let [c (find-class t)
r (refl/type-reflect c)
flags (:flags r)]
(cond
(= c Object)
:abstract
(contains? flags :interface)
:interface
(contains? flags :final)
:final
(contains? flags :abstract)
:abstract
(= flags #{:public})
:public
:else
(throw (ex-info (format "disjoint? type %s flags %s not yet implemented" t flags)
{:error-type :invalid-type-flags
:a-type t
:flags flags})))))here's an easier one
(defmacro exists
"Test whether there exists an element of a sequence which matches a condition."
[[var seq] & body]
`(some (fn [~var]
~@body) ~seq))(defn conversion-C3
"(and A ( not A)) --> SEmpty, unit = STop, zero = SEmpty
(or A ( not A)) --> STop, unit = SEmpty, zero = STop"
[td]
(if (exists [n (operands td)]
(and (gns/not? n)
(member (operand n) (operands td))))
(zero td)
td))@jimka.issy I have a prototype working now
also you can use https://github.com/borkdude/carve to detect unused vars
@jimka.issy If you're struggling with reading clojure.test's (is (= <expected> <actual>)), you might prefer expectations.clojure.test which is 100% compatible with clojure.test but lets you write (expect <expected> <actual>) instead -- and also lets you write things like (expect ::my-spec <actual>) to validation against a Spec or (expect <predicate> <actual>) to validate using a predicate -- https://cljdoc.org/d/com.github.seancorfield/expectations/
funny thing. I had a test which was taking a long time to run. I spend some time investigating why the function being tested was so slow. After a while, I looked at the actual test, and it had a repeat loop generating 100000 samples. It really needed 1% of that amount. sometimes the bug is not where you think it is.
Helped a colleague optimize some algorithm he implemented in Clojure once. While I did speed it up significantly, an equal amount of speedup was gained by moving computation from run-time to when the data structure was built.
That's a great advantage of having the full power of the language at compile time.
This specifically was just done by shoving computation after reading configuration, so it was still in the running application and not "compile time", but yes, having entire language at compile time, and compile time being all the time is very powerful
has anybody here tried clojure with the new zgc collector on Linux? Any aberrations or unexpected memory explosions?
yeah I was wondering what the difference was... shenandoah is still single-generation mark sweep, but zgc does crazy things with memory regions (each memory cycles as from-space -> to-space) and tagging pointers
linux only however. not sure why that is but i suspect if I reread the docs it would be enlightening
> It scales from a few hundred MB to TB-size Java heaps, while consistently maintaining very low pause times—typically within 2 ms
the most important part though is your worst case pauses (see: the tail at scale, a google paper). zgc and shenandoah are looking at like... 10ms, maybe 20ms at most worst case pauses, and those are your 99th percentile pauses
the heap stuff is a reason I'm interested, I found out recently that Go has some pathologically bad behavior for small heap sizes... and big ones!
the way all concurrent collectors do it is to pace the collector thread(s) with the allocs. so the more you alloc, the more you collect, and one kind of follows the other. the disadvantage (with any concurrent collector) is that you lose throughput. but that's the tradeoff for being able to satisfy those soft realtime constraints
the most important part though is your worst case pauses (see: the tail at scale, a google paper). zgc and shenandoah are looking at like... 10ms, maybe 20ms at most worst case pauses, and those are your 99th percentile pauses
> having a GC per "process" is just such a cute hack super agree. Google tried to do something similar to this and the generational hypothesis where they were like "most garbage is scoped to the lifetime of a request" and tried to do something they were calling "request oriented garbage collection". but then it turned out it slowed down most of their go programs so they killed the idea
I think the best I saw was a person trying to hardware accelerate the GC, I thought that was an interesting avenue
I bet for a highly concurrent clojure application though, with true independence between requests, and because data is immutable, you would get a LOT of mileage out of per-request nurseries. fast bump allocs, good cache coherence and you have a better idea of when the objects are going to die (end of the request). would be an interesting experiment
