Fork me on GitHub
deadghost01:10:25 > Note that parent generator (in the spec or overrides map) will supersede those of any subtrees. Is there a better way to have a custom generator in a subtree than doing nested gen overrides?

;; Example

;; We love wasteful packaging so we want to create a ::crate that will always
;; contain a ::big-box. We want each ::big-box to contain one or more
;; ::small-box. In each small box, we want to have one Light Bulb.

(s/def ::crate (s/keys :req-un [::big-box]))
(s/def ::big-box (s/coll-of ::small-box :kind vector?))
(s/def ::small-box (s/coll-of string? :kind vector?))

(let [gen-override
       ;; Override so ::big-box is not empty
         (s/gen ::big-box
                ;; And that each ::small-box in ::big-box only has a Light Bulb
                {::small-box (fn [] (s/gen (s/coll-of #{"Light Bulb"}
                                                      :kind vector?
                                                      :count 1)))}))}]
  ;; Generate a ::crate with our nested overrides
  (gen/generate (s/gen ::crate gen-override)))

;; =>
 [["Light Bulb"]
  ["Light Bulb"]
  ["Light Bulb"]
  ["Light Bulb"]
  ["Light Bulb"]
  ["Light Bulb"]
  ["Light Bulb"]
  ["Light Bulb"]]}

Tom Helmuth01:10:37

Not sure if this is the right place to ask this, but here's an oddity that greatly confuses me.

Tom Helmuth01:10:59

I understand what is going on here:

> ('foo {'bar 5 'foo 3})

Tom Helmuth01:10:59

But what is up with this behavior:

> ('* 3 2)

Tom Helmuth01:10:43

Is it trying to look up the symbol '* in the "map-like thing" 3, and when it doesn't find it, returns the default value of 2?


Yes, that is the explanation for that behavior.

Tom Helmuth01:10:08

Why is it happy to treat 3 as a map-like thing without throwing an error? I guess that's the part that really baffles me.


I do not know the reason, but I think it is for the same reason that (get 3 '* 2) is happy to treat 3 as a map-like thing.


It has been that way in Clojure I believe since its beginning, and a JIRA bug was files in 2012 suggesting such expression throw an exception. Rich Hickey responded in 2014 that such a change would be a breaking change, and the Clojure core team values backwards compatibility very highly. I know that doesn't explain why it was like that in Clojure's original versions -- again, I do not know the reason, just giving you a little bit of history that this wasn't a recent thing, nor can I find anything in Clojure FAQ nor entry for 'get' that gives a reason:


If it's any consolation, I think pretty much everyone new to Clojure asks this... 🙂


Symbols and keywords look themselves up in their argument and it's actually very convenient when writing generic code because you don't have to worry about type checking when you simply get nil from an "inapplicable operation".


It feels like this question should be added to in some form -- @U017SU36TG9 do you think you could formulate your question in a form suitable for that page? I'd be happy to help turn it into a contribution for that page (I just recently contributed the ?/`!` FAQ there).

Tom Helmuth02:10:24

Sure, I'd be happy to.

Tom Helmuth02:10:27

FWIW, I've been using Clojure for 10 years and just discovered that (get 3 5 :default) is happy to treat 3 as a map-like thing. @U0CMVHBL2 I appreciate the historical context!


Feel free to DM me to chat about it (tomorrow -- it's a bit late tonight for me to coherently process that).

Tom Helmuth02:10:35

Looks like this will happen with any type of value that isn't a thing that expects to be used for lookup. My student who ran into this issue found the source code, and if none of those conditions are met, it just returns the notFound value:


Yup, it's a very deliberate design decision: if the type is something you could "get" a thing from, attempt the lookup (and return notFound if it isn't there), else return notFound.


But it surfaces in a number of expressions and that's the part I'm not sure how to formulate as a "frequently asked question".

Tom Helmuth03:10:05

The crazy part is that it makes the one-character typo of ('* 3 2) behave in a totally unexpected way. I totally get the two parts making sense independently (symbols can be used as functions for lookup and get returning notFound if you can't get from that type), but the combination is lethal.


As you can see from the JIRA issue, at least some people wish that get being called on a non-associative first arg were considered lethal (i.e. throw an exception instead of returning nil) all by itself, but that is unlikely to ever be changed, given backwards compatibility, plus whatever the original design reason was.


every time I see this familiar question, i remember

(get get get get)  
;;=> #function[clojure.core/get]


@U04V70XH6 Perhaps the frequently asked question might be "Why do several ways of looking up a key in an associative collection return nil when given something that isn't an associative collection?" starting with examples of both the get and symbol-as-first-form-in-parens expressions?


Granted that 'get' and the "symbol lookup as first form within parens" don't have good strings I can think of that people would be Google-searching for when they have this question, but folks like us who know what FAQs exist would know it was there on that page somewhere.


I wouldn't be surprised if Rich Hickey himself may have addressed the "why is it designed this way?" question around 2009-2010 in the Clojure Google group, but I didn't find such a thing in 10 mins of searching, again given the difficulty of finding the right search terms.


This old thread doesn't answer the question, but Chouser gives his quick rules of thumb for choosing between (key-or-symbol some-map) versus (some-map maybe-a-key) versus (get some-map maybe-a-key) alternatives.


Ah, found one where Rich himself addresses the question of get returning nil vs. throwing an exception!


It's a bit of a "throw away" reference to Common Lisp's behavior tho'...


Understood it doesn't explain with a lot of words ... Haven't seen anything more than that yet, though.


And I am sure that no matter what quotes I might find, any new FAQ will be reviewed by Alex and perhaps even Rich to avoid giving incorrect reasons/rationale.


FYI +10 from me if you do end up creating a new FAQ entry for this. It is slightly above my activation energy threshold to create one, but interested to see how it turns out.


And those 10 points are redeemable, you know, in all the usual places 🙂


A beer at a conference sometime in the post-plague era? 🙂 🍺


Sure. I could probably also arrange to have something from a place like Total Wine & more deliver to your doorstep 🙂


So far the only chain of stores in USA that I have found carries Rauchbier from Bamburg, Germany. An acquired taste.


I've been ordering a lot from Belching Beaver in So. Cal. Lots of fruity sours, if you're into that (I'm not, but I love their other stuff). Also, I've clipped this entire convo into OneNote so I can use fragments of it to distill down to an FAQ at some point. Thanks everyone!


And looking more carefully at the last Clojure Google group thread I linked above, it appears Rich is only addressing the question of get returning nil vs. throwing exception when the key is not found. I don't think it addresses the issue of throwing exception when the collection is not an associative thing.

Tom Helmuth01:10:22

This just caused some massive headaches when accidentally wrote '* instead of *', but luckily stumbled into the problem.

ryan echternacht13:10:06

I’m looking to build a “job queue”. It’d poll a db for work (I’m doing that with in java), and then fire off work. The work is mostly file imports, that are safe to be run concurrently. What’s the best clojure threading primitive for this type of work?

ryan echternacht13:10:39

I care that jobs finish, and their finishing status. but i’m not sure how much more I really care about them while running.

ryan echternacht13:10:46

I could see using core.async for it, but I don’t think I have that much of a pipelining need. Futures seem like they might work, but almost seem to barebone — I feel like I’ll be polling them for status, whereas I’d rather react to when they’re done.


It’s not so much polling so much as iterating over a vector of go-loop channels doing a blocking read from each, which will stop at the first unfinished go-loop, “forgetting” loops as they finish. I use core.async in cases like this even without a pipelining requirement. fwiw.

ryan echternacht00:10:06

I’m working through building it with core.async, and i’m pretty happy. I think I had the mental block that core.async was only for “big” things, but that’s a bad model

🍺 3

@ryan072 I borrowed this bit of code from CLJS to run an amount of tasks in parallel (to fully utilize CPU cores) and wait until they're done: Not sure if that's similar to your problem

👍 3

Each of the parallel tasks pick work from a queue, until there is no more work, then they lower some countdown latch.

ryan echternacht13:10:09

thanks, that does help. Gives me some confidence on rolling my own, and a some good java primitives to look into


HI, I've got an error saying java.lang.NoClassDefFoundError: manifold/bus/IEventBus when running my backend service using uberjar profile, which configed in project.clj as :profiles {:uberjar {:main hello-ring.main, :aot :all} . Finally I found the reason, the class loader of manifold/bus/IEventBus`` has a hierarchy as below

#object[clojure.lang.DynamicClassLoader 0xc7aac7c clojure.lang.DynamicClassLoader@c7aac7c]
#object[clojure.lang.DynamicClassLoader 0x2186d797 clojure.lang.DynamicClassLoader@2186d797]
#object[sun.misc.Launcher$AppClassLoader 0x23fc625e sun.misc.Launcher$AppClassLoader@23fc625e]
#object[sun.misc.Launcher$ExtClassLoader 0x50313382 sun.misc.Launcher$ExtClassLoader@50313382]
While the class loader of the AOTed class of my business code is #object[sun.misc.Launcher$AppClassLoader 0x23fc625e "sun.misc.Launcher$AppClassLoader@23fc625e"] , so it can not find the class which is loaded by child class loader(`#object[clojure.lang.DynamicClassLoader 0xc7aac7c clojure.lang.DynamicClassLoader@c7aac7c]`). Any one have an idea about this issue?

Alex Miller (Clojure team)14:10:10

I think you'll need to share more to understand what you're seeing. where do IEventBus and the impl come from? how is this app run? are you using java -jar with the uber jar to launch that main class?


I got this error either run the program through java -jar or through repl .

Endre Bakken Stovner18:10:33

I'd like to get the line and column number of every element in the below data structure. {:a {:aa 2} :b {:ba 0 :bb 3}} So the first { (the whole map) has coordinates 0 0, the :a has 0, 1 {:aa 2} has 0, 4, 3 has 1, 15. Is there anything in clojure or the ecosystem that will help me do this?


Not directly, but you could write a macro for that that analyzes &form and its metadata:

user=> (defmacro form-meta [] (meta &form))
user=> (form-meta)
{:line 1, :column 1}

👍 3

My approach would probably be use to use Would you need a need it to return a new data structure or adorn the passed in data?


oh wait, you're looking for line and character positions?

Alex Miller (Clojure team)18:10:33

if you use a LineNumberingPushbackReader to read it, you can get some of that from the reader (in metadata)

👍 3
Alex Miller (Clojure team)18:10:55

the maps can carry metadata but keywords and numbers cannot so you won't get those

Alex Miller (Clojure team)18:10:32

I believe there are other tokenizing parser libs for Clojure that can though, just not sure which is the best recommendation


But keywords can't have metadata, so any method that relies on it to return position information won't tell you where the keywords are

Endre Bakken Stovner19:10:45

I wonder if any libraries are able to work around that limitation in some way? I will have to investigate tomorrow.


The only way I can think of to get position information for things where Clojure does not support metadata, e.g. keywords, numbers, strings, is to return NOT a Clojure collection that is equal to what clojure.edn/read would return, but instead to return some kind of 'parse tree' representation that had an explicit line/column data for each object of the parse tree.


For example, if when reading {:a 2} you tried returning a value x such that (= x {:a 2}) was true, then there is no place "inside" of there to put the line/column info of the number 2. If you instead return a value y such that (= y {:a 2}) was false, you have many choices for how y represents the string that it read and its parts, and many choices for how the line/column info is embedded in there.


@U04V15CAJ wrote very recently, so should have fresh on his mind whether it supports line/col info on all sub-parts of the returned data, or not.


@UT770EY2K rewrite-edn is based on rewrite-cljc and rewrite-cljc can give you that metadata There is also which will parse directly to sexprs with adding location metadata when possible. Using a post-process step you can also have metadata on numbers etc when you coerce them in something that can have metadata. See this test:

Endre Bakken Stovner06:10:26

Thanks for all the replies so far. Perhaps I should explain my use-case. I want to write a highlighter (like for so that I can see what result my specter-expression returns in real-time. Then it would be neat if I could annotate every value in the data structure I want to query with line/column info. Then a specter highlighter could merely: 1. annotate every value 2. run specter as usual 3. pick out the line/column from the returned results afterward But if one cannot annotate keywords/numbers this approach won't work. I guess I can use the edamame wrapper-approach, but then I would need to convert all specter-queries for numbers/keywords into a specter-query for wrapper(number/keywords). If you can think of simpler strategies please advise.

Endre Bakken Stovner06:10:48

I was unable to see how rewrite-cljs can help me with the above, but many of the linked to pages in their docs were 404s.


I'm trying to extend this interface: like so:

  (completion [position]))
but I'm getting:
Can't define method not in interfaces: completion
And I'm confused why, because it's definitely there!


Apparently I wanted proxy, my bad!


Did you need proxy because of the default method?


I think Alex got it, it's because I forgot this


Ah! Thanks.

Alex Miller (Clojure team)20:10:19

the completion method in your reify should take 2 args - a "this" arg and then the args on the method

Alex Miller (Clojure team)20:10:51

  (completion [this position]))

Alex Miller (Clojure team)20:10:23

you probably don't actually need that arg, so fine to replace this with _ in that case


Oh, I didn't know clojure-lsp also used Lsp4j, convenient as a reference!


I was going to take a stab at a runtime-LSP for Clojure, but LSP4J feels like the worst of Java 😁 Maybe I'll leave that for a more determined Dominic in the future.


Someone asked me a beginner question that turned out to be deeper than I expected.

(defn a [l]
  (prn "a:" (type l))
  (prn "a:" l))

(defn b [l]
  (prn "b:" (type l))
  (prn "b:" l)
  (a l))

user> (b (list 1 2))
"b:" clojure.lang.PersistentList
"b:" (1 2)
"a:" clojure.lang.PersistentList
"a:" (1 2)
b evaluates (list 1 2) and it becomes the list (1 2), which it passes to a. a is a function, and functions evaluate their arguments, and lists are evaluated as function calls. Why doesn't a evaluate the list (1 2) and throw an error (given that 1 isn't a function)? I'm sure there's a simple answer to that, but damned if I can come up with it right now.


From having implemented small lisps, I can handwave about (1 2) having already been evaluated into an data structure in the host language, but that feels a bit shaky to me.

Jan K22:10:27

The list (1 2) is not getting evaluated anywhere in the code. When a is called its argument, the symbol l is evaluated, resulting in the list.


b does not evaluate (list 1 2). the repl does


There are two kinds of evaluation. One is "what does this mean" and the other is "what value does it have". The reader does the former, a function call does the latter.


I would say the reader does not eval and neither does a function call (unless that function is eval)


I think I agree on the former but not the latter. A function doesn't evaluate its arguments. But a function call does IMO. But maybe we mean a bit different things by "a function call".


That’s the doc I’ve been poring over. “Non-empty Lists are considered calls to either special forms, macros, or functions. A call has the form (operator operands*).” “If the operator is not a special form or macro, the call is considered a function call. Both the operator and the operands (if any) are evaluated, from left to right. The result of the evaluation of the operator is then cast to IFn (the interface representing Clojure functions), and invoke() is called on it, passing the evaluated arguments.” It seems like b is receiving a non-empty list, so it should be evaluated and therefore treated as a function call.


> There are two kinds of evaluation. One is “what does this mean” and the other is “what value does it have”. The reader does the former, a function call does the latter. I’d be interested to see where that’s documented; I don’t feel like I’ve heard of that before.


> A function doesn't evaluate its arguments. But a function call does IMO. But maybe we mean a bit different things by "a function call". I guess that sounds right based on the reference page.


That quote above about non-empty lists talks about list expressions, not values.


It talks about "what it means", not "what its value".

Jan K22:10:23

> Non-empty Lists are considered calls That is literal lists, but there is no literal (1 2) in the code.


(def a ())
Here, a means "a symbol that denotes a var that has a value of a list". It's different from () that means "a list".


Right, "literal" is a good word, although that evaluation page linked above doesn't mention it.


I’m curious whether you all see yourselves as saying the same thing in different words, or whether you think there are real disagreements there. I’m certainly not sure myself. Some of the statements above seem wrong to me, but I may be misinterpreting them.


I definitely don’t feel like I’ve got this nailed down to the point where I can explain it clearly to a beginner.


(list 1 2) evaluates to a data structure that looks like (1 2) -- and that's it: it's just data at that point. The call (a l) is calling a and passing in a data structure.


Personally, I don't feel like I have any authority here. So feel free to use my ramblings to adjust your mental models but don't really rely on them. I still feel like Alex Miller is about to come and correct us all at the same time. :) Oh, or Sean Corfield. 👋


There's no additional level of "evaluation" going on because in (a l) the argument is just data already.


The evaluation there is that the local symbol l is "evaluated" (looked up) to get its value, which is a data structure.


That seems like what @U2FRKM4TW is getting at with the expression/value distinction above.


Or, to put it another way: (b (list 1 2)) evaluates its argument (list 1 2) to get a data structure (1 2) which is passed into b (a l) evaluates its argument l to get the data it is bound to -- the data structure (1 2) which is passed into a


Evaluating a symbol means "look up the symbol's value" -- but there's no "evaluate the thing that the symbol is bound to".


If you had (a (eval l)) then, yes, you'd be evaluating what l is bound to... and that would then evaluate (1 2) and fail...


(let [l (list 1 2)] (println (eval l)))


I think that makes sense to me. It certainly fits with “The expressions exprs [of a fn] are compiled in an environment in which the params are bound to the actual arguments” (from the special forms


After five years or so of Clojure I’ve gotten to the point where I rarely get confused by any of the specifics (writing macros, obscure core functions), but I periodically go through a fresh stage of confusion about the most basic things 😜


Heh, like noticing how exactly you're breathing. Or walking.

☝️ 3

Thanks, y’all!


Hi all. Does anyone know why behaves this way? I have this situation where some keys contain multiple slashes. In the first example below, the resulting map is tagged with #:a which appears to come from the key, and the resulting key is missing the a/ prefix. This only happens when the JSON object contains only one key with multiple slashes. In the second example though the object contains two such keys and the behavior changes, it doesn’t add any tag to the resulting map and preserves the form of the original key.

( "{\"a/b/c/d\":1}" :key-fn keyword)
;; => #:a{:b/c/d 1}

( "{\"a/b/c/d\":1,\"e/f/g/h\":2}" :key-fn keyword)
;; => {:a/b/c/d 1, :e/f/g/h 2}
The same behavior can be observed with Cheshire.
(cheshire.core/parse-string "{\"a/b/c/d\":1}" true)
;; => #:a{:b/c/d 1}

(cheshire.core/parse-string "{\"a/b/c/d\":1,\"e/f/g/h\":2}" true)
;; => {:a/b/c/d 1, :e/f/g/h 2}
Interesting though is that keyword does what I expect, it doesn’t cut off the key’s prefix, so I suspect the behavior described above is caused by something else.
(keyword "a/b/c/d")
;; => :a/b/c/d
Is there any way to prevent the key from being parsed without the prefix, as in the first example?

Lennart Buit23:10:26

In your first example, it appears to be a keyword with namespace a and name b/c/d


@UDF11HLKC At first I was surprised to see that the parser was taking off the a as a namespace and using it to tag the map. But that doesn’t explain the second example’s results.


this isn't about the parser(s). it's just how the map is being printed out:

user> {:a/b/c/d 1, :e/f/g/h 2}
{:a/b/c/d 1, :e/f/g/h 2}
user> {:a/b/c/d 1, :a/f/g/h 2}
#:a{:b/c/d 1, :f/g/h 2}


the prefix is just a shorthand

Lennart Buit23:10:14

Well the #:a{...} syntax denotes all keys share the namespace :a, in your second example it can’t shorten it because there is no common prefix

👆 3

the key is still the same

user> (get {:a/b/c/d 1, :a/f/g/h 2} :a/b/c/d )


Hah! Interesting.

Lennart Buit23:10:33

(map namespace (keys ...)) out of the top of my head will tell you what the namespace parts of the keywords are :)



(let [m (cheshire.core/parse-string "{\"a/b/c/d\":1}" true)]
  (keys m))
;; => (:a/b/c/d)


I know that you didn't really ask about it but perhaps you might want reconsider keywordizing such maps where keys contain special symbols like /. Otherwise, it increases the potential to make things worse down the road.

👆 3

That’s comforting. Now I need to find out why the key :a/b/c/d is getting stored as b/c/d in mongodb.


Oh no. That's exactly what I'm talking about. :)


Because it uses name, I bet.


@U2FRKM4TW Totally agree. Unfortunately in this particular case I cannot avoid that. 😞

Lennart Buit23:10:14

Hmm? Why do you need to keywordize while parsing?


@UDF11HLKC It’s a huge map that comes thru a REST endpoint. 99% of the keys are just your good old normal keys but there’s this little map deep inside which contains those bizarre keys with slashes.


I guess I’ll have to give that little map a special treatment—i.e.: converting those keys to strings—before storing in mongo… ugh!


Or just don't keywordize the keys, separate the things that need to be keywordized from the things that don't need that, and keywordize the former manually.

👍 3