Fork me on GitHub

Hi all - with deps.edn, is there a command to pull in remote deps (and their deps) without actually invoking a test or exec-fn/main? I see clj -X:deps prep mentioned here ( - something that is equivalent to npm install in nodejs land?


Prepare clojure [clj-opt*] -P [other exec opts]


i always have to look it up in clj --help


its at the top in the "Usage" section



👍 1

Note that if using -P with other execution option flags, e.g. -M -X or -T , usually to include more dependencies, then -P should come before those flags

clojure -P -X:env/test:package/uberjar

Alastair Hole15:09:44

Is there a recommended way to get clojure-lsp/unused-public-var to not complain about top level public defn? i.e. they are unused because they are just exported automatically from the ns and used elsewhere


something sounds off. lsp should be aware of the other callsites and consider it used. Maybe you could ask for some help in checking configuration in #lsp?

Alastair Hole15:09:26

Ah I see, that’s good to know, thanks. I suspect then it’s due to the fact that it’s used via gen-class and then passing the class name to the caller (the Kafka Java library). Maybe I can add an exclusion just for that ns


oh yeah. it won’t get any runtime locations like that


there’s an annotation you can add to suppress the unused warning


#_{:clj-kondo/ignore [:unused-binding]}

Alastair Hole16:09:42

Ah cool I can do it for the whole ns in config.edn

:linters {:clojure-lsp/unused-public-var {:exclude #{my.ns}}}


@U11BV7MTK The`:unused-binding` rule can only be set in a config file. > I don't think it's possible to support this easily since the unused publics vars are linted outside of the namespace. - @U04V15CAJ >


it works for me


note the unused binding on the first form and no warning on the second with the annotation


Sorry, I meant to say unused-public-var, which is what OP asked about, and which I linked to a relevant issue. Does the unused-binding rule somehow solve @U02UHLGQ0BE's problem?

Alastair Hole16:09:26

I didn’t realise you could set rules in ns which is good to know, and even better to know that the rule I would need to use won’t work! 🙂 excluding the module by name in the config.edn works for me


good point @U90R0EPHA. I get lost between the two sometimes (clj-kondo and lsp)


I've been exploring transducers more and more and was curious about the different approaches. I took a small little coding challenge I had which was "Given an integer n, count the total number of 1 digits appearing in all non-negative integers less than or equal to n." My typical approach to something like this was:

(defn n->digits                                                                                  
  "ex. (n->digits 14) => [1 4]"                                                                  
  (if (< n 10)                                                                                   
    (conj (n->digits (quot n 10)) (rem n 10))))                                                  
(defn number-of-ones [n]                                                                         
  (->> (range (inc n))                                                                           
       (mapcat n->digits)                                                                        
       (filter #(= 1 %))                                                                         

;; (number-of-ones 14) ;; 7
And my various attempts at converting to using transducers are here:


(defn number-of-ones-into [n]                                                                    
  (let [xf (comp (mapcat n->digits)                                                              
                 (filter #(= 1 %)))]                                                             
      (into [] xf (range (inc n))))))                                                            
;; (number-of-ones-into 14) ;; 7                                                                 
(defn number-of-ones-transduce [n]                                                               
  (let [xf (comp (mapcat n->digits)                                                              
                 (filter #(= 1 %)))]                                                             
      (transduce xf conj (range (inc n))))))                                                     
;; (number-of-ones-transduce 14) ;; 7                                                            
(defn number-of-ones-eduction [n]                                                                
    (into []                                                                                     
          (eduction (mapcat n->digits) (filter #(= 1 %)) (range (inc n))))))                     
;; (number-of-ones-eduction 14) ;; 7                                                             
;; (time (number-of-ones 10000000))           ;; Elapsed time: ~8.5 seconds                      
;; (time (number-of-ones-into 10000000))      ;; Elapsed time: ~4.5 seconds                      
;; (time (number-of-ones-transduce 10000000)) ;; Elapsed time: ~5 seconds                        
;; (time (number-of-ones-eduction 10000000))  ;; Elapsed time: ~4.6 seconds 


I'm curious about the different pros and cons to using the into, transduce, and eduction approaches. Is it just a matter of taste and readability or is there something else I should be taking into account here? The different transducer approaches seem to be all almost twice as quick which is what I've read is sort of typical for such things.


eduction seems to be creating a lazy sequence out of a transducer process. Am I thinking about that correctly? I haven't quite figured out in which scenarios you would want to go back to laziness if you are already using transducers (which is more of an eager approach, right?). So in my problem above, after using eduction I had to go back to using into [] to realize it so I assume eduction would not be a proper approach in this particular problem.


from the eduction docs:

Returns a reducible/iterable application of the transducers
  to the items in coll.
It's lazy in a sense, but it should be faster and produce less garbage the the lazy sequence equivalent.


transduce is the more general form of collection building, where you can take it as far an not actually building a collection, but do the counting in the reducing function that you're transforming

(transduce (comp (mapcat str) (filter #{\1})) (completing (fn count-rf [c _] (inc c))) 0 (range 15)) ;; => 7
here the reducing function labeled count-rf is ignoring the input that's been filtered and just increasing the initial value


eduction will also not cache the results like a lazy-seq will


there's not really any reason I can think of to use eduction if you're already using into


if you just need the count, you can do:

(defn number-of-ones-eduction2 [n]
  (reduce + 0 (eduction (mapcat n->digits) (filter #(= 1 %)) (range (inc n)))))
This should be faster than the into version


for your transduce version, you can generate a count directly rather than producing a sequence and then counting the elements in the sequence



(defn number-of-ones-transduce2 [n]
  (let [xf (comp (mapcat n->digits)
                 (filter #(= 1 %)))]
    (transduce xf + (range (inc n)))))


Ok, great stuff fellas, thank you. That transduce2 looks quite readable and is the fastest as well (about 4.3 seconds on my machine). This is my first time seeing completing so I will look into that as well.


> I'm curious about the different pros and cons to using the into, transduce, and eduction approaches. My approach is to use into if you need to produce a collection, transduce if you're producing some other kind of result (eg. count, mean, max, etc), and eduction otherwise. I generally avoid using sequence .


Ahh, that kind of general rule of thumb is exactly what I was hoping for.


oh, actually @U0P0TMEFJ’s reducing function is probably preferred to mine since his actually counts and mine only counts when all the inputs are equal to one.


I like sequence and do sometimes use it in preference to long ->> chains, but find it most useful when constructing the transducer process independently of applying it. I think this is the best use case for transducers, the separation of process construction and application. But I do use into the most by far 😉


@U0P0TMEFJ, wouldn't eduction be preferred in that case?


not if you want to produce a lazy seq


an eduction isn't a lazy seq - it won't cache the results. It will reapply the transducer every time you go through the eduction


right, I guess if you are partially reusing the results multiple times. I find that pretty rare, but it's not that wild


well ... the point of an eduction is to combine your input with your process for processing later. So if want to filter the results later, you will reapply the transducer

(let [xf (map (fn [x] (prn '> x) x))
        e  (eduction xf (range 10))]
    [(count (filter odd? e)) (count (filter even? e))])


but sequence will only print the range once because it stores the result of each calculation


it all depends on what your problem is, I guess 😉


I guess the use case would be: • want to partially consume results (if fully consuming results, prefer into) • want to reuse multiple times (if used once, prefer eduction) • don't care about chunking (ie. possibly producing more than required)


for your example, I would prefer either into to fully realize, or transduce to just produce the evens and odds in one pass


sure. It's difficult to have these discussions on the internet with little toy examples. I guess I was kinda imaging a bigger codebase with existing code and assumptions that need to be worked within ...


clearly I came up with a bad example 😉


right. I'm not trying to be contrarian. I was just trying to figure out if the rule of thumb I started with works or should somehow include sequence. the only time I think sequence might be preferred is if it matches the 3 constraints I listed and I have a hard time coming up with a use case.


I'm not saying there isn't one.


I tend to prefer sequence over eduction because non-transducer code generally assumes that collections are lazy-seqs and whatever you pass that resulting sequence to may well do things you didn't expect if you give it something different. But I guess that depends on what you're doing with the result. I often feel like these rules of thumb have so many exceptions that it's just confusing. Every project has it's own problems 😉


The notes at the end of the eduction entry in clojuredocs links to another great discussion about eduction vs sequence if you are interested:

🆒 1

the linked thread compares eduction and sequence , but doesn't seem to mention into. I wonder how into and sequence compare if 1) you plan on realizing fully and 2) you're reusing the result. Just doing a naive comparison based on your sample code, it seems like (into [] ...) wins both in performance and the result let's you access elements by index. I suspect that into also produces less garbage.

Abhi Saxena18:09:34

Hi All, I have been writing crud operations on a new table in Postgres DB (initially empty), while writing test cases I realized that order of tests is not fixed, I was trying this order - INSERT->FETCH->UPDATE->DELETE so I can create a record and finally delete it leaving the database in same state, but as expected test cases are failing. What's the way to achieve this in a Clojure test.


AFAIK there is no way you can guarantee an specific order in a set of tests. Moreover in testing database operations, you really should wrap every test in a transaction, so the database state is clean after each test. So, the only way that I can think of would be including those operations in the same test function, in the order you need, and wrapping everyting in a transaction, anyway.

👍 1
Abhi Saxena18:09:01

I did include them in a single function and it worked fine however, I was told that they need to be separated out so I am trying to explore other options.


hard disagree with using transactions to roll back test state


it is a thing some people like to do, but if you actually use transactions at all in your app you end up with nested sql transactions which can be very weird

Bob B20:09:29

A test has pre-conditions, so if we're testing a delete of a row that's present, the pre-condition is that the row is present. I'd say tests shouldn't depend on other tests having run first (implicit dependencies), so e.g. the delete test would need to ensure that its pre-conditions are met (that the row is present, whether it's already there or the test setup adds it or there's a fixture that adds it or whatever), independent of what other tests may or may not have done.

Abhi Saxena03:09:39

thanks Bob for your response, I have used use-fixture for setup and teardown to make sure necessary rows are available before each test and it worked fine.


Hello! Is there any way to generate data based on the result of another generator? I'm trying to generate a map that has multiple fields and all of those fields have their own generator Here's a generic example: Let's say I have a map with :day-type (weekday or weekend) and :day-name (mon, tue, ...). When I generate the map, how can I specify that the generator should generate "sat" or "sun" if the :day-type generator returns "weekend"? Currently we're using a gigantic gen/let to populate our map, but I was hoping to do this in a way that was more "decentralized" (i.e each field would know how to generate itself based on its dependencies)

Martin Půda20:09:21

In Spec, you will probably need

[clojure.spec.alpha :as spec]
[clojure.spec.gen.alpha :as gen]

(def day
  (gen/bind (spec/gen #{:weekday :weekend})
            #(gen/hash-map :day-type (spec/gen #{%})
                           :day-name (if (= :weekday %)
                                   (spec/gen #{:mon :tue :wed :thu :fri})
                                   (spec/gen #{:sat :sun})))))

(gen/sample day)

Ben Sless20:09:32

gen/let expands to bind, so it's probably not much of an improvement 🙁

Bob B20:09:37

I imagine that, for the bigger picture question, we could create sub-maps based on dependency groupings and then merge them; dependencies would have to be specified somewhere, and let is one way of doing that; maybe there's a core.logic idea there that could take unordered declarations and determine a correct order of generation such that each specific dependency is generated at most once (which might end up generating a big let form)