Fork me on GitHub

do you tap> users have multiple monitors? what value does it have over cider-inspect?


@tessocog I only have one monitor (27"): I have VS Code 3/4 screen on the left and Reveal 1/4 screen on the right. The value of tap> is that it can stay in the code all the time, including in production code, and you can attach (and remove) listeners for it any time you want -- and in particular you can attach multiple listeners at different times.


For example, you could have a plain prn attached to see values at the console, you can have Reveal attached, you can have Cognitect's REBL attached, you can have Portal attached -- in any combination you want, at any time.


had the same idea


ofc if directly interacting with code, cider-inspect buffer will always present the last evaluation


Also, cider-inspect kind of requires you're using Emacs 🙂 I switched away from Emacs some years ago (after using it on and off for nearly two decades).


I'm using leiningen and want's to create a minimal html file containing git head sha before the uberjar is bundled. So my idea was to create the resources/version.html file as here: But that doesn't seem to work - the problem might be that :injections aren't executed for jar/uberjar:

;; Forms to prepend to every form that is evaluated inside your project.
  ;; Allows working around the Gilardi Scenario: 
  ;; Note: This code is not executed in jars or uberjars.
  :injections [(require 'clojure.pprint)]


I don't like :injections much you can run arbitrary code in a project.clj by placing it before defproject, shows this particularly


Interesting, thanks. I ended up using a shell script which is run by Make before lein uberjar

Jim Newton10:08:15

I just discovered a gotcha in clojure which was different than I had supposed for a long time. (first (filter pred lazy-collection)) might call pred again after the first time it has returned true. This could have an unfortunate effect on performance if pred has a high computational complexity. To remedy this, I am refactoring 10 to 20 call-sites in my application to avoid this dangerous pattern. The approach I have chosen (for the moment) is a scheme-like continuation-passing-style (CPS) pattern. Given a predicate, a collection, and an if-found function, call the if-found function on the first item of the collection which satisfies the predicate, and return a call-site-given return value if no such element exists. The functional version of this is somewhat awkward, but could be made more beautiful with some macrology if necessary.

(defn call-with-found
  "Call the given predicate, pred, on successive element of the collection
  until the first time pred returns a truthy value, at which time if-found
  is called with that element of the collection, and call-with-found returns
  the return value of if-found.   If no such element of collection is found
  (including if collection is empty) then the value if-not-found (defaulting
  to false) is returned."
  ([pred coll & {:keys [if-found if-not-found]
                 :or {if-found (constantly true)
                      if-not-found false}}]
   (reduce (fn [_ item]
             (if (pred item)
               (reduced (if-found item))
Here is an example of the call-site.
(call-with-found (fn [x] (dual-combination? this x)) operands
                   :if-not-found this
                   :if-found (fn [dual-td]
                               (create-dual this (map (fn [y]
                                                        (create this (map (fn [x]
                                                                            (if (= x dual-td)
                                                                              y x))
                                                      (rest dual-td)))))
(let [[dual-td & _ :as tds] (filter (fn [x] (dual-combination? this x))
    (if (empty? tds)
      (create-dual this (map (fn [y]
                               (create this (map (fn [x]
                                                   (if (= x dual-td)
                                                     y x))
                             (rest dual-td))))))


do you know why pred is been called after first true?


if it is about lazy-collection been chunked by default then have a look at

(defn re-chunk [n xs]
    (when-let [s (seq (take n xs))]
      (let [cb (chunk-buffer n)]
        (doseq [x s] (chunk-append cb x))
        (chunk-cons (chunk cb) (re-chunk n (drop n xs)))))))

  (filter (fn [l]
            (prn l)
            (even? l))
    (re-chunk 1 (range 1 1000))))
prints only 2 items instead of 32

Jim Newton10:08:12

I don't in fact know that it is. However, I read a response from alexmiller saying that clojure does not guarantee that this pattern halts the traversal when the predicate returns true.

👍 4
Jim Newton10:08:37

wrt unchuck, I found the following function in my odds-and-ends utils.clj file.

;; code thanks to 
(defn unchunk [s]
  (when (seq s)
      (cons (first s)
            (unchunk (next s))))))
although this link no longer works for me, so I'm not really 100% what it is doing. However, it looks very much like it might be useful here.

Jim Newton10:08:47

I am finding many places in my code base where I'm depending on my mis-understanding of filter

(defn conversion-C12
  "AXBC + !X = ABC + !X"
  (let [combos (filter gns/combo? (operands self))
        duals (setof [td combos] (dual-combination? self td))
        comp (filter (fn [n]
                       (and (gns/not? n)
                            (exists [td duals]
                                    (member (operand n) (operands td)))))
                     (operands self))]
    (if (empty? comp)
      (letfn [(f [td]
                (cond (not (gns/combo? td))

                      (not (dual-combination? self td))

                      (create td (remove-element (operand (first comp))
                                                 (operands td)))))]
        (create self (map f (operands self)))))))


Why would that unchunk work at all? If s is chunked lazy sequence it will still load 32 elements when first s is called


you are wrapping a chunked sequence into an additional lazy sequence which doesn’t solve anything


The "unchunk" would be used on a collection before handing it to an expensive lazy operation. But there's a reason there's no "unchunk" in core - if you need to control how many times an operation is performed you shouldn't be using laziness in the first place.

Michael Gardner18:08:32

you can also use transducers to avoid laziness:

user=> (first (map prn [1 2 3 4 5]))
user=> (into [] (comp (map prn) (take 1)) [1 2 3 4 5])


it's not the transducers doing that though, it's using take before map

user=> (map prn (take 1 [1 2 3 4 5]))

Michael Gardner21:08:29

comp works in "reverse" order with transducers, so map is being done first in my example


My personal approach is not to use first/filter, but rather some

(some #(when (pred %) %) lazy-collection)
this only calls pred on each element once, and does a recur on the next when the predicate returns falsey, so it’s not lazy. The bonus of some is that a lot of predicates return the value being tested when they pass, so the when wrapper isn’t needed

Jim Newton14:08:11

the problem with some, if I'm not mistaken, is that it returns the Boolean value returned by the predicate, not the value on which the predicate was called. The additional problem with the #(when (pred %) %) trick is that it cannot be used to find false nor nil in a sequence.


You could use transducers instead:

  (filter #(> % 10))
  (take 1))
 #(or %2 %1)
 (range 100))
;> 11

Jim Newton10:08:23

The function call-with-found uses reduce to efficiently iterate through the collection making no assumption about whether it is list-like or vector-like, and uses reduced to terminate the iteration when finished.


@jimka.issy Is this perhaps a case where memoize can prove useful?

Jim Newton13:08:25

what's the connection to memoize that you've noticed. I do make heavy use of memoize, but I don't see the connection here.


This would be to avoid re-execution of the predicate when the value has already been seen. That way if the filter executes the memorized predicate the second time, it just becomes a map lookup and doesn’t execute the predicate code

☝️ 4
Jim Newton08:08:02

ah, I sort of see what you mean. But I don't see how that would work in the global scope of the program. Are you suggesting that every predicate argument to filter should be a global function which is memoized for the duration of the program, or are you suggesting that I should just wrap the predicate argument of filter in a call to memoize? The former would be difficult as I often pass a closure to filter . The latter would be difficult because it would only have an effect if the predicate is called multiple times within the same call to filter.


Is there a way to capture Var meta-data changes? add-watch sees the Var before the meta-data has been changed.


silly demo:

(defn ^{:sweet 1} sour [])

  #'sour ::watch
  (fn [_k v _old _new]
    (print (-> v meta :sweet))
    ;; separate thread
    (future (println " .." (-> v meta :sweet)))))

(defn ^{:sweet 2} sour []) ;; =prints=> 1..2
(defn ^{:sweet 3} sour []) ;; =prints=> 2..3


looking at the Java classes, I would say no, but some non-hacky workarounds?


def executes all the watches before setting the metadata of the Var. I don't have much experience with reflection but maybe ASM ( has a way to modify .setMeta of the Var instances. I am assuming you do not want to change any of the defn calls to something else so that the changes in metadata are detected without changing any code that changes the metadata.


Hmm.. maybe the Var could be proxied/reified instead of adding a watcher :thinking_face:


The problem with it is that the proxied Var object cannot be be placed into the current namespace object. It can be retrieved by using #'symbol-name . If Clojure or Java had a simple mechanism to advice existing methods (similar to Emacs Lisp's advice mechanism: ), we could advice the setMeta method and make it execute something every time it is called. The only option that might achieve this would be to modify the clojure.lang.Var class before it is loaded (or maybe it is also possible to modify it after it is loaded).


What java profiler do you recommend to use today for Clojure? Do you feel some of them are simpler (more efficient) than others? Which one do you use and why? Why not others?


If I can shamelessly plug (because I made it :p and because it's a ~simple defn replacement)


thank you, but it is not a tool for my case. We have memory leak on production in old complex system and I have to debug it.

Elliot Stern15:08:45

I’ve used visualvm in the past for Scala. It’s pretty decent, though I’ve never tried it with clojure


I am watching YT and so far jprofiler looks like show information in simple way. But I think it is not free (still researching)


not for clojure particularly, but I have used thread dumps and to pin point memory problems on java


Why eclipse vs visualvm?


The evil part of visualvm (and maybe other profiles) is thousands of:

Profiler Agent Warning: Cannot convert 0x7fc988c13320
Profiler Agent Warning: Cannot convert 0x7fc988c13328
Profiler Agent Warning: Cannot convert 0x7fc988c13330
Profiler Agent Warning: Cannot convert 0x7fc988c13338
Profiler Agent Warning: Cannot convert 0x7fc988c13318
Profiler Agent Warning: Cannot convert 0x7fc988c13320
Profiler Agent Warning: Cannot convert 0x7fc988c13328
Profiler Agent Warning: Cannot convert 0x7fc988c13330
Profiler Agent Warning: Cannot convert 0x7fc988c13338
which make me to not trust results


YourKit, you won't regret it :) it's scary at first but you don't have to use every tab. You can grow in knowledge as-you-go 90% of the time I use the Threads tab, that one is a fairly simple UI

💯 2

rationale: it's powerful, just works, doesn't look like a cheap Swing app

👍 2

considering licences it is not free too, but I agree it can be good choice


BTW From youtube jprofiler was looking even better

👀 2

but it is only first impression


every xmas they make a deal for 100 bucks


still expensive, I need it first time to fix memory leak in about 5-6 years of coding Clojure


:) well, each to his own... after you acquire one of these hammers you'll see nails more often there's hardly a week where YK doesn't help me in some way. TIL about jprofiler btw!


this week's: I could quickly visualize that a thread pool wasn't being stopped, so my Reloaded workflow was leaking threads on each (reset)

👍 2

There is also XRebel worth to check

🍻 2

please let me know if you will do research about them


oh not atm, I feel quite well served it's good to know about them though!


for me all of them are new stuff and it is hard to judge for me


YK and JP look fairly similar in areas of focus. maybe JP looks a little bit easier? that will be subjective of course. You could simply try their demos and see which feels more intuitive


XRebel seems a little more tailor-made for webapps and microservices instead of being a generic JVM thing which is still interesting, but probably more than one needs for debugging a mem leak


I am trying jprofiler right now, but the main issue the out of memory exception is happening only on production


I can’t recreate it locally or even on preprod


I have plan to use XX:+HeapDumpOnOutOfMemoryError maybe

👍 2

But it will be my first time


normally these can connect to a remote process


I have only used YourKit but it was very impressive and easy to use. I would recommend it. It also has a “15 day fully functional trial”. (I have no relationship to the company)

👍 2

visualvm is a profiler, eclispse-mat is a memory (thread-dump) analyzer, the later can produce a "leak suspects" report from a thread-dump, I don't remember if it can connect to a running jvm. Also, OutOfMemory errors not necessarily implies a memory leak it may be that your current workload has increased, and well, your memory is not enough under current gc configuration. Monitoring jvm-memory spaces and app throughput/workload/rpm may hint this. Also you can look at the gc logs for hints of gc memory problems.


> “leak suspects” Can YourKit / jprofiler / XRebel do the same?


> OutOfMemory errors not necessarily implies a memory leak In this specific case we can observe increasing heap in time and constantly repeating OOM after a few days.


Ok to figure out out of memory issue I think the best will be to first prepare app for which one I can control leak memory intentionally. Do you know good sophisticated memory leak code example in Clojure? Just something more tricky to debug, than “add to array”.


@hiredman are you running with this patch applied in prod?


but still using core async?


so it’s not impactful enough in your app or did you work around it by avoiding alts!?


Hard to say, the leak is actually subtle and needs some precise circumstances to trigger. When our core.async heavy app was initially deployed it was very rocky and needed a lot of work to stabilize for a variety of reasons. I am not sure how much this leak played into that, but as part of stabilizing the application we removed a lot usages of timer channels just going on the idea that timers are global and potentially holding references in some way.


I didn't make the connection with alts! until much later


We also have a sort of custom channel that only weakly references callbacks that is used strategically in a few places that may also make us less suspectable


this is very helpful, thank you!

Ivar Refsdal14:08:01

Hi. I have some code like this:

(defonce buf (async/chan (async/sliding-buffer 1)))
(defonce running? (atom true))
  (async/go-loop [do-sleep false]
    (if @running?
        (when do-sleep
          (async/alts! [(async/timeout 200)]))
        (let [timeout (async/timeout 1000)
              [v ch] (async/alts! [buf timeout])]
          (if (nil? v)
            (recur false)
              (doseq [c @clients]
                (s/put! c (json/generate-string v)))
              (recur true)))))
      (log/info "exiting!"))))
and very often (async/>!! buf new-val) is done. Would this code (`pusher`) trigger a memory leak? Edit: From this issue I think yes, but I am not sure. I think that the timeout channels will "hang" around (for 1s) and potentially grow and grow (if there is enough traffic). I'm using org.clojure/core.async "0.3.443" included by yada/async "1.2.16".. I've only recently added this code, and only recently experienced OOM (I think!) in production. And OOM only happens after some time in prod, and not in stage, it seems.


I can't say for sure, but I think it is very unlikely your leak is related to that alts! (Using a loop in a go block like that mitigates parts of it, and you just have one instance of that loop, and you aren't closing over big chunks of memory)


I'd like to annotate any Throwables thrown from a certain context with some additional context-specific info. This seems like a job for metadata. However, Throwables don't appear to implement IObj:

(with-meta (Exception. "oh no") {:extra :context})
; (err) class java.lang.Exception cannot be cast to class clojure.lang.IObj
I could implement IObj (there's only one method: withMeta()) for Throwables, but would it be more idiomatic to throw an (ex-info ...) wrapping the original Throwable? It seems cleaner to just add stuff to the original but there's no obvious way to do that with the Lack of arbitrary annotations for errors seems like the central shortcoming that ex-info and ex-data were designed to solve, but I want to be sure before I add a bunch of indirection...


you could implement withMeta perhaps, but you couldn't really make it work correctly right?

Russell Mull18:08:46

This sound a little like chained exceptions, in the java world

Russell Mull18:08:34

Perhaps (ex-info "oh no!" {:key :val} original-exception) would serve?

Russell Mull18:08:52

Ah, As you already said. That seems most idiomatic.

Russell Mull18:08:43

What are your indirection concerns?


We generally just wrap exceptions in ex-info whenever we want to convey more information. They are also handled "better" by certain runners if they are otherwise unhandled (e.g., exec function execution prints a neat error message for ex-info but a complete stacktrace for other exceptions).


Okay, that makes sense. No specific concerns with indirection, just wanted to make sure I was on the right track/not overthinking things. Thanks all.


what do you need arbitrary annotations for @ctamayo?


this system has a hooks API so I want to annotate errors thrown inside hook callbacks with info about the hook being dispatched


Which system? A java framework?


and it uses annotations to dispatch?


sorry, not talking about annotations in the reader sense, I'm just talking about extra runtime data


"annotate" = call with-meta/`ex-info`


you should prefer ex-info


unless you can represent failures as plain old data (maps)


does it make sense to use spec for separating part of data that conforms to a spec? say

(s/MATCH!!! spec mycollection)


instead of

(filter pred mycollection)

Alex Miller (Clojure team)23:08:14

(s/filter (s/valid? spec %) mycollection)