Fork me on GitHub

@lepistane it can be done automatically using something like


@lepistane @schmee Or, which is faster, supports async and is extensible


@juhoteperi cool, haven’t seen that!


does anyone use Newrelic with Clojure?


Have you looked at It's a vendor neutral spec for collecting distributed tracing data with multiple language bindings and multiple backends including Jaeger (Uber), Zipkin (Twitter), Appdash, LightStep and others.


I wrote a clojure wrapper around the opentracing-java API which was relatively easy to do. Needs work though, because the opentracing spec is evolving quickly and they made a significant change to the way trace information is propagated within a threaded java program. And I just haven't gotten back to look at that.


ah no I didn't know about that


looks interesting thanks


for this project to be fair I don't really need it, but since with Heroku it would have been more or less free it would have been nice to have


You're welcome. On my project, we discovered we had access to unused appdynamics licenses from another team, not exactly free but we just had to pick up the annual maintenance. So we did that as a starting point. But opentracing is on our radar for a distributed, streaming raytracer we have.


it supports java but not sure if it would be actually useful


or other similar alternatives?


This seems quite interesting. Will we see this work reflected in Clojure eventually? Speedups Compared to Clojure’s Maps. In every runtime measurement CHAMP is better than Clojure. CHAMP improves by a median 72 % for Lookup, 24 % for Insert, and 32 % for Delete. At iteration and equality checking, CHAMP significantly outperforms Clojure. Iteration (Key) improves by a median 83 %, and Iteration (Entry) by 73 %. Further, CHAMP improves on Equality (Distinct) by a median 96 %, and scores several magnitudes better at Equality (Derived). Speedups Compared to Clojure’s Sets. The speedups of CHAMP for sets are similar to maps across the board, with exception of insertion and deletion where it scores even better. CHAMP reduces the footprint of Clojure’s maps by a median 16 % (32-bit) and 23 % (64-bit), ranging from 3 % to 48 %. The median reductions for sets are 30 % (32-bit) and 41 % (64-bit) respectively, ranging from 19 % to 60 %.


probably same content just in video format


@ashnur Not yet, but it’s now on the to-see list!


So there’s been some work in the community on these things it seems.


those speedups don't take into account that clojure's collections use a different hashing/equality than the default hashCode/equals


Does that mean that Clojure collections are doing hashing/equality better than this CHAMP thing?


they're doing it differently


in particular, clojure uses murmur3 on longs/ints, java uses the value itself


Ah, so CHAMP might have more hash collisions for maps keyed by a collection, is that the idea?


no, my point is that the hash calculation used by clojure adds overhead


that overhead is ignored in the CHAMP perf measurment


that benchmark is not just benchmarking CHAMP vs HAMT, it's benchmarking CHAMP w/hashCode vs HAMT w/ hasheq (the clojure hashing that uses murmur3)


it is possible that CHAMP is still faster than clojure's HAMT impl even after using the same hashing/equivalence checks that clojure's coll use, but I suspect the margin might be significantly reduced


Ah, interesting. Thanks for explaining, @bronsa.


@bronsa They seem to believe that Scala’s implementation can be improved with this, it seems:


i'm not arguing that there's no performance benefits to be had


Oh sorry, I didn’t mean to imply that you were


Just throwing this in there


i'm just saying that the order of magnitude figures they give don't take into account other factors in the clj impl


What exactly makes Clojure special in this case? That is, what does the overhead consist of and why would it be necessary in Clojure, while being unnecessary in other implementations?


>>>Clojure’s hashing strategy for numbers, sequences/vectors, sets, and maps mimics Java’s. In Clojure, however, it is far more common than in Java to use longs, vectors, sets, maps and compound objects comprised of those components (e.g., a map from vectors of longs to sets) as keys in other hash maps. It appears that Java’s hash strategy is not well-tuned for this kind of usage. Clojure’s hashing for longs, vectors, sets, and maps each suffer from some weaknesses that can multiply together to create a crippling number of collisions.


@cgrand and I have separately looked at implementing CHAMP on the java side. It's a lot more complicated than 'free performance'


And @ztellman reproduced my measurements that CHAMP vs Clojure differences are hash algorithms differences.


so the perf side is a wash; did you ever quantify whether the memory improvements were worth?


No. There should be memory gains, but not those claimed.


I also expect iteration to be faster but not for reasons evoked in the paper.


The by far most common key in Clojure maps must surely be the keyword. I don’t know what situation would demand that I use “a map from vectors of longs to sets” as a key in a map.


@ghadi I bet it is. I barely comprehend it.


clojure.spec apparently thinks non-keyword keys is such an uncommon thing that it doesn’t even support them.


Is there some weirdness with *ns* while running tests? I’m seeing a failure that I can only explain if ns can somehow be a different test ns


  (:require [ :as bio]
            [clojure.test :as t]))

(t/deftest ns-keyword-tests
    (in-ns '
    (t/is (= (bio/ns-keyword *ns* "a")))
    (t/is (= (bio/ns-keyword *ns* :a)))))
That test only passes with lein test when I add the in-ns. (I know the try is superfluous)

Sam H15:08:03

I’ve noticed *ns* evaluates to a higher level namespace. Possibly what ever lein test calls under the hood. I don’t really fully understand it tho.


I’d grok that but it’s just another test namespace


not even the first one or the previous one


@henrik Hm, I use vectors for keys where vectors are coordinates. [0] [0 0] [0 0 0]


@urbank Sure, i definitely won’t say it doesn’t happen, far from it. In any case, there’s nothing stopping us from adding different implementations with different performance characteristics and using them as appropriate.


Wasn't aware that clojure spec doesn't support it though. In that s/keys only allows keywords, I assume? (Haha, guess I just gave away that I didn't try to spec it.)


Yeah, unfortunately. I discovered this precisely because I had a non-keyword map. It may be that it gets covered in the future. After all, spec is just alpha.


Hm, that is unfortunate. Though I guess in my case the exact keys aren't determined in advance so s/keys wouldn't be applicable anyway.


Hi. I have a basic clojure spec for a map, e.g. (s/keys :req [:a :b] :opt [:c]) and I would like to get the list of required and optional keys back out; how can I do this?


s/describe* on the spec returns (clojure.spec.alpha/keys :req [:a :b] :opt [:c]) (a list) which I could then parse to get what I want


but is there an easy way to convert something like this (clojure.spec.alpha/keys :req [:a :b] :opt [:c]) to something like this {:req [:a :b] :opt [:c]}?


@henrik the problem with having multiple hashmap impls is then you get killed by callsites becoming megamorphic. That was one of the problems with using tuples for small vectors. Using just tuples was fast, using just vectors was fast, using both caused the callsites to to megamorphic and the code got slower than just using one or the other.


Gotcha. That makes a lot of sense. Could spec be leveraged for some kind of static analysis? I.e., if I describe my data structure in one way, one implementation could be used, and if I describe it in another, a different one is invoked?


The JVM already does that, the problem is when the implementation changes from one invocation of a function to the next


oh, wow, that’s a big gotcha


Is there a way with the clj tool to dump/cache the classpath (resolved) without launching anything? Pure resolution?


@richiardiandrea that’s a hard thing to google, who provides this “clj” binary?


if it can run arbitrary clojure code, you can always do (println (System/getProperty “java.class.path”))


yes but that always starts a repl if I understood correctly


I wanted something that just dumps like lein classpath > cp


@richiardiandrea ${PROJECT_DIR}/.cpcache/default/default.cp


that's where it gets cached


yes I see that it is there, but I don't want to start any repl here, because I want to launch lumo instead...


i don't follow


lumo is cljs -- tools.deps is clj


do you want to use lumo to connect to a socket repl?


how would you get the info created by a clojure library without running a clojure process?


The idea (as I understand) is to use tools.deps to: - fetch missing dependencies into m2 - generate a classpath string And then pass that to the lumo cli


but how would lumo use your jvm deps?


@noisesmith currently there is jvm cljs and bootstrapped cljs, the deps can be resolved by maven means


if they work or not...that depends on the library


@richiardiandrea btw lein cp starts a clojure process (only one, instead of two like lein repl does…)


@noisesmith that's good, but I guess this is the fastest I can get:

$ time lein trampoline cp > cp
cp did not run any project code for trampolining.

real	0m8.430s
user	0m28.768s
sys	0m0.420s
$ time lein trampoline classpath > cp
classpath did not run any project code for trampolining.

real	0m8.453s
user	0m28.892s
sys	0m0.396s


I guess they are the same 😄


What's the preferred way of using Kafka in clojure? pyr/kinsky? Raw java interop?


I use a combo of clj-kafka and interop and that works fine


are multimethods / protocols Clojure's take on abstract-data-types ? the point being "people program this interface / set of functions, instead of the direct underlying implementation"


yes but I’d express it differently - ADT and prototypes and multimethods are each ways of doing parametric polymorphism, or more generally handling the expression problem


@qqq multimethods/protocols are clojure's take on polymorphism (useful!), in clojure land we tend to think of language like "ADT" as data-hiding (not useful)


ie, you implement protocols using records, but records are still mostly just data, there's no requirement for getters setters etc


but I think data hiding is useful. without data hiding, if I change the layout of a structure, I have no idea where all the pieces of code I have to change are. with data hiding, if I change the layout of a structure, I just reimplement the protocol / multimethods its involved in


implementing a protocol / multimethod to access data isn’t a popular thing in clojure though - we just use the regular data structures


@qqq you don't need data hiding for that, you just need common functions and discipline


in general though, you're not going to have a great time if you're trying to do data hiding in clojure, the community and the creators are pretty explicit about thinking it's a bad idea

bfabry18:08:35 <-- specifically ""It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures." - Alan J. Perlis" and "Many functions defined on few primary data structures (seq, map, vector, set)."


is there an idiom or function for wrapping an object in a vec or list?


so, basically #(do [%])?


some reason I keep forgetting about that, since I see vec so much and get disappointed that it just does type coercion


Hello local geniuses, I just forked this small clojure text editor because I saw a cool opportunity to explore an idea. Let's say I wanted to have a plugin directory where you could drop a .jar or a .clj file and have it loaded, I know you can load .clj files, but if someone wants to make a big plugin (wouldn't happen, but if), a jar is just how it comes packaged. tl;dr clojure plugins for clojure projects?


Another possibly silly pattern question: I have a reducer function (fn [acc v] (conj v acc)) — are there any facilities for converting a function into one of the opposite argument order?


you just wrote one though @shader


is there such thing as "branch prediction" in clojure? e.g. is there any performance benefits/penalties based on if branches order? I like to have short then branches, and long else ones. Do I pay for this anything, other than extra (not) call in predicate?


@shader you could just use the function with the -> or ->> macro depending on which order


@shader there is also #(-> [%]), if you parse literals and threading macros faster than recall function signatures


@alice plugins as jars were discussed here like last week, try searching log (I did not follow the discussion though)


@misha no, and if it did it would probably be abused a lot. The CPU branch predictor has something like a 95% correct rate


Once the code gets JIT-ited the CPU branch prediction will kick in, and it will do a good enough job.


That being said, if you can remove if statements and replace them with math, that's almost always a good idea.


java does branch profiling before it JITs


yeah, that too


then the CPU will do branch prediction


What would be a good way to chain/thread update calls so that the intermediate values are accessable (the following code is just wishful thinking)

(-> {:index 0 :dummy 0}
    (update :index inc)
    (update :dummy (fn [dummy] (+ dummy (:index %)))))
the wishfullness here is that I want to access :index when Im updateing :dummy so that dummy accesses :index after application, there is to say it would have the number 1.


@hlolli (-> foo (bar baz) (as-> x (update x :dummy + (:index x))))


as-> is really just a let block that keeps assigning to the same symbol, but formatted to work inside -> nicely


ah! as-> eye opener


yeah, it’s great


up to this point I dismissed it blindly for some syntactical sugar. Thanks!!

Aleh Atsman20:08:48

Hi guys, i am trying to test my macros. Basically i want to test param validation. In macros definition i have some asset calls. In repl everything works fine. But with lein test not. I do not have any ideas what is wrong. Console output is lile this:

Testing lambada.core-test

FAIL in (def-lambda-fn-test) (core_test.clj:10)
should throw assert error if name is not a symbol

expected: (thrown?
            (quote (l/def-lambda-fn "" [in out context] (println "hello world")))))
  actual: nil
Test code.
(deftest def-lambda-fn-test
  (testing "should throw assert error if name is not a symbol"
    (is (thrown? java.lang.AssertionError 
                  '(l/def-lambda-fn ""
                    [in out context]
                    (println "hello world"))))))) 
Source of the macros:
(ns lambada.core
  (:require [clojure.spec.alpha :as s])
  (:import [ RequestStreamHandler]))

(s/def ::name symbol?)
(s/def ::args vector?)
(s/def ::body list?)
(s/def ::arguments (s/cat :name ::name :args ::args :body ::body))

(defn- assert-args [args]
  (assert (s/valid? ::arguments args)
          (s/explain ::arguments args)))

(defmacro def-lambda-fn
  "Create a named class that can be invoked as a AWS Lambda function."
  [name args body]
  (assert-args [name args body])
  (let [prefix (gensym)
        handleRequestMethod (symbol (str prefix "handleRequest"))]
        :name ~name
        :prefix ~prefix
        :implements [])
       (defn ~handleRequestMethod
         ~(into ['this] args)

Aleh Atsman20:08:57
replied to a thread:Hi guys, i am trying to test my macros. Basically i want to test param validation. In macros definition i have some asset calls. In repl everything works fine. But with lein test not. I do not have any ideas what is wrong. Console output is lile this: Testing lambada.core-test FAIL in (def-lambda-fn-test) (core_test.clj:10) should throw assert error if name is not a symbol expected: (thrown? java.lang.AssertionError (macroexpand (quote (l/def-lambda-fn "" [in out context] (println "hello world"))))) actual: nil Test code. (deftest def-lambda-fn-test (testing "should throw assert error if name is not a symbol" (is (thrown? java.lang.AssertionError (macroexpand '(l/def-lambda-fn "" [in out context] (println "hello world"))))))) Source of the macros: (ns lambada.core (:require [clojure.spec.alpha :as s]) (:import [ RequestStreamHandler])) (s/def ::name symbol?) (s/def ::args vector?) (s/def ::body list?) (s/def ::arguments (s/cat :name ::name :args ::args :body ::body)) (defn- assert-args [args] (assert (s/valid? ::arguments args) (s/explain ::arguments args))) (defmacro def-lambda-fn "Create a named class that can be invoked as a AWS Lambda function." [name args body] (assert-args [name args body]) (let [prefix (gensym) handleRequestMethod (symbol (str prefix "handleRequest"))] `(do (gen-class :name ~name :prefix ~prefix :implements []) (defn ~handleRequestMethod ~(into ['this] args) ~@body))))

Any ideas?


you should use syntax quote for the macro form you are expanding


almost certainly for (long chain of reasons) macroexpand is failing to resolve l/def-lambda-fn to what you think it should resolve to, so it doesn't know it is a macro and does nothing


but if you use syntax quote l/def-lambd-fn will be resolved at read time to what you expect

Aleh Atsman20:08:20

but it does so in repl… How should i quote it? Because i have quoted it already


with syntax quote


the compilation and runtime contexts for tests run via a tool are often different, but for the repl they are almost always the same

Aleh Atsman20:08:08

‘(l/def-lambda-fn, here i have a quote symbol, or should i call “quote” explicitly?


do you now know what syntax quote is? because the def-lambda-fn macro is using it

Aleh Atsman20:08:36

i understand what ` symbol do, but i am not sure what do u mean


Warning, the current releases of leiningen and boot will fail on that without modifications to them ^


clojure, however is fine


What did they end up doing with Jigsaw then?


(and how come Clojure runs but Leiningen/Boot are broken?)

Aleh Atsman20:08:25

i got it. i used ' instead of `

Aleh Atsman20:08:39

@hiredman thank you for answers


@tbaldridge can you give an example of replacing if with math?


(my question comes from day-to-day application/ui code, not from some low level or algorithmic one, if that makes any difference)


@misha sure, in Clojure sometimes you have to count the number of set bits in an int


lein repl takes about 50s to start on my laptop for a small project with no so many dependencies. Is that to be expected?


FWIW, on my iMac desktop, lein repl outside a project has these timings 4.24 real 3.53 user 0.33 sys


@hmaurer how about java -jar clojure.jar (you can get the jar from or find it under ~/.m2/repository/org/clojure/clojure/


@hmaurer What do you have in ~/.lein/profiles.clj? Maybe something there is causing the problem?


Else perhaps share your project.clj?


clojure starts in 3.8 seconds on a Raspberry Pi 3


... while Running Java 9. 4.37 seconds on java 8.


# raspberry pi 3
[alarm@alarmpi ~]$ time ./jdk-9/bin/java -jar clojure-1.8.0.jar -e :foo

real	0m3.876s
user	0m4.170s
sys	0m0.140s
[alarm@alarmpi ~]$ time ./jdk1.8.0_144/bin/java -jar clojure-1.8.0.jar -e :foo

real	0m4.379s
user	0m3.840s
sys	0m0.150s


Not bad, right?


they must have made some ARM backend improvements for Java 9, because Java 9 startup has regressed a smidge on x86-64


(for comparison, 2012 iMac, Clojure startup is around 0.750s so, yeah, Go Pi!)


It improves to 3.55s on java 9 when Clojure is compiled with JDK8 level bytecode.


A Raspberry Pi is a totally sufficient remote REPL if you are fine with a 920MB of usable heap.


You, however, would not enjoy running lein on it. tools.deps will soon be sufficient for bootstrapping a JVM.


I'm going to try to measure lein startup on it. It'll take a moment because I don't have it installed


[alarm@alarmpi ~]$ time JAVA_CMD=$PWD/jdk1.8.0_144/bin/java ./lein run -m clojure.main -e :foo
Java HotSpot(TM) Client VM warning: TieredCompilation is disabled in this release.

real	0m6.555s
user	0m7.670s
sys	0m0.260s


not as bad as I thought, but still double clojure.main. lein currently fails 💥 outright on Java 9 because it puts Clojure on the special bootclasspath and there are more restrictions on that in Java 9