@lepistane it can be done automatically using something like https://github.com/ngrunwald/ring-middleware-format

@lepistane @schmee Or https://github.com/metosin/muuntaja, which is faster, supports async and is extensible

@juhoteperi cool, haven’t seen that!

does anyone use Newrelic with Clojure?

it supports java but not sure if it would be actually useful

or other similar alternatives?

This seems quite interesting. Will we see this work reflected in Clojure eventually? https://michael.steindorfer.name/publications/phd-thesis-efficient-immutable-collections.pdf Speedups Compared to Clojure’s Maps. In every runtime measurement CHAMP is better than Clojure. CHAMP improves by a median 72 % for Lookup, 24 % for Insert, and 32 % for Delete. At iteration and equality checking, CHAMP significantly outperforms Clojure. Iteration (Key) improves by a median 83 %, and Iteration (Entry) by 73 %. Further, CHAMP improves on Equality (Distinct) by a median 96 %, and scores several magnitudes better at Equality (Derived). Speedups Compared to Clojure’s Sets. The speedups of CHAMP for sets are similar to maps across the board, with exception of insertion and deletion where it scores even better. CHAMP reduces the footprint of Clojure’s maps by a median 16 % (32-bit) and 23 % (64-bit), ranging from 3 % to 48 %. The median reductions for sets are 30 % (32-bit) and 41 % (64-bit) respectively, ranging from 19 % to 60 %.

Implementation of the same: https://github.com/usethesource/capsule

@henrik https://www.youtube.com/watch?v=GibNOQVelFY you saw this? 🙂

probably same content just in video format

@ashnur Not yet, but it’s now on the to-see list!

Also, https://github.com/bendyworks/lean-map

So there’s been some work in the community on these things it seems.

those speedups don't take into account that clojure's collections use a different hashing/equality than the default hashCode/equals

Does that mean that Clojure collections are doing hashing/equality better than this CHAMP thing?

they're doing it differently

in particular, clojure uses murmur3 on longs/ints, java uses the value itself

see https://dev.clojure.org/display/design/Better+hashing

Ah, so CHAMP might have more hash collisions for maps keyed by a collection, is that the idea?

no, my point is that the hash calculation used by clojure adds overhead

that overhead is ignored in the CHAMP perf measurment

that benchmark is not just benchmarking CHAMP vs HAMT, it's benchmarking CHAMP w/hashCode vs HAMT w/ hasheq (the clojure hashing that uses murmur3)

AFAICT

it is possible that CHAMP is still faster than clojure's HAMT impl even after using the same hashing/equivalence checks that clojure's coll use, but I suspect the margin might be significantly reduced

Ah, interesting. Thanks for explaining, @bronsa.

@bronsa They seem to believe that Scala’s implementation can be improved with this, it seems: https://github.com/scala/collection-strawman

i'm not arguing that there's no performance benefits to be had

Oh sorry, I didn’t mean to imply that you were

Just throwing this in there

i'm just saying that the order of magnitude figures they give don't take into account other factors in the clj impl

yeah nw

What exactly makes Clojure special in this case? That is, what does the overhead consist of and why would it be necessary in Clojure, while being unnecessary in other implementations?

>>>Clojure’s hashing strategy for numbers, sequences/vectors, sets, and maps mimics Java’s. In Clojure, however, it is far more common than in Java to use longs, vectors, sets, maps and compound objects comprised of those components (e.g., a map from vectors of longs to sets) as keys in other hash maps. It appears that Java’s hash strategy is not well-tuned for this kind of usage. Clojure’s hashing for longs, vectors, sets, and maps each suffer from some weaknesses that can multiply together to create a crippling number of collisions.

quoting from https://dev.clojure.org/display/design/Better+hashing

@cgrand and I have separately looked at implementing CHAMP on the java side. It's a lot more complicated than 'free performance'

The by far most common key in Clojure maps must surely be the keyword. I don’t know what situation would demand that I use “a map from vectors of longs to sets” as a key in a map.

@ghadi I bet it is. I barely comprehend it.

clojure.spec apparently thinks non-keyword keys is such an uncommon thing that it doesn’t even support them.

Is there some weirdness with *ns* while running tests? I’m seeing a failure that I can only explain if ns can somehow be a different test ns

(ns bleeker.io-test
  (:require [ :as bio]
            [clojure.test :as t]))

(t/deftest ns-keyword-tests
  (try
    (in-ns 'bleeker.io-test)
    (t/is (= :bleeker.io-test/a (bio/ns-keyword *ns* "a")))
    (t/is (= :bleeker.io-test/a (bio/ns-keyword *ns* :a)))))

@henrik Hm, I use vectors for keys where vectors are coordinates. [0] [0 0] [0 0 0]

@urbank Sure, i definitely won’t say it doesn’t happen, far from it. In any case, there’s nothing stopping us from adding different implementations with different performance characteristics and using them as appropriate.

Hi. I have a basic clojure spec for a map, e.g. (s/keys :req [:a :b] :opt [:c]) and I would like to get the list of required and optional keys back out; how can I do this?

s/describe* on the spec returns (clojure.spec.alpha/keys :req [:a :b] :opt [:c]) (a list) which I could then parse to get what I want

but is there an easy way to convert something like this (clojure.spec.alpha/keys :req [:a :b] :opt [:c]) to something like this {:req [:a :b] :opt [:c]}?

@henrik the problem with having multiple hashmap impls is then you get killed by callsites becoming megamorphic. That was one of the problems with using tuples for small vectors. Using just tuples was fast, using just vectors was fast, using both caused the callsites to to megamorphic and the code got slower than just using one or the other.

oh, wow, that’s a big gotcha

Is there a way with the clj tool to dump/cache the classpath (resolved) without launching anything? Pure resolution?

@richiardiandrea that’s a hard thing to google, who provides this “clj” binary?

oh sorry: https://github.com/clojure/tools.deps.alpha

if it can run arbitrary clojure code, you can always do (println (System/getProperty “java.class.path”))

yes but that always starts a repl if I understood correctly

I wanted something that just dumps like lein classpath > cp

@richiardiandrea ${PROJECT_DIR}/.cpcache/default/default.cp

that's where it gets cached

yes I see that it is there, but I don't want to start any repl here, because I want to launch lumo instead...

@richiardiandrea btw lein cp starts a clojure process (only one, instead of two like lein repl does…)

@noisesmith that's good, but I guess this is the fastest I can get:

$ time lein trampoline cp > cp
cp did not run any project code for trampolining.

real	0m8.430s
user	0m28.768s
sys	0m0.420s

$ time lein trampoline classpath > cp
classpath did not run any project code for trampolining.

real	0m8.453s
user	0m28.892s
sys	0m0.396s

I guess they are the same 😄

What's the preferred way of using Kafka in clojure? pyr/kinsky? Raw java interop?

I use a combo of clj-kafka and interop and that works fine

are multimethods / protocols Clojure's take on abstract-data-types ? the point being "people program this interface / set of functions, instead of the direct underlying implementation"

yes but I’d express it differently - ADT and prototypes and multimethods are each ways of doing parametric polymorphism, or more generally handling the expression problem

@qqq multimethods/protocols are clojure's take on polymorphism (useful!), in clojure land we tend to think of language like "ADT" as data-hiding (not useful)

ie, you implement protocols using records, but records are still mostly just data, there's no requirement for getters setters etc

but I think data hiding is useful. without data hiding, if I change the layout of a structure, I have no idea where all the pieces of code I have to change are. with data hiding, if I change the layout of a structure, I just reimplement the protocol / multimethods its involved in

implementing a protocol / multimethod to access data isn’t a popular thing in clojure though - we just use the regular data structures

@qqq you don't need data hiding for that, you just need common functions and discipline

in general though, you're not going to have a great time if you're trying to do data hiding in clojure, the community and the creators are pretty explicit about thinking it's a bad idea

https://clojure.org/about/rationale#_object_orientation_is_overrated <-- specifically ""It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures." - Alan J. Perlis" and "Many functions defined on few primary data structures (seq, map, vector, set)."

is there an idiom or function for wrapping an object in a vec or list?

so, basically #(do [%])?

@shader vector

some reason I keep forgetting about that, since I see vec so much and get disappointed that it just does type coercion

@schmee thanks

Hello local geniuses, I just forked this small clojure text editor because I saw a cool opportunity to explore an idea. Let's say I wanted to have a plugin directory where you could drop a .jar or a .clj file and have it loaded, I know you can load .clj files, but if someone wants to make a big plugin (wouldn't happen, but if), a jar is just how it comes packaged. tl;dr clojure plugins for clojure projects?

Another possibly silly pattern question: I have a reducer function (fn [acc v] (conj v acc)) — are there any facilities for converting a function into one of the opposite argument order?

nope

you just wrote one though @shader

is there such thing as "branch prediction" in clojure? e.g. is there any performance benefits/penalties based on if branches order? I like to have short then branches, and long else ones. Do I pay for this anything, other than extra (not) call in predicate?

@shader you could just use the function with the -> or ->> macro depending on which order

@shader there is also #(-> [%]), if you parse literals and threading macros faster than recall function signatures

@alice plugins as jars were discussed here like last week, try searching log (I did not follow the discussion though)

@misha no, and if it did it would probably be abused a lot. The CPU branch predictor has something like a 95% correct rate

Once the code gets JIT-ited the CPU branch prediction will kick in, and it will do a good enough job.

That being said, if you can remove if statements and replace them with math, that's almost always a good idea.

java does branch profiling before it JITs

yeah, that too

then the CPU will do branch prediction

What would be a good way to chain/thread update calls so that the intermediate values are accessable (the following code is just wishful thinking)

(-> {:index 0 :dummy 0}
    (update :index inc)
    (update :dummy (fn [dummy] (+ dummy (:index %)))))

@hlolli (-> foo (bar baz) (as-> x (update x :dummy + (:index x))))

as-> is really just a let block that keeps assigning to the same symbol, but formatted to work inside -> nicely

ah! as-> eye opener

yeah, it’s great

up to this point I dismissed it blindly for some syntactical sugar. Thanks!!

Hi guys, i am trying to test my macros. Basically i want to test param validation. In macros definition i have some asset calls. In repl everything works fine. But with lein test not. I do not have any ideas what is wrong. Console output is lile this:

Testing lambada.core-test

FAIL in (def-lambda-fn-test) (core_test.clj:10)
should throw assert error if name is not a symbol

expected: (thrown?
           java.lang.AssertionError
           (macroexpand
            (quote (l/def-lambda-fn "" [in out context] (println "hello world")))))
  actual: nil

(deftest def-lambda-fn-test
  (testing "should throw assert error if name is not a symbol"
    (is (thrown? java.lang.AssertionError 
                 (macroexpand 
                  '(l/def-lambda-fn ""
                    [in out context]
                    (println "hello world"))))))) 

(ns lambada.core
  (:require [clojure.spec.alpha :as s])
  (:import [com.amazonaws.services.lambda.runtime RequestStreamHandler]))

(s/def ::name symbol?)
(s/def ::args vector?)
(s/def ::body list?)
(s/def ::arguments (s/cat :name ::name :args ::args :body ::body))

(defn- assert-args [args]
  (assert (s/valid? ::arguments args)
          (s/explain ::arguments args)))

(defmacro def-lambda-fn
  "Create a named class that can be invoked as a AWS Lambda function."
  [name args body]
  (assert-args [name args body])
  (let [prefix (gensym)
        handleRequestMethod (symbol (str prefix "handleRequest"))]
    `(do
       (gen-class
        :name ~name
        :prefix ~prefix
        :implements [com.amazonaws.services.lambda.runtime.RequestStreamHandler])
       (defn ~handleRequestMethod
         ~(into ['this] args)
         [email protected]))))

you should use syntax quote for the macro form you are expanding

almost certainly for (long chain of reasons) macroexpand is failing to resolve l/def-lambda-fn to what you think it should resolve to, so it doesn't know it is a macro and does nothing

but if you use syntax quote l/def-lambd-fn will be resolved at read time to what you expect

but it does so in repl… How should i quote it? Because i have quoted it already

with syntax quote

the compilation and runtime contexts for tests run via a tool are often different, but for the repl they are almost always the same

‘(l/def-lambda-fn, here i have a quote symbol, or should i call “quote” explicitly?

do you now know what syntax quote is? because the def-lambda-fn macro is using it

https://clojure.org/reference/reader#syntax-quote

i understand what ` symbol do, but i am not sure what do u mean

Java 9 RC1 http://mail.openjdk.java.net/pipermail/jdk9-dev/2017-August/005940.html

Warning, the current releases of leiningen and boot will fail on that without modifications to them ^

i got it. i used ' instead of `

@hiredman thank you for answers

@tbaldridge can you give an example of replacing if with math?

(my question comes from day-to-day application/ui code, not from some low level or algorithmic one, if that makes any difference)

@misha sure, in Clojure sometimes you have to count the number of set bits in an int

you could use a loop, or use this: http://graphics.stanford.edu/~seander/bithacks.html#CountBitsSetParallel

got it

lein repl takes about 50s to start on my laptop for a small project with no so many dependencies. Is that to be expected?

@hmaurer What do you have in ~/.lein/profiles.clj? Maybe something there is causing the problem?

Else perhaps share your project.clj?

clojure starts in 3.8 seconds on a Raspberry Pi 3

... while Running Java 9. 4.37 seconds on java 8.

# raspberry pi 3
[[email protected] ~]$ time ./jdk-9/bin/java -jar clojure-1.8.0.jar -e :foo
:foo

real	0m3.876s
user	0m4.170s
sys	0m0.140s
[[email protected] ~]$ time ./jdk1.8.0_144/bin/java -jar clojure-1.8.0.jar -e :foo
:foo

real	0m4.379s
user	0m3.840s
sys	0m0.150s

Not bad, right?

they must have made some ARM backend improvements for Java 9, because Java 9 startup has regressed a smidge on x86-64

😄

(for comparison, 2012 iMac, Clojure startup is around 0.750s so, yeah, Go Pi!)

It improves to 3.55s on java 9 when Clojure is compiled with JDK8 level bytecode.

A Raspberry Pi is a totally sufficient remote REPL if you are fine with a 920MB of usable heap.

You, however, would not enjoy running lein on it. tools.deps will soon be sufficient for bootstrapping a JVM.

I'm going to try to measure lein startup on it. It'll take a moment because I don't have it installed

[[email protected] ~]$ time JAVA_CMD=$PWD/jdk1.8.0_144/bin/java ./lein run -m clojure.main -e :foo
Java HotSpot(TM) Client VM warning: TieredCompilation is disabled in this release.
:foo

real	0m6.555s
user	0m7.670s
sys	0m0.260s

not as bad as I thought, but still double clojure.main. lein currently fails 💥 outright on Java 9 because it puts Clojure on the special bootclasspath and there are more restrictions on that in Java 9