Fork me on GitHub
#code-reviews
<
2021-12-31
>
Fra06:12:10

Hi, I am trying a little experiment in order to compare java and clojure multithreading performance with shared resources. This is wolfam elementary ca, I have a https://paste.ofcode.org/6YyKBFwjvQ4cSUuQqDZQjM and https://paste.ofcode.org/33pFbyTqsBkb3RZgDDj5QJG. I am trying to use cyclic-barrier in order to start threads at the almost same time, but I can see that it works in java but not in clojure. Do you know why? Thanks for any help

seancorfield06:12:23

@francesco.losciale You have the await calls commented out (and they don't look correct). Can you explain what you've tried and what error or incorrect behavior you're seeing?

seancorfield06:12:30

This looks... suspicious as well: (:require [jdk.util.concurrent.CyclicBarrier :as jdk]) -- that would require a Clojure namespace but I think you're trying to import a Java class?

Fra06:12:42

yes this was strange to me as well, if I open that file I can see a ns

seancorfield06:12:38

What library is that??

seancorfield06:12:11

Wow... why would anyone create a library for basic interop like that? That seems incredibly counter-productive...

seancorfield06:12:03

And it hasn't been updated for years. I definitely would not want to use that...

Fra06:12:57

I definitely agree with you @seancorfield, but it is quite handy for me I am writing throw away code anyway

Fra06:12:47

my goal is to see that the clojure version of wolfram performs better than the java one

Fra06:12:13

I am not sure if this makes sense

seancorfield06:12:36

What do you mean by "clojure version of wolfram"?

Fra06:12:17

I have written a https://paste.ofcode.org/6YyKBFwjvQ4cSUuQqDZQjM and a https://paste.ofcode.org/33pFbyTqsBkb3RZgDDj5QJG of wolfam (at least, I believe they are almost the same thing). I wanted to see what is the execution time of one compared to the other

seancorfield06:12:08

Well, they're not the same thing at all and the Clojure code you've written is going to perform worse than the Java code, even once you have it doing the "same" thing.

seancorfield07:12:49

The Java code is using mutable arrays so that's a big performance advantage over the agent with an immutable vector and assoc...

seancorfield07:12:48

And your Clojure code isn't concurrent (which is why it "hangs") -- you don't have a separate thread for each cell.

seancorfield07:12:40

Performance aside, you can certainly write a Clojure version of that Java code that is much shorter and much more elegant and readable. But getting raw performance out of Clojure to match "native" Java, that is using primitive int and mutable arrays, is going to require a non-idiomatic approach in Clojure and will make for some ugly code.

Fra07:12:06

the non idiomatic approach you mean is basically using arrays with java interop?

seancorfield07:12:07

To have control over threads in Clojure, you need to use the Java Executor classes. The Java code creates a fixed thread pool executor -- so that it can run each cell's process in a separate thread -- and you need to do that in Clojure too.

Fra07:12:54

I see, that doesn’t make sense then to use java in Clojure.

seancorfield07:12:12

To get performance equivalent to primitive int and mutable arrays, you're going to need to use a mutable array of primitive int in Clojure too.

seancorfield07:12:09

"that doesn’t make sense then to use java in Clojure" -- I'm not sure what you mean by this? It's common to use Java interop in Clojure, to leverage the JDK libraries for example.

Fra07:12:13

I am learning Clojure and I hear that it is very good for concurrency so I thought I could write something to prove it (comparing two different languages) but what I wrote is fundamentally wrong. I wonder how would you approach this problem?

seancorfield07:12:20

Clojure is great for concurrency -- because of immutability -- but we mean it is great for getting concurrency right with simpler code. Getting concurrent code correct in Java is much, much harder.

seancorfield07:12:58

That example is very narrowly-focused and if you're optimizing for performance-in-the-small, you're not going to see the benefits of Clojure.

seancorfield07:12:09

It's not a problem I'd be interested in solving -- but I deal with concurrency a lot in my day-to-day work and I'm very glad I'm using Clojure instead of Java because it's "simple" to get concurrent code "in-the-large" working with Clojure but very hard with Java.

seancorfield07:12:49

Performance benchmarking across languages is not a very interesting or useful problem -- as it typically optimizes for non-idiomatic code (I'd argue that your Java code is not idiomatic Java either).

Fra07:12:24

thanks @seancorfield for sharing your thoughts, it totally makes sense

raspasov13:12:30

@francesco.losciale IMO, the place where Clojure's immutability really shines out-of-the-box is concurrent, consistent, correct read access to shared state/data. Even with idiomatic Clojure, in a read-heavy scenario Clojure would probably beat Java, or come very, very close, given idiomatic code on both sides. The difference in code/cognitive complexity would be enormous, in favor of Clojure. The reason being that in Java the idiomatic approach would most likely look something like having a bunch of Objects dancing around, doing their thing, and being updated. When it comes time to read from multiple of those objects, what do you do? You'd probably have to stop all writers, or get involved in some locking scenarios, or get deep into the weeds of java.util.concurrent. In Clojure, you simply (deref an-atom) and you get consistent view of the world. You cannot have that with mutable data.

raspasov13:12:53

Now, you can definitely get immutable data structures into Java, and most likely beat Clojure. But that's definitely not idiomatic Java for 99% of people of doing Java, I think 🙂

raspasov13:12:38

I completely agree with @seancorfield that such across language benchmarks are generally a waste of time. If the JVM/Java performance is good enough for you, chances are very high Clojure will be suitable also. It's not an order of magnitude difference, it's pretty close. The only thing I've heard people worry about is garbage collection pressure in high traffic scenarios due to the fact that immutable data structures on average generate more garbage when being updated. I haven't experienced such case myself, so I can't speak from experience.

👍 1
Drew Verlee01:01:38

Would it be correct to say it will be easier for a dev to write the concurrent functionality they wanted in clojure because immutability will negate the need to manually check state dependencies?

Drew Verlee01:01:41

The difference isn't the language, per say, but the use of more space (immutable structures) to negate the hard to follow behavior of a program/story that you can't read linearly. That a var mutation at a later stage might influence one that happens earlier.

raspasov11:01:50

@U0DJ4T5U1 "immutability will negate the need to manually check state dependencies" I believe this is valid statement. But I think the problem with mutability is way deeper than simply "dependencies". Mutable objects are not information nor data. When you hold a reference to a mutable objects, what are you really holding? In the presence of concurrency and/or asynchrony (even in the case of one thread), you can never really be sure. For any given object, you have to worry "Is this object going to be the same next time I check it?". For certain objects, you might know. You created the object, and nobody else is touching it. You are aware of everything the object does and everything that you do with that object. That is already plenty of things to worry about in my book! For other objects, not so much. Multiple people working on the project, background processes involving IO updating objects/state. Those two talks by Rich Hockey cover the topic in excellent detail, way better than I can describe here: https://www.youtube.com/watch?v=dGVqrGmwOAw https://www.youtube.com/watch?v=E4RarTAZ2AY&amp;t=21s

Fra06:12:55

yes thanks @seancorfield , the behaviour that I see if I use the cyclic-barrier is that everything hangs on, it seems the thread are not started at all

seancorfield06:12:41

Agents process things one at a time, not concurrently. The Java version uses an Executor to queue everything up, ready to run concurrently.