Fork me on GitHub
#clojure
<
2022-04-09
>
jobhdez03:04:12

Hey guys and ladies, I dont know if you know Peter Norvig is but he wrote an essay on how to become a programmer: https://norvig.com/21-days.html. In this essay he says to learn 6 programming languages one of which is clojure as it emphasizes parallelism. can you guys tell me how exactly clojure emphasizes parallelism? and if you can mention any books or articles related to clojure's parallelism

dgb2308:04:55

Idiomatic Clojure is by default free of race conditions. The workhorse here is the atom. It’s a reference type that holds state and changing its contents with swap is atomic (will succeed in full or retry). This allows for a consistent, sequential view of the world in a concurrent program. Pair it with functional code and you are mostly freed from concurrency issues. A bunch of keywords to explore: sequential consistency compare and swap mutual exclusion data race race condition atomic operation

👍 4
mars0i18:04:30

I endorse the previous comments, but: You get can get a very simple demonstration of parallelism in Clojure with pmap (or one of its siblings such as pcalls). Take a map expression in which you map a function that is slow because it does a lot of computation (or because you put a delay in it) over a sequence. Wrap it in doall. Then time it, ideallying using the Criterium library, or you can use time if you do the same thing a few times so that the just-in-time compiler will figure out how to optimize your computation. Next, do the same thing, but replace map with pmap. If the function that you are mapping over the sequence is slow, you should find that the pmap version is significantly faster, because it's splitting the application of the mapped function across different cores on your machine. That's it. Just add "p" to "map". Note, however, that if the function you're mapping is not computationally expensive or otherwise slow, the version with pmap might be slower than the version with map, because there's a cost to splitting up the computation across cores. (I love pmap. So simple. It's mentioned in the Eric Norman tutorial, but it's kind of buried. Obviously not good for every case.)

👍 1
didibus20:04:22

To add clarity, the others were mostly talking about concurrency and that is not the same thing as parallelism, even though Clojure is good at that as well, and will tend to dispatch concurrent tasks in ways that make them parallel when possible. Clojure emphasizes parallelism by supporting multi-threaded and multi-core programming. The run-time can execute computations on multiple computer cores at the same time, thus making the computation parallel. It is well suited for this kind of work, because generally in order to split computation across many "compute node", you need some form of a data manipulation abstraction that does map/reduce like work. Split up the input data, send it to the multiple "compute node", recombine their results back. This itself is based on an immutable foundation; you don't want each core to share mutable variables, because that means they will contend on reading/writing to them. Like @U4HSMRQQJ said, you can do so in Clojure using pmap, pcalls, pvalues and r/fold. These are your parallel map/reduce functions.

didibus21:04:54

I'd say the downside here is that parallel computation hasn't proved itself to be that much faster in practice on current CPU. At the time Clojure came out, and I believe Norvig wrote that article, people thought that soon enough, CPUs would be massively multi-core, and it be the norm to have 32, 64 or more cores on a CPU. But somehow, it is still mostly common to only have around 8 cores. It also seems the hardware architecture still have overhead when parallelizing compute, and all the book-keeping tends to eat at the speedups of what 8 cores gives you, often not resulting in much speedup. That means, for now, it turns out maximizing for cache hits design on a single core still seem to yield better compute performance sometimes. Or people have just skipped the CPU altogether, going to the GPU for compute, where GPUs tend to have hundred+ cores.

👍 1
Benjamin C22:04:52

Hmm, make me wonder if someone has made a plet (& etc.) that utilizes the gpu. :thinking_face:

didibus23:04:03

Hum, I'm not sure the GPU can be used like that. Operations on a GPU are different than on a CPU, so I'm not sure you can get let like semantics there. But yes, you can use the GPU from Clojure, these are the top 2 libs I know for it: • https://clojurecuda.uncomplicate.org/https://clojurecl.uncomplicate.org/ Though you might be better served to use the higher-level Neanderthal directly: https://neanderthal.uncomplicate.org/articles/tutorial_opencl.html Which supports using GPU as a backend for matrix computation, though there are some caveats again, because GPUs are so different to CPUs, which that article explains.

👀 2
jtth19:04:52

Is there an idiomatic way of converting a nested map to a namespaced map? e.g.,

{:name {:title "Ms", :first "Lydia", :last "Brooks"},
 :email "",
 :phone "(257)-666-4912",
 :picture {:large "",
           :medium "",
           :thumbnail ""},
 :department "Legal"}
becomes
{:name/title "Ms"
 :name/first "Lydia"
 :name/last "Brooks"
 :phone "(257)-666-4912"
 :picture/large ""
 :picture/medium ""
 :picture/small ""}
 :department "Legal"
This does not have to be robust, just wondering if this is a solved problem or if there’s a handy util?

didibus20:04:46

Don't think so, your scheme for flattening the hierarchy would probably be custom anyways, like if there is more than one level? clojure.walk is probably a good place to start to implement this.

didibus20:04:25

Or just a reduce if you only care about one level deep

didibus22:04:21

(defn flatten-entity
  [entity]
  (reduce-kv
   (fn[acc k v]
     (if (map? v)
       (merge
        acc
        (reduce-kv (fn[acc2 k2 v2]
                     (assoc acc2 (keyword (name k) (name k2)) v2))
                   {}
                   v))
       (assoc acc k v)))
   {}
   entity))

didibus22:04:27

For example, this does it for one level only. For more levels, you'll have to decide what you want the keys to become, like say :name/title.something or :name.title/something

jtth23:04:07

Ahhh, I see! Thank you so much, both for your example and your explanation about why there isn’t a general method!

mike_ananev21:04:17

Hello! Is there any way to run transcriptor tests in .repl files using kaocha and get correct report?

didibus23:04:59

Is there a way to have a function return a primitive array in Clojure? Or can functions only return primitive long or double? I was under the impression you could at least pass-in a primitive array without downcasting from object, am I wrong here too?