Fork me on GitHub
#beginners
<
2022-07-05
>
dumrat01:07:51

Is there any construct like this in clojure where all sub expression would be run parallel and results will be returned after the longest running expression returns?

(run-parallelly
  (slurp "")
  (Thread/sleep 2000)
  (println "All run parallel")) 

=> ("<!doctype....", nil, nil)

hiredman02:07:12

(map deref [(future ...) ...])

hiredman02:07:56

There are actually built in things for this, maybe pcalls? I forget the name, but it does something slightly more complicated similar to pmap, which I have a low opinion of and never use

dumrat02:07:25

Why do you have a low opinion of pmap and what's an alternative?

hiredman02:07:33

It doesn't launch parallel tasks until you consume the lazy seq it produces, but also produces a little more than what you ask for in parallel, and also produces results in order

👍 1
hiredman02:07:21

Basically bakes in a bunch of decisions that don't often match what I want from parallel execution

hiredman03:07:08

I think the claypool library tries to provide more flexible versions of pmap and pcalls, but I've never used those either

hiredman03:07:56

I (perhaps too readily) just build what I want out of java.util.concurrent

hiredman02:07:39

The docstring for pvalues would make you believe it is run-parallelly, but it is implemented using pmap, the parallel execution is entwined with the lazy realization of the seq, which complicated things

dumrat03:07:29

I tried writing a macro to run all expressions in a body in parallel. This seems to work:

(defmacro run-parallel [& body]
  (list 'doall
    (list 'map 'deref
      (into []
        (for [x body] (list 'future x))))))

(macroexpand-1 '(run-parallel (+ 1 2) (Thread/sleep 1000))) => (doall (map deref [(future (+ 1 2)) (future (Thread/sleep 1000))]))

(run-parallel (+ 1 2) 
              (Thread/sleep 1000)) => (3 nil)
I can't seem to write this macro with backquote though. Everything I try fails. Any help to write with backquote?

tomd06:07:52

(defmacro run-parallel [& body]
  `(doall
    (map deref 
         ~(into []
                (for [x body]
                  `(future ~x))))))

tomd06:07:43

although sticking with vectors:

(defmacro run-parallel [& body]
  `(mapv deref
         ~(mapv (partial list `future)
                body)))
may be preferable

👍 1
popeye12:07:23

why so ?

user=> (compare 2 10)
-1
user=> (compare "2" "10")
1
user=> 

popeye12:07:21

thanks for response @U04V4KLKC, Is there any way to handle it? except changing to Numbers ?

popeye12:07:41

becasue my column may also have string , so I have converted all the numbers into strings and testing

delaguardo12:07:24

sort accepts optional comparator. you can configure it to do anything you want including comparison of numbers with strings

popeye12:07:50

including comparison of numbers with strings what that means ?

delaguardo12:07:35

(compare "0" 1) will throw by default because doesn’t know how to compare string and number. you can create a function that will return 0, 1 or -1 in that case and use it as a comparator

Pedro Boschi12:07:51

@U01J3DB39R6 if your data is always a number, but sometimes it as string, you can create a function that first "left-pad" the shorter string with zeros, and then calls the comparison. This will ensure that even string's lexicographical comparison will work for your number comparison

vanelsas12:07:20

because in characters "1" comes before "2" ?

Alastair Hole15:07:21

I am trying to get my head around the idea that Clojure code is executed at compile time: I have just switched to using Omniconf (after having just hardcoded strings for config during local prototyping) - where previously compilation was successful due to all the config values being available (at this point I was even unaware that code is executed at compile time), now they are loaded at runtime with Omiconf I am getting a raft of null pointer exceptions and similar at compile time. What is the recommended method to mitigate this? Should I somehow defer execution of code that relies on runtime config?

Alex Miller (Clojure team)15:07:56

compiling will load every namespace being compiled, so top-level def's are usually problematic in this regard (I treat them all with suspicion) - consider pushing more into defn's, invoked via starts, or using delay

👍 4
Alastair Hole15:07:01

Thanks 🙂 Presumably then top-level defs should be used sparingly, for things with no/few side effects/dependencies? Would it be sufficient to move the defs to an ‘init’ (start?) defn and call that at runtime (via -main)?

hiredman15:07:45

Defs inside things is bad

Alastair Hole15:07:51

Ah OK, yeah it did seem like cheating

Alastair Hole15:07:28

What is meant by ‘starts’?

Alex Miller (Clojure team)15:07:39

sorry, meant a starting function, whatever your entry point is

Alastair Hole15:07:15

Ah I see, thanks 🙂

Alex Miller (Clojure team)15:07:15

which presumably will not be called during compilation (just loading) but will be called when you run your app

Alex Miller (Clojure team)15:07:49

this may require you to pass more state through things rather than using global defs. that is a good direction to go.

Alastair Hole15:07:42

Great, yeah I think that was starting to dawn on me. Here I guess really the top level defs are just a lazy way to share a config dependency across defns

Alastair Hole16:07:26

So yeah I think I will pull them into a defn and call that in the entry point and pass them into the defns that need it

Alex Miller (Clojure team)16:07:53

you'll thank me a year from now :)

Alastair Hole16:07:25

😄 I certainly also thank you today

Alastair Hole16:07:28

I was definitely over-thinking it, the concept of code executing at compile time blew my tiny mind and threw me off common sense