Fork me on GitHub
#beginners
<
2020-09-15
>
Trung Dinh05:09:59

Hi team, I’m trying to make concurrent calls to apis to gather paginated results, I’m thinking of using atom, future and loop for that, my take so far is as follows, but not sure how do I ensure not breaking out of loop before all future calls finish? Any advise please? Thanks

(let [panda-mappings (atom {})
      range 200]
  (loop [offset 0]
    (future (swap! panda-mappings assoc offset "val"))
    (if (> offset range)
      (println @panda-mappings)
      (recur (+ offset 50))))
  (println @panda-mappings))

Trung Dinh05:09:04

I’m trying to get something like this instead…

{100 val, 200 val, 150 val, 250 val, 0 val, 50 val}

dpsutton05:09:12

clojure.core.async has some ideas about how to communicate like this. you could check out clojure.core.async/pipeline-blocking

3
Trung Dinh05:09:45

you mean future is not the right tool for this kind of problem?

dpsutton06:09:59

there are many tools for this. but core.async can give you good coordination tools for your problem of waiting until all the requests finish

dpsutton06:09:26

pipeline-blocking will actually use thread underneath it but with async channels to communicate results and pending results

cgrand07:09:29

I second looking at pipeline-blocking. However the way you use swap! and future can’t work. If you want to use futures you must use them to perform the actual fetch and you have to store the future inside the map. Then once all futures started you have to wait.

👍 3
cgrand07:09:06

(loop [futures {} [offset & offsets] (range 0 250 50)]
  (if offset
    (recur (assoc futures offset (future (DO-FETCH offset))) offsets)
    ; done, let's wait
    (into {} (map (fn [[o f]] [o @f])) futures)))

parrot 9
Trung Dinh08:09:02

nice @U3E46Q1DG, thanks for the idea of putting all the future in one collections and wait

Trung Dinh08:09:29

my attempt was

(let [panda-mappings (atom {})
      range 200
      futures []]
  (loop [offset 0]
    (conj futures (future (swap! panda-mappings assoc offset "val")))
    (when (< offset range)
      (recur (+ offset 50))))
  (when (every? realized? futures)
    (println @panda-mappings)))

Trung Dinh08:09:13

and it works 🙂 , will need to learn your example a bit more, seem a bit cryptic for me the map part…, anyway, thanks

baptiste-from-paris12:09:27

@U018XDZJ4RG, completing @U3E46Q1DG example, you *could use (deref ref timeout-ms timeout-val) with a timeout value. It’s not perfect as you don’t handle errors but it *might be better than nothing depending on your context

👍 3
noisesmith17:09:50

many people find it counter-intuitive, but when I do side effects collecting results across a collection, I prefer reduce or into over loop since they bake in the traversal pattern

(into {}
      ;; wait on and deref futures via transducer
      (map (fn [[k v]]
             [k @v]))
      ;; eagerly start futures before further processing
      (mapv (fn [o]
              [o (future (DO-FETCH o))])
            (range 0 250 50)))

noisesmith17:09:37

the whole thing turns into a few very common idioms (something I aim for)

Trung Dinh02:09:12

cool @U051SS2EU, really like this idiomatic way, thanks. One more question please, when I try doing (map (fn [[k v]] @v)) instead of (map (fn [[k v]] [k @v])) , to get all results without the indices to the map, I get “Vector arg to map conj must be a pair” IllegalArgumentException, I’ve been mucking around without luck, any thoughts?

noisesmith02:09:58

you can only conj a key/value pair to a hash map

noisesmith02:09:47

replace the {} with #{} or [] and it works, if you didn't need the keys

3
zackteo12:09:39

Am following a Clojure Tutorial - Anyone has an idea why (shutdown-agents) causes .... : "Exception updating the ns-cache #error"? I can skip this but am just wondering o:

(defn agent-ex []
  (def tickets-sold (agent 0))
  (send tickets-sold + 15)
  (println)
  (println "Tickets " @tickets-sold)
  
  (send tickets-sold + 10)
  (await-for 100 tickets-sold)
  (println "Tickets " @tickets-sold)
  (shutdown-agents))

(defn -main [& args]
  (agent-ex)
  )

seancorfield17:09:41

@UUSQHP535 def should only be used at the top-level. If you want a local binding, use let instead.

seancorfield17:09:23

And, yeah, most REPL tooling will break if you call (shutdown-agents) since it will kill off stuff inside the tooling, not just your program.

seancorfield17:09:45

The safer idea is to move the (shutdown-agents) call into -main so it's

(defn -main [& args]
  (agent-ex)
  (shutdown-agents))
that makes your agent-ex function safe to call from the REPL.

Alex Miller (Clojure team)13:09:57

That's not a message from Clojure itself, must be something from your tooling. If something in your tooling is using agents or futures, shutdown-agents could be breaking the tooling itself possibly

zackteo13:09:43

Right! Maybe NREPL itself then o:

zackteo13:09:39

Right! Thanks for your help! 🙂

Matthew Pettis19:09:40

I'm trying to use Double/isNaN as a predicate in some, but clojure interprets it as a static field. Is this the idiomatic way of dealing with such things?

Matthew Pettis19:09:47

rules-engine.thrashing02=> (Double/isNaN ##NaN)
true

rules-engine.thrashing02=> (some Double/isNaN [##NaN 2])
Syntax error compiling at (form-init5367077877367404867.clj:1:1).
Unable to find static field: isNaN in class java.lang.Double

rules-engine.thrashing02=> (some #(Double/isNaN %) [##NaN 2])
true

dpsutton19:09:34

#(Double/isNan %) is the idiomatic way

👍 3
dpsutton19:09:10

i believe the reason is that methods are not first class objects on the JVM.

✔️ 3
Matthew Pettis19:09:22

Ok, thanks. Only quirk I have is that this is already in an anonymous function, which I cannot nest, so I have to do something else. But this makes sense, thank you.

dpsutton19:09:06

make either the outer or the inner a (fn [thing] ...) version. probably the outer since #(Double/isNan %) is quite simple

Matthew Pettis19:09:32

yep, that's what I did:

👍 3
Matthew Pettis19:09:39

(fn [it] (Double/isNaN it))

Matthew Pettis19:09:20

and yep, it's simple, so the fn construct works well.

Alex Miller (Clojure team)20:09:14

it might be worth making a nan? function in core

9
teodorlu07:09:22

I really appreciate the built-in predicates for string?, vector?, etc, rather than having to go through another function, like Python's isinstance("sheep", str), especially in combination with other functions like some? or map.

Nazar Trut21:09:40

(def first-test '(not x))
(def second-test '(not (not a)))
(def third-test '(not (not (and a b))))
(def fourth-test '(not (not (not (not c)))))
(defn not-elmination
  [expression]
  (cond
    (= first-test expression) set '{}
    (= second-test expression) set '{a}
    (= third-test expression) set '{(and a b)}
    (= fourth-test expression) set '{(not (not c))}

    :else set '{})

  )
I am trying to return sets here and im getting this error "Map literal must contain an even number of forms"

Nazar Trut21:09:49

Anyone know why?

bronsa21:09:10

#{} is the set literal, {} is the map literal

bronsa21:09:49

set there makes no sense, what are you trying to do

Nazar Trut21:09:40

So if the user enters '(not x), the cond checks and i want to return a set {(not x)}

bronsa21:09:09

ok, set '{(not x)} makes no sense, you want #{'(not x)}

bronsa21:09:59

'#{(not x)} also works, it'll quote the entire set instead of the specific element but in this case they're the same thing

Nazar Trut21:09:46

So i should be returning #'{a} for example

bronsa21:09:32

those are equivalent to #{(quote a)} and (quote #{a}) respectively

bronsa21:09:48

#'{ is not valid syntax

bronsa21:09:09

the syntax for a set is #{ elements.. }

bronsa21:09:49

so you can use ' either in front of the whole set to quote all the values in the set, or in front of an element in the set

Nazar Trut21:09:58

Ok thanks for the information, I will apply that to my code rn

Nazar Trut21:09:47

hmm now im getting "cond requires an even number of forms"

Nazar Trut21:09:52

(def second-test '(not (not a)))
(def third-test '(not (not (and a b))))
(def fourth-test '(not (not (not (not c)))))
(defn not-elmination
  [expression]
  (cond
    (= second-test expression)  #{'a}
    (= third-test expression) #{'(and a b)}
    (= fourth-test expression) #{'(not (not c))}
    :else set '#{})

  )

seancorfield21:09:56

@ntrut Parentheses are important in Clojure and mean "function call" (in general) so set '{} is two unrelated expressions but (set '{}) would be a function call -- and would call set on '{}

bronsa21:09:04

:else set '#{}is wrong

seancorfield21:09:22

(because it is three separate expressions)

Nazar Trut21:09:41

Sorry, its hard getting used to clojure, just recently starting learning it in one of my classes

seancorfield21:09:05

No worries. Lisps are always a bit tricky to get used to.

seancorfield21:09:18

And it also depends what languages you're used to.

seancorfield21:09:57

In Java or similar: obj.method(arg) In Clojure (method obj arg)

Nazar Trut21:09:21

Defiantly not used to this, I code in Java/C/Pyhon

Nazar Trut21:09:34

New territory for me here

parens 9
Nazar Trut21:09:59

What is wrong with my else?

dpsutton21:09:31

:else expresssion but you have :else a-function an-empty-set

dpsutton21:09:35

(also, you can just use #{} as an empty set. there are no forms in it that you are quoting)

Nazar Trut21:09:46

Yep using #{} fixed my else problem and now my program works

Nazar Trut21:09:48

thanks people

eval-on-point22:09:41

Regarding the infer-externs cljs compiler option: Are you intended to move the generated externs in inferred_externs.js to a file that you use for advanced compile? Or is it supposed to be enough to use type hinting and infer-externs when you use advanced compile?

eval-on-point00:09:53

I think I found the answer https://gist.github.com/swannodette/4fc9ccc13f62c66456daf19c47692799:

Simply add a new compiler option :infer-externs true to your compiler config. Now when you build your project you will get a new file in your :output-dir named inferred_externs.js. When you do an advanced build, this externs file will be used.
So, your inferred_externs should be in the :output-dir of your advanced build. Therefore, you might need to check it in to source control

seancorfield22:09:25

@mitchell_clojure If no one answers here, you might try #clojurescript -- I'm not sure how many active folks here are familiar with ClojureScript compilation issues.

eval-on-point22:09:00

Thanks Sean! Hard to know sometimes if I am missing something simple sometimes

Joel23:09:30

if i have an defrecord instance, is there a simple way to create a new one with just one value substituted, sort of like update-in does for maps?

Alex Miller (Clojure team)23:09:57

you can use update

💯 3
Alex Miller (Clojure team)23:09:44

user=> (defrecord Point [x y])
user.Point
user=> (def r (->Point 1 2))
#'user/r
user=> (update r :x inc)
#user.Point{:x 2, :y 2}
user=> (assoc r :x 10)
#user.Point{:x 10, :y 2}