I was going to start with these after looking at the java implementation: https://github.com/SparkFund/google-apps-clj https://gist.github.com/arohner/8d94ee5704b1c0c1b206186525d9f7a7 https://github.com/jonneale/gcal

Inspired by your inspiration @dharrigan

what was it ?

Yeah, I'd like to know too

I also used pmap, to fire off a couple of long running sql queries

(let [workers (pmap #(cleanup-stagnant-data % app-config) [:funky-table-1 :funky-table-2 :funky-table-3])]
    (map #(deref % five-minutes-ms :failure) workers)

I'm sure there are multitude of ways of skinning that cat, but this works for me 🙂

pmap doesn't return a sequence of futures so that doesn't look right to me...

oooh....newbie alert 🙂

I sense a rewriting coming on 🙂

I'll have to read up on the docs again 🙂

(doall (pmap ...)) will realize the whole result. Or you could map a function that does produce a future over a collection and then map deref over that (but you'd still need to force realization).

we map over manifold streams for that use-case @dharrigan , and use a buffered-stream to get concurrency control

okay, I'll have another go! 🙂

And you also have the problem that if the first few elements take a long time, pmap won't necessarily have even started the futures for elements further along the sequence so you don't get the concurrency you might desire anyway 🙂

bummer 😞

Hence why people say pmap is pretty much always the wrong solution 🙂 And executors are easy to use from Clojure anyway.

And yet, we don't know what was @U052SB76M’s usage of pmap 😁

Why is your SQL query taking 5 min? 😬

maybe worth looking at https://github.com/reborg/parallel it has its own pmap implementation https://github.com/reborg/parallel#ppmap

Crikey, I didn't even see this thread

I am familiar with the advice surrounding pmap (and should probably look at reborg's alternative implementation) but my use case fit the bill so well it was too hard to resist

For the record it's some DB queries (n<5) which are all eventually merged into a cache object. Simplified...

(->> queries
     (pmap execute-query)
     (reduce merge))

@U9CQCSTUZ You may be already aware of that, but all the following writings are equivalent:

(:my-kw {} :not-found)
({} :my-kw :not-found)
(get {} :my-kw :not-found)

Note: this form won’t work when the map is nil however.

(nil :my-kw :not-found)

Oh, wow, yes, of course, maps are fns too! and yeah, then it makes sense the NullPointerException on the last form.. 🙂 Thanks!