core-async 2021-11-09 | Slack Archive

rende1107:11:07

Hello! I wanna make several async fetches transform somehow responses from each one and collect all results. Is it's possible with core.async ? Something like this:

;; asynchronously retrieve and transform data
(let [users (xf-users (<! (<get-users)))
      articles (xf-articles (<! (<get-articles)))
      comments (xf-comments (<! (<get-comments)))
;; somehow wait until all results will available and return
  {:users users
   :articles articles
   :comments comments})

How to do it in a right way?

dergutemoritz10:11:15

So you want to do the three fetches + transforms in parallel instead of sequentially like you do in your example?

rende1116:11:38

Yes!

rende1116:11:17

Some sort of Promise.all in JS

dergutemoritz11:11:16

You could do something like this:

(let [users (go (xf-users (<! (<get-users))))
      articles (go (xf-articles (<! (<get-articles))))
      comments (go (xf-comments (<! (<get-comments))))]
  {:users (<! users)
   :articles (<! articles)
   :comments (<! comments)})

But you need to be careful with the xf- functions throwing errors - these would go pretty much unnoticed in this snippet (you'd end up with a nil value but the exception itself would not be logged anywhere by default). There are various ways to deal with this, e.g. you could use https://github.com/alexanderkiel/async-error. Alternatively, since you're apparently familiar with the promise-based approach, you could check whether https://github.com/funcool/promesa might fit your usecase better.

dergutemoritz10:11:11

Hey folks, just spent a few hours chasing down a very erratic error which turned out to be caused by a known bug: https://clojure.atlassian.net/browse/ASYNC-204 - would really appreciate if somebody could apply @cgrand's attached patch there, looks quite straight forward. I'm sure this would save other poor souls like me quite some nerves 🙂 Thanks 🙇

Alex Miller (Clojure team)14:11:56

I will try to take a new look at it this week. I know I have looked at it in the past, don't remember why I did not apply it

Alex Miller (Clojure team)15:11:06

might just have been the time to adequately review the proposal

dergutemoritz15:11:06

@alexmiller Thanks a lot, that's great news 🙂

Alex Miller (Clojure team)16:11:36

this code is both very dense, and in the path of everything using core.async so understanding the code, and the problem, and the proposed fix and considering other possible solutions can take hours

dergutemoritz16:11:14

Heh, not as straight forward as it appears then, I guess!

Alex Miller (Clojure team)16:11:38

and depending what the change is, I may also need to run it past Rich

2021-11-09

Channels