i'm having a bit of trouble wrapping my head around core.asyc in the browser. specifically, i have a handful of channels that were returned from HTTP requests and i want to get their results in the order that I made the requests, not in the order they were returned. if i manually bind each value from each channel individually then it works. but if i map <! over the channels i just get back a list of channels. can anyone explain what i'm doing wrong?

(let [response-channels (map #(http/get "" {:with-credentials? false}) (range 3))]
  ; Response is now three channels generated by http/get:
  ;(#object[cljs.core.async.impl.channels.ManyToManyChannel]
  ; #object[cljs.core.async.impl.channels.ManyToManyChannel]
  ; #object[cljs.core.async.impl.channels.ManyToManyChannel])

  ; If I want the results back in the guaranteed order that I made them, I can do this:
  (go (let [response1 (<! (nth response-channels 0))
            response2 (<! (nth response-channels 1))
            response3 (<! (nth response-channels 2))]
        (println "this works fine" response1 response2 response3)))


  ; But if I try to map <! over the channels instead, I just get back a list of channels
  (let [responses (into [] (map (fn [c] (go (<! c))) response-channels))]
    (println "responses" responses)
    ; This is still just a vec of many-to-many channels
    ; [#object[cljs.core.async.impl.channels.ManyToManyChannel]
    ; #object[cljs.core.async.impl.channels.ManyToManyChannel]
    ; #object[cljs.core.async.impl.channels.ManyToManyChannel]]
    )
  )

and this returns an error that i'm using <! outside of a go block:

(go (into [] (map <! response-channels)))

i was tempted to use core.async/merge and core.async/reduce over the channels which works but there's no guaranteed order which i need

scratch that, solved by doseq!

okay, retracting the resolution. doseq is used for side effects and does not return results in a data structure, therefore i can't return them from the function. so the problem remains.

@rodeorockstar: Simple, kinda hackish solution I just came up with: Just tag the response with something that lets you determine original order.

Actually, rereading your problem... hm.

thanks @meowy, that's definitely an option! i've done just that in the past for simple requests. in this particular case i'm running complex queries through a web service, so i think the "key" to matching the results back to their original requests would be the entire input to the request itself. that doesn't feel as efficient as it could be.

but i suppose (=) is cheap

I suppose that could work.

an alternative to doseq could be for, perhaps?

https://github.com/clojure/core.async/blob/master/examples/walkthrough.clj#L97-L108

I'm not sure if this is applicable, but... if you have a "batch job" of HTTP requests, you could apply a similar strategy as in this example, using alts!.

Note that a response must be returned by all channels that you use like this, because otherwise, this will deadlock.

You’re not mapping <! over the channels, you’re mapping (go (<! …)) over the channels. The go macro returns a channel.

okay that's making some sense

and thanks meowy. i'll take a look at the walkthrough

@hrathod i tried for but ended up with the same problem: a list of channels rather than their results, presumably because of the go block inside as donaldball just mentioned

You probably want something along the lines of (go (map <! response-channels)) and I guess take! from that?

oh, maybe i could merge the resulting channels from that and then reduce over them. hopefully they'd be in the correct order

There will only be one resulting channel from that, and it will contain the first value put on each of the response channels in order

in a seq

What I'm worried about with that approach is that channel 2 might return a value before channel 1 does, if you actually process them in order.

But then... it probably doesn't matter, anyway, because there's not really a time difference, is there?

It doesn’t matter. The <! from the first channel will block until the channel receives a value or closes. The fact that there are values pending on the other channels is irrelevant because no one is trying to take from them yet.

Yeah.

You need the results of all the channels, anyway, so it doesn't matter whether you get channel 2 and then 1 (because 2 was faster), or just wait on 1 and then get the value from 2 immediately (because it was done already).

interesting. i'm testing now.

@rodeorockstar: take a look to http://clojure.github.io/core.async/#clojure.core.async/merge

hmm, i can't seem to avoid Uncaught Error: <! used not in (go ...) block

@niwinz thanks. the problem with merge is that the ordered isn't guaranteed.

you need order?

ok wait a moment

Guess you could tag the output then, as I suggested. Given that it's HTTP requests, you could tag them with the time of arrival, and given sufficient precision of the timestamp, there shouldn't be an issue sorting them.

i shoudn't say that's the problem with merge, i should say that's the problem with me 😉

That, or you associate an input with some sort of tag, and get the right things for the right inputs after the merge.

actually yeah, i could just return some sort of index

https://gist.github.com/stuartsierra/8364220

https://groups.google.com/forum/#!topic/clojure/kjdgOdRpogM

the concat described on the gist should work

is like merge but works sequentially

@rodeorockstar: If you wait for all channels to have a result anyways, then you can just use core.async/map

would i wait via blocking?

(i'm trying the gist now, niwinz)

Uncaught Error: No protocol method ReadPort.take! defined for type cljs.core/LazySeq: (#object[cljs.core.async.impl.channels.ManyToManyChannel] #object[cljs.core.async.impl.channels.ManyToManyChannel] #object[cljs.core.async.impl.channels.ManyToManyChannel])

seems like you are passing a wrong data to concat

ah yeah, oops.

concat expects call similar to (concat ch1 ch2 ch3)

not (concat [ch1 ch2 ch3])

if you have a vector, just use apply

(apply concat [ch1 ch2 ch3])

apologies - i've been staring at this stuff for a while. can't quite see the forest through the trees. 😉

hmm, close, but i'm only getting back the value from one channel instead of all three:

(defn concater [& chans]
  (let [out (chan)]
    (go-loop [[ch & more] chans]
             (when ch
               (let [val (<! ch)]
                 (if (nil? val)
                   (recur more)
                   (do (>! out val)
                       (recur chans))))))
    out))

(let [response-channels (map #(http/get "" {:with-credentials? false}) (range 3))]
  (go (println "results:" (<! (apply concater response-channels))))
  )

results: {:status 200, :success true, :body {:time 02:43:11 PM, :milliseconds_since_epoch 1466692991958, :date 06-23-2016}, :headers {content-type application/json; charset=ISO-8859-1, cache-control private}, :trace-redirects [ ], :error-code :no-error, :error-text }

yes, becuase the concater returns a channel of results

if you only take 1 only

if you want a coll of results

try to put the concater return value in (<! (a/into [] (apply concater response-channels)))

a is a namespace alias for clojure.core.async

okay, just tried and i get back nothing at all this time.

(let [response-channels (map #(http/get "" {:with-credentials? false}) (range 3))]
  (go (println "results:" (<! (a/into [] (apply concater response-channels))))))

oh yes, this is because the concater has a bug 😞

it does not closes the channel, sorry for point you to a badly implemented concat

that's fine! it's a good start. i was about to ask about closing channels anyway.

otherwise i'll never know when i'm finished

(defn concater 
  [& chans]
  (let [out (chan)]
    (go-loop [[ch & more] chans]
      (if ch
        (let [val (<! ch)]
          (if (nil? val)
            (recur more)
            (do (>! out val)
                (recur chans))))
        (close! out)))
    out))

try this concater,

that did it!

thanks a lot, niwinz (and meowy and everyone else)

😉

What's the idiomatic way in Java 8 to make #insts if you have year, month, day as numbers?

(java.util.Date. year month day) is deprecated.

And you could

(java.util.Date/from
 (.. (java.time.LocalDate/of year month day)
     (atStartOfDay)
     (toInstant java.time.ZoneOffset/UTC)))

(I know about JodaTime.)

I’ve been using java-time recently

It aims to be the wrapper for java8’s java.time library

I’ve been sticking with https://clojars.org/clj-time.

@zane Oh, many minds have been lost trying to work with java dates and times. The first "official" replacement was the Calendar classes, they're (IMO, obv) a bunch of crap and worth avoiding. The sane options now are joda, clj-time (which uses joda internally) and the newly added java.time which is heavily based on Joda. If you have to work with j.u.Date or don't want to use joda/clj-time then I'd really recommend just using that same deprecated constructor and ignoring the deprecation warnings from it