Clojurians Log v2

I'd just use js/Promise.all.

You can just use map and then Promise.all to turn the promises in a single Promise of an array of values

Does that work with Clojure-specific collections, then? Like a seq returned from map?

I think it does

If not you could turn it into an array first

That's true. I'll test it out. Thanks!

It does work, yes.

Currently I have a map function like:

(map #(update % :data decrypt key) messages)

(js/Promise.all
 (map (fn [{:keys [data] :as msg}]
        (-> (decrypt data key)
            (.then #(assoc msg :data %))))
      messages))

(defn promised-update [m k f & args]
  (-> (apply f (m k) args) (.then #(assoc m k %)))

Promises are basically monads except not in JS since you can't nest a promise, it automatically unnests. Anyway that's a tangent

Out of curiosity, what is the next step in the process? What are you doing with the decrypted messages in this case?

Rendering them on the page. The server stores only the encrypted messages, which are then decrypted client-side.

I see - I would say that https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled is also worth knowing about. And @borkdude's ongoing work around await will also likely be relevant; ideally a beautiful (not overly indented and .then-laden) piece of code could support decrypting the messages separately and individually and put them in the frontend db for rendering as they became ready without waiting for them all to be done and succeed. It's a good real-world case.

What's the difference between all and allSettled?

.allSettled returns an array of objects that contain a value and an error status. .allSettled runs until all promises are handled. .all returns as soon as one of the promises fails with just one error

Promise.allSettled([
  Promise.resolve(33),
  new Promise((resolve) => setTimeout(() => resolve(66), 0)),
  99,
  Promise.reject(new Error("an error")),
]).then((values) => console.log(values));

// [
//   { status: 'fulfilled', value: 33 },
//   { status: 'fulfilled', value: 66 },
//   { status: 'fulfilled', value: 99 },
//   { status: 'rejected', reason: Error: an error }
// ]

Thanks for the explanation!

(defn- ^:async decrypt&
  [{:keys [data] :as msg}]
  (-> (assoc msg :data (await (decrypt data key)))
      (put-in-frontend-db!)))

(run! decrypt& messages)

run! doesn't return a promise and isn't async aware.

Does it matter? It's only for the side effects, here.

Depends on whether or not the order is important and whether or not partial results are OK.

Great thoughts @p-himik... @weavejester mentioned above that they have sequence numbers, so order is not important --- and partial results being OK seems good to me here in case decrypt does anything weird.

It probably matters since you probably want to do something after the results finish. Anyway, depends on context

The idea is that the put-in-frontend-db! function would take the promises and handle them.

I tend to prefer keeping everything functional up until the last moment. The code I ended up with looks like:

(defn- promised-update [m k f & args]
  (-> (apply f (get m k) args) (.then #(assoc m k %))))

(defn- decrypt-messages [key messages]
  (js/Promise.all (map #(promised-update % :data crypto/decrypt key) messages)))

(defn fetch-messages [key start end]
  (let [query (m/assoc-some {} :start start :end end)]
    (go (let [messages  (-> (<! (http/get "logs" query)) :body :logs)
              decrypted (<p! (decrypt-messages key messages))]
          (m/index-by :id decrypted)))))

(defn update-messages!
  ([store key] (update-messages! store key nil nil))
  ([store key start end]
   (go (let [mesgs (<! (fetch-messages key start end))]
         (swap! store update :messages into mesgs)))))

The messages are stored in a sorted-map, ordered by :id, which is a sequential integer added by the server.

Ahh, if you already use core.async then it might make sense to rely on it more. I myself prefer to avoid it as much as possible on the frontend.

Ah, if you're already using core.async, then https://github.com/funcool/promesa is probably also worth consideration.

🤝 @p-himik

Thanks for the library tip, @hhausman. I'm using core.async because I also have a websocket that listens for incoming events. I use HTTP to retrieve historic/missing logs as then I can preload them in a parallel connection.

That's cool - I imagine that's a lot of fun to program... We don't get to use core.async because there is a perception that it bloats our js artifacts.

It definitely used to, haven't checked in a while. It is (or was) also harder to debug. And go loses ^js metadata. All of that is not a deal breaker by itself, but so far the implementations of my async needs haven't been too ugly without core.async.

Wait until async/await gets merged, then we can rebuild the IOC go macro using that and it will be much smaller (I've heard)