Fork me on GitHub
#clojurescript
<
2023-12-16
>
Fredrik Andersson15:12:08

Hi, I'm trying to use promesa.core/doseq. And I thought that it should be used like this (p/doseq [panel-choice panel-choices] (something-returning-promise panel-choice))))

Fredrik Andersson15:12:56

Where panel-choices is a seq and panel-choice would be bound to each item. Everything executing in sequence. But I get "Use of undeclared Var" for panel-choice

Fredrik Andersson15:12:09

also, if I wanted to do a sequential map. How could I accomplish that?

Chris McCormick07:12:37

> Promesa has a somewhat similar wait-all* I guess p/all is more analogous to js/Promise.all.

Chris McCormick07:12:41

Similar to what Eugene said, if you don't care about computation (but want to use promesa):

(p/all (mapv #(something-returning-promise %) panel-choices))

Chris McCormick07:12:04

You can wait for the result of that computation, which will be an array of the results of something-returning-promise on each member of panel-choices as follows:

(p/let [result-vector
        (p/all (mapv #(something-returning-promise %) panel-choices))]
      (print result-vector))

Chris McCormick07:12:31

If you genuinely care about the order of computation and want it to be sequential like you asked then I think you're either going to have to use reduce to create a stack of promises which wait on the previous result, or else you might be able to use p/loop and p/recur. In my experience this code will end up a bit messy so maybe it's something that can be contributed to the promesa library. think_beret

Chris McCormick07:12:11

If you are using clj-kondo then you can get p/doseq to lint as doseq like this:

{:lint-as {promesa.core/doseq clojure.core/doseq}}
For example try pasting this into your ns:
{:clj-kondo/config '{:lint-as {promesa.core/doseq clojure.core/doseq}}}

Chris McCormick07:12:42

@U01JYUZRS4V ok I managed to figure out how to do the computations sequentially using reduce and waiting for the previous result:

(p/let [result
          (p/all
            (reduce
              (fn [col panel-choice]
                (conj col
                      (p/let [_ (last col)]
                        (something-returning-promise panel-choice))))
              []
              panel-choices))]
    (print "reduce" result))
This code is not the nicest. 😅 What it does is create a sequence of promises with each promise using p/let to wait for the result of the promise before it before running something-returning-promise .

Chris McCormick07:12:09

Definitely use the p/map method if you don't actually care about computation order but only results order.

Chris McCormick07:12:25

Sorry, I can't leave this problem alone. 😁 Just for good measure here is a p/loop/`p/recur` solution which also does sequential computation like the reduce solution:

(p/let [result
          (p/loop [choices panel-choices results []]
            (if (empty? choices)
              results
              (p/let [result (something-returning-promise
                               (first choices))]
                (p/recur (rest choices)
                         (conj results result)))))]
    (print "loop/recur" result))

Chris McCormick07:12:33

Here's a file you can run with nbb which executes all four solutions to show the timing differences.

$ npx nbb promises.cljs 
--> doseq strategy
started 1
done 1
started 2
done 2
started 3
done 3
started 4
done 4
started 5
done 5
RESULT doseq 10 

--> map strategy
started 1
started 2
started 3
started 4
started 5
done 5
done 2
done 3
done 4
done 1
RESULT map [2 4 6 8 10] 

--> reduce strategy
started 1
done 1
started 2
done 2
started 3
done 3
started 4
done 4
started 5
done 5
RESULT reduce [2 4 6 8 10] 

--> loop/recur strategy
started 1
done 1
started 2
done 2
started 3
done 3
started 4
done 4
started 5
done 5
RESULT loop/recur [2 4 6 8 10]

Fredrik Andersson08:12:43

Sorry for being away. Now I'm back and appreciate your efforts. To be clear, order doesn't matter for this particular problem. What really matters is that there must not be any parallel execution because I am using puppeteer to extract data from links. So I need to descend into a link, extract data, get back up and start over.

Fredrik Andersson08:12:12

p/doseq doesn't seem to exist. Must check my versions of promesa

Fredrik Andersson08:12:33

I will continue comment on the other things you've written as I go along

Fredrik Andersson09:12:23

It turns out that I had an old version (~8 :face_with_rolling_eyes: ) that was missing doseq. It performs as expected. But thanks for all the help!

👍 1
p-himik10:12:53

> there must not be any parallel execution In JS, there is no parallel execution, only concurrent one via async. Not sure how the distinction is important in your case since a linear workflow "get a link, get the data, go back" doesn't seem to interfere with any other similar workflow regardless of how it's executed.

Fredrik Andersson10:12:49

puppeteer is very async in it's API

Fredrik Andersson10:12:30

and the state is in the browser

p-himik10:12:54

And different tests, if run concurrently, will interfere with each other?

Fredrik Andersson10:12:20

yes, if they don't spawn their own browser

Fredrik Andersson10:12:46

I'm not doing tests. I'm trying to scrape my laundary booking system 😄

Fredrik Andersson10:12:49

in theory I could spawn a new browser and login in each and scrape all available laundaries in parallel

p-himik10:12:33

Yeah. Or maybe the tool allows you to use multiple tabs or persist cookies and local store across new browser spawns.

Fredrik Andersson10:12:49

yes, it's possible

Fredrik Andersson10:12:30

the laundary system is written in older http://asp.net which had the convention to use session variables alot

Fredrik Andersson10:12:16

so it's almost impossible to navigate with URL alone. I have to click buttons and links in order to make the server session happy

p-himik10:12:27

Ohhh, archaeology!

Fredrik Andersson10:12:20

yes, I usually pulls up the http://ASP.NET example for anyone who say that big companies must know what they're doing