More details or repro? Afk now but can check later when you have one

will try to make a repro

(defn -main
  [_]
  (p/let [;; Run non-DB tests. In order for simpler reporting
          p1 (p/promise 1)]
    (when p1
      (-> (p/let [;; Run DB required tests. In order for simpler reporting
                  tests1 (postgres-test/run-tests)]
            (js/console.log "TESTS conclude" tests1))
          (p/finally (fn [_ _]
                       (js/console.log "FINALLY Called Now")
                       (p/promise 1)))))))

replaced postgres tests with this

(ns postgres-test
  (:require
    [cljs-bean.core :as cb]
    [cljs.test :as t :refer [async deftest is testing]]
    [postgres]
    [promesa.core :as p]))


(deftest p1
  (testing "p1"
    (async done
      (-> (p/let [p1 (p/promise 1)]
            (js/console.log "p1")
            (is (= 1 p1)))
          (p/finally done)))))


(deftest p2
  (testing "p2"
    (async done
      (-> (p/let [p1 (p/promise 1)]
            (js/console.log "p2")
            (is (= 1 p1)))
          (p/finally done)))))


(defn run-tests
  []
  (t/run-tests 'postgres-test))

I checked this with other tests and any time you have more than one deftest using a promise the finally executes after the first promise is resolved

npx nbb --classpath src:test -m runner Testing postgres-test p1 TESTS conclude true p2 FINALLY Called Now Ran 2 tests containing 2 assertions. 0 failures, 0 errors.

p2 is called after tests conclude and then the finally and the p2 are in a race

so big question for me ... am I holding it wrong?

I'll try locally

When I run this:

(ns postgres
  (:require
   [cljs.test :as t :refer [async deftest is testing]]
   [promesa.core :as p]))


(deftest p1
  (testing "p1"
    (async done
           (-> (p/let [p1 (p/promise 1)]
                 (js/console.log "p1")
                 (is (= 1 p1)))
               (p/finally done)))))


(deftest p2
  (testing "p2"
    (async done
           (-> (p/let [p1 (p/promise 1)]
                 (js/console.log "p2")
                 (is (= 1 p1)))
               (p/finally done)))))


(t/run-tests 'postgres)

$ nbb /tmp/postgres.cljs

Testing postgres
p1
p2

Ran 2 tests containing 2 assertions.
0 failures, 0 errors.

If you want to provide a more complete repro, please make a github repo which I can clone locally so we're speaking about exactly the same thing

sorry, I didn't notice you posted multiple files

it would be easier for me to be able to clone it though, than to replicate the same setup by copy/pasting things from slack

I'll make a small repo

https://github.com/raymcdermott/nbb-promise-tests

npx nbb -m runner

Testing promise-test
p1
TESTS conclude true
p2
FINALLY Called Now

Ran 2 tests containing 2 assertions.
0 failures, 0 errors.

That's what I'm seeing locally. What is unexpected?

no harm here but bad things happen when p2 is not resolved reliably before the let bindings are completed

should see p1, p2 ... then TESTS concude

unless I am missing something about the p/let contract

ah ok, now looking into it then

thanks... I would love for it to be one of these

run-tests does not return a promise, so awaiting that doesn't really wait on anything

https://cljs.github.io/api/cljs.test/#run-tests

(defn run-tests
  []
  (p/let [_ (t/run-tests 'promise-test)]
    true))

> >

To detect test
> completion add a :end-run-tests method case to the cljs.test/report
> multimethod.

does the job

This returns true immediately, before the tests run (or somewhere in the middle)

ha! ok, thanks ... I'll do that too

It's indeed a but mostly because CLJS doesn't really embrace promises here

You could solve this by resolving a promise inside that multimethod though

It would be pretty nice to have a promise-based test runner instead of the cljs.test one

yes, that's what I'll do

thanks for your help - totally excellent

funny

(defn run-tests
  []
  (p/let [_ (t/run-tests 'promise-test)]
    true))

reliably produces the correct results

adding a promise to that multimethod has no effect

(defmethod t/report [::t/default :end-run-tests]
  [m]
  (p/let [ok? (t/successful? m)]
    (if ok?
      (println "Tests passed!")
      (println "FAIL"))))

no change in the chaos

Do you see the println?

Yes

Does it work for you?

What you're seeing is purely co-incidental, just add a _ (p/delay 1000) in one of those tests and you'll see that it doesn't work with p/let + true

This works:

(def end-resolve (atom nil))
(def end-promise (js/Promise. (fn [resolve _reject]
                                (reset! end-resolve resolve))))

(defmethod t/report [:cljs.test/default :end-run-tests]
  [m]
  (let [ok? (t/successful? m)]
    (if ok?
      (println "Tests passed!")
      (println "FAIL"))
    (@end-resolve)))

(defn run-tests
  []
  (t/run-tests 'promise-test)
  end-promise)

So, in the multimethod you resolve the promise and in run-tests you return the promise that will be fulfilled

Blimey, I'm gonna need a minute 🤯 Thanks for digging into it. Like you say, dealing with this stuff is annoying compared to what a promise based library could offer.

It is a mess when there are two test nses with N deftests

both the clause after the let bindings and the finally on the runner never executes

Repro or it didn't happen. :)

actually I think the finally shit is me holding it wrong - I should use per ns fixtures instead

In this case we should be seeing some console output

(ns runner
  (:require
    [promesa.core :as p]
    [promise-second-test]
    [promise-test]))

(defn -main
  [_]
  (-> (p/let [_ (promise-test/run-tests)
              _ (promise-second-test/run-tests)]
        (js/console.log "TESTS conclude"))
      (p/finally (fn [_ _]
                 (js/console.log "Finally TESTS conclude")))))

but there is none

it's more of a curiosity at this point cos the tests and fixtures all run and behave how I want

I have pushed the updates

I'm seeing:

$ nbb -m runner

Testing promise-test
stand up promise-test
tear down promise-test
Tests passed!

Ran 3 tests containing 3 assertions.
0 failures, 0 errors.

Testing promise-second-test
stand up promise-second-test
tear down promise-second-test
Tests passed!

Ran 2 tests containing 2 assertions.
0 failures, 0 errors.

oh yes, I see, the console logs from the runner

when I remove #_#__ (promise-second-test/run-tests) I do see the output... hmm

so what you're doing here is that a defmethod in the first ns overrides the defined method of the other ns

Yes, like I say it gets messy when you do more things. Usually you should be able to scope these things to the specific ns

or maybe more to the point that's what I would like to be able to do

the whole point of this is to run two test suites with promises and for them to conclude in an orderly manner

otherwise the async is leaking

it's not so urgent in the sense that we have a workable test suite now but I feel like the silent console is trying to tell us something

This is probably because you're overriding one defmethod + dispatch with another one so one of the promises never gets fulfilled

If you want to have one promise per ns, you should resolve a namespace specific promise in the defmethod

If you like to run tests from different namespaces in order, you can just do:

(run-tests 'namespace-one 'namespace-two)

I should just run all tests. I think this runner is overly complex

https://gist.github.com/borkdude/cd9eb17b7d32a8d972441a848541debd

that test runner plus the fixtures .... everything just seems to work without the need for the promise resolving hack

I know I could do it but if you have a version that runs all the test nses .... would be nice

not to worry, I've adjusted it

ha, the adjustments are falling over after I have three. 😿

forget that ... just using run-tests after the various adjustments is working

You probably know already but all of the promise hacks are not needed in that case

core.async isn't supported in nbb. the recommended approach is to use promises directly in combination with promesa.core

Righto, thank you.

Thanks! Exactly what I was trying to remember. Was looking in the examples folder, should have also checked docs.

no problem :)