Fork me on GitHub
#nbb
<
2022-10-28
>
alex03:10:51

Currently we are running clojurescript e2e tests using a combination of nbb + playwright. Additionally, we have an implementation for parallelizing tests that splits test vars into N groups, then invokes N`babashka.process/process` for running nbb and t/test-vars against each subset of test vars. Unfortunately, running even just 2 processes significantly slows down my M1 mac I looked around a bit to see if there were performance issues with running multiple Playwright browser instances and didn't find much. In fact, it seems like the Playwright test runner uses parallelization by default, and folks have generally had good experiences. Out of curiosity, I ran the Playwright test runner via npx playwright test against 3 sample JS test files (by default, test files are run in parallel), and everything felt snappy. I replicated those sample test files into clojurescript and ran those tests with our parallelized babashka.process/process implementation -- unfortunately it again felt quite slow compared to the Playwright test runner. Is this expected when using something like process to enable parallelization vs. an optimized Playwright test runner? Given the observations above, I was curious to try compiling our .cljs test code files into JS modules with #cherry, so they could be run with the Playwright test runner. Is that currently supported? I tried to run the following examples in https://github.com/nextjournal/clerk/blob/cherry-ui-tests-poc/ui_tests/cherry_playwright.cljs (they don't existon the main branch) but ran into the below issue.

$ npx cherry compile cherry_macros.cljs # works fine
$ npx cherry compile cherry_playwright.cljs
[cherry] Compiling CLJS file: cherry_playwright.cljs
node:internal/errors:464
    ErrorCaptureStackTrace(err);
    ^

TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string or an instance of Buffer or URL. Received null
    at Object.openSync (node:fs:577:10)
    at Module.readFileSync (node:fs:453:35)
    at file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cli.js:976:342
    at file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cli.js:126:365
    at qS (file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cli.js:126:378)
    at Function.uS.j (file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cli.js:496:321)
    at Function.uS.o (file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cli.js:496:448)
    at Function.$APP.Ch.B (file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:793:398)
    at vS (file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cli.js:131:71)
    at Function.iU.j (file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cli.js:497:205) {
  code: 'ERR_INVALID_ARG_TYPE'
}

borkdude09:10:32

My expectation is that when running two different playwright commands via process it is slower than when playwright does running in parallel itself since it can share resources.

borkdude09:10:50

Let me take a look at the cherry code

borkdude09:10:43

yeah, so how macros work has changed, I'll rectify this

borkdude11:10:47

Now fixed the branch. You can run:

npm install
npx cherry-cljs run cherry_playwright.cljs
to run the tests now

alex16:10:38

> My expectation is that when running two different playwright commands via process it is slower than when playwright does running in parallel itself since it can share resources. Thanks, that makes sense. Do you see a viable path for being able to use the Playwright test runner but still maintaining REPL capabilities? I guess we would move away from deftest and cljs.test/is assertions and towards Playwright's test and expect assertions >

npm install
> npx cherry-cljs run cherry_playwright.cljs
Appreciate the quick attention & fix! It seems like requiring other Clojure namespaces is not yet supported -- is that correct? For example, if I add [clojure.string :as str] to :require, I can no longer compile & run the file
(ns cherry-playwright
  {:clj-kondo/config '{:skip-comments false}}
  (:require ["child_process" :as cp]
            [clojure.string :as str]
            ["playwright$default" :refer [chromium]])
  (:require-macros [cherry-macros :refer [assert!]]))

$ npx cherry run src/sample-project/cherry_playwright.cljs
[cherry] Running src/sample-project/cherry_playwright.cljs
file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/compiler.js:188
qy=function(a){a=$APP.z(a);var b=$APP.A(a);a=$APP.C(a);var c=$APP.ah(a);a=$APP.Qe.g(c,$APP.py);var d=$APP.Qe.g(c,$APP.qs);c=b.split("$",2);b=$APP.D.l(c,0,null);c=$APP.D.l(c,1,null);c=$APP.z($APP.q(c)?c.split("."):null);var e=$APP.A(c);$APP.C(c);c=$APP.v.h($APP.q($APP.q(d)?$APP.Sd.g("default",e):d)?Ox(nr.l?nr.l("import %s from '%s'",d,b):nr.call(null,"import %s from '%s'",d,b)):null);d=$APP.v.h($APP.q($APP.q(d)?$APP.Ya(e):d)?Ox(nr.l?nr.l("import * as %s from '%s'",d,b):nr.call(null,"import * as %s from '%s'",
                                                                                                                              ^

TypeError: b.split is not a function
    at qy (file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/compiler.js:188:127)
    at file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:884:362
    at file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:884:375
    at Cg (file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:538:417)
    at $APP.Bg.$APP.g.W (file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:771:21)
    at Object.$APP.z (file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:508:94)
    at $APP.lg.$APP.g.za (file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:761:454)
    at Object.$APP.C (file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:509:199)
    at Function.$APP.Px.g (file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/compiler.js:395:109)
    at file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/compiler.js:564:50

alex17:10:04

Playing around with the compiled .mjs files and running the Playwright test runner against them

(ns cherry-playwright.spec
  {:clj-kondo/config '{:skip-comments false}}
  (:require ["@playwright/test" :as pw :refer [expect]]
            ["child_process" :as cp]
            ["playwright$default" :refer [chromium]]
            #_[clojure.string :as string])
  (:require-macros [cherry-macros :refer [assert!]]))

(pw/test "this test should work ok" (fn ^:async [{:keys [page]}] <-- error
                                      (prn page)))

;; running in terminal
$ npx cherry compile tests/cherry_playwright.spec.cljs && npx playwright test

;; getting this error

Error: First argument must use the object destructuring pattern: p__12
  defined at tests/cherry_playwright.spec.mjs:7:9
https://github.com/microsoft/playwright/blob/ee83694372e2cd8a03131c042189a5359504f1b6/packages/playwright-test/src/fixtures.ts#L421-L422 and expects a destructured form. I'm not sure how to modify the .cljs so that it generates something compatible
test('this test should work ok', async ({ page }) => {

borkdude20:10:44

A bit busy this weekend with #C047C0BE3HC - I will install a reminder so I can read this and reply next week

alex20:10:28

Sounds great. Enjoy #C047C0BE3HC!

borkdude20:10:09

You can generate this:

test('this test should work ok', async ({ page }) => {
by writing this:
(test "this test should work ok" ^:async (fn [^:js {:keys [page]} ...))
I think

alex18:10:27

That doesn't seem to generate a destructured form in the arguments

(pw/test "this test should work ok"  ^:async (fn [^:js {:keys [page]}]
                                               (println page)))
pw.test.call(null, "this test should work ok", async function (p__12) {
let map__1314 = p__12;
let map__1315 = __destructure_map.call(null, map__1314);
let page16 = map__1315["page"];
return println.call(null, page16);
});

borkdude18:10:23

hmm, right. as a workaround you can just write (.-page argument) . can you file a bug?

alex18:10:19

Yep, happy to file a bug sorry for being a bit dense, but I don't think I understand the (.-page argument) suggestion. Where would that go? Would it generate the ({ page }) destructured form that Playwright seems to require for its test function?

borkdude18:10:25

Like this:

^:async (fn [x] (let [page (.-page x)] ...))

alex19:10:32

So the issue is not about accessing the page field from the argument but that Playwright actually checks the syntax of the argument provided to the test function and fails it if it's not a destructured form (starting with { ending with } ) https://github.com/microsoft/playwright/blob/ee83694372e2cd8a03131c042189a5359504f1b6/packages/playwright-test/src/fixtures.ts#L421-L422

alex19:10:59

Curious about the destructing form being a requirement; this is the only explanation I've found thus far: https://github.com/microsoft/playwright/issues/8798#issuecomment-973948093

borkdude19:10:45

This is a pretty weird requirement

alex19:10:16

Yeah. I don't think I would expect any compile-to-JS languages to product this exact syntax

borkdude19:10:35

You can hack around it like this:

$ ./node_cli.js --show -e "(js* \"async function foo ({page}) {return ~{}}\" (+ 1 2 3)) (prn (foo #js {:page 1}))"

alex19:10:27

Tried to bb build on the repo and got the following error:

Error building classpath. Manifest type not detected when finding deps for io.github.squint-cljs/compiler-common in coordinate #:local{:root "/Users/alex/code/cherry/compiler-common"}
----- Error --------------------------------------------------------------------
Type:     clojure.lang.ExceptionInfo
Message:
Data:     {:proc #object[java.lang.ProcessImpl 0x22eea198 "Process[pid=31429, exitValue=1]"], :exit 1, :in #object[java.lang.ProcessBuilder$NullOutputStream 0x5eb92fe4 "java.lang.ProcessBuilder$NullOutputStream@5eb92fe4"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x61336731 "java.lang.ProcessBuilder$NullInputStream@61336731"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x61336731 "java.lang.ProcessBuilder$NullInputStream@61336731"], :prev nil, :cmd ["npx" "shadow-cljs" "--config-merge" ".work/config-merge.edn" "release" "cherry"], :type :babashka.process/error}
Location: /Users/alex/code/cherry/bb/tasks.clj:42:3

----- Context ------------------------------------------------------------------
38:   (fs/create-dirs ".work")
39:   (fs/delete-tree "lib")
40:   (fs/delete-tree ".shadow-cljs")
41:   (spit ".work/config-merge.edn" (shadow-extra-config))
42:   (shell "npx shadow-cljs --config-merge .work/config-merge.edn release cherry"))
      ^---
43:
44: (defn publish []
45:   (build-cherry-npm-package)
46:   (run! fs/delete (fs/glob "lib" "*.map"))
47:   (shell "npm publish"))

----- Stack trace --------------------------------------------------------------
babashka.process/check                    - <built-in>
babashka.process/shell                    - <built-in>
tasks/build-cherry-npm-package            - /Users/alex/code/cherry/bb/tasks.clj:42:3
tasks/build-cherry-npm-package            - /Users/alex/code/cherry/bb/tasks.clj:37:1
user-44205ee8-7a2b-44b7-b660-38e68cd0ad5f - <expr>:26:1

alex19:10:16

Ah that folder is empty for me. Guessing I need to put something there

borkdude19:10:17

@UGGU8TSMC We should document this, but you should clone the repo with --recursive because it has a submodule

alex19:10:35

Thanks for the tip. I was able to resolve via git submodule update --init

👍 1
borkdude19:10:15

Feel free to PR this to the README

alex20:10:57

thank you! :)

alex21:10:44

Thanks for the code snippet earlier. Took a little while for me to wrap my head around what might be available in scope i.e. how do I get page into my scope... but this seems to work

(pw/test "this test should work ok"  (js* "async function foo ({page}) {return ~{}}"
                                          (js/await (.goto page ""))))
For example page is not in scope on the Clojurescript side (hence clj-kondo complains), but this doesn't prevent Cherry compilation. thanks to the js* compilation, page is available in JS land

alex21:10:17

pw.test.call(null, "this test should work ok", async function foo ({page}) {return (await page.goto(""))});

borkdude21:10:09

it's a hack nonetheless, but it works. what you could also do is define this as a wrapper function in a JS file, like this:

function wrapper(f) {

  return ({page}} => f(page) 

} 

borkdude21:10:32

and then use wrapper in your test by providing a function to it which expects the page

borkdude21:10:40

so then playwright must be happy

alex21:10:11

Hmm how would I call that wrapper function? Does it still need the js*? Naively tried

(pw/test "this test should work ok" (wrapper ^:async (fn [page] (js/await (.goto page ""))))
         #_(js* "async function foo ({page}) {return ~{}}"
                (js/await (.goto page ""))))
and
(pw/test "this test should work ok" (js* wrapper ^:async (fn [page] (js/await (.goto page ""))))
         #_(js* "async function foo ({page}) {return ~{}}"
                (js/await (.goto page ""))))
The first doesn't produce the correct output; the second errors

alex21:10:03

;; ./wrapper.js

export function wrapper(f) {

  return ({page}) => f(page)
}

;; ./test.spec.cljs
(ns cherry-playwright.spec
  {:clj-kondo/config '{:skip-comments false}}
  (:require ["@playwright/test" :as pw :refer [expect]]
            ["child_process" :as cp]
            ["playwright$default" :refer [chromium]]
            ["./wrapper.js" :as w :refer [wrapper]]
            #_[clojure.string :as string])
  (:require-macros [cherry-macros :refer [assert!]]))

(pw/test "this test should work ok" (w/wrapper ^:async (fn [page] (js/await (.goto page ""))))
         #_(js* "async function foo ({page}) {return ~{}}"
                (js/await (.goto page ""))))

borkdude21:10:10

I would (naively) expect it to work - why does it not?

borkdude21:10:25

maybe you need to write the wrapper as:

export async function wrapper(f) {

  return async ({page}) => await f(page)
}

alex21:10:15

Seems like 2 issues with my most recent code snippet: 1. generates the function without the destructured ({page})

import * as w from './wrapper.js';
import * as pw from '@playwright/test';
import { expect } from '@playwright/test';
import * as cp from 'child_process';
import { chromium } from 'playwright';
pw.test.call(null, "this test should work ok", w.wrapper.call(null, async function (page) {
return (await page.goto(""));
}));
2. the wrapper import doesn't seem to be defined

borkdude21:10:19

The call to w.wrapper returns a function that has the destructuring - isn't that what counts?

borkdude21:10:55

Can you try:

["./wrapper.js" :as w]

borkdude21:10:07

without :refer and then (w/wrapper ...)?

borkdude21:10:19

this is probably a bug in cherry, feel free to post an issue about that

alex21:10:58

> The call to w.wrapper returns a function that has the destructuring - isn't that what counts? Ah yes, if that's the case, that should (hopefully) work

alex21:10:00

The extra . didn't work. No file called ./.wrapper.js

borkdude21:10:24

that's not what I meant

alex21:10:41

Oh whoops. I misread your suggestion

borkdude21:10:58

What I meant was: get rid of the refer and just go via the alias

borkdude21:10:07

this is probably a bug in cherry

alex21:10:30

Yep. Same result w/o the refer. I logged w

[Module: null prototype] { default: { wrapper: [Function: wrapper] } }
so this works
["./wrapper.js$default" :as w]

alex21:10:45

Not sure if intended or not 🙂

borkdude21:10:10

huh... wrapper isn't the default export right?

borkdude21:10:31

or wait, this might be .cjs vs .mjs

borkdude21:10:42

if you rename that file to wrapper.mjs it might work as intended

borkdude22:10:06

or add "type": "module" in your package.json - same effect

alex22:10:08

Ah cool, changed extension to .mjs and that works

borkdude22:10:36

OK, checking out now. Will read your messages tomorrow

1
alex04:11:47

Thanks for your help. I'm probably blocked on converting actual tests over to cherry until there is support for importing clojurescript namespaces e.g. Clojure core ones like clojure.string , regular libraries like promesa.core , and our own util namespaces e.g. [test.utils :as t-utils] . Is that not currently supported? I get errors like

(:require [clojure.core :as c])

file:///Users/alex/code/cherry-test/ui_tests/node_modules/cherry-cljs/lib/compiler.js:188
qy=function(a){a=$APP.z(a);var b=$APP.A(a);a=$APP.C(a);var c=$APP.ah(a);a=$APP.Qe.g(c,$APP.py);var d=$APP.Qe.g(c,$APP.qs);c=b.split("$",2);b=$APP.D.l(c,0,null);c=$APP.D.l(c,1,null);c=$APP.z($APP.q(c)?c.split("."):null);var e=$APP.A(c);$APP.C(c);c=$APP.v.h($APP.q($APP.q(d)?$APP.Sd.g("default",e):d)?Ox(nr.l?nr.l("import %s from '%s'",d,b):nr.call(null,"import %s from '%s'",d,b)):null);d=$APP.v.h($APP.q($APP.q(d)?$APP.Ya(e):d)?Ox(nr.l?nr.l("import * as %s from '%s'",d,b):nr.call(null,"import * as %s from '%s'",
                                                                                                                              ^

TypeError: b.split is not a function
    at qy (file:///Users/alex/code/cherry-test/ui_tests/node_modules/cherry-cljs/lib/compiler.js:188:127)

borkdude15:11:18

clojure.string can be arranged promesa you likely won't need with async/await

👍 1
borkdude15:11:22

Adding your own utils [test.utils :as t-utils] : not sure if this works already but in worst case you can do: ["./test/utils.mjs" :as t-utils]. Please create issues for clojure.string and the latter one.

alex15:11:26

> Adding your own utils [test.utils :as t-utils] : not sure if this works already but in worst case you can do: ["./test/utils.mjs" :as t-utils]. Thanks. Just to confirm, that would require me to first run npx cherry compile test/utils.cljs right?

alex17:11:04

For clojure.string, this exists: https://github.com/squint-cljs/cherry/issues/18 Do you want a separate issue? I added one for importing user namespaces

borkdude17:11:57

Ah yes. For squint we already have the transpiled one, we can do it for cherry too

alex15:11:03

I just returned to this after a few days and converted a test over to the Playwright test runner. This is really promising, thanks for all your support!

(ns cherry-playwright.spec
  {:clj-kondo/config '{:skip-comments false}}
  (:require ["./wrapper.mjs" :as w]
            ["@playwright/test" :as pw :refer [expect]]
            ["child_process" :as cp]
            ["playwright$default" :refer [chromium]])
  (:require-macros [cherry-macros :refer [assert!]]))

(def index
  "")

(pw/test "homepage should load"
         (w/wrapper ^:async
          (fn [page]
            (js/await (.goto page index))
            (js/await (.waitFor (.locator page "[data-testid='this should fail']")))
            (js/await (.waitFor (.locator page "h1:has-text(\"Crack the culture code\")"))))))

alex15:11:51

> Reducing friction between ClojureScript and JS tooling.