This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-10-28
Channels
- # announcements (5)
- # babashka (7)
- # beginners (101)
- # biff (9)
- # calva (46)
- # cider (6)
- # clj-yaml (2)
- # cljsrn (13)
- # clojure (11)
- # clojure-europe (43)
- # clojure-nl (13)
- # clojure-norway (22)
- # clojurescript (20)
- # conjure (1)
- # cursive (7)
- # data-science (2)
- # datomic (26)
- # emacs (38)
- # graphql (27)
- # gratitude (5)
- # hoplon (8)
- # hugsql (22)
- # humbleui (2)
- # hyperfiddle (6)
- # introduce-yourself (8)
- # joyride (3)
- # lsp (79)
- # malli (6)
- # nbb (67)
- # portal (16)
- # rdf (27)
- # reagent (42)
- # releases (2)
- # remote-jobs (1)
- # shadow-cljs (36)
- # test-check (17)
- # tools-deps (1)
- # xtdb (15)
Currently we are running clojurescript e2e tests using a combination of nbb + playwright. Additionally, we have an implementation for parallelizing tests that splits test vars into N groups, then invokes N`babashka.process/process` for running nbb
and t/test-vars
against each subset of test vars. Unfortunately, running even just 2 processes significantly slows down my M1 mac
I looked around a bit to see if there were performance issues with running multiple Playwright browser instances and didn't find much. In fact, it seems like the Playwright test runner uses parallelization by default, and folks have generally had good experiences. Out of curiosity, I ran the Playwright test runner via npx playwright test
against 3 sample JS test files (by default, test files are run in parallel), and everything felt snappy. I replicated those sample test files into clojurescript and ran those tests with our parallelized babashka.process/process
implementation -- unfortunately it again felt quite slow compared to the Playwright test runner. Is this expected when using something like process
to enable parallelization vs. an optimized Playwright test runner?
Given the observations above, I was curious to try compiling our .cljs
test code files into JS modules with #cherry, so they could be run with the Playwright test runner. Is that currently supported?
I tried to run the following examples in https://github.com/nextjournal/clerk/blob/cherry-ui-tests-poc/ui_tests/cherry_playwright.cljs (they don't existon the main
branch) but ran into the below issue.
$ npx cherry compile cherry_macros.cljs # works fine
$ npx cherry compile cherry_playwright.cljs
[cherry] Compiling CLJS file: cherry_playwright.cljs
node:internal/errors:464
ErrorCaptureStackTrace(err);
^
TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string or an instance of Buffer or URL. Received null
at Object.openSync (node:fs:577:10)
at Module.readFileSync (node:fs:453:35)
at file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cli.js:976:342
at file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cli.js:126:365
at qS (file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cli.js:126:378)
at Function.uS.j (file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cli.js:496:321)
at Function.uS.o (file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cli.js:496:448)
at Function.$APP.Ch.B (file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:793:398)
at vS (file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cli.js:131:71)
at Function.iU.j (file:///Users/alex/code/icebreaker/ui_tests/node_modules/cherry-cljs/lib/cli.js:497:205) {
code: 'ERR_INVALID_ARG_TYPE'
}
My expectation is that when running two different playwright commands via process it is slower than when playwright does running in parallel itself since it can share resources.
Now fixed the branch. You can run:
npm install
npx cherry-cljs run cherry_playwright.cljs
to run the tests now> My expectation is that when running two different playwright commands via process it is slower than when playwright does running in parallel itself since it can share resources.
Thanks, that makes sense. Do you see a viable path for being able to use the Playwright test runner but still maintaining REPL capabilities? I guess we would move away from deftest
and cljs.test/is
assertions and towards Playwright's test
and expect
assertions
>
npm install
> npx cherry-cljs run cherry_playwright.cljs
Appreciate the quick attention & fix! It seems like requiring other Clojure namespaces is not yet supported -- is that correct? For example, if I add [clojure.string :as str]
to :require
, I can no longer compile & run the file
(ns cherry-playwright
{:clj-kondo/config '{:skip-comments false}}
(:require ["child_process" :as cp]
[clojure.string :as str]
["playwright$default" :refer [chromium]])
(:require-macros [cherry-macros :refer [assert!]]))
$ npx cherry run src/sample-project/cherry_playwright.cljs
[cherry] Running src/sample-project/cherry_playwright.cljs
file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/compiler.js:188
qy=function(a){a=$APP.z(a);var b=$APP.A(a);a=$APP.C(a);var c=$APP.ah(a);a=$APP.Qe.g(c,$APP.py);var d=$APP.Qe.g(c,$APP.qs);c=b.split("$",2);b=$APP.D.l(c,0,null);c=$APP.D.l(c,1,null);c=$APP.z($APP.q(c)?c.split("."):null);var e=$APP.A(c);$APP.C(c);c=$APP.v.h($APP.q($APP.q(d)?$APP.Sd.g("default",e):d)?Ox(nr.l?nr.l("import %s from '%s'",d,b):nr.call(null,"import %s from '%s'",d,b)):null);d=$APP.v.h($APP.q($APP.q(d)?$APP.Ya(e):d)?Ox(nr.l?nr.l("import * as %s from '%s'",d,b):nr.call(null,"import * as %s from '%s'",
^
TypeError: b.split is not a function
at qy (file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/compiler.js:188:127)
at file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:884:362
at file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:884:375
at Cg (file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:538:417)
at $APP.Bg.$APP.g.W (file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:771:21)
at Object.$APP.z (file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:508:94)
at $APP.lg.$APP.g.za (file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:761:454)
at Object.$APP.C (file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/cljs_core.js:509:199)
at Function.$APP.Px.g (file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/compiler.js:395:109)
at file:///Users/alex/code/sample-project/ui_tests/node_modules/cherry-cljs/lib/compiler.js:564:50
Playing around with the compiled .mjs
files and running the Playwright test runner against them
(ns cherry-playwright.spec
{:clj-kondo/config '{:skip-comments false}}
(:require ["@playwright/test" :as pw :refer [expect]]
["child_process" :as cp]
["playwright$default" :refer [chromium]]
#_[clojure.string :as string])
(:require-macros [cherry-macros :refer [assert!]]))
(pw/test "this test should work ok" (fn ^:async [{:keys [page]}] <-- error
(prn page)))
;; running in terminal
$ npx cherry compile tests/cherry_playwright.spec.cljs && npx playwright test
;; getting this error
Error: First argument must use the object destructuring pattern: p__12
defined at tests/cherry_playwright.spec.mjs:7:9
https://github.com/microsoft/playwright/blob/ee83694372e2cd8a03131c042189a5359504f1b6/packages/playwright-test/src/fixtures.ts#L421-L422 and expects a destructured form. I'm not sure how to modify the .cljs
so that it generates something compatible
test('this test should work ok', async ({ page }) => {
A bit busy this weekend with #C047C0BE3HC - I will install a reminder so I can read this and reply next week
You can generate this:
test('this test should work ok', async ({ page }) => {
by writing this:
(test "this test should work ok" ^:async (fn [^:js {:keys [page]} ...))
I thinkThat doesn't seem to generate a destructured form in the arguments
(pw/test "this test should work ok" ^:async (fn [^:js {:keys [page]}]
(println page)))
pw.test.call(null, "this test should work ok", async function (p__12) {
let map__1314 = p__12;
let map__1315 = __destructure_map.call(null, map__1314);
let page16 = map__1315["page"];
return println.call(null, page16);
});
hmm, right. as a workaround you can just write (.-page argument)
. can you file a bug?
Yep, happy to file a bug
sorry for being a bit dense, but I don't think I understand the (.-page argument)
suggestion. Where would that go? Would it generate the ({ page })
destructured form that Playwright seems to require for its test
function?
So the issue is not about accessing the page
field from the argument but that Playwright actually checks the syntax of the argument provided to the test function and fails it if it's not a destructured form (starting with {
ending with }
)
https://github.com/microsoft/playwright/blob/ee83694372e2cd8a03131c042189a5359504f1b6/packages/playwright-test/src/fixtures.ts#L421-L422
Curious about the destructing form being a requirement; this is the only explanation I've found thus far: https://github.com/microsoft/playwright/issues/8798#issuecomment-973948093
Yeah. I don't think I would expect any compile-to-JS languages to product this exact syntax
You can hack around it like this:
$ ./node_cli.js --show -e "(js* \"async function foo ({page}) {return ~{}}\" (+ 1 2 3)) (prn (foo #js {:page 1}))"
Tried to bb build
on the repo and got the following error:
Error building classpath. Manifest type not detected when finding deps for io.github.squint-cljs/compiler-common in coordinate #:local{:root "/Users/alex/code/cherry/compiler-common"}
----- Error --------------------------------------------------------------------
Type: clojure.lang.ExceptionInfo
Message:
Data: {:proc #object[java.lang.ProcessImpl 0x22eea198 "Process[pid=31429, exitValue=1]"], :exit 1, :in #object[java.lang.ProcessBuilder$NullOutputStream 0x5eb92fe4 "java.lang.ProcessBuilder$NullOutputStream@5eb92fe4"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x61336731 "java.lang.ProcessBuilder$NullInputStream@61336731"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x61336731 "java.lang.ProcessBuilder$NullInputStream@61336731"], :prev nil, :cmd ["npx" "shadow-cljs" "--config-merge" ".work/config-merge.edn" "release" "cherry"], :type :babashka.process/error}
Location: /Users/alex/code/cherry/bb/tasks.clj:42:3
----- Context ------------------------------------------------------------------
38: (fs/create-dirs ".work")
39: (fs/delete-tree "lib")
40: (fs/delete-tree ".shadow-cljs")
41: (spit ".work/config-merge.edn" (shadow-extra-config))
42: (shell "npx shadow-cljs --config-merge .work/config-merge.edn release cherry"))
^---
43:
44: (defn publish []
45: (build-cherry-npm-package)
46: (run! fs/delete (fs/glob "lib" "*.map"))
47: (shell "npm publish"))
----- Stack trace --------------------------------------------------------------
babashka.process/check - <built-in>
babashka.process/shell - <built-in>
tasks/build-cherry-npm-package - /Users/alex/code/cherry/bb/tasks.clj:42:3
tasks/build-cherry-npm-package - /Users/alex/code/cherry/bb/tasks.clj:37:1
user-44205ee8-7a2b-44b7-b660-38e68cd0ad5f - <expr>:26:1
@UGGU8TSMC We should document this, but you should clone the repo with --recursive
because it has a submodule
Thanks for the code snippet earlier. Took a little while for me to wrap my head around what might be available in scope i.e. how do I get page
into my scope... but this seems to work
(pw/test "this test should work ok" (js* "async function foo ({page}) {return ~{}}"
(js/await (.goto page " "))))
For example page
is not in scope on the Clojurescript side (hence clj-kondo
complains), but this doesn't prevent Cherry compilation. thanks to the js*
compilation, page
is available in JS landpw.test.call(null, "this test should work ok", async function foo ({page}) {return (await page.goto(" "))});
it's a hack nonetheless, but it works. what you could also do is define this as a wrapper function in a JS file, like this:
function wrapper(f) {
return ({page}} => f(page)
}
and then use wrapper
in your test by providing a function to it which expects the page
Hmm how would I call that wrapper
function? Does it still need the js*
?
Naively tried
(pw/test "this test should work ok" (wrapper ^:async (fn [page] (js/await (.goto page " "))))
#_(js* "async function foo ({page}) {return ~{}}"
(js/await (.goto page " "))))
and
(pw/test "this test should work ok" (js* wrapper ^:async (fn [page] (js/await (.goto page " "))))
#_(js* "async function foo ({page}) {return ~{}}"
(js/await (.goto page " "))))
The first doesn't produce the correct output; the second errors;; ./wrapper.js
export function wrapper(f) {
return ({page}) => f(page)
}
;; ./test.spec.cljs
(ns cherry-playwright.spec
{:clj-kondo/config '{:skip-comments false}}
(:require ["@playwright/test" :as pw :refer [expect]]
["child_process" :as cp]
["playwright$default" :refer [chromium]]
["./wrapper.js" :as w :refer [wrapper]]
#_[clojure.string :as string])
(:require-macros [cherry-macros :refer [assert!]]))
(pw/test "this test should work ok" (w/wrapper ^:async (fn [page] (js/await (.goto page " "))))
#_(js* "async function foo ({page}) {return ~{}}"
(js/await (.goto page " "))))
maybe you need to write the wrapper as:
export async function wrapper(f) {
return async ({page}) => await f(page)
}
Seems like 2 issues with my most recent code snippet:
1. generates the function without the destructured ({page})
import * as w from './wrapper.js';
import * as pw from '@playwright/test';
import { expect } from '@playwright/test';
import * as cp from 'child_process';
import { chromium } from 'playwright';
pw.test.call(null, "this test should work ok", w.wrapper.call(null, async function (page) {
return (await page.goto(" "));
}));
2. the wrapper
import doesn't seem to be definedThe call to w.wrapper
returns a function that has the destructuring - isn't that what counts?
> The call to w.wrapper
returns a function that has the destructuring - isn't that what counts?
Ah yes, if that's the case, that should (hopefully) work
Yep. Same result w/o the refer
. I logged w
[Module: null prototype] { default: { wrapper: [Function: wrapper] } }
so this works
["./wrapper.js$default" :as w]
Thanks for your help. I'm probably blocked on converting actual tests over to cherry
until there is support for importing clojurescript namespaces e.g. Clojure core ones like clojure.string
, regular libraries like promesa.core
, and our own util namespaces e.g. [test.utils :as t-utils]
. Is that not currently supported?
I get errors like
(:require [clojure.core :as c])
file:///Users/alex/code/cherry-test/ui_tests/node_modules/cherry-cljs/lib/compiler.js:188
qy=function(a){a=$APP.z(a);var b=$APP.A(a);a=$APP.C(a);var c=$APP.ah(a);a=$APP.Qe.g(c,$APP.py);var d=$APP.Qe.g(c,$APP.qs);c=b.split("$",2);b=$APP.D.l(c,0,null);c=$APP.D.l(c,1,null);c=$APP.z($APP.q(c)?c.split("."):null);var e=$APP.A(c);$APP.C(c);c=$APP.v.h($APP.q($APP.q(d)?$APP.Sd.g("default",e):d)?Ox(nr.l?nr.l("import %s from '%s'",d,b):nr.call(null,"import %s from '%s'",d,b)):null);d=$APP.v.h($APP.q($APP.q(d)?$APP.Ya(e):d)?Ox(nr.l?nr.l("import * as %s from '%s'",d,b):nr.call(null,"import * as %s from '%s'",
^
TypeError: b.split is not a function
at qy (file:///Users/alex/code/cherry-test/ui_tests/node_modules/cherry-cljs/lib/compiler.js:188:127)
Adding your own utils [test.utils :as t-utils]
: not sure if this works already but in worst case you can do: ["./test/utils.mjs" :as t-utils]
. Please create issues for clojure.string and the latter one.
> Adding your own utils [test.utils :as t-utils]
: not sure if this works already but in worst case you can do: ["./test/utils.mjs" :as t-utils]
.
Thanks. Just to confirm, that would require me to first run npx cherry compile test/utils.cljs
right?
For clojure.string
, this exists: https://github.com/squint-cljs/cherry/issues/18 Do you want a separate issue? I added one for importing user namespaces
I just returned to this after a few days and converted a test over to the Playwright test runner. This is really promising, thanks for all your support!
(ns cherry-playwright.spec
{:clj-kondo/config '{:skip-comments false}}
(:require ["./wrapper.mjs" :as w]
["@playwright/test" :as pw :refer [expect]]
["child_process" :as cp]
["playwright$default" :refer [chromium]])
(:require-macros [cherry-macros :refer [assert!]]))
(def index
"")
(pw/test "homepage should load"
(w/wrapper ^:async
(fn [page]
(js/await (.goto page index))
(js/await (.waitFor (.locator page "[data-testid='this should fail']")))
(js/await (.waitFor (.locator page "h1:has-text(\"Crack the culture code\")"))))))