Fork me on GitHub
#babashka
<
2022-07-18
>
pinkfrog01:07:48

Open a bb repl, and

(require '[taoensso.timbre.appenders.core :refer [println-appender]])
It errs with
; : Unable to resolve classname: java.util.concurrent.CountDownLatch user 
So it seems like at the current phase we cannot configure the appenders for timbre

borkdude03:07:25

Try referring that var from the core namespace

pinkfrog03:07:13

Why does that error occur? Are we use some shim layer to bypass the j.u.concurrent package?

borkdude03:07:37

Only the core namespace is built into bb. You should not have to include timbre in your bb.edn which is probably what you're doing now

pinkfrog03:07:13

Yes. A dependency indirectly imports timebre

borkdude03:07:54

Oh I see. And this dep is using the appenders namespace ?

borkdude03:07:32

Issue welcome, we could add the appenders ns with the println appender too but as I said earlier, it's already available through the core namespace

robert-stuttaford10:07:33

@borkdude how pleasant is it to use etoain + babashka? i guess the slow bits would now be between the driver and the browser?

borkdude10:07:15

I haven't used it myself much yet, but @UE21H2HHD has been actively working on this

borkdude10:07:38

I guess it should be easy to switch between bb and JVM so you could try both

borkdude10:07:38

Note that you have to use the master branch of etaoin - there is no tagged release yet

borkdude10:07:53

clj-commons/etaoin

robert-stuttaford10:07:03

that's for JVM right - i can just use the pod with bb?

borkdude10:07:24

I recommend just using clj-commons/etaoin now instead of the pod

robert-stuttaford10:07:38

good thing i asked 😄

borkdude10:07:55

once clj-commons/etaoin has a tagged release, we can deprecate the pod

lread14:07:06

Hey ho! Yeah, borkdude and I have volunteered to care for and maintain Etaoin. I've been digging in deep over the past while, I've learned a ton, but would not claim that I am an expert. I have one more issue I'd like to resolve before I cut a release to clojars. As for usage from bb, the fast startup time is really nice, otherwise I haven't really noticed behaviour differences between bb/jvm-clojure (which probably means we've done a good job with bb support!).

robert-stuttaford14:07:46

huzzah! thanks Lee!

borkdude14:07:01

I wanna try it myself on clerk, if @U5H74UNSF agrees

borkdude14:07:25

But I'm definitely going to use it to write tests for scittle.

borkdude14:07:05

I've been using etaoin on previous Clojure projects as well - aside from the quirkiness of UI tests in general, it's a nice lib to work with.

robert-stuttaford14:07:48

i've tried browser-driver testing a number of times in the past, i've never had a good time

robert-stuttaford14:07:12

but never with etoain, though - the pure Clojure impl may just be the ticket!

borkdude14:07:12

It's always flaky no matter what you use

robert-stuttaford14:07:55

yeah, it's crucial to design your tests well i expect, probably not something i ever succeeded at 😅

borkdude14:07:56

flaky as in: it sometimes needs extra care to cover a particular scenario robustly

lread14:07:18

FWIW, the Etaoin test suite seem to rarely fail on chrome/firefox on linux. I did have some test stability challenges on macOS and Windows, which may be somewhat resolved... not sure yet.

lread14:07:23

But yeah, there are sometimes bugs in web drivers, or browsers, or environmental issues on CI.

lread14:07:38

youarerite

mkvlr14:07:20

I think playwright & puppeteer enable way more reliable tests thanks to the more powerful capabilities like waitForFunction, see eg https://dev.to/checkly/avoiding-hard-waits-in-playwright-and-puppeteer-272

mkvlr14:07:10

we use this for example to check that the re-frame event queue is fully processed

robert-stuttaford14:07:01

@U5H74UNSF if it's not too much of an imposition, would it be possible to spend 20 mins on a zoom sometime, so you can show me how you're using P+P?

borkdude14:07:19

@U0509NKGK I've actually ported some of nextjournal tests (http://nextjournal.com + clerk) to run with #nbb on playwright

robert-stuttaford14:07:36

mmm i would love to see that

robert-stuttaford14:07:27

a constraint we're struggling with at the mo is a lack of testing confidence. our whole system of work is great, except when we make foundational changes, and then we get stuck in the mud with lots of poorly organised manual testing. looking for leverage

borkdude14:07:47

but I did this at a time when nextjournal wasn't aware of the Java API in playwright.

borkdude14:07:09

and the async stuff in JS is still a bit annoying compared to synchronous clojure

robert-stuttaford14:07:22

ok interesting! nbb is bb + node right (i'm such a node noob)

lread14:07:35

etaoin has wait-* fns, but maybe p&p has more of them and lower level ones ?

borkdude14:07:30

I think you could accomplish "wait until the reframe queue is full by evaluating some JS in the client" can also be accomplished with etaoin, so I'm not exactly sure what benefits playwright will give you

mkvlr14:07:12

playwright and puppeteer have more direct access to the browser runtime iirc, so you can wait until a function returns true and then wait two more js frames until it's rendered by react

borkdude14:07:55

I also asked to the organizer of ClojureD who is working on a browser testing project for clojure, what his thoughts on playwright was and if he was using it: his reply was, well it isn't a standard, I'm sticking with selenium/webdriver for my stuff. I guess everyone has different trade-offs.

mkvlr14:07:57

I don't think you can do that request animation frame stuff on eatoin

mkvlr14:07:31

our testing needs with content-editable stuff on the nextjournal editor are also pretty demanding, certainly not your standard use case

borkdude14:07:35

One could argue that you're testing on a fairly low level, lower than normal end-to-end tests operate on.

robert-stuttaford14:07:09

the content-editable stuff must surely set your hair on fire

mkvlr14:07:52

@borkdude clerk or nextjournal?

borkdude14:07:26

I mean, by inspecting the re-frame queue in a test, you're not running the test in terms of "now the user does this, than that, and then I see this"

mkvlr14:07:41

that's just a tool to get the tests stable

mkvlr14:07:51

do some interaction, wait the minimum amount of time until those changes should be propagated to the dom (without any timeouts), then fail or succeed.

mkvlr14:07:45

to be fair, we still see occasional flakes on CI so it's not perfect. But it's the best testing environment in the browser we've used and we've been trying for a while. Selenium (which then got standardized into webdriver) is crap in comparison imo.

borkdude14:07:08

Makes sense

lread14:07:16

Interesting

robert-stuttaford14:07:07

thanks for the input fellas, appreciate it!

lread19:07:55

FWIW, while doing some research for Etaoin I was digging around Selenium content and stumbled on the fact that a https://github.com/w3c/webdriver-bidi is under development. Might bear some interesting fruit someday.

mkvlr19:07:50

interesting, though most (all?) of the things https://github.com/w3c/webdriver-bidi/blob/master/explainer.md#goals are possible today with puppeteer & playwright

borkdude20:07:01

yes, but ... but ... playwright doesn't work with #babashka :)

robert-stuttaford05:07:07

............ yet 😆

mkvlr05:07:35

it should actually: #babashka can you http, json & websockets, right? Should be all that’s needed: https://chromedevtools.github.io/devtools-protocol/#endpoints

borkdude05:07:50

There is this project which works with bb. Does that mean it can integrate with playwright? https://github.com/tatut/clj-chrome-devtools

mkvlr05:07:48

playwright is basically the same as puppeteer (heard it was even created by some of the same folks that moved from google to microsoft) but also working in Safari & Firefox using custom builds of those browsers with the patches above

borkdude05:07:36

I know but the Playwright APi is Java so you would have to rewrite all that stuff in bb right

mkvlr05:07:06

there’s java & node, right?

mkvlr05:07:29

and python & .net

mkvlr06:07:26

not sure how much https://github.com/tatut/clj-chrome-devtools differs from the puppeteer / playwright api, maybe it’s already sufficient

borkdude06:07:31

I guess the part you would need to figure out is how to talk to a playwright-instrumented/patched browser without using the playwright API.

mkvlr06:07:08

that's just the devtools protocol afaik

borkdude06:07:40

Sure, I mean the logistics of getting playwright to download a browser, spinning up and then talking to it. Maybe it's easy, maybe not. Worth a shot

mkvlr06:07:02

npm install playwright does that I think

mkvlr06:07:20

also care less about playwright vs puppeteer, they're almost the same. It's nice to be able to also run in safari & firefox but not a deal breaker.

borkdude06:07:07

I'll give it a try later this week

sheepy 1
borkdude07:07:51

lol, I screwed up my local Chrome install by executing:

$ node_modules/playwright/bin/reinstall_chrome_stable_mac.sh
+ rm -rf '/Applications/Google '

borkdude07:07:18

Ah: npx playwright install

borkdude07:07:31

Got something working:

((requiring-resolve 'babashka.deps/add-deps)
 '{:deps {tatut/devtools {:git/url ""
                          :git/sha "cc96255433ca406aaba8bcee17d0d0b3b16dc423"}
          org.babashka/spec.alpha {:git/url ""
                                   :git/sha "1a841c4cc1d4f6dab7505a98ed2d532dd9d56b78"}}})

(ns 
  (:require
   [babashka.process :refer [process destroy]]
   [clj-chrome-devtools.automation :as auto]
   [clj-chrome-devtools.core :as core]))

(def proc (process "/Users/borkdude/Library/Caches/ms-playwright/chromium-930007/chrome-mac/Chromium.app/Contents/MacOS/Chromium --remote-debugging-port=9222"))

(let [browser (auto/create-automation (core/connect))]
  (try
    (auto/to browser "")
    ;; NPE:
    ;; (prn (auto/text-of "body"))
    (finally
      (destroy proc))))
So I'm launching the browser installed by playwright with a specific debugging port which clj-devtools can then connect to

borkdude07:07:43

I'm getting an NPE though, not sure why

mkvlr09:07:50

lol, meanwhile “later this week

lread13:07:15

I’m frankly a little bummed having just spent a ton of time on Etaoin (borkdude and I volunteered to maintain it), but good to learn about issues and alternatives to webdriver! Thanks @U5H74UNSF! I’ll likely add a note to Etaoin’s “interesting alternatives” section.

borkdude13:07:54

@UE21H2HHD There's plenty of etaoin users and I've used it with satisfaction in the past

lread13:07:24

Thanks, that helps! ❤️

borkdude13:07:29

And I'll probably keep using it for new projects (like scittle), since I can now use it with #babashka :)

lread13:07:22

Yeah, I’m using it to generate contributor badges for READMEs. Great for stuff like that too.

borkdude13:07:52

I'm also using it for my watch script for bb book, to re-render the rendered book in the browser while developing

borkdude13:07:25

I probably don't write sophisticated UIs enough (like http://nextjournal.com) to experience any of the downsizes ;)

lread14:07:50

There is a section in docs listing companies using Etaoin, but adding other more specific example usages would also likely be helpful to folks…. I’ll likely do that.

borkdude14:07:21

http://grep.app will probably show more repos using it

lread14:07:49

Yeah, good idea.

borkdude13:07:16

Interesting tidbit about playwright + java API vs Node:

borkdude21:08:25

Got a minimal thing working here. @U11SJ6Q0K came back with the solution, needed the reset! https://gist.github.com/borkdude/627be1593c0abe20d5c53caf6aa34d85 Note that text-of sometimes complains with "no element with that id", I think maybe due to having to wait for it to appear or something. Note sure how to do that, but voila, it seems possible to re-use a playwright-downloaded browser with this library and bb.

athomasoriginal16:07:42

I'm looking to run a figwheel process and a postcss process at the same time. The CSS process is just a file watcher. I want to be able to interact with figwheel as I normally would. I'm good with the CSS logs being interspersed between the figwheel logs. I also want to kill both processes when the terminal gets the ctrl-c. would sh or processBuilder be the right choice for this? :thinking_face:

borkdude16:07:57

@U6GNVEWQG bb tasks has a solution for this:

{:tasks {
    figwheel (clojure "-M:figwheel") 
    postcss (shell "npx whatever")
    -dev {:depends [figwheel postcss]}
    dev {:task (run '-dev {:parallel true})}
}}

borkdude16:07:14

And then you can run bb dev and have those processes run in parallel

athomasoriginal16:07:49

oh lord. I can't believe I missed that :face_palm:. Sad that I overcomplicated just an obvious thing 😞 Thanks!!

steveb8n23:07:23

Q: is there something like Enlive or Enliven in the bb world? I have a html website that I want to use as a template

steveb8n23:07:09

maybe Enlive already works with bb but its a stagnant project so I’m wondering if there’s a modern take on it

steveb8n23:07:26

I’m also planning to use https://github.com/ptaoussanis/tempura . Is this known to work in bb land?

steveb8n23:07:50

I see neither of these libs in https://babashka.org/toolbox/ hence the questions

steveb8n23:07:36

I’m fine with using these libs in jvm Clojure but I always try bb first these days 🙂

borkdude04:07:05

There is the bootleg pod which supports enlive. Also bb comes with Selmer and hiccup

steveb8n06:07:25

great I’ll give it a try. I’ll also see if tempura works OOTB