Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2023-03-08

Channels

#etaoin
<
2023-03-08
>
shem14:03:10

i have a babashka/etaoin script that stopped working a couple of weeks ago. i've tried with chrome and firefox, also with :path-browser defined but it quits with “clojure.lang.ExceptionInfo: gave up trying to launch :chrome after 1 tries [at etaoin/api.clj:3504:9]”. i've tried maxing out logging but it won't elaborate on the failure. i get the same error in a minimal repl session. this is on arch linux, babashka v1.2.174, etaoin 1.0.39

☝️ 2
lread14:03:18

Hi @U067EQZL4, can you extract a minimal example so we can try to reproduce the issue? One thing that trips me up sometimes is a WebDriver/browser version mismatch. Chrome might get update on my system, but then I also need to update chromedriver.

shem14:03:33

user=> (require '[babashka.deps :as deps])
nil
user=> (deps/add-deps '{:deps {etaoin/etaoin {:mvn/version "1.0.39"}}})
nil
user=> (require '[etaoin.api :as e])
nil
user=> (def driver (e/chrome))
2023-03-08T14:03:12.293Z UnknownHost DEBUG [etaoin.api:?] - Created driver: chrome 127.0.0.1:45035
2023-03-08T14:03:17.296Z UnknownHost DEBUG [etaoin.api:?] - Starting process: chromedriver --port=45035
2023-03-08T14:03:17.300Z UnknownHost DEBUG [etaoin.api:?] - Waiting until 127.0.0.1:45035 is running
clojure.lang.ExceptionInfo: gave up trying to launch :chrome after 1 tries [at etaoin/api.clj:3504:9]
user=> (def driver (e/chrome {:headless false :path-browser "/opt/google/chrome-beta/chrome"
                :args "--ozone-platform-hint=auto" :log-level :all :driver-log-level "DEBUG"}))
2023-03-08T14:07:13.899Z UnknownHost DEBUG [etaoin.api:?] - Created driver: chrome 127.0.0.1:43547
2023-03-08T14:07:18.900Z UnknownHost DEBUG [etaoin.api:?] - Starting process: chromedriver --port=43547 --log-level=DEBUG
2023-03-08T14:07:18.904Z UnknownHost DEBUG [etaoin.api:?] - Waiting until 127.0.0.1:43547 is running
clojure.lang.ExceptionInfo: gave up trying to launch :chrome after 1 tries [at etaoin/api.clj:3504:9]

shem14:03:25

i have checked that the webdriver version matches browser version

lread14:03:14

cool, thanks, I'll give this a whirl sometime soon. If you have a moment can you clone https://github.com/clj-commons/etaoin and then from etaoin run bb tools-versions for me? This is a etaoin developer sanity task, but can help in troubleshooting situations like this one.

shem14:03:27

i just tried in a fresh jvm clojure project and that also dies with {:message "unknown error: Chrome failed to start: exited abnormally.\n (unknown error: DevToolsActivePort file doesn't exist)\n (The process started from chrome location /opt/google/chrome-beta/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)\n (Driver info: chromedriver=111.0.5563.41 (976ef12907ef9b413c2d929c043307b415d27b9e-refs/branch-heads/5563@{#737}),platform=Linux 6.2.2-arch1-1 x86_64)"}},

shem14:03:16

|-------------------+-----------------------------------------------------------------------------------------------------+-----------------------|
|        Name       |                                               Version                                               |          Path         |
|-------------------+-----------------------------------------------------------------------------------------------------+-----------------------|
| Java              | openjdk version "11.0.18" 2023-01-17                                                                | /usr/bin/java         |
|                   | OpenJDK Runtime Environment (build 11.0.18+10)                                                      |                       |
|                   | OpenJDK 64-Bit Server VM (build 11.0.18+10, mixed mode)                                             |                       |
| Babashka          | babashka v1.2.174                                                                                   | /usr/bin/bb           |
| Image Magick      | Version: ImageMagick 7.1.0-62 Q16-HDRI x86_64 20885                          | /usr/bin/identify     |
| Chrome            | ** ERROR: bin not found: google-chrome **                                                           |                       |
| Chrome Webdriver  | ChromeDriver 111.0.5563.41 (976ef12907ef9b413c2d929c043307b415d27b9e-refs/branch-heads/5563@{#737}) | /usr/bin/chromedriver |
| Firefox           | Mozilla Firefox 110.0.1                                                                             | /usr/bin/firefox      |
| Firefox Webdriver | ** ERROR: bin not found: geckodriver **                                                             |                       |
|-------------------+-----------------------------------------------------------------------------------------------------+-----------------------|

shem14:03:09

chrome is version 111.0.5563.64-1

lread14:03:03

cool, thanks, I'll take a peek after I have some breakfast.

Zach Thomas15:03:16

I guess I came to the right place. I have this issue, too. Strange thing is that it's the same for firefox as well as chrome. 

user=> (require '[etaoin.api :as e] '[etaoin.keys :as k])
nil
user=> (def driver (e/chrome))
2023-03-08T15:37:29.961Z L-SAN-220728-01 DEBUG [etaoin.api:?] - Created driver: chrome 127.0.0.1:57139
2023-03-08T15:37:29.997Z L-SAN-220728-01 DEBUG [etaoin.api:?] - Starting process: chromedriver --port=57139
2023-03-08T15:37:30.003Z L-SAN-220728-01 DEBUG [etaoin.api:?] - Waiting until 127.0.0.1:57139 is running
clojure.lang.ExceptionInfo: gave up trying to launch :chrome after 1 tries [at etaoin/api.clj:3504:9]
user=> (def driver (e/firefox))
2023-03-08T15:37:40.917Z L-SAN-220728-01 DEBUG [etaoin.api:?] - Created driver: firefox 127.0.0.1:57144
2023-03-08T15:37:40.918Z L-SAN-220728-01 DEBUG [etaoin.api:?] - Starting process: geckodriver --port 57144
2023-03-08T15:37:40.920Z L-SAN-220728-01 DEBUG [etaoin.api:?] - Waiting until 127.0.0.1:57144 is running
clojure.lang.ExceptionInfo: gave up trying to launch :firefox after 1 tries [at etaoin/api.clj:3504:9]

lread15:03:23

ya, ok, can reproduce on Pop!_OS

lread15:03:00

Interesting, huh? I'll try to figure out what is going on.

Zach Thomas15:03:40

Thank you! I'm happy to try any repro steps you want.

Zach Thomas15:03:21

|-------------------+-----------------------------------------------------------------------------------------------------+--------------------------------------------|
|        Name       |                                               Version                                               |                    Path                    |
|-------------------+-----------------------------------------------------------------------------------------------------+--------------------------------------------|
| Java              | openjdk version "17.0.6" 2023-01-17                                                                 | /Users/zachthomas/.asdf/shims/java         |
|                   | OpenJDK Runtime Environment Temurin-17.0.6+10 (build 17.0.6+10)                                     |                                            |
|                   | OpenJDK 64-Bit Server VM Temurin-17.0.6+10 (build 17.0.6+10, mixed mode)                            |                                            |
| Babashka          | babashka v1.2.174                                                                                   | /opt/homebrew/bin/bb                       |
| Image Magick      | Version: ImageMagick 7.1.0-62 Q16-HDRI aarch64 20885                         | /opt/homebrew/bin/magick                   |
| Chrome            | 111.0.5563.64                                                                                       | /Applications/Google             |
| Chrome Webdriver  | ChromeDriver 110.0.5481.77 (65ed616c6e8ee3fe0ad64fe83796c020644d42af-refs/branch-heads/5481@{#839}) | /opt/homebrew/bin/chromedriver             |
| Firefox           | 110.0.1                                                                                             | /Applications/Firefox.app                  |
| Firefox Webdriver | geckodriver 0.32.2                                                                                  | /opt/homebrew/bin/geckodriver              |
| Edge              | ** ERROR: mac app not found: Microsoft Edge **                                                      |                                            |
| Edge Webdriver    | ** ERROR: bin not found: msedgedriver **                                                            |                                            |
| Safari            | 16.3                                                                                                | /Applications/Safari.app                   |
| Safari Webdriver  | Included with Safari 16.3 (18614.4.6.1.6)                                                           | /System/Cryptexes/App/usr/bin/safaridriver |
|-------------------+-----------------------------------------------------------------------------------------------------+--------------------------------------------|

lread15:03:12

@U024E7BEV8D you seem to be running from babashka too, yeah?

Zach Thomas15:03:43

I am. Want me to use a plain ol' clojure repl?

borkdude15:03:42

could it be the syntax change in babashka/process? should not have to matter but perhaps something broke

lread15:03:45

Etaoin tests are failing for babashka, but seem to be passing for jvm.

lread15:03:59

Good guess, @U04V15CAJ, probably.

borkdude15:03:59

are you using the newest version of babashka/process in the JVM?

borkdude15:03:13

if not, then that would explain the difference

borkdude15:03:46

note: this should not have broken, but may accidentally have. I repent

lread15:03:48

Right. Nope have not updated etaoin, still using 0.3.11.

borkdude15:03:00

I'll take a look

borkdude15:03:43

@UE21H2HHD I want to trigger the tests by pushing to a branch but that doesn't seem to work: 

on:
  push:
    branches: ['master']
  pull_request:

borkdude15:03:55

I dont' see why this isn't just [push, pull_request]

borkdude15:03:11

I'll change it unless there's a good reason

lread15:03:06

I think I didn't want to build branches that were not PRs... maybe? Go ahead and change if you like.

lread15:03:28

I think that is still probably my preference, but if your workflow is to always run tests on branches, then that's ok.

borkdude16:03:58

how else would I know if my commits are valid. I only create a PR once I'm done

lread16:03:13

run tests locally?

borkdude16:03:16

anyway, locally my tests seem to work

borkdude16:03:35

we all know that "works on my machine" isn't the best way to test things

lread16:03:18

do CI tests run for folks before they create PRs?

borkdude16:03:38

yes, under their own account on Github

borkdude16:03:05

thank god they do, else I'd have to look at some PR that failed every minute

borkdude16:03:13

that gets old fast

borkdude16:03:37

anyway, I want to solve the problem, not get into a bikeshed about who prefers what CI config

borkdude16:03:12

and since locally they pass but on CI they fail, any help is welcome

lread16:03:15

yeah, no, I'm not bikeshedding, just learning about something I might have misunderstood

lread16:03:46

but we can talk about that separately

lread16:03:55

etaoin bb test:bb fails for me locally

lread16:03:08

I can poke around @U04V15CAJ, maybe easier for me because I have a local failure.

borkdude16:03:32

$ bb test:bb
----- Error --------------------------------------------------------------------
Type:     java.lang.Exception
Message:  File does not exist: test:bb

lread16:03:23

silly question: are you synched to master?

borkdude16:03:01

maybe not...

borkdude16:03:33

nope 25 commits behind 😆 this day isn't going too well, damnit

lread16:03:23

ha! that'll make the good days feel even better maybe...

borkdude16:03:33

Failed to launch WebDriver binary `msedgedriver`.

borkdude16:03:42

is there a way to limit to chromedriver?

Zach Thomas16:03:55

bb test:bb -b chrome

lread16:03:09

Zach beat me to it.

borkdude16:03:47

Most things run fine here, but: 

ERROR in (test-screenshot-element) (/Users/borkdude/dev/etaoin/test/etaoin/api_test.clj:600)
chrome
expected: (valid-image? path)
  actual: clojure.lang.ExceptionInfo: please install image magick, we use it for screenshot image verification
{}

lread16:03:57

Also bb tools-versions can help you sanity check your installs.

borkdude16:03:02

brew install imagemagick

lread16:03:06

yeah we use that in tests.

Zach Thomas16:03:13

Now you have time to get a coffee. 😄

borkdude16:03:51

can I limit my test run to a failing test? 

bb test:bb -b chrome ...?

borkdude16:03:02

so I don't have to download half the internet

borkdude16:03:25

ah there's an -h option, excellent

borkdude16:03:40

but which test failed for you @UE21H2HHD?

borkdude16:03:49

just give me one :)

lread16:03:18

So... etaoin.api-test test setup it seems

borkdude16:03:48

is that etaoin.api-test/test-setup ?

borkdude16:03:03

Could not resolve var: etaoin.api-test/test-setup

borkdude16:03:09

I don't know what you mean

lread16:03:19

No the test fixture.

borkdude16:03:02

$ bb test:bb -b chrome -v etaoin.api-test/test-visible

[ TASK test:bb -b chrome -v etaoin.api-test/test-visible ]-----------------------

[ Running tests ]----------------------------------------------------------------
suites: <none specified>
browsers: chrome
runner-args: [:vars etaoin.api-test/test-visible]

Running tests in #{"test"}

Testing etaoin.api-test
Testing with browsers: [:chrome]
:args ("chromedriver" "--port=51098") :type clojure.lang.LazySeq
=== test-visible [bb][chrome]

Ran 1 tests containing 4 assertions.
0 failures, 0 errors.

TASK test:bb done.

lread16:03:02

❯ bb test:bb -b chrome -v etaoin.api-test/test-visible

[ TASK test:bb -b chrome -v etaoin.api-test/test-visible ]-----------------------

[ Running tests ]----------------------------------------------------------------
suites: <none specified>
browsers: chrome
runner-args: [:vars etaoin.api-test/test-visible]

Running tests in #{"test"}

Testing etaoin.api-test
Testing with browsers: [:chrome]
----- Error --------------------------------------------------------------------
Type:     clojure.lang.ExceptionInfo
Message:  gave up trying to launch :chrome after 1 tries
Data:     {}
Location: /home/lee/proj/oss/clj-commons/etaoin/src/etaoin/api.clj:3504:9

----- Context ------------------------------------------------------------------
3500:   (let [max-tries (inc webdriver-failed-launch-retries)]
3501:     (loop [try-num 1
3502:            ex nil]
3503:       (if (> try-num max-tries)
3504:         (throw (ex-info (format "gave up trying to launch %s after %d tries" (:type driver) max-tries) {} ex))
              ^--- gave up trying to launch :chrome after 1 tries
3505:         (do
3506:           (when ex
3507:             (log/warnf ex "unexpected exception occurred launching %s, try %d (of a max of %d)"
3508:                        (:type driver) (dec try-num) max-tries)
3509:             (Thread/sleep 100))

----- Stack trace --------------------------------------------------------------
etaoin.api/-run-driver                         - /home/lee/proj/oss/clj-commons/etaoin/src/etaoin/api.clj:3504:9
etaoin.api/-run-driver                         - /home/lee/proj/oss/clj-commons/etaoin/src/etaoin/api.clj:3460:1
etaoin.api/boot-driver                         - /home/lee/proj/oss/clj-commons/etaoin/src/etaoin/api.clj:3663:56
etaoin.api/boot-driver                         - /home/lee/proj/oss/clj-commons/etaoin/src/etaoin/api.clj:3640:1
etaoin.api-test                                - /home/lee/proj/oss/clj-commons/etaoin/test/etaoin/api_test.clj:71:3
... (run with --debug to see elided elements)
cognitect.test-runner.api/do-test              - /home/lee/.gitlibs/libs/io.github.cognitect-labs/test-runner/dfb30dd6605cb6c0efc275e1df1736f6e90d4d73/src/cognitect/test_runner/api.clj:6:1
cognitect.test-runner.api                      - /home/lee/.gitlibs/libs/io.github.cognitect-labs/test-runner/dfb30dd6605cb6c0efc275e1df1736f6e90d4d73/src/cognitect/test_runner/api.clj:28:30
exec-7a6b831b-ba00-48d8-becf-22d3f87d8245/exec - <expr>:22:1
exec-7a6b831b-ba00-48d8-becf-22d3f87d8245      - <expr>:4:1
user-6966f77b-250c-46d6-803f-3372c14518b1      - <expr>:27:46


-ERROR:--------------------------------------------------------------------------
  exited with 1 for: ["bb" "-test:bb" ":vars" "etaoin.api-test/test-visible"]    
 --------------------------------------------------------------------------------

lread16:03:41

❯ bb tools-versions

[ TASK tools-versions  ]---------------------------------------------------------
|-------------------+-----------------------------------------------------------------------------------------------------+----------------------------------------------------|
|        Name       |                                               Version                                               |                        Path                        |
|-------------------+-----------------------------------------------------------------------------------------------------+----------------------------------------------------|
| Java              | openjdk version "17.0.6" 2023-01-17                                                                 | /home/lee/.sdkman/candidates/java/current/bin/java |
|                   | OpenJDK Runtime Environment Temurin-17.0.6+10 (build 17.0.6+10)                                     |                                                    |
|                   | OpenJDK 64-Bit Server VM Temurin-17.0.6+10 (build 17.0.6+10, mixed mode, sharing)                   |                                                    |
| Babashka          | babashka v1.2.174                                                                                   | /home/linuxbrew/.linuxbrew/bin/bb                  |
| Image Magick      | Version: ImageMagick 6.9.11-60 Q16 x86_64 2021-01-25                         | /usr/bin/identify                                  |
| Chrome            | Google Chrome 111.0.5563.64                                                                         | /usr/bin/google-chrome                             |
| Chrome Webdriver  | ChromeDriver 111.0.5563.64 (c710e93d5b63b7095afe8c2c17df34408078439d-refs/branch-heads/5563@{#995}) | /home/lee/bin/chromedriver                         |
| Firefox           | Mozilla Firefox 110.0.1                                                                             | /usr/bin/firefox                                   |
| Firefox Webdriver | geckodriver 0.32.2 (602aa16c20d4 2023-02-08 00:09 +0000)                                            | /home/lee/bin/geckodriver                          |
|-------------------+-----------------------------------------------------------------------------------------------------+----------------------------------------------------|

TASK tools-versions done.

borkdude16:03:56

can you get rid of that retry logic so we can better see what's going wrong?

Zach Thomas16:03:45

If it's helpful, here's a one-liner that fails: 

bb -Sdeps '{:deps {etaoin/etaoin {:mvn/version "1.0.39"}}}' -e "(require '[etaoin.api :as e]) (def driver (e/chrome))"

borkdude16:03:22

That's very helpful but for me that just works

borkdude16:03:34

what is your bb version?

borkdude16:03:01

what's the error message

Zach Thomas16:03:19

❯ bb -Sdeps '{:deps {etaoin/etaoin {:mvn/version "1.0.39"}}}' -e "(require '[etaoin.api :as e]) (def driver (e/chrome))"
2023-03-08T16:22:06.163Z L-SAN-220728-01 DEBUG [etaoin.api:?] - Created driver: chrome 127.0.0.1:58201
2023-03-08T16:22:06.183Z L-SAN-220728-01 DEBUG [etaoin.api:?] - Starting process: chromedriver --port=58201
2023-03-08T16:22:06.186Z L-SAN-220728-01 DEBUG [etaoin.api:?] - Waiting until 127.0.0.1:58201 is running
----- Error --------------------------------------------------------------------
Type:     clojure.lang.ExceptionInfo
Message:  gave up trying to launch :chrome after 1 tries
Data:     {}
Location: etaoin/api.clj:3504:9

----- Context ------------------------------------------------------------------
3500:   (let [max-tries (inc webdriver-failed-launch-retries)]
3501:     (loop [try-num 1
3502:            ex nil]
3503:       (if (> try-num max-tries)
3504:         (throw (ex-info (format "gave up trying to launch %s after %d tries" (:type driver) max-tries) {} ex))
              ^--- gave up trying to launch :chrome after 1 tries
3505:         (do
3506:           (when ex
3507:             (log/warnf ex "unexpected exception occurred launching %s, try %d (of a max of %d)"
3508:                        (:type driver) (dec try-num) max-tries)
3509:             (Thread/sleep 100))

----- Stack trace --------------------------------------------------------------
etaoin.api/-run-driver - etaoin/api.clj:3504:9
etaoin.api/-run-driver - etaoin/api.clj:3460:1
etaoin.api/boot-driver - etaoin/api.clj:3663:56
etaoin.api/chrome      - etaoin/api.clj:3655:4
etaoin.api/chrome      - etaoin/api.clj:3686:1
user                   - <expr>:1:43

borkdude16:03:20

$  bb -Sdeps '{:deps {etaoin/etaoin {:mvn/version "1.0.39"}}}' -e "(require '[etaoin.api :as e]) (def driver (e/chrome))"
2023-03-08T16:24:08.574Z m1.local DEBUG [etaoin.api:?] - Created driver: chrome 127.0.0.1:51141
2023-03-08T16:24:08.581Z m1.local DEBUG [etaoin.api:?] - Starting process: chromedriver --port=51141
2023-03-08T16:24:08.583Z m1.local DEBUG [etaoin.api:?] - Waiting until 127.0.0.1:51141 is running
2023-03-08T16:24:08.920Z m1.local DEBUG [etaoin.api:?] - Waiting until 127.0.0.1:51141 is running
2023-03-08T16:24:08.925Z m1.local DEBUG [etaoin.impl.client:?] - chrome 127.0.0.1:51141   POST session {:desiredCapabilities {:loggingPrefs {:browser "ALL"}}}
#'user/driver

borkdude16:03:45

I think the retry logic is in the way

borkdude16:03:49

of seeing the real error

lread16:03:23

Can bb emit the cause? The ex ? Or does not it not work that way. 

3504:         (throw (ex-info (format "gave up trying to launch %s after %d tries" (:type driver) max-tries) {} ex))
              ^--- gave up trying to launch :chrome after 1 tries

borkdude16:03:54

maybe with bb --debug, not sure

borkdude16:03:14

I don't think it prints more than one msg. I'll just get rid of the retry logic myself and push to CI

lread16:03:00

Ah yes. 

❯ bb --debug -Sdeps '{:deps {etaoin/etaoin {:mvn/version "1.0.39"}}}' -e "(require '[etaoin.api :as e]) (def driver (e/chrome))"
2023-03-08T16:26:02.751Z pop-os DEBUG [etaoin.api:?] - Created driver: chrome 127.0.0.1:46473
2023-03-08T16:26:02.754Z pop-os DEBUG [etaoin.api:?] - Starting process: chromedriver --port=46473
2023-03-08T16:26:02.758Z pop-os DEBUG [etaoin.api:?] - Waiting until 127.0.0.1:46473 is running
----- Error --------------------------------------------------------------------
Type:     clojure.lang.ExceptionInfo
Message:  gave up trying to launch :chrome after 1 tries
Data:     {}
Location: etaoin/api.clj:3504:9

----- Context ------------------------------------------------------------------
3500:   (let [max-tries (inc webdriver-failed-launch-retries)]
3501:     (loop [try-num 1
3502:            ex nil]
3503:       (if (> try-num max-tries)
3504:         (throw (ex-info (format "gave up trying to launch %s after %d tries" (:type driver) max-tries) {} ex))
              ^--- gave up trying to launch :chrome after 1 tries
3505:         (do
3506:           (when ex
3507:             (log/warnf ex "unexpected exception occurred launching %s, try %d (of a max of %d)"
3508:                        (:type driver) (dec try-num) max-tries)
3509:             (Thread/sleep 100))

----- Stack trace --------------------------------------------------------------
etaoin.api/-run-driver - etaoin/api.clj:3504:9
etaoin.api/-run-driver - etaoin/api.clj:3460:1
etaoin.api/boot-driver - etaoin/api.clj:3663:56
etaoin.api/chrome      - etaoin/api.clj:3655:4
etaoin.api/chrome      - etaoin/api.clj:3686:1
user                   - <expr>:1:43

----- Exception ----------------------------------------------------------------
clojure.lang.ExceptionInfo: gave up trying to launch :chrome after 1 tries
{:type :sci/error, :line 3504, :column 9, :message "gave up trying to launch :chrome after 1 tries", :sci.impl/callstack #object[clojure.lang.Volatile 0x3397b914 {:status :ready, :val ({:line 1, :column 43, :ns #object[sci.lang.Namespace 0x7523ce50 "user"], :file "<expr>", :sci.impl/f-meta {:name chrome, :ns #object[sci.lang.Namespace 0x7d413ed5 "etaoin.api"], :file "etaoin/api.clj", :arglists ([] [opts]), :line 3686, :column 1, :doc "Launch and return a Chrome driver.\n\n  `opts` map is optionally, see [Driver Options](/doc/01-user-guide.adoc#driver-options)."}} {:line 3655, :column 4, :ns #object[sci.lang.Namespace 0x7d413ed5 "etaoin.api"], :file "etaoin/api.clj", :sci.impl/f-meta {:name boot-driver, :ns #object[sci.lang.Namespace 0x7d413ed5 "etaoin.api"], :file "etaoin/api.clj"}} {:line 3663, :column 56, :ns #object[sci.lang.Namespace 0x7d413ed5 "etaoin.api"], :file "etaoin/api.clj", :sci.impl/f-meta {:name -run-driver, :ns #object[sci.lang.Namespace 0x7d413ed5 "etaoin.api"], :file "etaoin/api.clj", :private true, :arglists ([driver {:keys [webdriver-failed-launch-retries], :as opts}]), :doc "Runs a driver process locally.\n\n  Creates a UNIX process with a Webdriver HTTP server. Host and port\n  are taken from a `driver` argument. Updates a driver instance with\n  new fields with process information. Returns modified driver.\n\n  Arguments:\n\n  - `driver` is a map created with `-create-driver` function.\n\n  - `opts` is an optional map with the following possible parameters:\n\n  -- `:path-driver` is a string path to the driver's binary file.\n\n  -- `:path-browser` is a string path to the browser's binary\n  file. When not passed, the driver discovers it by its own.\n\n  -- `:webdriver-failed-launch-retries` number of times to retry launching webdriver process.\n\n  -- `:log-level` a keyword to set browser's log level. Used when fetching\n  browser's logs. Possible values are: `:off`, `:debug`, `:warn`, `:info`,\n  `:error`, `:all`.\n\n  -- `:driver-log-level` a keyword to set driver's log level.\n  The value is a string. Possible values are:\n  chrome: [ALL, DEBUG, INFO, WARNING, SEVERE, OFF]\n  phantomjs: [ERROR, WARN, INFO, DEBUG] (default INFO)\n  firefox [fatal, error, warn, info, config, debug, trace]\n\n  -- `:log-stdout` and `:log-stderr`. Paths to the driver's log files as strings.\n  Specify `:inherit` to inherit destination from calling process (ex. console).\n  When not set, the output goes to /dev/null (or NUL on Windows)\n\n  -- `:args-driver` is a vector of additional arguments to the\n  driver's process.\n\n  -- `:env` is a map with system ENV variables. Keys are turned into\n  upper-case strings.", :line 3460, :column 1}} {:ns #object[sci.lang.Namespace 0x7d413ed5 "etaoin.api"], :file "etaoin/api.clj"} {:line 3504, :column 9, :ns #object[sci.lang.Namespace 0x7d413ed5 "etaoin.api"], :file "etaoin/api.clj", :special true})}], :file "etaoin/api.clj"}
 at sci.impl.utils$rethrow_with_location_of_node.invokeStatic (utils.cljc:129)
    sci.impl.analyzer$analyze_throw$reify__4264.eval (analyzer.cljc:968)
    sci.impl.analyzer$return_if$reify__4219.eval (analyzer.cljc:849)
    sci.impl.fns$fun$arity_2__1171.invoke (fns.cljc:108)
    sci.impl.analyzer$return_call$reify__4502.eval (analyzer.cljc:1402)
    sci.impl.analyzer$analyze_let_STAR_$reify__4183.eval (analyzer.cljc:629)
    sci.impl.analyzer$analyze_let_STAR_$reify__4181.eval (analyzer.cljc:618)
    sci.impl.analyzer$analyze_let_STAR_$reify__4187.eval (analyzer.cljc:663)
    sci.impl.fns$fun$arity_2__1171.invoke (fns.cljc:108)
    sci.lang.Var.invoke (lang.cljc:202)
    sci.impl.analyzer$return_call$reify__4504.eval (analyzer.cljc:1402)
    sci.impl.analyzer$return_if$reify__4219.eval (analyzer.cljc:849)
    sci.impl.analyzer$analyze_let_STAR_$reify__4181.eval (analyzer.cljc:618)
    sci.impl.analyzer$analyze_let_STAR_$reify__4185.eval (analyzer.cljc:642)
    sci.impl.analyzer$analyze_let_STAR_$reify__4189.eval (analyzer.cljc:686)
    sci.impl.fns$fun$arity_2__1171.invoke (fns.cljc:108)
    clojure.lang.AFn.applyToHelper (AFn.java:156)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:667)
    sci.impl.analyzer$analyze_fn_STAR_$reify__4147$f__4148.doInvoke (analyzer.cljc:529)
    clojure.lang.RestFn.invoke (RestFn.java:421)
    sci.lang.Var.invoke (lang.cljc:202)
    sci.impl.analyzer$return_call$reify__4504.eval (analyzer.cljc:1402)
    sci.impl.fns$fun$arity_1__1166.invoke (fns.cljc:107)
    clojure.lang.AFn.applyToHelper (AFn.java:154)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:667)
    sci.impl.analyzer$analyze_fn_STAR_$reify__4147$f__4148.doInvoke (analyzer.cljc:529)
    clojure.lang.RestFn.invoke (RestFn.java:408)
    clojure.core$partial$fn__5908.invoke (core.clj:2640)
    sci.lang.Var.invoke (lang.cljc:198)
    sci.impl.analyzer$return_call$reify__4496.eval (analyzer.cljc:1402)
    sci.impl.evaluator$eval_def.invokeStatic (evaluator.cljc:27)
    sci.impl.analyzer$analyze_def$reify__4206.eval (analyzer.cljc:761)
    sci.impl.interpreter$eval_form.invokeStatic (interpreter.cljc:40)
    sci.impl.interpreter$eval_string_STAR_.invokeStatic (interpreter.cljc:62)
    sci.core$eval_string_STAR_.invokeStatic (core.cljc:270)
    babashka.main$exec$fn__31074$fn__31112$fn__31113.invoke (main.clj:1042)
    babashka.main$exec$fn__31074$fn__31112.invoke (main.clj:1042)
    babashka.main$exec$fn__31074.invoke (main.clj:1032)
    clojure.lang.AFn.applyToHelper (AFn.java:152)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.core$with_bindings_STAR_.invokeStatic (core.clj:1990)
    clojure.core$with_bindings_STAR_.doInvoke (core.clj:1990)
    clojure.lang.RestFn.invoke (RestFn.java:425)
    babashka.main$exec.invokeStatic (main.clj:824)
    babashka.main$main.invokeStatic (main.clj:1155)
    babashka.main$main.doInvoke (main.clj:1124)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:667)
    babashka.main$_main$f__31186__auto____31196.invoke (main.clj:1194)
    clojure.lang.AFn.run (AFn.java:22)
    java.lang.Thread.run (Thread.java:1589)
    com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine (PlatformThreads.java:775)
    com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine (PosixPlatformThreads.java:203)

lread16:03:38

Hmmm... firefox passes for me: 

$ bb --debug -Sdeps '{:deps {etaoin/etaoin {:mvn/version "1.0.39"}}}' -e "(require '[etaoin.api :as e]) (def driver (e/firefox))"

2023-03-08T16:28:19.743Z pop-os DEBUG [etaoin.api:?] - Created driver: firefox 127.0.0.1:38401

2023-03-08T16:28:19.747Z pop-os DEBUG [etaoin.api:?] - Starting process: geckodriver --port 38401

2023-03-08T16:28:19.751Z pop-os DEBUG [etaoin.api:?] - Waiting until 127.0.0.1:38401 is running

2023-03-08T16:28:19.752Z pop-os DEBUG [etaoin.api:?] - Waiting until 127.0.0.1:38401 is running

2023-03-08T16:28:19.752Z pop-os DEBUG [etaoin.impl.client:?] - firefox 127.0.0.1:38401   POST session {:desiredCapabilities {:loggingPrefs {:browser "ALL"}}}

#'user/driver
``` Maybe we are dealing with a chrome change... but wonder why it fails only on bb, not jvm...

borkdude16:03:20

the tests also fail for bb + edge etc

borkdude16:03:48

:args ("chromedriver" "--port=49183") :type clojure.lang.LazySeq
----- Error --------------------------------------------------------------------
Type:     java.lang.IllegalArgumentException
Message:  No matching method sleep found taking 1 args
Location: /Users/runner/work/etaoin/etaoin/src/etaoin/api.clj:2504:8

Zach Thomas16:03:51

For me, (e/firefox) fails in exactly the same way. :man-shrugging:

borkdude16:03:23

Can you try: (Thread/sleep 1000) in your bb?

lread16:03:31

The old sleeparoo?

borkdude16:03:52

Just: 

bb -e '(Thread/sleep 1000)'

lread16:03:18

no errors when run

borkdude16:03:34

it seems to go wrong in etaoin await

borkdude16:03:06

(Thread/sleep ^long (* seconds 1000))

lread16:03:08

@U04V15CAJ, if your day is going sideways, I can dig in.

borkdude16:03:18

This also works for me: 

bb -e '((fn [x] (Thread/sleep ^long x)) 1000)'

lread16:03:19

ah maybe a jdk19 thread sleep signature change (what I meant by the old sleeparoo).

borkdude16:03:40

This also works for me: 

bb -e '((fn [x] (Thread/sleep ^long (inc x))) 1000)'

lread16:03:05

the above run fine for me

borkdude16:03:29

Btw I can also repro with the JDK19 one, I was running an old local dev build: 

/opt/homebrew/bin/bb -Sdeps '{:deps {etaoin/etaoin {:mvn/version "1.0.39"}}}' -e "(require '[etaoin.api :as e]) (def driver (e/chrome))"

lread16:03:16

oh well that's good

borkdude16:03:26

well, at least it isn't process then

borkdude16:03:30

but some interop thing

borkdude16:03:35

now on to a repro

borkdude16:03:53

I can only repro this when I don't pass a long: 

(wait 1.0)

lread16:03:13

Ditto. (wait 1) is fine, but (wait 1.0) I get No matching method sleep found taking 1 args

borkdude16:03:10

I'll change the wait function to: 

(defn wait
  "Sleep for `seconds`."
  (#_{:clj-kondo/ignore [:unused-binding]} [driver seconds]
   (wait seconds))
  ([seconds]
   (Thread/sleep (long (* seconds 1000)))))

borkdude16:03:15

This works for both JVM 19 and bb

lread16:03:11

Yeah that looks fine to me!

lread16:03:41

So bb didn't grok the type hint maybe?

borkdude16:03:55

it did but I suspect the type of the thing passed not being a long

borkdude16:03:19

(e/wait (/ windowTimeout 1000))

lread16:03:38

Hmmm... so jvm and bb behave differently for this. But I think your (long ...) is probably what we really want to do anyway.

lread16:03:50

So also bb not showing exception cause made things a bit tricky to diagnose.

borkdude16:03:51

do you run tests with jdk19?

borkdude16:03:05

because if you don't I get why the JVM is accepting this, else not sure

lread17:03:09

Not on CI, but did so locally. Lemme double check. (CI tests are already hugely long so only doing one jdk for now, could spot check jdk19 tho...).

lread17:03:36

❯ java --version
openjdk 19.0.2 2023-01-17
OpenJDK Runtime Environment Temurin-19.0.2+7 (build 19.0.2+7)
OpenJDK 64-Bit Server VM Temurin-19.0.2+7 (build 19.0.2+7, mixed mode, sharing)
❯ bb test:jvm -b chrome -v etaoin.api-test/test-visible

[ TASK test:jvm -b chrome -v etaoin.api-test/test-visible ]----------------------

[ Running tests ]----------------------------------------------------------------
suites: <none specified>
browsers: chrome
runner-args: [:vars etaoin.api-test/test-visible]

Running tests in #{"test"}

Testing etaoin.api-test
Testing with browsers: [:chrome]
=== test-visible [jvm][chrome]

Ran 1 tests containing 4 assertions.
0 failures, 0 errors.

TASK test:jvm done.

lread17:03:47

I put in that sleep fix a while ago for folks using JDK19, I expect I would have heard back from folks if there were issues.

borkdude17:03:37

I guess JVM Clojure puts an explicit long cast in there when emitting bytecode

lread17:03:32

I guess it must?

borkdude17:03:08

not if you always pass a long

borkdude17:03:29

perhaps bb could do the long cast via the long function to fix this, but right now doing this manually works

borkdude17:03:32

I'll make a PR after dinner

lread17:03:03

$ clj
....
user=> (def x "foo")
#'user/x
user=> (Thread/sleep ^long x)
ClassCastException class java.lang.String cannot be cast to class java.lang.Number (java.lang.String and java.lang.Number are in module java.base of loader 'bootstrap')  user/eval156 (NO_SOURCE_FILE:12)
user=> (def x 3/4)
#'user/x
user=> (Thread/sleep ^long x)
nil

borkdude17:03:07

it casts to a number ?

borkdude17:03:17

and then the JVM coerces it into a long probably

borkdude17:03:29

would be good to inspect the bytecode with decompiler

borkdude17:03:55

afk now, dinner time. but I will be back!

lread17:03:50

bon appetit!

borkdude18:03:45

yeah, but I'm also using this code in bb

borkdude18:03:06

I'll create a separate issue about this

borkdude18:03:26

This works: 

bb -e '(Thread/sleep (/ 1 2))'

borkdude18:03:56

No, it doesn't: 

/opt/homebrew/bin/bb -e '(Thread/sleep (/ 1 2))'
----- Error --------------------------------------------------------------------

borkdude18:03:19

this is because bb isn't passing type hints along with the reflective call

borkdude19:03:47

I did see a firefox failure, but I suspect that doesn't have to do with my PR

lread19:03:45

Annoyingly, there are sporadic failures still... I typically just have GitHub Actions rerun failed jobs.

lread19:03:39

I'll cut a release after merge.

🙏 2
lread19:03:40

Hey @U04V15CAJ, having bb somehow share an exception cause(s) would help in a situation like this. I might have missed it, but I don't see an existing bb issue for this, want one?

borkdude19:03:46

yes, with a repro plz!

👍 2
borkdude19:03:56

also with a comparison to what clj does

👍 2
Zach Thomas19:03:10

Y'all are amazing. I confirmed that I'm back up and running (both Chrome and Firefox) with this branch.

lread19:03:32

Thanks for chipping in and helping @U024E7BEV8D, much appreciated!

borkdude19:03:58

Thank you all for putting up with my grumpiness!

2
Zach Thomas19:03:37

Maybe your day is turning around now. 😉

🙌 2
lread19:03:51

And thank you for putting up with my future-grumpiness in some coming day. simple_smile

❤️ 2
borkdude19:03:47

grump-driven development

lread19:03:21

Well, a large portion of the Clojure community is, I hear, grumpy old men. Maybe you just got pickled in the brine.

lread19:03:14

Blech, CI tests are flakier than I remember.

Zach Thomas19:03:17

In fact, I am also a grumpy old man. But Clojure is what I do to feel better. 😄

😄 4
lread20:03:07

I'm gonna take a peek at that test-doc failure before merging. This might mean getting my macOS VirtualBox setup more than it is now (which might be not so easy). I'll try some quick tests in a branch first.

lread21:03:27

Etaoin and WebDrivers. Almost as fun as YAML!

💯 2
lread21:03:21

@U04V15CAJ regarding triggering on push, pull-request, the thing that might have irked me was that tests are triggered twice for one commit to a PR. But that's the trade-off, I guess.

borkdude21:03:32

right. that's an issue that's bothered me with Github Actions, it's the only CI I've seen to behave like that

lread21:03:47

It really kinda sucks. Unnecessary waste of compute.

borkdude21:03:56

climate change 🚀

😿 2
lread22:03:48

I'm kinda tired of trying to get CIs to do what I'd like them to do but maybe I'll play a bit with this one sometime later. The GitHub Actions default activities for the pull_request trigger are opened, synchronize or reopened . I'm now wondering if removing synchronize would avoid the double triggering on push. https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request

shem06:03:41

Amazing session, guys! Can confirm that with 1.0.40 this works with babashka. Still fails with jvm clojure though...

👍 4
shem06:03:41

no, actually it works when :headless true

lread13:03:17

@U067EQZL4 that jvm issue you hit is something different. Do you need any help looking into it?

shem08:03:50

@UE21H2HHD it's not acute, headless mode is fine

👍 2