Fork me on GitHub
#etaoin
<
2022-12-13
>
lucian30309:12:53

is there any way to make clicking work with chrome-headless instead of getting the exception below? it works well in regular chrome.

:cause throw+: {:response {:sessionId "7a2cc6b0a96f60b94cd65ee27c0f3cea", :status 64, :value {:message "element click intercepted: Element is not clickable at point (323, 1046)\n  (Session info: headless chrome=108.0.5359.40)\n  (Driver info: chromedriver=108.0.5359.40 (280b5fcaab3e877562b06cfaf2eb51121e13c3b9-refs/branch-heads/5359@{#689}),platform=Linux 5.15.0-53-generic x86_64)"}}, :path "session/7a2cc6b0a96f60b94cd65ee27c0f3cea/element/0.5304828470177068-11/click", :payload nil, :method :post, :type :etaoin/http-error, :port 39431, :host "127.0.0.1", :status 200, :webdriver-url nil, :driver {:args ("chromedriver" "--port=39431"), :headless true, :capabilities {:loggingPrefs {:browser "ALL"}, :chromeOptions {:prefs {:download.default_directory "/opt/gplimport/downloads/", :download.prompt_for_download false, :browser.helperApps.neverAsk.saveToDisk "application/zip"}, :args ("--headless")}}, :process {:proc #object[java.lang.ProcessImpl 0x40250ddd "Process[pid=1441195, exitValue=\"not exited\"]"], :exit nil, :in #object[java.lang.ProcessImpl$ProcessPipeOutputStream 0x715059be "java.lang.ProcessImpl$ProcessPipeOutputStream@715059be"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x7f2b6dd0 "java.lang.ProcessBuilder$NullInputStream@7f2b6dd0"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x7f2b6dd0 "java.lang.ProcessBuilder$NullInputStream@7f2b6dd0"],:prev nil, :cmd ["chromedriver" "--port=39431"]}, :locator "xpath", :type :chrome, :port 39431, :host "127.0.0.1", :url "", :session "7a2cc6b0a96f60b94cd65ee27c0f3cea"}}
:data {:response {:sessionId 7a2cc6b0a96f60b94cd65ee27c0f3cea, :status 64, :value {:message element click intercepted: Element is not clickable at point (323, 1046)
(Session info: headless chrome=108.0.5359.40)
(Driver info: chromedriver=108.0.5359.40 (280b5fcaab3e877562b06cfaf2eb51121e13c3b9-refs/branch-heads/5359@{#689}),platform=Linux 5.15.0-53-generic x86_64)}}, :path session/7a2cc6b0a96f60b94cd65ee27c0f3cea/element/0.5304828470177068-11/click, :payload nil, :method :post, :type :etaoin/http-error, :port 39431, :host 127.0.0.1, :status 200, :webdriver-url nil, :driver {:args (chromedriver --port=39431), :headless true, :capabilities {:loggingPrefs {:browser ALL}, :chromeOptions {:prefs {:download.default_directory /opt/gplimport/downloads/, :download.prompt_for_download false, :browser.helperApps.neverAsk.saveToDisk application/zip}, :args (--headless)}}, :process {:proc #object[java.lang.ProcessImpl 0x40250ddd Process[pid=1441195, exitValue="not exited"]], :exit nil, :in #object[java.lang.ProcessImpl$ProcessPipeOutputStream 0x715059be java.lang.ProcessImpl$ProcessPipeOutputStream@715059be], :out #object[java.lang.ProcessBuilder$NullInputStream 0x7f2b6dd0 java.lang.ProcessBuilder$NullInputStream@7f2b6dd0], :err #object[java.lang.ProcessBuilder$NullInputStream 0x7f2b6dd0 java.lang.ProcessBuilder$NullInputStream@7f2b6dd0], :prev nil, :cmd [chromedriver --port=39431]}, :locator xpath, :type :chrome, :port 39431, :host 127.0.0.1, :url , :session 7a2cc6b0a96f60b94cd65ee27c0f3cea}}
or if not, to get etaoin to try firefox. w/ the geckodriver installed i just get:
#error{:cause "error=2, No such file or directory",
       :via [{:type clojure.lang.Compiler$CompilerException,
              :message "Syntax error macroexpanding at (festinger.clj:3:17).",
              :data #:clojure.error{:phase :execution, :line 3, :column 17, :source "festinger.clj"},
              :at [clojure.lang.Compiler$InvokeExpr eval "Compiler.java" 3719]}
             {:type clojure.lang.ExceptionInfo,
              :message "Failed to launch WebDriver binary `geckodriver`.
                        Please ensure you have the driver installed.
                        If it is not on the PATH specify its location.
                        For driver installation, check out the Etaoin user guide: ",
              :data {:args ("geckodriver" "--port" 46007)},
              :at [etaoin.impl.proc$run invokeStatic "proc.clj" 40]}
             {:type java.io.IOException,
              :message "Cannot run program \"geckodriver\": error=2, No such file or directory",
              :at [java.lang.ProcessBuilder start "ProcessBuilder.java" 1143]}
             {:type java.io.IOException,
              :message "error=2, No such file or directory",
              :at [java.lang.ProcessImpl forkAndExec "ProcessImpl.java" -2]}],
       :trace [[java.lang.ProcessImpl forkAndExec "ProcessImpl.java" -2]
               [java.lang.ProcessImpl <init> "ProcessImpl.java" 314]
               [java.lang.ProcessImpl start "ProcessImpl.java" 244]
               [java.lang.ProcessBuilder start "ProcessBuilder.java" 1110]
               [java.lang.ProcessBuilder start "ProcessBuilder.java" 1073]
               [babashka.process$process invokeStatic "process.cljc" 350]
               [babashka.process$process invoke "process.cljc" 284]
               [babashka.process$process invokeStatic "process.cljc" 329]
               [babashka.process$process invoke "process.cljc" 284]
               [etaoin.impl.proc$run invokeStatic "proc.clj" 37]
               [etaoin.impl.proc$run invoke "proc.clj" 28]
               [etaoin.api$_run_driver_STAR_ invokeStatic "api.clj" 3455]
               [etaoin.api$_run_driver_STAR_ doInvoke "api.clj" 3426]
               [clojure.lang.RestFn invoke "RestFn.java" 423]
               [etaoin.api$_run_driver invokeStatic "api.clj" 3510]
               [etaoin.api$_run_driver invoke "api.clj" 3460]
               [etaoin.api$boot_driver invokeStatic "api.clj" 3663]
               [etaoin.api$boot_driver invoke "api.clj" 3640]
               [clojure.core$partial$fn__5908 invoke "core.clj" 2641]
               [clojure.lang.AFn applyToHelper "AFn.java" 154]
               [clojure.lang.RestFn applyTo "RestFn.java" 132]
               [clojure.lang.Compiler$InvokeExpr eval "Compiler.java" 3714]
               [clojure.lang.Compiler$DefExpr eval "Compiler.java" 457]
               [clojure.lang.Compiler eval "Compiler.java" 7199]
               [clojure.lang.Compiler eval "Compiler.java" 7183]
               [clojure.lang.Compiler eval "Compiler.java" 7149]
               [clojure.core$eval invokeStatic "core.clj" 3215]
               [clojure.core$eval invoke "core.clj" 3211]
               [nrepl.middleware.interruptible_eval$evaluate$fn__1595$fn__1596 invoke "interruptible_eval.clj" 87]
               [clojure.lang.AFn applyToHelper "AFn.java" 152]
               [clojure.lang.AFn applyTo "AFn.java" 144]
               [clojure.core$apply invokeStatic "core.clj" 667]
               [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1990]
               [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1990]
               [clojure.lang.RestFn invoke "RestFn.java" 425]
               [nrepl.middleware.interruptible_eval$evaluate$fn__1595 invoke "interruptible_eval.clj" 87]
               [clojure.main$repl$read_eval_print__9206$fn__9209 invoke "main.clj" 437]
               [clojure.main$repl$read_eval_print__9206 invoke "main.clj" 437]
               [clojure.main$repl$fn__9215 invoke "main.clj" 458]
               [clojure.main$repl invokeStatic "main.clj" 458]
               [clojure.main$repl doInvoke "main.clj" 368]
               [clojure.lang.RestFn invoke "RestFn.java" 1523]
               [nrepl.middleware.interruptible_eval$evaluate invokeStatic "interruptible_eval.clj" 84]
               [nrepl.middleware.interruptible_eval$evaluate invoke "interruptible_eval.clj" 56]
               [nrepl.middleware.interruptible_eval$interruptible_eval$fn__1628$fn__1632
                invoke
                "interruptible_eval.clj"
                152]
               [clojure.lang.AFn run "AFn.java" 22]
               [nrepl.middleware.session$session_exec$main_loop__1698$fn__1702 invoke "session.clj" 218]
               [nrepl.middleware.session$session_exec$main_loop__1698 invoke "session.clj" 217]
               [clojure.lang.AFn run "AFn.java" 22]
               [java.lang.Thread run "Thread.java" 833]]}
Of course, firefox and geckodriver are installed and in the path and when i test etaoin, i just get
bb test all                                                          69 ↵ ──(Tue,Dec13)─┘
----- Error --------------------------------------------------------------------
Type:     java.io.FileNotFoundException
Message:  test (Is a directory)

lread13:12:06

Hiya @U053KNT7C. It is easier for me to help if you share a reproducible example that I can run on my machine. For the Firefox issue, https://clojurians.slack.com/archives/C7KDM0EKW/p1667800817209179 might help.

lucian30301:12:41

hi @UE21H2HHD i'm not sure if i can reproduce it easily, but i'll try. i noticed a similar message w/ firefox. seems to be possibly some loading div covering things up so that is at least a clue ... as for firefox, i got it running on my local system (pop os) by symlinking the geckodriver to /usr/local/bin and using the :path-browser option to explicitly set the path /usr/bin/firefox ... but on my prod machine (which runs ubuntu, no screen) after doing that, i get a new error:

{:clojure.main/message
"Execution error (ExceptionInfo) at etaoin.api/running? (api.clj:2230).\nWebDriver process exited unexpectedly witha value: 1\n",
:clojure.main/triage
{:clojure.error/class clojure.lang.ExceptionInfo,
:clojure.error/line 2230,
:clojure.error/cause
"WebDriver process exited unexpectedly with a value: 1",
:clojure.error/symbol etaoin.api/running?,
:clojure.error/source "api.clj",
:clojure.error/phase :execution},
:clojure.main/trace
{:via
[{:type clojure.lang.Compiler$CompilerException,
:message
"Syntax error macroexpanding at (/tmp/form-init13003331923583710995.clj:1:74).",
:data
{:clojure.error/phase :execution,
:clojure.error/line 1,
:clojure.error/column 74,
:clojure.error/source "/tmp/form-init13003331923583710995.clj"},
:at [clojure.lang.Compiler load "Compiler.java" 7665]}
{:type clojure.lang.ExceptionInfo,
:message "gave up trying to launch :firefox after 1 tries",
:data {},
:at [etaoin.api$_run_driver invokeStatic "api.clj" 3504]}
{:type clojure.lang.ExceptionInfo,
:message "WebDriver process exited unexpectedly with a value: 1",
:data {},
:at [etaoin.api$running_QMARK_ invokeStatic "api.clj" 2230]}],
:trace
[[etaoin.api$running_QMARK_ invokeStatic "api.clj" 2230]
[etaoin.api$running_QMARK_ invoke "api.clj" 2218]
[etaoin.api$wait_running$fn__11054 invoke "api.clj" 2630]
[etaoin.api$wait_predicate$fn__10979 invoke "api.clj" 2503]
[etaoin.api$wait_predicate invokeStatic "api.clj" 2502]
[etaoin.api$wait_predicate invoke "api.clj" 2474]
[etaoin.api$wait_running invokeStatic "api.clj" 2630]
[etaoin.api$wait_running doInvoke "api.clj" 2617]
[clojure.lang.RestFn invoke "RestFn.java" 410]
[etaoin.api$_run_driver$fn__11494 invoke "api.clj" 3512]
[etaoin.api$_run_driver invokeStatic "api.clj" 3511]
[etaoin.api$_run_driver invoke "api.clj" 3460]
[etaoin.api$boot_driver invokeStatic "api.clj" 3663]
[etaoin.api$boot_driver invoke "api.clj" 3640]
[gplimport.scraper.core$run$fn__21444 invoke "core.clj" 14]
[gplimport.scraper.core$run invokeStatic "core.clj" 14]
[gplimport.scraper.core$run invoke "core.clj" 11]
[gplimport.scraper.core$_main invokeStatic "core.clj" 59]
[gplimport.scraper.core$_main invoke "core.clj" 57]
[clojure.lang.Var invoke "Var.java" 380]
[user$eval477 invokeStatic "form-init13003331923583710995.clj" 1]
[user$eval477 invoke "form-init13003331923583710995.clj" 1]
[clojure.lang.Compiler eval "Compiler.java" 7194]
[clojure.lang.Compiler eval "Compiler.java" 7184]
[clojure.lang.Compiler load "Compiler.java" 7653]
[clojure.lang.Compiler loadFile "Compiler.java" 7591]
[clojure.main$load_script invokeStatic "main.clj" 475]
[clojure.main$init_opt invokeStatic "main.clj" 477]
[clojure.main$init_opt invoke "main.clj" 477]
[clojure.main$initialize invokeStatic "main.clj" 508]
[clojure.main$null_opt invokeStatic "main.clj" 542]
[clojure.main$null_opt invoke "main.clj" 539]
[clojure.main$main invokeStatic "main.clj" 664]
[clojure.main$main doInvoke "main.clj" 616]
[clojure.lang.RestFn applyTo "RestFn.java" 137]
[clojure.lang.Var applyTo "Var.java" 705]
[clojure.main main "main.java" 40]],
:cause "WebDriver process exited unexpectedly with a value: 1",
:data {},
:phase :execution}}

lread03:12:25

(I've just recently switch to Linux from macOS and am trying out Pop!_OS, I quite like it!) Glad to hear that :path-browser got you past your Firefox launch issue. One thing to remember is that Etaoin is a very thin wrapper around the WebDriver implementations. When things get odd, I'll often do a general search on the web. Sometimes it helps. If you find a solution on your own, please do report back here, so others can benefit. If you are still stuck, I am happy to continue to listen, but without more details and something I can reproduce locally on my dev box, it will be hard for me to help you here.

lucian30306:12:43

me too, a few months ago. for development (clojure, c++, php, js) pop is actually better, imo, since i'm working in the target env and things are generally smoother and simpler so yeah, it's got to be a webdriver/chromium issues at this point. i'm actually pretty close. i increased the window size using the :size option and made some mods to the code (some popup was appearing sometimes unexpectedly) and it works w/ chromium headless on pop (my local) just not ubuntu (prod). i will try to ask in the webdriver forum. i didn't figure out why ff works on pop and not ubuntu yet, but i'm hoping to fix this element not clickable and ideally keep in all in chromium @UE21H2HHD

lucian30308:12:56

i think it's finally working on my prod machine too. i was setting size wrong in the prod code. so i think that was the main thing, setting the :size of the headless chrome. the firefox issue i'm still looking into but it's not a big deal atm

lread14:12:52

Cool, glad you are on a path that is working. So were you clicking on a specific x y coordinate and missing?

lucian30318:12:10

no, i was using a query but there was an overlay

👍 1