2023-06-13 etaoin | Clojure Slack Archive

etaoin

anonimitoraf 2023-06-13T22:45:21.518499Z

Hi folks, just wondering what it would take to support https://github.com/ultrafunkamsterdam/undetected-chromedriver? Main goal is to write webscrapers that can avoid triggering anti-bot mechanisms

2023-06-26T13:34:47.857889Z

@nicdaoraf A bit late but I've had some initial luck using etaoin undetected with he following settings:

(e/chrome 
    {:args         ["--disable-blink-features=AutomationControlled"]
     :capabilities {:chromeOptions {:args            ["--disable-blink-features=AutomationControlled"]
                                    :excludeSwitches ["enable-automation"]}}})

not sure if all those are necessary or doing anything, or if this is sufficiently undetected, but it seemed to let me do some stuff I couldn't before. Ultimately I ended up using the Chrome DevTools protocol instead of webdriver to control a "normal"/non-webdriver controlled browser (started with --remote-debugging-port=.... https://chromedevtools.github.io/devtools-protocol/ which seems to offer more capabilities than webdriver and might be worth checking out depending on what you're doing

👀 1

anonimitoraf 2023-06-14T09:52:42.619449Z

Hey @lee, sorry for my poorly worded question, I’ve not tried anything yet. I guess the biggest challenge is that all the documentation/tutorials/etc seem to require python . I’ll see if I can find a way to convert it to an executable, and hopefully it’s just a matter of setting the driver path

lread 2023-06-14T14:06:23.409139Z

No prob @nicdaoraf, let us know if you find something that works for you. Admittedly I only glanced at them, but I didn't find the docs for undetected-chromedriver particularly clear. Not sure exactly what it does or why or how. Out of the box, etaoin https://cljdoc.org/d/etaoin/etaoin/1.0.40/doc/user-guide#user-agent. Not sure if that helps you at all.

lread 2023-06-14T03:08:14.804449Z

Hi @nicdaoraf, dunno, never heard of it! But if it is just a chromedriver drop-in replacement, might just-work? What have you tried?

Clojurians Log v2

etaoin