etaoin 2023-06-13 | Slack Archive

anonimitoraf22:06:21

Hi folks, just wondering what it would take to support https://github.com/ultrafunkamsterdam/undetected-chromedriver? Main goal is to write webscrapers that can avoid triggering anti-bot mechanisms

lread03:06:14

Hi @UR37CBF8D, dunno, never heard of it! But if it is just a chromedriver drop-in replacement, might just-work? What have you tried?

anonimitoraf09:06:42

Hey @UE21H2HHD, sorry for my poorly worded question, I’ve not tried anything yet. I guess the biggest challenge is that all the documentation/tutorials/etc seem to require python . I’ll see if I can find a way to convert it to an executable, and hopefully it’s just a matter of setting the driver path

lread14:06:23

No prob @UR37CBF8D, let us know if you find something that works for you. Admittedly I only glanced at them, but I didn't find the docs for undetected-chromedriver particularly clear. Not sure exactly what it does or why or how. Out of the box, etaoin https://cljdoc.org/d/etaoin/etaoin/1.0.40/doc/user-guide#user-agent. Not sure if that helps you at all.

jjttjj13:06:47

@UR37CBF8D A bit late but I've had some initial luck using etaoin undetected with he following settings:

(e/chrome 
    {:args         ["--disable-blink-features=AutomationControlled"]
     :capabilities {:chromeOptions {:args            ["--disable-blink-features=AutomationControlled"]
                                    :excludeSwitches ["enable-automation"]}}})

not sure if all those are necessary or doing anything, or if this is sufficiently undetected, but it seemed to let me do some stuff I couldn't before. Ultimately I ended up using the Chrome DevTools protocol instead of webdriver to control a "normal"/non-webdriver controlled browser (started with --remote-debugging-port=.... https://chromedevtools.github.io/devtools-protocol/ which seems to offer more capabilities than webdriver and might be worth checking out depending on what you're doing

👀 2

2023-06-13

Channels