Fork me on GitHub
#off-topic
<
2023-03-01
>
Gerome15:03:26

I want to recursively walk down a tree structure in ClojureScript and while I was writing this loop-recur, CoPilot suddenly showed loading for a minute or two and spat out this construct. :D

๐Ÿ˜‚ 7
โœ… 1
Gerome15:03:21

I just accepted the suggestion and it randomly stops at some point and then I can continue this as long as I want.

Gerome15:03:20

Haha, it now shows this error icon and when I hover over it, it says "CoPilot is encountering temporary issues (click for details)" and if I click, nothing happens. I think, I might have broke sumfin.

dgb2315:03:51

I guess copilot isn't optimized for languages that are sufficiently expressive...

skylize20:03:42

Anyone who's attempted building an "unofficial" 3rd party API have any good tips for working out what's happening on a webpage, besides looking at the HTTP requests?

p-himik21:03:19

> besides looking at the HTTP requests The only thing besides the requests is what drives them. Usually it's user actions, sometimes it's other code (`setTimeout` or some other non-user communication).

Rupert (All Street)22:03:38

Built in browser Dev tools give a pretty good picture (A) network tab shows reqs/response (B) script tab can show pretty printed code and let you add break points (C) Inspect tab lets you see the DOM after manipulation by Javascript (D) application tab lets you see what's in cookies and local storage. When doing an unoffial API - I expect a programmer to be spending a long time looking at how it works (often this takes more time than the coding part).

๐Ÿ‘ 1
skylize22:03:10

> (B) script tab can show pretty printed code and let you add break points > How would you go about looking for somewhere to put breakpoints in almost-certainly-minified, likely-otherwise-obfuscated 3rd party code?

skylize22:03:22

Err, I guess the first part is partially answered already, that pretty printing is on offer. But still, you've got code that's been crammed together, with build artifacts scattered about, rather than a nice collection of organized modules. Then a good chance of obfuscation, whether intentional or from optimization. How do you find a starting point to start digging from in such code?

jjttjj22:03:06

depends what you're doing and you might already know this, but oftentimes it helps for this unofficial stuff to start with a webdriver, ie https://github.com/clj-commons/etaoin rather than http requests

Rupert (All Street)23:03:59

> certainly-minified, likely-otherwise-obfuscated 3rd party code Not all websites are obfuscated, but when they are it can be painstaking work. You can back track from key events (e.g. HTTP requests, click handlers etc). You can also sometimes find strings on the page and look for them in the code. Sometimes it not worth looking at the code at all. e.g. just letting the code run (e.g. in a headless browswer) and interacting with the DOM or emulating a user actions (e.g. emulating click events).

๐Ÿ‘ 1
skylize23:03:03

> oftentimes it helps for this unofficial stuff to start with a webdriver ... rather than http requests > Could you clarify what you mean by this? I don't quite follow

Rupert (All Street)23:03:00

I think they mean - don't look at the requests. Just let the page operate with javascript enabled trapped inside a headless browser (webdriver) then interact with the page.

๐Ÿ‘† 1
skylize04:03:02

etaoin does look pretty cool. Any thoughts on using it for discovery of functionality that can then be used without necessarily having a webdriver session?

Gerome11:03:29

ChatGPT can help de-obfuscate code. As with all things you put through that box though, take it with a pinch of salt. https://news.ycombinator.com/item?id=34503233

Rupert (All Street)11:03:12

There is a warning in that comment thread > Misleading, ChatGPT generated both obfuscated and de-obfuscated code It's less surprising that it can de-obfuscate what it obfuscated- it's possible it literally had the answer in it chat history that it could use.

Rupert (All Street)11:03:08

Also obfuscated javascript can be megabytes in size (and definitions are scattered order) - there's no way to paste in that much into ChatGPT.

Rupert (All Street)11:03:07

Appreciate that you did say take with a pinch of salt.

Gerome11:03:48

I wouldn't paste megabytes of code into chatGPT unless I want to understand and read megabytes of code. What I had in mind was to take maybe one function and then use the output as a starting point. I just tried it with a random piece of code from slack. It did prettify and spell out what each line does but that was not super helpful. Thanks @UJVEQPAKS for pointing this out. I will practice some more due diligence next time.

Gerome11:03:27

It asked for more context, though. If there is some more context e.g. a response arrived or something, this might become more useful.

Rupert (All Street)11:03:58

I think your original due dil was good and your post had an appropriate disclaimer (pinch of salt) too. So there's absolutely no criticism from me. ๐Ÿ˜€ Thanks for the extra info. Agree there may be approaches to integrate ChatGPT into the deobfuscation without trying to give it the whole javascript file.

dgb2323:03:56

Something that has always slightly bothered/confused me but I haven't quite understood is why I tend not not use protocols (or interfaces, or prototypes etc.) that much. I had the nagging feeling that don't really get them, because I rarely reach for that general mechanism in any language I use. I noticed that when do I use them, they tend to be in code that coordinates or stitches together stuff, typically stateful things as well. So really at the most upper layer. But otherwise I just dispatch on some concrete data, or use (conceptual) jump tables (I think router libs like reitit do this?) and so on. I'm still not sure if I'm missing out or not. When I reach for interfaces/protocols etc. It's usually very obvious to me that I should. There's usually a set of functions that I want to call, while I don't care about what happens behind it. Or in more rare cases it's that I "configured" a function in advance (closure/partial application etc.). But especially in pure code that rarely seems to happen?

kenj23:03:54

I think the most powerful/oft-used point of interfaces is hiding conditionals with polymorphism. If your code doesn't have a need for that, then I could see interfaces being rather not helpful.

๐Ÿ‘ 2
dpsutton23:03:40

i look for tasteful uses of them. Reading and think about a non-protocol design

๐Ÿ‘ 2
respatialized23:03:08

I think it's also important if you want to make a library extensible by users - ordinary conditionals are fine for internals but can't be modified or extended, so if you're thinking about any kind of system that implements its own types or custom data structures protocols can help other users get their data into your system's format. Granted the idioms around just using plain data structures as a lingua franca between subsystems in Clojure are quite strong so there's less of a need for dynamic type-based dispatch than in other languages that emphasize classes but there are definitely contexts where you can't quite do that. One application where protocols seemed to "click" for me recently was in the context of some GUI & graphics programming - I wanted an extensible way to get shapes on a canvas that wasn't limited to a subset of prespecified types of shape and multimethods fit very nicely.

๐Ÿ‘ 2
phronmophobic23:03:06

Empirically, theyโ€™re not used that often, https://blog.phronemophobic.com/dewey-analysis.html#Var-Definitions-and-Macros , slightly more than macros. They have their uses, but I also think theyโ€™re part of the story for convincing OO devs to try clojure. Look! Clojure has polymorphism (even if you end up not needing it as much in practice).

๐Ÿ‘€ 2
p-himik23:03:17

Maybe this can be of some help (although somewhat outdated because doesn't list protocol impl via metadata): https://github.com/plumatic/eng-practices/blob/master/clojure/20130926-data-representation.md

๐Ÿ™ 4
โค๏ธ 2
dgb2310:03:06

Thanks to all the tips and responses! @UFTRLDZEW : > One application where protocols seemed to "click" for me recently was in the context of some GUI & graphics programming - I wanted an extensible way to get shapes on a canvas that wasn't limited to a subset of prespecified types of shape and multimethods fit very nicely. For me multimethods are a in-between thing! I typically have a dispatch function like this: (:events/type event)` Or maybe some composite thing. But conceptually it is still data driven and not type driven dispatch right? I think of multimethods as a kind of hybrid between a manual jump table (map of keys to functions) that I can mutate with defmethod and a slow version of cond if that makes sense.

dgb2311:03:53

My mental model for how for example the #C7YF1SBT3 router works is a mapping like this: route string -> value Or as a ring/router: route string -> function Where "->" is one of the routing implementations (linear, tree etc.) that is chosen for me. This is entirely generic in my eyes and very extensible as I can just ask for a piece of data in order to add/change routes. --- I'll look at the links and tips in your comments and think about it a bit more! Ty

p-himik11:03:41

Yeah, I use them all the time when I don't need something to be extendable from elsewhere.

๐Ÿ‘ 2
dgb2311:03:10

apparently that's how multimethods work too?

(defmulti table identity)
(defmethod table :on-connect [_] (constantly :connected))
(defmethod table :on-text [_] (fn [message] message))
(methods table)
=>
{:on-connect #function[user/eval7627/fn--7628], :on-text #function[user/eval7631/fn--7632]}

p-himik11:03:06

Almost - the registry is global and mutable and you can use derive to establish a hierarchy of dispatch values.

๐Ÿ‘ 2