Fork me on GitHub
#clojure-dev
<
2023-08-28
>
Noah Bogart19:08:47

i've been reading the code for defprotocol and defrecord and extend recently, and I'm unsure how they intersect. defprotocol generates a java interface and dynamically loads it (in genclass.clj). defrecord generates an interface as well using deftype*, which is handled by Compiler.java. I only sort of follow what's happening, but it seems like the compiled record interface has defined methods for all of the implemented protocols. And then extend just adds the implemented methods to the method cache on the protocol's var. how do these line up with each other?

2
hiredman19:08:52

Protocols are a composite of different satisfaction strategies

hiredman19:08:34

A given object can satisfy a protocol by implementing an interface, by having the protocol extended to some super type of the object, or by adding some special metadata to an object

hiredman19:08:49

Deftype implementing an interface is the interface route

hiredman19:08:14

Extend adding to the var is the extending to a super type route

hiredman19:08:55

The metadata route you'll see if you look at how protocol callsites are compiled (and protocols have to opts into supporting)

hiredman19:08:52

The metadata route is the most dynamic, the interface route has the best performance

Noah Bogart19:08:56

that makes sense. thanks. when compiling a protocol method, is it purely the compiler that determines which route to build?

hiredman19:08:06

The compiler generates a callsite that potentially tries all 3

hiredman19:08:52

With an inline cache to try and avoid having to do that dispatch every time

hiredman19:08:30

I am not looking at it right now, so I don't recall if the inline cache covers all the cases, or just the extend case, and the jvm is expected to make the instance check for the interface fast

Noah Bogart19:08:52

this is InvokeExpr, right? i think i'm looking at the right spot

hiredman19:08:43

I believe there is a distinct Expr type for protocol callsites

Noah Bogart19:08:15

InvokeExpr has an emitProto, I haven't seen a different one but this is a really big file and there's plenty to miss

hiredman19:08:41

Yeah, I could be mistaken

Noah Bogart19:08:54

no worries. this is really helpful, thank you

hiredman19:08:06

Yeah, I think I am muddling how the cache for keyword callsites work with the protocol sites

👍 2
hiredman20:08:51

https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Compiler.java#L3750-L3755 is the calling a protocol function (what does the extends dispatch logic)

👍 2
hiredman20:08:05

the metadata bits must be tucked away inside the functions together with the extends stuff

👍 2
Noah Bogart20:08:28

here's my understanding leading up to that: emit-protocol defs (interns) the symbol names of the protocol methods with maps, including {:protocol (var ~name)} (the protocol itself). at construction time, InvokeExpr looks for that metadata and looks at the protocol's :on to see if it's an actual Class. If it is, grab the :method-map which is a map of the method names as keywords to the :on class, get the class instance using the protocol method from the map, and store it on the InvokeExpr object.

Noah Bogart20:08:02

then in emitProto as you noted, if the protocol actually implements the interface it emits an interface call, and if it doesn't it emits a "normal" clojure function call to the protocol var, which is the function produced by emit-method-builder , where the method cache stuff happens

Noah Bogart20:08:23

hell yeah, thank you for helping out, i don't think I would have put together how emitProto works without your help