Fork me on GitHub
#clojure-dev
<
2020-12-14
>
dominicm08:12:21

is it a bad idea to use the clojure compiler options in my code, something like so:

(if (:direct-linking *compiler-options*)
  handler
  #(apply handler  %&))
Thinking about the impacts this might have on hot swappable code.

borkdude08:12:04

@dominicm Is this to circumvent direct linking? In that case you can make your var ^:redef or ^:dynamic if it should be a dynamic var.

andy.fingerhut09:12:57

It is also possible that some parts of the Clojure code were compiled with direct linking enabled, and other parts were compiled with it disabled, and I am not sure whether and expression like that, even if it does appear to work, would actually "remember" which code was compiled how. clojure.core that most people use is compiled with direct linking enabled, and unless someone enables it, other Clojure code is compiled with it off.

dominicm09:12:08

@borkdude the broader context of this would be use with jetty or anywhere you pass a function for later execution. Making your ring functions reloadable without a reset.

borkdude09:12:36

The above still applies to your situation I think

dominicm10:12:18

@ben.sless yes, I'm wondering if direct linking could be used as a proxy for switching between those two routers, yes.

dominicm10:12:36

@borkdude I've not heard of redef, so I need to check that out.

borkdude10:12:08

You can probably even get away with making that metadata dynamic based on dev or prod

dominicm10:12:16

Okay, not after redef @borkdude. I'm not trying to bypass direct linking. I want to allow redefining a function in development, but not in production. But I'm really working around the fact I'm passing a function value around, rather than a var.

borkdude10:12:47

Then why did you bring up direct linking?

borkdude10:12:49

#(foo 1 2 3) will see redefinitions of foo, but (partial foo ...) won't

dominicm10:12:49

I was thinking that direct linking would be an appropriate flag to use to switch between the two.

seancorfield19:12:04

@dominicm I'm curious if you've measured the overhead of using the Var in production vs a function value? We tend to just leave the Var references in place for handlers in our apps, but we do have direct linking enabled when we AOT compile for uberjars (so, in production, we can't redef functions on the fly in general anyway).

dominicm19:12:53

@seancorfield I have, https://github.com/SevereOverfl0w/direct-performance But the difficulties slip in cases like the reitit one where compiling a router is an expensive operation.

seancorfield19:12:31

@dominicm That seems to indicate that indirect use is faster than direct use 👀 and neither of those tests use a Var reference?

hiredman19:12:48

"the difference appears insignificant."

borkdude19:12:06

And the ability to hotfix a var in production might outweigh the small perf hit :P

hiredman19:12:41

that is a quote from the repo readme

borkdude19:12:02

I understood that

hiredman19:12:20

but I should also point out, that the code in the repo is not testing direct linking, and is not testing using a var reference directly

hiredman19:12:15

so you should be very careful about using that to justify making decisions about direct linking and var reference usage

hiredman19:12:16

my understanding is directly linking is mostly a win around avoiding the volatile field which vars use to hold data

hiredman19:12:40

reading and writing a volatile is, relative to a lot of user code, very cheap

hiredman19:12:07

but I believe it limits the amount of inlining the jvm jit can do

hiredman19:12:46

so in a hot loop, direct linking can be a huge performance win, because it lets the jit do more inlining, apply more optimizations, get more opportunities to inline, and so on

borkdude19:12:47

for these cases you could maybe turn on direct linking temporarily, compile the function with hot loops and then disable it again

borkdude19:12:31

but that would get confusing if you patch a var that's used in the body of such a function

borkdude19:12:36

and elsewhere

borkdude19:12:31

you could program against this by de-refing the vars outside the loop and then using the bindings in the loop as well

borkdude19:12:05

(let [x @#'inc] (loop [] (x 2)))

borkdude19:12:26

or even without @#' this works already

borkdude19:12:43

but then you would get a lookup in the lexical context. I'm not sure how expensive that is, maybe similar to a var deref?

borkdude19:12:55

user=> (defn foo [x] (inc x))
#'user/foo
user=> (time (dotimes [i 10000000000] (foo 1)))
"Elapsed time: 6210.834273 msecs"
nil
user=> (time (let [foo foo] (dotimes [i 10000000000] (foo 1))))
"Elapsed time: 2476.871089 msecs"
nil

hiredman19:12:56

@borkdude I am not sure this is a conversation for #clojure-dev

hiredman20:12:51

I mean, I don't know, it just seems like a tutorial about how clojure compiles references to different kinds of names would be better somewhere like #clojure

hiredman20:12:21

I think of #clojure-dev to be more about the development of clojure, where that kind of thing can usually be assumed. But I don't know that I've seen a statement of purpose for #clojure-dev , so I may just be wrong

borkdude20:12:02

I was exploring a local variation of direct linking but I probably could have done that in private and not while thinking out loud. You're right.

Alex Miller (Clojure team)20:12:51

Direct linking also directly affects bytecode size and thus load time because it does need to load or store vars in fields

hiredman20:12:25

Ah of course, should have remembered that, the impact to load times is why we have it turned on at work

Ben Sless20:12:33

I have also found it decreases GC pressure