Fork me on GitHub
#clojure-dev
<
2020-07-24
>
seancorfield16:07:03

This one's pretty interesting https://ask.clojure.org/index.php/9499/inconsistent-unmatching-primitive-recur-arg -- I was trying to help @kenny with that last night and it seems that under some situations with inline functions, the compiler does something that loses the primitive return type and instead uses a promoted type (effectively Object).

hiredman16:07:28

the compiler never actually knows the return type

hiredman16:07:19

what happens is if the function is inlined, there is no function call, just the static method call which the compiler can see the type of

hiredman16:07:13

if the function isn't inlined, there is just the function call, which the compiler doesn't know the return type of

seancorfield17:07:08

@hiredman This situation is a bit more complex tho'. It knows the boolean primitive type of the local but for (boolean x) it thinks that returns boolean only in some situations...

seancorfield17:07:11

So having figured out x (pos? y) should be treated as a boolean primitive, when you recur with (boolean 1) it fails the primitive type check in the compiler, but with (boolean nil) it accepts it.

seancorfield17:07:41

Which means the compiler is doing some amount of "pre-evaluation" in there.

hiredman17:07:58

it is hard to say exactly, because the the compilation is failing so you can't check the bytecode

hiredman17:07:29

but for some reason the compiler is deciding not to inline the call to boolean on (boolean 1) which is why you aren't getting return type information

seancorfield17:07:38

user=> (loop [x (boolean (first []))] (when x (recur true)))
Syntax error (IllegalArgumentException) compiling fn* at (REPL:25:1).
 recur arg for primitive local: x is not matching primitive, had: java.lang.Boolean, needed: boolean
user=> (loop [x (boolean (first []))] (when x (recur (boolean true))))
nil
user=> (loop [x (boolean (first []))] (when x (recur (boolean 1))))
Syntax error (IllegalArgumentException) compiling fn* at (REPL:27:1).
 recur arg for primitive local: x is not matching primitive, had: Object, needed: boolean
user=> (loop [x (boolean (first []))] (when x (recur (boolean nil))))
nil
user=> (loop [x (boolean (first []))] (when x (recur (boolean (first [])))))
nil
user=> 

seancorfield17:07:05

It's not fooled by this either:

user=> (loop [x (boolean (first []))] (when x (recur (boolean (long 1)))))
Syntax error (IllegalArgumentException) compiling fn* at (REPL:30:1).
 recur arg for primitive local: x is not matching primitive, had: Object, needed: boolean
user=> 

seancorfield17:07:42

But

user=> (loop [x (boolean (first []))] (when x (recur (boolean (+ 1)))))
nil
user=> 
So I guess it is doing some literal expression "pre-evaluation" and not others here?

hiredman17:07:45

I am not sure, I don't recall the specifics of inlining, but I don't think it is any kind of pre-evaluation

hiredman17:07:17

it is the reflection

hiredman17:07:27

(which you don't get a warning about)

mikerod17:07:18

This lack of warning was a source of a large perf bottleneck in our codebase the other day. Hah. Was with alength

devn15:07:29

interesting

devn15:07:41

how did the bottleneck present itself?

devn15:07:13

what did the code look like roughly? would like to bank it in case i run into something like this

mikerod16:07:59

@U06DQC6MA When something was taking a long time, I did some CPU sampling with a profiler (visualvm here) and saw a lot of reflection being used. When looking at the stack for that reflection it was coming out of alength. Granted, this isn’t always a problem - this was an actual hot spot and it was being called a lot across a lot of data.

mikerod16:07:11

I’m just saying, it was easy to have reflection here, that had no warnings

mikerod16:07:29

Consider:

(fn [xs] (map alength xs)) ;;= no warnings
(fn [xs] (map #(alength %) xs)) ;;= now has warnings

mikerod16:07:32

So the inline stuff is subtle

devn03:07:06

Thanks for sharing that. It is indeed subtle. I think a long time ago I profiled something like this and tinkered with it enough to fix the performance issue by accident, but did not recognize a hidden reflection issue.

👍 1
hiredman17:07:44

the static method call that boolean is inlined to is overloaded for Object and boolean arguments, but doesn't have a method for longs

hiredman17:07:40

(boolean (+ 1)) gets the object overload, (boolean 1) looks for a method that takes a long and doesn't find it and does something different (not sure exactly what)

hiredman17:07:40

Clojure 1.10.1
user=> (set! *warn-on-reflection* true)
true
user=> (loop [x (boolean (first []))] (when x (recur (boolean 1))))
Reflection warning, NO_SOURCE_PATH:1:47 - call to static method booleanCast on clojure.lang.RT can't be resolved (argument types: long).
Syntax error (IllegalArgumentException) compiling fn* at (REPL:1:1).
 recur arg for primitive local: x is not matching primitive, had: Object, needed: boolean
user=> (loop [x (boolean (first []))] (when x (recur (boolean (+ 1)))))
nil
user=> 

seancorfield17:07:00

Ah! That makes more sense than what I was thinking was happening (that the compiler was a lot smarter about inlining than it clearly is).

seancorfield17:07:48

I was imagining it was doing the sort of smart code inlining that I had worked on back in the '80s and '90s when I wrote compilers and virtual machine runtimes for a living 🙂