Fork me on GitHub
#clojure-dev
<
2019-07-19
>
souenzzo01:07:36

After read tons of clojure source, I still can't understand this behavior.

(= ##NaN ##NaN)
=> false
(not= ##NaN ##NaN)
=> false
(apply = [##NaN ##NaN])
=> false
(apply not= [##NaN ##NaN])
=> true

andy.fingerhut01:07:19

You are curious what in the Clojure/Java implementation causes that to happen?

andy.fingerhut01:07:38

Hmm, now that you ask, I don't know if I have noticed that (not= ##NaN ##NaN) returns false before. That is pretty weird.

Alex Miller (Clojure team)01:07:39

it's the weird thing about ##NaN that causes all the problems :)

Alex Miller (Clojure team)01:07:37

##NaN is identical but not equals, which is very rare in Clojure

Alex Miller (Clojure team)01:07:47

user=> (identical? ##NaN ##NaN)
true
user=> (= ##NaN ##NaN)
false

andy.fingerhut01:07:05

But does not= make sense to you? I am curious enough to dig for a few minutes and find out, but ... weird

andy.fingerhut01:07:07

It is probably switching between primitive double vs. Object equiv methods in clojure.lang.Util there, is my best guess before experiments confirm

Alex Miller (Clojure team)01:07:26

does seem weird that the not= and apply not= are different

Alex Miller (Clojure team)01:07:13

user=> (apply not= [##NaN ##NaN])
true
user=> (apply not= ##NaN [##NaN])
true
user=> (apply not= ##NaN ##NaN [])
false

Alex Miller (Clojure team)01:07:49

maybe something weird with the rest seq

Alex Miller (Clojure team)01:07:30

or inlining? = is inlined, not= is not

andy.fingerhut01:07:35

user=> (#'= ##NaN ##NaN)
true
user=> (= ##NaN ##NaN)
equiv(double NaN, double NaN) returning false
false

andy.fingerhut01:07:56

That equiv(...) ... line is extra debug print in my local Clojure Java code.

Alex Miller (Clojure team)01:07:18

if you go through the var, probably not getting the inlining

Alex Miller (Clojure team)01:07:54

prob need to compare the compiled bytecode

andy.fingerhut01:07:45

I can make a table for Clojure 1.10.1 of the cases and stick them up on a page somewhere, for the morbidly curious

andy.fingerhut01:07:23

Obviously the person creating such a page is among the morbid 🙂

Alex Miller (Clojure team)01:07:46

oh, maybe primitive long vs boxed long?

Alex Miller (Clojure team)01:07:12

the inlined will probably be primitives whereas non-inlined is boxed

andy.fingerhut01:07:16

I think (#'= ##NaN ##NaN) and (not= ##NaN ##NaN) go through equiv(Object, Object) method

Alex Miller (Clojure team)01:07:13

thankfully jshell to the rescue...

Alex Miller (Clojure team)01:07:18

jshell> Double.NaN == Double.NaN
$2 ==> false

jshell> Double.valueOf(Double.NaN).equals(Double.valueOf(Double.NaN))
$3 ==> true

Alex Miller (Clojure team)01:07:34

prims are false, boxed are true

Alex Miller (Clojure team)02:07:33

pretty likely there is some place that's not handled right

andy.fingerhut02:07:29

I think the equality guide's recommendation of avoiding NaN's is unlikely to change as a result of any of this 🙂

Alex Miller (Clojure team)02:07:49

same issue as -0.0 and 0.0 iirc

Alex Miller (Clojure team)02:07:28

jshell> -0.0 == 0.0
$5 ==> true

jshell> Double.valueOf(-0.0).equals(Double.valueOf(0.0))
$6 ==> false

Alex Miller (Clojure team)02:07:52

"This definition allows hash tables to operate properly." orly?

gfredericks17:07:45

does negative zero equal positive zero in some other context?

andy.fingerhut17:07:29

I think Java == on primitive doubles is defined to return true in that case?

andy.fingerhut17:07:41

because IEEE 754 requires it, perhaps?

andy.fingerhut17:07:16

And here is a mention in the Java language spec that says IEEE 754 requires +0.0 == -0.0 to return true: https://docs.oracle.com/javase/specs/jls/se7/html/jls-15.html#jls-15.21.1

andy.fingerhut18:07:11

clojure.core/= and clojure.core/== return true when comparing +0.0 and -0.0 in the latest Clojure (that was a change in the last few years as a response to a filed issue)

andy.fingerhut18:07:15

Not having done so, I have the impression that reading the IEEE 754 spec, and rationale if one exists, would lead one to believe that there were many heated discussions during its creation process.

Alex Miller (Clojure team)18:07:44

IEEE 754 is trying to solve many use cases, some where +/- 0 is meaningfully, some where it's not. unsurprisingly, there are tradeoffs. :)

andy.fingerhut18:07:18

Oh, I'm immensely glad the standard was created. I can easily imagine the frustrated state of numerical software developers before it existed, trying to port software.

Alex Miller (Clojure team)18:07:23

as I'm sure you are both well aware, most things are weirder when examined closely :)

andy.fingerhut18:07:29

Recently read the first chapter of the book "Microbe Hunters" describing early days of Leeuwenhoek making the best microscopes of his day and spending most of his life looking at all sorts of things. Excellent book so far.

Alex Miller (Clojure team)18:07:17

was he the guy that stabbed himself in the eye with a needle to figure out how light worked? or was that someone else

Alex Miller (Clojure team)18:07:27

also, don't ever google that

andy.fingerhut18:07:28

That is a level of scientific curiosity I don't think I will personally reach.

gfredericks23:07:50

it's also unclear to me what theories you rule out after doing that

andy.fingerhut02:07:41

I guess I can create a ticket at least listing the current behavior and the root causes. Personally I'm not expecting high priority bugs out of those results, but that hasn't stopped me from creating tickets before 🙂

andy.fingerhut02:07:23

I think this should go into a Github repository named Batman -- hat tip to Gary Bernhardt's video https://www.destroyallsoftware.com/talks/wat

potetm03:07:15

(not= ##NaN
      ##NaN)
=> false
(not= Double/NaN
      Double/NaN)
=> true

andy.fingerhut03:07:20

Thanks for the examples. I will be adding them to my list of expressions to explain their results.

andy.fingerhut03:07:30

will send link to the results here when they are published.

andy.fingerhut07:07:23

OK, that might not have been the best way to spend that many hours of my life, but here are the results of a bunch of Clojure expressions involving NaN, most with explanations for why they return the result they do, based upon adding some debug print statements to Clojure 1.10.1 source code and then evaluating the expressions: https://github.com/jafingerhut/batman

💯 4
andy.fingerhut07:07:52

Sample output of running the command in the doc directory, in case you just want to look through them: https://github.com/jafingerhut/batman/blob/master/doc/ubuntu-18.04.2-jdk-openjdk-11.0.3-clojure-1.10.1.txt

andy.fingerhut15:07:51

@alexmiller It is not clear to me from that fun collection of results about = and not= on various flavors of NaN, called in different ways, ought to do. Consistently return false when comparing NaN's for =, and consistently return true when comparing them for not=? I guess that could be a semi-reasonable desire.

andy.fingerhut17:07:51

OK, PR created for the few NaN weird cases found: https://clojure.atlassian.net/browse/CLJ-2526

andy.fingerhut20:07:47

OK, also created a ticket with the potential speed improvement for identical keys in array-maps: https://clojure.atlassian.net/browse/CLJ-2527 The improvement could affect anything that uses EquivPred -- I will do a quick check of the code to see whether it is used for anything other than array-map key lookup.