Fork me on GitHub
#clojure-dev
<
2019-07-19
>
souenzzo01:07:36

After read tons of clojure source, I still can't understand this behavior.

(= ##NaN ##NaN)
=> false
(not= ##NaN ##NaN)
=> false
(apply = [##NaN ##NaN])
=> false
(apply not= [##NaN ##NaN])
=> true

andy.fingerhut01:07:19

You are curious what in the Clojure/Java implementation causes that to happen?

andy.fingerhut01:07:38

Hmm, now that you ask, I don't know if I have noticed that (not= ##NaN ##NaN) returns false before. That is pretty weird.

alexmiller01:07:39

it's the weird thing about ##NaN that causes all the problems :)

alexmiller01:07:37

##NaN is identical but not equals, which is very rare in Clojure

alexmiller01:07:47

user=> (identical? ##NaN ##NaN)
true
user=> (= ##NaN ##NaN)
false

andy.fingerhut01:07:05

But does not= make sense to you? I am curious enough to dig for a few minutes and find out, but ... weird

andy.fingerhut01:07:07

It is probably switching between primitive double vs. Object equiv methods in clojure.lang.Util there, is my best guess before experiments confirm

alexmiller01:07:26

does seem weird that the not= and apply not= are different

alexmiller01:07:13

user=> (apply not= [##NaN ##NaN])
true
user=> (apply not= ##NaN [##NaN])
true
user=> (apply not= ##NaN ##NaN [])
false

alexmiller01:07:49

maybe something weird with the rest seq

alexmiller01:07:30

or inlining? = is inlined, not= is not

andy.fingerhut01:07:35

user=> (#'= ##NaN ##NaN)
true
user=> (= ##NaN ##NaN)
equiv(double NaN, double NaN) returning false
false

andy.fingerhut01:07:56

That equiv(...) ... line is extra debug print in my local Clojure Java code.

alexmiller01:07:18

if you go through the var, probably not getting the inlining

alexmiller01:07:54

prob need to compare the compiled bytecode

andy.fingerhut01:07:45

I can make a table for Clojure 1.10.1 of the cases and stick them up on a page somewhere, for the morbidly curious

andy.fingerhut01:07:23

Obviously the person creating such a page is among the morbid 🙂

alexmiller01:07:46

oh, maybe primitive long vs boxed long?

alexmiller01:07:12

the inlined will probably be primitives whereas non-inlined is boxed

andy.fingerhut01:07:16

I think (#'= ##NaN ##NaN) and (not= ##NaN ##NaN) go through equiv(Object, Object) method

alexmiller01:07:20

same for apply

alexmiller01:07:22

or sorry, doubles

alexmiller01:07:13

thankfully jshell to the rescue...

alexmiller01:07:18

jshell> Double.NaN == Double.NaN
$2 ==> false

jshell> Double.valueOf(Double.NaN).equals(Double.valueOf(Double.NaN))
$3 ==> true

alexmiller01:07:34

prims are false, boxed are true

alexmiller02:07:33

pretty likely there is some place that's not handled right

andy.fingerhut02:07:29

I think the equality guide's recommendation of avoiding NaN's is unlikely to change as a result of any of this 🙂

alexmiller02:07:49

same issue as -0.0 and 0.0 iirc

alexmiller02:07:28

jshell> -0.0 == 0.0
$5 ==> true

jshell> Double.valueOf(-0.0).equals(Double.valueOf(0.0))
$6 ==> false

alexmiller02:07:52

"This definition allows hash tables to operate properly." orly?

gfredericks17:07:45

does negative zero equal positive zero in some other context?

andy.fingerhut17:07:29

I think Java == on primitive doubles is defined to return true in that case?

andy.fingerhut17:07:41

because IEEE 754 requires it, perhaps?

andy.fingerhut17:07:16

And here is a mention in the Java language spec that says IEEE 754 requires +0.0 == -0.0 to return true: https://docs.oracle.com/javase/specs/jls/se7/html/jls-15.html#jls-15.21.1

andy.fingerhut18:07:11

clojure.core/= and clojure.core/== return true when comparing +0.0 and -0.0 in the latest Clojure (that was a change in the last few years as a response to a filed issue)

andy.fingerhut18:07:15

Not having done so, I have the impression that reading the IEEE 754 spec, and rationale if one exists, would lead one to believe that there were many heated discussions during its creation process.

alexmiller18:07:44

IEEE 754 is trying to solve many use cases, some where +/- 0 is meaningfully, some where it's not. unsurprisingly, there are tradeoffs. :)

andy.fingerhut18:07:18

Oh, I'm immensely glad the standard was created. I can easily imagine the frustrated state of numerical software developers before it existed, trying to port software.

alexmiller18:07:23

as I'm sure you are both well aware, most things are weirder when examined closely :)

andy.fingerhut18:07:29

Recently read the first chapter of the book "Microbe Hunters" describing early days of Leeuwenhoek making the best microscopes of his day and spending most of his life looking at all sorts of things. Excellent book so far.

alexmiller18:07:17

was he the guy that stabbed himself in the eye with a needle to figure out how light worked? or was that someone else

alexmiller18:07:00

that was Newton, nvm

alexmiller18:07:27

also, don't ever google that

andy.fingerhut18:07:28

That is a level of scientific curiosity I don't think I will personally reach.

gfredericks23:07:50

it's also unclear to me what theories you rule out after doing that

alexmiller02:07:12

just tease us

alexmiller02:07:43

anyhow, ticket welcome

andy.fingerhut02:07:41

I guess I can create a ticket at least listing the current behavior and the root causes. Personally I'm not expecting high priority bugs out of those results, but that hasn't stopped me from creating tickets before 🙂

andy.fingerhut02:07:23

I think this should go into a Github repository named Batman -- hat tip to Gary Bernhardt's video https://www.destroyallsoftware.com/talks/wat

potetm03:07:15

(not= ##NaN
      ##NaN)
=> false
(not= Double/NaN
      Double/NaN)
=> true

andy.fingerhut03:07:20

Thanks for the examples. I will be adding them to my list of expressions to explain their results.

andy.fingerhut03:07:30

will send link to the results here when they are published.

andy.fingerhut07:07:23

OK, that might not have been the best way to spend that many hours of my life, but here are the results of a bunch of Clojure expressions involving NaN, most with explanations for why they return the result they do, based upon adding some debug print statements to Clojure 1.10.1 source code and then evaluating the expressions: https://github.com/jafingerhut/batman

💯 1
andy.fingerhut07:07:52

Sample output of running the command in the doc directory, in case you just want to look through them: https://github.com/jafingerhut/batman/blob/master/doc/ubuntu-18.04.2-jdk-openjdk-11.0.3-clojure-1.10.1.txt

andy.fingerhut15:07:51

@alexmiller It is not clear to me from that fun collection of results about = and not= on various flavors of NaN, called in different ways, ought to do. Consistently return false when comparing NaN's for =, and consistently return true when comparing them for not=? I guess that could be a semi-reasonable desire.

andy.fingerhut17:07:51

OK, PR created for the few NaN weird cases found: https://clojure.atlassian.net/browse/CLJ-2526

andy.fingerhut20:07:47

OK, also created a ticket with the potential speed improvement for identical keys in array-maps: https://clojure.atlassian.net/browse/CLJ-2527 The improvement could affect anything that uses EquivPred -- I will do a quick check of the code to see whether it is used for anything other than array-map key lookup.