Fork me on GitHub
#clojure
<
2021-03-03
>
borkdude12:03:23

I am trying to extend transit for arrays which (surprisingly) are not supported out of the box. E.g.:

(def array-write-handler (transit/write-handler "pods/array" (fn [o] (vec o))))
(def array-read-handler (transit/read-handler (fn [o] (into-array o))))

(defn transit-json-read [^String s]
  (with-open [bais (java.io.ByteArrayInputStream. (.getBytes s "UTF-8"))]
    (let [r (transit/reader bais :json
                            {:handlers {"pods/array" array-read-handler}})]
      (transit/read r))))

(def array-type (type (into-array [])))
(def long-array-type (type (into-array Long [])))

(defn transit-json-write [^String s]
  (with-open [baos (java.io.ByteArrayOutputStream. 4096)]
    (let [w (transit/writer baos
                            :json
                            {:handlers {long-array-type array-write-handler
                                        array-type array-write-handler}})]
      (transit/write w s)
      (str baos))))
The problem here is that there is no generic array type on which :handler can dispatch? Or is there?

borkdude13:03:45

I have trouble believing nobody has ever run into this before

borkdude13:03:35

I think I'll just use a postwalk instead...

dnolen17:03:22

anybody figured out workarounds for git deps and Apple M1?

ghadi17:03:59

@dnolen what's the problem needing working around?

dnolen18:03:14

git deps fail

ghadi18:03:30

a repro with any error messages would be helpful

ghadi18:03:30

I realize the error messages coming from that subsystem are not very useful, but e.g. is it just ssh: protocol, or https: too? is an ssh-agent active?

dnolen18:03:33

Error building classpath. /private/var/folders/q4/3d296lg10qd4nt3sxxl98fmw0000gn/T/jna--541193536/jna3022155084463275496.tmp: dlopen(/private/var/folders/q4/3d296lg10qd4nt3sxxl98fmw0000gn/T/jna--541193536/jna3022155084463275496.tmp, 1): no suitable image found.  Did find:
	/private/var/folders/q4/3d296lg10qd4nt3sxxl98fmw0000gn/T/jna--541193536/jna3022155084463275496.tmp: no matching architecture in universal wrapper
	/private/var/folders/q4/3d296lg10qd4nt3sxxl98fmw0000gn/T/jna--541193536/jna3022155084463275496.tmp: no matching architecture in universal wrapper
java.lang.UnsatisfiedLinkError: /private/var/folders/q4/3d296lg10qd4nt3sxxl98fmw0000gn/T/jna--541193536/jna3022155084463275496.tmp: dlopen(/private/var/folders/q4/3d296lg10qd4nt3sxxl98fmw0000gn/T/jna--541193536/jna3022155084463275496.tmp, 1): no suitable image found.  Did find:
	/private/var/folders/q4/3d296lg10qd4nt3sxxl98fmw0000gn/T/jna--541193536/jna3022155084463275496.tmp: no matching architecture in universal wrapper
	/private/var/folders/q4/3d296lg10qd4nt3sxxl98fmw0000gn/T/jna--541193536/jna3022155084463275496.tmp: no matching architecture in universal wrapper
	at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
	at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:383)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:227)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:169)
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2383)
	at java.base/java.lang.Runtime.load0(Runtime.java:746)
	at java.base/java.lang.System.load(System.java:1857)
	at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:761)
	at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:736)
	at com.sun.jna.Native.<clinit>(Native.java:131)
	at com.jcraft.jsch.agentproxy.usocket.JNAUSocketFactory$CLibrary.<clinit>(JNAUSocketFactory.java:47)
	at com.jcraft.jsch.agentproxy.usocket.JNAUSocketFactory.open(JNAUSocketFactory.java:114)
	at com.jcraft.jsch.agentproxy.connector.SSHAgentConnector.open(SSHAgentConnector.java:93)
	at com.jcraft.jsch.agentproxy.connector.SSHAgentConnector.<init>(SSHAgentConnector.java:54)
	at com.jcraft.jsch.agentproxy.ConnectorFactory.createConnector(ConnectorFactory.java:104)
	at clojure.tools.gitlibs.impl$fn__1257.invokeStatic(impl.clj:31)
	at clojure.tools.gitlibs.impl$fn__1257.invoke(impl.clj:29)
	at clojure.lang.Delay.deref(Delay.java:42)
	at clojure.core$deref.invokeStatic(core.clj:2324)
	at clojure.core$deref.invoke(core.clj:2310)
	at clojure.tools.gitlibs.impl$call_with_auth.invokeStatic(impl.clj:51)
	at clojure.tools.gitlibs.impl$call_with_auth.invoke(impl.clj:43)
	at clojure.tools.gitlibs.impl$git_clone_bare.invokeStatic(impl.clj:73)
	at clojure.tools.gitlibs.impl$git_clone_bare.invoke(impl.clj:70)
	at clojure.tools.gitlibs.impl$ensure_git_dir.invokeStatic(impl.clj:112)
	at clojure.tools.gitlibs.impl$ensure_git_dir.invoke(impl.clj:102)
	at clojure.tools.gitlibs$resolve.invokeStatic(gitlibs.clj:33)
	at clojure.tools.gitlibs$resolve.invoke(gitlibs.clj:29)
	at clojure.tools.gitlibs$procure.invokeStatic(gitlibs.clj:47)
	at clojure.tools.gitlibs$procure.invoke(gitlibs.clj:41)
	at clojure.tools.deps.alpha.extensions.git$eval1323$fn__1325.invoke(git.clj:42)
	at clojure.lang.MultiFn.invoke(MultiFn.java:239)
	at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:422)
	at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:390)
	at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:495)
	at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:475)
	at clojure.tools.deps.alpha$calc_basis.invokeStatic(alpha.clj:655)
	at clojure.tools.deps.alpha$calc_basis.invoke(alpha.clj:629)
	at clojure.tools.deps.alpha.script.make_classpath2$run_core.invokeStatic(make_classpath2.clj:91)
	at clojure.tools.deps.alpha.script.make_classpath2$run_core.invoke(make_classpath2.clj:57)
	at clojure.tools.deps.alpha.script.make_classpath2$run.invokeStatic(make_classpath2.clj:119)
	at clojure.tools.deps.alpha.script.make_classpath2$run.invoke(make_classpath2.clj:113)
	at clojure.tools.deps.alpha.script.make_classpath2$_main.invokeStatic(make_classpath2.clj:169)
	at clojure.tools.deps.alpha.script.make_classpath2$_main.doInvoke(make_classpath2.clj:140)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:705)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.main$main_opt.invokeStatic(main.clj:514)
	at clojure.main$main_opt.invoke(main.clj:510)
	at clojure.main$main.invokeStatic(main.clj:664)
	at clojure.main$main.doInvoke(main.clj:616)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:705)
	at clojure.main.main(main.java:40)

Alex Miller (Clojure team)18:03:39

Yeah, that’s just not going to work

borkdude20:03:37

@dnolen Can you run this under Rosetta?

markbastian20:03:24

Hey all, can anyone help me make sense of this one:

(not (= ##NaN ##NaN))
=> true
(not= ##NaN ##NaN)
=> false

manutter5120:03:18

It’s basically part of the definition of NaN

manutter5120:03:36

“NaN is not equal to anything, including itself.”

markbastian20:03:52

Right, but that isn’t the issue.

manutter5120:03:03

Oh wait, not=

markbastian20:03:04

not= and (not (=))

manutter5120:03:16

Yeah, I’m catching up.

markbastian20:03:31

Here’s some more fun. I am working in a mixed clj/cljs env. Just discovered this:

(= (not (= ##NaN ##NaN)) (not= ##NaN ##NaN))
=> false
^In a clj repl
(= (not (= ##NaN ##NaN)) (not= ##NaN ##NaN))
=> true
^In a cljs repl

markbastian20:03:01

but the cljs version is the one that behaves as expected

andy.fingerhut20:03:04

The answer for Clojure (not ClojureScript) is probably somewhere in this annotated output: https://github.com/jafingerhut/batman/blob/master/doc/macos-10.13.6-jdk-adoptopenjdk-11.0.3-clojure-1.10.1.txt

markbastian20:03:48

Yeah, I was looking at https://clojuredocs.org/clojure.core/not=, but that just the docs, not the src.

manutter5120:03:10

The source doesn’t really help:

(defn not=
  "Same as (not (= obj1 obj2))"
  {:tag Boolean
   :added "1.0"
   :static true}
  ([x] false)
  ([x y] (not (= x y)))
  ([x y & more]
   (not (apply = x y more))))

manutter5120:03:30

For 2 args, not= just calls (not (= x y))

andy.fingerhut20:03:35

Here be dragons. Avoid NaN whenever you can.

💯 9
bronsa20:03:08

= is inlined w/o boxing

bronsa20:03:12

not= isn't

bronsa20:03:19

user=> (not (= ##NaN ##NaN))
true
user=> (not (= (identity ##NaN) (identity ##NaN)))
false
user=> (not= ##NaN ##NaN)
false

andy.fingerhut20:03:27

IIRC, (= ##NaN ##NaN) can sometimes return true because of the optimization to check for identical? first, and sometimes two ##NaN objects are identical.

andy.fingerhut20:03:06

If two ##NaN objects are not identical, then (= ##NaN ##NaN) returns false

bronsa20:03:25

indeed, when comparing ##NaN for equality as an object, it may return true

bronsa20:03:35

but as primitive doubles, it will not

manutter5120:03:08

Ok, but for 2 args, not= just calls (not (= x y)), so why doesn’t it behave the same?

bronsa20:03:25

because = has intrinsics in the compiler

andy.fingerhut20:03:37

There are some expressions here where the Clojure compiler inlines, vs. not, object vs. primitive double, etc. The transcript I linked above might cover all the possibilities, but it might not. Feel free to suggest additions if it is missing anything.

bronsa20:03:37

which avoid boxing the arguments if they are literal primitive values

bronsa20:03:53

but not= isn't special cased, so the primitive values always get boxed

bronsa20:03:20

if not= was a macro instead of a function (or if it had an inline annotation), it'd behave differently

manutter5120:03:30

Ah, ok, the boxing is invoked in the process of evaling the arguments, and by the time they get to =, they’re already boxed.

manutter5120:03:37

That’s interesting.

andy.fingerhut20:03:40

There is almost no rule you can give about ##NaN behavior which is true in all cases, other than it will drive you nuts if you try to find general rules.

3
manutter5120:03:40

“The only way to win is not to play.” — WOPR.

bronsa20:03:49

##NaN is the only value where this can happen

bronsa20:03:59

and should just not be used in equality comparisons

andy.fingerhut20:03:01

Play with doubles/floats carefully, if you need them (read as much of https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html as you need for your purposes), but just stay away from NaN whenever you can.

markbastian20:03:50

In my current use case, I definitely do not want ##NaN (or even doubles), but this does seem like a bug.

andy.fingerhut20:03:04

which part? 🙂

markbastian20:03:20

(not= not producing the same behavior as (not (=.

andy.fingerhut20:03:57

I haven't tried to examine what it would take to "fix" that, but I suspect it would mean slowing down operations on values that were not ##NaN

markbastian20:03:18

But it isn’t anything I’ll lose any sleep over. As we all say, the best solution is to prevent ##NaN in the first place.

bronsa20:03:22

what it would take to fix this would be slowing down every call to = to check for isNan before identity?

bronsa20:03:25

never gonna happen

bronsa20:03:22

I suppose the ##NaN literal could return a new instance every time and that would "fix it" for the ##NaN literal

markbastian20:03:29

Yeah, I wasn’t even suggesting it be fixed. Mainly just wondering why it was behaving as it was in the first place.

Filipe Silva20:03:53

well there's also this

Double/NaN
##NaN
(not= Double/NaN Double/NaN)
true
(not= ##NaN ##NaN)
false

andy.fingerhut20:03:24

That is probably because of Object identity in one case, but two non-identical objects in the other

3
andy.fingerhut02:03:41

Clojure 1.10.1
user=> (identical? Double/NaN Double/NaN)
false
user=> (identical? ##NaN ##NaN)
true

bronsa20:03:27

but there's still programmatic way to make it behave like this, it's just an edge case of how NaN work as a primitive (as defined by the float standard) and how it behaves as an object

andy.fingerhut20:03:04

##NaN eats babies for breakfast, shoots your dog, steals your Bible, and drives Darth Vader insane.

😂 6
Lennart Buit21:03:24

floats remain one of the more interesting inventions of computer science. They are incredibly clever, and incredibly … sharp and pointy sometimes

andy.fingerhut22:03:33

I believe that most arithmetic operations are not associative using floating point arithmetic, at least for a large class of values, e.g. it is often the case that you get different results from ((a+b)+c) vs. (a+(b+c)). I believe Fortran and a few other programming language mandate that order of operations emitted by the compiler may not change from what is implied in the source code, so people can write floating-point numerical libraries and not have the compiler mess up what they intended.

raspasov06:03:09

@U0CMVHBL2 so you’re saying that something like:

(= 
 (+ (+ 1.111 1.111) 1.111) 
 (+ 1.111 (+ 1.111 1.111)))
=> true

raspasov06:03:43

… might return false, depending on the numbers involved?

Filipe Silva20:03:43

'tis evil, I tell you

bringe22:03:07

https://clojure.org/reference/atoms Could someone explain to me why, in this example, the memoized fib call returns in a fraction of the time of the not memoized fib call even on the first call to the memoized fib? I ran the example and verified that it does run way faster on the first call, but how does this work if the result for those args hasn't been saved yet? I understand that subsequent calls with the same args should return much faster since the results have been saved, just unsure about how that happens with the first call.

bringe22:03:21

Furthermore, if I run something like (fib 444) unmemoized it ties up the repl, naturally, but if I memoize fib then call the same thing I get an actual result in milliseconds. :thinking_face:

bringe22:03:43

And that result hasn't been saved before ^

p-himik22:03:01

fib is recursive. (fib 5) calls (fib 4) and (fib 3). And if those are in cache, then (fib 5) will be fast. So now you have (fib 5) and (fib 4) in cache - exactly what's needed for (fib 6). And so on.

bringe22:03:36

Ohhhh 🙃

p-himik22:03:09

I strongly suggest reading the 2nd edition of SICP. It has some great material that deals with all sorts of problems in the functional world and outside of it - fib is one of them.

bringe22:03:28

Oh cool. I actually have this on my list.

👍 6
bringe22:03:00

Maybe I should be clear I'm using clojurescript targetting node, not sure if that matters though.

dpsutton22:03:11

because fib calls itself. fib 35 calls fib 34 and fib 33. each of which call .... So "the first call" has lots of other first calls that are duplicated

bringe22:03:19

Oh I see! 😄 Thank you.

Alex Miller (Clojure team)22:03:50

it's the difference between 444 and 444! :)

simple_smile 3