cool, thanks! you always have the best core.async gists

Ah, so this is interesting:

(type (to-array [1 2 3]))
;; => [Ljava.lang.Object;
(type (into-array [1 2 3]))
;; => [Ljava.lang.Long;
(type (into-array Integer/TYPE [1 2 3]))
;; => [I
(type (int-array [1 2 3]))
;; => [I
(type (aget (int-array [1 2 3])
            0))
;; => java.lang.Integer

the last one with aget - is that always an object?

I don't think so:

> (type (int 1))
java.lang.Integer

> (identical? (int 1) (Integer. 1))
false
> (identical? (int 1) (int 1))
true

I'm not sure how test for boxed Integer vs int

user=> (let [is (into-array Integer [1 2 3])] (class is))
Execution error (IllegalArgumentException) at java.lang.reflect.Array/set (Array.java:-2).
array element type mismatch
user=> (let [is (into-array Integer [(int 1) (int 2) (int 3)])] (class is))
[Ljava.lang.Integer;
user=>

As for identical? I believe Java caches small values of numbers so (int 1) is always the same object, but (Integer. 1) is going to construct a new instance (with the same value) but a unique identity?

the docs say aget /`aset` are overloaded for arrays of primitives, https://clojure.org/reference/java_interop#primitives

regarding identical?, it looks like it just uses the java == operator, https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Util.java#L134

which I think is value equality for primitives like int

user=> (take 1 (drop-while #(identical? (int %) (int %)) (range)))
(128)

So the first 128 (0..127) are cached.

This is true for byte and long as well.

I think working with primitives is very tricky, every time arguments are passed to a function, primitives are boxed again. So I wonder if the call to type will box the return of aget

See https://insideclojure.org/2014/12/15/warn-on-boxed/

ohhh, so https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Util.java#L134 is using the == operator, but only after converting the ints to Integers?

because the method signature is identical(Object k1, Object k2)

rather than also having identical(int k1, int k2)

Ya, identical also boxes, all function by default will box their arguments and return value, unless they are type hinted as long or double (no other primitive type hint will work to remove the boxing)

Unless they also have an overload in Java, like aget does

I don't think type can actually return a primitive type, right?

I guess it couldn't, since it probably can't actually be passed one as argument

(a great discussion, thanks!)

And like that article sean posted says: > You probably noticed we didn’t see the boxing occurring on the way in/out of the function. That’s not actually boxed math, just function invocation, so it’s a little harder to detect and warn on automatically Which is why its extra tricky, cause :warn-on-boxed won't show you if you're boxing by accident through function application

But also, in a primitive context, you can't use any of Clojure core, except for arithmetics and the array functions, everything else will box again.

also, referencing @seancorfield’s comment about small Integers being cached. you can use that to make 1+1=3, https://pedrorijo.com/blog/java-integer-cache/

Hahaha... 🙂

You can write functions in Clojure with (I think) up to at most 4 arguments that take a mix of Object, and long and double primitives, and return Object, or a long or double primitive, using type hints, but I don't recall are any built-in Clojure functions that do this.

And that technique works only for those primitive types, no others.

buddy? https://funcool.github.io/buddy-core/latest/api/04-dsa.html

Yes, but, I don't have enough Kotlin experience to know for sure what they do, I suspect that Kotlin is not lazy, and does something more like Java streams. So lazy-seq would not be the same. Transducers are more likely to be similar to what Kotlin does I'd suspect

That said, lazy seq will not go through the whole data in sequence, each element is pulled and then goes through the whole pipeline of transformation, and then the next element is pulled, etc.

Awesome, that's exactly what I wanted to know, thanks!

Hum, a quick read of the Kotlin doc on sequences, so maybe they are much more like Clojure sequences.

And not like Java streams

one difference might be with chunking - clojure has chunked collections which are evaluatred N elements (often 32) at a time, rather than one by one

this won't matter if you don't use laziness for side effects (don't use laziness for side effects)

Anyways, I'm not sure on the Kotlin side, but I'm sure on the Clojure side 😛 From Clojure, think of it as "pulling". When you define a pipeline:

(->> [1 2 3]
  (map inc)
  (filter odd?)
  (map #(* 2 %)))

And like noisesmith said, there's a small caveat in that because Clojure support ChunkedSeq as well (not all lazy-seq are chunked but most are). Well when you have a ChunkedSeq you can't ask for the next element, you can only ask for the next chunk. So the default chunk size is 32 it means even if you say please give me (first ...) you're actually saying please give me the first result of the first chunk. Which means 32 elements will go through the pipeline, but not one at a time, it'll be 32 go through the first step fully, and then 32 go through the second step, etc.

This is a performance optimization, but it also means if you really want to be one at a time lazy, you have to be careful you're not using ChunkedSeq.

\end-wall-of-text

> you can't ask for the next element, you can only ask for the next chunk you can ask for one item, but a chunk at a time is realized and cached on the implementation side

> you're actually saying please give me the first result of the first chunk

@U016XBH746B Ya so the infographic at the end of the Kotlin doc https://kotlinlang.org/docs/reference/sequences.html this is exactly how the Clojure lazy sequence work when not chunked.

{:version "1.10.1.492"
 :config-files ["/usr/local/Cellar/clojure/1.10.1.492/deps.edn" "/Users/pgearon/.clojure/deps.edn" "deps.edn" ]
 :config-user "/Users/pgearon/.clojure/deps.edn"
 :config-project "deps.edn"
 :install-dir "/usr/local/Cellar/clojure/1.10.1.492"
 :config-dir "/Users/pgearon/.clojure"
 :cache-dir ".cpcache"
 :force false
 :repro false
 :resolve-aliases ""
 :classpath-aliases ""
 :jvm-aliases ""
 :main-aliases ""
 :all-aliases ""}

Yup, that's too old for the new -M behavior.

that's really old

ah, thank you

1.10.1.763 is current. A lot of changes happened after 1.10.1.536

I’ve only been paying attention to the 1.10.1

Yup, pulling it in now. Thank you!

https://clojure.org/releases/tools lists all the changes.

While it’s trying to pull it in now, it’s failing to do so:

Cloning: 
Error building classpath. Destination path "test-runner" already exists and is not an empty directory

I suspect that's in ~/.gitlibs

thank you!

either you had a failed thing there that's messing with it or you are encountering a known race condition with downloading gitlibs

adding -Sthreads 1 avoids the latter for the moment

OK, I’ve hurt myself here. I blew away the ~/.gitlibs directory, but now it says:

Cloning: 
Error building classpath. [email protected]:cognitect-labs/test-runner.git: USERAUTH fail

maybe it’s my 2FA on github

it's not finding your user auth. 2fa isn't specifically an issue

https://clojure.org/reference/deps_and_cli#_git has some further tips

Thank you

first, check

ssh-add -l

then most thing wrong is in ~/.ssh/config

using IdentityFile ~/.ssh/id_rsa there is usually problematic - try commenting that out (with #)

hang on, I think it's probably something else

you're using https (public) but it's trying git (private)

are you on CI or local?

sorry, I don’t follow that question

oh, continuous integration

local

this is all my local command line

do you have a ~/.gitconfig with a insteadOf directive?

enabled with something like git config --global url."[email protected]:".insteadOf

(don't run that, but that's how it would be enabled)

grep for insteadOf in ~/.gitconfig

could also be set on a per-project basis

ah, yes, I do. Last set up some years ago, so I forgot it

yeah, that's forcing it to try to use your ssh key

Finally got it. Thank you!

(actually running the tests now)

cool

appreciate it all. That was a lot 🙂

this my iterator based implementation:

(defn kv-loop-iterator [f init coll]
  (let [it (clojure.lang.RT/iter coll)]
    (loop [out init]
      (if (.hasNext it)
        (let [entry (.next it)
              k     (key entry)
              v     (val entry)]
          (recur (f out k v)))
        out))))

bench results (also add some other things I tried):

| title                   | mean          | bar                  | variance-str |
|-------------------------+---------------+----------------------+--------------|
| reduce-kv-native        | 136.966296 ns | █████▏               | 0.000x       |
| reduce-kv-loop-iterator | 301.646678 ns | ███████████▍         | 1.202x       |
| reduce-kv-simple-reduce | 412.819192 ns | ███████████████▋     | 2.014x       |
| reduce-kv-loop-seq      | 530.523741 ns | ████████████████████ | 2.873x       |

I think you’re unlikely to beat reduce-kv, or self reduction on a map

Maps don’t store entries, just kvs. So anything that makes entries that you have to take apart is just additional cost in making and destroying objects

reduce-kv avoids that

Working on that right now, coming in 1.11... :)

@U064X3EF3 Great to hear! Can you say how it's going to be addressed?

All in good time

You know me: as soon as there are changes on master (or another accessible branch), I'm happy to try them out at work! 🙂

well obviously nothing before 1.10.2 ... likely rc3 coming tomorrow

Will get it on dev a.s.a.p. and onto QA early next week...