(let [ed (->> (range 500000)
              #_(into [])
              (eduction (map #(doto % print)))
              (eduction (map #(doto % (->> (str "    ") println)))))]

  (println "taking 3")
  (->> (into [] (take 3) ed)
       (println "taken 3:"))

  (println "getting first")
  (->> (first ed)
       (println "got first:"))

  (println "reducing first")
  (->> (reduce (comp reduced {}) nil ed)
       (println "reduced first:")))
taking 3
0    0
1    1
2    2
taken 3: [0 1 2]
getting first
0    0
1    1
2    2
3    3
4    4
5    5
6    6
7    7
8    8
9    9
10    10
11    11
12    12
13    13
14    14
15    15
16    16
17    17
18    18
19    19
20    20
21    21
22    22
23    23
24    24
25    25
26    26
27    27
28    28
29    29
30    30
31    31
32    32
got first: 0
reducing first
0    0
reduced first: 0
=> nil

This is on clojure 1.11.1

and 1.12.0-alpha1

It occurred to me whether perhaps eduction eagerly consumes its input collection as opposed to eagerly running the calculations within so I did another test where I added prints when the input is realized:

(let [new-input (fn []
                  (->> (range 100)
                       (map #(doto % (->> (println "     init:"))))))
      xf1       (map #(doto % (->> (print "  processing:"))))
      xf2       (map #(doto % (->> (println "    "))))
      ed        (->> (new-input)
                     (eduction xf1)
                     (eduction xf2))]

  (println "=================================================\nTaking 3")
  (->> (into [] (take 3) ed)
       (println "Taken 3:"))

  (println "=================================================\nTaking 3 again")
  (->> (into [] (take 3) ed)
       (println "Taken 3 again:"))

  (println "\n=================================================\nGetting first")
  (->> (first ed)
       (println "Got first:"))

  (println "\n=================================================\nGetting first again")
  (->> (first ed)
       (println "Got first again:"))

  (println "\n=================================================\nReducing first")
  (->> (reduce (comp reduced {}) nil ed)
       (println "Reduced first:"))

  (println "\n=================================================\nTransducing first without eduction from new input")
  (->> (new-input)
       (transduce (comp xf1 xf2) (completing (comp reduced {})) nil)
       (println "Transduced:")))
=================================================
Taking 3
     init: 0
     init: 1
     init: 2
     init: 3
     init: 4
     init: 5
     init: 6
     init: 7
     init: 8
     init: 9
     init: 10
     init: 11
     init: 12
     init: 13
     init: 14
     init: 15
     init: 16
     init: 17
     init: 18
     init: 19
     init: 20
     init: 21
     init: 22
     init: 23
     init: 24
     init: 25
     init: 26
     init: 27
     init: 28
     init: 29
     init: 30
     init: 31
  processing: 0     0
  processing: 1     1
  processing: 2     2
Taken 3: [0 1 2]
=================================================
Taking 3 again
  processing: 0     0
  processing: 1     1
  processing: 2     2
Taken 3 again: [0 1 2]

=================================================
Getting first
  processing: 0     0
  processing: 1     1
  processing: 2     2
  processing: 3     3
  processing: 4     4
  processing: 5     5
  processing: 6     6
  processing: 7     7
  processing: 8     8
  processing: 9     9
  processing: 10     10
  processing: 11     11
  processing: 12     12
  processing: 13     13
  processing: 14     14
  processing: 15     15
  processing: 16     16
  processing: 17     17
  processing: 18     18
  processing: 19     19
  processing: 20     20
  processing: 21     21
  processing: 22     22
  processing: 23     23
  processing: 24     24
  processing: 25     25
  processing: 26     26
  processing: 27     27
  processing: 28     28
  processing: 29     29
  processing: 30     30
  processing: 31     31
     init: 32
     init: 33
     init: 34
     init: 35
     init: 36
     init: 37
     init: 38
     init: 39
     init: 40
     init: 41
     init: 42
     init: 43
     init: 44
     init: 45
     init: 46
     init: 47
     init: 48
     init: 49
     init: 50
     init: 51
     init: 52
     init: 53
     init: 54
     init: 55
     init: 56
     init: 57
     init: 58
     init: 59
     init: 60
     init: 61
     init: 62
     init: 63
  processing: 32     32
Got first: 0

=================================================
Getting first again
  processing: 0     0
  processing: 1     1
  processing: 2     2
  processing: 3     3
  processing: 4     4
  processing: 5     5
  processing: 6     6
  processing: 7     7
  processing: 8     8
  processing: 9     9
  processing: 10     10
  processing: 11     11
  processing: 12     12
  processing: 13     13
  processing: 14     14
  processing: 15     15
  processing: 16     16
  processing: 17     17
  processing: 18     18
  processing: 19     19
  processing: 20     20
  processing: 21     21
  processing: 22     22
  processing: 23     23
  processing: 24     24
  processing: 25     25
  processing: 26     26
  processing: 27     27
  processing: 28     28
  processing: 29     29
  processing: 30     30
  processing: 31     31
  processing: 32     32
Got first again: 0

=================================================
Reducing first
  processing: 0     0
Reduced first: 0

=================================================
Transducing first without eduction from new input
     init: 0
     init: 1
     init: 2
     init: 3
     init: 4
     init: 5
     init: 6
     init: 7
     init: 8
     init: 9
     init: 10
     init: 11
     init: 12
     init: 13
     init: 14
     init: 15
     init: 16
     init: 17
     init: 18
     init: 19
     init: 20
     init: 21
     init: 22
     init: 23
     init: 24
     init: 25
     init: 26
     init: 27
     init: 28
     init: 29
     init: 30
     init: 31
  processing: 0     0
Transduced: 0
=> nil

But it appears that isn't the case either (this is on 1.11.1, chunking doesn't seem to be working as I had expected in 1.12)

On the first take 3 the first 32 elements of the input are realized, then the first 3 are processed by the eduction(s). On the second take 3 enough elements of the input are realized already so no further init happens, only the re-processing of the first 3 as per the eduction docs. first wants to look at the first 33 elements for some reason so on the first invocation the second 32 element chunk is realized, then the first 33 elements are processed. On the second invocation, all the elements needed are already realized so eduction reprocesses the first 33. Reducing first only needs a single element and that's what eduction reprocesses so. For a comparison I reimplemented the reducing first with transduce to see how it works without the eduction and it doesn't seem any different.

sorry, I was really referring to reduce in that sentence

Thanks for the response Alex 🙂 Does it also apply to this comment on slack? > eduction is pretty specialized in that it is a delayed eager evaluation https://clojurians.slack.com/archives/C053AK3F9/p1567446198032000

it is delayed

it is often used in eager scenarios

but it isn't inherently eager then?

no, it's iterator based, but often used with reduce, which is eager

Okay, thanks so much for taking the time to reply!

sorry, have been too busy to reply to the other ask question yet

Not at all, I honestly really appreciate your approachability and how you interact with the community! I re-raised the question here mainly because my understanding of what you had stated was at odds with how I had understood eduction. I had refactored code in multiple places to use it once I got to know eduction and part of the reason for those refactorings was the capability of eduction to only calculate as many elements as absolutely necessary. I feared that if I misunderstood eduction these might cause problems and wanted to ask for community wisdom to help decide whether I need to revisit those places in code. Thank you again for your time!

x# is just a symbol. It will turn into some other symbol only inside the syntax quote.

user=> `x#
x__2__auto__

it is not auto-gensymmed. defn itself is a macro so you can expand it to the code:

(def Integer# (fn* ([x] {:type 'Integer, :bytes 4, :value x})))

(defmacro foo []
  `(do
     (prn "!!!")
     (prn (Integer# 1))))

(foo) ;; expands to (do (prn "!!!") (prn (Integer__11986__auto__ 1)))
;; => Syntax error compiling at (src/dev/dotfox/matchete.cljc:295:1).
;;    Unable to resolve symbol: Integer__11986__auto__ in this context

Integer# will stay as is even when used as (defn Integer# ...).

appreciate it :) ty

@U03CZA5A539 You can use (ns-unmap *ns* 'Integer) and then simply call your function Integer. But sometimes people like to call these constructor functions ->Something, so maybe ->Integer isn't too bad either

@U04V15CAJ I like ->Something! I do need java.lang.Integer in the same namespace as my Integer constructor function, so unmapping java.lang.Integer wouldn't work off-the-cuff. Maybe I could rename java.lang.Integer instead? If so, how would I do that?

You could always refer to it by its full name. But please don't shadow it. It's much better to use ->Integer or any other variation if possible.

fully qualified java.lang.Integer will be there even after unmapping

great info from both of you! this is going to solve my problem

% cat thread-sleep.clj           
(println (format "clojure-version %s" (clojure-version)))
(println (format "java.version %s" (System/getProperty "java.version")))
(println "(Thread/sleep (int 100))")
(Thread/sleep (int 100))
(def int-millis (int 100))
(println "(Thread/sleep int-millis)")
(Thread/sleep int-millis)
(println "done")

% clj -Srepro -M thread-sleep.clj
clojure-version 1.11.1
java.version 17.0.6
(Thread/sleep (int 100))
(Thread/sleep int-millis)
done

% clj -Srepro -M thread-sleep.clj
clojure-version 1.11.1
java.version 19.0.2
(Thread/sleep (int 100))
(Thread/sleep int-millis)
Execution error (IllegalArgumentException) at user/eval11 (thread-sleep.clj:7).
No matching method sleep found taking 1 args

Any ideas what I’m missing?

They added a new overload to sleep that takes a Duration and that is screwing with clojures ability to determine what method you are trying to call

Gotcha. Thanks!

If you turn on reflection warnings you'll see it complain about not being able to figure it out

And if you pass a long instead of an int it will figure it out

Good call.

Yup, we found the long solution, but I hadn’t figured out the why. Thanks!

I think I've opened issues against several repos about reflection warnings from that change (we keep a pretty close eye on reflection warnings at work).

I've ran into this specific case in a number of projects already

@U04V15CAJ Is there anything in clj-kondo that can check whether I’ve remembered to enable reflection warnings for namespaces where I’ve done java interop?

yes: https://github.com/clj-kondo/clj-kondo/blob/master/doc/linters.md#warn-on-reflection

Perfect!

Thanks, all!

@alexmiller Maybe Clojure should have a sleep function to avoid this interop/reflection issue over several version of the JVM :)

What’s going on with the difference between (Thread/sleep (int 100)) and (Thread/sleep int-millis) ? What’s the difference in how the types are resolved?

Maybe answering my own question: the var doesn’t have a type hint, and int likely does

@U0EHU1800 The type of a var is never inferred

Yup. Makes sense.

another shout-out to @U04V15CAJ and clj-kondo. It’s really been life-changing for my development and particularly maintenance of clojure projects. Thank you!

@U0EHU1800 I always use clj-kondo, but when doing interop I often also add https://github.com/jonase/eastwood to my lint pipeline to help find this kind of issue.

I use the clojure compiler for this: https://github.com/http-kit/http-kit/blob/290c7f9cb1d022e453a3f8fc02f18044bed2044a/test-native/bb.edn#L15-L18

along with this: https://github.com/http-kit/http-kit/blob/290c7f9cb1d022e453a3f8fc02f18044bed2044a/test-native/dev/user.clj

Nice!

It's so useful to see how others approach these types of things.

we use https://github.com/athos/clj-check at work, which scans for clj files and then loads each one it finds with *warn-on-reflection* set

Not totally sure it's the only problem, but the _ symbol in the argument list should probably be a gensym _#

(defmacro def-ttype-head [it]
  `(defmethod ttype-head ~it [_#]
     (s/keys :req [::ttype-head ::bytes-kind ::dimensions])))

I think you can also do:

(s/def ::ttype-head (s/keys :req [::ttype-head ::bytes-kind ::dimensions]))
(defmethod ttype-head ::Integer [_]
  ::ttype-head)

yeah, it's the _ arg

note that CIDER does not use the error triage and printing system that we spent so much time making in Clojure 1.10, but if you use a clojure.main repl it would have said:

user=> (def-ttype-head ::Integer)
Syntax error macroexpanding clojure.core/fn at (REPL:1:1).
(user/_) - failed: Extra input at: [:fn-tail :arity-1 :params] spec: :clojure.core.specs.alpha/param-list
user/_ - failed: vector? at: [:fn-tail :arity-n :params] spec: :clojure.core.specs.alpha/param-list

@U7RJTCH6J would the spec ::ttype-head collide with the multi-spec (s/multi-spec ttype-head ::ttype-head)?

Yea, it would. You would have to use a different name

roger. Thanks for the sharp reading :)

ASIDE: this discussion helped me a lot. I just wrote a significant new macro and debugged the names given your teachings 🙂

(defmacro def-ttype-sugar [the-head]
  `(defn ~the-head
     [& {kind# :kind, dimensions# :dimensions,
         :or {kind# 4, dimensions# []}}] ;; defaults
     (let [cnf# (s/conform
                 ::asr-ttype-head
                 {::ttype-head
                  (keyword (str ~the-head))
                  ::bytes-kind kind#,
                  ::dimensions dimensions#})]
       (if (s/invalid? cnf#)
         (keyword "masr.specs" (str "invalid-" ~the-head))
         cnf#))))

that has nested multi-specs :)