Fork me on GitHub
#clojure
<
2023-01-27
>
stuartrexking06:01:53

Why is the former faster than the latter?

(require '[criterium.core :as c])
  (def data (vec (range 100)))

  (c/report-result
   (c/benchmark
    (into [] (comp (filter even?) (map inc)) data) {}))

  ;; Evaluation count : 54365280 in 60 samples of 906088 calls.
  ;;              Execution time mean : 1.112239 µs
  ;;     Execution time std-deviation : 10.197468 ns
  ;;    Execution time lower quantile : 1.099641 µs ( 2.5%)
  ;;    Execution time upper quantile : 1.144325 µs (97.5%)
  ;;                    Overhead used : 2.033354 ns
  
  ;; Found 5 outliers in 60 samples (8.3333 %)
  ;; 	low-severe	 1 (1.6667 %)
  ;; 	low-mild	 1 (1.6667 %)
  ;; 	high-mild	 3 (5.0000 %)
  ;;  Variance from outliers : 1.6389 % Variance is slightly inflated by outliers 

  (c/report-result
   (c/benchmark
    (into [] (comp (map inc) (filter even?)) data) {}))
  
  ;; Evaluation count : 39379800 in 60 samples of 656330 calls.
  ;;              Execution time mean : 1.537396 µs
  ;;     Execution time std-deviation : 12.310657 ns
  ;;    Execution time lower quantile : 1.515765 µs ( 2.5%)
  ;;    Execution time upper quantile : 1.568869 µs (97.5%)
  ;;                    Overhead used : 2.033354 ns  ;; 

  ;; Found 3 outliers in 60 samples (5.0000 %) 
  ;;  low-severe	 3 (5.0000 %)
  ;;  Variance from outliers : 1.6389 % Variance is slightly inflated by outliers

p-himik06:01:51

Because you invoke inc half as many times there.

didibus06:01:14

In the first one, you only increment the even ones. In the second one you increment everything and then filter out the even ones. The two shouldn't even have the same result by the way, so it's not equivalent logic. If you increment first, you make the odds even and then remove the evens, leaving you with odds. On the other you remove the evens and increment the odds, leaving you with only evens.

(into []
 (comp 
  (map inc)
  (filter even?))
 [1 2 3 4])
> [2 4]
(into []
 (comp
  (filter even?)
  (map inc))
 [1 2 3 4])
> [3 5]

stuartrexking09:01:56

@U0K064KQV @U2FRKM4TW I think I’m misunderstanding something. The doc for comp says it applies to the right most function when passes the result to the left. If that’s true, the first example will inc first then filter, the second will filter then inc. if that’s true then the second example will inc half the number of the first, so it should be faster, no?

p-himik09:01:23

In this case, you're comping not inc and even? but rather (map inc) and (filter even?), which are transducers. It might not be obvious at the first glance why it's so, but it becomes rather clear when you learn more about transducers. Here's a documentation section on comp with transducers specifically: https://clojure.org/reference/transducers#_defining_transformations_with_transducers

🎯 2
2
💯 2
2
stuartrexking09:01:29

Oh! Thanks!

👍 2
didibus16:01:22

Ya, when using comp for transducers, the order becomes left to right. Comp does compose right to left, but with transducer it ends up applying them left or right.

didibus16:01:49

A little confusing I know.

valerauko07:01:28

can i rely on this for "normal" clojure maps regardless of size?

(= (zipmap (keys m) (vals m)) m)

p-himik07:01:13

Yes.

👍 2
phill10:01:06

Consistent ordering is promised by the docs of keys and vals: https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/keys

Al Z. Heymer10:01:36

Why is it that if you create an ex-info with a nil ex-data field, an IllegalArgumentException is thrown? I can use nil (nearly) everywhere else, but an ex-data-map is not valid with nil ?

Al Z. Heymer11:01:42

Additionally, the doc-string tells us nothing about this behaviour, which I find quite puzzling.

delaguardo11:01:57

> that carries a map of additional data. nil is not a map

Al Z. Heymer11:01:26

Ok, but (ex-data (Exception. "foo")) is nil . Subsequently, you would need to introduce an instance check for ExceptionInfo or an (or (ex-data e) {}) "catch". I think that is bad reasoning, tbh. If you provide nil data, the intention is clear: No additional data can be provided. An IllegalArgumentException makes no sense to me.

p-himik11:01:12

Not a lengthy explanation but an explanation, in the original ticket: https://clojure.atlassian.net/browse/CLJ-733?focusedCommentId=18813

👀 2
Al Z. Heymer12:01:38

I understand that there was an uncertainty to whether nil should be a valid argument, yet this wasn't picked up and discussed, because of the initial doc-string, which still does not explicitly tell you that it throws an IllegalArgumentException in that rather possible "intuitively correct" case. I do understand the reasoning to why ex-data should return nil, but it isn't really coherent with the behaviour of non-existent exception data in java exceptions and an illegal argument of ex-info

Alex Miller (Clojure team)13:01:10

There is a newer ticket about this (can’t easily search on my phone to find it) and I am hoping to fix it in 1.12. I agree with you that this is not good (and it’s entirely too easy to write error handling code that fails at runtime when there’s an error).

4
p-himik13:01:05

Seems like this is the one: https://clojure.atlassian.net/browse/CLJ-2640 > ex-info should be tolerant of nil data param

🙌 2
jumar15:01:05

What about adding a single-arity variant of ex-info that would simply use empty map (or nil) as the second argument?

p-himik15:01:05

That would still make things like

(let [data (something-that-can-return-nil)]
  (throw (ex-info "I might never be thrown" data)))
possible.

jumar15:01:40

Oh yeah; I was just wondering some many times why ex-info doesn't accept a single, msg, argument 🙂

☝️ 2
Alex Miller (Clojure team)15:01:15

there's more work on this elsewhere that isn't visible here

Alex Miller (Clojure team)15:01:40

I'm working on several ex-info things together

👍 4
wombawomba11:01:30

I'm looking for something that acts as a promise, but lets me call a function on deref/`realized?` (to see if done yet). Does this exist or should I just go ahead and write it myself?

lispyclouds11:01:40

(realized? a-promise) works right? if i understand your requirement correctly.

Al Z. Heymer11:01:47

I think this could work with a one-element lazy-seq . Once you consume it, the task is spawned

wombawomba11:01:42

@U02PS2BNULE hmm no, I want to perform the check each time it's checked/deref:ed until "done"

wombawomba11:01:19

@U7ERLH6JX hmm no, I want something like

(def p (promise (fn [] (println "checking...") (are-we-done-yet?)))

(realized? p) ;; prints "checking..."

wombawomba11:01:02

basically, I have an interface that takes a promise, but in this particular case I don't have a thread I can use to deliver the promise, so it'd be convenient to 'deliver' the promize lazily instead

lispyclouds11:01:55

what's the use case youre trying to model? maybe that helps in understanding more?

delaguardo11:01:24

sounds like a delay

delaguardo11:01:16

but it won't call the body when checked for realized?

wombawomba11:01:08

There's a library function (defn do-something [a-promise]) that expects a promise. I have a function are-we-done-yet?, that I can call to check if the promise should be delivered. I want to connect these two without having to create a thread that repeatedly calls are-we-done-yet? just so it can deliver the promise.

wombawomba11:01:56

yeah delay doesn't quite do the trick

lispyclouds11:01:07

so essentially like a callback when the promise is delivered?

wombawomba11:01:49

no, like a callback that the promise uses to check if it should act delivered

lispyclouds11:01:53

hmm, not sure if there's something off the shelf for this

wombawomba11:01:31

yeah alright, guess I'll write it myself

delaguardo11:01:23

why do you need to call are-we-done-yet? repeatedly? That could be a single deliver call to unlock other threads which are waiting for the promise.

wombawomba11:01:10

@U04V4KLKC there's really only one thread here, and it's managed by do-something

wombawomba11:01:20

since are-we-done-yet? is a (non-blocking) function and not a promise, I'd need to loop to see if it's done

delaguardo11:01:40

is it clojurescript?

wombawomba11:01:03

essentially what I'm trying to avoid is

(let [*p (promise)]
  (future
    (while-not (deref *p 0 false)
      (when (are-we-done-yet?)
        (deliver *p true))))
  (do-something *p))

wombawomba11:01:39

no, regular old Clojure

wombawomba11:01:50

but it's fine, I'll just write the thing I need

delaguardo11:01:59

so you have a library that expects a promise provided by your code and you want to realize it only when its needed by the library code?

wombawomba11:01:23

without doing any unnecessary work

delaguardo11:01:34

that is how delay works

wombawomba11:01:07

no, not quite

wombawomba11:01:21

with delay, I'd still need the loop

wombawomba11:01:47

and there'd still be a thread

delaguardo11:01:24

> with delay, I'd still need the loop why? because when you can create a delay you don't have a value for it?

wombawomba11:01:58

delays are only computed once, and are-we-done-yet? isn't blocking

wombawomba11:01:31

so I'd have to do something like

(delay (while (not (are-we-done-yet)))) true)

wombawomba11:01:07

also, realized? works differently on delays than on promises

delaguardo11:01:08

hm... differently how?

wombawomba11:01:56

well, I could be wrong, but afaik realized? doesn't trigger the computation

Matthew Downey15:01:24

Does a more fully-featured promise abstraction solve this? https://github.com/funcool/promesa

Jo Øivind Gjernes15:01:58

How can i close an “infite” io.reader ? (.close reader) seems to block :thinking_face:

emilaasa15:01:15

Do you have some more code to look at?

emilaasa15:01:34

What does read-from do?

Jo Øivind Gjernes15:01:27

Parsing json in my case

Jo Øivind Gjernes15:01:33

But i don’t think that matters :thinking_face:

Jo Øivind Gjernes15:01:45

It’s basically having an infinite source of messages

emilaasa15:01:05

And when do you want to break out?

emilaasa15:01:27

doseq docs:

Repeatedly executes body (presumably for side-effects) with
bindings and filtering as provided by "for".  Does not retain
the head of the sequence. Returns nil.

delaguardo15:01:28

(let [close? (promise)]
  (with-open [rdr (create-a-reader)]
    (loop []
      (when-not (.deref close? 1 false)
        (let [msg (read-from rdr)]
          (process-msg msg)
          (recur)))))
  (reify java.io.Closeable
    (close [_]
      (deliver close? true))))

delaguardo15:01:22

with-open will take care about closing the reader

emilaasa16:01:07

I often use line-seq and it does all right on many occasions

emilaasa16:01:24

but it depends on what you want to do 🙂

chrisn12:01:14

Charred returns an auto-closeable json supplier for these types of purposes - https://cnuernber.github.io/charred/charred.api.html#var-read-json-supplier.

Ben Lieberman18:01:57

is clojure.core/time's printed value analogous to the real time displayed by the time utility or something else?

seancorfield18:01:54

(source time) shows it is just Java's System/nanoTime

seancorfield18:01:06

(and purely an elapsed value from that)

Ben Lieberman18:01:53

I am forever forgetting about source :face_palm::skin-tone-2: thanks Sean

Lycheese19:01:21

Is there a macro-less obvious way to implement a call to for with a variable number of arguments that I missed or is this an acceptable case for writing a macro?

(defmacro cartesian [& args]
  (let [sym-arg-pairs (map #(list (gensym) %) args)
        syms (mapv first sym-arg-pairs)]
    `(for ~(into [] (reduce concat sym-arg-pairs))
       ~syms)))

(comment
  (= (for [a (range 3)
           b (range 3)
           c (range 3)
           d (range 3)
           e (range 3)]
       [a b c d e])
     (cartesian (range 3)
                (range 3)
                (range 3)
                (range 3)
                (range 3)))
  *e)

hiredman19:01:38

there is no reason it can't be a function

RollACaster19:01:55

here's a function version of cartesian product

(defn cart [colls]
  (if (empty? colls)
    '(())
    (for [more (cart (rest colls))
          x (first colls)]
      (cons x more))))

Lycheese19:01:14

Oh yeah that is obvious XD Thanks Maybe it's time to go to sleep

wevrem22:01:45

For cartesian product (if that is what you are really after, and it wasn’t just a placeholder example) there is also https://github.com/clojure/math.combinatorics

d._.b21:01:23

What's the thread pool size for futures again? Something like logical cores * 2 + 1 or something?

phronmophobic21:01:03

If the size of the thread pool is important to your use case, you should create your own thread pool.

hiredman21:01:11

clojure, the language runtime, has 2 threadpools built in, one is unbounded and the other is Runtime.getRuntime().availableProcessors() + 2

hiredman21:01:15

core.async has its own threadpools, one is unbounded and the other is maybe capped at 8 (I forget) but there is a system property you can use to change it

hiredman21:01:31

the java has also gained a "default" threadpool since clojure was released, and a lot of newer stuff uses that by default if you don't specify a pool to run on (completionstage stuff, the new httpclient)

hiredman21:01:47

the thread pool clojure.core/future uses is the unbounded clojure runtime one

d._.b22:01:14

thanks hiredman!

emccue23:01:26

enable preview and set them all to v-threads. f* da police

jumar16:01:10

Afaik, Virtual threads are mostly for io bound tadks, not cpu bounded computations