Fork me on GitHub
#beginners
<
2024-06-28
>
Felix Dorner10:06:07

Ultrastupid question: I have a list of messages I want to log. How can I log each message individually with clojure.tools.logging?

pyry10:06:27

Like so?

(let [xs ["a" "b" "c"]]
    (doseq [x xs]
      (log/info x)))

🙇 1
Felix Dorner10:06:47

Dont understand why but (run! (fn [x] (log/info x)) xs) seems to work better for me when the xs come in very slow, and each line seems to be printed as soon as possible, whereas the version above with doseq seems to flush it all out in the end only.

growthesque12:06:01

is it better to use reduce or apply for summing collections?

jpmonettas12:06:04

(reduce str (range 10000))

Evaluation count : 12 in 6 samples of 2 calls.
             Execution time mean : 60.754565 ms
    Execution time std-deviation : 7.039782 ms
   Execution time lower quantile : 54.607000 ms ( 2.5%)
   Execution time upper quantile : 69.546219 ms (97.5%)
                   Overhead used : 8.661299 ns

(apply str (range 10000))

Evaluation count : 1524 in 6 samples of 254 calls.
             Execution time mean : 450.917359 µs
    Execution time std-deviation : 51.664164 µs
   Execution time lower quantile : 400.642425 µs ( 2.5%)
   Execution time upper quantile : 514.938083 µs (97.5%)
                   Overhead used : 8.661299 ns

Ben Sless12:06:31

You're cheating, string is not summing

Ben Sless12:06:27

When building a string the fastest solution is reducing into a StringBuilder with .append being the reducing function

growthesque12:06:46

why is this cheating, apply is still much slower in this example as well

Ben Sless12:06:36

No, apply str is faster, it's iterating over the sequence and accumulating into a string builder, while reduce creates a new string every element

jpmonettas12:06:37

(reduce + (range 10000))
Evaluation count : 4512 in 6 samples of 752 calls.
             Execution time mean : 159.437816 µs
    Execution time std-deviation : 24.200161 µs
   Execution time lower quantile : 139.749231 µs ( 2.5%)
   Execution time upper quantile : 187.718169 µs (97.5%)
                   Overhead used : 8.661299 ns

(apply + (range 10000))
Evaluation count : 3594 in 6 samples of 599 calls.
             Execution time mean : 183.463103 µs
    Execution time std-deviation : 21.677155 µs
   Execution time lower quantile : 171.652766 µs ( 2.5%)
   Execution time upper quantile : 220.698771 µs (97.5%)
                   Overhead used : 8.661299 ns

growthesque12:06:04

oh I didn't see it was microseconds instead of milliseconds

👀 1
growthesque12:06:15

or is this nanoseconds

jpmonettas12:06:58

I also got confused, because I misread both as micros facepalm

jpmonettas12:06:59

but then I guess it depends on how you use it, because this str reduction is a normal application of apply also

Ben Sless12:06:46

Not sure I got your last point

jpmonettas12:06:31

so I read the question more generic, like "accumulating collections" with (apply fn coll) vs (reduce fn coll), and I see there is no one that is always faster

jpmonettas12:06:31

(reduce concat coll-of-colls) is another one that is slower compared to (apply concat coll-of-colls)

Ben Sless12:06:54

It really depends on the accumulating function in question, although there is always an equivalent rf which is faster, and an implementation using reduce faster than using apply

💯 1
Ben Sless12:06:21

Reduce give you better theoretical performance. Give it a bad rf and that's put the window

Jim Newton12:06:31

Here's a question about something always get confused about when I return to clojure after being away for a while. When writing macros, in particular when using quasiquote, how do I distinguish between vars and symbols? I have two macros that both work but they are extremely similar. I want to refactor common code into a helper function. The difference is which variable names are used in the macro expansion. I want to write a function which takes the macro parameters, as well as the three variable names which differ between the two macros, and this helper function should simply return the macro expansion. In Common Lisp there's no difference between a symbol and a var, so this type of refactoring means just writing a function that returns the correct list, but it doesn't have to worry about distinguishing symbols with meaning from symbols with no meaning.

Jim Newton12:06:23

If I pass quoted variable names to the helper function, I think they'll be symbols, rather than vars. right?

Jim Newton12:06:22

Here are the two macros:

(defmacro +exists
  "Existential quantifier syntax.  body is expected to evaluate
  to a heavy-bool"
  [[var coll & others :as var-coll] & body]

  (cond (empty? var-coll)
        `(do ~@body)

        (= var :let)
        `(let ~coll
           (+exists [~@others]
               ~@body))

        (= var :when)
        `(if ~coll
           (+exists [~@others]
               ~@body)
           +false)

        (empty? others)
        `(+exists- '~var
                   (fn [~var] {:post [(heavy-bool? %)]}
                     ~@body)
                   ~coll)

        :else
        `(+exists [~var ~coll]
             (+exists [~@others]
                 ~@body))))

Jim Newton12:06:42

(defmacro +forall 
  "Universal quantifier syntax.  body is expected to evaluate
  to a heavy-bool"
  [[var coll & others :as var-coll] & body]
  (cond (empty? var-coll)
        `(do ~@body)

        (= var :let)
        `(let ~coll
           (+forall [~@others]
               ~@body))

        (= var :when)
        `(if ~coll
           (+forall [~@others]
               ~@body)
           +true)
        
        (empty? others)
        `(+forall- '~var
                   (fn [~var] {:post [(heavy-bool? %)]}
                     ~@body)
                   ~coll)

        :else
        `(+forall [~var ~coll]
             (+forall [~@others]
                 ~@body))))

Jim Newton12:06:12

I think I'd like to write a helper function which I can call within the defmacro

(defmacro +forall [[var coll & others :as var-coll] & body] 
  (helper-expand var coll others var-col body 
                 '+forall '+forall- +true))

(defmacro +exists [[var coll & others :as var-coll] & body] 
  (helper-expand var coll others var-col body 
                 '+exists '+exists- +false))

Noah Bogart12:06:14

syntax-quote doesn't resolve symbols to a var, it fully qualifies symbols. the symbol is resolved to a var at run time

Jim Newton15:06:26

OK, here is an example: I've written the function. and it fails to work.

(defn expand-quantifier [var coll others var-coll body
                         macro-name f-name ident]
  (cond (empty? var-coll)
        `(do ~@body)

        (= var :let)
        `(let ~coll
           (~macro-name [~@others]
               ~@body))

        (= var :when)
        `(if ~coll
           (~macro-name [~@others]
               ~@body)
           ~ident)
        
        (empty? others)
        `(~f-name '~var
                   (fn [~var] {:post [(heavy-bool? %)]}
                     ~@body)
                   ~coll)

        :else
        `(~macro-name [~var ~coll]
             (~macro-name [~@others]
                 ~@body))))

Jim Newton15:06:52

Now I redefine the two macros as follows:

(defmacro +exists
  "Existential quantifier syntax.  body is expected to evaluate
  to a heavy-bool"
  [[var coll & others :as var-coll] & body]
  (expand-quantifier var coll others var-coll body
                     '+exists '+exists- '+false))



(defmacro +forall 
  "Universal quantifier syntax.  body is expected to evaluate
  to a heavy-bool"
  [[var coll & others :as var-coll] & body]
  (expand-quantifier var coll others var-coll body
                     '+forall '+forall- '+true))

Jim Newton15:06:21

When I expand the expression (+forall [x (range 10)] (+heavy-bool (> x 0))) it returns the following

(+forall-
  'x
  (fn [x] {:post [(heavy-bool? %)]} (+heavy-bool (> x 0)))
  (range 10))
and when I explicitly copy and paste that expansions into the buffer and evaluate it, it evaluates just find and gives me the result I hoped for. However when I try to evaluate the expression explicitly, I get an error

Noah Bogart15:06:44

Oh, yeah, that's due to the "qualifying symbols" stuff. you need to say ~'%

Jim Newton15:06:00

I'm happy to link the the entire project if the issue is not obvious. I'm pretty sure the problem is my lack of understanding of when vars get created. Clojure macros are not really homoiconic

Jim Newton15:06:41

but when I define the macro without using the intermediate function, I don't need any such ~% .

Noah Bogart15:06:16

at read time, syntax quote is expanded to a (list ...) style expression, and symbols that aren't unquoted are resolved to either the current namespace or an aliased namespace. in the user namespace, foo` is read in as user/foo and example/foo` is read in as some.other.example/foo (if (require '[some.other.example :as example]) exists)

Noah Bogart15:06:53

~ and ' are right associative, read as their equivalent (unquote ...) and (quote ...) forms. so ~'% is read in as (unquote (quote %)), which is read in by the syntax quote as '% because unquote is telling the reader to treat it as the given form

Noah Bogart15:06:07

; eval (root-form): '`(pos? %)
(clojure.core/seq
 (clojure.core/concat
  (clojure.core/list 'clojure.core/pos?)
  (clojure.core/list 'user/%)))
vs
; eval (root-form): '`(pos? ~'%)
(clojure.core/seq
 (clojure.core/concat
  (clojure.core/list 'clojure.core/pos?)
  (clojure.core/list '%)))

Noah Bogart15:06:53

does that make sense? do you want more examples?

Jim Newton15:06:42

sorry, doesn't make sense. Are you retracting your statement from above? > syntax-quote doesn't resolve symbols to a var, it fully qualifies symbols. the symbol is resolved to a var at run time (edited)

Jim Newton15:06:32

OK, let's push that aside. I'll accept '~% as axiomatic for the moment. Next problem: I made the change you suggested. Now when I evaluate the following expression in the same file as the macro definition, it works fine.

(+forall [x (range 1 10 3)] [(odd? x) '({:forall true})])
However, when I put that line in a unit test in a different file it fails.
(deftest t+and
  (testing "and"
    (is (sut/+and (sut/+forall [x (range 1 10 3)] [(odd? x) '({:forall true})])
                  (sut/+exists [x (range 1 10 3)] [(odd? x) '({:exists true})])))))

Jim Newton15:06:15

It macro-expands to this in the testing file

(+forall-
  'x
  (fn [x] {:post [(sut/heavy-bool? %)]} [(odd? x) '({:forall true})])
  (range 1 10 3))

Noah Bogart15:06:00

remove the leading ' from +forall- in your expand-qualifier call

Jim Newton15:06:08

but it macro expands to this in the original file

(+forall-
  'x
  (fn [x] {:post [(heavy-bool? %)]} [(odd? x) '({:forall true})])
  (range 1 10 3))

Jim Newton15:06:46

that will insert the function object in the macro expansion.

Noah Bogart15:06:05

oh hm. maybe syntax-quote it then

Noah Bogart15:06:30

instead of '+forall-, do +forall-`

Noah Bogart15:06:03

you want the fully-qualified name to be inserted

Jim Newton15:06:01

OK, that works. using back-quote in the function call.

👍 1
Noah Bogart15:06:33

if you want, i can attempt to explain how the macro expansion works so you can understand why this is why it is

Jim Newton15:06:56

do you know how macro expansion works in other lisps?

Noah Bogart15:06:15

i don't, i've only dabbled in racket and common lisp but don't have deep knowledge of them

Jim Newton15:06:24

I'd like to understand how clojure macro expansion differs from what a lisper would expect.

Jim Newton15:06:09

is the problem that there is no reader syntax to distinguish var from symbol but the compiler treats them differently?

Noah Bogart15:06:44

there's #' which is read as (var ...) which is a special form understood by the compiler to be reference to a var

Jim Newton15:06:25

If I make the function calls like this, it seems to work (haven't run all the test cases yet)

(defmacro +exists
  "Existential quantifier syntax.  body is expected to evaluate
  to a heavy-bool"
  [[var coll & others :as var-coll] & body]
  (expand-quantifier var coll others var-coll body
                     (var +exists) (var +exists-) (var +false)))



(defmacro +forall 
  "Universal quantifier syntax.  body is expected to evaluate
  to a heavy-bool"
  [[var coll & others :as var-coll] & body]
  (expand-quantifier var coll others var-coll body
                     (var +forall) (var +forall-) (var +true)))

Jim Newton15:06:16

that's a bit surprising given that I have a local variable named var as well.

🤪 1
Noah Bogart15:06:05

that will work until you try to use var as a function, and then it will not work lol

Jim Newton15:06:20

OUCH!!! now the macro expansion is really ugly

(#<Var@6f5841e4: #function[heavy-bool/+forall]>
  [a (range 10)]
  (#<Var@6f5841e4: #function[heavy-bool/+forall]>
    [b (range 10) :when (> a 5) :when (> b a)]
    (sut/+heavy-bool (> b 5))))

Noah Bogart15:06:40

you shouldn't ever be looking at the macro expansion

Jim Newton15:06:58

Now even when I'm trying to debug a macro?

Noah Bogart15:06:18

that can be helpful, but it's not something you should worry about looking "pretty"

exitsandman15:06:08

macroexpand is broken in clj because it doesn't care about envs

exitsandman15:06:20

use tools.macro's version

exitsandman15:06:50

(it doesn't care about &env, specifically)

Jim Newton15:06:15

If I replace the call to var with backquote, then the macroexpansion is prettier.

(sut/+forall
  [a (range 10)]
  (sut/+forall
    [b (range 10) :when (> a 5) :when (> b a)]
    (sut/+heavy-bool (> b 5))))

👍 1
exitsandman15:06:45

oh wait, that's how it was supposed to look in that case, mb

Noah Bogart15:06:09

you're correct about the built-in macroexpand's &env bug, but it's not applicable here

exitsandman15:06:42

anyway, the reason var still works in macro position is because var is a special-form

👍 1
exitsandman15:06:24

on the contrary, things like let , fn and reify are actually macros that forward to a real special-form (respectively let*, fn* and reify*) and can be messed with

Jim Newton15:06:06

I really with :post hook would give better feedback when the assertion fails.

Noah Bogart15:06:24

yeah, it's not a great way to check invariants, sadly.

Jim Newton15:06:27

It ought to give me the value of % in the error message.

exitsandman15:06:22

I think spec might do a better job at that

Noah Bogart16:06:29

changing your impl to use a let with a better assert is pretty easy, thankfully: (fn [x] (let [ret# [(odd? x) '({:forall true})]] (assert (heavy-bool? ret#) (str "Expected heavy-bool, got " ret#)) ret))

exitsandman16:06:43

I personally use my own assert-info macro that throws an AssertionError & IExceptionInfo with a map of info of my choosing

Jim Newton16:06:02

What does this spec error mean?

(sut/value) - failed: Extra input at: [:fn-tail :arity-1 :params] spec: :clojure.core.specs.alpha/param-list
sut/value - failed: vector? at: [:fn-tail :arity-n :params] spec: :clojure.core.specs.alpha/param-list

Noah Bogart16:06:27

you gave a function that doesn't have a params vector: (fn (+ 1 1))

Jim Newton16:06:26

Here's what I wrote:

(fn [~var] {:post [((fn [value]
                                (assert (heavy-bool? value)
                                        (format "value=%s" value)))
                              ~'%)]}
                     ~@body)

Jim Newton16:06:58

isn't this valid syntax: ((fn [x] do-something) value-of-x) ?

Noah Bogart16:06:24

oh i see, replace value with value# in all usages in that function

Jim Newton16:06:49

why? it's a local variable. right?

exitsandman16:06:30

this is within a quasiquoted form judging by the unquotes, right?

Noah Bogart16:06:00

all symbols are read in as qualified symbols and use the current namespace if there's no namespace. if a symbol ends with a #, it's treated as an gensym, an auto-generated symbol literal, which is tracked for the duration of the syntax quote: value# is treated as the symbol literal value6845

Noah Bogart16:06:20

> For Symbols, syntax-quote resolves the symbol in the current context, yielding a fully-qualified symbol (i.e. namespace/name or fully.qualified.Classname). If a symbol is non-namespace-qualified and ends with '#', it is resolved to a generated symbol with the same name to which '' and a unique id have been appended. e.g. x# will resolve to x123. All references to that symbol within a syntax-quoted expression resolve to the same generated symbol.

exitsandman16:06:31

yeah Clojure's syntax-quote is a bit different from Common Lisp's, but I find it much better. It's great that you have to go out of your way to capture variables instead of the opposite being true

Jim Newton16:06:21

capture variablesin this case I have to go out of my way to define a local variable which I don't want to capture. I don't understand why I need value# rather than just value. but OK, I'll accept it as axiomatic. I'll write the function a different way to avoid the problem.

Noah Bogart16:06:31

the reasoning above doesn't explain it well enough for you?

Jim Newton16:06:49

`(~f-name '~var
          (fn [~var] {:post [(assert-heavy-bool ~'%)]}
                     ~@body)
          ~coll)
and then just define a normal function assert-heavy-bool which prints is argument if the assertion fails
(defn assert-heavy-bool [hb]
  (assert (heavy-bool? hb)
          (format "value=%s" hb)))

👍 1
exitsandman16:06:26

Avoiding to bind names to fixed symbols is important in macros. I'm sure you've seen the good old anaphoric macros like:

(defmacro aif [test then else] `(if (let [~'test ~test] (if ~'test ~then ~else)))))
These work great until somebody has a test binding outside of the macro, which will always be overridden inside the macro. Now, with anaphorics this is the intended behavior, but in CL it's quite easy to inadvertently introduce such captures. Clojure makes it hard by auto-namespacing symbols in syntax-quote and forcing you to write the very intentional '~test . To avoid this problem, the approach is to use gensym, which creates a symbol which will still be captured in the body, but which has a name so obscure that it will never come up in human written code.

👍 1
Jim Newton16:06:57

yes, but code like this does not insert user code inside the scope of value

((fn [value] (assert (> value 4))) %)
so I don't understand why I need a gensym.

Noah Bogart16:06:40

it's resolving value in the local context: example/value

Noah Bogart16:06:01

so it's ((fn [example/value] (assert (> example/value 4))) %)

👆 1
Jim Newton16:06:13

but what's the problem with that, it's a local variable which is referenced nowhere else. right?

Noah Bogart16:06:29

parameters aren't local variables, and parameters can't be qualified

Jim Newton16:06:17

ahhhhhhhhhhhhh! I absolutely didn't know that. In CL a local variable can be in any package (namespace) except keyword

exitsandman16:06:18

yeah which is why CL can be problematic in that regard. If you want to splice exactly value there, you can quote it and it'll work

Jim Newton16:06:03

is CL there is no such thing as "exactly value"

Jim Newton16:06:15

every symbol is in a namespace

exitsandman16:06:21

I was referring to Clojure ofc

Jim Newton16:06:04

ahhh. sorry. I think macros are confusing. I thought they were easy 40 years ago, the more I know about them, the more difficult they are.

exitsandman16:06:56

it's why functions are preferred if possible (also they compose much better than macros)

Jim Newton16:06:24

What does the :while modifier do in for and doseq ? how is it different from :when ?

👍 1
Jim Newton16:06:59

the code is different for :while and :when

Noah Bogart16:06:11

:while early exits, :when skips interations. break vs continue in a traditional for loop

Jim Newton16:06:49

ahhh :while breaks but still lets for return whatevery has been collected so far?

hiredman17:06:12

:when is filter :while is take-while

👍 3
Sam Ferrell22:06:58

tangentially something i wasn't initially aware you could do is repeat :let , :while, and :when expressions and they'll be evaluated in order ex. https://github.com/samcf/advent-of-code/blob/main/2020-11-seating-system.clj#L4-L21

Jim Newton09:06:59

I'm experimenting with :while having just discovered it. I'm not sure what's wrong with this code. can someone help?

(deftest t-example
  (doseq [a (range 10)
          b (range 10)
          :while (is (< (+ a b) 15))]))
My reasoning is that when the is test fails, then is will return false and thus cause doseq to abort. Instead here's the output I see.

Jim Newton09:06:48

maybe is is not evaluating to false?

Jim Newton09:06:32

modified the test slightly

Jim Newton09:06:16

it might have something to do with laziness??? I have trouble reasoning about the semi-laziness of clojure

Jim Newton13:06:35

I'm pretty puzzled about the semantics of :while actually. Look at the following loop:

user> (doseq [a (range 10)
              b (range 10)
              :let [_ (println [:sum (+ a b)])]
              :while (< (+ a b) 15)]
        (println [:print :a a :b b :sum (+ a b)]))

Jim Newton13:06:01

Here is the output, we see that the sum reaches 15 several times before the loop terminates.

[:sum 0]
[:print :a 0 :b 0 :sum 0]
[:sum 1]
[:print :a 0 :b 1 :sum 1]
[:sum 2]
[:print :a 0 :b 2 :sum 2]
[:sum 3]
[:print :a 0 :b 3 :sum 3]
[:sum 4]
[:print :a 0 :b 4 :sum 4]
[:sum 5]
[:print :a 0 :b 5 :sum 5]
[:sum 6]
[:print :a 0 :b 6 :sum 6]
[:sum 7]
[:print :a 0 :b 7 :sum 7]
[:sum 8]
[:print :a 0 :b 8 :sum 8]
[:sum 9]
[:print :a 0 :b 9 :sum 9]
[:sum 1]
[:print :a 1 :b 0 :sum 1]
[:sum 2]
[:print :a 1 :b 1 :sum 2]
[:sum 3]
[:print :a 1 :b 2 :sum 3]
[:sum 4]
[:print :a 1 :b 3 :sum 4]
[:sum 5]
[:print :a 1 :b 4 :sum 5]
[:sum 6]
[:print :a 1 :b 5 :sum 6]
[:sum 7]
[:print :a 1 :b 6 :sum 7]
[:sum 8]
[:print :a 1 :b 7 :sum 8]
[:sum 9]
[:print :a 1 :b 8 :sum 9]
[:sum 10]
[:print :a 1 :b 9 :sum 10]
[:sum 2]
[:print :a 2 :b 0 :sum 2]
[:sum 3]
[:print :a 2 :b 1 :sum 3]
[:sum 4]
[:print :a 2 :b 2 :sum 4]
[:sum 5]
[:print :a 2 :b 3 :sum 5]
[:sum 6]
[:print :a 2 :b 4 :sum 6]
[:sum 7]
[:print :a 2 :b 5 :sum 7]
[:sum 8]
[:print :a 2 :b 6 :sum 8]
[:sum 9]
[:print :a 2 :b 7 :sum 9]
[:sum 10]
[:print :a 2 :b 8 :sum 10]
[:sum 11]
[:print :a 2 :b 9 :sum 11]
[:sum 3]
[:print :a 3 :b 0 :sum 3]
[:sum 4]
[:print :a 3 :b 1 :sum 4]
[:sum 5]
[:print :a 3 :b 2 :sum 5]
[:sum 6]
[:print :a 3 :b 3 :sum 6]
[:sum 7]
[:print :a 3 :b 4 :sum 7]
[:sum 8]
[:print :a 3 :b 5 :sum 8]
[:sum 9]
[:print :a 3 :b 6 :sum 9]
[:sum 10]
[:print :a 3 :b 7 :sum 10]
[:sum 11]
[:print :a 3 :b 8 :sum 11]
[:sum 12]
[:print :a 3 :b 9 :sum 12]
[:sum 4]
[:print :a 4 :b 0 :sum 4]
[:sum 5]
[:print :a 4 :b 1 :sum 5]
[:sum 6]
[:print :a 4 :b 2 :sum 6]
[:sum 7]
[:print :a 4 :b 3 :sum 7]
[:sum 8]
[:print :a 4 :b 4 :sum 8]
[:sum 9]
[:print :a 4 :b 5 :sum 9]
[:sum 10]
[:print :a 4 :b 6 :sum 10]
[:sum 11]
[:print :a 4 :b 7 :sum 11]
[:sum 12]
[:print :a 4 :b 8 :sum 12]
[:sum 13]
[:print :a 4 :b 9 :sum 13]
[:sum 5]
[:print :a 5 :b 0 :sum 5]
[:sum 6]
[:print :a 5 :b 1 :sum 6]
[:sum 7]
[:print :a 5 :b 2 :sum 7]
[:sum 8]
[:print :a 5 :b 3 :sum 8]
[:sum 9]
[:print :a 5 :b 4 :sum 9]
[:sum 10]
[:print :a 5 :b 5 :sum 10]
[:sum 11]
[:print :a 5 :b 6 :sum 11]
[:sum 12]
[:print :a 5 :b 7 :sum 12]
[:sum 13]
[:print :a 5 :b 8 :sum 13]
[:sum 14]
[:print :a 5 :b 9 :sum 14]
[:sum 6]
[:print :a 6 :b 0 :sum 6]
[:sum 7]
[:print :a 6 :b 1 :sum 7]
[:sum 8]
[:print :a 6 :b 2 :sum 8]
[:sum 9]
[:print :a 6 :b 3 :sum 9]
[:sum 10]
[:print :a 6 :b 4 :sum 10]
[:sum 11]
[:print :a 6 :b 5 :sum 11]
[:sum 12]
[:print :a 6 :b 6 :sum 12]
[:sum 13]
[:print :a 6 :b 7 :sum 13]
[:sum 14]
[:print :a 6 :b 8 :sum 14]
[:sum 15]
[:sum 7]
[:print :a 7 :b 0 :sum 7]
[:sum 8]
[:print :a 7 :b 1 :sum 8]
[:sum 9]
[:print :a 7 :b 2 :sum 9]
[:sum 10]
[:print :a 7 :b 3 :sum 10]
[:sum 11]
[:print :a 7 :b 4 :sum 11]
[:sum 12]
[:print :a 7 :b 5 :sum 12]
[:sum 13]
[:print :a 7 :b 6 :sum 13]
[:sum 14]
[:print :a 7 :b 7 :sum 14]
[:sum 15]
[:sum 8]
[:print :a 8 :b 0 :sum 8]
[:sum 9]
[:print :a 8 :b 1 :sum 9]
[:sum 10]
[:print :a 8 :b 2 :sum 10]
[:sum 11]
[:print :a 8 :b 3 :sum 11]
[:sum 12]
[:print :a 8 :b 4 :sum 12]
[:sum 13]
[:print :a 8 :b 5 :sum 13]
[:sum 14]
[:print :a 8 :b 6 :sum 14]
[:sum 15]
[:sum 9]
[:print :a 9 :b 0 :sum 9]
[:sum 10]
[:print :a 9 :b 1 :sum 10]
[:sum 11]
[:print :a 9 :b 2 :sum 11]
[:sum 12]
[:print :a 9 :b 3 :sum 12]
[:sum 13]
[:print :a 9 :b 4 :sum 13]
[:sum 14]
[:print :a 9 :b 5 :sum 14]
[:sum 15]
nil
user> 

Jim Newton13:06:18

simplified a bit:

user> (doseq [a (range 10)
              b (range 10)
              :let [_ (println [:sum (+ a b)])]
              :while (< (+ a b) 15)])
Here is the output:
[:sum 0]
[:sum 1]
[:sum 2]
[:sum 3]
[:sum 4]
[:sum 5]
[:sum 6]
[:sum 7]
[:sum 8]
[:sum 9]
[:sum 1]
[:sum 2]
[:sum 3]
[:sum 4]
[:sum 5]
[:sum 6]
[:sum 7]
[:sum 8]
[:sum 9]
[:sum 10]
[:sum 2]
[:sum 3]
[:sum 4]
[:sum 5]
[:sum 6]
[:sum 7]
[:sum 8]
[:sum 9]
[:sum 10]
[:sum 11]
[:sum 3]
[:sum 4]
[:sum 5]
[:sum 6]
[:sum 7]
[:sum 8]
[:sum 9]
[:sum 10]
[:sum 11]
[:sum 12]
[:sum 4]
[:sum 5]
[:sum 6]
[:sum 7]
[:sum 8]
[:sum 9]
[:sum 10]
[:sum 11]
[:sum 12]
[:sum 13]
[:sum 5]
[:sum 6]
[:sum 7]
[:sum 8]
[:sum 9]
[:sum 10]
[:sum 11]
[:sum 12]
[:sum 13]
[:sum 14]
[:sum 6]
[:sum 7]
[:sum 8]
[:sum 9]
[:sum 10]
[:sum 11]
[:sum 12]
[:sum 13]
[:sum 14]
[:sum 15]
[:sum 7]
[:sum 8]
[:sum 9]
[:sum 10]
[:sum 11]
[:sum 12]
[:sum 13]
[:sum 14]
[:sum 15]
[:sum 8]
[:sum 9]
[:sum 10]
[:sum 11]
[:sum 12]
[:sum 13]
[:sum 14]
[:sum 15]
[:sum 9]
[:sum 10]
[:sum 11]
[:sum 12]
[:sum 13]
[:sum 14]
[:sum 15]
nil

Jim Newton07:06:31

doseq is not lazy at allAre you sure it's not lazy? When I look at the macro expansion I see lots of laziness coded in. Maybe, I'm mistaken but there's lots to do with chunking in there.

(loop [seq_13095 (seq (range 10))
       chunk_13096 nil
       count_13097 0
       i_13098 0]
  (if (< i_13098 count_13097)
    (let [a (.nth chunk_13096 i_13098)]
      (loop [seq_13103 (seq (range 10))
             chunk_13104 nil
             count_13105 0
             i_13106 0]
        (if (< i_13106 count_13105)
          (let [b (.nth chunk_13104 i_13106)]
            (let [_ (println [:sum (+ a b)])]
              (when (< (+ a b) 15)
                (do)
                (recur
                  seq_13103
                  chunk_13104
                  count_13105
                  (unchecked-inc i_13106)))))
          (when-let [seq_13103 (seq seq_13103)]
            (if (chunked-seq? seq_13103)
              (let [c__6065__auto__ (chunk-first seq_13103)]
                (recur
                  (chunk-rest seq_13103)
                  c__6065__auto__
                  (int (count c__6065__auto__))
                  (int 0)))
              (let [b (first seq_13103)]
                (let [_ (println [:sum (+ a b)])]
                  (when (< (+ a b) 15)
                    (do)
                    (recur (next seq_13103) nil 0 0))))))))
      (recur
        seq_13095
        chunk_13096
        count_13097
        (unchecked-inc i_13098)))
    (when-let [seq_13095 (seq seq_13095)]
      (if (chunked-seq? seq_13095)
        (let [c__6065__auto__ (chunk-first seq_13095)]
          (recur
            (chunk-rest seq_13095)
            c__6065__auto__
            (int (count c__6065__auto__))
            (int 0)))
        (let [a (first seq_13095)]
          (loop [seq_13099 (seq (range 10))
                 chunk_13100 nil
                 count_13101 0
                 i_13102 0]
            (if (< i_13102 count_13101)
              (let [b (.nth chunk_13100 i_13102)]
                (let [_ (println [:sum (+ a b)])]
                  (when (< (+ a b) 15)
                    (do)
                    (recur
                      seq_13099
                      chunk_13100
                      count_13101
                      (unchecked-inc i_13102)))))
              (when-let [seq_13099 (seq seq_13099)]
                (if (chunked-seq? seq_13099)
                  (let [c__6065__auto__ (chunk-first seq_13099)]
                    (recur
                      (chunk-rest seq_13099)
                      c__6065__auto__
                      (int (count c__6065__auto__))
                      (int 0)))
                  (let [b (first seq_13099)]
                    (let [_ (println [:sum (+ a b)])]
                      (when (< (+ a b) 15)
                        (do)
                        (recur (next seq_13099) nil 0 0))))))))
          (recur (next seq_13095) nil 0 0))))))

Jim Newton07:06:32

When I look at the behavior, the doseq DOES NOT quit as soon as the :while clause is encountered. As you can see from the printed output [:sum 15] is printed multiple times before the loop aborts.

Jim Newton07:06:24

> In general the problem people have understanding for and doseq, is they nest Are you suggesting that the :while only aborts the inner-most loop? I.e. the b (range 10) loop? That's an interesting explanation.

Jim Newton07:06:58

In fact if the doseq only contains a single loop, then the :while does cause an abort as soon as the condition fails.

user> (doseq [a (range 10)
              :let [b 7]
              :let [_ (println [:sum (+ a b)])]
              :while (< (+ a b) 15)])
[:sum 7]
[:sum 8]
[:sum 9]
[:sum 10]
[:sum 11]
[:sum 12]
[:sum 13]
[:sum 14]
[:sum 15]
nil
user> 

hiredman07:06:12

Seqs are lazy when produced not when consumed e.g. it is lazy when you call lazy-seq, but first and rest are not lazy, same for traversing chunks

Jim Newton19:06:15

Is the behavior of :while explained somewhere? I think I understand it after a bunch of experimentation. But I thing it ought t be explained either in the for or the doseq docstrings.

hiredman20:06:48

Not that I have seen, to my mind if you understand the mechanical procedure for transforming a for or doseq into calls to mapcat, map, filter, etc, the behavior of take-while falls out of that same model. But I use for a lot and never use :while

adi04:07:28

@U010VP3UY9X doseq with two bindings is like a loop within a loop. In your t-example, doseq loops for every a until the :while condition holds.

adi04:07:38

This will be easier to see with a slightly modified example:

(doseq [a [1 2 3 4 5]
        b [0 1]
        :while (< (+ a b) 5)]
...
)

adi04:07:16

for will behave similarly, except it will be lazy and return the generated sequence.

(for [a [1 2 3 4 5]
      b [0 1]
      :while (< (+ a b) 5)]
  [a b])

=> ([1 0]
 [1 1]
 [2 0]
 [2 1]
 [3 0]
 [3 1]
 [4 0]) ; [4 1] fails the test and so does every pair after that. ([5 0] [5 1]) in this case.

adi07:07:38

P.S. Also I realise why :when and :while are confusing in the original example cited. The choice of example produces identical results and masks the difference between :when and :while. In the original example, both a and b are ranges of contiguous integers. So as long as the sequences are increasing by one in the same direction, the point at which the condition fails is the same in the progression ([1 0] [1 1] [2 0] ....). Slightly modifying the smaller example I chose above shows that :while causes early termination and that :when is a filter clause.

(for [a [1 2 3 4 5]
      b [1 0] ; instead of [0 1]
      :while (< (+ a b) 5)]
  [a b])

=> ([1 1] [1 0] [2 1] [2 0] [3 1] [3 0]) ; early termination

(for [a [1 2 3 4 5]
      b [1 0] ; instead of [0 1]
      :when (< (+ a b) 5)]
  [a b])

=> ([1 1] [1 0] [2 1] [2 0] [3 1] [3 0] [4 0]) ; filtering

Jim Newton08:07:14

I think most people would think that :while would cause the for for doseq to abort. That fact that it only causes the inner-most loop to abort is not intuitive, without understanding the implementation. Now, I understand, but it took a while [pun].

😅 1
hiredman20:06:48

Not that I have seen, to my mind if you understand the mechanical procedure for transforming a for or doseq into calls to mapcat, map, filter, etc, the behavior of take-while falls out of that same model. But I use for a lot and never use :while