Fork me on GitHub
#cljs-dev
<
2019-08-30
>
markw23:08:39

Hi All - I have a question about the implementation of LazySeq, actually several questions, but I'll stick with the main one first: Since the lazy-seq macro expands into a LazySeq deftype (which wraps the body in a thunk) at macro-expand time, it is not until runtime afaict when first or rest is called on the LazySeq that the the body is evaluated. However, when it is evaluated, the recursive nature of the body results in another call to lazy-seq, but this is after compile / macro-expansion time, right? How does that work?

markw23:08:16

Calling first or rest results in a call to seq on itself, which finally calls .sval. Inside the .sval call (and this is now runtime) the thunk bound to the deftype member fn is called resulting in a new call to the lazy-seq macro. It seems there is some interleaving of macroexpansion and runtime, or I'm misunderstanding the implementation.

andy.fingerhut23:08:54

Take my comment with a big grain of salt, since I am much less familiar with ClojureScript than Clojure/Java, but at least in Clojure/Java when you do a defn containing one or more uses of macros, those macros are expanded while compiling the entire defn form, not at run time.

andy.fingerhut23:08:58

i.e. at run time, there should be no remaining occurrences of lazy-seq or any other macro call.

markw23:08:25

I agree w/ your second statement, not sure how the first related though

markw23:08:57

What's confusing me is that at compiletime, (lazy-seq some-body) gets macroexpanded into the LazySeq deftype

andy.fingerhut23:08:03

Part of your original statement was "resulting in a new call to the lazy-seq macro". I do not think that is true.

markw23:08:12

e.g. (new cljs.core/LazySeq nil (fn [] ~@body) nil nil))`

markw23:08:20

doh.. (new cljs.core/LazySeq nil (fn [] ~@body) nil nil))

markw23:08:49

that's the macroexpansion right there

markw23:08:21

so that's what you have at runtime (i think), right?

markw23:08:16

and when you call any ISeq function on that, there is another call to lazy-seq that occurs, which appears to be at runtime

andy.fingerhut23:08:25

Every occurrence of a lazy-seq macro should be replaced by an expression like its body at compile time, yes, and what you pasted above is what I see in ClojureScript source code as its body.

andy.fingerhut23:08:59

That body contains no occurrences of the lazy-seq macro, though, so how would lazy-seq be involved at run time?

markw23:08:11

the body typically would return a recurisve call to a procedure which calls lazy-seq

markw23:08:26

for example: `(defn my-iterate [f x] (lazy-seq (cons x (my-iterate f (f x)))))`

andy.fingerhut23:08:40

A procedure which, at compile time, had all of its occurrences of lazy-seq replaced with the body of lazy-seq, too.

andy.fingerhut23:08:34

You could write my-iterate without using lazy-seq at all, just using the body of lazy-seq yourself, and you should get the same effect as if you used lazy-seq

andy.fingerhut23:08:24

Unless I have messed something up in my copy and paste (possible), you could write my-iterate like this:

(defn my-iterate [f x]
  (new cljs.core/LazySeq
       nil (fn [] (cons x (my-iterate f (f x))))
       nil nil))

andy.fingerhut23:08:02

and that is what the compiler sees and actually compiles, after it has done macro invocation

markw23:08:50

OK yes i'm with you

markw23:08:00

so now we agree macroexpansion is over, and ew have a lazyseq type

markw23:08:06

well.. LazySeq

andy.fingerhut23:08:18

right. Nothing named lazy-seq remains after macro expansion

markw23:08:23

oen of the fields of that LazySeq is fn which is bound to that body

markw23:08:37

bound to (fn [] (cons x (my-iterate f (f x))))

andy.fingerhut23:08:57

or slightly more precisely, the function that expression evaluates to.

markw23:08:58

now we call first, rest, or anything else... and it evals that body

markw23:08:10

it calls the function

andy.fingerhut23:08:10

it calls that already-compiled body.

markw23:08:48

which returns (cons x (my-iterate f (f x)))

markw23:08:01

and my-iterate expands into... a call to lazy-seq

andy.fingerhut23:08:14

there is no lazy-seq remaining anywhere at this point.

dpsutton23:08:32

my-iterate has been compiled into javascript at this point. its not expanded

andy.fingerhut23:08:35

there is the function that looks like what I pasted above.

markw23:08:38

ok, what happens when (my-iterate f (f x))) is evaluated

andy.fingerhut23:08:04

It creates and returns a new LazySeq object.

markw23:08:24

ok I think I get what you're saying now

andy.fingerhut23:08:05

(after first evaluating the (f x) part of the expression)

markw23:08:43

I think this is a nuance of macroexpansion i've not encountered...

andy.fingerhut23:08:12

Think of them as a code-writing convenience, and they are all gone by the time "the real compiler" sees the ClojureScript code.

andy.fingerhut23:08:49

not exactly accurate, since macro expansion is part of the compiler, but perhaps useful in thinking of the distinction of macro expansion time.

markw23:08:01

I understand the concept of macroexpansion time and the fact that all macro calls are replaced with their expansions by the time runtime comes around

markw23:08:48

this is odd because at runtime there is a call that uses some already expanded code.. ors omething

markw23:08:54

need to think on this a bit

markw23:08:20

the way I have been thinking about it is that any macros the compiler can see, in source text, get replaced... and then you're done and on to runtime

andy.fingerhut23:08:21

I completely understand if that plus recursion can throw one for a mental loop, though.

markw23:08:24

but that's not quite the case there

andy.fingerhut23:08:30

It seems to me to be the case here. Macros are expanded at compile time, the post-macroexpanded code is compiled to JavaScript, and then you are completely in the JavaScript runtime.

andy.fingerhut23:08:33

The post-macroexpanded code is a function that returns a new object. One field of that object happens to be a reference to an already-compiled function, in this case an anonymous one that happens to call the outer function.

markw23:08:29

I've got it

markw23:08:37

I don't know why I was so confused

markw23:08:56

the second call to my-iterate was already macroexpanded at that time

markw23:08:51

Wow that was a bit of a headsplosion for me

markw23:08:00

thanks for taking the time to patiently explain it

andy.fingerhut23:08:25

Cut yourself a little slack -- I think it has that effect on lots of people the first time they understand it (and many just use it without trying to pick it apart).

markw23:08:59

there is always the draw over time to try and pick apart clojure/script source for me... hah

andy.fingerhut23:08:13

It reminds me of the saying "In order to understand recursion, one must first understand recursion"