Clojurians Log v2

not familiar or forgot, would have to go through the history

If I'm not completely off the entire thing composes, so that no super complicated analyzer passes are needed. it likely won't be optimal and do many more assignments than necessary, but :advanced can take care of that for us. For development it may actually be beneficial because of lots more opportunities for breakpoints in JS debugger

cljs.core.foo = function cljs$core$foo(bar) {
  let expr_if_121814;
  let expr_do_121815;
  cljs.core.prn.cljs$core$IFn$_invoke$arity$variadic(
    cljs.core.prim_seq.cljs$core$IFn$_invoke$arity$2(
      [new cljs.core.Keyword(null, "foo", "foo", 1268894036)],
      0,
    ),
  );

  expr_do_121815 = bar();
  if (cljs.core.truth_(expr_do_121815)) {
    let expr_do_121816;
    cljs.core.identity(1);

    expr_do_121816 = 1;
    expr_if_121814 = expr_do_121816;
  } else {
    expr_if_121814 = 2;
  }
  var x = expr_if_121814;
  return x;
};

(defn foo [bar]
  (let [x (if (do (prn :foo) (bar)) (do (identity 1) 1) 2)]
    x))

lots of lots of extra symbols, but no more IIFE in if/do

this could probably be optimized: > expr_do_121816 = 1; > expr_if_121814 = expr_do_121816; as a direct assignment to the expr_if one, by forwarding the assigntarget to the do AST instead. With this change in place, are you able to spin up the CLJS node REPL and run the tests? I'd be interested in seeing the change

doesn't quite work yet. will upload the branch after cleanup

I tested if Closure optimizes the IIFEs, it doesn't

(defn ^:async foo []
  (let [x (let [y (await (js/Promise.resolve 1))] y)]
    (js/console.log (inc x))))

(foo)

(async function() {
  var $x$jscomp$662$$ = await async function() {
    return await Promise.resolve(1);
  }();
  return console.log($x$jscomp$662$$ + 1);
})();

(require '[cljs.build.api :as b]
         '[clojure.java.io :as io])

;; Create test source
(let [src-dir "/tmp/cljs-test/src"
      src-file (io/file src-dir "demo/core.cljs")]
  (.mkdirs (.getParentFile src-file))
  (spit src-file
        "(ns demo.core)

(defn ^:async foo []
  (let [x (let [y (await (js/Promise.resolve 1))] y)]
    (js/console.log (inc x))))

(foo)"))

;; Build by passing the source FILE directly (not using :main)
(doseq [opt [#_:simple :advanced]]
  (b/build "/tmp/cljs-test/src/demo/core.cljs"
           {:output-dir "/tmp/cljs-test/out-adv"
            :output-to (str "/tmp/cljs-test/" (name opt) ".js")
            :optimizations opt
            :pretty-print true
            :pseudo-names true}))

(println (slurp "/tmp/cljs-test/advanced.js"))

yeah didn't expect it for async fns. I don't think that would be technically correct since the behavior changes.

let's see if it does for non async

lol, it optimizes this:

(defn foo []
  (let [x (let [y 1] y)]
    (js/console.log (inc x))))

(foo)

console.log(2)

A quickwin would be to check if there's an await happening in the let (or anything else that needs IIFE) and if not, just emit a non-async IIFE.

Yea that's why I said it's already "insanely optimized" :p Ho yes if async is propagating from the fn def it might be a bit too much. Propagation from the await itself to the top should be more than enough for most real-world applications.

to the top of what?

async context (fn def)

literally what you said

I can't parse your last utterance, it seems you said: if A, that's bad. A is more than enough.

from the async (fn def) to the bottom (all IIFE recursively) => your current implementation, if I understood correctly from the await (js-await) to the top (fn def) => will await only IIFEs that have await calls under

I don't understand why you put it like that, but yeah, the parts after => make sense to me

it was from an AST point of view (top/bottom of the tree), sorry it that wasn't clear

only the scope (let ... (await )) matters, not, all the way up to the function body

I should take a look at your current implementation instead of trying to understand what you've done from your words.

Words are difficult I guess ;). What I mean: for the optimization to turn an async IIFE originated from a let into a non-async one, we should track await usages in the let (bindings and body). This is what I mean with: only that scope matters. Mentioning "top of the function" confuses things for me.

I thought it would also apply for others IIFE too (do, case, ... idk all of them)

there is so much weird code in cljs.analyzer. every time I look at it I want to clean it up, but that just distracts me from what I'm actually trying to do 😛

this is going to require a lot more work than I anticipated, so will continue some other time. not sure its the correct path either, so threw away the branch. failed quite hard at compiling cljs.core, so not all that useful.

the core.async go macro does SSA which is even stricter than ANF - we could maybe just use that? ;)

well the idea works. I think at least, just not the shortcut I was trying to take 😉

from what I can tell the concept of an expression needs to be removed everywhere. or rather rewritten always.

yes, all expressions need to be transformed to assignments of simple values. ANF

perhaps a good path to write a clj -> webassembly compiler ;)

then it would be possible to emit block statements yes. But still a lot of work to do in analyzer for converting to ANF

(I haven't looked at webassembly, let me not side-track this discussion with it, sorry)

if only if JS had block expressions, our problems would be solved

that would certainly make things much easier yeah

there are these TC39 do expression proposal babel plugins: they emit IIFEs

I wonder how they would emit async/await, let me check that

async function foo() {
    let x = do { 
      foo();
      let y = do { await bar() };
    };
}
=>
async function foo() {
  let x = await async function () {
    return foo();
    let y = await bar();
  }();
}

that seems incorrect? why is there a return?

I think it's a bug (and it does even feel familiar to me since it happened to me in squint as well lol) They do optimize this:

async function foo() {
    let x = do { 
      foo();
      let y = do { bar() };
    };
}

they also optimize do that doesn't introduce any locals to comma-expressions, but that's a very specific optimization

https://github.com/babel/babel/blob/main/packages/babel-plugin-proposal-do-expressions/src/index.ts implementation doesn't seem to be straightforward

JS is an imperative language with control flow, which makes stuff even more complicated than CLJS. We live in a relatively simple language

if you enable https://babeljs.io/docs/babel-preset-env#forcealltransforms it gives different results than IIFEs

I think it's because it will transform async/await into some pre ES6 stuff which is way more complicated

for async yes but IIFEs sometimes got transformed to this syntax:

let x = do {
    let y = 40;
    let z = 2;
    foo(y + z);
};

var y, z;
var x = (y = 40, z = 2, foo(y + z));

that's a comma expression, mentioned it above already

ah you mean, when you introduce locals

yeah, I guess it hoists.

hmm, which specific setting is this? could be interesting

idk, now it works without the link I sent. https://babeljs.io/repl#?config_lz=N4IgZglgNgpgdgQwLYxALhAJxgBygOgCsBnEAGhB22JgBdS0BtRkeAN3NFoUwHM6GwAL4UkAewAmAV1gMwCKDSEBdMi2Ld-AWgAMnEBJgBjMZgS1TxAGoxMxCGLjoQAJh0uAzFoCM38pQgcGCgIOBgABUwxHDFiBWcACwQjAGsQFTUQDQRtPzJQQxMzCzsbOwcnDDdPHzyAoJCwyOjY-Iwk1PTVdU0YLRd9QtNzSzL7R2dqr19_HEDg0IiomLioROS0jJ6cvo9B42GS61txytd3abq5hsXmlbaQDs3lVSyxKUwjGAAVAE8g5ziaSwdJAA&code_lz=FAGwpgLgBAHlC8UAmB7KBvYVtXNAnglACwAMA3FjoQNTJqY5O6RQBeRATJczh3QEYeOAL6UxwIA&version=7.28.6

and yes it's hoists, adding some name conflicts confirm

$ clj -M:cljs-repl -m cljs.main -re node
ClojureScript 0.0.1213971316
cljs.user=> (defn ^:async foo [] (let [x 1 y (let [x 2] (inc x))] (await (+ x y))))
#'cljs.user/foo
cljs.user=> (str foo)
"async function cljs$user$foo(){\nvar x = (1);\nvar y = (function (){var x__$1 = (2);\nreturn (x__$1 + (1));\n})();\nreturn (await (x + y));\n}"

this avoids async for: var y = (function (){var x__$1 = (2);\nreturn (x__$1 + (1));\n})()

which can then be optimized by Closure

After this optimization:

(defn ^:async foo []
  (let [x (let [y 1] y)]
    (await 1)
    (js/console.log (inc x))))

(foo)

(async function() {
  await 1;
  return console.log(2);
})();

although a more likely case will actually do an await somewhere "deeper", so with IIFE

true, but many sub-expressions could be clean

I implemented it like this: https://github.com/clojure/clojurescript/commit/ae967ca49f9941f560b5a4a25e5ac4c20c46dbd3

(easy to extend to the other IIFE-producing forms)

I thought I undid the whitespaces changes. If I'm going to add this to the patch, I will. This is a different branch just in case

nice!

I'd be very surprised too see any benchmark giving IIFEs significantly slower than any "exploded/inlined" version of the same code. They are already insanely optimized. It might add some optimizations too as they have a smaller local scope, better type consistency between calls, and all the craziest micro optimizations you could think of. Plus it adds more complexity on the upper function (and so can break some optimizations too). Just a guess tho.

Its been a long time since I looked at the code, but I don't think its going to be that hard. We already do some of it in the invoke case where IFn is considered and the arguments are bound first.

I would be surprised if it isn't faster, especially in the context of async/await/generators. let alone all the extra code being saved. :advanced does remove some IIFEs but not all.

I might have some time next month to dig into this a bit, but I have been meaning to do that for years, so we'll see I guess.

@thheller Do you mean this thing in parse-invoke*?

(analyze env
          `(let [~@bindings]
             ~(with-meta
               `(~(analyzed (if bind-f-expr? f-sym f))
                 ~@(if bind-args? arg-syms args))
               {:tag tag})))

yes, or the https://github.com/thheller/shadow-cljs/blob/4973aea864f16c34368d3a3bc1fa62ceb46ef866/src/main/shadow/build/cljs_hacks.cljc#L1046-L1082 I wrote in shadow-cljs

I might miss something, but how a function call can slow down any async/await code that is extremely slow by ending the current microtask anyway?

the issue is creating nested IIFEs "get infected" and turn async (and add another await), which I'm fairly certain creates an entire new promise/microtask, which just isn't needed

you can see an example of this here: https://squint-cljs.github.io/squint/?src=gzip%3AH4sIAAAAAAAAE9NISU3LU4izSiyuzEtWSMvPV4iO5VJQ0MhJLVGIrkDQWcW6ieWJmSUgln5AUX5uZnGqXlFqcX5OWaqCsaYmSBMCaGTmJStUwERhPE0ATfVt7m0AAAA%3D

totally makes sense to do some basic benchmarks to see if it actually matters, but my instinct says yes.

also don't want to be doing the same stuff for generators, would be nice if it just worked

I don't see a significant speedup here: https://squint-cljs.github.io/squint/?src=gzip%3AH4sIAAAAAAAAE61PSQrCMBTd9xQPVwniUN0p9Qzui0ra%2FkKkTSCJWhHvLmnUDm59m58%2F5A2soBLSkbGIlx48ilhBpcJxI%2Bxd5Si1RnqIAFaRQ9p09Wxn4iak86%2FF3uhaWpobsrq6Etac%2B08dmFQ5ms%2F003k59pVSbyUnawIrtK8W6Sk47PP1xEuteYvgHJkwrSO6igqTwMw4kh0efUM%2BRRNvf0YrJAjMo0hszQfXTYzEn08xZDHkLkb59Xjz7DeTf2TPhAnZX6G%2FyzDIAQAA&repl=false

not sure what you mean with generators

never trust any benchmark that used eval. that is always unoptimized code

yea ok for nested code blocks, any additional await costs a lot

I generated a benchmark from the above code in pure JS. for iters = 10000000:

$ node /tmp/foo.mjs
bar (flat async): 615.81 ms
foo (extra async IIFE): 927.87 ms
foo (extra async IIFE): 889.02 ms
bar (flat async): 580.44 ms

would also be interesting to see without async?

as for generators I mean that it just works out of the box with no modifications in the compiler. your branch has a lot of (when async ...) don't want to be adding (when generator ...) as well 😛

you mean, with A-normal form in place

assuming all of this is actually doable, but I see no reason why it wouldn't be

yes.

without async/promise:

$ node /tmp/foo.mjs
bar (flat async): 297.14 ms
foo (extra async IIFE): 292.80 ms
foo (extra async IIFE): 282.46 ms
bar (flat async): 283.65 ms

would it be easier to implements variables scopes as blocks instead of hoist variables?

var foo = async function () {
    const x1 = await (async () => {
        const x2 = await Promise.resolve(3);
        return x2 + 1;
    })();
    return x1 + 1;
};

var foo = async function () {
    let x1;
    {
        const x2 = await Promise.resolve(3);
        x1 = x2 + 1;
    }
    return x1 + 1;
};

that's what we are discussing right

pretty sure that extra block does nothing useful, but could be done

(yes i got back to original subject as for async it can be optimized)

oh that block. yeah, it might actually be useful to some optimizer that sees that x2 is out of scope or so

it might only change how the code is emitted, but I don't really know how cljs compiler work

I actually tried it today but it's not sufficient to just change the emitter I think (or you'll have to modify the envs recursively of existing AST nodes, which is a sign that it's better done in the analyzer probably)

definitely needs support from the analyzer side yeah

IMHO the emitter shouldn't do any "analysis" work anyway. just leftovers from before the separation for the cases it still does.

makes sense

to hoist variables, we do need to always have a "statement" boundary somewhere and collect all the assigment targets there. This is more complicated than it seems due to nested expressions, etc. It's doable but you need analyze stuff more than once or make it an analyzer pass even?

or you could pass along an atom and update it along the way

btw, I also benchmarked the old core.async vs the async/await one and async/await based go macro was slower