This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-05-22
Channels
- # ai (1)
- # announcements (1)
- # babashka (9)
- # beginners (18)
- # calva (19)
- # clerk (136)
- # clj-http (3)
- # clj-kondo (13)
- # cljs-dev (166)
- # clojure (39)
- # clojure-europe (133)
- # clojure-nl (1)
- # clojure-norway (5)
- # clojure-uk (12)
- # clr (1)
- # community-development (6)
- # conjure (8)
- # cursive (13)
- # data-science (1)
- # datomic (26)
- # events (5)
- # fulcro (12)
- # gratitude (3)
- # honeysql (9)
- # hyperfiddle (33)
- # introduce-yourself (6)
- # kaocha (1)
- # lambdaisland (5)
- # malli (4)
- # off-topic (3)
- # rdf (4)
- # re-frame (3)
- # releases (3)
- # scittle (11)
- # specter (2)
- # sql (4)
- # tools-deps (4)
- # vim (10)
@thheller @borkdude I’m assuming you all handle the ES module problem (in Node.js), are you conditionally generating import
or always generating import
?
I am using shadow's :esm
target if I'm using shadow-cljs.
For #cherry which is a little alternative CLJS compiler I just generate import
directly from :require
.
e.g. in Node.js you cannot use require
1. from modules
2. to load other libraries (unless you use some Node.js specific API, module.createRequire(import.meta.url)
For this reason, always import
In #nbb what I do is use dynamic import, always. But this is a CLJS interpreter, not targeted at emitting code, but just executing on the fly. So (require '["whatever-lib"])
executes as a dynamic import of whatever lib and then defines whatever is necessary in the environment regarding referred functions, aliases etc. Each top level REPL expression returns a promise (not automatically wrapped like JS promises) and it's "awaited" result becomes the REPL evaluation result.
dynamic imports don't play well with optimizations done by JS tooling though
#cherry doesn't have much of a REPL yet and this is indeed one of the unsolved annoying ES6 problems.
$ npx nbb
Welcome to nbb v1.2.173!
user=> (do (require '["fs" :as fs]) (js/console.log fs))
[Module: null prototype] {
Dir: [class Dir],
Dirent: [class Dirent],
F_OK: 0,
for squint (similar project as cherry, but no immutable data structures, etc) I've implemented something that would be closer to the CLJS compiler problem since it emits code.
$ npx squint-cljs repl
user=> (require '["fs" :as fs])
user=> fs/readFileSync
[Function: readFileSync]
What I've done there is sneakily compile the REPL expression to a JS module, emit this to a file (which you shouldn't have to, base64 also works) and then dynamically import that file. So there I just emit a top level import
, no dynamic onebut it does not unwrap promises directly:
user=> (js/Promise.resolve 1)
Promise { 1 }
This is something you should be aware of, the default JS behavior automatically chains a promise result within another promiselast time I asked the target :esm
didn't have a REPL, but curious about hearing more from @thheller
no, the import
is only emitted for :target :esm
. which currently doesn't support requiring "new" stuff not part of the build yet
otherwise for regular builds import
is added as a prepend, so the closure compiler never sees it
@thheller What do you mean with "regular build", aside from target esm you don't need import
, right?
@thheller hrm so shadow-cljs just doesn’t support requiring libs in Node.js that for some wild reason declare "module"
only
the way I have done it in shadow is emitting import * as some$alias$var from "x"
. which is added as a prepend and loaded normally via node
(require '["x" :as x])
then maps to that some$alias$var
, but as of now does NOT do a dynamic import for create that var if it not already exists
yep :( I'm considering just doing nested global objects for development in cherry and squint. I mapped "CLJS" namespaces to ES6 modules directly, but this doesn't play well with a REPL for sure
yeah, thats why during development :esm
maps everything in globalThis
to emulate "everything is global" as much as possible
I wish there would be a more dev-friendly ES6 module format with at least mutable bindings, async or not
ok, I thought the library that I was looking at only supports module
but actually that’s not true - it does have a CJS file in the build
dir
right I remember chatting about this two years ago - like any day now - but based on how bad this stuff is I think it will never happen
@U050B88UR #C029PTWD3HR defaults to loading ESM but falls back to CJS via module.createRequire
if nothing else is available
well, yeah, the certainly didn't adopt "top level expression is the compilation unit" idea...
Some tools like next.js (I can't keep up with what they have now again, vite, or so?) do hot-reload ES6 modules, but how they do it is a complete mystery to me
I share the "do nothing" sentiment. Always feels like things should settle around this stuff but they somehow don't
bun, which is a Node.js competitor allows more stuff like sync loading ES6 stuff and mixing import and require, I believe.
☠️ - this is how I feel after reading this thread (fixed). I never realized how repl-unfriendly this es6 has been. Enlightening at least.
@borkdude afaik vite works by keeping track of the module graph and reloading the relevant part of the tree by pretending it’s a new module via some query param. Modules can opt into hot-module-reloading (HMR) see https://vitejs.dev/guide/api-hmr.html
I mean, it's great they solved the problem, but it shouldn't have to be that complex
new thread - package.json
exports
- require
vs. import
field, does shadow-cljs provide knobs for this? @thheller
no support for exports
at all as of now. same nightmare of inconsistent support and undefined uses.
directories, wildcards, ordering matters, ambiguities all over the place. just horrible.
there is a :js-options {:entry-keys ["browser" "main" "module"]}
config option that lets you re-order so "module"
is picked first for example. just not in the context of package.json
"exports"
ok, it also seems way too complex to me, I think I will only solve a couple of cases so that there are some clear examples
community can submit patches to expand the functionality using the refactored code / tests as launching pad
if you add the same simplification that shadow-cljs does for strings you don't really need to do anything
ie. any (:require ["any-string" :as x])
is just passed through, without attempting to resolve for node at all
you only needed to know node_modules
packages for symbols, so that (:require [any-sym :as x])
can be checked if it exists
symbols you'd still need to check. or is your intent to allow (:require [react-select/creatable :as x])
?
for shadow-cljs in any build where it doesn't try to bundle npm dependencies (so node, react-native, etc) it just accepts any string and lets webpack/node/whatever figure it out
I think it is breaking because you could have used a string for a regular CLJS require
for symbols it does the node_modules
presence check, just to complain if they don't exist
well you could disallow that. shadow-cljs doesn't allow that, and I have never seen it used in the wild.
I know these aren't the official CLJS rules, but some tools (shadow, nbb, clj-kondo) rely on string = JS lib, symbol = CLJS lib. Keeps things much simpler
I know, but there's no other way for clj-kondo (static analysis) to know whether you're using a JS lib or not (and hence you can use the alias as a JS object or not)
strings are supported because symbols have character restrictions - that’s it, nothing about JS or not
but clojure namespaces can't have those characters, so it makes sense and everything so much clearer
at this point I’m just pointing out that it’s just very unclear who would be affected so it’s not really worth altering anymore
@borkdude btw, you could probably call into the ClojureScript functions to figure this out
yeah, it could do that, but clj-kondo is pretty unassuming, it works on random .clj(s) files without looking at the rest of your project and incrementally learns more if you throw more code at it
I haven't gotten any complaints about it, I think most people like the shadow convention. But I understand that making breaking changes is a no-go
the other thing to consider is some point we could parse the entrypoints as we do w/ GCL so we validate usages / :refers
etc.
I’m not saying that clj-kondo should do anything - but figuring this stuff does have value on the ClojureScript side
yeah at some point it might. we're scratching the surface with Java analysis (source + byte code) now to improve Java interop linting/completions
@mikerod https://github.com/clojure/clojurescript/commit/8a9d38eea79a24f3f8e43199689dc026ece7c537
I believe this fixes the exports problem for react-select
which seems like a more typical problem
That is nice if it is working for react-select. I’ve seen a few libs in recent times that have this same structure. So that certainly covers the cases I’ve been exposed to so far (I read the threads and know that package.json “exports” can get a lot more messy & we aren’t dealing w/that now).
I like the refactored code org. It makes it clearer where this stuff happens and I just think its easier to read through than the previous.
I just realized there is a release version of CLJS that has this change. I am going to try it out. Been hitting the same react-select style indexing in many node modules I’ve encountered lately.
I’m still messing with it, but so far it’s looking good. @codemirror
modules were another major example of this and we have a bunch of them that now work with CLJS v1.11.121. Quite nice.
I also changed the the indexing to include .cjs
files because this does seem to occur in the wild
w/ GCL adopting await
and async fns I think we should support this stuff out of the box
I get (js-await ...)
but where would you use js-async
? Some experimental projects and ClojureDart have used:
(defn ^:async foo [])
that might be sufficient, honestly I haven’t been following along too closely, so if there’s prior art that is good to know
there are very many cases in the compiler that generate IIFEs. that will break await
expectations
let
being the most prominent I guess, but pretty much all expressions get wrapped in (function() { return ... })();
so async function() { var x = (function() { return await something() + 1 })() }
breaks
Just emitting the keyword won't be enough since expressions that result into compiled fn
expressions (to make JS behave as an expression oriented language) also need to be marked async. So (I think) it's necessary to use the CLJS analyzer to remember if you are in an async function and then propagate that to the emitted self-calling fn helpers. I've been doing this work in cherry.
E.g.:
$ ./node_cli.js --show --no-run -e '(defn ^:async foo [] (let [x (do (js/await 1) 2 3)] 1))'
import * as cherry_core from 'cherry-cljs/cljs.core.js';
var foo = (async function () {
let x1 = (await (async function () {
(await 1);
2;
return 3;
})());
return 1;
})
;
export { foo }
So far the CLJS project (and also shadow) resisted adding async/await support, I'd be happy to contribute or help
So far shadow covered this use case by implementing js-await
as a macro (shadow.utils.js-await..., don't remember the exact namespace) which just compiled into a .then
expression. It was the user-space solution which gets you 90% there (no top level await), but because JS isn't Lisp they have to solve it at the language committee level
async/await in CLJS will be super tedious unless let
is changed to not compile down to IIFEs, I think
it's not that complex, at least not how I did it in cherry, just remember some state in the context and pass it down
e.g. one may end up doing sth like
(defn ^:async f [x]
(let [y x]
(yield ...)
(await ...)))
and with the current implementation of let
, we’d have something like this emitted
(function(){ // <-- inserted by compiler
yield ...; // <-- error (must be inside function*)
await ...; // <-- error (must be inside async function)
}();
also how does recur
work in an async context? async iteration is relatively new to ES (though before es2020 people in JS/TS world missed Promise.all
and would incorrectly use await
in for-each
loops)
yeah that’s right… I remember searching around the net seeing what would be the pros/cons to async/await in CLJS, and so far it seems like a huge refactoring of the compiler would be needed…
with that said, I agree with sufficient effort and refactoring we could likely have async/await in CLJS, but it seems like a huge amount of effort for something that is honestly kinda “solved” by core.async, promesa, and JS interop with promises
imho js-await
should look like this https://clojureverse.org/t/promise-handling-in-cljs-using-js-await/8998
> it seems like a huge refactoring of the compiler would be needed I don't think it will be a huge change, but only a PoC will make this evident I guess
in cherry I only have a few lines of code that are about async/await and I haven't come across an example that didn't work sufficiently. this is why my hunch was that the change to the CLJS compiler wouldn't have to be that big, but I could be wrong
if it is a lot of work - then probably not, but if can be done as a pass then probably we can do this