This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-09-01
Channels
- # announcements (10)
- # aws (1)
- # babashka (19)
- # beginners (104)
- # calva (50)
- # cider (17)
- # cljs-dev (135)
- # cljsrn (56)
- # clojure (240)
- # clojure-dev (4)
- # clojure-europe (19)
- # clojure-nl (2)
- # clojure-uk (7)
- # clojurescript (22)
- # conjure (2)
- # css (1)
- # cursive (10)
- # data-science (1)
- # datomic (60)
- # emacs (2)
- # events (2)
- # exercism (1)
- # figwheel-main (3)
- # fulcro (13)
- # graalvm (5)
- # gratitude (1)
- # inf-clojure (4)
- # introduce-yourself (5)
- # jobs-discuss (21)
- # lsp (36)
- # malli (6)
- # meander (8)
- # missionary (12)
- # off-topic (14)
- # pathom (13)
- # pedestal (10)
- # polylith (42)
- # re-frame (5)
- # reagent (12)
- # reitit (3)
- # releases (8)
- # sci (10)
- # shadow-cljs (37)
- # sql (5)
- # tools-deps (6)
Can anyone tell me where where clojurescript imports its reader? (I mean, it seems like its from clojure directly, but the question is where it does the importing.)
@leif do you mean this? https://github.com/clojure/clojurescript/blob/master/src/main/clojure/cljs/analyzer.cljc#L27-L28
@borkdude maybe better to talk about the details here - what went wrong w/ the require?
@dnolen first let's establish the right context for discussing the problem.
My goal is to create a node script that imports an ES module from vanilla CLJS, to compare if it ostensibly looks the same as what nbb scripts are going to look like.
I am able to require term-size
and some other ES module using shadow's `:esm` build type:
https://gist.github.com/borkdude/7e548f06fbefeb210f3fcf14eef019e0
I have no idea how to accomplish the same thing with vanilla CLJS.
Then there is another thing:
First I used createRequire to implement :require
in nbb. But this disallowed using ES Modules from NPM and some libraries (like term-size) only offer their new versions as ESM. That is the only reason I switched to ESM-based :require again implemented using shadow.esm which is essentially js/import
. Simply because this lets you deal with all libs available in nodeJS and not only the commonJS ones.
require
itself is forbidden in ES modules in Node.js and importing ES modules from CommonJS doesn't work either
but let's just focus on how would one do this in normal CLJS in the ns form: how does one go about making a script using vanilla CLJS like this one: https://gist.github.com/borkdude/7e548f06fbefeb210f3fcf14eef019e0. I can make the nbb scripts look the same, there is enough flexibility
@borkdude right then it probably doesn't work in ClojureScript in the same way it doesn't work in JS since we generate CommonJS
My main concern is compatibility: I want nbb scripts to look like CLJS scripts and not make any breaking changes later on. But it seems so far nobody every really tried to use ES modules from nodeJS via CLJS? This surprises me.
@borkdude but I think you already solved the problem wrt. compatibility? that what it ends up looking like on the Web & for RN
I get the whole "we don't care about ES modules" but if there is no other choice, they are forced upon you now.
I think probably you are overestimating the number of people actually doing Node.js specific stuff
truly popular libraries probably haven't to switched to only ES because of these issues
@borkdude all that said, this createRequire
thing seems like a simple enough workaround - does it work regardless of what you try to require?
I'm not using createRequire
anymore since you can't load ES modules with it, only CommonJS modules from an ES module
well I am using it indirectly. I am using (createRequire "script.cljs")
then I use its resolve method to find library in the relative node_modules directory and feed that into dynamic import.
I think Node.js lets you require CommonJS modules using import as well, which is probably why it works.
There are two things: you can define "type": "module"
in your package.json to define your package as ESM and if you're going to use the import ...
syntax, then you use .mjs
, at least, this is how far my understanding currently goes
note that this is more or less my first encounter with this ecosystem in any serious way, so I could totally be missing something
In my naivety I thought: let's try to replicate the experience of lumo scripts with what I've currently got with SCI and see what happens.
yes, dynamic import returns a promise. but nbb deals with this by evaluating everything async from top to bottom
but this is ok, this solves a different problem I had with SCI for a while: it will enable loading code asynchronously from other places lazily, so I'm not unhappy that I was forced down this path
yeah, makes sense. I'm also curious about @thheller's input here since he has made the :esm
target in shadow already.
:esm
in shadow-cljs does a bunch of stuff and support more usecases - not really interested in using it as a reference
Just .js
files so I'm probably wrong about that .mjs
stuff. This is why I would appreciate thheller's input, he's already figured this stuff out.
$ head out/nbb_main.js
import "./nbb_core.js";
import "./cljs-runtime/shadow.module.nbb_main.prepend.js";
SHADOW_ENV.setLoaded("shadow.module.nbb_main.prepend.js");
import "./cljs-runtime/shadow.esm.esm_import$module.js";
SHADOW_ENV.setLoaded("shadow.esm.esm_import$module.js");
import "./cljs-runtime/nbb.main.js";
SHADOW_ENV.setLoaded("nbb.main.js");
import "./cljs-runtime/shadow.module.nbb_main.append.js";
SHADOW_ENV.setLoaded("shadow.module.nbb_main.append.js");
$ head out/nbb_main.js
import * as esm_import$module from "module";
import { $APP, shadow$provide, $jscomp } from "./nbb_core.js";
const shadow_esm_import = function(x) { return import(x) };
'use strict';
var $sci$core$alter_var_root$$ = function($var_args$jscomp$464$$) {
for (var $args__4824__auto__$jscomp$93$$ = [], $len__4818__auto___58591$$ = arguments.length, $i__4819__auto___58592$$ = 0;;) {
if ($i__4819__auto___58592$$ < $len__4818__auto___58591$$) {
$args__4824__auto__$jscomp$93$$.push(arguments[$i__4819__auto___58592$$]), $i__4819__auto___58592$$ += 1;
} else {
break;
you don't need await import ...
, only when you are using the import(...)
function I think (aka dynamic import)
> Node.js will treat .cjs files as CommonJS modules and .mjs files as ECMAScript modules. It will treat .js files as whatever the default module system for the project is (which is CommonJS unless package.json says "type": "module",).
note that this kind of build can also benefit the deno stuff (although that sounds a bit speculative)
I think you can also emit import * as something from "./nbb_core.js"
; and then do something with something, which is probably similar to what CLJS is doing now with require
FWIW the only thing shadow-cljs really does is add glue code to make import
work. so it compiles CLJS as regular code with (:require ["thing" :as x])
setting an alias. eg. module$thing
, so any use of x/foo
gets turned into module$thing.foo
. after cljs compilation is done and "packaging" starts shadow-cljs basically just adds import * as module$thing from "thing"
via a prepended string
all the extra glue it does is just to work around quirks in the closure compiler not being configurable enough when I did this
it still isn't. one hurdle with async import()
will be that closure will want to optimize it (or others in case of :target :bundle
)
@thheller so you want to support bundling for Node.js? Is there are any reason for that? or you were trying to make this work for both Web and Node.js?
I wanted this for platforms that don't support commonjs, see https://clojureverse.org/t/generating-es-modules-browser-deno/6116
so the point really is supporting the standard, assuming everything moving to it over time
@thheller but is there any requirement for this to work for the Web given all the bundlers solve this for you?
many bundlers starting going ESM first (eg. webpack v5). so the whole interop between commonjs <-> ESM will get much stricter than it used to be
and as such more and more people moving over to ESM. until we get to the point where its mostly ESM and commonjs is the problem case
works fine. in development its kinda fake ESM. just puts everything in the global scope and operates on globalThis
I think @thheller would be an ideal candidate for this issue, much more ideal than me ;)
the thing :esm
does "more" than :bundle
is the usual npm processing and actually bundling the packages
It's not like I have seas of time either, but to re-iterate, the main issue I'm after is: nbb should behave the same as (future) CLJS with regards to this stuff. If I can help ensuring that, then I would give it some time.
but with :js-provider :import
it is basically :target :bundle
, just import
not require
@borkdude if all you care about is compatibility - then there's nothing more to do - somebody else can work on this as well
the impact is too small for me or anyone else at the moment - I don't hear people clamoring for this
at least I was able to verify how it works with the :target :esm
in shadow-cljs already: the nbb script looks identical there
and at least people have 1 option to migrate from interpreted scripts to compiled scripts with shadow, CLJS could follow later (and hopefully the .cljs code stays the same)
I just care a lot of about not making breaking changes, that's why I brought this all up. I'd be ok if CLJS waited a bit on this issue as there are options now.
I could add some docs to nbb how people can migrate to shadow if they want more perf out of their scripts.
you could provide a shadow-cljs run nbb.shadow my-script.cljs out/my-script.js
(or I could)
yeah, not all nbb projects have just one single script though, there's already one with 3 and a util namespace: https://github.com/chr15m/c64core
@thheller Ah, okay, so it just uses the parser of the version of clojure its written in then?
no. tools.reader is a separate library https://github.com/clojure/tools.reader