Fork me on GitHub
#shadow-cljs
<
2023-03-23
>
scknkkrer00:03:53

Hello fellas, do you know any document or guide to host front-end and back-end in one project with shadow-cljs? I was thinking to use Macchiato for back-end. Actually, I’ve been trying out some templates and POCs but, no luck, they all fails. Any suggestions?

thheller05:03:56

what question do you have? you do the frontend as usual, you do the backend as usual. really nothing changes. you just don't use the shadow-cljs :dev-http to serve your .js files, you use your backend. thats it.

thheller05:03:23

> they all fail

thheller05:03:26

what exactly fails?

hifumi12300:03:09

I believe District0x on GitHub has several projects that are pure CLJS on both front and backend. But they mix deps.edn with shadow-cljs IIRC

cjohansen08:03:54

Hi, I maintain a library that has a JS dependency. The dependency is inlined in my source code like this: https://github.com/cjohansen/dumdom/blob/master/src/deps.cljs This seems to not work for shadow-cljs users. Is there some way to make this work with shadow while still working as it does today with figwheel/cljs-build?

hifumi12308:03:17

instead of deps.cljs try declaring an NPM dependency on snabbdom instead

hifumi12308:03:48

in general, shadow-cljs works against CLJSJS and manually injecting dependencies this way — the best chance of getting support on shadow-cljs is to declare an npm dependency and fall back to CLJSJS for figwheel users

hifumi12308:03:04

while there are ways to get stuff like CLJSJS libraries into shadow-cljs, it is usually not recommended for several reasons: • you have full access to npm already • npm libs go through simple optimizations • externs inference eliminates the need for handwritten externs almost all of the time using npm dependencies via cljsjs is usually a “last resort” and involves shimming the namespaces from the shadow-cljs side

thheller08:03:13

I would recommend not making it a foreign lib in the first place

cjohansen08:03:26

Ok, but I don’t have exhaustive overview over my options 😅

thheller08:03:38

you can fairly trivially rewrite this file to a goog.module

thheller08:03:51

then it'll go through :advanced with the rest of the code

thheller08:03:04

which a) eliminates the need for externs, and b) benefits from DCE

hifumi12308:03:09

@U9MKYDN4Q Is there a way to use NPM dependencies with figwheel besides your current route? I would expect that is the “path of least resistence” here.

thheller08:03:57

with goog.module("dumdom.snabbdom");

thheller08:03:20

then at the bottom the

thheller08:03:36

become all just exports.array = array, so exports instead of snabbdom

thheller08:03:44

that should be it

thheller08:03:05

in the CLJS code you then just (:require [dumdom.snabbdom :as snab]) and snab/array etc

thheller08:03:16

no messing with js/ foreign-libs or whatever

thheller08:03:19

it just becomes a normal source file

cjohansen08:03:28

very cool, I did not know this

thheller08:03:29

which you also place next to your regular CLJS files

cjohansen08:03:46

I won’t need the config file either?

thheller08:03:49

so in src/dumdom/snabbdom.js

thheller08:03:55

no config file needed no

cjohansen08:03:07

awesome, thanks a lot! I will give it a shot

thheller08:03:14

should work out of the box

cjohansen08:03:32

do I still need the externs file?

thheller08:03:13

well, depends. if you were defensive in your CLJS code and use goog.object to access stuff maybe

thheller08:03:16

otherwise no

thheller08:03:22

regular interop will just work

thheller08:03:47

also don't need the .min.js since it'll go through :advanced

thheller08:03:18

(note that I never actually tried this with figwheel, no clue if it actually works)

thheller08:03:28

it should work in theory, it does work in shadow-cljs 😛

cjohansen08:03:35

I will try that 🙂

thheller08:03:40

after the goog.module you can also goog.module.declareLegacyNamespace();

thheller08:03:14

optional but lets you access dumdom.snabbdom in the browser console during dev. might be useful

cjohansen08:03:28

definitely sounds useful

cjohansen08:03:08

once I moved the file to match the namespace provided to goog it works perfectly with figwheel 👍

👍 2
thheller08:03:27

good to know 😉

thheller08:03:02

make sure you verify :advanced too, since that was not previously done to the code

cjohansen08:03:33

I’ll do that before pushing it. Gotta pop in a meeting, but will let you know how it works later 👍

thheller08:03:20

in case it doesn't work just adding :npm-deps {"snabbdom" "the-version"} to the deps.cljs should be enough to make it work with shadow-cljs

thheller08:03:33

it'll just ignore the :foreign-libs and use the npm package instead

thheller08:03:50

I see some code that might not survive :advanced, but definitely try it first

cjohansen13:03:51

@thheller Finally got around to trying a advanced build with this solution. Unfortunately, cljs-build doesn’t seem to understand the module No such namespace: dumdom.snabbdom

cjohansen13:03:11

Which seems weird, as I thought figwheel used it under the hood

thheller15:03:36

hmm yeah that should definitely work. or are you maybe on an ultra ancient cljs version with cljsbuild?

cjohansen17:03:37

Hah, could be. Will check 👍

cjohansen21:03:45

That was it. Got the build going, but ran into this runtime: TypeError: t.getAttribute is not a function. There are only two calls to this function - can I extern my way out of this @thheller or am I better off with reverting to deps.edn with the npm-deps?

thheller21:03:17

I don't know. need more context for the error. getAttribute is not getting renamed there, so likely a different issue

cjohansen07:03:46

I guess. But just to try: how would I use the old externs with the goog module? The externs goes var snabbdom = { ,,, } but I’m assuming they need to target something else now?

thheller07:03:35

externs are just a workarround. usually better to just figure out the cause first, even if the fix might be externs

thheller07:03:09

externs also don't really matter "what" they are on, so var snabbdom is fine.

thheller07:03:44

as in it tries to look these up as strings, but they were renamed, thus t ends up as nil

thheller07:03:53

if you are using shadow-cljs to test this I suspect these to be the problem

thheller07:03:40

so if you are using shadow-cljs to test create a externs/your-build-id.txt

thheller07:03:50

with just these, one per line, without the quote

cjohansen07:03:31

I’ve only tried it with advanced compilation with cljsbuild so far

thheller07:03:42

or just add the full externs. shadow-cljs via :compiler-options {:externs ["path/to/externs.js"]}

cjohansen07:03:06

ok, thanks will try those

thheller07:03:14

can't remember how to add externs for cljsbuild

cjohansen07:03:27

I have that in place 🙂

thheller07:03:28

would help to have a stacktrace

thheller07:03:44

know where t.getAttribute failed, just to see some code for context

thheller07:03:10

there are only two uses in snabbdom, they all look safe

cjohansen07:03:34

return function(t, v) {
        var y;
        const z = [];
        for (y = 0; y < q.oh.length; ++y)
            q.oh[y]();
        if (void 0 === t.Gc) {
            y = t.id ? "#" + t.id : "";
            var A = (A = t.getAttribute("class")) ? "." + A.split(" ").join(".") : "";
            t = w_(r.tagName(t).toLowerCase() + y + A, {}, [], void 0, t)
        }
        y_(t, v) ? l(t, v, z) : (y = t.oa,
        A = r.parentNode(y),
        c(v, z),
        null !== A && (r.insertBefore(A, v.oa, r.nextSibling(y)),
        g(A, [t], 0, 0)));
        for (y = 0; y < z.length; ++y)
            z[y].data.me.insert(z[y]);
        for (y = 0; y < q.post.length; ++y)
            q.post[y]();
        return v
    }

cjohansen07:03:58

this is the context from the stack trace, looks like toVNode

thheller07:03:08

try with :pseudo-names true

thheller07:03:36

but yeah it does

cjohansen07:03:25

externs got me another error at the very least 🙂

cjohansen07:03:34

I suspect the pseudo names gave me another issue, trying another compile without it

thheller07:03:42

might be safer to just go with npm-deps then, sometimes these libs just don't survive :advanced

thheller07:03:59

if pseudo-names changes behavior that usually points to extern problems

cjohansen07:03:05

Failed to set a named property on 'DOMStringMap': 'data-$dumdom-id$' is not a valid attribute name.

cjohansen07:03:14

Yeah, looks like a pseudo-name issue

cjohansen07:03:17

:drum_with_drumsticks:

cjohansen07:03:31

Works perfectly with the externs and without pseudo names 🎉

cjohansen07:03:59

Awesome, thank you so much! I will try to run it with shadow later today, but hopefully this works now.

thheller07:03:20

THIS IS NOT THE FIX!

thheller07:03:37

trust me. it just happens to work. there is no guarantee this won't break randomly

thheller07:03:54

> 'data-$dumdom-id$

thheller07:03:11

this means it is using a renamed property name as some sort of identifier

thheller07:03:44

this is guaranteed to break. I very strongly recommend NOT using goog.module then, or finding where this is coming from and changing the code

thheller07:03:06

if :pseudo-names break anything that is a bug and means code cannot be relied upon

cjohansen08:03:13

Alright, I'll try to track it down then.

cjohansen08:04:29

@thheller you were right, of course - this didn’t work 😅 I’m now trying to find a way to make this work in both figwheel and shadow and both simple and advanced optimizations… The previous inline JS source as a foreign-lib works with figwheel. I’m looking at this https://shadow-cljs.github.io/docs/UsersGuide.html#publish-deps-cljs but it is unclear how I would go about using the library after doing this? js/snabbdom?

cjohansen08:04:07

I’m starting to think my time would be better spent just writing the damn thing in ClojureScript 😂

thheller08:04:52

first of all you never use js/snabbdom at all, not in fighwheel not in shadow

thheller08:04:18

in both you just use (:require ["snabbdom" :as snab]) and then (snab/whatever ...)

cjohansen08:04:22

seems like [snabbdom :as snabbdom] actually worked

thheller08:04:28

so, just a normal regular alias

cjohansen08:04:37

yep. I was a little quick - I got it working after a clean

thheller08:04:58

for figwheel it just needs to proper global declaration in :foreign-libs

cjohansen08:04:14

Woot! I got it working in all quadrants 😄

hifumi12308:03:35

This is probably a non-sense question, but is it possible to have a root module that does not depend on cljs.core? My use case is to have a small module that attempts authentication. As part of the authentication process, the user is redirected to an external identity provider, and I want to avoid having to make the user download some 400 kb bundle twice. The authentication code is very small and makes up a tiny fraction of this bundle, so loading that twice is not a big deal to me. I want to lazily load the app when authentication is successful

thheller08:03:07

there is nothing implicit about cljs.core being in the "root". so yes

hifumi12308:03:40

Sweet. Is it okay to use dynamic ES import to load the app module, or should I look into using closure-library’s ModuleLoader?

thheller08:03:59

if you use :target :esm that works, otherwise no

👍 2
pmooser09:03:03

@thheller A couple of months ago I was trying to get automerge 2 working with shadow-cljs, via webpack (as an external js provider), but I was never able to actually correctly import anything. I've created a minimal project that just attempts to do this - would you be willing to take a look?

pmooser13:03:37

Can I send you a zip? Or a tar.gz ?

thheller15:03:23

either works

pmooser15:03:40

I will send it directly on slack - gmail doesn't like it for some reason.

pmooser15:03:59

This is the archive

pmooser15:03:49

You'll have to do an npm install in that directory (I removed the node modules to make it smaller), and if you look at the "run" script, you can see what I was trying to do to package it up for webpack and shadow-cljs.

pmooser15:03:26

Thanks for any time you spend looking at this.

thheller15:03:50

(ns minimal.core
  (:require
   ;; automerge docs do:
   ;;   import * as Automerge from '@automerge/automerge'
   ["@automerge/automerge" :as am]
   [shadow.cljs.modern :refer (js-await)]))

(defn init
  []
  (js-await [^js mod am]
    (js/console.log "init result" (.init mod))
    (println "Initialized!")))

thheller15:03:50

the import returns a promise, don't know why but it does

thheller15:03:06

check with (js/console.log am).

pmooser16:03:31

Oh, hmmm. I hadn't realized that.

pmooser16:03:43

I suppose that means it just utterly, completely doesn't work?

thheller16:03:20

I don't have a clue how you use any of this, but with the js-await you get what you'd otherwise get. whether that makes it work or not I cannot tell

thheller16:03:34

but the init call succeeds just fine

thheller16:03:54

what you do with it after I don't know

pmooser16:03:59

Interesting - we have js-await in cljs ?

thheller16:03:14

see the snippet I posted above?

pmooser16:03:34

I'm sorry, I missed that part. I see it now !

pmooser17:03:22

I think that will give me enough to go a bit further and see if maybe this stuff will work .. so thank you so much for your time and help !

👍 2
thheller17:03:30

I think it is the experiments: { asyncWebAssembly: true }, in webpack.config.js

pmooser17:03:46

Ah, interesting. That's there because they (the automerge people) said it was required. But interestingly, none of their js code does anything with promises to refer to that import.

pmooser17:03:01

I will follow up with those guys next and see if they have anything interesting to say about it ...

pmooser11:03:48

@thheller If I were going to use js-await like you showed me, do I have to do that in every function that accesses the import ? I think so ?

pmooser11:03:16

The webpack docs say that for their async modules: > Importing them via import is automatically handled and no additional syntax is needed and difference is hardly notice-able. > Importing them via require() will return a Promise that resolves to the exports.

pmooser11:03:58

Is there anything we can do via cljs that maps to that notion of import vs require ?

thheller15:03:30

> every function that accesses the import ?

thheller15:03:30

no, you only access the import once. and after that only the await'd result

thheller15:03:31

shadow-cljs does not yet support "managed" async imports

pmooser07:03:14

@thheller When you say after the first time, you only access the await'd result ... do you mean that once you access it, you store a reference the module or whatever (in an atom or something) ?

thheller08:03:32

I mean that when (:require ["async-thing" :as x]) accessing the x directly will ALWAYS return a promise. Always, no matter where or when. So, yes you await that somewhere once, preferably in the init, then store the reference and use that instead. could be a simple (def automerge-actual nil) and then in the await body (set! automerge-actual the-result). then anywhere you want to access automerge stuff you use automerge-actual. probably create some helper functions to do so

pmooser08:03:12

Ok, thanks for the explanation - that is what I suspected you meant, but I just wanted to be sure. Thank you !

pmooser10:03:47

@thheller In the old library, which we could just require and refer to as am, I had some references to data types (which I used with extend-type) that I could just refer to like am/Text ... with this promise-based thing, when I have the result of the js-await, how do I do the equivalent of am/Text ? (.- the-result Text) or something ?

pmooser08:03:16

@thheller I have one more question. With this external js-provider, when I use the project nextjournal-markdown, I get an error from shadow-cljs upon loading the page:

pmooser08:03:28

Any idea how one might debug this ? It happens consistently with this specific project.

thheller09:03:02

this is not an error from a build using :js-provider :external. it does not call shadow.js.jsRequire ever at all

pmooser09:03:04

Not sure if that could be caused by:

pmooser09:03:28

This error only occurs if you use the external js-provider with webpack.

pmooser09:03:58

I mean obviously you know what you are talking about, but the markdown project works fine with normal shadow-cljs ... this error only appears when you do this webpack packaging step.

thheller09:03:34

feel free to make a repro and I'll take a look

pmooser09:03:48

Sure. I'll send it as an archive again. Gimme a min to package it up - thank you.

thheller09:03:56

from what you posted I can tell you that the CLJS output you are loading is not from a build using :js-provider :external

thheller09:03:08

might be a bad cache or something if you are switching arround alot

pmooser09:03:57

I can blow away all caches but this happens in multiple projects that I've tried this in, including a brand new minimal project for the repro case. Let me try again before sending it to you, after blowing away all subdirs etc.

thheller09:03:22

I'm not talking about shadow-cljs caches here

thheller09:03:26

I'm talking about your browser cache

thheller09:03:07

(for example if you are not using the built-in :dev-http)

pmooser09:03:07

Ok let me clear the browser cache as well.

pmooser09:03:28

(I am using the dev-http)

thheller09:03:51

then it should be fine. might still be bad service workers, bad browser extensions, etc

thheller09:03:05

but I'll take a look at the repro, could of course be a bug as well

pmooser09:03:24

Ok, let me disable all browser extensions and stuff like that too

pmooser09:03:14

Here you go - I removed the node_modules and generated js as well.

thheller09:03:58

note that the files you are sending have a lot of ._ files. don't know what this is about but they are useless 😛

thheller09:03:38

the nextjournal markdown lib is using a JS file

thheller09:03:04

this is not currently supported for :external

thheller09:03:23

let me see how hard it would be to implement

pmooser09:03:05

Ah, ok at least that explains why it doesn't work at least !

pmooser09:03:18

Maybe I can just find another library to use ... (or write my own, I suppose)

thheller11:03:54

just release 2.22.8 which fixes the shadow-cljs side

thheller11:03:09

however your repro then just fails with a different non-shadow-cljs error

thheller11:03:27

not exactly sure what is happening there

pmooser12:03:11

Thank you for doing that - maybe I can take a look with the new version.

pmooser12:03:27

Assuming that the error you are seeing is the complaint about plugin.apply not being a function, there is some discussion of similar issues elsewhere in their library here: https://github.com/valeriangalliat/markdown-it-anchor/issues/55 but I think it probably means that for now, it's best for me to find another library. I definitely don't know enough about js or js packaging to make sense of this stuff.

Ben Lieberman16:03:11

I'm getting "http2 is not available" trying to use amplify / AWS SDK. I see this was a bug in shadow previously, but it's still not working for me now. I tried setting the resolve to false in :js-options but that causes the build to fail with the cause of missing a different dependency.

thheller16:03:53

when/where do you get that? false just disables it, if its actually required and accessed that will cause problems

Ben Lieberman16:03:54

When I require "@aws-amplify/ui-react"

thheller16:03:08

I need more context to help

thheller16:03:15

> When I require "@aws-amplify/ui-react"

thheller16:03:19

so you get a build error? or a runtime error?

thheller16:03:43

what is the full error message?

Ben Lieberman16:03:15

Build error, yeah.

Build failure: the required JS dependency "http2" is not available, it was required by "node_modules/@aws-sdk/node-http-handler/dist/cjs/node-http2-handler.js". Dependency Trace:
        aoda/amplify/app.cljs
        aoda/amplify/views.cljs
        node_modules/@aws-amplify/ui-react/dist/index.js
        node_modules/aws-amplify/lib/index.js
        node_modules/@aws-amplify/core/lib/index.js
        node_modules/@aws-amplify/core/lib/Providers/index.js
        node_modules/@aws-amplify/core/lib/Providers/AWSCloudWatchProvider.js
        node_modules/@aws-sdk/client-cloudwatch-logs/dist/cjs/index.js
        node_modules/@aws-sdk/client-cloudwatch-logs/dist/cjs/CloudWatchLogsClient.js
        node_modules/@aws-sdk/client-cloudwatch-logs/dist/cjs/runtimeConfig.js
        node_modules/@aws-sdk/node-http-handler/dist/cjs/index.js
        node_modules/@aws-sdk/node-http-handler/dist/cjs/node-http2-handler.js

Searched for npm packages in:
        /home/bhlieberman/dev/clj/cljs/amplify-ui-demo/node_modules

thheller17:03:15

ok, so this is a node build I presume? or is it for the browser?

thheller17:03:26

but it appears to try to include a bunch of node related things

thheller17:03:44

so I'd have little hope of this working at all

Ben Lieberman17:03:21

no, but I'll give that a go

thheller17:03:38

maybe it has some webpack specific things

Ben Lieberman17:03:04

that worked @thheller thanks a bunch. I did want to note something (I think is) strange. The output of webpack created a directory named libs.js, and I had to reference the main.js inside that directory to get the build to run.

thheller17:03:21

that is your webpack config doing that, but yes sometimes webpack outputs more than one file. all depends on what you are building

2