I'm currently attempting to reimplement data_readers.cljc for the project, but maybe there is a more graceful way to handle the issue?

hmm yeah reader literals are not handled by aliases

Any idea what I might be doing wrong (presumably with webpack) that would cause this error?

not loading the webpack output before the shadow-cljs output

you should have two script tags in the html

I do - but maybe something changed and my webpack config has to be different - I'll work on it some more and see if I can resolve the error through the webpack config.

Thank you!

@thheller I'm still on this path of trying to use a wasm library. I previously had it working, and the wasm library would show up in my cljs code as a promise, and you helped me use js-await to access the contents of that promise, and then I was able to use that module from there. They've made an update to the library, and now it doesn't appear as a promise anymore, but when I try to do things with it, it complains that .use hasn't been called on the module, which sounds like maybe it's not being loaded correctly. I know this isn't exactly supported territory for shadow-cljs, but do you have any ideas for things I might try? I'm currently using an import like this:

["@automerge/automerge" :rename {next automerge}]

this means absolutely nothing to me. can you provide more context? JS examples/docs?

Yes, I'm sorry - the project I'm trying to use is: https://github.com/automerge/automerge/ I'm using webpack as an external js-provider with shadow-cljs. A javascript example of using it with webpack is provided here, about half a page down: https://automerge.org/docs/quickstart/

Essentially, I'm just trying to call .init on automerge, as they show there. And this has worked previously with this wasm setup, but at some point they upgraded their library in such a way that it no longer works, and notably, as I mentioned, it doesn't show up as a promise anymore in cljs.

and why are you not following the docs exactly, but try to do something else?

AFAICT the docs don't mention calling .next on anything?

In what way have you identified that I'm not following the docs? Ah.

Ah --

I see const Automerge = require('@automerge/automerge') and then let doc1 = Automerge.init()

next is the API they're migrating to, so certain things are essentially only possible in next, so despite it not being mentioned in the quickstart, it is supported and in fact recommended in their library.

translated to cljs that is (:require ["@automerge/automerge" :as Automerge]) and then (def doc (Automerge/init))

Here - I think for the purposes of this discussion, I can probably reproduce the same error without next. One minute, and I can verify.

I see no mention of next, so I have no clue what you are talking about

Are you an expert in automerge?

In that case, I'm not surprised you haven't heard of next.

I just explained it to you.

no, I have never looked at it

Ok -

So if I use this import instead:

["@automerge/automerge" :as am]

And I just try to do (.init am) exactly the same error as I described occurs.

So in fact it has nothing to do with next.

and what is the error?

And speaking with the developers of automerge, they said that users should never have to call use themselves, and that it normally happens as a result of loading the module.

looks like a JS problem to me?

did you configure webpack correctly?

shadow-cljs is not involved in the JS bundler with :js-provider :external

Yes, I mean, in so far as: • this configuration worked with a previous version of the library • webpack runs and emits js that I can load in the browser • it's the same webpack config present in the quickstart I linked above

Which part of what you see above is related to the js bundler? Just webpack doing the packaging?

and you verified that a basic pure JS example works?

The developers say it does. Maybe I can go try to do that next.

Anyway, it sounds like you think this isn't anything related whatsoever to shadow-cljs.

Thanks for taking a look.

I would put this in a JS file

import * as Automerge from "@automerge/automerge"
let doc = Automerge.init()
console.log(doc)

run webpack over it and load it in a browser

Ok, I will give it a try, thank you.

if that works we can look further and figure out what adding shadow-cljs into the mix does

Ok, thanks. I'll go work on this. I appreciate the pointers you're giving me.

@thheller Ok, so I did what you asked, and it works for both the "normal" library and the "next" submodule. The output looks like this in the console (not very useful, but just to prove I did it):

and what does it look like if you translate that JS code to CLJS. only that. nothing else?

Do you mean that I should try to create the most minimal cljs project that does just this? I can certainly try that and report back.

yes, ideally one you can share so I can look at it

Ok, I will try to do exactly that - but it will probably take me a bit. I'll report back to you once I have it set up.

always hard to debug anything without seeing the code

Of course, that's very reasonable.

Thanks. I'll let you know.

@thheller Ok, I have it, with the same results as in my larger project (ie, I get the error which doesn't occur in plain js). How can I send you a .tgz of the project ?

And I will just nuke the node_modules directory rather than include it I assume.

<mailto:[email protected]|[email protected]> or just slack

Ok, let me package things up.

Here you go. I put some minor comments in the readme, but you just have to npm install and then you can execute the shell script run and then you can load index.html in the browser.

@thheller Let me know if you have any issues.

ok, I'm guessing this is some sort of issue in webpack and its ESM/WASM bundling when used from a commonjs require file

here is what basically happens

the automerge package does this

import * as wasm from "./automerge_wasm_bg.wasm";

but .wasm imports in the browser are non-standard and therefore each bundler does something slightly different

webpack in this case rewrites it to use the regular wasm initialization steps, which are async

and it therefore uses its alternative loading mechanism that turns EVERY import async

at least when working of a ESM file in the first place

I changed the build config to

{:dev-http {8000 "resources/web"}
 :builds {:app {:target :browser
                :output-dir "resources/web/js"
                :modules {:main {:init-fn minimal.core/init}}
                :js-options {:js-provider :external
                             :external-index "target/index.js"
                             :external-index-format :esm}}}}

which appears to work?

but I think it can run into issues with the async loading, as in trying to use the dependencies before they finish loading

since the webpack async loading mechanism is hidden from the user, there is no way to reliably detect it

but at least I don't get an error on load

@thheller Thank you so much for looking into this - I've been in meetings all afternoon so I haven't had a chance to give this a try, but I really appreciate your thorough investigation and I'm hopeful that this can get things working for me again (although I understand what you said about async loading) ... I'm curious if it will change how the module "appears" to me (since, as I said, in the previous versions, the actual module showed up as a promise) ...

I'll let you know if it actually works for me, in case that's of interest.

It does appear to work! That's great. But what you said about async loading and not being able to reliably wait for it is disturbing, but I understand we can't do much about it. I'll give the feedback back to the automerge guys. Thank you again for your efforts on this - I didn't even know about the :external-index-format option.

somewhat coincidence this works. that option exists for other reasons 😛

@thheller So as I said, that option seems to fix that fundamental issue, but what I see now is that if I load my application in a new browser window, I get the following error (followed of course by others):

When I refresh the page, I don't get the error anymore --

could this be the async loading issue that you mentioned,

where we don't have any real way to wait for webpack's async loading to be done?

And yeah, it's not even consistent - sometimes it takes more than 1 reload for the problem to go away, so it may just be pure luck whether or not it happens.

yes, that is likely the async loading stuff

the problem is that webpack thinks it is in control of everything and only ensures its own code is loaded properly and in order

that of course doesn't work for us

Ok, thanks very much - this approach is dead in the water then. I'll go talk to the automerge devs and see if I can come up with something else !

Yes, that makes sense - but I appreciate your response and explanation !

would be good if automerge had a way where you can initialize the wasm stuff yourself. i.e. not use import with a .wasm file

as no browser supports that and is as of now a bundler feature, which shadow-cljs doesn't yet have

Yeah, I just asked them if there was a way to load the wasm myself (by coincidence) because it seems like that would let me bypass all of this and just use shadow-cljs like a normal person instead of in this webpack way anyway ...

it'd be so simple if their Automerge.init just took an argument that is the wasm stuff

import * as Automerge from "@automerge/automerge"
import { wasm } from "@automerge/automerge-wasm"
let doc = Automerge.init(wasm)
console.log(doc)

it would be trivial to make that work with CLJS 😛

but of course .. breaking change, so never gonna happen

Yes - that would be very nice indeed. Although in their typescript declarations they directly import some stuff from the wasm project (like import type { ... } from "@automerge/automerge-wasm") so I'm not sure if that's even possible for them. But I'll ask ...

well its JS world, so maybe 😉

Do you have a patreon or something like that?

Nevermind, I found it.

github sponsors, patreon yes 😛

Ok, I sent you something through the github sponsorship program - I'd rather have my company do something, but at least this shows that I appreciate the help you have given me over the past few years. Thank you again.

thank you. much appreciated

Hey @U07VBK5CJ, I’m curious if you made progress here!

@U017QJZ9M7W was it auto merge specifically you were interested in ?

Yea that’s right

So for the time being I am stuck, because using web pack as a bundler in this way seems to introduce a loading race condition with the initialization of auto merge. However, I am talking to the auto merge guys, and they are going to eventually provide an interface that will let me load the wasm module myself, and then initialize auto merge with it, meaning that I won’t have to use the web pack bundler anymore at all. The timeline isn’t clear yet, but I am waiting on them to give me an example of how to do this.

Nice who are you talking to? Are you in their discord?