Fork me on GitHub
#sci
<
2021-01-24
>
mkvlr09:01:51

I’d like to implement require but lazy load a :namespace map compiled to cljs instead of a Clojure source file, is this possible?

mkvlr10:01:57

looking at https://github.com/borkdude/sci/blob/19242bf2cf549fd7895696e65761c04e32c61f7f/src/sci/impl/evaluator.cljc#L193 it seems it’s not but seems easy enough to add. Would a PR be welcome for it?

borkdude10:01:57

@mkvlr Can you sketch the scenario of what should happen on require?

borkdude10:01:28

As long as you return the source string using the :load-fn it should work

mkvlr11:01:09

@borkdude I want to load an cljs advanced compiled module, basically merge the namespaces map on load

mkvlr11:01:20

not Clojure source

mkvlr11:01:23

so basically load https://github.com/sicmutils/sicmutils/blob/master/src/sicmutils/env/sci.cljc#L105 lazily but having it part of my advanced compiled bundle (but as a separate module)

borkdude11:01:49

@mkvlr ah, yeah, I think you can mis-use :load-fn for this by checking the namespace, then updating your ctx manually and then return an empty source string

borkdude11:01:16

(update ctx :env swap! assoc-in [:namespaces your-ns-name] your-ns)
something like this

mkvlr11:01:40

@borkdude thanks, will give it a try and let you know!

mkvlr13:01:15

@borkdude returning an empty source and changing the env works. But I still need a async variant of :load-fn . https://github.com/borkdude/sci/issues/511

borkdude13:01:41

@mkvlr if this is going to be async, how do you prevent using the required namespace in successive expressions?

borkdude13:01:44

I was thinking, maybe you can fire the async operation and then wait in some loop until the async operation has completed?

borkdude13:01:01

eh, this is JS async hell again

mkvlr13:01:33

I thought we can contain it but there’s not really a way, right?

mkvlr13:01:55

it basically means sci/eval has to become async?

borkdude13:01:39

you are fetching the module via some http request?

borkdude13:01:01

is something like this not possible? async operation -> marks flag when done loop { wait for flag to become true }

borkdude13:01:54

we need await in CLJS ;)

borkdude13:01:47

I'm not sure what the consequences will be if load-fn has an async counterpart. Will then everything become async?

mkvlr13:01:41

I think so, that’s why its called a leaky abstraction right?

mkvlr13:01:55

https://eloquentjavascript.net/11_async.html says: > If we had used the handler’s return value as the response value, that would mean that a request handler can’t itself perform asynchronous actions. A function doing asynchronous work typically returns before the work is done, having arranged for a callback to be called when it completes. So we need some asynchronous mechanism—in this case, another callback function—to signal when a response is available.

borkdude13:01:56

So an alternative would be to include the module inside some dom node or JS variable as a string and load it from there maybe? Or will this have negative consequences?

borkdude13:01:15

Isn't is possible to get synchronous loading by creating a dom node which refers to that file and then reading the contents from that dom node?

borkdude13:01:28

(thinking of workarounds)

borkdude13:01:37

a dom node which does a GET request and gets populated with a string response from your server

borkdude13:01:53

and then that is fed to load-file

borkdude13:01:03

load-fn rather

mkvlr13:01:08

I think it’s just not possible to do async work from a non-async function or do I have a fundamental misunderstanding?

borkdude13:01:23

yes, the workaround I suggest is making everything sync

borkdude13:01:58

it will block your UI, but for require I think it's reasonable, since this is what happens in CLJ too

mkvlr13:01:17

will try it, thanks!

borkdude13:01:22

they even added serialized-require to CLJ now to make it more synchronous

mkvlr13:01:01

yeah, but you can still have other threads running in this case?

borkdude13:01:06

of course, but for loading a namespace from an external resource, I don't think it's unreasonable to show a spinner and wait for it to be loaded. it simplifies the model drastically

borkdude13:01:37

we can make everything async, but this will be a major change

mkvlr13:01:55

this doesn’t sound so good either

borkdude13:01:21

> Just because appendChild() has returned does not in anyway guarantee that the script has finished executing He is talking about the JS eval

borkdude13:01:35

but the response is there. We have sync eval in sci

borkdude13:01:40

so this should work

mkvlr13:01:45

no, I’m loading a js module

mkvlr13:01:29

think I need to look at how async could be supported in sci and sleep on it

borkdude13:01:30

so what would happen if you wrapped the sync sci API functions in your own async functions?

borkdude13:01:56

so any eval string you do would be async and the string could be coming from an http request?

mkvlr13:01:59

would that be problematic when you eval code that does the require and uses it in the same cell?

borkdude13:01:23

maybe it wouldn't be so bad to introduce some async API functions

borkdude13:01:32

don't know how deep the rabbit hole would be

borkdude13:01:43

but worth an experiment I'd say

borkdude13:01:58

probably the analyzer can stay sync

borkdude13:01:04

also the parser can stay sync

borkdude13:01:36

I mean, we would add async functions, not remove the sync ones

mkvlr13:01:02

I’ll give it a try, probably tomorrow

mkvlr13:01:12

thanks for your help!

borkdude13:01:46

@mkvlr btw, I'm not entirely sure about the comment on SO:

eval("1 + 1")
2
this is entirely sync btw, you do need to allow this, many sites forbid it

borkdude13:01:17

but there might be a way to do this safely

borkdude13:01:30

of course this is hacky, so an async API would be the best

borkdude13:01:15

but in theory, I think you can do it all synchronously: the request/response + js eval