Fork me on GitHub
#cljs-dev
<
2021-07-16
>
leif16:07:47

@dnolen If you care, I think I figured out the problem. Its that the macro expansion itself doesn't use the eval-fn you give it and expects that eval-fn modified the global environment.

leif16:07:06

(Rather than what my eval-fn does, which is run the compiled code in a sandbox.

dnolen16:07:01

@leif I think is back to my point yesterday about the compilation model

dnolen16:07:15

macros cannot be separated from the compiler

leif16:07:30

Of course they can't.

dnolen16:07:43

so they can't eval somewhere else

leif16:07:39

That part doesn't seem inherently true to me. Is there something special about the clojurescript compiler that prevents it from calling eval?

leif16:07:18

(At least the self-hosted one.)

leif16:07:27

(Obviously the jvm one can't easily do that.)

dnolen16:07:14

we intentionally make anything that would lead to different behavior difficult

dnolen16:07:40

the point of self-hosted is just to have a nearly fully compatible thing that can run in JavaScript

dnolen16:07:52

not to investigate things we can't do on the JVM

leif16:07:38

That's fair. But then I highly recommend you document that. 😉

leif16:07:08

Like, it sounds like your eval-fn function is expected to modify the global state.

leif16:07:50

But the docs just seem to give the function signature.

leif16:07:55

(Unless I'm reading the wrong docs?)

dnolen16:07:01

if you are doing self-hosted you are already deep in the weeds 🙂

dnolen16:07:13

it's simply not a recommended thing to do - and most people aren't doing it

dnolen16:07:18

so the value of documenting such things is very low

leif16:07:26

That's also fair, but then I would recommend putting that in the docs. (Unless I missed it?)

leif17:07:22

(To be fair, I very well could have missed that self-hosted is never recommended. 🙂 )

dnolen17:07:37

never recommended unless

dnolen17:07:06

you are willing to figure it out on your own, you're building some kind of online tooling, some kind of Node.js thing etc. etc.

dnolen17:07:35

@leif those docs are auto-generated and we link to it

dnolen17:07:04

we should probably maintain our thing to have more control - but the amount of time in the day on such a large project is limited

leif17:07:54

Welp, I do certainly fit into that class of people. 🙂

leif17:07:04

(And ya, it does make sense that its not a focus.)

dnolen17:07:40

pretty light on details

leif17:07:44

But it also seems to only indicate that the only issue is residual code size.

dnolen17:07:27

right self-hosted is ~300k gzipped

leif17:07:28

Anyway, no big deal. I'll probably just modify the analyzer to run via eval rather than running directly.

dnolen17:07:45

ClojureScript even with nearly the entire standard lib is <30k gzipped

leif17:07:02

Ya, that is quite large. 🙂

dnolen17:07:02

so we don't emphasize because people get the wrong idea

dnolen17:07:11

ooooh ClojureScript w/o the JVM

dnolen17:07:23

"what the output is so big, this is is no good" etc.

dnolen17:07:12

that said it would be cool to see external efforts to document self-hosted

dnolen17:07:19

there's stuff scattered about here and there

leif17:07:21

> "what the output is so big, this is is no good" etc. Oh, my artifact is 1-2 MB gzipped, which for a normal webpage is huge, but for an ide is okay. 🙂

dnolen17:07:51

the other thing is of course JavaScript and ClojureScript are both super hackable

leif17:07:54

> that said it would be cool to see external efforts to document self-hosted Oh don't worry, I will. 🙂 (At least if its okay with you. I'd like to not step on people's feet.)

dnolen17:07:15

nothing is stopping you from monkey-patching something in the compiler to get the behavior you want

dnolen17:07:34

in the end what we hand people is the recommended thing

dnolen17:07:45

but go do whatever you want etc.

leif17:07:50

> the other thing is of course JavaScript and ClojureScript are both super hackable I noticed. ^.^ (I've actually already done that to make the eval-fn and load-fn both take callbacks.

leif17:07:00

(IIRC, only load-fn did.)

leif17:07:31

> in the end what we hand people is the recommended thing That makes sense, thanks.

leif17:07:41

If there's a better way than literally pushing a 'slightly patched' build to a github repo I'm always ears. 🙂

leif17:07:24

(Not super experienced with state of the art java build environments. I'm more familiar with macro expanders and embedded systems. 😉 )

dnolen17:07:44

ClojureScript is a library

dnolen17:07:09

we wrote a js.cljs in the way we felt seemed sound - it won't be what everyone wants

dnolen17:07:37

you could patch or you just go do your own thing js2.cljs etc.

dnolen17:07:05

like a make a project that depends on ClojureScript, take the parts of js.cljs that make sense to you and do it different

dnolen17:07:32

all the choices about asychrony etc. are made there - the rest of the compiler is fully generic free of I/O concerns

dnolen17:07:43

reading, analysis, code gen etc.

dnolen17:07:12

to simplify maintaining a thing - that's how I would do it - no conflicts in js.cljs etc.

leif17:07:34

Mmm...makes sense.

leif17:07:48

Anyway, thanks for the feedback. 😄

dnolen17:07:26

@plexus welp it turns out it already was possible to do what you wanted just writing a macro that emits ns - mind checking if this satisfies your requirements - if so just going to revert your changes

3
dnolen17:07:27

if it does satisfy then after the revert we'll cut a release - and we can do a post that explains it always possible - and this will get rid of these nasty issues

dnolen17:07:29

also the parse-ns approach would be problematic for self-hosted - and writing macros that emit ns should work in all situations

borkdude17:07:04

> self-hosted is ~300k gzipped I get more like 800kb gzipped like numbers, ~8mb unzipped, e.g. here: https://borkdude.github.io/re-find.web/

dnolen17:07:35

@borkdude hah ok, I'm going off the old days - we probably have a lot more stuff now

borkdude17:07:51

that project is around 2 years old

borkdude17:07:41

I could do a more proper analysis because this also includes cljs.spec, and a bunch of core specs, so it might be slightly less

dnolen17:07:38

383k gzipped if you look at main.js

dnolen17:07:45

but that was 6 years ago

borkdude17:07:10

k, now I'm curious, I'm going to use shadow's report functionality to see the size.

thheller17:07:51

self-host is :simple only so it'll grow very quickly once you add stuff like reagent, re-frame, cljs.pprint, etc. just bare-bones cljs.core isn't that large on its own

dnolen18:07:43

https://github.com/clojure/clojurescript/pull/99 - all tests passing, all code reverted, but the original 3276 test still passes

borkdude19:07:56

ok, I get about ~2mb of output for (self-hosted) CLJS itself with that project:

thheller07:07:09

don't forget to count the :bootstrap output though. regular CLJS will at least contain the analyzer data for cljs.core directly in the main build while shadow-cljs will keep that all separate and loaded via transit

borkdude07:07:06

@U05224H0W Where in the report can I find this?

borkdude07:07:09

I may not have correctly set this project up with shadow. I didn't know how to port the cljsjs deps to npm, could not find the codemirror addons.

thheller18:07:56

the report doesn't mention :bootstrap stuff since its a separate build. guess you can just look at the files 😉

thheller18:07:32

https://code.thheller.com/blog/shadow-cljs/2017/10/14/bootstrap-support.html this I mean. you'll need that for any kind of self-hosted stuff

borkdude18:07:09

I see. I was hoping to port this project with minimal changes, but it's a bit more work. Perhaps it's better to just start from scratch and evaluate a trivial expression, without all the UI/codemirror stuff

thheller18:07:39

only the bootstrap/init stuff is new and the load-fn it provides. everything else should be the same

borkdude18:07:51

I also could not port the cljsjs stuff to npm, could not find the deps on npm

borkdude19:07:32

so 300kb gzipped sounds about right

dnolen19:07:53

ok there was one thing missing, I changed it so ns always just merges info - this probably in the end closer to the semantics of Clojure

dnolen19:07:34

something like this was always necessary for REPLs to behave in the expected way but it was done in a more a roundabout way and only for REPLs

dnolen19:07:26

I also enhanced the test so that it actually tests that multiple ns forms are cumulative wrt. the compiler state

dnolen19:07:16

@plexus if you have time to look at this over the weekend briefly that would be cool - would like to merge soon and get release out the door