@thheller does shadow provide npm-deps to gclosure to be included in dead-code elimination and other whole program optimisations as described here? https://clojurescript.org/news/2017-07-12-clojurescript-is-not-an-island-integrating-node-modules

it can yes but not by default

https://code.thheller.com/blog/shadow-cljs/2017/09/15/js-dependencies-the-problem.html

https://code.thheller.com/blog/shadow-cljs/2017/09/15/js-dependencies-going-forward.html

has some background on why it doesn’t by default

bascially set :js-options {:js-provider :closure} and see if your build still works

if yes great, it no set it back to :shadow 😉

at this point I’d say the chances are less than 50% .. way too many incompatible packages

ok - i’ll give it a spin 🙂 thanks!

@thheller picking up on an old thread, I got maria building on circleci w/ shadow-cljs: https://github.com/mhuebert/maria/blob/master/.circleci/config.yml (finally got a docker image with java + node + git working.) I ran into weird issues where my circleci would stop build commands in the middle of what they were doing, and call them “successful”. no idea yet what the cause was. i was able to “get it to work” by breaking things up into smaller commands (hence server/start, then three independent server/release commands) but i am not convinced it will be reliable until i find out what was happening there.

@mhuebert npx shadow-cljs start shouldn’t need the background option. it will exit when the server has finished starting. at least in theory.

I started setting up circleci for shadow-cljs itself too. still need to actually run something though. https://github.com/thheller/shadow-cljs/blob/master/.circleci/config.yml

@thheller hmm, i thought the problem was that circleci would somehow cause that process to exit without the background option

but i don’t fully understand it

hmm from what I understand from the docs the background will just run the process itself in the background

but the node process that starts the process itself will exit anyways

and only waits until the server is started

so that process does not run in the background, it starts a process that runs in the background by itself

Why does the guide [1] say "node 7.0.0+" while the npm package [2] says "6.0.0+"? Who is wrong? [1] https://shadow-cljs.github.io/docs/UsersGuide.html#_standalone_via_code_npm_code [2] https://github.com/thheller/shadow-cljs/blob/master/packages/shadow-cljs/package.json#L24

oops, the guide

@mhuebert maybe if you combine background with npx shadow-cljs server that works? (instead of start)

@holyjak to be honest the most recent version is always preferred and I don’t actually know the minimum version. I know some code breaks when using node v5

thats why it has v6 as minimum

@thheller maybe what i really need to do is figure out why i can’t just run everything in a single shadow-cljs clj-run maria.build/release.. i don’t see any real reason why i should have to run these commands separately

and how they are being abruptly stopped and circleci declaring that a success

maybe the logfile in target/shadow-cljs has some clues?

i’ll try that later today

do another build and ssh in

but yeah the release fn should work fine

although if that is all you do https://github.com/mhuebert/maria/blob/master/editor/src/maria/build.clj

you can just run npx shadow-cljs release live trusted bootstrap

yeah, i only created the build script as part of the debugging, to see if it made a difference

i was using that command earlier

either way, the process exits

at unpredictable times and without any message in the circleci logs

wow, very nice site! I have handful of people I could recommend this to 🙂 @mhuebert

@thheller here is the circleci log for an example of the problem: https://circleci.com/gh/mhuebert/maria/322

if you click on the pwd; ls; cd editor; npm install; npm run release; command

it shows up as green,

but it ended itself just after beginning the :bootstrap build. this is the very end of the log:

[:bootstrap] Compiling ...
-> Converting 31 JS sources
<- Converting 31 JS sources (3215 ms)
-> JS Cache write: 31 JS files
<- JS Cache write: 31 JS files (70 ms)
-> Compile CLJS: cljs/core.cljs
<- Compile CLJS: cljs/core.cljs (15735 ms)

hmmm

now I remember. running the builds in parallel wasn’t great with circleci

but yeah thats not parallel. no idea why it would exit

here is an example where I put npx shadow-cljs start into a command (without background: true), then ran subsequent builds - https://circleci.com/gh/mhuebert/maria/325 the build after shadow-cljs start:

#!/bin/bash -eo pipefail
cd editor; npm run release;

> @ release /root/project/editor
> rm -rf public/js/bootstrap; shadow-cljs clj-run maria.build/release;

shadow-cljs - config: /root/project/editor/shadow-cljs.edn version: 2.0.139
shadow-cljs - connected to server
shadow-cljs - socket connect failed, server process dead?

circleci’s VMs are pretty constrained. we switched to using our own instances with http://buildkite.com - clojure / cljs builds need lots of cpu

@mhuebert if you use background try server instead of start so circleci can take back of backgrounding the process

instead of letting the node script do that

what happens if you just run npx shadow-cljs release bootstrap and the other 2 builds after each other without the server?

just let it start 3 times?

@thheller I can’t remember if i tried that

i will try using server instead of start

hmm

maybe as the last command just cat target/shadow-cljs/shadow-cljs.log.0 just in case

circleci’s docs say: The background attribute allows for executing commands in the background. In this case, job execution will immediately proceed to the next step rather than waiting for the command to return.

yeah probably need to sleep for a bit

for the cache key, do you think v1-dependencies-{{ checksum "editor/package.json" }}--{{ checksum "editor/shadow-cljs.edn"}} would be sufficient

ie. package.json + shadow-cljs.edn

am wondering how long to sleep for, if dependencies have changed it might take longer for the server to start?

hmm true. no idea. start is supposed to solve that

since it exits when the server is started

i will try that again

I’ve got an interesting issue. We’re using hiccup-like syntax for shared views in .cljc files, and it works with both server- and client-rendered libs. In one particular form field, :phone_number, the input’s name attribute is set to phone_number in development mode (watch), but after doing a shadow-cljs release, that field becomes phone-number. Other underscored field names seem to be unaffected.

tried using start again and got the same,

shadow-cljs - config: /root/project/editor/shadow-cljs.edn version: 2.0.139
shadow-cljs - connected to server
shadow-cljs - socket connect failed, server process dead?
deleted pid file, please retry command to start new server
Exited with code 1

will try now not using the server at all

@jgdavey no idea? shadow-cljs definitely never rewrites strings/keywords in any way? what does the code do?

(I’m using the latest version of shadow-cljs, and have tried dumping caches, etc)

It’s hiccup-style view code. I’ll dig further into our libs, but the fact that it acts differently between dev/release seemed odd

AH!

yeah .. doh … so in shadow-cljs release keywords are optimized

so they are only allocated once

so it replaces the new cljs.core.Keyword(null, "phone_number", hash, meta) calls with cljs$cst$kw$phone_number variables

Ah, interesting. But wouldn’t :phone-number be different than :phone_number?

BUT if you have :phone_number AND :phone-number in your code the normalized JS variable name replace each other

since - is just replaced by _ blindly

Interestingl…

let me fix that damn munge function from closure

🙂

Cool. Everything else’s been working great, in prod. First issue we’ve hit with optimized js

@thheller ok, i think i will just abandon the server idea. for deploy I just want something as reliable as possible and a few extra seconds isn’t so important

@mhuebert I’ll try to figure out whats going on. I need it for shadow-cljs CI itself, just first need to figure out the npm side of things since I can’t npm install shadow-cljs there

@jgdavey should be fixed in [email protected]

That was quick 🙂

@thheller i added that cat log command. are any of these “waiting for…” messages significant? https://circleci.com/gh/mhuebert/maria/335

funny enough I fixed that issue once before in CLJS itself. https://dev.clojure.org/jira/browse/CLJS-1105 😉

@mhuebert not really. its just the parallel compile waiting for other files to compile first

@thheller Nice work. 🙂 Thanks again!

they will timeout eventually but it will log the abort

@mhuebert what is odd is the number of logs. there should be more than <number of cpu cores> log messages

@thheller weird, i just tried a new deploy after clearing the caches, and after everything ran successfully it said:

#!/bin/bash -eo pipefail
cat target/shadow-cljs/shadow-cljs.log.0
cat: target/shadow-cljs/shadow-cljs.log.0: No such file or directory
Exited with code 1

hmm guess there was nothing to log? the file is created lazily on the first log output

ah ok

now it will try printing the log if the build failed & do nothing if the log doesn’t exist:

- run:
          command: cat target/shadow-cljs/shadow-cljs.log.0 2> /dev/null
          working_directory: ~/project/editor
          when: on_fail

is there a way to get lein’s “checkouts” functionality working with shadow-cljs?

@thheller typo in 7.1.2 of the guide: «Add start/top hook functions.»

@lilactown just add the :source-paths and keep it in :dependencies. or you use the lein integration