I’m thinking hard about the caching provided by shadow and how to avoid wrecking it with my ideal stack. Just curious whether FactUI breaks caching due to being based on Clara. https://github.com/arachne-framework/factui

@grounded_sage yeah clara-rules is not very cache friendly. IIRC the problematic part of fact-ui is the f/rulebase call

so the ns that calls that has to be tagged with ^:dev/always

@richiardiandrea the threadpool was Terminated? did you ctrl+c or something? normally a watch never terminates the threadpool? not even when the watch is stopped since its shared on the entire system (only stop! does that)

Hey, does shadow-cljs have issues with cond->

or re-frame maybe?

no?

Sorry, it's a re-frame thing https://github.com/Day8/re-frame-debux/issues/22

@thheller just gotta say, the repl fix that uses the current code NS is awesome! much better experience, and with your tips to remove warnings and the latest shadow my compilation times got down from 15s (for file with lots of dependants) to 3s!! awesome!!

for future reference: if anyone is seeing regular watch recompile times of 10sec+ please let me know. there is probably something to be improved just waiting to be found! do not assume that 10s+ is normal

heck even 5sec is probably too much 😛

hmm there's a new REPL?

no, just fixed a bug

So umm how to get this new repl? Update? I use emacs and connect to nrepl

:s/update/update shadow version

the change was for Cursive only I think. not sure emacs actually sends the :ns when evaling from a file

the behaviour in Cursive is that it uses the namespace of a file when you send a form from a file to the REPL

eg. if you send ::foo from that will be :

even if the REPL is currently in cljs.user

my emacs setup is completely busted so I have no idea if emacs actually sends :ns or not

Had this problem agian in CI:

[2018-10-19 19:41:40.023 - INFO] :shadow.build.classpath/jar-cache-read-ex - {:file #object[java.io.File 0x1f0f5174 ".shadow-cljs/jar-manifest/1539977882000-core.rrb-vector-0.0.12.jar.manifest"]}
RuntimeException java.io.EOFException
	com.cognitect.transit.impl.ReaderFactory$ReaderImpl.read (ReaderFactory.java:114)
	cognitect.transit/read (transit.clj:319)
	cognitect.transit/read (transit.clj:315)
	shadow.build.cache/read-cache (cache.clj:49)
	shadow.build.cache/read-cache (cache.clj:33)
	shadow.build.classpath/find-jar-resources/fn--11757 (classpath.clj:565)
…	cognitect.transit/read (transit.clj:319)

sounds like maybe a job got killed while writing cache output?

its no biggy as the cache will just be re-built

its just a warning not a critical error

I'm using 2.6.10 and getting very intermittent errors when doing a release build in CI:

aborted par-compile, [:shadow.build.classpath/resource "clojure/string.cljs"] still waiting for #{cljs.core}
{:aborted [:shadow.build.classpath/resource "clojure/string.cljs"], :pending #{cljs.core}}
ExceptionInfo: aborted par-compile, [:shadow.build.classpath/resource "clojure/string.cljs"] still waiting for #{cljs.core}
	clojure.core/ex-info (core.clj:4739)
	clojure.core/ex-info (core.clj:4739)
	shadow.build.compiler/par-compile-one (compiler.clj:760)
	shadow.build.compiler/par-compile-one (compiler.clj:729)
	shadow.build.compiler/par-compile-cljs-sources/fn--12969/iter--12970--12974/fn--12975/fn--12976/fn--12977 (compiler.clj:822)
	clojure.core/apply (core.clj:657)
	clojure.core/with-bindings* (core.clj:1965)
	clojure.core/with-bindings* (core.clj:1965)
	clojure.core/apply (core.clj:661)
	clojure.core/bound-fn*/fn--5471 (core.clj:1995)
	java.util.concurrent.FutureTask.run (FutureTask.java:266)
	java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
	java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
	java.lang.Thread.run (Thread.java:748)

might this be fixed by some of the changes to parallel compilation that I see since 2.6.10? It's hard for me to test since it only happens intermittently

hi all

a question about minification/advanced optimizations

I've tried manually specifying :optimizations :advanced, :infer-externs :auto and :shadow-keywords true, and still, a release build looks like this:

(not sure if the image uploaded correctly, as slackbot is saying something about no storage left)

basically: #1, a curious unminified polyfill always shows up, and #2 plenty of full names visible in code

if I include reagent and re-frame, I get ~300k of release bundle; I tried adding a button and textfield component from material-ui, and the bundle size shoots up to 600k +

(it gets up to 1MB if I include all of material-ui-core)

is this normal behaviour, or am I missing some extra minification setting?

material-ui is incredibly heavy, yeah

I thought so, but then I stumbled upon some issues about material-ui bundle size, and people apparently expect a release bundle to be ~100-300k max

are you using a minified version of react & material-ui?

they might be talking gzipped

@temporal.pl run this to get a details breakdown of all the stuff in your build https://shadow-cljs.github.io/docs/UsersGuide.html#_build_report

and yes material-ui is huge if you import all of it

you typically want to limit it to only import what you are actually using

@thheller thanks

http://paste.temporal.pl/shadow-cljs-build-report.html

this is the report I get; do sizes there look normal?

thats not the report 😉

the html file is more detailed

I'm new to shadow-cljs, and cljs in general, so I have no point of reference

uh, ok 😄

those sizes look pretty normal

@temporal.pl use things like (:require ["@material-ui/core/Button" ...]) to only include parts of the package

yes, this case uses only

["@material-ui/core/Button" :default muiButton]
["@material-ui/core/TextField" :default muiTextField]

if I include it all, it's + 400k to total size

still; it's a fresh project, and I'm not married to any UI library yet, so if I could ask you for recommendation?

friends over the JS land discouraged me from using react-bootstrap

dunno really .. they all get pretty large pretty quickly

I just bumped to 2.6.14 and I'm getting this error in CI

ok, thanks for help and good night!

@lilactown thats a bad closure compiler version

aackclkasdlfds

I do this everytime I update shadow-cljs. I'm sorry

thats what you get for not using shadow-cljs to manage your deps 😉

not a fan that closure keeps changes their public interfaces. they don't seem to care much about backwards compatibility 😞

well to be fair that interface wasn't public and I had to cheat to get access to it 😛

heh, when I try and run the release command locally i get an OOM error

one release build works fine, but concurrent ones: npx shadow-cljs release server client dev-client

hm, fixed by killing my watch's

ah I bet I was using the old version shadow-cljs I had running as a server

OK, the previous error occurred again after upgrading

hmm don't scare me like that ... I actually changed how parallel compile works. don't tell me I actually broke it?

are you absolutely 100% sure that the correct code is loaded

no bad AOT cache or anything?

I've been seeing this error intermittently since 2.6.10

it could be cache related

in the past, re-running the build has resolved it, but it was popping up relatively often

thats definitely not normal and should never actually happen. its just a safeguard so the process doesn't wait forever

I'm re-running it right now so we'll see if it happens twice 😛

run with :verbose true in the config please

otherwise its gonna be impossible to reconstruct what happens

(or with --verbose on the CLI)

I'll add that once this job finishes

hmm is the machine maybe just overloaded?

if the other 2 builds proceed normally the third may just not be getting a chance to run before its timeout

still need to add that --parallel=false option

you should try creating a clojure fn that just does the builds sequentially

I might do that

ok, this job succeeded

the way this works is that all builds more or less start at the same time and enqueue all the CLJS compiles onto a threadpool

which then compiles them in order they were submitted

so its entirely possible that the queue just gets too long

hmm I see

kinda unlikely but possible

you were running on a toaster before 😉

I wonder if it might have to do with other jobs running on the same machine

lol, well we upgraded the toaster

we now have a group of toasters in AWS ECS for our pipeline

I'm wondering if a particular EC2 gets used for multiple concurrent builds if it's what causes the error

it might be poisoning the cache, or just overloading it

definitely try running with verbose AND running them after each other instead of parallel

cache is always isolated per build

so they can't interfere with each other

alright, I wrote a simple script to do the build:

(ns release
  (:require [shadow.cljs.devtools.server :as server]
            [shadow.cljs.devtools.api :as shadow]))

(defn -main [& args]
  (server/start!)
  (shadow/release :server {:verbose true})
  (shadow/release :client {:verbose true})
  (shadow/release :dev-client {:verbose true})
  (server/stop!)
  (System/exit 0))

don't really need a server instance for release

ah OK

shadow-cljs run release/all also works a bit better probably

(defn all [] ...) of course .. not -main

or wrap the calls in (shadow/with-runtime ...) yourself

(defn -main [& args]
  (shadow/with-runtime
    (shadow/release :server {:verbose true})
    (shadow/release :client {:verbose true})
    (shadow/release :dev-client {:verbose true})))

what does with-runtime do?

share some components between the builds

otherwise they all go off and re-index the classpath 3 times and stuff

if you use shadow-cljs run thats done automatically

(defn all [& args]
  (shadow/release :server {:verbose true})
  (shadow/release :client {:verbose true})
  (shadow/release :dev-client {:verbose true}))