Fork me on GitHub
#shadow-cljs
<
2019-02-06
>
jorda0mega04:02:28

for the calva plugin in vscode, is there a way to pretty print the results on the terminal window?

anmonteiro15:02:01

@thheller Hey Thomas, I think we might have run into a weird bug in shadow cljs

anmonteiro15:02:31

shadow.cljs.util/is-jar? does a simple endsWith check to see if the string is a jar

anmonteiro15:02:53

however in our internal setup, Buck generates directories that end with .jar ¯\(ツ)

anmonteiro15:02:02

that’s causing a weird error in our build

anmonteiro15:02:24

FileNotFoundException: /Users/anmonteiro/Documents/github/ladder/buck-out/gen/src/ladder/__common_lib__/common.jar (Is a directory)
        java.util.zip.ZipFile.open (ZipFile.java:-2)                        
        java.util.zip.ZipFile.<init> (ZipFile.java:219)               
        java.util.zip.ZipFile.<init> (ZipFile.java:149)             
        java.util.jar.JarFile.<init> (JarFile.java:166)              
        java.util.jar.JarFile.<init> (JarFile.java:130)           
        shadow.build.classpath/find-jar-resources* (classpath.clj:455)
        shadow.build.classpath/find-jar-resources* (classpath.clj:452)
        shadow.build.classpath/find-jar-resources (classpath.clj:579)
        shadow.build.classpath/find-jar-resources (classpath.clj:553)     
        shadow.build.classpath/find-resources (classpath.clj:627)    
        shadow.build.classpath/find-resources (classpath.clj:625)            
        shadow.build.classpath/index-path* (classpath.clj:854)        
        shadow.build.classpath/index-path* (classpath.clj:851)
        clojure.lang.PersistentVector.reduce (PersistentVector.java:343)
        clojure.core/reduce (core.clj:6827)             
        clojure.core/reduce (core.clj:6810)                               
        shadow.build.classpath/index-classpath/fn--1940/fn--1941 (classpath.clj:973)
        clojure.lang.Atom.swap (Atom.java:37)                              
        clojure.core/swap! (core.clj:2352)                        
        clojure.core/swap! (core.clj:2345)                        
        shadow.build.classpath/index-classpath/fn--1940 (classpath.clj:973)
        shadow.build.classpath/index-classpath (classpath.clj:972)   
        shadow.build.classpath/index-classpath (classpath.clj:967)    
        shadow.build.classpath/index-classpath (classpath.clj:969)
        shadow.build.classpath/index-classpath (classpath.clj:967)
        shadow.cljs.devtools.server.common/fn--6874 (common.clj:89)   
        shadow.cljs.devtools.server.common/fn--6874 (common.clj:87) 

anmonteiro15:02:29

this is the relevant stack trace

thheller15:02:43

yeah I can fix that

anmonteiro15:02:50

expanded with a little more info

anmonteiro15:02:01

@thheller this is using 2.7.25 btw

anmonteiro15:02:20

I tried to override is-jar? with:

(defn is-jar? [^String name]
  (let [f (java.io.File. name)]
    (and
     (not (.isDirectory f))
     (.endsWith (str/lower-case name) ".jar"))))

thheller15:02:28

yeah the check just naively assumes that things ending in .jar are actually jars not directories

anmonteiro15:02:31

but that doesn’t work either because name is just "common.jar" in this case

anmonteiro15:02:50

so I think it’d need to be the full path

anmonteiro15:02:59

not sure if that makes things harder or not 😛

thheller15:02:17

nah easy fix.

anmonteiro15:02:32

this also only started happening when I bumped shadow from 2.7.12 to 2.7.25

anmonteiro15:02:48

not sure why that would be, but it could be my fault instead of shadow’s

anmonteiro15:02:54

given that our dep management is pretty gnarly

ccann15:02:01

kind of new to this bit — is there any way to specify a path in shadow-cljs.edn to an npm dependency’s CSS such that it works in dev and in a release artifact?

ccann15:02:52

ah I think this is one of those “use other tools for now” bits yeah?

thheller15:02:40

no CSS is not supported currently

ccann15:02:35

okay cool

ccann15:02:48

do you know of any tools ppl reach for in this instance?

thheller15:02:02

I use the node-sass CLI

thheller15:02:24

@anmonteiro nothing in that regard really changed since 2.7.12 so maybe something in your build setup changed?

anmonteiro15:02:50

yeah that might be the case, I need to investigate further

thheller15:02:23

does buck only do this for release builds? some dev features treat files in jars a bit differently and won't warn as aggressively for example

anmonteiro15:02:50

I’m not sure

anmonteiro15:02:06

hrm so the difference between 2.7.25 and 2.7.12 in our setup is that the directory ending in .jar is never in the dep graph in 2.7.12

anmonteiro15:02:31

I’m still not sure why that is though

thheller15:02:31

that is weird. the error happened when the classpath was indexed which happens before any CLJS compilation is even involved

thheller15:02:57

@anmonteiro 2.7.26 should fix it

anmonteiro15:02:15

I actually built a jar from master now that I saw your commit

anmonteiro15:02:39

yeah I think it did work

thheller15:02:59

was about to push a release anyways since I wanted to test a new feature I was working on 🙂

anmonteiro15:02:14

@thheller soo there’s another “issue” in 2.7.26 related to this

anmonteiro15:02:25

you’re now throwing an error classpath entry that is not a directory or jar file

anmonteiro15:02:33

but I’m not sure if you need to be this strict

thheller15:02:50

what the heck? that was supposed to never happen?

anmonteiro15:02:51

we have a shared library .so in the classpath that is neither a jar nor a directory 😛

thheller15:02:08

that is allowed? I though java would blow up if you do that?

anmonteiro15:02:24

I think it is, for bindings to native code

anmonteiro15:02:27

through the JNI

anmonteiro15:02:42

I can work around this one, just wanted to provide feedback

thheller15:02:07

pretty sure that still needs the full filename and can't be the classpath entry itself

anmonteiro15:02:26

this seems to be allowed

anmonteiro15:02:36

we’ve had this for a while and java hasn’t complained

anmonteiro15:02:37

wrt why we started hitting this, did you perhaps change anything between 2.7.12 and 2.7.25 wrt. laziness / eagerness?

thheller15:02:54

nothing related to classpath indexing

thheller15:02:47

@anmonteiro pushed 2.7.27 that just silently ignores anything but directories or regular files that end in .jar

anmonteiro15:02:57

thank you sir

thheller15:02:24

the last related change to the classpath was in 2.7.6 which reworked how symlinks are handled

anmonteiro15:02:53

@thheller does clojurescript need to be a hard dependency for shadow btw?

thheller15:02:21

no but also yes

thheller16:02:22

yes because sometimes things are added in master that break shadow-cljs. like when 449 added new bindings in cljs.analyzer that would cause nullpointerexceptions since shadow-cljs didn't properly bind them

thheller16:02:37

but since adding them older releases don't work since those bindings don't exist

thheller16:02:18

most of the time master just works, just sometimes things break so its safer to always set a depdendency

anmonteiro16:02:21

yeah being a hard dep for API compat makes sense

anmonteiro16:02:57

I just asked because we use the "slim" classifier and were getting the normal AOTed compiler on every shadow dep resolution

anmonteiro16:02:37

in fact the whole reason why I found this bug in the first place was because we had 1.10.516 on shadow 2.7.12 and it was telling me clojure.edn didn’t exist

anmonteiro16:02:46

so I went to upgrade shadow

thheller16:02:58

hmm if you had 1.10.516 you should have had clojure.edn? your dep should always be picked over the shadow-cljs dep?

anmonteiro16:02:12

so it compiled fine without trouble

anmonteiro16:02:29

but when I changed the config.edn file, the browser said the bundle was out of date

anmonteiro16:02:45

and the terminal gave a warning that clojure.edn didn’t exist so it would skip compilation

thheller16:02:17

oh you are using shadow-cljs straight from npm to resolve deps I assume?

thheller16:02:32

yeah that currently does not allow setting custom CLJS versions

anmonteiro16:02:32

I don’t know what those words mean 😄

thheller16:02:46

you are not using deps.edn or project.clj for deps

thheller16:02:41

the npm package is rather strict because people in the past had old clojurescript in their :dependencies

thheller16:02:03

it should warn if you have have clojurescript in shadow-cljs.edn :dependencies

anmonteiro16:02:28

that might be the case

anmonteiro16:02:32

(I didn’t set it up)

thheller16:02:36

I've never used buck. kinda curious to see what it does now 🙂

anmonteiro16:02:58

it’s a pretty frankensetup

anmonteiro16:02:29

this would be the (very outdated) extracted version of it https://github.com/ladderlife/loonie

thheller16:02:50

shadow-cljs already does pretty extensive caching and validation thereof

thheller16:02:24

shouldn't need anything beyond just keeping the .shadow-cljs/builds directory around between compiles

anmonteiro16:02:55

Buck’s notion of caching is a little different, and more holistic

anmonteiro16:02:03

@thheller last issue we’ve found with Shadow

anmonteiro16:02:24

we’re running a fork of shadow actually, because the source maps were embedding our entire source code in production 🙂

anmonteiro17:02:06

^ this is the current diff we applied to shadow

anmonteiro17:02:25

but we aren’t sure that’s the correct patch

thheller17:02:26

happy to make this configurable

anmonteiro17:02:42

we’d love to not run a fork too 🙂

thheller17:02:59

but what good are source maps without the sources?

anmonteiro17:02:38

well in the stacktraces we report to sentry we just need to get the filenames and line numbers

anmonteiro17:02:48

we’d rather keep our code private

5
🤫 5
thheller23:02:52

I could use some help in testing this new thing I added today https://clojureverse.org/t/smaller-builds-with-lots-of-js-dependencies/3742

👀 5
thheller23:02:26

some before/after numbers would be nice and if it actually works for your builds