Fork me on GitHub
#boot
<
2015-06-13
>
estsauver06:06:28

I’m having trouble with Boot, it seems like some dependencies aren’t being resolved.

estsauver06:06:46

Earls-MacBook-Pro:wingapp earljstsauver$ java -jar target/wingapp.app-0.1.0-SNAPSHOT.jar Exception in thread "main" java.lang.NoClassDefFoundError: clout/core/CompiledRoute

martinklepsch11:06:20

@estsauver: is this an uberjar?

estsauver11:06:29

It looks like from what I can find it might be a resource issue.

martinklepsch12:06:45

estsauver: what do you mean by resource issue?

martinklepsch12:06:57

Only cause I can think of is that you’re using a dependency that’s not added to the uberjar — are you adding dependencies as part of development tasks maybe?

estsauver12:06:30

I don’t think I am.

estsauver12:06:16

I set dependencies in one place in the set-env

estsauver12:06:37

I’m not currently making a distinction between dev dependencies and prod.

alandipert12:06:48

can you share your build.boot?

alandipert13:06:55

@estsauver:

alan@alanputer:/tmp/foop$ java -cp target/wingapp.app-0.1.0-SNAPSHOT.jar clojure.main
Clojure 1.7.0-RC1
user=> clout.core.CompiledRoute       
CompilerException java.lang.ClassNotFoundException: clout.core.CompiledRoute, compiling:(NO_SOURCE_PATH:0:0) 
user=> (require 'clout.core)
nil
user=> clout.core.CompiledRoute
clout.core.CompiledRoute

alandipert13:06:30

CompiledRoute is a defrocord in clout.core which doesn't exist until you (require 'clout.core). perhaps you are trying to use CompiledRoute before doing this?

alandipert13:06:09

you may also add clout.core to the list of NSes you AOT

alandipert13:06:50

i gotta run, hope this helps! back later

micha14:06:13

@tsdh: BOOT_CLOJURE_VERSION sets the version of clojure that boot will use for the actual bootstrapping (i.e. evaluating your build.boot file), it can't know what :dependencies are without evaling build.boot in clojure, so you can't set the version of clojure the build.boot script will be evaluated in in build.boot itself

micha14:06:36

@tsdh: things go much smoother currently if you use maven repo for all jar deps

micha15:06:36

@tsdh: if you install your dependency jars to local maven repo tests should work, unless the java classes severely abuse classloader hierarchy by modifying classloaders it doesn't own etc

micha15:06:54

@tsdh: you should make your tests such that they perform the setup you need in the test context rather than referring to things in the build.boot context, because the tests will run in a fresh pod each time so they won't have access to the build.boot classloader context

micha15:06:23

if that makes any sense lol

micha15:06:32

it looks like gibberish now

tsdh16:06:29

@micha I guess I can use (getenv "B_C_V") as dependency version so that I need to declare the right clojure version only in boot.properties.

tsdh16:06:37

@micha Sorry, but I don't understand your last statement. Is that meant for the third party lib's JavaCompiler?

jeluard16:06:36

Are changes to env (say :resource-paths) effective directly or only inside new pods?

micha16:06:38

@jeluard: they're effective immediately

micha16:06:13

they aren't propagated to existing pods, but they will be effective in pods created after the env changes

micha16:06:52

@jeluard: changes to :source-paths and :resource-paths are even effective immediately in the REPL

micha16:06:22

i.e. you can remove or add directories from the classpath dynamically at runtime, something you usually can't do

micha16:06:40

normally you can't remove URLs from a URLClassLoader instance

jeluard16:06:15

I only care about adding but this is good to know.

micha16:06:46

for adding you may want to look at the merge-env! function in boot.core

micha16:06:05

(merge-env! :source-paths #{"new-src"})

micha16:06:29

that adds "new-src" to :source-paths without overwriting the existing ones

micha16:06:29

otherwise you can do like (set-env! :source-paths (fn [old-source-paths] (conj old-source-paths "new-src")))

jeluard16:06:41

So if I modify resource-paths in a task defined in my buid.boot (say dev) only sub-tasks creating new pods will see those changes?

micha16:06:55

it depends where you do it

micha16:06:22

if you do it before returning the handler it will happen when the pipeline is being constructed

micha16:06:58

(deftask foo []
  (set-env! ...)
  (with-pre-wrap [fs] ...))

micha16:06:14

that's where i'd do it i think

micha16:06:30

that will happen before the pipeline is composed

micha16:06:15

like when you do boot foo bar baz on the command line

micha16:06:44

what's actually constructed is the expression (boot (foo) (bar) (baz)) in clojure

micha16:06:07

so when (foo) is evaluated is when the set-env! would happen

micha16:06:15

in the above example

jeluard16:06:28

great! Is it safe to load resources from Thread.currentThread().getContextClassLoader() in this case?

micha17:06:58

@jeluard: yes, i think so

micha17:06:36

the only problems arise when loading clojure classes into the parent of that classloader

micha17:06:32

you should look at boot.pod/add-classpath and friends

micha17:06:00

those will do the right thing, respecting classloaders marked by boot as being immutable

micha17:06:10

boot uses dynapath for that

micha17:06:39

but that's just a convention for marking classloaders with metadata that indicates that they shouldn't be altered

micha17:06:54

it's not supported at the JVM or Java level

micha17:06:54

the boot.pod namespace is available in any bot pod

micha17:06:04

s/bot/boot/

jeluard17:06:47

Thanks! I have some trouble reading files from the classpath from a boot task but it does not look related to 'dynamic' env actually. Now that I know what I can expect I will dig more.

ragge17:06:52

I've struggled with a quite weird AOT/uberjar issue while trying to port a leiningen build component based program to boot.

micha17:06:01

@jeluard: if you have a simple test case i can have a look at it if you want

ragge17:06:01

Have created a repo demonstrating the issue here: https://github.com/ragnard/boot-uberjar-mystery

ragge17:06:49

Have created an issue (https://github.com/boot-clj/boot/issues/227) but if anyone has any insight or want to discuss it I'd be happy to

ragge17:06:12

is suspect it's something related to the aot compilation, protocols and classfile mismatch

micha17:06:49

@ragge: i'll have a look

ragge17:06:20

@micha: thanks! i'm probably doing something silly somewhere

micha17:06:17

@ragge: have you looked in the jar yet?

micha17:06:26

to make sure the things you need are in there?

micha17:06:36

like any .clj files you may need specifically

micha17:06:02

from your own project

ragge17:06:37

would I need any .clj in this case? main is aot compiled, the two other namespaces should be transitively compiled too, no?

micha17:06:49

oh ok, yeah that makes sense

ragge17:06:52

i def have classfiles for all those namespaces

ragge17:06:39

i could sort of see this issue happening if the component/Lifecycle protocol classfiles don't match the component/Lifecycle that the TestComponent implements

ragge17:06:07

in that case, when i call start, the TestComponent would not implement the start of the correct interface

ragge17:06:12

even though they're called the same thing

ragge17:06:23

and then start would fallback to the implementation for Object

ragge17:06:27

which is a no-op I believe

micha17:06:02

i recently fixed a bug with uber task that might be relevant

ragge17:06:30

let me know if you need any help re-producing this (or if it simply works with recent bug fix)

micha17:06:52

does that look like it could be a problem?

micha17:06:24

that particular issue is fixed in master

juhoteperi17:06:35

By checking the clojure-version inside pod I can catch cases where user has added clojure to dependencies using set-env

micha17:06:05

deraen: that looks perfect

micha17:06:29

one minor thing is maybe check *clojure-version* instead?

micha17:06:15

(let [{:keys [major minor]} *clojure-version*]
  (when-not (and (= major 1) (> minor 6))
    (warn "Requires minimum Clojure version 1.7.0")))

micha17:06:02

it's a dynamic var that has the clojure version parsed into a map

micha17:06:23

this would just help in the case where someone uses this version of boot-cljs with future versions of clojure

ragge17:06:50

@micha: hard to judge, but I can give master a go

micha17:06:02

@ragge: awesome, thanks!

juhoteperi17:06:02

I also took a stab at implementing asset-path for boot-reload: https://github.com/adzerk-oss/boot-reload/pull/22

juhoteperi17:06:03

@micha: Should work correctly with string compare also, but comparing numbers is better

ragge17:06:27

@micha: can I just clone boot, do a make install and change boot.properties in my project to the master snapshot version?

micha17:06:16

@juhoteperi: not sure what to do about the major version

micha17:06:41

maybe (>= major 1) is the right thing there

juhoteperi17:06:40

@micha: (when (and (<= major 1) (< minor 7)) ...)

juhoteperi17:06:06

@micha: Or no, that doesn warn with version 0.9 :D

micha17:06:18

that's the hardest thing in lisp lol

micha17:06:31

macros are for children

micha17:06:43

the experts struggle with (< 1 2)

juhoteperi17:06:45

This should do it: (or (< major 1) (and (= major 1) (< minor 7)))

micha17:06:13

what about "0.9.0" ?

juhoteperi17:06:19

@micha: That matches the first or branch

jeluard17:06:34

By the time Clojure hits 2.x I am pretty sure this boot task will have different constraints! Maybe just force major = 1.

micha17:06:40

right i was thinking in reverse, matching the acceptable versions

ragge17:06:49

@micha: it works using master so something has fixed it simple_smile

micha17:06:50

instead of matching the versions to warn about

ragge17:06:04

@micha: thanks for helping me resolve it, will close the issue

micha17:06:20

@ragge: haha that was easy, awesome!

ragge17:06:40

@micha: next obvious question is when is next release? 😉

jeluard17:06:01

@micha: about my issue: it looks related with dynamic env after all. I will work on creating a small test case.

micha17:06:02

@ragge: i can make one today

ragge17:06:23

@micha: that would be much appreciated! thanks for the great work on boot, I'm enjoying exploring it

micha17:06:14

@ragge: thanks 🍻

jeluard17:06:30

Related to dynamic env: can one of sub-tasks be defined in a source-path a task just added? i.e.

(deftask dev []
  (merge-env! :source-paths #{"src-dev"})
  ; some call to let boot discover my task
  (comp .. (start)))
with start defined in src-dev. This way my dev related code does not ends up in an uberjar.

meow18:06:54

I'm looking for confirmation that I understand the typical cljs repl workflow using boot. Here is what works for me now:

;; => boot repl -c
;; boot.user=> (start-repl)
;; cljs.user=> (js/alert "This is a test of the emergency broadcasting system.")
;; cljs.user=> (require '[app.core :as app] :reload)
;; cljs.user=> (app/foo 3 4)
;; 7
;; cljs.user=> :cljs/quit
;; boot.user=> (quit)
;; Bye for now!

meow18:06:32

Not sure how unified mode fits in, which I'm not using.

micha18:06:18

@jeluard: you need to require the namespace and then eval the expression

micha18:06:31

otherwise clojure will choke compiling the expression

micha18:06:42

(deftask dev []
  (merge-env! ...)
  (require 'some.namespace)
  (comp ... (eval '(some.namespace/start))))

micha18:06:45

or someting like that

micha18:06:17

@meow: unified mode is always on now in boot-cljs, so you're using it i think

jeluard18:06:30

Ah great thanks

meow18:06:35

@micha: then I guess I am using unified... simple_smile

meow18:06:09

Now, if I add a (defn ...) to my core.cljs file it doesn't get recognized in my repl namespace so it looks like the :reload option isn't working or perhaps I'm still a bit confused by this stuff.

micha18:06:38

hmm, i'll be back in a while

micha18:06:46

time to get a truck and move a couch

juhoteperi18:06:24

@micha: I took a look at boot-cljs-repl update and I think boot-cljs-repl should depend at least on tools.nrepl, looks like if any lib in project depends on tools.nrepl it will bring in an old version

micha18:06:57

deraen: i think maybe :scope "provided" for these issues?

juhoteperi18:06:57

@micha: With scope provided the dependency for cljs-repl is not added to the project?

juhoteperi19:06:43

boot-cljs-repl already has [org.clojure/tools.nrepl "0.2.10" :scope "provided"]. Removing the :scope fixes the problems,.

micha22:06:44

deraen: oh i see what you mean

micha22:06:09

deraen: i don't like the idea of boot tasks adding dependencies to your project

micha22:06:15

but maybe we need to

juhoteperi22:06:32

@micha: I would rather make the tasks warn if user doesn't have necessary deps

juhoteperi22:06:40

User should probably have cljs dependency in the project

micha22:06:01

right, or we could force the correct versions of the deps

micha22:06:17

by filtering the project deps before making the pod

micha22:06:24

and adding our fixed versions to the pod

juhoteperi22:06:35

The problem is that repl doesn't use pods

micha22:06:09

yeah the repl is always a special case because of that

micha22:06:34

still maybe warnings is the right way to go with cljs-repl too

micha22:06:49

ok be back in a bit

ragge22:06:09

hello again simple_smile

ragge22:06:25

because of "Too many files open"

ragge22:06:54

investigated a bit

ragge22:06:05

and I think there may be a file handle leakage in copy-url

ragge22:06:47

it's happening in uber task

ragge22:06:58

in unpack-jar

ragge22:06:13

i ran strace on my build and saw this pattern:

ragge22:06:02

8924  open("/home/ragge/.m2/repository/cheshire/cheshire/5.4.0/cheshire-5.4.0.jar", O_RDONLY) = 147
8924  open("/nix/store/c00cymmlb0p05wrr7wq1h11yqqd8i75r-oraclejdk-8u45/jre/lib/content-types.properties", O_RDONLY) = 148
8924  close(148)                        = 0
8924  open("/home/ragge/.boot/cache/tmp/home/ragge/projects/github/uswitch/mobiles-data-hub/6v2/4naatj/project.clj", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 148
8924  close(148)                        = 0
8924  open("/home/ragge/.m2/repository/cheshire/cheshire/5.4.0/cheshire-5.4.0.jar", O_RDONLY) = 148
8924  open("/home/ragge/.boot/cache/tmp/home/ragge/projects/github/uswitch/mobiles-data-hub/6v2/4naatj/META-INF/leiningen/cheshire/cheshire/README.md", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 151
8924  close(151)                        = 0
...

ragge22:06:08

sorry, I'll include a few more lines for the pattern to make more sense:

ragge22:06:11

8924  open("/home/ragge/.m2/repository/cheshire/cheshire/5.4.0/cheshire-5.4.0.jar", O_RDONLY) = 147
8924  open("/nix/store/c00cymmlb0p05wrr7wq1h11yqqd8i75r-oraclejdk-8u45/jre/lib/content-types.properties", O_RDONLY) = 148
8924  close(148)                        = 0
8924  open("/home/ragge/.boot/cache/tmp/home/ragge/projects/github/redacted/redacted/6v2/4naatj/project.clj", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 148
8924  close(148)                        = 0
8924  open("/home/ragge/.m2/repository/cheshire/cheshire/5.4.0/cheshire-5.4.0.jar", O_RDONLY) = 148
8924  open("/home/ragge/.boot/cache/tmp/home/ragge/projects/github/redacted/redacted/6v2/4naatj/META-INF/leiningen/cheshire/cheshire/README.md", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 151
8924  close(151)                        = 0
8924  open("/home/ragge/.m2/repository/cheshire/cheshire/5.4.0/cheshire-5.4.0.jar", O_RDONLY) = 151
8924  open("/home/ragge/.boot/cache/tmp/home/ragge/projects/github/redacted/redacted/6v2/4naatj/META-INF/leiningen/cheshire/cheshire/LICENSE", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 152
8924  close(152)                        = 0
8924  open("/home/ragge/.m2/repository/cheshire/cheshire/5.4.0/cheshire-5.4.0.jar", O_RDONLY) = 152
8924  open("/home/ragge/.boot/cache/tmp/home/ragge/projects/github/redacted/redacted/6v2/4naatj/cheshire/core.clj", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 153
8924  close(153)                        = 0
8924  open("/home/ragge/.m2/repository/cheshire/cheshire/5.4.0/cheshire-5.4.0.jar", O_RDONLY) = 153
8924  open("/home/ragge/.boot/cache/tmp/home/ragge/projects/github/redacted/redacted/6v2/4naatj/cheshire/custom.clj", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 154
8924  close(154)                        = 0

ragge22:06:21

this is unpacking the cheshire jar

ragge22:06:40

you can see that the open calls to the jar file has no corresponding close

ragge22:06:27

saturday maybe not best day to get people to star at strace logs...