Fork me on GitHub
#boot
<
2016-10-31
>
jannis10:10:18

Is there some clean way of adding dependencies to a project (and the uberjar + pom built from it) from a custom-written task? set-env! or merge-env! successfully update :dependencies in the boot env but if you have a pipeline with uber and pom, those will use the dependencies as they were when constructing the pipeline - so prior to executing earlier tasks like the one that would add the dependencies.

martinklepsch13:10:51

@jannis I remember seeing something for this but I don't find it in the API docs right now. @micha will know. That said generally I think what you're trying to do might not be advisable since it affects dependency resolution and you might get different set of deps than if you'd run it without your task

jannis13:10:08

@martinklepsch Yes, although that's kind of what I'm looking for. I may have found a better way though: have a function similar to bootlaces! that I call before the task definitions. That can then add the dependencies so it doesn't have to be done in a task.

jannis14:10:50

Has anyone seen this error before, when building with boot (I think it's from aot): java.lang.ClassNotFoundException: java.util, compiling:(clojure/tools/reader/default_data_readers.clj:1:2)?

martinklepsch14:10:34

not seen it but general consensus around here has often been that avoiding AOT is best as long as you don't absolutely need it

jannis14:10:27

Isn't it needed when building an uberjar for standalone execution?

martinklepsch14:10:20

yes but you can make a "shim namespace" that essentially does something like this:

(require ')
((resolve '))

martinklepsch14:10:45

then you only AOT that namespace which has zero dependencies and you don't get into any of the AOT related troubles

jannis14:10:16

Let me try that quickly. šŸ™‚

martinklepsch14:10:28

good luck šŸ™‚

jannis14:10:47

Thanks šŸ™‚

martinklepsch14:10:37

@dominicm whats the benefit over the clj approach?

jannis14:10:12

Ok, that worked (compiling at least). Let's see if it runs.

dominicm14:10:53

@martinklepsch my understanding is that it's simpler to understand, but I might be wrong. I know the equivalent clojure code might be considered confusing. I think it was mostly about skipping a step, given how short the java code is. There was some talk about injecting that file into the fileset as an option for the aot task.

martinklepsch14:10:12

Was also thinking that this might be ncie

martinklepsch14:10:17

anyways, gotta go

jannis14:10:53

Thanks @martinklepsch, that put me in the right direction, I think!

micha14:10:17

@jannis the pom task also accepts dependencies as an option, overriding the env ones

micha14:10:29

and you can have uber omit build dependencies if you make those build dependencies scope "test"

micha14:10:38

by default the uber task only includes dependencies of scope compile, runtime, and provided

micha14:10:36

generally it is sufficient to use scope test for all the build dependencies, the pom and uber etc will all work correctly then automatically

zane14:10:06

> if you make those build dependencies scope "test" This was a surprise to me!

zane14:10:28

Seems kind of magical / runs counter to boot-clj's philosophy.

micha14:10:25

it's really just the pom spec

micha14:10:57

not really a boot thing

jannis14:10:10

@dominicm How does the boot-uberjar-example ensure my-namespace is available in the built uberjar (since it's not required anywhere other than in main.Main)? Is it the sift task?

micha14:10:09

the pom dependency scopes work correctly with any maven client

dominicm14:10:42

@jannis I believe it needs to be on your resource path, instead of your source path

dominicm14:10:56

Your code should only be in source-paths if you're aot'ing it

jannis14:10:11

Oh! That would make sense.

micha14:10:22

^^^ and if you don't want the source in the jar

jannis14:10:23

Yep, that's probably it.

zane14:10:39

@micha: Ah! TIL. Thanks.

jannis14:10:06

@micha Good point šŸ™‚

micha14:10:15

both source and resource are on the class path, but source are excluded from packaging like jars

jannis14:10:17

I remember that now. It's why all CLJS libraries have their sources in :resource-paths.

jannis15:10:29

The trick with main.Main and javac is nice. However, it seems to also build .java sources from dependencies, generating duplicate classes and, thereby, a java compiler error.

jannis15:10:42

I guess I should put javac before uber. Ah. Better.

jannis15:10:46

Hm. And after all that, I get the Exception in thread "main" java.lang.ClassNotFoundException: java.util, compiling:(clojure/tools/reader/default_data_readers.clj:1:2) error at runtime, rather than during AOT. So it wasn't aot. I wonder what it is then.

micha15:10:47

java.util isn't a class

jannis15:10:17

Yep. But somehow Clojure thinks it is when it sees that (:import [java.util Calendar ...]) line in tools.reader: https://github.com/clojure/tools.reader/blob/master/src/main/clojure/clojure/tools/reader/default_data_readers.clj#L12

jannis15:10:32

No, it's probably not about that line.

jannis15:10:53

I'm just not sure how to interpret the error message and where to look for the cause.

dominicm15:10:06

Seems like some kind of misdirection is going on here..

micha15:10:17

in a repl can you require the tools namespace?

micha15:10:41

the tools reader namespace

jannis15:10:01

The whole project starts and works fine if I run it from the REPL.

jannis15:10:30

I add one dependency via merge-env! and aot fails. If I leave the dependency out it compiles fine.

jannis15:10:45

Could it be a problem that this dependency is a Clojure project that was also pre-compiled with aot?

jannis15:10:58

And that it doesn't include the sources in its JAR?

micha15:10:56

@jannis if you're only aot a namespace that has no :requires it shouldn't try to compile any other namespaces

jannis15:10:57

That's not the problem here. I'm looking at "compile project X with aot, which depends on Y, which was also precompiled with aot". X doesn't even refer to Y anywhere in the code. It's just added as a dependency and it breaks aot compilation of X... I wish this project was open source and simpler, then I could share it.

micha15:10:50

are you sure you need to compile X?

micha16:10:54

although if X doesn't refer to Y there isn't any way that i can think of where they would interfere with each other

jannis16:10:26

It is to be built into a standalone uberjar, so, yeah. I tried with the shim instead of aot but I get the same ClassNotFoundException: java.util error at runtime instead of at compile time that way. That's the only difference.

jannis16:10:19

Hmm. Y is also an uberjar, perhaps that's the problem.

micha16:10:28

that looks to me like something is trying to import java.util, which isn't a class

micha16:10:40

it's a package

jannis16:10:41

Yeah, no idea where that should be coming from...

micha16:10:44

uberjar in maven is highly unusual and definitely not recommended:)

micha16:10:07

it defeats all of the maven dependency machinery

micha16:10:24

and is almost impossible to debug

micha16:10:00

it's basically cloaking dependencies and snuggling them in under a different name and with no version info

micha16:10:14

smuggling even

micha16:10:29

highly likely that you'll see very strange bugs because of it

jannis16:10:52

Yeah, hm. This is supposed to become a modular system, where each module can be executed as a standalone application but can also define code that can be imported into other projects.

micha16:10:18

that's OK you can have the dependencies separate

micha16:10:26

boy does this in fact

micha16:10:36

boot does this i mean

jannis16:10:42

How do you mean?

micha16:10:45

phone autocorrect

micha16:10:10

boot ships with an uberjar inside of boot.jar

micha16:10:48

that uberjar contains all the maven stuff boy needs to resolve is own dependencies the first time you run it

micha16:10:19

like if you download boot.sh and have an empty maven cache

micha16:10:51

but that uberjar is constructed from normal maven dependencies

micha16:10:05

boot/pod, etc

micha16:10:45

point is that the uberjar is built from normal separate maven dependencies

micha16:10:25

it doesn't contain the modules itself except via its dependencies

micha16:10:58

put the module in a normal maven package and make a separate uberjar that depends on it

jannis16:10:37

I was wondering what you meant but that last line is all I needed to know šŸ˜‰

micha16:10:47

also having two uberjars on the class path at the same time is a recipe for madness

micha16:10:07

because they could contain conflicting classes

jannis16:10:13

Yeah, that makes sense. Build one "library" jar and one "runtime" uberjar for each module basically.

jannis16:10:19

Yeah, absolutely

jannis16:10:42

Cool, thanks!

ag21:10:13

hey guys.. can someone help meā€¦ Iā€™m trying to serve stuff in ring handler from node_modules, what should I use? wrap-resource, wrap-file, something else?

ag21:10:27

whatā€™s the right syntax?

ag21:10:23

can I combine use both wrap-resource and wrap-file?

ag21:10:29

canā€™t figure this out

richiardiandrea21:10:11

@ag if your node_modules is in resource-paths you should be able to use wrap-resource only

richiardiandrea21:10:25

but keep in mind that if you have #{"node_modules"} then boot/java will include the content within the folder only

richiardiandrea21:10:22

I mean, you won't see the node_modules folder in there and wrap-resource should use "." as root path

ag21:10:30

okā€¦ let me try, it never occurred to me to tweak resource-paths

richiardiandrea21:10:59

you have two options actually, you could avoid that and use wrap-file as well

richiardiandrea21:10:28

probably the latter is more easy to use as it does not clutter your mind with the resource trickery

ag21:10:58

my mind is already quite cluttered.

ag21:10:19

tried doing this: :resource-paths #{"resources" ā€œn ode_modulesā€}. That hangs boot-clj

anmonteiro21:10:15

@ag try this:

(deftask add-node-modules []
  (with-pre-wrap fileset
    (let [nm (io/file "node_modules")]
      (when-not (and (.exists nm) (.isDirectory nm))
        (dosh "npm" "install" "react"))
      (-> fileset
        (add-resource (io/file ".") :include #{#"^node_modules/"})
        commit!))))

richiardiandrea21:10:21

ok so probably because of the number of the files

richiardiandrea21:10:46

ok see above ^ šŸ˜„

anmonteiro21:10:50

as @richiardiandrea said, you need to include "." to include node_modules

anmonteiro21:10:59

or else its contents will be included

richiardiandrea22:10:49

@anmonteiro it looks like boot hangs before that even

anmonteiro22:10:03

now a word of caution: it's probably OK if you don't have a lot of modules, but you'll probably see substantial slow down in your build pipeline if you have a lot of modules

richiardiandrea22:10:29

^ this is why I think your solution with io/file might be better

richiardiandrea22:10:47

and wrap-file for serving

ag22:10:43

can someone give me insight on deraen.boot-sass I canā€™t figure out how to feed a single .sass file into it and get a single .css ?

ag22:10:24

or maybe thereā€™s better alternative for sass in boot?

richiardiandrea22:10:52

say you want a css/app.css on your class path, you need to add a folder with css/app.scss within :source-paths

mattly23:10:29

alternately you can set :output-dir in your options for the sass task

mattly23:10:25

oh, wait, I'm using mathias.boot-sassc, sorry

ag23:10:08

@richiardiandrea thatā€™s the thing - it just grabs ALL sass files in the folder and tries to compile them. I need to tell it to pick up a single file.

richiardiandrea23:10:50

@ag ok then I did not understand the issue in the first place šŸ™‚ ..to pick one you would probably need to use sift with a regex and filter out the unwanted files...

ag23:10:18

pfffā€¦ migrating to boot turned out to be a lot more work than I initially expectedā€¦ Kinda having hard time to convince my colleagues that this is still worthy effort

richiardiandrea23:10:15

in this case tough, you have a bunch of scss files in the src-paths... it makes sense for the task to compile all of them right? can I ask you the reason why you want to pick one only?

richiardiandrea23:10:16

also, I think that if you put an underscore as first character of the file name, it will be ignored

richiardiandrea23:10:23

should be documented

ag23:10:24

@richiardiandrea yeah I found ā€œ_ā€ thingy

ag23:10:11

needless to say; ā€œI hate sass. Wish I could use Garden"

ag23:10:24

now this thing is trying to compile every sass/scss file it can lay hands onā€¦ remember my problem with npm? itā€™s trying to compile everying in node_modulesā€¦ sigh

ag23:10:33

java.lang.Exception: Error: File to import not found or unreadable: ../node_modules/rs-breakpoints/index.scss
                            Parent style sheet: /Users/ag.ibragimov/.boot/cache/tmp/Users/ag.ibragimov/DevProjects/finops-admin/1hwn/6qp7pj/node_modules/rs-golden-ratio-grid/sandbox/../src/index.scss
                             on line 2 of node_modules/rs-golden-ratio-grid/sandbox/../src/index.scss
                     >> @import '../node_modules/rs-breakpoints/index.scss';
why the hell is it doing so?