Fork me on GitHub
#boot
<
2016-08-15
>
stuarthalloway11:08:46

I am trying to get boot approved in an environment with private, locked down, maven repositories, and I am encountering a problem where boot seems to dynamically need clojars on the fly

stuarthalloway11:08:11

In particular, the aot task seems to want to hit Clojars

stuarthalloway11:08:05

I have narrowed this down to the following line:

stuarthalloway11:08:14

(future (pod/make-pod (core/get-env)))

stuarthalloway11:08:55

if clojars is not listed in :repositories, this causes

stuarthalloway11:08:35

ArtifactNotFoundException Could not find artifact boot:pod:jar:2.6.0 in ...
        org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap (WagonRepositoryConnector.java:947)
        org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap (WagonRepositoryConnector.java:941)
        org.sonatype.aether.connector.wagon.WagonRepositoryConnector$GetTask.run (WagonRepositoryConnector.java:669)
        org.sonatype.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:60)
        java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1142)
        java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:617)
        java.lang.Thread.run (Thread.java:745)

stuarthalloway11:08:21

;; sneaky clojars consumption?
(set-env! :repositories [["central" {:url ""}]])
(require '[boot.pod :as pod]
         '[boot.core :as core])
(def pref (future (pod/make-pod (core/get-env))))
@pref

stuarthalloway11:08:56

^^ ok, so that seems to end up down at aether/resolve-dependencies

stuarthalloway11:08:07

I don’t think aot compilation (or any task that is not specifically about deps) should talk to maven repos

stuarthalloway11:08:22

Hm, this might be fixable by passing :retrieve false to aether calls used by aot

stuarthalloway11:08:57

Is this kind of aether use common elsewhere in boot?

tcrawley13:08:55

@stuarthalloway: Is this the first execution of boot on the current machine? I believe the boot binary bootstraps on first exec, downloading the rest of boot. So I think you would see that behavior with any task call, not just aot

stuarthalloway13:08:39

no matter how many times you run it, aot makes a network call

stuarthalloway13:08:55

even if everything is locally resolved already

tcrawley13:08:17

Ah, yes, I see the same behavior here. The issue is likely aether's repository checking

tcrawley13:08:31

it stores the repository where it pulled the artifact from in ~/.m2/repository/boot/pod/2.6.0/_maven.repositories, then, when loading that artifact, it confirms that one of the repos listed there is in :repositories

stuarthalloway13:08:13

definitely wrong behavior for a task named aot

tcrawley13:08:57

it's not the aot task, it's boot trying to build it's classpath - it asks aether for the jars for its deps, and it triggers this

stuarthalloway13:08:04

would :retrieve false not be the right thing in all aot cases

stuarthalloway13:08:20

you can tell aether to ask only for jars it already has cached locally

stuarthalloway13:08:44

and some pre-aot step should be responsible for talking over the network

tcrawley13:08:50

maybe so. I'm not a boot dev, @micha would probably have a better viewpoint

stuarthalloway13:08:52

your viewpoint looks upside down 🙂

tcrawley13:08:02

the repository tracking is a security feature - aether may still check that with :retrieve false. If so, it may still fail to use the artifact on disk, but I haven't tested that

stuarthalloway13:08:01

yeah. 😞 If that cannot be avoided aether is unsuitable for the job of building classpaths

Alex Miller (Clojure team)13:08:37

almost everything about aether is pluggable. I would be surprised if there is not something that could be done.

alandipert13:08:29

true also about boot 🙂

alandipert13:08:38

aot is a tiny task, you could easily make your own that doesn't even use a pod

alandipert13:08:15

it definitely does make sense to me to eliminate pod dep resolution where we can in boot though

alandipert13:08:47

the other thing you can do is deploy boot jars to yuor private maven server

alandipert14:08:23

when i was at Big Corp we had to do this for auditing anyway

borkdude14:08:29

New ClojureScript site is up. Boot is still missing on tools / guides page. http://clojurescript.org/tools

stuarthalloway14:08:29

@alandipert: yes, my current plan is to write our own just-the-aot-please task, but switching to a :retrieve false in boot is worth investigating

Alex Miller (Clojure team)14:08:56

@borkdude: please submit an issue or a PR to add it!

stuarthalloway14:08:57

@alandipert the “deploy boot jars to your server” makes sense but is tricky if one cannot enumerate the tasks that might talk over the network, e.g. i would not expect aot to fail in an environment with no network connectivity

stuarthalloway14:08:30

@alandipert should I make a ticket?

alandipert14:08:37

agreed, which is why i think you're right and that we should try to fix for boot

alandipert14:08:46

in the general case of open-world tasks tho it's insoluble

alandipert14:08:54

sure, please do

micha14:08:47

@stuarthalloway: I think you will need to have a fresh maven cache and set the BOOT_CLOJARS_REPO, BOOT_MAVEN_CENTRAL_REPO env vars

micha14:08:17

if your local mavan cache contains artifacts from clojars, aether will need to talk to clojars sometimes

micha14:08:37

also if you want to lock the build down to specific repos you don't want artifacts from clojars in your cache

micha14:08:11

once you have a maven coordinate cached in your .m2, aether will pin that coordinate to the repo you got it from

micha14:08:22

as i understand it

micha14:08:38

it seems like a security feature kinda

stuarthalloway14:08:54

that all makes sense, except the “need to talk to the repo again” part. If all the deps are local, there should be a way to avoid that

stuarthalloway14:08:14

admittedly if aether does not expose that we are kinda stuck

stuarthalloway14:08:22

I will be testing this later today

micha14:08:39

there could be a good reason it does what it does, too

micha14:08:50

getting deps right is tricky

stuarthalloway14:08:22

I will reserve judgment until I know what it does 🙂

micha14:08:31

i don't have access to a nexus setup or i would test the env var setup

micha14:08:03

it would be good to establish a best practices guide to locking boot down, i will start a wiki page

vikeri15:08:52

I can’t get boot-cljs to work with closure defines. Anyone has run into the same issue? I tried both in the main.cljs.edn

{:require [env.android.main]
 :init-fns [env.android.main/main]
 :compiler-options {:optimizations :none
                    :closure-defines {’my.ns/platform ”android"}}}
as well as directly in the task
(cljs :compiler-options {:optimizations   :none
                                      :closure-defines {”my.ns.platform” "android"}}
                   :ids #{"main”})

juhoteperi15:08:32

:closure-defines {"goog.LOCALE" "fi"} this is working fine for me

juhoteperi15:08:17

Not sure how self defined vars work

vikeri15:08:46

Can’t get goog.LOCALE to work either 😐

micha15:08:07

in case you can't figure the gcl stuff out

vikeri15:08:35

@micha: Can I access them from within the cljs code at compile time?

micha15:08:05

that's the idea

micha15:08:11

it's a cljs macro

micha15:08:46

in your build.boot you can set the vars, or in your shell, whatever

micha15:08:00

then in your cljs you can access the compile-time values

micha15:08:34

it works the same in cljs and clj

vikeri15:08:50

Great! My use case is parametrizing android vs ios builds. I’ve had issues with goog-define before. So I’ll try that lib.

micha15:08:02

yeah that's what i do

micha15:08:05

worked fine so far

anmonteiro15:08:39

is there an option to pass a (Java) property to boot repl?

micha15:08:34

you can use the BOOT_JVM_OPTIONS env var to set system properties

micha15:08:04

or System.setProperty("foo.bar", "baz") in your build.boot file

vikeri15:08:40

@micha: Implemented! Not entirely clear how to use is from the readme but I figured it out.

micha15:08:55

what was missing in the readme?

bsima20:08:05

is there anything like lein-modules but for boot?

flyboarder22:08:01

@bsima: You can specify multiple different targets using the target task, this way you can create tasks for different builds

bsima22:08:14

mmm, good idea