Fork me on GitHub
#boot
<
2017-07-27
>
martinklepsch08:07:07

@lwhorton this gives the basic ideas

lwhorton14:07:30

i felt like you meant to send something but it didn’t go through?

richiardiandrea10:07:11

@micha @alandipert and all, I am planning on moving https://github.com/Lambda-X/boot-pack-source to boot-clj. It will be renamed as well to boot-sources. Any objection?

alandipert00:07:55

none from me

richiardiandrea14:07:41

Moved! https://github.com/boot-clj/boot-sources There will be a nice task for source file concatenation (needed by unrepl)

lwhorton14:07:24

this is just a general conceptual question: I seemed to have misunderstood the purpose of (commit! fileset). I had a task that was caching the previous fileset so it could do a (fileset-diff @old new). I would only commit to the fileset if 1) there was a diff between the prev fs and the current fs 2) there were files relating to my task (filtered using the fileset query api) that changed. If both 1 and 2 were not satisfied, I would only return the fileset exactly as provided, and would not invoke commit!. However on multiple reruns I kept getting invalid nameToPath (goog.closure + clojurescript’s file loader) errors. Looking further it seemed that boot’s .boot/cache/.../hash/hash caching mechanism caused the “problem”.

lwhorton14:07:09

So it seems that even if a task does not want to “do” anything for the given fileset diff, it still needs to commit it’s closed-over tmp-dir! to the fileset, otherwise boot will not spit out the proper files into the new cache directory?

djebbz14:07:47

Hello people, I have troubles merging the file BOOT_HOME/profile.boot and MY/PROJECT/buid.boot. I followed the instructions from https://github.com/jsofra/data-scope to have it always available, instructions are :

(set-env! :dependencies #(conj % '[jsofra/data-scope "0.1.2"]))
(boot.core/load-data-readers!)
(require 'data-scope.inspect)
Then my build.boot file does (set-env! :dependencies '[[w/x "x.x.x"] [y/z "x.x.x"] ...]) normally. But my profile deps aren't included when I launch a boot repl in my project. Then I thought, set-env! of my project erase the dependencies, so I tried updating them instead by following the official docs (https://github.com/boot-clj/boot/wiki/Boot-Environment#update-env-value). I tried various syntaxes with no success :
;; in project's build.boot
;; #1
(set-env! :dependencies #(conj % '[[...]])) ;; throws java.lang.ClassCastException: clojure.lang.PersistentVector cannot be cast to clojure.lang.Named

#2 and #3
(set-env! :dependencies #(conj % [[...]]) ;; dependencies vector not quoted, throws java.lang.ClassNotFoundException: org.clojure
(set-env! :dependencies #(concat % [[...]]) ;; concat + dependencies vector not quoted, throws java.lang.ClassNotFoundException: org.clojure

#4
(set-env! :dependencies #(concat % '[[...]]) ;; no error, not working

dominicm14:07:46

@djebbz you can't set-env! multiple times, try merge-env! in your build.boot

micha14:07:02

i think in your pasted build.boot you could try (set-env! :dependencies #(into % '[[...]]))?

micha14:07:18

because you want to join two vectors there

djebbz14:07:54

Will either work for people in team without BOOT_HOME/profile.boot

micha14:07:44

conj would do like (conj [['foo/bar "1.0"] ['baz/baf "2.0"]] [['quux "3.0"]]) => [['foo/bar "1.0"] ['baz/baf "2.0"] [['quux "3.0"]]]

micha14:07:15

yes dependencies is the empty vector by default

micha14:07:38

that what merge-env! does

djebbz14:07:31

not working : (pprint (get-env :dependencies)) after running boot repl does not include the jsofra/data-scope stuff

djebbz14:07:40

let me check...

djebbz14:07:25

$ ls $BOOT_HOME 
=> profile.boot

djebbz14:07:52

$ cat $BOOT_HOME/profile.boot 
(set-env! :dependencies #(conj % '[jsofra/data-scope "0.1.2"]))
(boot.core/load-data-readers!)
(require 'data-scope.inspect)

djebbz14:07:24

Could it be related to data-readers ?

djebbz15:07:10

Giving up for now... Too bad, I really wanted it to work.

djebbz15:07:47

Last try, do you have a merge-env! example ? There's none in the docs

dominicm15:07:50

same use as set-env!

djebbz15:07:37

Well, (merge-env! :dependencies #(into % '[[]])) throws java.lang.IllegalArgumentException: Don't know how to create ISeq from: boot.user$eval47$fn__48

dominicm15:07:40

@micha is it worth having docs & guides default to suggesting merge-env! all the time? Seems sensible, especially if you stick it in a task.

dominicm15:07:11

@djebbz sorry, I meant your original use of set-env!, (merge-env! :dependencies '[[yada "1.2.1"]])

micha15:07:15

@dominicm yeah i think it's a good default

djebbz15:07:33

@dominicm still no success, no error but profile.boot stuff still not included

micha15:07:49

@djebbz why do you say "not included"?

micha15:07:53

is it on the classpath?

djebbz15:07:08

I even added (println "***********\n PROFILE.BOOT \n***********") in profile.boot, not printed when I start a REPL

micha15:07:10

i.e. (require 'data-scope.inspect)

micha15:07:14

in the repl

djebbz15:07:49

java.io.FileNotFoundException: Could not locate data_scope/inspect__init.class ...

micha15:07:55

what is the command line you're using to launch boot?

djebbz15:07:51

a custom task that launches (repl :some 'options)

djebbz15:07:07

(repl :port 9091 :eval '(do (require 'user)))

micha15:07:14

i mean what do you type in the shell to start boot

djebbz15:07:23

boot back-dev

micha15:07:36

can you do boot -b back-dev please?

micha15:07:56

you should see the boot.user namespace it generates printed to stdout

micha15:07:05

does that contain your profile.boot contents?

djebbz15:07:50

no it doesn't

micha15:07:14

where is your profile.boot file located?

djebbz15:07:17

pretty sure now that this is the problem

micha15:07:24

is it at ~/.boot/profile.boot?

djebbz15:07:55

No :

echo $BOOT_HOME/profile.boot 
/home/djebbz/.config/boot/profile.boot

micha15:07:09

ok that looks legit to me

djebbz15:07:58

My complete profile.boot (very small) :

(set-env! :dependencies '[[jsofra/data-scope "0.1.2"]])
(boot.core/load-data-readers!)
(require 'data-scope.inspect)

(println "***********\n PROFILE.BOOT \n***********")

djebbz15:07:37

i expected the println to be displayed whenever I launched boot repl or boot back-dev

micha15:07:48

if you don't see that in the boot -b ... output there must be something going on with BOOT_HOME

micha15:07:04

is there a boot.properties file in the project maybe?

djebbz15:07:17

Yes there is, but no BOOT_HOME inside

djebbz15:07:31

boot.properties :

#
#Thu Dec 01 16:19:49 CET 2016
BOOT_CLOJURE_NAME=org.clojure/clojure
BOOT_CLOJURE_VERSION=1.9.0-alpha15
BOOT_VERSION=2.7.1
BOOT_EMIT_TARGET=no

micha15:07:34

try (System/getEnv "BOOT_HOME") in the repl

micha15:07:02

after launching boot

micha15:07:15

i guess that's not going to be helpful

djebbz15:07:16

boot.user=> (System/getEnv "BOOT_HOME")

     java.lang.IllegalArgumentException: No matching method: getEnv
clojure.lang.Compiler$CompilerException: java.lang.IllegalArgumentException: No matching method: getEnv, compiling:(/tmp/boot.user3111401912711538582.clj:1:1)

micha15:07:32

oh sorry, System/getenv

djebbz15:07:37

boot.user=> (System/getenv "BOOT_HOME")
nil

micha15:07:38

i always do that

micha15:07:49

so that's odd

micha15:07:54

and it would explain the issue

djebbz15:07:56

in a raw boot repl

djebbz15:07:13

Let me try from a fresh Terminal

dominicm15:07:01

@micha does boot.properties search upwards ever? '@djebbz do you have the same result when you start from /tmp ?

dominicm15:07:15

I'd also suggest checking your .bashrc or similar

micha15:07:50

as a quick check to convince yourself you can make a symlink from /.boot to /.config/boot or something

djebbz15:07:52

The equivalent of .bashrc for me is where I set BOOT_HOME

djebbz15:07:22

In /tmp, after boot repl :

boot.user=> (System/getenv "BOOT_HOME")
nil

micha15:07:29

which shell?

micha15:07:39

do you use

micha15:07:48

oh dear 🙂

micha15:07:04

i don't know how to debug that one

micha15:07:19

but it doesn't appear to be setting BOOT_HOME as an environment variable

micha15:07:27

just as a shell variable

micha15:07:37

so child processes won't see it

djebbz15:07:14

indeed, env | grep BOOT returns nothing

djebbz15:07:59

Damn, now looks ok :

env | grep BOOT
BOOT_HOME=/home/djebbz/.config/boot

micha15:07:30

that's /usr/bin/env right?

micha15:07:38

not a shell builtin?

djebbz15:07:48

which env
/usr/bin/env

micha15:07:49

sweet, so that should be all a-ok now

djebbz15:07:20

Strangely, I've just typed boot repl in /tmp, and it displayed this :

djebbz15:07:24

Downloading ...
Running for the first time, BOOT_VERSION not set: updating to latest.
#
#Thu Jul 27 17:21:24 CEST 2017
BOOT_CLOJURE_NAME=org.clojure/clojure
BOOT_VERSION=2.7.1
BOOT_CLOJURE_VERSION=1.7.0

micha15:07:50

you may have a boot.properties file in there

micha15:07:03

that tricked it into downloading a different version of boot

djebbz15:07:06

I have one in ~/.boot/

micha15:07:13

i mean in /tmp

micha15:07:21

it looks in CWD for a boot.properties file also

micha15:07:27

like if you have one pinned to your project

djebbz15:07:35

not in /tmp

micha15:07:38

so if you have a garbage one in /tmp it would do that

micha15:07:48

oh i get it

micha15:07:04

now that you fixed your BOOT_HOME setup it sees your boot.properties file in there for the first time

micha15:07:11

and gets itself set up accordingly

djebbz15:07:40

ok makes sense

djebbz15:07:55

Look, boot repl prints what's in my profile.boot !

micha15:07:12

haha what was the original thing you wanted to do again?

micha15:07:16

does it do it?

djebbz15:07:23

Let me try

djebbz15:07:27

Suspens...

micha15:07:34

antici...

djebbz15:07:44

Yo yo yo yippi yo yippi yeah

djebbz15:07:20

Thanks A WHOLE LOT for your support guys @dominicm @micha

djebbz15:07:35

If I ever meet you in some Clojure conf, remind me to offer you a drink !

micha15:07:52

🍻 certainly!

djebbz15:07:11

For the notice, I still have to change my original set-env! to have the deps from my profile.boot working in my project's repl

dominicm15:07:51

@micha I mean, boot should probably have an XDG compatible version 😉

djebbz15:07:33

For the notice, I open an issue in the new tools.deps.alpha repo from Clojure core for this exact thing, hoping that the new clj tool will store and search stuff in ~/.config/clojure

dominicm15:07:13

and cache into ~/.local!

djebbz15:07:11

By the spec, cache goes into ~/.cache

dominicm15:07:17

Good point. What is .local for.. I realise I'm using it for my buffer files.

micha15:07:44

i use it to install things in my account

micha15:07:54

like instead of /usr/local

djebbz15:07:07

But regarding Boot, it's already nice to have BOOT_HOME, it allows anyone to customize it

dominicm15:07:34

I think I'm using it wrong

djebbz16:07:57

Oh, the dependencies in profile.boot (loaded with (set-env! :dependencies '[[]]) worked in my project, which loaded them the same way. No merge-env!, no into, concat, whatever. Still worked.

djebbz16:07:26

I though they would be overridden what's in my build.boot

djebbz16:07:59

Anyway, it works without changing anything, cool

micha16:07:46

so the :dependencies in boot env are really just informational once the deps have been added to the classpath

micha16:07:59

that is because there is no way to remove a class from a classloader once it's in there

micha16:07:33

the only issue with overwriting :dependencies is that subsequent maven operations also won't know about those deps

djebbz16:07:52

oh, ok, so chain of (set-env! :dependencies '[[]]) in the various files that boot understands will always work

micha16:07:04

so the subsequent maven dependency graph will be computed without that info

djebbz16:07:12

Ah, it means this chain shouldn't be used inside a project

micha16:07:20

it's best to add all your dependencies in one call to set-env

djebbz16:07:29

Yes, I understand

micha16:07:42

for dev tooling it's usually ok to load some in the profile, because those generally won't have lots of transitive dependencies

micha16:07:00

if they do have transitive dependencies you could run into annoying issues though

djebbz16:07:12

Well, it just happens that I had 3 conflicts between dev deps and project deps

micha16:07:26

but boot has pods for that

djebbz16:07:28

Hopefully, using :exclusions in my profile.boot worked

djebbz16:07:47

Haven't looked closely at pods yet

micha16:07:52

you can always load dev dependencies into a pod, which eliminates transitive deps completely

micha16:07:06

the idea is this

djebbz16:07:07

Let me RTFM

djebbz16:07:14

ok I'll listen

micha16:07:16

a pod is an isolated clojure runtime

micha16:07:34

they're first class, meaning you can create an anonymous pod for example

micha16:07:58

and they have protocols and stuff that let you do things like evaluate expressions in the pod and obtain the result

micha16:07:18

so the idea is to create a pod with the dependencies you need for some dev task

micha16:07:42

and in the pod you have a namespace usually with functions that will form the pod's api

micha16:07:15

then you can take the code that creates the pod and so on and make that a clojure library and deploy a jar to clojars

micha16:07:46

now in a project you can add that dependency (it won't have any transitive dependencies because it loads all the things it needs into a pod)

micha16:07:23

and once that dependency is loaded it will usually have some api namespace with functions you can call in your dev environment

micha16:07:34

and all the work is done in the pod behind the scenes

micha16:07:43

does that make sense?

djebbz16:07:14

But do we have to go through deployment to clojars ?

micha16:07:36

you'd just do that if you wanted to have a unit of code you can use in many projects

micha16:07:23

like in boot for example

micha16:07:44

if you look on the classpath in the repl, none of the maven pomegranate things are on the classpath at all

micha16:07:00

set-env! actually delegates to a function that runs in a pod

djebbz16:07:08

So if understand correctly, if I put my deps in a pod and expose what I like in some namespace in profile.boot, I could always use this dev namespace for any project without ever having a conflict ?

micha16:07:09

and that pod has all the pomegranate and maven dependencies

micha16:07:58

yes i thonk you could do that

micha16:07:15

the one place where pods won't work is when the code needs to run in the same clojure runtime

micha16:07:29

like for example if it needs to have access to the same vars

micha16:07:52

so the nrepl server task for example

micha16:07:58

it can't start nrepl in a pod really

micha16:07:13

because the whole point of it is usually to manipulate the dev environment

micha16:07:37

if the nrepl server was started in its own isolated pod it wouldn't have access to your project vars

micha16:07:04

but most dev tooling stuff just needs to perform some work like compile, transform, etc

micha16:07:13

and usually that can be isolated just fine

djebbz16:07:45

I should give it a try. I discovered at work before the lucidity lib, which has in/`inject`, which take some deps declaration (with :refer etc.) and inject them all in a namespace named .. I was shown its usage in a lein repl, and the guy was able to do ./pprint for example from any namespace from his repl.

djebbz16:07:14

If I manage to combine this stuff and a pod, I would never have to worry about conflict.

micha16:07:13

conflicts between dev and project dependencies only arise in two cases really:

micha16:07:31

1. transitive dependency versions

micha16:07:57

2. your dev dependency is itself a dependency of your project, possibly via transitive dependency

micha16:07:26

so as long as the inject dep itself doesn't have transitive dependencies you're probably fine

micha16:07:42

because that's not something your project would ever depend on

djebbz16:07:42

Sorry, I'm totally new to this class of problems. Say in my dev deps (in profile.boot) I load deps X which depends on deps Y version 1. My project loads deps Z which depends on deps Y version 2. If I use the pod trick, there's never going to be a conflict ?

micha16:07:11

that is correct, yes

micha16:07:34

pods solve issue 1

djebbz16:07:23

Pods look awesome !

djebbz16:07:53

Thanks again for your support, you've just won another drink 🍹