Fork me on GitHub
#boot
<
2015-08-29
>
estsauver00:08:28

So, I’m thinking about writing some sort of task that implicitly allows me to share a set of pods across tasks. I would really like environ to be operating in the same pod as tests is, but that doesn’t seem to be possible by default.

micha00:08:25

estsauver: all pods see the same environment

micha00:08:41

so if you set env vars or system properties all pods will see them

estsauver00:08:05

The environ task isn’t setting system properties/env vars, it’s altering a clojure variable.

micha00:08:15

lexical bindings aren't "environment" simple_smile

estsauver00:08:58

Truth! So, I guess my question would be what’s the best way to work around a problem like this?

micha00:08:07

use system properties?

estsauver00:08:19

It looks like boot-test creates its own pods

micha00:08:42

there are 2 pods involved with starting boot

micha00:08:52

and then tasks will make more

estsauver00:08:49

Huh, okay, it seems like a super bizarre choice to have the pod-pool be a hidden object, so to speak.

estsauver00:08:09

It would be wonderful to have access to something like “middleware” for pods.

micha00:08:15

how do you mean?

micha00:08:20

middleware?

estsauver00:08:34

So, it would be nice to be able to do things to the clojure runtime in other tasks and have that effect “persist” into other tasks.

micha00:08:06

it's not really possible, unfortunately

micha00:08:20

no clojure objects can pass between pods

micha00:08:55

we use a system of pr-str and read-string to send java strings between pods so you can do things with them

estsauver00:08:00

Well, I guess the same way filesets are passed through boot middleware, I would like to see pods passed?

micha00:08:33

pods are intended to be private really

micha00:08:47

but you can pass them around of course

micha00:08:06

you can pass pods to functions etc

escherize05:08:01

I'm curious to use boot for making a blog, as a way to learn about boot for the first time. I think I remember reading about the existance of a.. #{project library template workflow} to do that.

escherize05:08:31

googling boot blog leads me to the boot blog. closurelib

escherize06:08:58

ahh hah! thanks crisptrutski :fist:

martinklepsch09:08:21

@estsauver: environ not working in pods is confusing, I agree. It makes many tasks that use pods almost useless if your program relies on env vars. :(

martinklepsch09:08:13

Don't know if something like middleware is the answer but it's definitely worth thinking about.

micha14:08:37

@martinklepsch: env vars work fine though

micha14:08:48

i mean actual environment variables

micha14:08:05

lexical bindings and clojure vars are not "environment variables"

micha14:08:16

i don't think it's a great idea to complect the two

micha14:08:54

environment variables come from the environment, which can be export FOO=bar in the shell before running the program or java -DFOO=bar, or even something like (System/setProperty "FOO" "bar") in your build.boot or something like environ that would set properties

micha14:08:53

if environ isn't working because of pods i would say that environ is maybe being misused

micha14:08:02

respectfully simple_smile

martinklepsch14:08:19

Maybe environ doing an actual setproperty call is an option for boot environ. Only thing I see is that a test suite or a ring handler might rely on an env var and that it's not supplied as people expect it. Not saying that this is wrong, only confusing/not "easy"

micha14:08:10

right, but i think it's fundamentally doomed to try to equate clojure vars to env vars

micha14:08:17

for exactly this reason

micha14:08:35

env vars are part of the environment, meaning the entire jvm sees them the same

micha14:08:44

clojure vars aren't like that, and they can never be like that

micha14:08:53

they belong to a specific classloader

micha14:08:04

that's why pods work

micha14:08:32

i think the 12-factor method is pretty good sense

micha14:08:01

they have a section about why environment vars are not the same as config files and variables in your program

micha14:08:14

personally my projects actually use env vars to configure themselves

micha14:08:31

i have an encrypted env file i source in the shell before i start boot

micha14:08:05

and when i deploy to elastic beanstalk for instance or tomcat or whateevr i set environment variables in the container when i create the environment

micha14:08:52

so the application runs anywhere, and the things that depend on where it's deployed are provided by the environment as env vars

micha14:08:20

so when i run locally for tests it's like a local deploy

micha14:08:31

very clean and simple to reason about and maintain

micha14:08:11

basically parameterize the environment with env vars

micha14:08:35

like i can't have database passwords in config files, that's just a no-go for my job

martinklepsch18:08:50

I’m trying to debug that multiple task invocation issue

micha18:08:25

@martinklepsch: can you try a test while you're in there, disabling the extra directories being passed to the cljs compiler?

micha18:08:41

i think that might be the cause of the recent regression with compile times

martinklepsch18:08:35

ehm, not sure what you mean. I’m looking into this: https://github.com/martinklepsch/boot-cljs-multiple-builds

micha18:08:50

i think cljs will compile faster if directories is nil

juhoteperi19:08:43

I asked dnolen if all the dirs should be passed to build

martinklepsch19:08:51

The issue is that on the second invocation the cljs edn file contains something like `{:require [app.out.boot.cljs.main864 app.out.boot-cljs-multiple-builds.app app.out.boot-cljs-multiple-builds.separate app.out.cljs.core boot-cljs-multiple-builds.app boot-cljs-multiple-builds. separate]}` which is apparently caused by cljs files from previous builds being in input-files

juhoteperi19:08:23

@martinklepsch: Do you have .cljs.edn file?

juhoteperi19:08:37

Automatically generated .cljs.edn could be broken

micha19:08:47

what did he say about the directories?

juhoteperi19:08:09

@micha: No aswer yet, I asked just few minutes ago

micha19:08:28

i think that definitely causes recompile of all things

martinklepsch19:08:41

@juhoteperi: no cljs.edn is generated

micha19:08:42

which explains the behavior i was seeing when compiling with optimizations

micha19:08:04

i.e. namespaces not required in :main included in build

juhoteperi19:08:52

@martinklepsch: Automatically generated cljs.edn should probably filter out files created by cljs compiler

juhoteperi19:08:32

Try calling remove-previous-cljs-output on default-main

martinklepsch19:08:55

@juhoteperi: even if both .cljs.edn files are “ handwritten"

juhoteperi19:08:24

Boot-cljs shouldn't include any cljs files from input-files in cljs.edn

juhoteperi19:08:07

Or boot-cljs shouldn't change .cljs.edn at all

martinklepsch19:08:24

yeah, default-main does scan the fileset for cljs files.

juhoteperi19:08:40

Default-main shouldn't do anyhting if there are .cljs.edn files

juhoteperi19:08:28

If it is doing something, you'll see a message about writing .cljs.edn file

martinklepsch19:08:27

@juhoteperi: so the conditional must be broken there somehow

juhoteperi19:08:32

Okay so it's creating separate.cljs.edn instead of sing the provided one

juhoteperi19:08:53

How do you run that? I did try the command on readme and couldn't reproduce

martinklepsch19:08:53

Yeah, but also app.cljs.edn

juhoteperi19:08:51

just clean repo

micha19:08:09

i did update the way ids are computed, so you can do --ids foo/bar

micha19:08:18

before it was just looking at the last segment

martinklepsch19:08:35

@juhoteperi: you might need to update the version number in build.boot

juhoteperi19:08:35

@micha: Yeah I see that now, that should have been mentioned in changelog etc.

juhoteperi19:08:48

@martinklepsch: Yes, reproduced with -3

micha19:08:49

i put it in the commit message

juhoteperi19:08:24

I only read the first line

juhoteperi19:08:23

I guess it makes sense.

juhoteperi19:08:37

Or well, I know it makes sense. I remember thinking it should work that way but as there wasn't utility fn to do that I just used by-name.

martinklepsch19:08:48

I even saw that but somehow was confused by another commit being reverted.

martinklepsch19:08:01

Anyways, thanks!

juhoteperi20:08:30

@micha: Dnolen says passing only one dir should work and will accept a patch to fix it.

juhoteperi20:08:07

And yeah, listing all the dirs causes all the files being compiled even though they wouldn't be required by anything

micha20:08:31

@juhoteperi: what would the patch do?

juhoteperi20:08:59

Fix the analysis so that cljs.test sees the new vars in files which are not inside dirs passed to build

juhoteperi20:08:08

That was the reason to list all dirs

micha20:08:20

ah ok cool

micha20:08:28

so we can back out the dirs thing then

micha20:08:35

and fix this upstream

juhoteperi20:08:10

I'm creating a ticket about this and I'll try debugging it with Cursive

juhoteperi23:08:09

Ahhah, I found out that even order of dirs in build matters

micha23:08:57

we can pass no dirs, right?

juhoteperi23:08:11

We need pass one dir

micha23:08:14

just the url of the main12345 ns on the classpath?

juhoteperi23:08:18

And there is definitely a bug

micha23:08:11

dir seems unnecessary when you have :main set

juhoteperi23:08:49

Hm yeah I guess passing just a File or URL would work

juhoteperi23:08:26

But that doesn't fix the problem with stuff not being recompiled when another ns is changed

micha23:08:11

i think if you change a deftest you only change metadata on vars, not the values the vars point to

micha23:08:35

maybe metadata is ignored when performing the dependency checks or whatever

micha23:08:00

like it doesn't recompile things if the var meta changes

juhoteperi23:08:17

Nope, looks like it's completely broken. I can reproduce it just with (def a "foo") and changing the value.