Fork me on GitHub
#boot
<
2015-10-27
>
micha00:10:59

well i think i found the problem, sort of

micha00:10:03

or rather a workaround

micha00:10:25

@alandipert: can you try that one?

micha00:10:46

it's bumped to version 2.4.1

alandipert00:10:00

i can in a few mins

micha00:10:17

i think this fixes it actually

micha00:10:45

but i'm worried that pod isolation has been compromised somehow

alandipert00:10:52

yeah i'm not a fan of the global exclusion

micha00:10:06

it's not excluding it from the deps

micha00:10:14

just from the jars that are added to the classloader

micha00:10:37

clearly clojure is already loaded, so adding the clojure jar again is redundant

micha00:10:47

but i don't understand why it all of a sudden breaks

alandipert00:10:05

if you add like, dunaj tho

micha00:10:15

it uses BOOT_CLOJURE_NAME

alandipert00:10:32

so maybe your BOOT_CLOJURE_NAME is dunaj and a dep brings in clj

micha00:10:36

if you're using dunaj presumably you'd have an explicit clojure exclusion

alandipert00:10:55

true, but this would still be new behavior for you

micha00:10:55

yeah it would just prevent dunaj from being loaded again

micha00:10:52

it only prevents something that's already on the classpath from being loaded again

micha00:10:38

if BOOT_CLOJURE_NAME is dunaj it wouldn't change anything about clojure deps, i don't think

alandipert00:10:10

oh now, but the problem may exhibit itself, no?

alandipert00:10:24

oh i suppose it won't be clojure.lang.Compiler$Expr

alandipert00:10:29

so it's good-ish

micha00:10:43

i want to know what is different now

micha00:10:54

why it break all of a sudden

alandipert00:10:59

yeah not cool man

alandipert00:10:11

compiled with java 7, right?

alandipert00:10:29

i need to do dishes but after that

alandipert00:10:32

i can plug in and turbocompute

alandipert00:10:38

bump the com truise

micha00:10:47

ok i'm going to head home

micha00:10:57

although i can't deploy from there

micha00:10:05

should i push 2.4.1 for now?

alandipert00:10:05

i'll be able to

micha00:10:35

if you deploy don't forget to make a release with boot.jar for 2.4.1

micha00:10:50

doesn't need boot.sh or boot.exe anymore, just boot.jar

micha00:10:55

from the bin/ dir

micha00:10:10

a github release i mean

micha00:10:18

for the download

micha01:10:00

ok i am ready to compute

alandipert01:10:16

me too, finally

alandipert01:10:59

i'm trying to get java 7 going

alandipert01:10:12

i can only remember how to maintain multiple JDKs on 2 OSes at a time

alandipert01:10:19

but i'm on the one that was not cached in my brain

micha01:10:33

sudo update-alternatives --config java

micha01:10:48

sudo update-alternatives --config javac

micha01:10:07

i don't know why java and javac are separately configured

micha01:10:10

but so it goes

alandipert01:10:39

all of them are

alandipert01:10:42

jar, javap, etc

alandipert01:10:45

tons of crap in the jdk bin

micha01:10:04

i need to learn how to use their tools

micha01:10:16

the profilers and debugger tools

micha01:10:34

the ones that come iwth the jdk are supposed to be good

alandipert01:10:35

oh wait, wrong talk i think

alandipert01:10:13

i could swear there's another one where he talks about jvm debugging

alandipert01:10:20

that one is nice coverage of invokedynamic tho

micha02:10:28

yeah the invokeDynamic talk is great

micha02:10:44

graal is still a great name

alandipert04:10:16

@martinklepsch: feel free to open that brew PR again at your leisure, 2.4.1 is good

juhoteperi06:10:04

@micha: Should boot.App be available in all pods?

juhoteperi06:10:08

Hmm, I guess the problem is that boot/aether doesn't depend on boot/base

juhoteperi06:10:58

Adding dependency to boot/base (with provided scope) to boot/aether seems to work

juhoteperi06:10:13

I had to remove ~/.boot/boot.properties to get Boot working at all after updating to 2.4.1

juhoteperi06:10:42

And now I'm getting exceptions like: java.lang.NoClassDefFoundError: clojure/lang/AFunction and java.lang.NoClassDefFoundError: clojure/lang/RT

pesterhazy08:10:45

Anyone know what issue this indicates:

clojure.lang.Compiler$CompilerException: java.lang.RuntimeException: Unable to resolve symbol: intern in this context, compiling:(/var/folders/rj/8lnf662s3mlgzv5mcq2r2fnc0000gn/T/boot.user250642385654527633.clj:1:715)
?

pesterhazy08:10:30

I get it when I run the repl task

martinklepsch08:10:39

@pesterhazy: what version? haven’t seen it before — is there a intern symbol in your build.boot?

martinklepsch08:10:01

is that all you got or are there more lines?

pesterhazy08:10:15

git grep intern has no matches

pesterhazy08:10:45

nothing more in the output

pesterhazy08:10:40

I had this once when there was a mistaken ns declaration.... I think

pesterhazy09:10:01

my.project.foo vs my/project/bar.clj

martinklepsch09:10:39

I see. How about bisecting then?

pesterhazy09:10:04

it's strange

pesterhazy09:10:33

it's now fixed; I looked at the git log and I only see the fixed ns declartion

pesterhazy09:10:58

I must have made a mistake somewhere

pesterhazy09:10:08

still, that is a strange error

pesterhazy09:10:31

Clojure often behaves weird if there's a (ns) vs filename mismatch

pesterhazy09:10:53

I'll bring it up if I see this again

martinklepsch09:10:50

@pesterhazy: did someone else fix it or what happened that it now works?

martinklepsch09:10:54

Seems to be similar to what @juhoteperi reported but removing boot.properties didn’t change the outcome.

juhoteperi09:10:25

Yeah, removing boot.properties didn't fix that. But before removing it, boot didn't start at all for me.

martinklepsch09:10:23

@juhoteperi: I noticed that when you set BOOT_VERSION to something else then 2.4.1 it just doesn’t do anything — is that what you mean?

martinklepsch10:10:20

@micha: @alandipert 2.4.{0,1} is really strange to me. We should have tested this more. What do you think about rolling these two commits back and releasing a 2.4.2 with only the stuff before micha’s classloader stuff?

pesterhazy10:10:16

yeah, if I s/2.2.0/2.4.1/ in boot.properties, boot dev just doesn't do anything anymore

pesterhazy10:10:28

or is that not how I update boot? simple_smile

martinklepsch11:10:04

@pesterhazy: sometimes (as with 2.4.{0,1}) you need to also download a new binary. 2.4.0 was supposed to be the last release that requires a new binary update. 😄

pandeiro11:10:31

what is the state of 2.4.1? i'm seeing something very weird where it looks like tasks in our CI are running twice per invocation

pandeiro11:10:22

Binary is 2.2.0, but it seems to pull in 2.4.1 versions of the libs

martinklepsch11:10:37

do you have a boot.properties file? calling boot -u?

pandeiro11:10:06

I do, and it specifies BOOT_VERSION=2.2.0

pandeiro11:10:43

If I specify a boot-core version in build.boot would that prevent Boot from retrieving and using 2.4.1?

martinklepsch11:10:31

@pandeiro: BOOT_VERSION is supposed to handle that, are you sure that env var is properly exported on CI?

martinklepsch11:10:57

@pandeiro: CircleCI? maybe ssh & boot -V could give some hint

pandeiro11:10:10

@martinklepsch: the boot.properties file is at the project root where boot ... commands are run -- would I need to do something more than that?

pandeiro11:10:28

I cleared the CircleCI cache, so I assume that is why it is retrieving those latest versions of the libs

pandeiro11:10:50

But if my :dependencies specify a version of boot, will it peg to that version of boot, pod, aether etc?

pandeiro11:10:00

(right now I don't specify anything)

martinklepsch11:10:45

@pandeiro: no that sounds right. are you running boot -u?

pandeiro11:10:59

nope, never

martinklepsch11:10:02

hm. the :dependencies thing should really not be necessary and I’m not sure if it’d help. only thing I can think of for troubleshooting is boot -V

martinklepsch11:10:24

besides that I have no clue why tasks would run twice in 2.2.0 or 2.4.1

pandeiro11:10:43

@martinklepsch: but the expected behavior of boot is to download the latest versions of the libs if none are present yet in the local maven cache, right?

martinklepsch11:10:16

or the ones specified via BOOT_VERSION. usually your project should not have a dependency on boot. maybe some of your deps imports it?

pandeiro11:10:38

i'm less worried about the tasks running twice (maybe just outputting twice -- could be a buggy circle ui too) but more about how i can specify and control which boot lib versions are obtained

pandeiro11:10:57

i don't have any deps that are newer than boot 2.4.1 though...

martinklepsch11:10:35

do any boot (aether/pod/worker/core) deps show up in boot show -d?

pandeiro11:10:56

are you sure BOOT_VERSION=2.2.0 for example means boot binary and libs version 2.2.0? I'm thinking it just means the binary?

martinklepsch11:10:53

no it definitely means libs. can you check boot -V? I’m thinking maybe it just downloads the deps but doesn’t use them? … similar to how boot show -u acts

pandeiro11:10:06

#
#Tue Oct 27 09:35:47 BRST 2015
BOOT_CLOJURE_VERSION=1.7.0
BOOT_VERSION=2.2.0
#App version: 2.2.0

martinklepsch11:10:06

is that from CI? if so I’m pretty sure no other deps then 2.2.0 are being used. Still weird that it downloads latest though.

pandeiro11:10:29

no that is local, I'll do it on CI too

pandeiro11:10:00

what would happen if boot.properties specifies a BOOT_VERSION lower than the version of the binary that was downloaded? Any idea?

martinklepsch11:10:21

binary version can’t be changed via BOOT_VERSION so it would still be whatever version the binary is that you downloaded

pandeiro11:10:58

so one issue is that a different version was being specified in circle.yml boot install step than in boot.properties...

pandeiro11:10:46

And it does seem that when those two don't match, it retrieves boot 2.4.1 and I get the double output issue (even with boot -V etc)

martinklepsch11:10:42

that sounds strange what version did you download and which did you specify in boot.properties?

pandeiro11:10:18

my local version is still 2.2.0; a recent update to our circle stuff made the install retrieve 2.3.0. properties version has always remained 2.2.0.

pandeiro11:10:44

But don't worry about it @martinklepsch -- I think I'm just having a twilight zone moment here...

pandeiro11:10:52

Thanks a bunch for sorting me out

pandeiro11:10:12

Ah I guess the ultimate question: would you recommend upgrading to 2.4.1 right now?

pandeiro11:10:21

Or would it maybe be smarter to wait on the next release?

martinklepsch11:10:49

I’d wait. I’ll try to reproduce your issue and log an issue if I can.

micha11:10:53

yeah we should roll back

micha12:10:04

do we have a list of issues with 2.4.1?

micha12:10:16

a lot of stuff in the scroll buffer

martinklepsch12:10:32

we should establish some testing process to avoid this in the future

micha12:10:56

definitely

martinklepsch12:10:14

@micha: two things 1) boot $task returns without any output 2) https://github.com/boot-clj/boot/issues/321

martinklepsch12:10:07

@pandeiro: what tasks did you invoke that caused duplicate output? any? even boot -V?

pandeiro12:10:43

@martinklepsch: yes even boot -V, boot show -d, as well as project-specific tasks

martinklepsch12:10:11

/v/f/s/4/T/boot-test> boot220 -V
#
#Tue Oct 27 12:59:37 CET 2015
BOOT_CLOJURE_VERSION=1.7.0
BOOT_VERSION=2.3.0
#App version: 2.2.0

pandeiro12:10:14

the combination was: binary 2.3.0 installed; BOOT_VERSION 2.2.0 specified; empty ~/.m2/repository

martinklepsch12:10:30

oh, other way around ;D

pandeiro12:10:26

@martinklepsch: feel pretty confident now that installing boot and then running a command with an empty maven will cause boot to download the latest version of the lib

pandeiro12:10:40

do you not see that behavior? is that not the expected behavior?

micha12:10:02

yeah this is expected behavior, but latest release is bad

pandeiro12:10:15

Additionally, I'm getting The command '/bin/sh -c boot -vv' returned a non-zero code: 254 in Docker Hub and Travis builds that bring in 2.4.1

pandeiro12:10:33

@micha: Do you know what 254 means?

pandeiro12:10:41

maybe since i'm trying to publish a 2.3.0 image, I could change that step to boot -d boot:2.3.0 -vv... I wonder if that would work

martinklepsch12:10:33

I’m confused:

~/c/d/boot-test2 rm -rf ~/.m2/repository/boot/
~/c/d/boot-test2 boot230 -V
#
#Tue Oct 27 13:21:33 CET 2015
BOOT_CLOJURE_VERSION=1.7.0
BOOT_VERSION=2.2.0
#App version: 2.3.0
~/c/d/boot-test2 boot230 show -d
Retrieving pod-2.2.0.jar from 
Retrieving core-2.2.0.jar from 
Retrieving worker-2.2.0.jar from 

pandeiro12:10:14

Huh, why is it not retrieving boot itself?

pandeiro12:10:44

I get "Retrieving boot-2.4.1.jar from ..." and then the additional deps are 2.2.0 as well, like yours

pandeiro12:10:01

Ah maybe the boot lib is not cached in .m2

martinklepsch12:10:29

@pandeiro: afaik boot-x.x.x.jar is just to get some sort of fake clojars transaction

pandeiro12:10:53

Hmm, my boot -d boot:2.3.0 -vv idea above didn't work... It still gets 2.4.1 versions

martinklepsch12:10:02

it’s what the binary checks for new releases and during deploy it’s uploaded after all other jars

pandeiro12:10:32

The scenario I mentioned above (binary 2.3.0, version 2.2.0, empty cache) produced this:

$ ./bin/circleci-deps
Retrieving boot-2.4.1.jar from 
Retrieving clojure-1.7.0.jar from 
Retrieving dynapath-0.2.3.jar from 
Retrieving pod-2.2.0.jar from 
Retrieving shimdandy-impl-1.1.0.jar from 
Retrieving core-2.2.0.jar from 
Retrieving aether-2.2.0.jar from 
Retrieving worker-2.2.0.jar from 
Retrieving parsley-0.9.3.jar from 
Retrieving regex-1.1.0.jar from 
Retrieving reply-0.3.5.jar from 
Retrieving cd-client-0.3.6.jar from 
Retrieving clj-http-lite-0.2.0.jar from 
Retrieving slingshot-0.10.3.jar from 
Retrieving clj-stacktrace-0.2.7.jar from 
Retrieving drawbridge-0.0.6.jar from 
...

pandeiro12:10:46

Note the boot version and the pod, aether, etc versions

martinklepsch12:10:33

@micha: as for rolling back I made a test with all commits until the new binary stuff and that seemed to work fine

ska12:10:37

I just followed https://github.com/boot-clj/boot/wiki/Cider-REPL and get a stacktrace when running boot repl Is that a known issue?

ska12:10:28

clojure.lang.ExceptionInfo: java.lang.NoClassDefFoundError: clojure/lang/RT, compiling:(deps/toolsreader/v0v9v2/clojure/tools/reader/impl/utils.clj:1:1)

alandipert12:10:06

@ska: it is, and it's new... we pushed a broken release last night

ska12:10:50

Heh, just the day I decided to give boot a try 😉. No worries. Anything I can do to help?

alandipert12:10:12

perfect timing 😉

martinklepsch12:10:20

@ska hey and welcome 😉

ska12:10:37

@martinklepsch: it's about time I join, eh?

alandipert12:10:43

i'm gonna update the README with 2.3.0 links

alandipert12:10:51

until we get this sorted

martinklepsch12:10:33

@ska: true words. anything specific you want to use boot for?

ska12:10:39

@martinklepsch: First, I love the claim. Second, just want to generally be informed. So, no, nothing special. Will try to compile mixed clj and java project.

martinklepsch12:10:55

@ska: that will be a breeze, just did that the other day simple_smile

pandeiro12:10:31

Nice, setting ENV BOOT_VERSION 2.3.0 in my boot 2.3.0 Dockerfile got it building fine. Works for me. I guess hard-coding that in is probably better any way? Thoughts?

micha12:10:58

yeah always pin versions in unattended deploys

ska12:10:31

@martinklepsch: hmm, breeze? Now boot fails silently with return code 254 😕

ska12:10:41

Need to find out what I am messing up

martinklepsch12:10:20

@ska: did you downgrade to 2.3.0?

ska12:10:55

yep and boot repl outside of the project dir gives a REPL

martinklepsch12:10:14

do you have a boot.properties file in the project dir?

ska12:10:34

Uhm, I had a REPL a few minutes ago in /tmp now it is not working anymore. So, not project related

ska12:10:28

Ah, different shell. Over there, I already had exported export BOOT_VERSION=2.3.0

martinklepsch12:10:13

@ska did you download the 2.3.0 binary as well?

ska12:10:38

I did, yes. Now I also removed ~/.boot/cache

ska13:10:25

OK, works in project

ska13:10:38

How do I add java source paths?

micha13:10:13

ok i think i have a fix

micha13:10:26

uploading

martinklepsch13:10:44

@ska: you just add them like any other source path, i.e (set-env! :source-paths #{“com/main/bla/src”})

martinklepsch13:10:10

@ska: there’s a javac task (`boot javac -h`) that can compile them for you.

martinklepsch13:10:13

basically you’ll want a combination of pom, javac and jar — maybe aot as well

martinklepsch13:10:38

@micha: why do we only want a partial revert?

micha13:10:54

it's just the loader

micha13:10:05

2.4.1 is ok

martinklepsch13:10:05

I’d have thought we fully revert the change and continue in another branch with testing

micha13:10:20

well we can't remove 2.4.1 from clojars

micha13:10:24

so that will remain an issue

micha13:10:35

we can still do the other thing but this at least fixes that problem

martinklepsch13:10:27

@micha: so you mean a new binary would fix it and the libs are ok?

micha13:10:28

all i'm doing is uploading boot.sh and boot.exe to github releases

micha13:10:45

my internet is slow so it's taking a minute

martinklepsch13:10:56

so we have a 2.4.2 or do we just override the binaries?

micha13:10:13

there were no binaries in that release

micha13:10:20

only boot.jar

micha13:10:22

so it's good

martinklepsch13:10:48

building and testing now

micha13:10:06

i think we need @tcrawley to consult with the loader simple_smile

martinklepsch13:10:33

haha, I thought of him as well

micha13:10:49

i'm doing something wrong with the classloader loading

alandipert13:10:58

i dunno man it looks legit

alandipert13:10:03

dotted i's, crossed t's

ska13:10:18

Can I reload build.bootwhen I am in the boot.repl?

micha13:10:35

(load-file "build.boot") @ska

ska13:10:02

@micha: no problems with side-effects like set-env! then?

micha13:10:31

well stateful things can cause problems with reloading

micha13:10:34

but that's always the case

micha13:10:41

set-env! should be okay though

micha13:10:16

judicious use of defonce can help with those kinds of things if you run into issues

ska13:10:45

Ah, interactive build-script development. Joy!

micha13:10:26

:thumbsup:

ska13:10:58

And C-c C-k from Emacs works, too. 😄

micha13:10:28

boot.sh is uploaded

martinklepsch13:10:12

@micha: btw, do things in boot.properties override env vars?

ska13:10:50

Ugh, missed a meeting while playing around with boot.

ska13:10:24

what is the supposed way of mimicking gradle wrapper? Check in boot.sh?

martinklepsch13:10:04

dunno about gradle wrapper, sorry

ska13:10:09

A thin wrapper supposed to be checked in. Eases installation pain and makes builds reproducible.

tcrawley13:10:41

@micha: sup? anything I can help with?

martinklepsch13:10:20

@ska: https://github.com/boot-clj/boot/issues/163 — I think now that parts of #300 have been reverted it’s not yet there. Wont be long though I guess

micha13:10:11

^^ this seems to fix the problem with the loader

ska13:10:14

@martinklepsch: Yeah, I read that earlier. That was the reason for me asking, if boot.sh (the Loader) should be checked in now.

martinklepsch13:10:58

@ska: you could check it in. I think the point of #317 is that the binary does not change at all anymore

martinklepsch13:10:25

@micha: what do you think about depending on 1.7 for all the libs? for some reason I can’t get this to build without it: https://github.com/boot-clj/boot/pull/318/files causing weird things:

java.lang.IllegalAccessError: tried to access method clojure.lang.RT.classForNameNonLoading(Ljava/lang/String;)Ljava/lang/Class; from class boot.from.digest$loading__5340__auto____25, compiling:(web.clj:1:1)

micha13:10:09

i dunno, how many people are using boot with 1.6?

martinklepsch13:10:13

I would think not a lot but well… trying one more thing.

micha13:10:50

ok i thnk this fixes the issues

micha13:10:54

can we test?

micha13:10:03

it should work with the 2.4.1 lib in clojars

martinklepsch13:10:52

@micha: one more thing worked ^

martinklepsch13:10:01

@micha how much did you sleep tonight? 😄

micha13:10:08

@martinklepsch: i'm down with splitting boot.tmpdir into a separate library btw

micha13:10:37

we'd need to keep the same namespace names of course

martinklepsch13:10:38

@micha: I was thinking the registry stuff could go in there too but fine too if not

micha13:10:59

yeah that's a separate thing, i think

micha13:10:53

ok i'm going to update the boot.sh and boot.exe files in the 2.4.0 release

micha13:10:57

since those are fubar anyway

micha13:10:02

can't hurt

nberger13:10:08

Is there any work done on https://github.com/adzerk-oss/boot-test/issues/9? I'd love to be able to use the junit formatter.

martinklepsch13:10:32

my thought was the fileset requires something like tmpregistry in most cases anyways

micha13:10:55

that's true, but the directories could come from anywhere

micha13:10:09

might make sense to separate those concerns

micha13:10:28

we should make a TmpRepo type

micha13:10:34

a mutable java type

micha13:10:38

that can hold the mutable state

micha13:10:11

or amybe just an atom i guess

martinklepsch13:10:58

@micha: with new binary App Version will always be $BOOT_VERSION is that right?

micha13:10:14

unless you're using an older version of boot

micha13:10:19

which will also work

martinklepsch13:10:44

“of boot” as in boot binary right?

micha13:10:30

BOOT_VERSION=2.1.2 for example

micha13:10:42

that will be using app version 2.0.0 and lib evrsion 2.1.2

micha13:10:46

or something like that

micha13:10:03

but 2.4.0 and later will have app and lib versions the same

micha13:10:30

should be a-ok now

martinklepsch13:10:38

k, then everything seems to work fine 👍 nice one!

micha13:10:03

i'll remove the boot.sh and boot.exe from 2.4.1 release when we are confident the issues are resolved

micha13:10:23

and point the README at the 2.4.0 .sh and .exe

martinklepsch13:10:24

are these the old ones?

micha13:10:32

no they're the new ones

micha13:10:37

i replaced them

micha13:10:19

i like having the binaries on the 2.4.0 release because that's where the new regime comes into effect

micha13:10:42

the commit message for 2.4.0 explains all about the new loader and whatnot

martinklepsch13:10:30

cool, muddling with releases seems like something we’d want to avoid but in this case parrot

micha13:10:43

i guess it's possible that the bad versions could be cached somewhere

micha13:10:56

lol that emoji is incredible

martinklepsch13:10:39

I wouldn’t be surprised, there have been previous occasions where people got old binaries somehow

micha13:10:50

when you feel confident about the loader can you PR to homebrew please?

micha14:10:19

i'm going to update the README to point to the 2.4.0 binaries now

micha14:10:40

@tcrawley: good morning simple_smile

micha14:10:10

i was creating a new URLClassLoader instance, loading the uberjar (the one with shimdandy) into it, setting it as TCCL, and then calling boot.App/-main in there via reflection

micha14:10:26

that seems to interfere with the clojure shim

micha14:10:06

the fix was using reflection to add the uberjar to the system classloader by exposing the protected add-URL() method via reflection

micha14:10:26

and not creating a new classloader

micha14:10:00

i don't really understand the problem, but it seems to work (tm)

micha14:10:35

i think maybe shimdandy needs to be at the top of the classloader hierarchy

alandipert14:10:15

@micha: i still get the problem with cider using the new boot.sh

micha14:10:52

what is the exception?

alandipert14:10:34

i will run this thing in docker to make sure

micha14:10:28

the cider thing works for me

alandipert14:10:48

yeah i think i was confus

micha14:10:49

i rm -rf /.m2/repository/boot and /.boot thoug

tcrawley14:10:44

@micha: ah, the issue may be that the shim resets the TCCL to its own CL, so the CL you set as the TCCL would be overwritten?

alandipert14:10:22

that sounds like it

micha14:10:08

calling the -main method in the uberjar is in "tail position" though

micha14:10:16

the loader doesn't do anything after that

micha14:10:37

doesn't need to load any classes or anything

micha14:10:53

classloaders are so mysterious

micha14:10:06

if you understand them you're like a shaman

tcrawley14:10:11

does that uberjar have clojure in it?

micha14:10:26

it's the old binary

micha14:10:38

the java program that loads the shimdandy api

micha14:10:48

and then starts clojure in shims

tcrawley14:10:03

and the shims can't find clojure.lang.RT at that point?

micha14:10:37

the first issue was loader problem where it tried to load clojure classes more than once

alandipert14:10:38

@micha: ok i can confirm, it works - totally clean with just the offending ~/.boot/profile.boot

micha14:10:11

clojure.lang.ExceptionInfo: loader constraint violation:
          loader (instance of java/net/URLClassLoader) previously initiated
          loading for a different type with name "clojure/lang/Compiler$Expr"

micha14:10:55

we fixed that in boot clojure code to filter out org.clojure/clojure from being added dynamically via set-env!

micha14:10:06

since it alerady is loaded

micha14:10:13

but that was a red herring i think

micha14:10:35

i think the real problem was pod isolation somehow compromised

micha14:10:10

because after that was fixed more subtle issues arose, like this one https://github.com/boot-clj/boot/issues/320

tcrawley14:10:30

what are you setting the parent loader to for the CL you are giving to the shim?

micha14:10:52

i tried setting it to TCCL, system classloader, and Loader.class.getClassLoader()

micha14:10:07

none of those made any difference as far as i could tell

micha14:10:31

the current thing that seems to work is to mutate the system classloader in place

micha14:10:41

via reflection, exposing addURL()

micha14:10:57

instead of creating a new URLClassLoader and loading the jar in that

micha14:10:38

so i get a handle on addURL() for the system cl, make it accessible, then add the app uberjar to it, and call -main via reflection

micha14:10:46

appears to solve the problem

tcrawley14:10:47

I'm glad that works. I'm just trying to figure out the cause, since I don't know what's going on either :)

micha14:10:07

i was thinking maybe shimdandy api needs to be at the tip of the classloader hierarchy?

tcrawley14:10:24

maybe, but I don't know why that would be the case

tcrawley14:10:59

hmm, the two places I've used it, it wasn't, so I don't know

micha14:10:13

oh interesting

micha14:10:41

you are loading shimdandy into a lower level classloader, at runtime?

micha14:10:46

like in immutant?

tcrawley14:10:28

yeah, in immutant 1.x and Vert.x, it was loaded dynamically, at runtime

micha14:10:46

i am going to build a sweat lodge

micha14:10:57

and make a classloader symposium

micha14:10:25

we can take peyote and learn about classloaders

tcrawley14:10:38

that's probably the only way to find the truth

micha14:10:48

it came to me in a dream

micha14:10:53

like watson and crick

tcrawley14:10:33

I've got to go work on something else now, but will let this stew in the back of my head and see if I can figure it out

micha14:10:43

awesome, thanks!

micha14:10:59

let me know if i can help in any way simple_smile

micha14:10:32

everything seems fine now with boot

tcrawley14:10:39

my pleasure! will do, and feel free to ping me any time with stuff like this, even if I usually don't have an answer :)

alandipert14:10:57

fortunately you usually do have an answer 🙌

micha14:10:13

you have the worst library to support, because people must come to you with the worst problems

micha14:10:35

"why don't my crazy classloader shenanigans work?"

martinklepsch14:10:32

Does this look like a sane way to set dependencies at task runtime?

micha14:10:13

sure, although it doesn't actually change the dependencies loaded in the jvm

micha14:10:22

so if you have conflicts it won't help with that

micha14:10:38

but it will change the dependencies for the pom task, which is what you want there i think

micha14:10:12

by "the jvm" i mean the build process runtime

martinklepsch14:10:16

well if it does not actually change the dependencies then aot will break

micha14:10:34

yeah so you can't use :all there i guess

micha14:10:08

there could be weird effects with aot, i'm not sure

martinklepsch14:10:40

The reason I’m exploring this is because when one build needs the jar of another one that hasn’t been built yet it will fail

micha14:10:52

i was thinking about that

micha14:10:02

i want to try the runBoot() method

micha14:10:16

you can make it so that the output of one build is the input for the other

micha14:10:26

and set a watcher on that

micha14:10:33

so the builds will cascade

micha14:10:52

like the build that makes the jar emits the jar to some target dir

micha14:10:01

which is the source-paths of the other build

micha14:10:09

so the watcher will pick up the change

micha14:10:21

with runBoot() you have isolated builds

micha14:10:24

in separate pods

micha14:10:29

so no aot issues

micha14:10:40

basically you're running boot in boot there

martinklepsch14:10:42

that sounds very much like how you’d orchestrate lein building interdependent things 😄

micha14:10:11

i think step 1 is just to continue to use the makefile, but have it call boot in each module instead of lein

micha14:10:45

when that's working and producing correct jars we can make a build.boot in the main dir that calls runBoot() on all of those

micha14:10:53

all in one process

micha14:10:28

and we can set up the targets and source-paths so they will all build correctly

micha14:10:41

eliminating the need for make's dependency tracking

micha14:10:38

yes i thnk that's the way to do it

micha14:10:32

ok since it looks like the world is no longer on fire i will start my day simple_smile

martinklepsch14:10:39

alternatively to multiple runBoot couldn’t we just use pods and pass them the fileset?

martinklepsch15:10:13

when I set BOOT_VERSION to a development build I just made on my machine I get errors that boot.jar can’t be downloaded from github

peterbak15:10:17

is there way to completely blow away boot from an OS X system and reinstall it from scratch?

martinklepsch15:10:50

@peterbak: delete the binary + rm -rf ~/.m2/repository/boot

micha15:10:03

martinklepsch: the makefile has a fix in there

micha15:10:10

to install the boot.jar in the right place

micha15:10:15

are you using the makefile?

micha15:10:22

or the boot-built-with-boot?

martinklepsch15:10:37

both right now

martinklepsch15:10:59

trying integrating boot into Makefile as you said, then we can merge and improve from there

peterbak15:10:14

thanks @martinklepsch trying that now

micha15:10:21

it won't try to download it if it's installed in BOOT_HOME/cache/bin/BOOT_VERSION/boot.jar

micha15:10:33

the makefile should do that for you

micha15:10:41

i thnk runBoot() would be much simpler

micha15:10:51

because you don't need to mock anything up

micha15:10:59

we can build a gulp-like library on that

micha15:10:20

to handle the interproject dependencies in a nice way

micha15:10:34

because with multiple projects you don't want a pipeline anymore

micha15:10:38

you really want a dependency graph

micha15:10:49

which is what you don't want for a single project

martinklepsch15:10:10

lol, how can I have a symbol ’boot in my build.boot? > boot already refers to: #'boot.core/boot in namespace: boot.user

micha15:10:36

boot.core/boot is a crucial function simple_smile

micha15:10:45

you don't want to replace that guy

micha15:10:57

but if you really want to you can do alter-var-root on it

martinklepsch15:10:08

I know but how can I pass ’boot to the pom task then?

micha15:10:10

and replace it with another thing

micha15:10:19

why would you do that?

martinklepsch15:10:39

to build the boot project?

micha15:10:51

you're getting the error because you're defining a function or defing a var named boot

micha15:10:07

which you shouldn't do, just name it something else

martinklepsch15:10:09

argh damn, sorry 😄

martinklepsch15:10:25

I forgot my task was also named boot 😛

micha15:10:31

:thumbsup: no worries!

micha15:10:08

i'll make a demo of what i mean today

micha15:10:21

a multi-module build

micha15:10:28

with watcher support

martinklepsch15:10:42

cool, I’ll integrate the thing I have with the make file now

chrisn15:10:09

this may be a bit trollish but why isn't boot self-hosting?

ska15:10:42

Ah, the joy of side effects... I added a dir to resource paths once to get a file into an uberjar, then wanted to get that file out of it again, but it still turned up. After a short detour searching for a clean task, I found that the set-env! was still in effect, because I did not set it back.

ska15:10:40

Uh, wait. The file is still there

ska15:10:02

Hm, what am I missing? A file was copied to the target-folder because I had a directory in the :resource-paths. I remove that setting from the call to set-env! and manually (set-env :resource-paths #{}). But the file is still in target and gets added to the uberjar.

ska15:10:25

s/set-env/set-env!/

alandipert15:10:43

@ska: you can't un-add resources from the classpath, but you can filter them from the fileset

alandipert15:10:24

not being able to remove from the classpath is a jvm limitation we can't work around

alandipert15:10:41

we mitigate by making it possible/easy to create new classpaths, aka pods

alandipert15:10:34

so what you want is to filter from the fileset, this is the set of things that will show up in target. you can check out the sift task

alandipert15:10:52

you can use it to filter out items in the fileset by regex

alandipert15:10:34

if you look around the build.boots for cljsjs packages, you can see many usages https://github.com/cljsjs/packages

ska15:10:36

on those rare cases I can also restart the JVM. Would that help? Would that remove the file from target dir?

martinklepsch16:10:57

it will reset the fileset which will probably lead to the result you want

micha16:10:15

@ska i think (set-env! :resource-paths #{}) should work

micha16:10:24

are you sure the file isn't getting in there some other way?

micha16:10:47

also removing a directory won't trigger the watch task, so maybe you need to manually trigger a build

micha16:10:00

by touching a file or something

onetom16:10:05

> i dunno, how many people are using boot with 1.6? we actually had to lock down one of our components to 1.6 because it was failing with some weird exception, iirc but otherwise are on 1.7 everywhere else

onetom16:10:45

we are rewriting that component in the coming days anyway though...

juhoteperi16:10:29

I think it would be best to keep 1.6 compatibility at least until 1.8 is released

micha16:10:43

yeah, also in boot we can do whatever needs 1.7 in the worker pod

juhoteperi16:10:44

Should 2.4.1 be working now?

micha16:10:49

and have the worker pod use 1.7

micha16:10:09

if you download boot.sh from the 2.4.0 release on github

juhoteperi16:10:59

Okay, started working after redownloading it

micha16:10:52

yeah the original one had a bug

danielsz16:10:37

Can't wait to try out the next release packed with goodies.

ska17:10:43

@micha: regarding the files which end up in the JAR or not.... I need to play with it some more. Will come back if I think something is not as it should be.

nberger17:10:47

Just sent a PR to boot-test adding junit formatter support. Feedback welcome simple_smile https://github.com/adzerk-oss/boot-test/pull/11

nberger17:10:37

checking the build error... not sure yet if it failed because of the failing test (that I put there on purpose) or something else

juhoteperi17:10:07

It's at least using historical version of boot, pre 2.0 release

nberger17:10:43

Oh, right. Will check that too

juhoteperi18:10:17

Updating .travis.yml should help and would also be good idea to update the dependency in build.boot

nberger18:10:35

cool, will do both

juhoteperi18:10:29

@nberger: Also, 2.4.1 would probably be preferred as 2.4.0 is quite broken

nberger18:10:54

Oh thx, I thought 2.4.1 was the broke one 😕

nberger18:10:03

I ended sending a separate PR to update boot to 2.4.1

esp118:10:52

how can i get the built-in boot web task to reload its serve function when code is changed?

alandipert18:10:17

@esp1: the web task only makes a web.xml and injects some java, it doesn't start any servers... what are you trying to do?

esp118:10:52

oh ok - i was trying to use tailrecursion/boot-jetty to serve a clojure backend

alandipert19:10:35

oh ok - yeah that task is handy if you're deploying as a WAR file and want to run a war locally

alandipert19:10:07

another option that doesn't imply the web task is https://github.com/pandeiro/boot-http

esp119:10:07

i tried that too - for some reason reloading isn’t working for me with either one

esp119:10:04

is there an example w/server handler reloading working somewhere?

nberger19:10:12

@alandipert: oops, seems like boot.sh is not available for download in the 2.4.1 release, so the boot-test build is still failing 😕

alandipert19:10:41

@nberger: would you mind outputting the junit reports to the fileset instead?

alandipert19:10:10

at least, that seems more future proof... subsequent tasks could process the xml and do things with it

nberger19:10:36

@alandipert: Yes, sounds like a good idea, I'll try to do that.

richiardiandrea19:10:57

hello guys, if I do boot -h in a Mac machine I don't see any output

martinklepsch19:10:05

@richiardiandrea: what’s the output of boot -V?

martinklepsch19:10:44

try rm -rf ~/.boot/cache/bin

richiardiandrea19:10:11

boot -V
#
#Tue Oct 27 20:46:26 CET 2015
BOOT_CLOJURE_VERSION=1.6.0
BOOT_VERSION=2.4.1
#App version: 2.0.0

martinklepsch19:10:42

oh. simple_smile you need to download the latest executable boot.sh: https://github.com/boot-clj/boot/releases/tag/2.4.0

richiardiandrea19:10:04

ah, I installed it with brew

roberto19:10:27

brew is evil troll

martinklepsch19:10:49

brew PR is outstanding but if you want to stick with brew you can set $BOOT_VERSION to 2.2.0 or so and upgrade boot via brew

martinklepsch19:10:20

2.0.0 is pretty old but I think 2.2.0 or so should be on homebrew

richiardiandrea19:10:21

that's good then, will try now

richiardiandrea19:10:42

but whatever works 😄

martinklepsch19:10:34

I see a boot 2.2.0 in homebrew — maybe you need to update brew itself?

martinklepsch19:10:56

like brew update I mean..

richiardiandrea19:10:50

ok i am trying without brew, it feels safer simple_smile

richiardiandrea19:10:41

i downloaded boot.sh and copied in /usr/local/bin and it downloads the right jar

richiardiandrea19:10:47

but still nothing

richiardiandrea19:10:03

#
#Tue Oct 27 20:56:25 CET 2015
BOOT_CLOJURE_NAME=org.clojure/clojure
BOOT_VERSION=2.4.1
BOOT_CLOJURE_VERSION=1.6.0
#App version: 2.4.1

richiardiandrea19:10:37

does it require 1.7 ?

richiardiandrea19:10:51

and how to update it?

richiardiandrea19:10:35

shall I dowload in .m2 clojure 1.7 ?

alandipert19:10:46

@richiardiandrea: you can edit ~/.boot/boot.properties

martinklepsch19:10:52

@richiardiandrea: what is the result of echo $? after you run boot -h and it returns nothing?

richiardiandrea20:10:25

on boot repl as well, same return value from the script

martinklepsch20:10:18

can you once again do rm -rf ~/.boot/cache/bin please?

richiardiandrea20:10:26

ok by changing version in ~/.boot/boot.properties it works

martinklepsch20:10:53

@richiardiandrea: what version did you change? clojure?

richiardiandrea20:10:07

i am reverting to 1.6 and try your stuff

richiardiandrea20:10:21

cleared cache and return of it is again 254

richiardiandrea20:10:50

i am going to keep 1.7 for now then

richiardiandrea20:10:53

great! if you want I can open an issue

michal20:10:58

hi. i’ve just upgraded boot to 2.4.1 and seems like it totally stopped working (eg. boot repl does completely nothing). any ideat wha happened?

esp120:10:49

@alandipert: i found my issue with reloading. i had relocated my source into a dir named “client/src/clj”. it looks like reload only works for source dirs that start with “src/"

michal20:10:16

#
#Tue Oct 27 21:12:34 CET 2015
BOOT_CLOJURE_VERSION=1.7.0
BOOT_VERSION=2.4.1
#App version: 2.0.0

alandipert20:10:17

@esp1: ah! i feel like i've run into that before myself and never figured it out

martinklepsch20:10:30

@michal: your binary is outdated, how did you install boot? (you can download the new binary from https://github.com/boot-clj/boot/releases/tag/2.4.0

michal20:10:19

a few minutes ago simple_smile

esp120:10:11

@alandipert: is this a bug in boot-http? or boot?

martinklepsch20:10:53

@michal: that is strange the checksum should be 9dd9f0ae594b1658fe79477ceb492289, maybe check that?

micha20:10:48

@richiardiandrea: can you paste the output of md5sum of your boot.sh file?

juhoteperi20:10:10

@esp1: Wrap-reload defaults to src. Boot-http should pass in options: {:dirs (boot/get-env :directories)\

micha20:10:32

@richiardiandrea:

barp $ md5sum ~/bin/boot
9dd9f0ae594b1658fe79477ceb492289  /home/micha/bin/boot

michal20:10:48

yup, checksum is identical:

MD5 (/usr/local/bin/boot) = 9dd9f0ae594b1658fe79477ceb492289

esp120:10:31

@juhoteperi: cool. do you want to post the pull request or should i? simple_smile

juhoteperi20:10:56

@esp1: You are welcome to do it. I have enough to do already .)

martinklepsch20:10:01

@michal, hm. only idea I now have is the boot you’re calling is not /usr/local/bin/boot

michal20:10:45

@martinklepsch: ok, i will try to download it again. give me a sec.

martinklepsch20:10:09

@michal: you already have the right file as far as I can see

martinklepsch20:10:28

@michal I assume which boot = /usr/local/bin/boot ?

juhoteperi20:10:00

@esp1: Note: wrap-reload is used inside a pod where boot.core is not available, you need to call get-env in the task and pass the value to boot-http.impl/server call.

michal20:10:12

~ % md5 /usr/local/bin/boot
MD5 (/usr/local/bin/boot) = 9dd9f0ae594b1658fe79477ceb492289
~ % /usr/local/bin/boot repl
~ %

esp120:10:41

@juhoteperi: good call, thx

michal20:10:53

does boot have any verbose mode to see what’s happening in the background ?

michal20:10:10

it quits surprisingly fast

micha20:10:22

@michal: did you try rm -rf $BOOT_HOME/cache?

michal20:10:34

nope. i’ll do that.

micha20:10:37

assuming you have BOOT_HOME set

micha20:10:41

default is ~/.boot

michal20:10:46

~ % rm -rf ~/.boot/cache
~ % /usr/local/bin/boot repl
Downloading .
~ %

michal20:10:08

yes, i have a .boot set correctly

micha20:10:32

ok that looks fine

micha20:10:48

your BOOT_VERSION must be set to something older than 2.2.0

michal20:10:19

you mean env variable?

michal20:10:36

~ % echo $BOOT_VERSION

michal20:10:48

i didn’t set it by my self

micha20:10:03

you must have a boot.properties file then

micha20:10:12

in CWD, or in BOOT_HOME

micha20:10:37

and in that properties file you must be setting the BOOT_VERSION

martinklepsch20:10:57

@micha: but wouldn’t that be shown when he runs boot -V?

michal20:10:57

as far as I remember I have a profile.boot only but i’ll double check

micha20:10:16

you can see it in boot -V

martinklepsch20:10:26

#
#Tue Oct 27 21:12:34 CET 2015
BOOT_CLOJURE_VERSION=1.7.0
BOOT_VERSION=2.4.1
#App version: 2.0.0

martinklepsch20:10:34

^ This is what @michal posted earlier

michal20:10:41

yup, that’s correct

michal20:10:07

ok, I have a boot.properties in .boot/

michal20:10:11

#
#Sun May 24 21:19:58 CEST 2015
BOOT_CLOJURE_VERSION=1.7.0-beta3
BOOT_VERSION=2.0.0

micha20:10:42

you can update that guy iwth boot -u

michal20:10:43

i don’t even remember i was placing something like this there simple_smile

micha20:10:25

but anyway that should work, i think

martinklepsch20:10:41

@micha: bug, that it prints the wrong BOOT_VERSION in boot -V?

micha20:10:28

martinklepsch: that bug only affects people who have an old version of boot.sh

micha20:10:33

and they do boot -u

michal20:10:57

guys, you saved my evening. repl works now perfectly simple_smile

micha20:10:14

martinklepsch: disregard that, i misunderstood your last statement

esp121:10:45

@juhoteperi: i submitted the pull request but looks like it failed ci. error 254 looks suspiciously like the boot version issue above. https://github.com/pandeiro/boot-http/pull/27

juhoteperi21:10:25

@esp1: Updating version of boot.sh in .travis.yml would probably help

esp121:10:51

@juhoteperi: done. ci passed cleanly this time. i added a boot.properties file as well to pin the version. thx!

martinklepsch21:10:57

^ if anyone could checkout that branch and run boot base —uberjar that would be great, there’s a magic jar file appearing! facepalm

martinklepsch21:10:31

@juhoteperi: do you think changelog should be updated for 2.4.1 & 2.4.2 as well? guess it should?

martinklepsch21:10:21

@micha: do you want to update for 2.4.2? not sure I could describe what was done

micha21:10:54

i just made sure that the error gets printed and flushed before the process exists

micha21:10:36

throwing an exception dumps you out of the main method, and then the JVM exits before the stack trace can be flushed from the sdtout buffers

micha21:10:24

yeah the commit message is probably detailed enough

micha21:10:35

it's an implementation detail as far as users are concerned

micha21:10:56

"fix issue where boot was completely broken"

micha21:10:34

"fix issue where the wrong classloader was being used to load the boot application"

martinklepsch21:10:58

Ok I’ll add this

micha21:10:03

sweet, thanks

micha21:10:14

man that was a tricky one

alandipert22:10:36

@nberger: btw https://github.com/alandipert/boot-jasmin/blob/master/src/alandipert/boot_jasmin.clj is a little example of doing work in a pod and outputting to a tmp-dir

nberger22:10:59

@alandipert: Cool, I'll check it out, thx

nberger22:10:02

@alandipert: Is it important that tgt is created in the outer let there? Or could it be created in the inner body?

alandipert22:10:02

it's idiomatic to create it in the outer layer and re-use it

alandipert22:10:36

but no, not important really

alandipert22:10:43

i can't even remember why we do it that way everywhere

alandipert22:10:02

probably mostly just to show how the outermost let is the part of the task where one initializes resources

micha22:10:27

if you allocate resources in the middleware layer then those resources must always be allocated in pipeline order

micha22:10:52

if you do it in the outermost let then someone could create the tasks in a different order than they compose them in the pipeline

nberger22:10:53

Sure. Btw, the branch is working in my machine, so I guess I'm not doing it that wrong with the fileset 😛. I think it's right. But I'm trying this on a project in a CircleCI build and the files are not being copied to the target dir, and I'm not sure why

micha22:10:10

and the resources would be allocated in the order in which the tasks were constructed

micha22:10:26

which is sometimes handy, like if there are dependencies between the things

micha22:10:57

it's rare to have that kind of state though

micha22:10:05

i don't know of any tasks that have it

nberger22:10:30

So, from what you are saying, it's better to allocate in the innermost let, or I'm not following you?

micha22:10:53

no, like alan said it doesn't really matter

micha22:10:05

i was just pointing out the difference

nberger22:10:08

I think I got it now simple_smile

nberger22:10:44

Doing it in the innermost would make it to not hold that state, might be more memory-friendly if that's an issue

nberger23:10:02

Ok, now I understand what's happening... when there's a test failure, boot-test throws an exception, so the pipeline stops processing, etc... of course that makes my commit! call to not happen, and target-path is not even created. Any idea?

nberger23:10:26

the issue can be reproduced by introducing a test failure in adzerk.boot-test.test then running boot test

nberger23:10:27

As we are here, I'd say that I don't love the stacktrace we always get when there's a test failure... but I guess an exception is the only way to stop the pipeline? I wonder if there's some way to make the throw on failing tests to be optional?

alandipert23:10:58

@nberger: hey, yeah - thre are several ways to do it

alandipert23:10:14

@nberger: another way that comes to mind is have a separate task that picks up the test report and maybe throws, or maybe just prints output

alandipert23:10:38

@nberger: i think what it should do depends on the context; like dev, you just want to see passing results; or ci, where you want the build to fail

alandipert23:10:21

@nberger: maybe the way is 2 tasks - one which runs tests and adds a report to the fileset, and another downstream task which looks for the report and throws or prints it depending on flag