Fork me on GitHub
#boot
<
2017-03-17
>
ag18:03:12

why boot always pulls every possible version of clojure?

ag18:03:41

it takes painfully long the first time you run it

ag18:03:49

does lein do the same thing?

juhoteperi18:03:50

If I remember correctly, Boot (aether) only pulls the pom files for each version

juhoteperi18:03:55

And this might be because you depend on some library which has dependency on the other clojure version, or even some package which uses version range to depend on Clojure, which I think causes aether to download ~all pom files

micha18:03:17

version ranges will definitely do that

micha18:03:33

even in transitive dependencies

micha18:03:57

version ranges are generally frowned upon in pom.xml files that are uploaded to a public repo

micha18:03:21

they can cause a number of problems with dependency resolution

ag18:03:37

I keep defending boot, I still have colleagues who are not convinced about the switch (but in this case I’m not completely sure if lein does absolutely the same thing)

juhoteperi18:03:02

I think this should work the same in Lein and Boot

micha18:03:07

boot and lein both use pomegranate and aether

ag18:03:25

ok… cool to know!

juhoteperi18:03:45

But there is still some room for differences/bugs, like https://github.com/boot-clj/boot/issues/577

micha18:03:15

interesting

micha18:03:31

only some dependencies have this issue it seems

juhoteperi18:03:50

It seems quite random, sometimes I see the correct versions and sometimes not

micha18:03:17

like random with the same test case?

micha18:03:25

or random across different dependencies

juhoteperi18:03:44

Random across different days, perhaps different dependencies

juhoteperi18:03:53

But running two times in row give the same results

juhoteperi18:03:23

@micha Btw, did you see my solution to moving exceptions (and other data) between pods: https://github.com/boot-clj/boot-cljs/blob/master/src/adzerk/boot_cljs/util.clj#L46-L75

juhoteperi18:03:41

This should work for all throwables, all Clj datastructures, Dates etc.

micha18:03:14

interesting one sec

juhoteperi18:03:20

One problem I found was Less4J error classes, which don't extend Serializable

juhoteperi18:03:56

(The exception itself is serializable, but it has properties which are not)

micha18:03:13

wait so objectoutputstream serialization can correctly serialize clojure data across clojure versions for example?

micha18:03:34

that's incredible, i wonder how it does it

juhoteperi18:03:34

Or well, I haven't tested between two versions, but works between two classloaders at least

juhoteperi18:03:39

that is what it is used for

micha18:03:58

i worry about binary compatibility with object serialization like that

juhoteperi18:03:01

for Java RMI or something

micha18:03:03

between clojure versions

micha18:03:28

i wrote a crazy thing to get around that

micha18:03:31

with pods

micha18:03:04

this uses the class name and reflection to build a copy of the thing

micha18:03:28

or if it's a native java object (like from the bootclassloader) it just passes it through

micha18:03:39

this method is used in with-pod

juhoteperi18:03:41

Aha, I also implemented similar solution to Boot-cljs which created Exceptions using reflection

juhoteperi18:03:25

What does that Stash do?

micha18:03:55

boot.user=> (def p (boot.pod/make-pod (get-env)))
#'boot.user/p
boot.user=> (def e (try (boot.pod/with-pod p (throw (Exception. "foop"))) (catch Throwable ex ex)))
#'boot.user/e
boot.user=> (type e)
java.lang.Exception
boot.user=> (.getMessage e)
"foop"

micha18:03:07

the Stash is how it passes java objects straight through

micha18:03:49

so the both pods are dealing with the same object rather than one pod making a copy of what is in the other pod

juhoteperi18:03:11

Okay. But yeah, the problem is not passing the java classes which are shared, but the problem is when those objects contain Clj objects which can't be passed straight throughh

juhoteperi18:03:11

I wouldn't worry much about serialization changing between clojure versions, e.g, PersistentHashMap properties haven't been changed in 7 years,

micha18:03:43

right so you can do things like this

micha18:03:49

boot.user=> (boot.pod/with-pod p (def xxx 100))
#object[clojure.lang.Var 0x46f62c15 "#'pod/xxx"]
boot.user=> (def f (boot.pod/with-pod p (fn [x] (+ x xxx))))
#'boot.user/f
boot.user=> (f 100)
200

juhoteperi18:03:45

hm how would that work?

micha18:03:56

that currently does work

micha18:03:06

i just did that in a 2.7.1 boot repl

juhoteperi18:03:21

oh right, both calculations are inside the pod

micha18:03:37

yeah the f that is returned is a proxy that calls through to the other one in the pod

juhoteperi18:03:22

^ re Java Serialization and changing the classes

micha18:03:19

still seems like an issue, no?

micha18:03:25

because it goes only one way

micha18:03:35

like you can add a new field, but you can't remove a field

micha18:03:56

that would mean that you could pass things onle one way betwen pods

micha18:03:05

the other way would break?

juhoteperi18:03:35

I think this kind of data passing is usually only needed from task pod -> main pod

juhoteperi18:03:27

Or well, currently I think I pass all file paths as strings to task impl

juhoteperi18:03:36

Could be useful to use File's

juhoteperi19:03:43

But mostly the serialization is interesting for cases where it is not possible to know what data is passed between pods

micha19:03:47

very interesting development with the serialization

juhoteperi19:03:49

like Cljs exception data

juhoteperi19:03:19

It would be easy to just select the few interesting fields for use in boot-cljs/reload

juhoteperi19:03:36

but Figwheel presumes that the exception is the same as from Cljs compiler

micha19:03:37

yeah i think i am ok with just assuming binary compatibility

micha19:03:42

the same as in the exact same instance?

juhoteperi19:03:57

no, but maps with all the same fields

juhoteperi19:03:00

and different errors and warnings have different metadata, files etc.

juhoteperi19:03:30

in one case the compiler state is atteched to warning metadata, that breaks the serialization because it has (I think) circular reference to itself

micha19:03:09

some things are not serializable i imagine

micha19:03:19

like {:a (fn [])}

juhoteperi19:03:48

Except functions are serializable

micha19:03:24

that must just be something they hacked to get the type system to not complain

juhoteperi19:03:51

Not sure if this really works between classloaders, but serializing function returns something and deserializing that returns a function

micha19:03:55

(def x 100)

(serialize (fn [y] (+ x y)))

micha19:03:05

that can't be serialized really

micha19:03:36

maybe it can but in some weird way

juhoteperi19:03:41

(def foo 100)
((deserialize-object (serialize-object (fn [] {:x foo}))))
;; => {:x 100}

micha19:03:40

that's bizarre

micha19:03:13

serialize-object fn should be in clojure.core

juhoteperi19:03:18

Okay, doesn't really work between classloaders

juhoteperi19:03:30

I think fn returns something which is serializable and refers to the real function

micha19:03:12

boot.user=> (def a (atom 100))
#'boot.user/a
boot.user=> (defn f [] (swap! a inc))
#'boot.user/f
boot.user=> (f)
101
boot.user=> (def f' (serialize-object f))
#'boot.user/f'
boot.user=> (type f')
java.lang.String
boot.user=> (def g (deserialize-object f'))
#'boot.user/g
boot.user=> (type g)
boot.user$f
boot.user=> (g)
102
boot.user=> (g)
103
boot.user=> (g)
104
boot.user=> (g)
105

juhoteperi19:03:55

(def p (boot.pod/make-pod))
(boot.pod/with-eval-in p
  (import [java.util Base64]
          [ ObjectOutputStream ByteArrayOutputStream])
  (defn serialize-object
    "Serialize given Object to String using Object Streams and encode the bytes
    as Base64 string."
    [e]
    (with-open [bos (ByteArrayOutputStream.)
                out (ObjectOutputStream. bos)]
      (.writeObject out e)
      (.encodeToString (Base64/getEncoder) (.toByteArray bos)))))
((deserialize-object (boot.pod/with-eval-in p
                       (def foo 100)
                       (serialize-object (fn [] {:x foo})))))
;; java.lang.ClassNotFoundException: pod$eval56$fn__57

juhoteperi19:03:39

Deserialization fails as the class created by fn is not found in another classloader

micha19:03:43

those classes that are generated on the fly

micha19:03:54

with gensym names

micha19:03:01

yeah that's a hard problem i guess

juhoteperi19:03:17

Yeah, I was surprised it works at all

micha19:03:40

the idea of serializing a lambda is kind of wild

micha19:03:20

my xform thing wraps lambdas with a proxy

micha19:03:27

when you cal lthe function it uses reflection to call it with args that are translated into things that are from the pod the function was created in

micha19:03:40

it's funny, you can pass functions back and forth between the pods as many times as you want, it will keep building more layers of proxies

ag18:03:45

now, I have another question: Can I (probably I can, but should I) override the repl task? Use case: when I get into repl in 99.5% cases I need to kick-off cljs compilation (what could go wrong if I rename boo.core/repl, create another repl task and pass all given args into repl* that composed with cljs?

mobileink18:03:32

just make a "dev" task that composes what u need?

micha18:03:34

yeah that's up to you

dominicm20:03:54

@ag I've found the only reason to want to override boot repl is for cider. But you can customize the start task to be dev which is an idiom in boot.

mobileink21:03:53

dominicm: do you override "repl" or just add a cider task? i follow the advice in "a better way" at https://github.com/boot-clj/boot/wiki/Cider-REPL, but that does not override anything, it just adds a task.

dominicm21:03:31

mobileink I add the cider task, and just run either boot cider repl or boot cider dev as appropriate

mobileink21:03:57

gotcha. no override there, though (so as not to confuse n00bs).

dominicm21:03:07

@U0LGCREMU I don't understand that

dominicm21:03:49

I try and keep things kinda "raw" I don't worry too much about beginners. I concentrate more on educating on the basic concepts which make the reasoning behind additional cider tasks & such make sense.

dominicm21:03:28

I suppose that > I don't worry too much about beginners would be better worded as "I don't worry much about making things easy for beginners."

mobileink21:03:33

when you add a cider task, you"re not overriding anything. just putting another task in the pipeline. the repl task is unchanged

dominicm21:03:06

Yeah. I don't override things if I can help it. I've only been inclined when hearing the easy argument from cider users.

dominicm20:03:07

Or you could stop using cider, but that's optional 🙂

phreed20:03:12

Is there a way to set --no-colors from within a task? [`boot --no-colors foo`]

ag20:03:47

@dominicm cider’s default value for cider-boot-parameters is "repl -s -H :: wait” having .dir-locals.el that sets the value right is not very “user friendly”. I was thinking if I can improve that

dominicm21:03:10

I find that interesting. As a vim user, that idea doesn't even remotely bother me. I've never had anything like cider-jack-in. I find it simpler to run boot/lein as appropriate, then vim connects to the nREPL that is opened. It's exactly the same.

ag21:03:42

hey guys, coleague just installed boot, now getting java.lang.illegalAccessError

ag21:03:58

he’s running Sierra

ag21:03:20

can someone advice?

mobileink21:03:47

don't do that?

mobileink21:03:38

seriously: mo info, please.

mobileink21:03:32

tell us exactly what he did that resulted in that very impolite msg, and we will fix it. pretty sure.

micha21:03:53

oh dear, maybe the error was more serious than we thought

ag22:03:37

alright… this scared me for a moment (there’s already heated discussion and I’m in minority defending boot). Turned out the colleague long-long time ago experimented with boot, and boot was actually pointing to the older executable.

ag22:03:46

sorry guys if I made you nervous

ag22:03:45

everything is fine. boot works fine.

richiardiandrea22:03:46

boot is awesome boot-clj

mobileink22:03:14

@ag: you owe us an account of how things turn out! 😉

ag23:03:02

guys, does anyone use CIDER with boot and set cider-boot-parameters ?

ag23:03:48

so it works on my machine, it works on machine of my colleague, it doesn’t work on machine of my other colleague: he sets cider-boot-parameters, runs jack-in clojurescript, yet cider opens Rhino repl - for me it does the right thing what could be wrong?

ag23:03:31

it seems cider is ignoring cider-boot-parameters