Fork me on GitHub
#boot
<
2016-01-08
>
danielgrosse08:01:50

How do I compile Clojurescript for production?

jethroksy08:01:25

is the setting of BOOT_VERSION to 2.4.2 intentional?

nha09:01:22

@jethroksy: it probably just haven't updated (but the README explains how to use another version).

jethroksy09:01:48

@nha: his latest commit seems to suggest he wants it to pull the latest version though

jethroksy09:01:52

seems like ideally it'd be better to programatically generate BOOT_VERSION and BOOT_CLOJURE_VERSION, perhaps with boot -BPV | sed ... (i'm terrible at bash)

martinklepsch10:01:18

@danielgrosse: optimizations :advanced

martinklepsch10:01:38

Check the clojurescript Compiler Wiki for more Information :)

danielgrosse11:01:45

When I do this, the whole compiled files are also copied to target. How can I generate a clean package for deployment?

dave11:01:58

@jethroksy: whoops -- yeah, that was totally unintentional. we do want it to pull in the latest version, not hard-code BOOT_VERSION and BOOT_CLOJURE_VERSION

dave11:01:11

i think we could just leave those 2 env vars out of the dockerfile

dave11:01:41

let me know if anyone thinks that's a bad idea, otherwise i'll make that change

jethroksy11:01:11

I'm good with that

martinklepsch12:01:21

@danielgrosse: the sift task can help with that. boot sift -h β€” combining --include and --invert will exclude entries from the fileset

martinklepsch12:01:38

for debugging try a line like boot build sift --invert --include "app.out" show --fileset

martinklepsch12:01:16

where build is the task to build your app. you'll likely need to adjust "app.out".

socksy14:01:04

if all shebangs are of the form #!/usr/bin/env boot, why get the base name for "boot" here? https://github.com/boot-clj/boot/blob/master/boot/core/src/boot/main.clj#L84-L91

socksy14:01:43

as the boot path is set incorrectly on NixOS it breaks all shebang scripts (not least because we can't use the program reported by the path)

alandipert14:01:25

@socksy: that logic is related to shebang difference between mac and linux - see 2nd to last comment on https://github.com/boot-clj/boot/issues/105

socksy14:01:39

then the code isn't doing that

socksy14:01:51

because this is saying check if this exists and whether it has boot in it

socksy14:01:23

but the code is checking that the file has the jvm property boot.app.path

socksy14:01:13

which on NixOS is being set in a .boot-wrapped file (??? no idea if this is a NixOS only thing, can't see it in the NixPkg description), which is not the path to boot

socksy14:01:54

/usr/bin/env boot exists, but this shebang function only returns if the file is /usr/bin/env .boot-wrapped

alandipert14:01:09

in the shebang case, isn't arg0 the name of the shebang script?

socksy14:01:12

the latter won't work, because the former sets the location of the JVM

alandipert14:01:36

i'd have to warm up on the code again, but i know at least some of this code is about finding the path of the script being executed. presumably the path to boot itself was already found

socksy14:01:55

if you look at https://github.com/boot-clj/boot/blob/master/boot/core/src/boot/main.clj#L84-L91 , arg is the path of the script being executed, but the base-path is just derived from (System/getProperty "boot.app.path")

socksy14:01:20

and it's boot.app.path that's at fault here

alandipert14:01:09

looks like its set in https://github.com/boot-clj/boot/blob/master/head.sh - is the logic there wrong?

socksy14:01:23

not per se

micha14:01:06

that property needs to be set to differentiate boot shebang scripts from other shebang scripts

micha14:01:20

so it doesn't try to execute a python script for example

socksy14:01:09

but the property is set as .boot-wrapped on my machine. All that is in the boot executable is the following:

#! /nix/store/7d2g3bvmbi5ps3vlkl54h7d7yasc4ynv-bash-4.3-p42/bin/bash -e
export JAVA_HOME=/nix/store/2h6lgjp9lyw8ifjiv1cwpyk0dm053mgc-openjdk-8u76b00
export PATH=/nix/store/2h6lgjp9lyw8ifjiv1cwpyk0dm053mgc-openjdk-8u76b00/bin${PATH:+:}$PATH
exec -a "$0" /nix/store/wm2101r6iwas1bxhhs95727s0cmqwivc-boot-2.4.2/bin/.boot-wrapped "${extraFlagsArray[@]}" "$@"

socksy14:01:22

.boot-wrapped has the head.sh in it

socksy14:01:40

so it gets the wrong $selfpath, and then passes the wrong thing to boot.app.path

micha14:01:53

that's not the head.sh that boot uses

micha14:01:00

where did that come from?

socksy14:01:54

I suspect from some nix installation package

socksy14:01:59

but I couldn't find it

socksy14:01:17

it's necessary because otherwise boot doesn't have Java

micha14:01:27

you can use the BOOT_JAVA_COMMAND env var

micha14:01:34

and BOOT_JVM_OPTS

micha14:01:48

instead of hacking boot.sh itself

socksy14:01:18

i could if i could find where whomever made this package hacked boot.sh in the first place

socksy14:01:44

wait I must be reading this bash script wrong

micha14:01:26

what's that boot-wrapped thing?

micha14:01:45

can you paste the cpotents of that file

micha14:01:47

*contents

socksy14:01:28

first 6 lines are head.sh, the rest is binary

socksy14:01:57

(only difference between head.sh and it is the patched shebang line to point at the real bash)

socksy14:01:27

@ragge apparently you made the nixpkg, perhaps you have an input simple_smile

ragge14:01:23

note that i've pushed an updated to nixpkgs master for boot

ragge14:01:27

no longer uses makeWrapper

ragge14:01:53

sets default value for BOOT_JAVA_COMMAND to nix location for java, instead of just java

ragge14:01:09

not sure what the issue you are having is, but worth trying master

micha14:01:45

@ragge: this is sweet, we should add a note to the README for the nixos package, you think?

socksy14:01:02

ok great, I'll give it a spin simple_smile

socksy14:01:13

very glad I don't have to try and fix it myself πŸ˜„

ragge14:01:37

@micha: sure, can do. boot is actually a quite pleasant nix citizen the way the script is currently written

crisptrutski14:01:38

thanks - just curious at appetite to add the feature to core, or help to do this as a library simple_smile

micha14:01:36

security stuff is always welcome in boot itself simple_smile

micha14:01:15

this PR looks great

crisptrutski14:01:21

i just added the ^{:boot/from :technomancy/leiningen} really

crisptrutski14:01:29

and looks like a fudged the whitespace there a bit >_<

micha14:01:37

haha no worries

micha14:01:02

looks great to me

crisptrutski14:01:06

awesome - this is last piece needed to get another shop over to all-boot-all-the-time simple_smile

crisptrutski14:01:11

just pushed the scope anyway

crisptrutski14:01:13

β€œsymmetry"

micha14:01:28

@crisptrutski: this won't be usable when boot loads its own dependencies, is that a problem?

micha14:01:46

boot uses aether to fetch itself before any build.boot or set-env! is evaluated

crisptrutski14:01:27

in my use case that’s fine - using secure repo only for non-clojars deps

micha14:01:52

what is the format of the :certificates key?

crisptrutski14:01:59

vector of file path strings

micha14:01:22

i will merge this for now

micha14:01:50

if you want toyu can add an env var for it

micha14:01:55

or i can do that

micha14:01:27

i just sent invite to boot-clj org btw

crisptrutski14:01:32

does this mean i qualify for free stickers now? doge😊

micha14:01:41

very much so!

socksy14:01:58

yes! Finally shebangs are working!

crisptrutski14:01:04

not quite sure what you mean with the var - would I be reading it in App.java before the boot.aether shimming?

micha14:01:31

no, it would be an env var like BOOT_CERTIFICATES

micha14:01:40

and be a colon delimited string perhaps

micha14:01:52

you could set that in your shell env or in boot.properties

crisptrutski14:01:09

so just another way to set it, not a solution for an initial pull

micha14:01:19

you'll see in boot.aether namespace where we handle the BOOT_CLOJARS_REPO env var for example

crisptrutski14:01:23

not quite sure how that chicken/egg would work seeing as aether is another jar simple_smile

micha14:01:37

you can modify aether.clj

martinklepsch14:01:38

@crisptrutski: if you feel like it at addition to the changelog would be great πŸ‘

crisptrutski14:01:00

yessirre - wiki too

micha14:01:28

i mean modify boot/aether/src/boot/aether.clj

micha14:01:38

not the aether.clj in pomegranate lol

crisptrutski15:01:32

@micha: updated, but super naive (should perhaps be delayed?)

crisptrutski15:01:14

also perhaps you’d prefer a PR to your 2.6.0-SNAPSHOT branch?

micha15:01:51

@crisptrutski: PR to 2.6.0-SNAPSHOT would be ideal, if you don't mind

micha15:01:02

excellent PR, this is going to be a good day

crisptrutski15:01:06

reopening this PR is the ideal excuse for my first push to a boot repo that isn’t a fork πŸ˜‰

crisptrutski15:01:19

rebase on the tag instead of the branch simple_smile

crisptrutski15:01:56

I’m off, have an awesome weekend all you bootaneers

crisptrutski15:01:32

@micha: thanks for all the awesome improvement you’ve been knocking out, being so patient and helpful, and for the wings boot-clj octocat

jethroksy15:01:01

when i run docker run -it adzerk/boot-clj repl the first time, it downloads all the dependencies

jethroksy15:01:32

when i run it the second time, I was expecting it to pick up the downloaded dependencies, but it redownloads all the dependencies for my project and for boot again

jethroksy15:01:39

is this behaviour expected?

jethroksy15:01:16

I was under the impression that ENV BOOT_LOCAL_REPO /m2 and this would do the caching:

# download & install deps, cache REPL and web deps
RUN /usr/bin/boot web -s doesnt/exist repl -e '(System/exit 0)' && rm -rf target

alandipert15:01:48

@jethroksy: if you want to keep the m2 across containers, you may consider making m2 a "shared volume"

alandipert15:01:58

otherwise each container has its own m2

jethroksy15:01:44

hmm yeah that just came across my mind

martinklepsch18:01:27

seeing a strange thing:

(println
  (pod/with-eval-in cpod
    (let [x (confetti.s3-deploy/dir->file-maps ( ~dir))]
      (println x)
      x)))

martinklepsch18:01:01

this first prints x but then the outer println seems to receive nil

martinklepsch18:01:18

does this make sense to anyone?

micha18:01:26

not to me simple_smile

micha18:01:39

actyually maybe

micha18:01:56

can you do (prn :x x) or something in the pod

micha18:01:18

you'll get nil returned from the pod if the result can't be read/printed

micha18:01:50

and println implicitly calls .toString on things

micha18:01:56

hiding problems

martinklepsch18:01:02

there are file objects

martinklepsch18:01:52

problem solved, thanks!

micha18:01:02

πŸ’₯

martinklepsch18:01:10

btw, did you have any more thoughts about the object passing between pods? I was thinking that the send! stuff is somewhat related and probably a solution should consider both problems?

micha18:01:22

yeah totally

micha18:01:37

i think i know how to do it but just got sidetracked

micha18:01:07

the plan would be to encode all the EDN supported clojure types as java Arrays

micha18:01:58

only instead of building json it would build live java ArrayList objects

micha18:01:05

those can pass between pods no problem

alandipert18:01:33

primitive object arrays ideal probly

micha18:01:40

and for each object of type not supported by EDN we would get the classloader for it

micha18:01:48

and determine whether it can pass between pods

martinklepsch18:01:54

I see. so the only problem really is passing stuff from clojure and up because this isn't shared between classloaders

micha19:01:20

yeah clojure protocol interfaces and classes are created dynamically and are local to a single clojure runtime

micha19:01:45

if you pass a clojure object, that object is an instance of a class that is unknown to other clojure runtimes

micha19:01:00

because they have created their own class definitions at runtime and they're different

martinklepsch19:01:07

yeah, I think I'm beginning to understand

micha19:01:11

so IFn in one pod is not the same as IFn in another

micha19:01:28

so passing a function you get "function doesn't implement IFn"

micha19:01:53

but the core java objects are the same in all pods

micha19:01:13

that still doesn't address the send! problem though

micha19:01:33

the send! problem is a result of the way clojure evals expressions

micha19:01:43

when you do (eval '(....))

micha19:01:01

it creates a new class dynamically

micha19:01:05

with a static method

micha19:01:33

if the expression you're evaling is too large then you get an error from the JVM, because there is a limit to how large the bytecode for a method can be

micha19:01:24

so the solution there is to not inline the unquoted things in the template

martinklepsch19:01:34

you mean things like ~big-data?

micha19:01:48

exactly yeah

micha19:01:05

so the with-eval-in macro would need to be able to detect large ones

micha19:01:15

and wrap them in send!

micha19:01:41

but because of the macro DSLs in clojure it's not foolproof

micha19:01:56

like (ns (send! ...)) doesn't work

micha19:01:07

because inside the macro dsl you're not really computing

micha19:01:18

so you can't replace one thing with another equivalent thing

micha19:01:55

but on the other hand it wouldn't be likely that you'd be making 10,000 clauses to ns

micha19:01:15

so i think it would work as expected like 99.9% of the time

martinklepsch19:01:32

I understand now, so these two things aren't actually as related as I thought they are πŸ˜›

micha19:01:37

this cljson approach will result in much better error messages too

micha19:01:49

like instead of returning nil there it could have told you what the problem was

micha19:01:59

or actually in your case there would be no problem in the first place

micha19:01:11

because File objects can pass between pods just fine

juhoteperi20:01:14

Should boot-cljs build identifiers start from 0 or 1? @martinklepsch did you have an opinion on this πŸ˜„?

juhoteperi20:01:47

Ha! Three commits in one minute. Vimagit is great.

martinklepsch21:01:04

@juhoteperi: build identifiers? πŸ˜„

juhoteperi21:01:52

the number in version after -

martinklepsch21:01:00

I'm usually using -0 for inital

martinklepsch21:01:21

I think cljsjs also usually has -0

micha22:01:58

hi dzajkowski

mccraigmccraig22:01:03

so i just upgraded all my boot infrastructure thus https://www.refheap.com/f95a6b757d3bfc307492515ca and it's mostly working fine, except that if i try to start a cljs-repl from the command-line with boot repl -c ... (start-repl) then my target directory gets emptied by the boot repl -c and i lose all the cljs/less compiled resources. if i connect from CIDER the cljs-repl works just fine ... any idea what i've borked ?

micha22:01:11

@mccraigmccraig: which version of boot?

mccraigmccraig22:01:50

@micha: 2.5.5 on clj 1.7.0

dzajkowski22:01:27

@micha oh sorry, hi πŸ˜‰

micha22:01:10

@mccraigmccraig: do you have BOOT_EMIT_TARGET=no set?

mjmeintjes23:01:33

@mccraigmccraig I had the same issue and setting BOOT_EMIT_TARGET=no worked for me