Fork me on GitHub
#boot
<
2016-02-13
>
sbrady00:02:34

Anyone know what the comprehensive list of directories are for OSX where boot caches files? I'm often having problems where boot is trying to find source files that no longer exist. This seems to have come after moving to 2.5+ (on 2.5.5)

micha00:02:10

@sbrady: ~/.boot/cache

micha00:02:15

that's it

sbrady00:02:23

Yeah, I rm -rf 'd that

sbrady00:02:29

Still having this problem

micha00:02:32

have you downloaded a recent version of the boot.sh binary

micha00:02:48

also if you used homebrew are you sure your homebrew is up to date?

sbrady00:02:58

pretty sure I'm on 2.5.5

sbrady00:02:10

After clearing the cache, it re-downloads 2.5.5

sbrady00:02:21

(not on homebrew btw)

micha00:02:28

sounds like it's working, no?

sbrady00:02:06

No, after clearing it, I still get java io can't locate foo on the classpath

sbrady00:02:28

I know it must be my machine because the CI server isn't looking for this errant classfile

micha00:02:58

try boot -BP

micha00:02:05

does that work?

micha00:02:55

so it must be something related to your build.boot or your boot.properties

sbrady00:02:04

that could be

micha00:02:11

those options disable loading boot.properties and build.boot

sbrady00:02:14

but the ci server is using the same build.boot

sbrady00:02:17

could be props

micha00:02:32

try boot -B

micha00:02:41

if that works then it's your build.boot

micha00:02:54

if boot -P works then it's your boot.properties

micha00:02:04

if neither one works then it's both simple_smile

sbrady00:02:11

Hmm, so boot -B seems to be giving a problem

sbrady00:02:15

So what does that mean to you?

micha00:02:32

the -B option disables loading of build.boot

micha00:02:50

so since boot -BP was ok, and the -P option disables boot.properties

micha00:02:01

that means your boot.proeperties is the problem

sbrady01:02:16

OK, so that would be consistent with the CI server saying OK. Got the problem fixed. Thanks!

richiardiandrea02:02:54

can boot or some custom task can bump release given +version+ in your build.boot?

shwx8403:02:35

hey all, is there a method of stripping out an input directory from being seen in subsequent tasks' input-dirs? I see rm, but this appears to just be for individual files.

shwx8403:02:36

my use case is that I have a task that transforms some source files -- the subsequent task should see the transformed files, but not the originals

alandipert03:02:58

rm or sift are the things you want i think

alandipert03:02:17

you can remove the files in one swoop w/ sift + regex

alandipert03:02:34

or if you need more control, do your own rm-ing

shwx8403:02:52

ah okay, I'll look at sift, thanks for the pointer

alandipert03:02:35

it's also worth knowing that the fileset works like git or S3

alandipert03:02:59

there aren't really directories, only paths to objects

alandipert03:02:14

(the classpath also works like this)

alandipert03:02:39

so, to remove a "directory" is going to be removing paths that match some pattern

shwx8403:02:54

ah, perhaps I'm going about my problem the wrong way then. Trying to solve the problem of using find-namespaces from clojure.tools.namespace.find, which expects a root directory of source files. Previously was just trying to feed it one of the temporary transactional directories that boot makes. But this might not make sense to do it this way.

alandipert03:02:07

i would do that by finding all the clj files in the fileset and copying them to a tmp-dir of your own making, then unleash find on the tmp-dir

bendy03:02:24

Hey all, I’m fairly new to clojure & clojurescript, even newer to boot. Coming from JavaScript, I really really like everything boot has to offer. The only thing that I think it is missing are some front end compilation tools. There are plenty of tools written in js (ie postcss) and I would love to port them over to boot, but I don’t really know the best way to start. Can you run clojurescript in a boot task? Can you run javascript libraries in clojure? Or is the best way to interface between them the clojure exec function?

bendy03:02:47

^ apologies for any outlandish beginner assumptions made simple_smile

alandipert03:02:14

@bendy: no worries that's a good question 😄 and welcome to boot-clj

shwx8403:02:28

@alandipert: I like that idea, I'll probably go with that -- thanks for the suggestion!

alandipert03:02:45

@shwx84: surely, enjoy

alandipert03:02:12

@bendy: you can run cljs in clojure via nashorn or another JS for the JVM like Rhino

alandipert03:02:47

what's missing is the nodejs runtime. node adds lots of things to the JS environment that the JS-in-JVM things i'm aware of don't provide

alandipert03:02:10

i found a few abandoned attempts but nothing that probably runs today 😦

alandipert03:02:25

so yeah, i would recommend exec

alandipert03:02:39

or dosh which comes with boot

alandipert03:02:46

(vendored from the conch library)

alandipert03:02:59

the recipe for a task that ran node on some files from the fileset would be to make a tmp-dir, like for @shwx84's problem

alandipert03:02:16

and then aim node at the tmp dir, and add whatever results you want from that tmp-dir to the fileset and then pass it on

bendy03:02:18

@alandipert: awesome, thanks! That’s definitely enough to go off of for now - I’ll report back if I make any progress 😉

alandipert04:02:20

@bendy: cool no problem! probably the fastest way to get cookin' is by looking at 'community tasks' on the wiki and finding the one closest to what you need and hackin' it

alandipert04:02:52

any of them that drive non-jvm translators/compilers are probably good references

richiardiandrea04:02:08

$ rm project.clj profiles.clj; git commit --all <- satisfactory

richiardiandrea04:02:46

Revisited build.boot for importing backend as well

New usage for building: boot -e flavor=fronted|backend build -t prod|dev
New usage for devving: boot -e flavor=fronted|backend dev
A little bit of fiddling with the deps (I had to start with all the deps in the classpath and then remove based on the flavor), but I love it! I will post the build.boot soon

kul08:02:21

any good practice for putting revision in pom.properties using boot?

mobileink12:02:58

Is there any way to set default values for task options?

mobileink12:02:24

I mean in the options spec rather than by writing a fn to do it.

jethroksy13:02:14

@kul I think it's usually done through task options

raywillig13:02:39

@juhoteperi do you know of any issues running on linux with your sass task?

raywillig13:02:14

getting this error: java.lang.UnsupportedOperationException: Platform linux:i386 not supported

jethroksy14:02:58

I didn't have problems running it on mine

juhoteperi14:02:37

@raywillig: it only works on 64bit

juhoteperi14:02:12

If your system is 64bit could be that you have 32bit Java. If your system is still 32bit I would highly recommend upgrade.

juhoteperi14:02:05

There is a issue about this at jsass repo: https://github.com/bit3/jsass/issues/11

mobileink15:02:21

Question about copying dependencies. I need to copy my jar dependencies to build/WEB-INF/lib. I have a task that does this using io/copy. That doesn't seem very bootish, but I haven't been able to figure out how to do it with a fileset. If I had the jars in a fileset I think I could use the target task, which seems more idiomatic. But I don't see how to get the jars into a fileset. Any advice would be appreciated.

jethroksy15:02:12

are you trying to build a jar?

mobileink15:02:37

No. This is for google app engine; for security purpose, the dev server will only look in its own dirs, so it cannot use the jars from ~/.m2/ - they must be copied to WEB-INF/lib, without unpacking.

mobileink15:02:46

pod/jars-in-dep-order gives me the jars, then io/copy works, but that doesn't seem like the Boot Way.

seancorfield16:02:33

We have a task to do almost exactly that… let me refheap it for you @mobileink

seancorfield16:02:47

We have multiple web apps under a common root folder ws-root and this gets out list of (web-libs) dependencies, and copies them to all our WEB-INF/lib folders.

seancorfield16:02:21

I expect that’s pretty much exactly what you came up with? https://clojurians.slack.com/archives/boot/p1455378406006079

richiardiandrea16:02:07

@kul I use boot-semver and set (def +version+ (semver/get-version))

mobileink16:02:47

@seancorfield: Looks like the same idea. But my understanding is that in Boot we should not directly muck about in the target tree, and instead use filesets. Hmm, now that I'm staring at the code some more it occurs to me that the way to go is to io/copy stuff to a core/tmp-dir! and then add that to the fileset for later handling by the target task. I've been using a hardcoded output path, io/copying straight into the target tree.

mobileink16:02:06

So I gather its a bit of both. You write to the filesystem, but always relative to a tmp dir controlled by boot, so you're kinda sorta writing to an unreal FS. Does that sound more or less right?

micha16:02:38

@mobileink: also if you write to your own temp dir and then add to fileset you have excellent control over caching strategies in your task

micha16:02:59

you can use boot.core/fileset-diff to obtain a fileset object containing only changed files

micha16:02:15

and then decide which things in your temp dirs need to be rebuilt

seancorfield16:02:43

One thing to be careful of with target is that its default mode is to erase everything in the target folder so using target with your WEB-INF/lib will require an option to not do that simple_smile

seancorfield16:02:18

You’ll also have to move those classpath JARs to your resource paths to get them into target.

micha16:02:46

@mobileink: re: the WEB-INF/lib dir, did you see the uber --as-jars option, with the war task?

seancorfield16:02:54

I had this discussion with @micha at the time and his recommendation was to do the io/copy pretty much as I did simple_smile

micha16:02:07

those two in combination will result in a war file with all dependency jars in WEB-INF/lib

seancorfield16:02:57

(But, yeah, if you’re prepackaging the entire app as a WAR to deploy to GAE, I’d do what @micha is suggesting here)

seancorfield16:02:06

That wasn’t/isn’t the case for us.

micha16:02:07

boot something something-else uber --as-jars war

micha16:02:24

plus maybe the web task to build a web.xml unless you want to make that your own way

mobileink16:02:43

I did see the uber task but now I see I misread it

micha16:02:10

the war task will find all jars in the fileset and put them in WEB-INF/lib

micha16:02:28

so it works with the uber task

mobileink16:02:00

@seancorfield: regarding target cleaning the target: learned that the hard way a few days ago. 😉

onetom16:02:44

@danielsz: im trying to decipher your latest advancements on boot.system. im a bit lost, because 1. i can't see git tags corresponding to the released versions i see on clojars 2. u have reverted to using ns-tracker from clojure.tools.namespace, though the docs still talks about c.t.n.repl (https://github.com/danielsz/system/commit/98457f6f1c1af1944dd0f89b5a8c21e9ecc7a83d) what will do the changed files detection eventually? boot.task.built-in/watch or c.t.n.repl?

mobileink16:02:24

looks like the uber task is just the thing. regarding WARs for GAE, the dev server doesn't like warfiles - the gradle plugin my boot-gae will replace goes to the trouble of making a warfile and then exploding it. so far it looks like the warfile can be skipped. then when up upload to the prod server the google stuff just knows what to do, no need to make an uberjar

micha16:02:45

yeah the uber and jar processes are separaet in boot

mobileink16:02:50

just tried boot uber --as-jars sift -i "jar$" target -d "build/WEB-INF/lib" and it worked like a charm. that is awesome. boot-gae should be ready to play with in a day or two but i can promise you right now that GAE with boot is about 10000 times easier than it is with gradle/maven. i hate to even think about how much time i spent trying to learn gradle/groovy and then the plugins - a nightmare.

mobileink16:02:38

i'm actually having fun figuring out how to do things with boot. fun doing config/build stuff - that's something i never ever thought i would say!

micha17:02:54

hahaha that's awesome!

onetom17:02:31

@mobileink: shall we have a Google App Engine page on the boot wiki for such recipes?

flyboarder17:02:24

@onetom: i think that might be a good idea, we can also post a blurb about deploying with docker and such

fenton18:02:57

how do i run a repl server, tried: 'boot repl -s', but it just returns...leaving no running process.

martinklepsch18:02:04

@fenton: by default most tasks don't block the pipeline, use boot repl -s wait to have a blocking task at the end

fenton18:02:25

@martinklepsch: thanks martin i'll try that...

fenton18:02:53

@martinklepsch: yay! simple_smile

onetom18:02:13

@flyboarder: actually if @mobileink makes a boot-gae task, then that can have the documentation and we should just link it from https://github.com/boot-clj/boot/wiki/Community-Tasks

onetom18:02:14

(i tried to create a page but then i realized im not sure what would i put into it, neither did it fit anywhere on the current TOC)

flyboarder18:02:25

@onetom: I think the community wiki should have a deployments section somewhere tho, the community projects page is almost big enough to need categories

flyboarder18:02:50

make that the tasks page

onetom18:02:00

what would be under it? • Deployments • Google App Engine • Heroku • Tomcat? • Docker? • VPS? eg Amazon AWS / Digital Ocean / RackSpace (just to have the keywords what ppl might search for)

flyboarder18:02:58

Yeah something like that, if there is a community project for one of them then direct link to that?

flyboarder18:02:25

I was thinking of a heroku task myself and there is already beanstalk

onetom18:02:54

it's actually more like u have certain ways to package and daemonize a clojure app in a 12factor compliant way: • Docker • WAR • Uber JAR • jsvc • Apache something something • Upstart • Systemd

seancorfield18:02:29

FYI, boot-new now has a first cut of generators in the 0.4.0-SNAPSHOT https://github.com/seancorfield/boot-new#boot-generators

onetom18:02:46

and for the various actual PaaS and IaaS provides u can link step by step instructions and document the peculiarities of them

flyboarder18:02:27

So we need 2 sections then, packaging and deploying

flyboarder18:02:08

packaging having the nitty gritty details and deployment having community projects and or instructions if there isnt one

onetom18:02:28

something like that, but im not very experienced in this regard yet. we are still running boot directly via upstart in production on EC2 using Amazon Linux.

flyboarder18:02:40

@onetom: i think you would benefit from the EC2 Container Service then, building the container image is very easy thanks to https://github.com/adzerk-oss/boot-clj-docker-image

fenton18:02:24

how to specify multiple options on the command line: $ boot repl --server --init-ns 'test.core' --eval '(start)' -- wait seems to only run the --server part and ignore the rest?

micha18:02:58

why do you say it's ignoring the rest?

micha18:02:13

if you do boot repl -c are you in the test.core namespace?

fenton18:02:41

let me check

fenton18:02:35

yes, correct namespace, but doesn't want to do my --eval part.

fenton18:02:11

though if i drop the server....the eval part works

micha18:02:52

right yeah

micha18:02:57

that's an option for the repl client

micha18:02:02

not the server

micha18:02:09

because it needs a session to do it

micha18:02:16

there is a ticket about that

fenton18:02:21

is there a way launch custom code from the server?

micha18:02:26

we need to make a separate option for that

micha18:02:33

there currently is not

micha18:02:47

well you can put the code in your init-ns

micha18:02:50

at the top level

micha18:02:52

that would do it

fenton18:02:00

i prefer to have the option of running the code...if it throws errors... hard to get any repl started if errors are thrown on simple file load... 😐

fenton18:02:51

use case was basically to have a repl i could attach to on my production code...but code needs to auto-restart on server reboot, etc...

onetom18:02:31

@flyboarder: i was dismissing docker until recently, because of the big upfront cost of getting the base images. BUT they started to transition to a 5MB Alpine Linux base, so I started to learn it finally. https://insights.ubuntu.com/2016/02/10/docker-alpine-ubuntu-and-you/ and https://www.brianchristner.io/docker-is-moving-to-alpine-linux/

fenton18:02:33

but would be nice to be able to come in, in multiple ways...my custom task, a regular repl no over aggressive NS loading...

micha19:02:16

there is a ticket about the repl option

micha19:02:27

it will be in 2.6.0 hopefully

flyboarder19:02:42

@onetom: yeah I would consider it for sure, if you have issues with the images let me know simple_smile

fenton19:02:50

@micha: okay cool.

micha19:02:03

if you want to give it a shot PRs are welcome simple_smile

micha19:02:33

we're a tiny bit backed up at the moment, but things will improve soon

fenton19:02:38

@micha: okay, maybe, I still don't feel that advanced...but hey why not!

flyboarder19:02:49

@onetom: also interesting if you are running your own PAAS for your apps there is rancheros 20mb linux running docker on the kernel

onetom19:02:49

fenton: i was battling with similar issues a few weeks ago. what makes it even more complicated is that we are also connecting to the repl from intellij we ended up having this:

(def repl-init
  '(require
     '[clojure.inspector :refer :all]
     '[adi.core :as adi]
     '[datomic.api :as d]
     '[org.httpkit.client :as http]
...
     '[midje.repl :refer :all]))

(task-options!
     repl {:eval repl-init})

(deftask dev []
  (alter-var-root #'midje.sweet/include-midje-checks (constantly false))
  (eval repl-init)

  (comp
    (environ :env {:http-port 3001})
    (watch)
    (system.boot/system
      :sys #'sys/dev
      :hot-reload true
      :auto-start true)

    ; (eval repl-init) above is necessary to require the convenience namespaces
    ; so we can have them when connecting from IntelliJ via lein, which can not
    ; simply use the `repl-init` defined in this build.boot
    (repl :server true)))

fenton19:02:11

@onetom: okay i'll muck around a bit and see what i can figure out...

onetom19:02:49

@flyboarder: thanks! (trying the adzerk image now. it will take probably 30mins on my connection to download it though...)

onetom19:02:29

officially 4Mbit/s connection...

flyboarder19:02:56

you and micha both have terrible interwebs 😛

onetom19:02:41

which propels us towards being efficiency conscious simple_smile

micha19:02:58

i have to kick my gf off the internet when i work from home and need to skype

onetom19:02:57

i need to stop my 1.5yr daughter watching "wheels on the bus go 'round and 'round"

onetom19:02:29

which is actually good, just loud simple_smile

seancorfield19:02:12

4Mb/s? Yikes, I couldn't function. I get about 100Mb/s down 12Mb/s up and even that doesn't feel very fast to me these days.

micha19:02:47

when my best carrier pigeon ran off with the squirrel i lost half my bandwidth

onetom19:02:05

5yrs ago i was every other month in Thailand, where i only had 11kbyte/s EDGE mobile connection and still i was administering 40 servers and monitoring them via Newrelic took a little patience but it was quite doable simple_smile

micha19:02:12

now i have packets hidden in acorn caches all over the place, lost for ever

micha19:02:46

@richiardiandrea: looks great

micha19:02:15

you may want to mention that the main.out directory is still needed if you want source maps to work simple_smile

micha19:02:40

yeah, that's why it's left there

micha19:02:43

by default

micha19:02:52

source maps refer to cljs files in that directory

richiardiandrea19:02:16

oh, I hoped it was a 1:1 mapping from main.js

micha19:02:16

we had to do a lot of work to get it to work transparently

micha19:02:36

becaue the source map settings are tightly coupled to a bunch of other settings

micha19:02:54

and you need to do things to make sure it generates the right paths to encode in the compiled js

micha19:02:13

well the source maps need to reflect your input files

micha19:02:27

because they're going to show you where in your code the error happened

micha19:02:38

so you need to have your original files available for that to work

richiardiandrea19:02:06

my JS skills are weak my friend simple_smile

micha19:02:35

i think main.out may already be removed if you have source maps disabled

micha19:02:44

i don't remember though

richiardiandrea19:02:56

I will do a couple of tests here

richiardiandrea19:02:26

with my brand new command: boot -e flavor=frontend build --type prod 😄

richiardiandrea19:02:19

ahahah (ottimo is Italian!)

micha19:02:35

what does it mean?

richiardiandrea19:02:47

it means perfect

micha19:02:07

haha sweet!

micha19:02:25

that sounds like a good name for a child

richiardiandrea19:02:32

ahahahhahahahaha

micha19:02:37

ottimo niskin

micha19:02:46

for a girl perhaps

richiardiandrea19:02:00

no for a girl is ottim*a*

richiardiandrea19:02:28

but it would be maybe too cocky to call your son/daughter like that 😄

richiardiandrea19:02:57

in Italian, we also say: ottimo, Andrea

richiardiandrea20:02:08

that would become: ottimo Ottimo

richiardiandrea20:02:21

a bit weird 😄

micha20:02:14

this is a great word, TIL

richiardiandrea20:02:19

I remember you did not need it ^, I am trying in firefox @micha

micha20:02:01

you will see the folders because it sees the references in the js file

micha20:02:08

but there won't be any sources there

micha20:02:13

it will just be blank

micha20:02:26

you can test it by throwing an exception or logging to the console

micha20:02:39

in chrome anyway you will get a stack trace that links to the source

micha20:02:21

getting source maps to Just Work was a huge pain

richiardiandrea20:02:35

will change the Readme

laforge4920:02:52

May have an issue relating to web workers.

laforge4920:02:21

specifically the cljs.edn file.

laforge4920:02:21

I have a library which creates/runs a web worker. The app should not need to know about it. But if I put the cljs.edn file in the library jar by using a resource path, it gets ignored.

laforge4920:02:53

I can only get it to work by adding the cljs.edn file to the app resource path. simple_smile

laforge4920:02:15

From a web-worker perspective, this makes no sense. Or am I missing something?

laforge4920:02:47

Perhaps I should be creating the js file somehow in the library jar?

laforge4920:02:26

Currently the only thing I do in the build.boot of the library is create the jar with all the source files.

micha20:02:21

is the cljs.edn file actually in the jar?

micha20:02:38

jar tf project-1.2.3.jar |grep cljs.edn

laforge4920:02:42

oh, I didn't check. back in 5.

laforge4920:02:06

(it isn't right now, I need to put it back.)

laforge4920:02:08

Yes. It is at the top level of the jar file.

micha20:02:43

ok yeah i see the issue now

micha20:02:57

the .cljs.edn file is an application spec

micha20:02:19

i don't think it's appropriate for applications to be built via transitive dependencies

micha20:02:50

like if you have some distant transitive dependency that is starting web workers

micha20:02:07

you need some way for the project to opt in

micha20:02:21

like a boot task perhaps

micha20:02:28

that can pull the things into the project

micha20:02:31

from the jars

laforge4920:02:43

I can live with that. simple_smile

micha20:02:10

it also avoids the need to scan the classpath

micha20:02:18

which can have hundreds of thousands of things in it

micha20:02:24

and can be quite slow to do

laforge4920:02:00

so this way the app only needs to know which jar files might be creating web workers.

micha20:02:45

i imagine you could do it like the cljs-reload task works

micha20:02:05

you'd have the web-worker task that pulls the things into the project

laforge4920:02:06

And to take it a step further, a library which depends on another library might want to do the curtesy of "pulling in" from its dependencies, so the app only needs to know about the top-level dependencies.

micha20:02:11

and performs the integration

micha20:02:36

yeah that's the benefit of using the task approach

micha20:02:41

the application doesn't need to know much

micha20:02:48

you just put the right tasks in the pipeline

micha20:02:51

like with boot-reload

micha20:02:59

the pplication knows nothing about reloading

laforge4920:02:05

So would it be fair to submit a request for such a web worker? simple_smile

micha20:02:09

that's added in the build pipeline

micha20:02:18

sure go for it

micha20:02:05

you could just make a tssk

micha20:02:24

it doesn't need to be part of boot-cljs

micha20:02:37

like boot-reload is a separate thing

micha20:02:42

and so on

laforge4920:02:43

which project then? I know it can be independent.

micha20:02:01

yeah i would probably make it a separate task, no?

micha20:02:25

how do you mean?

micha20:02:43

like in its own jar on clojars

laforge4920:02:43

you are saying a separate repository should be created?

micha20:02:02

probably yeah

laforge4920:02:16

then there is no place to add an issue.

micha20:02:31

what is the issue for?

laforge4920:02:41

web worker task

micha20:02:01

hm, i don't follow

micha20:02:12

you are going to make a web worker task, right?

laforge4920:02:40

perhaps. I'd need to dig into boot of course.

laforge4920:02:45

and I do have the time.

laforge4920:02:52

could be a good project for me.

micha20:02:15

yeah just try one, see if you like it simple_smile

micha20:02:25

after that we charge

laforge4920:02:45

I'll get this api change to durable-cells done first. Then I have a need for it. simple_smile

alandipert21:02:55

@onetom: i experimented with alpine recently but discovered it doesn't use or come with glibc, so a binary i needed to run in there didn't work

alandipert21:02:21

@onetom: just a PSA really - i think if you stick with what they have packaged or if you compile native stuff in the container, you're fine

seancorfield22:02:03

OK, https://github.com/seancorfield/boot-new 0.4.0 with generator support is up on Clojars. Built-in generators are ns and defn. PRs welcomed to add more. Generators can also be part of your own project (although you’ll need to depend on Boot new!).

richiardiandrea22:02:01

Oh man cool stuff!

richiardiandrea22:02:14

now we need cider integration 😄

richiardiandrea22:02:44

for defn I use yasnipped but ns is very useful indeed

seancorfield22:02:20

ns and defn were just the first two that popped into my head, and for defn I needed to update boot.new.templates/->files to support appending to an existing file so it was all good.

seancorfield22:02:05

I’ll probably switch my FW/1 web framework template from Leiningen to Boot now and add generators for controllers and views etc.

richiardiandrea22:02:15

Great yes, it would be nice for instance to generate re-frame handlers/subscriptions boiler plate