Fork me on GitHub
#boot
<
2016-08-03
>
typedeph00:08:05

I'd eventually like to use clojure and clojurescript side by side

typedeph00:08:14

hate to have 3-4 windows open just for that

micha00:08:33

it would be a lot to do at once for me though

micha00:08:50

but maybe that's just me

typedeph00:08:55

I think the thing that sucks

typedeph00:08:22

is I can get this done in leiningen with zero problems, but I really want to use boot as it appears similar to gulpjs

typedeph00:08:09

might just have to defer boot to clojurescript alone if I can't figure this out soon

micha00:08:12

leiningen is easier to google when you have an issue, i'm sure

micha00:08:45

yeah when i'm learning a new thing i hate getting bogged down in tooling

typedeph00:08:14

so do I but I keep thinking the payoff will but much higher in the long run

micha00:08:21

have you seen the modern-cljs book?

typedeph00:08:33

yea, was referencing it for this

micha00:08:42

that's a step by step progression

typedeph00:08:44

doesn't mention cider

typedeph00:08:55

but the benefits for clojurescript seem very real

micha00:08:58

you don't strictly need cider though

micha00:08:02

i mean eventually yes

micha00:08:25

but there is still lots of stuff to make progress on without that

micha00:08:44

and when you're more familiar with clojure and all that you can probably easily debug your cider configuration

micha00:08:44

without a good grasp of the underlying clojure stuff, it will be hard to maintain a complex tower of tooling

typedeph00:08:04

have a strong feeling its not my clojure config

typedeph00:08:24

I want C-k and C-e, just had a whole class on cl

typedeph00:08:34

those two things are invaluable for me

micha00:08:27

leiningen project.clj is a static map, so you just need to learn the names of different keys etc

micha00:08:37

which stack overflow is really good at providing

typedeph00:08:39

I don't have a strong motivation to use boot besides the point of "it will be a better tool once I use both clojure/clojurescript"

micha00:08:55

yeah maybe you should come back to it later

micha00:08:12

boot requires that you are familiar with the underlying stuff

micha00:08:27

and cider integration is tricky

typedeph00:08:28

I've seen the deftasks

typedeph00:08:31

their not that hard

typedeph00:08:37

its the cider integration like you say

micha00:08:52

an alternative is inferior lisp mode

micha00:08:56

with a socket repl

micha00:08:02

something to consider

micha00:08:47

i use vim so i don't know how to do things in emacs very well

micha00:08:10

i can say that vim fireplace works without any configuration

typedeph00:08:56

after making a new project

typedeph00:08:00

for some reason it wants to work now

micha00:08:24

inscrutible

typedeph00:08:29

thanks for the help anyways

micha00:08:38

have fun!

oahner00:08:27

@micha: it does cause a problem; windows doesn't recognize "/C:/file.txt" as being a valid path, it'd need to be "C:\file.txt", altho most of the win32 API will also accept "C:/file.txt"

oahner00:08:28

(map #(.getPath (URL. %))) returns the path component of a URL, I believe this would be more robust: (map #(.getPath (.toFile (Paths/get (.toURI (URL. %))))))

oahner00:08:11

if I'm not mistaken, this would convert the URL to a filesystem path in a platform-independent way

micha00:08:58

seems legit to me

oahner00:08:48

I think that logic would need to be applied to more than one place tho

oahner00:08:28

a couple keys under (core/get-env) also contains values where there's a leading forward slash before paths

oahner00:08:30

obviously doesn't cause issues internally, but as soon as a value is made available to the other windows programs, issues arise

oahner00:08:10

clojure's stdlib could really use a url->path function that works everywhere

micha00:08:27

this whole thing is a minefield of issues

micha00:08:56

because some tasks want to make correlation between paths on the filesystem and URLs for like HTTP for example

micha00:08:59

things like that

oahner00:08:14

indeed, minefield of issues is an understatement

micha00:08:06

then you have regexes that might match paths

micha00:08:24

there is no escape character that represents the path separator

oahner00:08:34

on the other hand, everything that trickles down to a value being made available through a System property should be in the format of the current system, no?

micha00:08:00

i guess they should be in the correct format for the intended consumer

micha00:08:23

like fake.class.path system property is for editors and IDEs i think

oahner00:08:56

in fact the next line (partial string/join (System/getProperty "path.separator")) joins it based on the system's path separator

micha00:08:15

which is guaranteed to be : lol

oahner00:08:15

fair to assume the paths should be in the system's format as well

oahner00:08:27

lol, really?

micha00:08:30

yeah i think so

oahner00:08:31

that's funny

oahner00:08:45

I thought some systems might have ;

micha00:08:05

i think path.separator is the classpath not the PATH

oahner00:08:33

I get ";" as path.separator on this system

micha00:08:41

ah interesting

oahner00:08:10

so in set-fake-class-path!, I believe both boot.class.path and fake.class.path would benefit from converting the the URL to a path using Paths.get

oahner00:08:25

unfortunately I only have a windows box to test it

oahner00:08:32

no idea what would happen on linux or osx

micha00:08:50

i think unix machines would be fine

micha00:08:02

i think your solution looks good

micha00:08:17

i can test it anyway

micha00:08:22

'm on linux here

oahner00:08:31

I wish I was on linux

oahner00:08:56

working at a windows shop has its fair share of downsides

micha00:08:14

i guess using windows is one of them 馃檪

micha00:08:39

is boot working okay on windows 7?

oahner00:08:03

I used to have file locking issues last time I tried it, I believe, in august 2015

oahner00:08:08

haven't gotten that far yet tho

micha00:08:12

i hear windows 10 solves those problems

oahner00:08:23

I'm currently trying to convert one of our lein/figwheel project to boot/boot-reload, I got it to work until this issue caused vim-fireplace to not be able to display the result of a REPL eval

oahner00:08:03

went through vim-fireplace for a couple hours until I realized boot was generating the wrong path format for windows in fake.class.path

oahner00:08:17

debugging vimscript is tedious

micha00:08:52

if you make a PR against master we can merge it

micha00:08:06

and i can make a 2.7.0-rc1 release

micha00:08:26

can you build boot locally?

oahner00:08:39

now would be a good time for me to find out how!

oahner00:08:54

I'm gonna grab dinner and look into that this evening

micha00:08:54

hopefully it's just clone repo and do make install

micha00:08:04

oh whoops

micha00:08:07

you need make

oahner00:08:12

I have msys2

oahner00:08:29

I dabble in rust on the weekends, lol

kenzaburo01:08:43

is anyone using :preloads with boot-cljs ? any pointers on where should I use that option? in the task or in the .edn?

oahner03:08:20

phew, I did it

richhickey12:08:15

Just installed boot (with brew install) on machine without it, running into this problem:

anmonteiro12:08:33

@richhickey: 2 things: does the error still happen if you run boot -B?

anmonteiro12:08:47

2nd: what鈥檚 the output of boot -b?

richhickey12:08:26

does not happen with boot -B

anmonteiro12:08:59

so the problem is somewhere in your build.boot or its dependencies

anmonteiro12:08:10

(or maybe profile.boot)

anmonteiro12:08:21

it鈥檚 right there

anmonteiro12:08:29

[depsfile (io/file "deps.edn")]

anmonteiro12:08:37

but no require for

richhickey12:08:52

ah, I鈥檓 in a project dir with boot stuff

richhickey12:08:07

that was it, thanks @anmonteiro !

anmonteiro12:08:16

yeah, it鈥檚 why boot -B works, it doesn鈥檛 load the boot script

anmonteiro12:08:20

np, happy to help

richhickey12:08:26

-b good to know

anmonteiro12:08:37

boot -h lists all this stuff

richhickey12:08:29

boot -h wasn鈥檛 working either

anmonteiro12:08:52

oh, of course it wasn鈥檛, silly me

micha14:08:37

@richhickey: do you still want to be able to use a development version of Clojure via BOOT_CLOJURE_VERSION, or do pods accomplish everything you need there?

micha14:08:07

assembling a new release, could include that

marc-schwartz18:08:50

Does anyone know how to use a standalone .jar file as a dependency in build.boot?

timothypratley18:08:07

@marc-schwartz: I have not, but just an idea; maybe :resource-paths #{"src" "lib/my.jar"}?

marc-schwartz18:08:49

@timothypratley: thanks! I will try that and if not, perhaps a local mvn repo

timothypratley18:08:54

Often recommended is installing the jar in your m2, or hosting a repo for a company

timothypratley18:08:52

something like mvn install:install-file -Dfile=<path-to-file>

alandipert18:08:06

@marc-schwartz: local mvn is good, there's also boot.pod/add-classpath

micha19:08:19

@marc-schwartz: example of alan's suggestion: (boot.pod/add-classpath "lib/my.jar")

marc-schwartz19:08:08

Thanks, everyone! Going with local mvn repo option...

kenny22:08:30

Is it possible to have task options be recursively merged? For example, I set up some default compiler options using like this:

(task-options! cljs {:compiler-options {:language-in :ecmascript5}})
I then have a task which then adds new :compiler-options:
(deftask do-stuff
         []
         (comp
           ;; do stuff before
           (cljs :compiler-options {:closure-defines {'myns.core "foo"}})
           ;; do stuff after
           ))
:compiler-options is replaced with the map I defined in the task. Instead, I would like to recursively merge :compiler-options. Is this possible?

micha22:08:45

@kenny: i believe so, check out (doc task-options!) in the repl

kenny22:08:22

Oh I see. Rather than setting the cljs task options inline I should use task-options! inside my task.

micha22:08:37

well probably not

micha22:08:53

this is what the .cljs.edn files are for

micha22:08:15

using task-options! like that isn't a good solution becuase you're mutating global state

micha22:08:25

the .cljs.edn file is scoped to the pipeline

micha22:08:56

so what you want to do is consume the existing .cljs.edn file, add options to it, then write your new version of it and add it to the fileset

micha22:08:59

using task-options! in the way you're describing will be complicated because it's not scoped to the pipeline

micha22:08:42

this is precisely the use case the .cljs.edn file is designed to accomodate

micha22:08:13

tasks that need to cooperate but don't need to know anything about each other

kenny22:08:29

So then is this the preferred way to set up dev and prod build tasks: Have your .cljs.edn file(s) contain the default options for both dev and prod build tasks. If you need to set specific options for either the dev or prod task you need to read the .cljs.edn file and merge it with the options you want to set.

micha22:08:30

yeah you'd have a task that automates that

kenny22:08:13

Okay that makes sense for that specific case. What about in the general case where you may want to inline merge task options?

micha22:08:50

you shouldn't need to do that

micha22:08:09

the task options are only really for static things

micha22:08:22

because the task itself, the function, is global and mutable

micha22:08:40

so you don't want to be messing with that inside the pipeline

micha22:08:02

a rule of thumb is that tasks shouldn't ever mutate any global things

micha22:08:09

that's why we have the fileset

micha22:08:26

so the mutation is hidden in temp dirs, etc

micha22:08:45

if your task mutates global state in the pipline, then consider something like the watch task

micha22:08:20

one of the things that makes tasks composable is the guarantee that every time the task is called it's called with a clean environment

micha22:08:55

if you're mutating vars you lose that

micha22:08:18

the fileset can be rewound in time, by just "commit!" an old fileset object

micha22:08:26

and pass it to the next task

micha22:08:43

task-options! is a wrapper around alter-var-root

micha22:08:47

it mutates the var

micha22:08:54

that doesn't rewind

micha22:08:30

i think maybe you want to do something like what the cljs-reload task does

micha22:08:42

when you use the reload task it looks for .cljs.edn files

micha22:08:55

it emits cljs code that it generates on the fly

micha22:08:13

then modifies .cljs.edn files such that they :require the namespaces it generated

micha22:08:32

so it can be composed with any tasks that use the .cljs.edn interface

micha22:08:57

compare that approach to for example having a :requires option for the cljs task

micha22:08:08

now the reload task would need to know about the cljs task

micha22:08:21

and it would need to mutate global state to set its options

micha22:08:48

and that isn't reversible, so the tasks that precede it in the pipeline would see the modifications it made to the cljs task

micha22:08:58

basically it's spaghetti now

micha22:08:05

everything coupled to everything

micha22:08:21

with the .cljs.edn method though, none of the tasks need to know anything about each other

micha22:08:31

they just need to know about the .cljs.edn format

micha22:08:42

in fact, you could be using a completely different cljs task

micha22:08:10

so that part works kind of like a leiningen project map

micha22:08:26

in that it's a schema for immutable data that some task will interpret

kenny22:08:01

So the key here is the rewinding -- task options are not passed around like the fileset is. If they were then you would get the rewind functionality. Correct?

micha22:08:46

task options though are for a different purpose

micha22:08:08

they're to pass information to the tasks about things that normally can't change over time

micha22:08:15

because the task needs to allocate long-lived resources

micha22:08:32

resources that live for the lifetime of the entire pipeline

micha22:08:41

not just one task execution

micha22:08:09

so basically for stuff that changes during the pipeline you can use the fileset to transmit info

micha22:08:14

the options are for things that the task constructor needs to know

micha22:08:31

and you can only construct a thing once

micha22:08:45

i mean once during its lifetime

kenny22:08:44

Just to get some clarification.. What are you thinking I mean when I say inline merge task options?

kenny22:08:24

Were you thinking this?

(deftask do-stuff
         []
         (task-options! cljs {})
         (comp
           ;; do stuff before
           (cljs :compiler-options {:closure-defines {'myns.core "foo"}})
           ;; do stuff after
           ))

micha22:08:35

ah i see, i was thinking you would be doing it in the pipline

micha22:08:39

like in the comp part

micha22:08:06

or more precisely in the handler function the task returns that becomes part of the pipline

micha22:08:34

if you do your global mutation where you have the task-options! in your example tat's better

micha22:08:46

but it does mutate that var

micha22:08:13

so if you're doing (boot (do-stuff)) in the repl you will have artifacts from the first time you do it that persist to the second time

kenny22:08:18

Right. What I had originally had wanted was to do something like this:

(deftask do-stuff
         []
         (comp
           ;; do stuff before
           (cljs :compiler-options #(merge % {:closure-defines {'myns.core "foo"}}))
           ;; do stuff after
           ))

micha22:08:07

for closure defines btw i usually just use a macro

kenny22:08:24

That was just for an example 馃檪

micha22:08:35

js doesn't have macros so they added a special case to the compiler

kenny22:08:05

Actually my specific use case is I want the {{:compiler-options {:language-in :ecmascript5}} set for all cljs tasks and I want to merge that with my :compiler-options in my task.

micha22:08:34

ok so what you want at the end of the day is to compile all your stuff as :ecmascript5?

kenny22:08:29

Well, yes. But it seems like the example I posted above may be a useful functionality.

kenny22:08:53

To do something like this:

(task-options! cljs {:compiler-options {:language-in :ecmascript5}})

(deftask do-stuff
         []
         (comp
           ;; do stuff before
           (cljs :compiler-options #(merge % {:verbose true}))
           ;; do stuff after
           ))

micha22:08:00

yeah that may be the best solution

kenny22:08:26

In that example I want {:compiler-options {:language-in :ecmascript5}} set as default for all cljs tasks. Then in the do-stuff task I want {:compiler-options {:verbose true}} merged with the defaults for cljs.

micha22:08:07

i really try to keep things simple with the cljs compiler

micha22:08:14

and not get fancy with it

kenny23:08:20

This is just an example though. It seems like it would be useful in other contexts.

micha23:08:46

the .cljs.edn way is the most powerful

micha23:08:05

the update-in way that you show above is almost as powerful

micha23:08:24

but not quite because now you need to decide which comes first in the merge

micha23:08:37

and how does that merge with the .cljs.edn file option

micha23:08:38

and so on

micha23:08:57

the .cljs.edn way you control the order in which you compose your tasks in the pipeline

micha23:08:09

so that is trivial to arrange properly

micha23:08:39

and it's not mutating anything global, so you don't need to worry about how the merge will affect things when they're applied over themselves

micha23:08:35

you can use the :id method of selecting which .cljs.edn file to operate on, like the reload and cljs-repl tasks do

micha23:08:55

so your pipeline would be like

micha23:08:28

(deftask doit
  []
  (comp
    (task1)
    (cljs-compiler-options :update #(update-in % [:closure-defines] into ['foo.bar "asdf"]))
    (taskN)
    (cljs)))

micha23:08:41

something like that

kenny23:08:54

Just to make sure I understand.. In the general case you are suggesting that the way you update task settings is by creating a custom file (e.g. .cljs.edn) that is understood by your task, do-stuff. If task1 or taskN wants to modify the settings for do-stuff before the do-stuff task is ran they would need to update the task specific file.

kenny23:08:42

And this method is preferred because then you do not need to worry about the ordering of how certain tasks update/merge task settings

micha23:08:50

pretty much

micha23:08:10

and also when you have one task that will do different things for different artifacts

micha23:08:21

most artifacts have some sort of manifest that describes them

micha23:08:29

and you can use that manifest to build the artifact

micha23:08:54

so for jars and maven we already have a .cljs.edn thing, the pom.xml

micha23:08:03

and for war files we have the web.xml

micha23:08:19

.cljs.edn is trying to be that, but for cljs webapps

kenny23:08:39

But then isn't allowing tasks to specify options inline (e.g. (task1 :some-option true)) going against that method? Because then you need to think about how your task-specific file is going to get merged with the options specified inline on the task.

micha23:08:37

yeah that's why the options should be distinct from each other

micha23:08:00

it's not like that in the cljs task because the whole .cljs.edn thing is different from what people expect

micha23:08:19

so there is overlap to make the task useful without it

micha23:08:26

it generates a default one for you if you don't make one

micha23:08:19

like the install task looks for a pom.xml in the jar file it's installing into local maven

micha23:08:36

but if you specify options to the task then those override the pom and it doesn't look for it

micha23:08:44

so it doesn't need to merge anything

micha23:08:33

but i think with the cljs task :compiler-options is an escape hatch

micha23:08:38

so things are going to be weird

micha23:08:01

the ecma5 thing maybe should be a cljs task option itself

micha23:08:50

you only run into the kinds of problems you're seeing when there is no real model for what we're doing

micha23:08:07

like arbitrary maps of things are extremely rare in boot tasks

micha23:08:29

usually we factor it out well enough so there are only a few options

micha23:08:02

the compiler options are tricky because they're coupled to each other in ways that are not easy to understand

micha23:08:51

there is a lot of work put into factoring the options to separate concerns

micha23:08:04

so we can generate the :compiler-options that work

micha23:08:22

the boot-cljs task has a whole namespace for processing that

kenny23:08:48

Agreed. I believe we are on the same page now -- I just happened to pick an edge case for my leading example which caused a tangent, but very insightful, discussion. The conclusion I have drawn is that tasks will generally not have a complex map for specifying options. If the task does need a map to specify options it should be set in a task-specific file that can be overridden by task-options. If the task-specific file is overridden by task options, it should be completely overridden so you don't introduce any additional complexity of merging.

micha23:08:32

precisely!

micha23:08:20

so like in your case maybe just do your :compiler-options configs in .cljs.edn files and not in the task options

micha23:08:36

nothing in boot will enforce that, but you could just do it that way

micha23:08:56

then you'd have very fine-grained control

kenny23:08:28

It seems like there could be a general task created to do this sort of thing. You specify your file and operation and the task would do the rest

micha23:08:45

yeah update nested map things

micha23:08:49

inthe fileset

micha23:08:13

a good thing to have in boot built-ins

micha23:08:57

could be something similar to the sift task maybe

kenny23:08:57

All you would need to specify is :file and :fn

kenny23:08:09

I guess you wouldn't be able to use it from the command line

kenny23:08:33

or maybe you would have to pass in a symbol to the function or something

micha23:08:43

yeah that sounds reasonable to me

micha23:08:55

and now boot tasks can take positional parameters

micha23:08:57

so maybe like

micha23:08:37

boot doit --match-files '\.cljs\.edn$' --expr '#(update-in % [:foo :bar] conj :baz)' 

micha23:08:14

i guess it doesn't need positional parameters lol

micha23:08:23

but it could work fine from the cli too

micha23:08:59

maybe specter has a dsl that works with the cli

micha23:08:16

i haven't used it much, but maybe there is something that matchesup with the cli there

kenny23:08:56

Not sure why specter is needed here

micha23:08:38

oh probably not

kenny23:08:48

What would it be replacing?

micha23:08:16

but like the way clojure.core/update-in unrolls the sexps into a sort of concatenative expression

micha23:08:36

like (-> foo (update-in [:bar] conj :baz))

micha23:08:49

i could see that working out nicely with a command line

micha23:08:35

i was thinking maybe specter has some unique possibilities for making nested data updates concatenative like that

kenny23:08:49

Yeah totally. If someone want's to use it they could just require it in their build.boot and use it as they please. Not worth introducing a new dependency

micha23:08:24

although it wouldn't necessarily introduce a dependency, because it would be loaded in the worker pod

micha23:08:43

so it wouldn't be a dependency introduced to the project or anything

micha23:08:51

overly complicated probably

kenny23:08:17

I think if you get to a case where you want specter to modify your task-specific options file then you probably need to rethink your task

kenny23:08:06

Clojure's default navigators should be more than sufficient. And, as you said, it is definitely the exception when a task needs complex config options.

micha23:08:25

yeah with more work we could factor all the compiler options into a cohesive whole

micha23:08:34

and then we'd have simple options to the cljs task

micha23:08:45

it's just a lot of work 馃檪

kenny23:08:56

Anyway, I'll be writing a task like the one discussed earlier which can certainly be used as a template/starting-place for a future built-in task.

kenny23:08:38

BTW, I was going to ask earlier but didn't want to distract from the conversation, you said you use a macro to set :closure-defines. What is the use-case for that? Right now I am just setting them using :compiler-options on the cljs task in my prod build task (obviously that isn't preferred after our discussion).

micha23:08:33

the main thing i use it for is to bring configuration settings that i make at compile time into my cljs application

micha23:08:53

i have a macro that can read the values from System/getProperty or System/getenv

micha23:08:14

also for things like logging

micha23:08:43

i have a logging macro that does a bunch of stuff but doesn't emit anything unless the log level is set in the environment or system properties when you compile

micha23:08:59

so when i compile for production i set the log level to none

micha23:08:09

and for dev to debug or whatever

micha23:08:41

imo it's superior to the compiler options, because now it's in a place where you're actually computing

micha23:08:54

not as a giant case statement jammed into the gcl compiler

micha23:08:28

so i can have compile-time constants that depend on each other in different ways for instance

micha23:08:40

with all that logic encapsulated in my cljs application

micha23:08:44

not in the build settings