Can I ask a question about bootlaces here?

@talexxx yes

I wrote my first package-worthy CLJS project, but I’m stumbling at the deploy process: https://github.com/adzerk-oss/bootlaces/issues/15

Someone suggested I make a ticket in #clojurescript, but if it’s something I can reason through I’d be happy to work it out

@talexxx can you please paste your build.boot?

it looks like you have overlapping :source-paths and :resource-paths or something

like for example:

(set-env!
  :source-paths #{"foo"}
  :resource-paths #{"foo/bar"})

in this case :resource-paths and :source-paths overlap, that is to say foo/bar directory is trying to be added to both

@seancorfield: Trying to get boot-new working but it seems it overwrites my file everytime due to the watch task, starts an infinite loop of appending

What exactly are you trying to do @flyboarder ?

@seancorfield: generate a namespace with version information

so it rebuilds the ns everytime, but this starts an infinite loop

https://github.com/flyboarder/boot-semver/blob/master/src/degree9/boot_semver.clj#L120-Lundefined

Wait... you're using watch to kick off the generator?

So every time the files change, you run the generator... which changes the files, so watch runs the generator... which changes the files...

Seems like that's exactly the expected behavior...?

except that generators are supposed to be “safe” as per the docs

@seancorfield: ^

so should they not write more than once

Not sure what you mean by that?

I was under the impression that the generators need force option to rerun if the file already exists

For the ns generator... it only creates a file if it doesn't exist.

If your generator appends stuff, it's going to append it every time it runs.

Unless you make the generator read the file and only add the new definition if it doesn't exist.

The generators don't overwrite a file, they just append to it.

I can clarify that in the readme 🙂

Ok, so I will need to parse the file somehow then and selectively update it

Is there a way to delay making the changes to the fileset, I would think running a generator would be like a compile task

There, see if the updated readme is clearer?

The generator does exactly what you program it to do.

The built-in ns creates a new file if and only if it does not exist (unless you force it). The built-in defn always appends a new defn form.

If you write a generator that parses an existing file and modifies it if necessary then... it will do exactly that...

right makes sense, I poorly assumed watch wouldn’t be called again by tasks that follow it

Yeah, the boot pipeline catches me out fairly often 😐

I wonder how things like cljs generate files without causing watch to fire again

They don't generate source files...

(`watch` is looking for source file changes)

@seancorfield: thanks so much, answers all my questions!

yeah if you write to the project directories you're escaping all the pipeline affordances

the pipeline is built around the fileset, which does have the expected behavior

the project directories are completely outside the scope of boot pipeline stuff

if you modify them in a task boot can't distinguish that from user interaction

why does boot-new write to project directories?

does :compiler-options {:preloads '[devtools.preload]} not work with boot-cljs?

i see no indication it’s inside the out build, and the preload module never seems to be loaded

@lwhorton: It works fine

@micha I think because it's designed to create projects.

@dominicm wouldn't it emit the project into the target directory?

via the fileset?

@lwhorton: Are you defining this in build.boot or .cljs.edn?

(def devtools-config {:devtools/config {:features-to-install [:formatters :hints :async] }})

...
    (cljs :compiler-options {
                             :preloads '[devtools.preload]
                             :external-config (merge {} devtools-config)
...

build.boot^

@micha: General usage is boot -d seancorfield/boot-new new -t app -n myapp where myapp is the folder to write to. So it possibly subverts the fileset

interesting

seems like emitting files would be what the target task would do normally

(target :no-clean true) allows you to dump files into a directory already populated with things, too, btw

https://github.com/seancorfield/boot-new/blob/master/src/boot/new_helpers.clj#L86 I chuckled

@juhoteperi does it belong in build.boot or a cljs.edn file?

@lwhorton: Both should work

Options in .cljs.edn compiler-options will overwrite options from task options

But if you haven't set preloads etc. in both it should woerk

do I have to define a boot.cljs.edn {:requires [boot.core] :init-fns [boot.core/enable-logger] }? that seems to be what the preload modules is for:https://github.com/binaryage/cljs-devtools/blob/master/src/lib/devtools/preload.cljs. I assumed just including this as a preload would automatically invoke (core/install!) before my main file was loaded?

you can add -v option to Boot to get Boot-cljs write the final Cljs options out

but that devtools.preload module never shows up in the build, I can’t search for it in devtools, and core/install! is never called…hmm

https://github.com/Deraen/saapas/blob/master/src/cljs/js/main.cljs.edn <- this is working

so peculiar ..

i’ve tried every which combination and nothing wants to work. i wonder if Im missing a ’ somewhere 😕

@micha if your curiosity is like mine, I found the relevant bit of code suggesting that it does indeed output outside the fileset (makes sense for it's purpose) https://github.com/seancorfield/boot-new/blob/master/src/boot/new/templates.clj#L163

@dominicm i don't see it, why not just write to a dir and add to fileset?

then use target task to write to final output dir

like why would the dir it writes to not be a temp dir?

@micha: Probably need to defer to @seancorfield for specifics. Might just have been a conception that the fileset is just for temporary files, not things you want to persist.

@lwhorton: I tested and managed to get arbitrary option from task option :compiler-options to show up on debug output enabled by -v

@micha because when you run boot-new you don't have a project: you are creating an entire file system structure and populating it with new files.

right but wouldn't the target dir be the resulting project dir, basically?

it's going to write to some directory, just like the target task does

(How come this discussion is only coming up now?)

sorry i didn't mean to complicate things 🙂

disregard me

how do you feel about rolling boot-new into boot for the 2.7.0 release, by the way?

Actually I think we did have this discussion.. And the consensus was boot-new correctly operates outside the pipeline because it is meant to be a one-off task used to create a project. No one gave a compelling reason for it to use the boot file system idiom at the time. But I guess we could revisit that ... ?

it would make it composable

with other tasks

For boot-new to use target to write the actual file tree, it would have to have control over the target folder name and whether to overwrite the directory or not

and the watch task etc. of course

you'd do that with your target task

doesn't seem bad to separate those concerns really

That would mean you'd need more command line arguments - it's already harder to use than lein new

we could make boot-new an option to boot itself

then it could do some stuff like add the target task and so on

and with the 2.7.0 feature of positional arguments to tasks we can clean up a bit maybe

like boot --new ...

but possibly allowing you to add more tasks to the pipeline or something

with target task schtupped onto the end

I am doubtful on benefit of composing boot-new tasks.

yeah it's a hard problem

Project template is already so complex problem that I don't see how several tasks could produce something which works together

And using boot-new tasks with watch task doesn't sound very useful

also that's what clojure is doing in our program

like we generate code with macros

not with templates

so yeah disregard please 🙂

when templates get too fancy that means we should probably look at the code

like the hoplon template is really just to generate a list of dependencies

@micha so what about for my use case, where I want a namespace generated on every run?

https://github.com/flyboarder/boot-semver/blob/master/src/degree9/boot_semver.clj#L120

That seems like It should work, if im using generators wouldnt it make sense to have a task which runs a bunch of generators in a safe way, template might be another story as you probably only want that once

there must be a better way to do that

writing to the project inside the pipeline

one option is to write to a file that's not in the watched directories

then generate the thing in the fileset, the VERSION file, from that

I guess I could have the generator write things to a tmp dir

but then i need access to the target output when I call the generator

why not just print the version.properties to stdout

boot version -x inc > version.properties

that preserves the boot way of leaving changes to project files with the user

so you don't end up with weird situations to debug

@micha the version.properties file isnt the issue, people are wanting a namespace to call in their project code that has the version information in it, im looking to statically generate this namespace

you can generate that in the fileset, no?

yes but I was under the impression that is what the boot-new generators are for

the alternative is to have a edn file in the project instead of version.properties

and just edit that

(sorry, had to step away in the middle of the boot new chat)

The thing with boot-new is that it’s usually generating a project.clj or build.boot file so I’m not sure how you’d really compose it with other tasks — you’d be in the wrong directory to start with (you’d normally start in a folder with no build.boot file — would you expect Boot to change down into the newly created folder and then start over with loading that new folder’s boot.properties and build.boot file?).

maybe it would be cool if it took an argument that put the generated project on the fileset? for dev-ing templates

that seems like the uncommon case, but a useful one to support, vs the common case of someone pulling it in and running at the CLI and not composing with any other tasks

actually that would be extremely usefue for developing templates

@seancorfield: I think your example is correct if you are generating a new project from a template, but in the case of a generator I think they should be able to work with existing projects also

so that you can compose them

are there any examples of composing generators?

@juhoteperi made a good point there i think

but like when i go to a directory and type boot new whatever --name foop and it creates a directory named foop with stuff in it, that is the same as boot some-task target -d foop

Perhaps there will be at least two kinds of generators? Those setting up new projects with boot-new and some that add files to project fileset?

or even project source-paths, I haven't needed that myself but maybe it would be useful some times

Well, the generator can do anything

So yes it absolutely could write to the boot fs abstraction etc

The examples just happen to create and modify source files

Because the lein project it was based on did exactly that

@seancorfield: I can just pass the filesystem as an arg to the generator tho correct?

If you’re generating a new source file (like the ns generator), how would that sit with a target task?

@flyboarder: currently the generate-code helper is not passed the fs argument from the base task… and current generators assume they get passed a prefix and "some args"

There would need to be some re-plumbing done in order to be able to pass the fs down into the (user-provided) generator...

I tried using the prefix and passing it a tmp-dir! path but the generator throws an error about relative paths

so that didnt work 😛

I feel like saying "of course"...

yeah XD

The task assumes pass-thru and does not let the generator do anything with the file system abstraction.

Again tho’, this comes back to the boot new / generate task being specifically for creating / modifying source files tho’...

And if you modify source files, and you have a watch task, then it is going to trigger because the source files changed.

right it all makes sense when it’s broken down, just not the expected behaviour compared to other boot tasks

Boot new is "not like other boot tasks" tho’...

right, and I think that would be a bit more clear if it gets pulled into boot 2.7 as an argument?

It is operating "outside the system" because it is (primarily) intended to create a project in which to run Boot...

Entirely possible. We discussed it before and @micha didn’t want to roll it into 2.6.0

boot —new I wouldnt expect that to work like a task

I restructured the source code to match Boot’s internal task structure in preparation for it being PR’d in — I even sent a PR 🙂

we can do it for 2.7.0 i think

yeah this is really just me assuming too much about the task behaviour

seems like a good time for it

I don’t know what it would have to look like to become a new command line argument (but having positional arguments would make it nicer to use!)

just bear in mind that there are quite a few modifying arguments (look for snapshots or not, etc) as well as the ability to pass arguments down to the template itself

I’m more than happy for the code to be included in Boot 2.7.0 is whatever form it needs to be.

I’m still not quite sure how you’d deal with the issue that it (normally) generates a new folder tree containing new source etc files — would the expectation be that the rest of the Boot pipeline ran in the context of that newly generated project?

Doesn’t Boot load boot.properties etc from the current directory before it runs any tasks?

i'd think we can have it do what it does now

What is the expectation if you’re using, say Clojure 1.8.0 by default and you boot —new a project based on a template that uses Clojure 1.9.0 or Clojure 1.7.0?

How would that compose?

it could work the same as it works now, no?

composition seems not that useful

like for the hoplon template, for example, i think the generator should just create a build.boot file, and that's it. dependencies plus some tasks

the demo app would be in a jar dependency

rather than generating source files

^ how I saw this in my head was being able to call the hoplon template as someother app template and build off of it

yeah that's really complicated though

think chef

but thats kinda what the filesystem does anyway

how so?

well a task modifies the filesystem and the next task builds off that filesystem

the fileset yeah

sorry fileset*

but like modifying source files is not really practical

right

or imho useful

we want to make better abstractions, not generate source

if we need to generate source files there's something wrong

still though ti would be possible if the generator output went into the fileset

the reason i love the idea is for template dev, i could use together with checkouts to live-reload code/update templates

like what would be an example of using it?

parsing source files and trying to infer stuff about them then modify them

that seems unsound

like as i'm making changes to the hoplon template, a generated project is continuously populating the target dir

we should make macros then

better ones

that seems to be the composable way to generate code

composable-ish

@micha are you responding to me or @flyboarder ?

^ now im confused isnt that a task?

@alan you lol

i mean, a template is a program that makes some text files

all i want to do is see my text files update continuously

which i could do if new could optionally write to the fileset, then its output becomes like the output of any other compile step

ok but they're not just text files

they're programs

you're going to be generating code

i just don't see that being workable at all

thats what the generators are for tho, generating code (templates)

so you think there is no value in having eg a hoplon simple template?

yeah kind of

i imagine the template generating a build.boot only

with dependencies and some tasks

the demo app being in a jar in clojars

that's what it does already

well currently it generates namespaces

which is where it gets dicey

liek how do you compose namespaces?

you don't

it shows people where they go

it's a starting point for a human, not a computer

it's a technical document generator basically, these templates

i agree that they are most valuable and good when they are small and show only the most important things

i can see that

like the things you compose are really orthogonal

they don't touch the same files

vs. the current trend toward massive ones that should be smaller and mostly library code that lives elsewhere

like the files generated by taskA and taskB are disjoint

if two generators modify the same source file, that's where i get skeptical

me too lol

but i don't object to other people doing it

for me the fileset just closes the template dev loop

i concur

if we make new a boot option we could make it pretty good, usability-wise

boot --new ...

@micha if people want to modify or compose a namespace or code itself that's what zippers are for

the program using the zipper can't understand the program though

all it sees is sexps

so yes it's trivial to generate sexps, i just don't see it being that useful to build programs that way

but thats fine since it should be at the task level

there will always be tons of edge cases

if it sees a namespace that already has stuff in it, it can't know how to correctly modify it

like boot —new shouldnt really worry about that

sure, it doesn't

the templates that boot new generates should only be a onetime thing right?

so its like the base starting point for the fs, then the project comes on top of that

so maybe we should have a new file type of :template

that would allow that layer to be swapped out by checkouts

i don't understand what the problem is we're trying to solve anymore

i thought you just use it to create some files

then you stop using it

and alan suggested it be dynamic

in that case you don't need composition, you can just run it once for each egnerator

so when a template changed the project was also updated

like if you're working on the template

right and i have a running app based on the template

the app would also update

i don't understand that

i guess you'd use boot-in-boot then

to runa new boot isntance or something

we should make a wiki page to decide what the functionality should be

i myself don't really use templates to generate projects, so i'm not really qualified to say what goes into boot for that

@micha I didn’t want to clobber the chat buffer. My build.boot is at https://github.com/talexand/hallo-color/blob/master/build.boot

@talexxx ah i see

bootlaces does a dumb thing, it adds "src" to :resource-paths by default

there is an option to disable that behavior

we should really make that the default, just haven't gotten around to it

Searching the boot docs for that option, but coming up empty

it's in bootlaces

it's the bootlaces function that's the problem

Ah

Am I biffing the function call? (bootlaces! +version+ {:dont-modify-paths? true})

java.lang.IllegalArgumentException: No value supplied for key: {:dont-modify-paths? true}

@talexxx look at the bootlaces function docstring

Found it in another repo

the best way to debug it is to look at the docstring though

That’s true 😄

whenever something like that happens

a lot quicker to find the problem 🙂

Isn’t the docstring supposed to be in quotes after the function signature?

https://github.com/adzerk-oss/bootlaces/blob/master/src/adzerk/bootlaces.clj#L22 has nothing like that

if you do (doc adzerk.bootlaces/bootlaces) in the repl you'll see the arglist

your issue is the & {:keys ...} part

Am I in the wrong namespace?

boot.user=> (doc adzerk.bootlaces/bootlaces)
nil

sorry, it's adzerk.bootlaces/bootlaces!

with a bang on it

basically you want (bootlaces! +version+ :dont-modify-paths? true)

Ah I should have caught that

ups wrong link

https://twitter.com/roman01la/status/767466809746419712

I wonder if a port to JavaScript would be that hard

whoa

yeah 😉 tooling should definitely go the JavaScript way

haha

I mean, it would be super fast to launch

the only part that we can't convert is the repl I guess, but a java -cp .... can be super quick

i dunno if it would be faster in js

it's not the jvm that's the problem

it's the dynamic nature of clojure

well if would speed up everything else (except the repl yes)

boot startup time as well

you would have to give up vars

boot uses vars for many things

yes you'd need some clojurescript-in-clojurescript for that, and no multi-threading I guess

if you have cljs-in-cljs and it provides the affordances of clojure, i think you'd be back where you started

just in nashorn instead of directly on the jvm

or in v8

yeah, planck does a very good job with cljs-in-cljs (but of course I haven't profiled anything)

just wondering actually what could be achieved and if it is worth it

(had a meeting… back to boot-new…)

So, it sounds like there are three distinct use cases, two of which boot-new currently covers: generate a whole new project as a one-off; generate or modify some source code (as a starter for a developer, e.g., new controller in an MVC app, new source file / namespace, new function stub, new test, etc); then there’s the "generate stuff into the fileset as part of a composable build pipeline".

That last case is not currently covered but could possibly be a valid thing for a generator to do — but not in the current form because the fileset is not passed into a generator and is not returned from the new task itself.

the generator wouldn't know or care about the fileset necessarily

So, if someone wants that to be possible within boot-new then go create a GitHub issue for it. I think it’s an interesting idea but I don’t need it. If someone writes up what they want and provides guidance on how to do it, I’m happen to implement it.

it just gets a tempdir

and writes to a tempdir

Right, but there’s no communication with the fs entity possible right now.

Go look at the source to understand what I’m saying.

The generator code does not do anything with the fs — it’s not used, not passed in. So a generator can’t write anything to the fileset as the code stands today.

I’m not against enabling that — but I wouldn’t want to break any existing template / text-based generators.

As for folding boot-new into Boot itself in some form, happy to work with @micha or whoever on what is needed there. In its current form, it’s easy to merge in as a built-in task (per the old PR — I think I could fairly easily create a new PR that followed that outline) but I’m not sure what additional work would be needed to wire it in as a —new option to Boot itself (because of all the additional options that the new task currently expects / requires)?

(I’m also not sure what would happen to the existing generator functionality in such a model?)

Re: three use cases: I don't think they need to be solved by single task (boot-new), the third one can be solved by something else

^agreed, i think the existing version works well for it’s intended use now that it has been explained, it seems like a good fit as an argument for a future version, i was trying to figure out where in boot the actual behaviour of the arguments is run

can someone help explain why Boot fails to compile this namespace? https://github.com/zcaudate/one.love/blob/master/src/one/love/command/ast.clj

@myguidingstar: what is the error you are getting?

my guess is that it has problem with finding path and grepping for .class file manually. That will only work in Lein

@flyboarder Error compiling one/love/command/ast.clj

line number?

(54:1)

i'd comment things out until you can require the namespace in the repl

because there is no useful information about what could be going wrong

all you can do is bisect the problem

line 54 appears to be an empty line

sorry, it was 64:1. Actually I only use one.love as a library and didn't use Boot to check its source

the use of ~'args etc looks odd

usually you'd use a gensym there

to avoid name clashes

like args# instead

looks like the author wants deterministic macro expansion 🙂

ah i see

he's trynig to find those java classes by looking for .class files in classpath (line #27). Is there a better way for that?

yeah line 17 is the problem there

a better way would be to get the classpath from a classloader, not from the system property

but then you need to choose which classloader, etc

you can probably prime the pump if you want

just get the classpath from boot and set java.class.path or something

you mean in Boot-powered projects that use the library, or shoud I change upstream source code?

oh this is your project?

no, it isn't

just using it as a library

i don't really understand all the ins and outs of what is going on there

but setting java.class.path seems like a good start to get your project to work with it

well I want to use this library. It is reported to work in Lein-powered projects and not Boot ones

right because boot manipulates the claspath at runtime

lein starts a second jvm with java.class.path set with a value computed by the first jvm

ideally you'd be able to provide the locations of things on the classpath to the library maybe

like in boot it's very easy to get the jar file for a dependency

without regexes 🙂

do you have any suggestion for the approach to fix in upstream (as a library)?

would it be possible to have a function that you'd call to initialize it?

i guess the issue there is that it needs to find the rethinkdb-driver jar

on the classpath

it uses the java.class.path plus regex to avoid scanning the classpath

if you know the name of a class in the jar you could use reflection

(lemme research a minute, I never get a thing about java class...)

you would just need to know the name of one class in the jar

oh you have one

com.rethinkdb.gen.Datum

so you can do like

(.getClassloader com.rethinkdb.gen.Datum)

to get the classloader that has its jar loaded

then to get the path to the jar on the filesystem you can do

(.getResource cl "com/rethinkdb/gen/Datum.class")

where cl is the classloader

that will return a URL, which you can then string munge on the path to get the jar

I see. Thanks a lot

you probably will need to convert the path to a java Path object to get platform correct path

because URLs don't use backslashes

i guess +rethink-snippet+ probably doesn't work on windows anyway, so it can't be worse

hmm, that rethink-ast function is called only once to derive a constant one.love.command.ast/classes whose value is a map of pairs like {:set-insert com.rethinkdb.gen.ast.SetInsert, :to-geojson com.rethinkdb.gen.ast.ToGeojson ...}

I guess we can just hardcode the list [SetInsert ToGeojson ...]

b/c maven version is fixed

the original dynamic function can be kept and use when upstream upgrade

(kept as comment)

or as a boot task

you can generate an edn file or whatever

package it in the upstream library jar

then get it with like ( "one/love/thingy.edn")

haha, the boot way 🙂

gotta sleep now

thanks a lot

np!

good luck 🙂