Fork me on GitHub
#boot
<
2016-03-15
>
richiardiandrea00:03:46

from inside the webjar? sorry that's why I was asking, I am not very familiar with webjar and I should have googled it 😉

juhoteperi00:03:57

There are no externs in webjars. Extern only exist in Cljsjs.

adamfrey03:03:26

there’s no way to see what options were passed to a task from within another task, right? Specifically knowing where the target task is outputting the fileset?

fenton04:03:47

is it possible to load and use code from inside your project inside the build.boot file? I'm having issues requiring code and then using it.

alandipert05:03:51

definitely possible

alandipert05:03:38

what issues are you having?

martinklepsch08:03:18

@adamfrey: there are ways to work around this but generally this introduces hidden coupling between tasks. Whats the motivation?

be911:03:55

is there a way to detect from my Clojure code if it's being run from boot in development mode or in production from uberjar?

be911:03:19

I have a command-line app that uses (System/exit ...) which is really bad for repl, so I want to use it only in production

dm311:03:29

does it make sense to split the code so that System/exit is only ever called when going through -main and -main is never called in development?

dm311:03:12

otherwise you can alter the fn that would delegate to System/exit in build.boot

be911:03:31

well, isolating -main is probably easier

dm311:03:00

another option is to throw some sort of exception that will only get caught in -main and trigger exit

dm311:03:13

won't work if multithreaded

be911:03:59

thanks @dm3, I think I just torture -main a bit

micha13:03:04

i use environment variables to hold that kind of configuration, like dev vs production environment etc

micha13:03:11

when i deploy to production i have an upstart script that launches the appliction and that's where the production environment variable setup happens

micha13:03:41

and i have scripts that can set up a shell with dev environment via source or eval in bash

dm313:03:34

I understood this was about a command-line app where dev/prod doesn't really make sense

adamfrey13:03:52

@martinklepsch: I’m adding a boot task to James Reeves’ codox library, and I was trying to match the lein plugin’s behavior of printing a message that lists the absolute path where the docs are generated. But James said that it was ok that the boot version prints a relative path, so I’m all good.

martinklepsch13:03:30

nice @adamfrey looking forward to giving it a try

juhoteperi16:03:42

@adamfrey: There is a boot task for codeina (which is codox fork): https://github.com/funcool/boot-codeina should be helpful to check how it has beem implemented

hlolli16:03:42

Im experimenting with making production version of my boot cljs project with "boot cljs -O advanced -- target", only thing missing is sass stylesheets. Any way to compile sass along into target/css ? For sass Im using [mathias/boot-sassc "0.1.5"]

hlolli16:03:34

ok nice, thanks

nha16:03:29

And an example here : https://github.com/Deraen/saapas/blob/master/build.boot#L80 I never used mathias/boot-sassc though so I can't compare

hlolli16:03:02

ahh ok, this is a different sass dependancy for boot. Well, I think what's standing in my way is that Im first time actually deploying something vs always developing. I think I need to read alot now. I notice differences from lein where a target directory was created with all the *js files ready to be lunched without serving the directory. (but my memory could be wrong) (sorry js not java)

martinklepsch16:03:14

@hlolli: are you developing a CLJ+CLJS app?

martinklepsch16:03:25

(as in a single project/build?)

hlolli16:03:02

no, it's all pure clojurescript built on tenzing actually. I see there is a production deftask, cant seem to wrap my head around how to use it.

martinklepsch16:03:30

boot production build

martinklepsch16:03:58

essentially the production task acts like a profile setting all the parameters required for production builds

hlolli16:03:10

ok, I wonder where the target directory is? Maybe I need to supply it with -- target ? I do boot production build but I see no files generated.

alandipert16:03:37

@seancorfield: hey, how would you feel about moving boot-new to the boot groupId instead of making it a builtin?

martinklepsch16:03:02

yeah, try boot production build target I usually have some task that syncs things to S3 or similar

martinklepsch16:03:30

@hlolli: if you're deploying a static site, checkout this: https://github.com/confetti-clj/confetti

adamfrey16:03:00

@juhoteperi: thanks! I didn’t know about that

dominicm16:03:31

is s3 free at low tiers or something?

juhoteperi16:03:22

@dominicm: For first year and for limited resources (but the limits are quite high)

seancorfield16:03:34

@alandipert: I’m fine with that but it’s still going to be a bit of a mouthful to type boot -d boot/boot-new new -t app -n myapp compared to boot new -t app -n myapp yes?

alandipert16:03:10

@seancorfield: i was thinking boot -d boot/new new, but yeah it will still be slightly longer

dominicm16:03:32

@seancorfield: Can't people just copy a snippet to their boot.profile?

juhoteperi16:03:21

Maybe boot-new could be maintained in separate repo and artifact but the task could be included in core?

juhoteperi16:03:36

Would make it easier to develop boot-new separately but would be easy to use for everyone

seancorfield16:03:44

How about if the task itself moved to Boot core but all the built-in templates and machinery stayed in boot/new and that dependency was brought in at run time?

alandipert16:03:55

the reason for waiting is just that new is just such a juicy builtin (and project task name), and micha and i talked ourselves into thinking that it might be good to wait and see a bit longer if something else makes more sense to go there

alandipert16:03:31

i think our reservations are less about where the code lives and more about the new builtin

seancorfield16:03:01

Agreed, which was why I was initially holding off wanting to integrate it — but with 2.6.0 coming up faster than I expected, it seemed like a good opportunity… and I think it was @micha who’d originally asked if I planned to send a PR 😸

alandipert16:03:02

i could still go the other way, just wanted to discuss

micha16:03:49

i think it might be good to let it percolate for a little while longer before fully integrating it, so it won't be too disruptive to make changes

seancorfield16:03:02

I’m happy to continue to maintain it separately tho’ if that’s what you would prefer… and making it boot/new rather than seancorfield/boot-new would be fine if I continue to maintain it there.

alandipert16:03:25

informing my thinking is making a new project is relatively rare compared to the other tasks, and that we will live with the builtins for probably decades

seancorfield16:03:59

Indeed. And lein new was lein-newnew as a plugin for ages before it was integrated.

micha16:03:14

perhaps a prominent note in the boot README about how to add it to the profile

micha16:03:43

so that we get a significant percentage of users who would use it to check it out and provide feedback?

micha16:03:14

or perhaps an addon type of thing that would include it

micha16:03:26

like a separate module for things we're considering adding to the core

micha16:03:39

but is versioned separately

micha17:03:01

that might have its own issues though

seancorfield17:03:18

Strangely I hadn’t even thought of adding it to profile.boot — because I didn’t want to drag in the dependency on every boot call simple_smile

micha17:03:47

you could make a new task in build.boot that pulls it in, perhaps

micha17:03:01

i mean in profile.boot

seancorfield17:03:56

Indirection solves everything simple_smile

seancorfield17:03:50

You’d have to duplicate all the argument processing tho’ … that’s quite a lot of code …

seancorfield17:03:58

It would be more convenient / easier to add a boot-new shell script that invoked boot -d seancorfield/boot-new new "[email protected]" or something...

alandipert17:03:21

alias n="boot -d seancorfield/boot-new new"

alandipert17:03:13

or i guess we could distribute multiple shell scripts that are different entrypoints into the app jar... but then we'd need a real installer

seancorfield17:03:59

(! 1157)-> cat ~/bin/boot-new 
#!/bin/sh
boot -d seancorfield/boot-new new "[email protected]"
OK, I’m happy now simple_smile

seancorfield17:03:31

(and, yeah, alias would work too but I don’t have any shell aliases at the moment … color me weird)

alandipert17:03:40

oh here's an idea

seancorfield17:03:59

Anyways, I’m happy to go whichever way you want at this point. I restructured the boot-new code to be easier to integrate and that’s an improvement anyway. I’m happy to maintain it under my user or move it to the boot org and maintain it there or… whatever…

alandipert17:03:00

so probably most new users obtain via homebrew? maybe we could add the stub script as part of the brew formula

alandipert17:03:44

@seancorfield: cool. thanks for making and contributing, it's super cool we don't rely on lein anymore

seancorfield17:03:41

Yeah, I’m using Boot for FW/1 stuff now as well which is nicer than Leiningen. And I’m starting to use it for generators in other projects too.

seancorfield17:03:55

Since you can just add a boot.generate.whatever namespace to your project for any new thing you want to generate simple_smile

fenton17:03:59

is it possible to load and use code from inside your project inside the build.boot file? I'm having issues requiring code and then using it. What I'm trying to do ultimately is support modern deep linking in a cljs application. it works fine in production but for testing i think i need to have a custom ring handler for the serve task, so urls like: http://host/blah/blah can be mapped to http://host/index.html

martinklepsch17:03:09

@hlolli: any success with the deploying?

fenton17:03:37

and i'd like to implement the ring handler in the current project versus build it somewhere out of context.

martinklepsch17:03:53

@fenton: if you're using boot-http you can have a custom handler that you supply as a fully namepspaced symbol

hlolli17:03:21

yes this worked! Just the way I wanted it. I will upload it tomorrow to a server, I just used python simple http to confirm. It still needs few last-minute text fixes, but I just wanted to know this process today in case it would drag on tomorrow.

fenton17:03:48

@martinklepsch: i tried to fully namespace it, but still complains it cant find it.

fenton18:03:44

@martinklepsch: sorry this example has too much crap in it...i'll make a streamlined one later today to show my issue.

fenton18:03:57

@martinklepsch: does that symbol have to come from an external library or can it be simply defined in the current project.

fenton18:03:39

and more generically can boot use code that is defined in the current project, or only code that comes in through external libraries?

micha18:03:00

@fenton: are you referring to the handler symbol?

micha18:03:36

if so it can be any function that can be resolved from the classpath

micha18:03:51

so it can be in a jar or in a file that's on the classpath etc

dominicm18:03:15

pcbe-web/ring-handler is this what you mean?

fenton18:03:20

micha: okay. i was able to require it, but when i tried to use it...it complained it couldn't be found...i'll try out the resolve part.

dominicm18:03:42

Your stack says pcbe-web.ring-handler @fenton

fenton18:03:55

@dominicm: pcbe-web.ring-handler/handler

micha18:03:05

the serve task should require the namespace and then resolve the handler function

micha18:03:31

oh i see the issue

flyboarder18:03:34

@fenton: you may need o quote the symbol

dominicm18:03:45

@fenton: ah, I see. Yeah, ^^ these guys got it then

fenton18:03:00

hmm... tried that too

dominicm18:03:50

(resolve 'pcbe-web.ring-handler/handler) any luck with that?

fenton18:03:55

single quote gave same error #' gave:

╭─[email protected] ~/projects/pcbe-web  ‹ft-auto2*› 
╰─➤  boot dev                                                                                                                  1 ↵
clojure.lang.ExceptionInfo: Unable to resolve var: pcbe-web.ring-handler/handler in this context
    data: {:file "/tmp/boot.user5558579707800723701.clj", :line 27}
java.lang.RuntimeException: Unable to resolve var: pcbe-web.ring-handler/handler in this context
               ...                  
boot.main/-main/fn     main.clj: 192
   boot.main/-main     main.clj: 192
               ...                  
  boot.App.runBoot     App.java: 390
     boot.App.main     App.java: 467
               ...                  
  boot.Loader.main  Loader.java: 253
╭─[email protected] ~/projects/pcbe-web  ‹ft-auto2*› 
╰─➤                                               

flyboarder18:03:44

@fenton: what error did the single quote give you? Is the namespace in resources?

fenton18:03:45

resolve does work...

micha18:03:49

odd, you should get an error if the option argument isn't a symbol

fenton18:03:23

single quote seems to work now...

fenton18:03:31

at least not complain.

fenton18:03:51

sorry thought i had tried that.

micha18:03:53

you can see the expected option argument types in the repl

micha18:03:00

(doc serve)

micha18:03:05

it will show something lke this:

dominicm18:03:16

I mean, symbol isn't particularly clear as a descriptor imo.

micha18:03:23

boot.user=> (doc serve)
-------------------------
pandeiro.boot-http/serve
([& {:as *opts*, :keys [help dir handler init cleanup resource-root port httpkit silent reload nrepl not-found]}])
  Start a web server on localhost, serving resources and optionally a directory.
  Listens on port 3000 by default.
  
  Keyword Args:
    :help           bool  Print this help info.
    :dir            str   The directory to serve; created if doesn't exist.
    :handler        sym   The ring handler to serve.
    :init           sym   A function to run prior to starting the server.
    :cleanup        sym   A function to run after the server stops.
    :resource-root  str   The root prefix when serving resources from classpath
    :port           int   The port to listen on. (Default: 3000)
    :httpkit        bool  Use Http-kit server instead of Jetty
    :silent         bool  Silent-mode (don't output anything)
    :reload         bool  Reload modified namespaces on each request.
    :nrepl          edn   nREPL server parameters e.g. "{:port 3001, :bind "0.0.0.0"}"
    :not-found      sym   a ring handler for requested resources that aren't in your directory. Useful for pushState.
nil

micha18:03:56

the second column is the type of the argument

micha18:03:04

sym for the :handler option

fenton18:03:20

@micha okay that looks good....sorry i'm still a clojure newbie..

pandeiro18:03:10

Re: boot-new, nobody asked me but I don't miss that part of the lein ecosystem, both because scaffolding new projects is not such a common task (and I kind of enjoy it), and because there's such a limited range of extendability, basically all configuration possibilities need to be thought of in advance and added to the template.

kanwei18:03:32

I'm writing a task to "touch" a file if certain other files change

kanwei18:03:40

but the (touch) method doesn't work on tmpfiles

kanwei18:03:46

what's the best way to achieve this?

micha18:03:12

@kanwei: yeah you don't want to modify files in the fileset

micha18:03:17

they should be immutable

micha18:03:29

so what you need to do is actually copy the file

pandeiro18:03:34

What I do think would be useful is for framework/architecture tools (like luminus, for example), had generator commands like rails or yeoman to quickly scaffold a particular new view or model

kanwei18:03:35

I basically want to trigger an update downstream

micha18:03:40

and update the modtime on that, then add it back

dominicm18:03:08

@pandeiro: Someone i know has been working on something like templates, but architected around git branches & tags, and when you want to get template updates, you "just" merge with the master. It's an interesting idea, trying to figure out the conflicts is hard though.

kanwei18:03:10

(when-let [changed-sql (seq (->> (fileset-diff @fs-prev-state fileset) input-files (by-ext [".sql"])))] (println "Changed these files: " changed-sql) (doseq [db-file (by-name ["db.clj"] (input-files fileset))] (touch db-file)))

kanwei18:03:20

I should cp db-file?

micha18:03:44

hmm @kanwei no that won't do what you want, i don't think

micha18:03:52

how do you mean "trigger a change"

kanwei19:03:02

I'm using system+hugsql

pandeiro19:03:08

@dominicm I think plain git clone with example repos solves the 95% case

micha19:03:14

i think there must be a more straightforward way to achieve this without messing with files

kanwei19:03:20

I want to make it so everytime a .sql file is changed, the db.clj file is "updated" so that system reloads the namespaces

dominicm19:03:48

@pandeiro: That's pretty much what he's suggesting. With a couple branches for specific styles (e.g. webapp)

micha19:03:50

why not just reload the namespaces?

kanwei19:03:08

how do I do that? not really sure how to trigger that

kanwei19:03:18

system kind of just takes care of it

micha19:03:21

@kanwei: i mean just do it, don't trigger something else to do it

micha19:03:04

@kanwei: system has some kind of function you can call, no?

micha19:03:08

to reload something?

kanwei19:03:27

i would basically reimplement half of that

pandeiro19:03:28

@kanwei if it's an HTTP app I think ring-reload would handle it

kanwei19:03:03

@pandeiro: it doesn't, since .sql files are separate and loaded into macros

kanwei19:03:08

and you need to refresh the ns

micha19:03:51

if the .sql files were .clj files it would work, because you could require them in db.clj

micha19:03:01

and tools.namespace would then know about those dependencies

pandeiro19:03:14

Writing a touch task shouldn't be hard though right?

micha19:03:17

you're trying to shoehorn in some ad-hoc dependencies

kanwei19:03:27

that seems pretty hacky

micha19:03:44

having a clojure file depend on some text files for compilation is also hacky

micha19:03:57

because clojure has a pretty good dependency system already

micha19:03:09

so bolting on an ad-hoc one is also a hack

kanwei19:03:29

well, then hugsql and yesql would be hacks by your definition

kanwei19:03:51

there's benefits from just using .sql files, editor support, etc

micha19:03:04

yeah i mean the way i would do that is to generate clojure namespaces from the sql files

pandeiro19:03:07

I don't understand why the macros don't reload in that case

micha19:03:22

so the compiler then knows about the dependencies

kanwei19:03:26

@pandeiro: the macros do reload, it's just the code running already has the old macros

micha19:03:30

instead of having to bolt something on

dominicm19:03:40

@kanwei: Does the reloaded workflow not work?

dominicm19:03:05

I've also used ns-trackerto some effect with similar situations.

micha19:03:06

the problem is that yesql generates code in memory from .sql files

pandeiro19:03:11

@kanwei the macro then doesn't redefine the vars of the generated query functions?

micha19:03:17

so there is no way to express that dependency relationship in the clojure compiler

pandeiro19:03:23

It generates vars no?

dominicm19:03:24

Obviously it's not perfect, because multimethods, but still.

micha19:03:26

it's all just a convention of that one library

micha19:03:44

there is no way for tools.namespace to know that this macro depends on doing IO to read some text file

micha19:03:54

that's the core problem

kanwei19:03:05

right, which is why a manual trigger on a clj file is required

dominicm19:03:05

oh, derp, yes.

kanwei19:03:13

i mean i get it's not perfect and clean

micha19:03:18

so my solution would be to geenerate namespaces from the .sql files

micha19:03:24

then you can require them like normal

kanwei19:03:24

but tons of people are using it just fine and i'm trying to simplify it for boot

micha19:03:28

and everything works

micha19:03:33

you can have a boot task that generates them

dominicm19:03:37

OR, you could use the namespace's refresh-all, OR require them manually after changes to them..

micha19:03:44

this is how we do hoplon and so forth

micha19:03:02

but you still have the problem of expressing the dependency relationships in a sane way

pandeiro19:03:05

Sounds like something better thought of and handled as part of the build chain, not app code

micha19:03:32

it would be pretty strightforward to have a task that finds .sql files and creates namespaces from them

micha19:03:44

then in your db.clj you just require those

micha19:03:49

and all will work smoothly

pandeiro19:03:52

But that would essentially be rewriting yesql as a boot task

micha19:03:03

yesql is tiny

pandeiro19:03:24

OK @kanwei we've figured out the solution

micha19:03:29

your task could generate just one namespace from all the .sql files

micha19:03:37

with one function for each .sql file

micha19:03:44

then you just require that in db.clj

micha19:03:29

if you do want to update the mtime of a thing in the fileset you can do it by copying the file to a temp dir, updating the mtime, then adding that temp dir back to the fileset

micha19:03:58

subsequent tasks in the pipeline will see the updated mtime, but it won't affect your actual project files

micha19:03:26

so it'll reload twice

micha19:03:09

so i think at the end of the day you have 3 options:

micha19:03:45

1. make a task that generates clojure namespaces that you can require in db.clj, thereby using the existing clojure dependency system and tools.namespace for reloading

micha19:03:31

2. extend tools.namespace to accomodate arbitrary dependencies in addition to what clojure already provides, that is adding some kind of annotation to the namespace that declares the dependency on those .sql files

micha19:03:55

and a distant 3rd option is to try to do the fileset thing

micha19:03:14

boot is exceptionally good at generating things like clojure namespaces from other formats

micha19:03:03

like with hoplon we had haml files that would be transformed into hoplon files which would be transformed into clojurescript namespaces + html files, which would be compiled by the cljs compiler into js

micha19:03:18

and all the dependencies worked normally

micha19:03:33

with no need to add arbitrary new dependency rules to the language

kanwei19:03:41

what would that new namespace file look like?

kanwei19:03:50

ns with a bunch of strings?

micha19:03:00

yesql transforms a .sql file into a function, no?

kanwei19:03:47

well, would it be even easier to create a new ns-file, dump all the text into a str var in the new ns, and pass that string var to yesql?

micha19:03:43

or just generate the calls to yesql

kanwei19:03:00

alright thanks, that makes sense

kanwei19:03:11

i'm gonna try implementing the "touch" method just for practice tho simple_smile

kanwei19:03:23

this part isn't working for me: (cp fs ^File src-file ^TmpFile dest-tmpfile)

kanwei19:03:40

(doseq [db-file (by-name ["db.clj"] (input-files fileset))]
                   (println db-file)
                   (cp fileset db-file ( tmp "db.clj")))

micha19:03:14

yeah like i said, you don't want to do that

micha19:03:21

you need to copy the file into a temp dir

micha19:03:35

then add that tempdir to the fileset

micha19:03:38

and commit

kanwei19:03:13

what's the use case for boot.core/cp ?

micha19:03:38

it's for when you want to copy the temp file to another path in the fileset

micha19:03:49

but without changing the contents or the mtime or anything

micha19:03:55

but it's not really easy to use

micha19:03:02

it's used in maybe one place in the boot internals

micha19:03:28

you get a small performance benefit from it, but it's not easy to set up for it

tcrawley19:03:41

I'm using pod/call-in* to execute code in a pod, and am getting nil back. If I add println to the code in the pod, I can see it returning a value - any tips on debugging that?

micha19:03:42

@tcrawley: that's probably because the pod is returning something that isn't printable/readable

micha19:03:04

if you do pr-str in the pod you will probably see #object[...] in there

tcrawley19:03:13

ah, probably. it is a map of strings to File objects

tcrawley19:03:23

I'll turn those into string paths

micha19:03:48

there is a with-pod macro in 2.6.0-SNAPSHOT that can pass almost anything between pods

micha19:03:58

java objects no problem there

micha19:03:10

the only things it can't handle yet are things like refs and atoms

tcrawley19:03:26

cool. this is for the boot-immutant plugin, so it should probably support 2.5.x

micha19:03:28

but records are ok as long as the pod has access to the definition in its classpath

kanwei20:03:19

alright, last issue for me I think:

kanwei20:03:36

getting

java.io.FileNotFoundException: /Users/kanwei/.boot/cache/tmp/Users/kanwei/Projects/doubledonation/nwf/dkd4k1/doublethedonation/db.clj (No such file or directory)

kanwei20:03:07

tmp is a (tmp-dir!) and I'm following the lc->uc tutorial

micha20:03:15

you need to use io/make-parents before you write to the output file

micha20:03:22

before you copy

micha20:03:34

to create the subdirectories the output file will need

kanwei20:03:48

ah I see that in the tut too

micha20:03:02

so if you update the mtime on the file you will see a strange interaction with the watch task

micha20:03:19

because when you updte the mtime in your task it is only in the pipeline

micha20:03:30

it doesn't change the mtime of the db.clj file in your project

micha20:03:47

so the next time you change a file in your project the mtime will be reset back to the older mtime

micha20:03:51

kind of like how assoc returns a new map instead of mutating the existing one

kanwei20:03:05

it totally works though! awesome

micha20:03:28

it should be reloading db.clj twice though

micha20:03:37

like update a .sql file

micha20:03:43

you will see db reloaded

micha20:03:52

then modify some other .cljs file

micha20:03:02

you should see db reloaded again, for no reason

micha20:03:12

i mean some other .clj file

kanwei20:03:22

yup you're right

micha20:03:47

that's why, because it's resetting the mtime back to the original value

micha20:03:02

probably fine for your use

kanwei20:03:03

side-question.. how I make cljs not recompile when editing non-cljs files? it seems to do it for every file

micha20:03:13

that's a cljs compiler issue

micha20:03:34

the cljs compiler should be caching everything it needs

micha20:03:40

so it should be compiling anything

micha20:03:46

*shouldn't

kanwei20:03:56

soo known issue or is my setup non-optimal?

micha20:03:32

are you sure it's actually compiling anything?

kanwei20:03:05

Compiling ClojureScript... • public/js/ddplugin.js Elapsed time: 1.567 sec

kanwei20:03:08

every edit to any file

micha20:03:48

you can add :verbose true to the watch task

micha20:03:55

what does that print?

micha20:03:02

can you paste?

kanwei20:03:04

this is me modifying controller.clj:

kanwei20:03:06

◉ :cp doublethedonation/controller.clj

#'doublethedonation.system/dev-system:refreshing:doublethedonation.controller
#'doublethedonation.system/dev-system:refreshing:doublethedonation.handler
#'doublethedonation.system/dev-system:refreshing:doublethedonation.system
Compiling ClojureScript...
• public/js/ddplugin.js
Elapsed time: 1.024 sec

kanwei20:03:38

i can try without system too

kanwei20:03:15

same behavior without system

micha20:03:31

so that means that the cljs compiler is for some reason recompiling things

micha20:03:48

you can add verbose to the :compiler-options option of the cljs task

micha20:03:56

that will have the cljs compiler print some stats

micha20:03:12

that will show you if the 1.024s is the cljs task or the cljs compiler itself

kanwei20:03:19

yeah it's just the task

kanwei20:03:00

shouldn't the task only watch for cljs changes though? simple_smile

micha20:03:16

there's no way to really do that

micha20:03:20

you might have macros etc

micha20:03:30

and other things that the cljs compiler keeps track of

kanwei20:03:31

true i just thought of that

micha20:03:34

like deps.cljs files

kanwei20:03:48

it's a fine price to pay

micha20:03:59

we could try to fix the cljs compiler

micha20:03:14

and use the output of the compiler analysis to make a better decision of when to call it

micha20:03:44

wait so it's just the task?

kanwei20:03:59

◉ :cp doublethedonation/controller.clj

#'doublethedonation.system/dev-system:refreshing:doublethedonation.controller
#'doublethedonation.system/dev-system:refreshing:doublethedonation.handler
#'doublethedonation.system/dev-system:refreshing:doublethedonation.system
Compiling ClojureScript...
• public/js/ddplugin.js
Elapsed time: 1.041 sec

kanwei20:03:06

that's a non-related file updating

kanwei20:03:14

◉ :cp doublethedonation/integrations/router.cljs

Compiling ClojureScript...
• public/js/ddplugin.js
Compiling /Users/kanwei/.boot/cache/tmp/Users/kanwei/Projects/doubledonation/o3d/-rcsl8f/doublethedonation/integrations/router.cljs
Compiling /Users/kanwei/.boot/cache/tmp/Users/kanwei/Projects/doubledonation/o3d/-rcsl8f/doublethedonation/integrations/core.cljs
Reading analysis cache for file:/Users/kanwei/.boot/cache/tmp/Users/kanwei/Projects/doubledonation/o3d/-rcsl8f/doublethedonation/integrations/router.cljs
WARNING: Use of undeclared Var doublethedonation.integrations.core/update-data at line 370 src-cljs/doublethedonation/integrations/core.cljs
Compiling /Users/kanwei/.boot/cache/tmp/Users/kanwei/Projects/doubledonation/o3d/-rcsl8f/doublethedonation/core.cljs
Compiling /Users/kanwei/.boot/cache/tmp/Users/kanwei/Projects/doubledonation/o3d/y82kcs/boot/cljs/main30310.cljs
Reading analysis cache for file:/Users/kanwei/.boot/cache/tmp/Users/kanwei/Projects/doubledonation/o3d/-rcsl8f/adzerk/boot_reload.cljs
Copying file:/Users/kanwei/.boot/cache/tmp/Users/kanwei/Projects/doubledonation/o3d/-rcsl8f/doublethedonation/integrations/router.cljs to /Users/kanwei/.boot/cache/tmp/Users/kanwei/Projects/doubledonation/o3d/-b1oyye/public/js/ddplugin.out/doublethedonation/integrations/router.cljs
Elapsed time: 1.270 sec

kanwei20:03:19

is an actual cljs file updating

micha20:03:32

interesting

micha20:03:42

i wonder what it's doing

micha20:03:00

do you have a .cljs.edn file in your project?

kanwei20:03:16

{:require [doublethedonation.core]
 :compiler-options {:asset-path "/api/js/ddplugin.out"
                    :externs ["public/js/externs.js"]
                    :preamble []}}

micha20:03:41

do you have a ton of files in the fileset maybe?

kanwei20:03:55

not really

kanwei20:03:05

not sure why that's relevant?

micha20:03:17

just trying to think what could be taking 1s to do

micha20:03:31

one thing is doing lots of stuff with files in the fileset

micha20:03:05

if you have 20k files in the fileset you'll see things take longer to do

kanwei20:03:19

it's always been around 1s for all my projects

kanwei20:03:35

including like, brand new ones

micha20:03:56

what kind of computer are you using?

micha20:03:26

this is what i get on my machine, and it's compiling hoplon etc, with reload and whatnot

micha20:03:59

sorry for the flood there

micha20:03:11

oh wait, the elapsed time includes all the system stuff

micha20:03:27

what if you remove the cljs task from the pipeline?

kanwei20:03:41

I have a 5ghz i7

kanwei20:03:50

without cljs

◉ :cp doublethedonation/controller.clj

#'doublethedonation.system/dev-system:refreshing:doublethedonation.controller
#'doublethedonation.system/dev-system:refreshing:doublethedonation.handler
#'doublethedonation.system/dev-system:refreshing:doublethedonation.system
Elapsed time: 0.497 sec

kanwei20:03:19

ok I didn't realize the elapsed time was for ALL watches

kanwei20:03:21

not just cljs

micha20:03:41

it's for the pipeline

kanwei20:03:52

yeah I thought it was cljs printing that

micha20:03:12

it's still doing something for 500ms, which is odd

micha20:03:59

i mean some task is anyway

kanwei20:03:00

let me like disable all the tasks

kanwei20:03:58

◉ :cp doublethedonation/controller.clj

Elapsed time: 0.306 sec

micha20:03:43

you have BOOT_EMIT_TARGET=no set?

kanwei20:03:55

echo $BOOT_EMIT_TARGET no

micha20:03:04

it could be taking 300ms to copy all your files to the target dir

micha20:03:23

i see like 7ms here

micha20:03:30

for no tasks just watch

kanwei20:03:07

that's 2 orders of magnitude faster than mine simple_smile

micha20:03:28

if you just do boot watch you see 300ms update times?

kanwei20:03:31

i'm on mac

kanwei20:03:31

yup this is 500ms to start, then 300ms to update

kanwei20:03:34

(deftask dev
         "Run when developing"
         []
         #_(lein-generate)
         (comp #_(environ :env {:dev "true"
                              :http-port 3000})
               (watch :verbose true)
               #_(refresh-sql-files :db-files ["db.clj"])
               #_(serve :init 'doublethedonation.handler/init
                        :handler 'doublethedonation.handler/app
                        :reload true)
               #_(system :sys #'dev-system
                       :auto true
                       :files ["handler.clj"
                               "db.clj"])
               #_(sass)
               #_(reload :asset-path "/api/")
               #_(cljs :source-map true
                     :compiler-options {:output-wrapper true
                                        :devcards true}
                     :optimizations :none)
               #_(repl :port 62909
                     :server true)))

kanwei20:03:24

maybe it's the OS? what are you running

kanwei20:03:34

yeah could be that

micha20:03:51

it's not doing anything, which is the weird part

micha20:03:05

it's not doing any IO

kanwei20:03:14

could be a delay on the fswatch or something

micha20:03:27

that won't be counted in the elapsed time though, i don't think

kanwei20:03:07

would be easy to tell if someone else here could run it and see what they get too simple_smile

kanwei20:03:28

both my macs give 300ms

kanwei20:03:33

on different projects

micha20:03:12

the only thing i can think of is a lot of files in the source or resource paths

micha20:03:29

because it does need to compute a diff there

micha20:03:40

and that's included in the elapsed time

kanwei20:03:10

yeah you're right a boot-new project gives 0.003 ms

kanwei20:03:21

idk, my project is pretty vanilla

micha20:03:33

the fsevents stuff is really bad on pretty much all operating systems

micha20:03:45

so it can't be used to compute deltas

micha20:03:56

boot has to walk all the files and diff them

micha20:03:00

to know what changed

micha20:03:27

otherwise it's firing all the time spuriously

kanwei20:03:42

i mean, it's not watch the ./target/ dir is it?

micha20:03:12

the target dir shouldn't be involved at all

micha20:03:30

if you don't have the target task in the pipeline that is

kanwei20:03:38

as in, I have a target folder in the dir

kanwei20:03:53

so it's technically in the dir

micha20:03:57

but not in source or resource paths

micha20:03:05

it only watches those

kanwei20:03:12

:source-paths #{"src-cljs" "src-cljc" "src-clj"} :resource-paths #{"resources"}

micha20:03:25

but if you have the target task it might need to write files there, which can take time

kanwei20:03:26

yeah I have less than 100 files total

micha20:03:38

that shouldn't be a big deal

micha20:03:20

i wonder if stat'ing files is really slow or something?

micha20:03:41

still seems like an awful long time

micha20:03:20

300ms to stat 100 files seems like a lot

kanwei20:03:26

ok it's the resource-path that's the issue

micha20:03:47

which version of boot btw?

micha21:03:36

can you try with 2.6.0-SNAPSHOT?

micha21:03:06

there are a couple of fileset performance optimizations there that might help

kanwei21:03:41

bringing it from 800 to 250 is down to 0.175

kanwei21:03:46

let me try snapshot

laforge4921:03:03

How do you get correct aot build ordering? It seems to ignore imports and requires in determing the order of .cljc files to compile.

micha21:03:04

at least it's linear

micha21:03:40

@laforge49: how do you mean?

laforge4921:03:49

I specify both base and AAVector to be aot's. and indeed, it compiles 1/2 and can't find the interface defined in the second because it is compiling in the wrong order.

kanwei21:03:54

yeah it's down to 0.090 in 2.6

kanwei21:03:07

still slow but w/e

micha21:03:10

@kanwei: that's great news

micha21:03:44

it will never be as fast as banging on a mutable filesystem

laforge4921:03:45

my AAVector starts like this...

micha21:03:54

at least fort the simple things like that

micha21:03:11

but it might be faster for more complex things when you need to get fileset diffs and such

micha21:03:24

because then it can leverage immutablity

laforge4921:03:42

snipit not working. 😞

laforge4921:03:43

ns durable.AAVector #?(:clj (:gen-class :main false :extends clojure.lang.APersistentVector :implements [clojure.lang.IObj durable.base.INoded] :constructors {[java.lang.Object clojure.lang.IPersistentMap] [] [java.lang.Object clojure.lang.IPersistentMap clojure.lang.IPersistentMap] []} :init init :state state)) (:require [durable.base :as base]) #?(:clj (:import (durable AAVector) (durable.base INode) (clojure.lang IPersistentMap))))

micha21:03:59

the 90ms there consists of doing the copying needed to make things immutable

micha21:03:30

the aot task just uses the clojure compile function

micha21:03:01

i would recommend not using AOT there

laforge4921:03:28

aot is required with gen-class, yes?

micha21:03:37

is gen-class needed?

laforge4921:03:46

unfortunately, yes

micha21:03:58

if anything you can make a shim and gen-class that

laforge4921:03:09

:extends clojure.lang.APersistentVector :implements [clojure.lang.IObj durable.base.INoded]

laforge4921:03:27

Building alternatives to clojure vector.

laforge4921:03:42

problems only occur when using .cljc.

micha21:03:30

i don't know if :gen-class is supported with cljc at all

micha21:03:37

seems like something that isn't suited to it at all

laforge4921:03:58

well it was working until I created a dependency. 😞

laforge4921:03:18

Compile ordering is the issue.

laforge4921:03:56

OK, well thanks.

laforge4921:03:42

Imay need to move my protocol dependency to a clj file.

laforge4921:03:07

Ordering seems to work when the dependency is a clj file

micha21:03:29

seems like that's doing very platform specific stuff there

micha21:03:44

which might as you say be better in a clj file

micha21:03:00

and pull it all together in the cljc file

laforge4921:03:08

I doubt that the dependency checker for ordering even considers .cljc files.

micha21:03:14

like expose the api that way

micha21:03:24

so the only cljc file is the api file

laforge4921:03:52

yeah, but then massive duplication of code. Not so fun to maintain.

micha21:03:19

you could put anything that's duplaicated in the cljc file

laforge4921:03:22

I'm keeping the aot'd namespaces small at least.

laforge4921:03:47

yeah, I've been refactoring like crazy for decoupling.

laforge4921:03:00

cljc files over an aot'd file is ok, the reverse is not.

laforge4921:03:07

...at least so far as access to interfaces goes. everything else is fine.

laforge4921:03:37

(what works and what does not work is a bit complex.)

micha21:03:25

i can imagine

micha21:03:46

we'll need autotools soon to generate makefiles with all the #ifdefs

laforge4921:03:00

I'm thinking that anything that needs to be imported by an aot'd file must be in a clj file.

micha21:03:11

have you considered proxy?

micha21:03:16

that might get you out of the aot game

pesterhazy21:03:33

Is there a way to prepend a javascript file to my output bundle in boot-cljs?

laforge4921:03:00

@micha looks possible, thanks!

juhoteperi21:03:21

@pesterhazy: Define the file as foreign-lib using deps.cljs in local project and require the namespace from cljs. Saapas should have example: https://github.com/Deraen/saapas/blob/master/src/cljs/deps.cljs

pesterhazy21:03:56

@juhoteperi: I think I actually need the file to be at the top of the file, before everything else

juhoteperi21:03:03

@pesterhazy: Foreign libs are placed before anything else, and if you require this foreign lib the first (main namespace, first require line) it should be before anything else

pesterhazy21:03:12

@juhoteperi: I'll try that then

pesterhazy21:03:28

@juhoteperi: that worked fabulously

martinklepsch21:03:20

@pesterhazy: another option might be just creating a new file and appending the built js? doesn't require foreign libs hackery...

micha21:03:16

there is also the :preamble compiler option if you want to prepend something like a license or copyright comment

micha21:03:43

only works with optimizations on though, naturally

pesterhazy21:03:16

foreign libs seem to work, but :preamble seems an even better

pesterhazy21:03:39

and all just to make the react-native packager happy

pesterhazy21:03:18

@micha, I actually need it for optimizations simple and above only

martinklepsch22:03:14

@pesterhazy: @micha still thinking prepending with a task might be better, user could have their own preamble set.

micha22:03:09

that's a good point

pesterhazy22:03:34

prepending with a task sounds good except I'd need to figure out how to do that with boot simple_smile

pesterhazy22:03:11

hmm preamble doesn't seem to work

pesterhazy22:03:51

WARNING: Preamble resource file not found: prelude.js

pesterhazy22:03:27

needs to be in the resource dir I assume

micha22:03:54

looks like you have no choice but to do it the right way 😄

pesterhazy22:03:58

I'll always find a way to do it in a hacky way

micha22:03:10

haha that's the spirit

pesterhazy22:03:11

it's in my nature

pesterhazy22:03:53

thanks guys you've helped a lot

pesterhazy22:03:03

if only react-native had a channel like this

micha22:03:36

good luck!

martinklepsch22:03:00

@pesterhazy:

(with-pre-wrap [fs]
  (let [path "some.js"
        out (tmp-dir!)
        f   (tmp-file (get-in fs [:tree path]))
        newf (io/file out path)]
    (io/make-parents newf)
    (spit newf (str "mypreamble" (slurp f)))
    (-> fs (add-resource out) commit!)))

martinklepsch22:03:33

@pesterhazy: didn't try any of this but should give you at least a starting point simple_smile