Fork me on GitHub
#boot
<
2016-04-10
>
richiardiandrea00:04:34

By the way now we have make test :)))

micha01:04:03

totally tested

richiardiandrea02:04:42

Ah maybe boot is not in the PATH?

richiardiandrea02:04:24

'bout the failing build, maybe the Makefile needs little tweaking :)

richiardiandrea02:04:32

Maybe boot test actually is bin/boot test

onetom16:04:39

I cleaned my ~/.m2 folder the other day and to my surprise our app triggered the download of 5 different versions of Clojure. boot show -d didn't show any other version than the explicitly specified 1.8.0 boot show -p however showed the following:

[✔] org.clojure/clojure
    ✔ 1.8.0
      org.clojure/clojure
      org.clojure/clojurescript
    ✘ 1.7.0
      com.andrewmcveigh/cljs-time
      com.andrewmcveigh/cljs-time
      hoplon/jquery-daterange-picker
      org.clojure/test.check
    ✘ 1.5.1
      binaryage/devtools
      hoplon/castra
    ✘ 1.3.0
      hoplon

onetom16:04:50

I have the feeling that the downloads triggered while walking the dependency tree is a bit too eager...

onetom16:04:32

My downloads looked something like this:

onetom@retinatom ~/e/appboard> boot dev
Starting reload server on 
Writing boot_reload.cljs...
Started Jetty on .
...
Retrieving clojure-1.3.0.jar from 
Retrieving clojure-1.6.0.jar from 
Retrieving clojure-1.6.0.jar from 
...
...
Retrieving clojure-1.5.1.jar from 
Retrieving clojure-1.2.1.jar from 
...
Retrieving clojure-1.4.0.jar from 
...

onetom16:04:19

There are 2 notable things in the output 1. why does 1.6.0 is there twice? 2. how come 1.2.1 & 1.4.0 doesn't appear in the output of boot show -p?

onetom16:04:09

It happened with boot-2.5.5 I'm trying it now with 2.6

micha16:04:32

@onetom: if you build from master you can see more information about what it's downloading

micha16:04:18

@onetom: I suspect that maybe some dependency uses a version range instead of a concrete version number, which is pretty much deprecated now

micha16:04:59

when you have a dependency like [org.clojure/clojure "(1.2,)"] for example

micha16:04:16

that will cause aether to have to dowload all the pom.xml files at least

onetom16:04:35

so if i just specify 2.6.0-SNAPSHOT in the boot.properties file it's not new enough?

micha16:04:07

no, there is a bunch of new stuff in master

onetom16:04:12

i just wanted to ask if i see it well that the date mentioned in the jar filename suggests it's just a february 2nd release

onetom16:04:17

ok, i will try

micha16:04:05

boot probably could be more selective about what it downloads

onetom16:04:22

it takes 5 minutes to download all the dependencies w that version even on an otherwise decent internet connection because there are some misterious pauses between the downloads

micha16:04:24

there might be some setting in aether somewhere to disable downloading things when you resolve them

micha16:04:15

but the whole situation is pretty complicated there

onetom16:04:57

im also seeing ~30kbyte/s download rates in the activity monitor often while - mostly for files coming from maven central - i see 1+Mbyte/s

micha16:04:53

it shouldn't be downloading anything once the cache is populated though

onetom16:04:23

sure, i just noticed this when i cleaned by ~/.m2 folder and if it's downloading several versions of clojure then probably it might have other unnecessary downloads which hurts the overall boot experience...

micha16:04:15

the maven code uses memoization for like a level1 cache which speeds things up a ton

micha16:04:36

because you don't need to be making requests over the network all the time

micha16:04:56

but that also makes it more difficult to be nuanced about how aether is called

micha16:04:04

it would be good to separate the dependency graph building from the jar fetching

micha16:04:51

not sure how to do it though lol

onetom17:04:30

if aether doesn't do it then probably it can only be done by reimplementing what aether does, right?

micha17:04:41

there is an option to aether i think

micha17:04:53

you can tell it not to download jars

micha17:04:13

it might not actually work though, because some of the things are not as simple as they appear in maven world

micha17:04:55

but it probably does work

micha17:04:30

but that would mean we'd need to change the way dependencies work in boot

onetom17:04:16

which would be confusing for plain java deps i guess, if those would resolve to different versions than in a pure maven project

micha17:04:06

oh, that would be fine

micha17:04:16

i mean we can't change how maven works

micha17:04:35

but if you have a separate set of functions for computing the dependency graph without downloading

micha17:04:44

and then more functions to download jars

micha17:04:56

that's a big change

micha17:04:17

right now if you tell boot to resolve-dependencies you get back a data structure with all the info about the depdnecies

micha17:04:32

the graph, the versions, and the jars in maven cache .m2

micha17:04:49

so we have lots of functions built on top of that

micha17:04:26

if you separate the the jars now you can't use that one function as the basis of all the other functions

micha17:04:03

i mean it would be great to do, it just seems like it would be more than just a bugfix

micha17:04:28

it would need some thought and design work probably

onetom17:04:29

simple_smile i was just about to ask about this because i noticed 1.6 during install

(cd boot/pod && lein install)
...
Retrieving org/clojure/clojure/1.6.0/clojure-1.6.0.pom from central

micha17:04:57

default is 1.7.0 now, updating to 1.8

micha17:04:40

i think it fetches a lot of pom.xml files because those are maybe the parent of other pom.xml files

micha17:04:48

i think clojure does that in their pom

onetom17:04:08

btw, i wanted to try boot master without ruining the production deployment, so i thought i could just export env BOOT_LOCAL_REPO=~/.m2/test26 time make deps install but obviously it doesn't work... would it make sense to tell lein where should it look for a maven repo if there is a BOOT_LOCAL_REPO env var?

micha17:04:54

we could add something to the project.clj files, you mean?

micha17:04:06

or the Makefile?

onetom17:04:18

Makefile i guess

micha17:04:32

i think lein has an env var for that?

onetom17:04:49

"i grew up on boot, so i wouldn't know" 😉

onetom17:04:19

honestly i dont really want to know either, because i hope it will be obsolete knowledge soon 😉

micha17:04:50

hmm, i think there is only a key in the project map

onetom17:04:24

thats not very nice 😕

micha17:04:44

we could add it

micha17:04:25

:local-repo ~(or (System/getenv "BOOT_LOCAL_REPO") nil)

micha17:04:27

something like that

onetom17:04:35

hmmm... i've just compiled the tip of master and when i run

pair@parrot ~/g/b/boot> ~/github.com/boot-clj/boot/bin/boot -V
Downloading ...
why is it trying to download that version? i've cleared my /.boot and /.m2 and ~/.lein and im standing in /Users/pair/github.com/boot-clj/boot

micha17:04:45

it's relative to the project.clj file though, so that would need to be figured out

micha17:04:09

do you have a ~/.boot/boot.properties?

onetom17:04:24

no, i did a mv ~/.boot{,.bak}

micha17:04:40

ah that is the default

micha17:04:44

for boot.sh

micha17:04:53

that's an old version of boot.sh too

micha17:04:01

if you do make deps in the boot dir

micha17:04:09

you can use ./bin/boot

onetom17:04:18

i've just built it:

pair@parrot ~/g/b/boot> ls -lah ~/github.com/boot-clj/boot/bin/boot
-rwxr-xr-x  1 pair  staff   6.3K Apr 11 01:14 /Users/pair/github.com/boot-clj/boot/bin/boot

micha17:04:19

make deps downloads lein and boot

onetom17:04:58

well, by built i mean make deps downloaded it for me

onetom17:04:55

i've even mv ~/github.com/boot-clj/boot{,.bak} and cloned it again to make sure i have a fresh tree

micha17:04:36

without BOOT_VERSION set, boot.sh needs to start something

micha17:04:45

so it has a version of boot hardcoded in it

micha17:04:57

and that's what it is downloading i think

onetom17:04:12

sure, but i was expecting 2.5.5 since that is the latest stable for quite a while now

micha17:04:23

i don't know why it's trying to download 2.5.2, it should be trying to download 2.5.5

onetom17:04:32

but if u r saying 2.5.2 is the expected behaviour, im fine

micha17:04:08

it's annoying because github doesn't have a "latest" URL for releases

micha17:04:32

so i have to manually trick it by deleting and recreating a "latest" tag pointing to the latest release

micha17:04:04

maybe i did that wrong

onetom17:04:11

aww... that's crap indeed

micha17:04:29

managing software versions is exhausting

onetom17:04:01

someone said it's a tradeoff between copy-pasting code vs modularizing it simple_smile

onetom17:04:15

btw, does this look correct to u?

pair@parrot ~/g/b/boot> ~/github.com/boot-clj/boot/bin/boot -V
Downloading ...
Running for the first time, BOOT_VERSION not set: updating to latest.
Retrieving clojure-1.7.0.jar from 
#
#Mon Apr 11 01:51:57 HKT 2016
BOOT_CLOJURE_NAME=org.clojure/clojure
BOOT_VERSION=2.5.5
BOOT_CLOJURE_VERSION=1.7.0

micha17:04:35

that looks legit

onetom17:04:37

based on your latest commit message i was expecting 1.8

micha17:04:47

it's using 2.5.5 though

micha17:04:53

which load 1.7.0 by default

micha17:04:13

if you do BOOT_VERSION=2.6.0-SNAPSHOT boot -V it should be different

onetom17:04:23

so why the fuck was it saying Downloading 2.5.2 BUT not mentioning downloading 2.5.5?

micha17:04:39

it downloaded 2.5.2 because that was hardcoded in

micha17:04:54

that version of boot was used to upgrade boot to the newest release

micha17:04:02

because you didn't have BOOT_VERSION set

micha17:04:10

so it says ok i'll get the latest version

micha17:04:16

the latest release is 2.5.5

micha17:04:20

so it looks correct

onetom17:04:05

ok, so newer versions of the boot binary will get the latest version of the boot.jar immediately?

micha17:04:05

BOOT_VERSION=2.6.0-SNAPSHOT boot repl

micha17:04:14

only if no BOOT_VERSION is set

micha17:04:26

because it needs to set that

micha17:04:46

if you set BOOT_VERSION it will use that

onetom17:04:11

im thinking from a newbie standpoint, who has a virgin machine

micha17:04:21

right you'll get the latest release version

micha17:04:31

and it will set BOOT_VERSION in your boot.properties

onetom17:04:43

they shouldn't specify env vars. that's "bad UX"

micha17:04:59

they don't need to unless they want a specific version

richiardiandrea18:04:25

so @micha, if I want to execute things in the core pod, and skip boot.main/-main for boot.parallel, I just need to set pod specific stuff like verbosity right?

micha18:04:06

there is a bunch of initialization happening in there

micha18:04:22

it's weird though that only your tasks show this issue

micha18:04:35

you can see it when you build, too

micha18:04:44

just compiling the namespace triggers it

micha18:04:54

so i suspect a bug in the warning maybe

richiardiandrea18:04:14

I can look into that

micha18:04:42

we also need to make sure that boot.user isn't somehow getting into a jar file

micha18:04:49

or something like that

richiardiandrea18:04:37

you know, the warning is in a delay

richiardiandrea18:04:12

I am not sure why and if this can be a problem, thinking...

micha18:04:02

ah this is the problem

micha18:04:04

i see it now

micha18:04:09

it's the AOT

micha18:04:16

those things are getting AOT compiled

richiardiandrea18:04:39

you mean parallel.boot right?

richiardiandrea18:04:53

actually everything is AOT-ed

micha18:04:10

yeah i'm fixing that

micha18:04:05

well, that isn't it

richiardiandrea18:04:13

i will println to see ns and sym at that point

richiardiandrea18:04:41

but well, probably they will match ah ah

micha18:04:14

trying something else now

onetom18:04:53

after doing a pair@parrot ~/g/b/boot> rm -rf ~/.boot/ ~/.m2/repository/; time make clean deps install; env BOOT_VERSION=2.6.0-SNAPSHOT bin/boot -V i can observe a bunch of clojure versions, but as it turns out some of them are just the pom files:

pair@parrot ~/g/b/boot> du -hsc ~/.m2/repository/org/clojure/clojure/*
 12K	/Users/pair/.m2/repository/org/clojure/clojure/1.2.0
 12K	/Users/pair/.m2/repository/org/clojure/clojure/1.2.1
 16K	/Users/pair/.m2/repository/org/clojure/clojure/1.3.0
 16K	/Users/pair/.m2/repository/org/clojure/clojure/1.3.0-beta1
 16K	/Users/pair/.m2/repository/org/clojure/clojure/1.4.0
 16K	/Users/pair/.m2/repository/org/clojure/clojure/1.5.1
3.5M	/Users/pair/.m2/repository/org/clojure/clojure/1.6.0
3.6M	total
Which is funny because there is no 1.8.0 though the ~/.boot/boot.properties refers to that..

micha18:04:40

did you run an actual boot task or just boot -V?

micha18:04:47

cause boot -V doesn't actually load boot

micha18:04:57

it just reads the metadata in the .properties files and whatnot

micha18:04:16

so it makes sense that clojure 1.8 isn't loaded yet

micha18:04:18

in that case

onetom18:04:48

sure i have not run anything else, just boot -V, but why clojure 1.6.0 is there though... 😕

onetom18:04:38

i think it was used to build aether

micha18:04:14

yeah boot's worker pod probably

micha18:04:41

@richiardiandrea: is it only tasks defined via deftesttask that show the warnings?

onetom18:04:48

ah, i see now:

pair@parrot ~/g/b/boot> ag 1.6.0
boot/aether/project.clj
16:  :dependencies [[org.clojure/clojure               "1.6.0"  :scope "compile"]

boot/core/project.clj
16:  :dependencies   [[org.clojure/clojure "1.6.0"  :scope "provided"]

boot/pod/project.clj
19:                   [org.clojure/clojure                     "1.6.0"  :scope "provided"]

boot/worker/project.clj
18:  :dependencies [[org.clojure/clojure         "1.6.0"  :scope "provided"]

onetom18:04:07

wouldnt it make sense to upgrade these too to 1.8?

onetom18:04:33

or u r keeping it at 1.6 to say that it's the lowest supported version?

richiardiandrea18:04:49

@micha: let me double check that (also boot -vv shows the stack trace) I have tried only that, but it is a macro calling deftask

micha18:04:49

i mean it works reliably now

micha18:04:04

i don't like to update things if they're working well as they are

micha18:04:18

that's the nice thing about pods

onetom18:04:45

sure, just seeing 7 different clojure versions make me think there is some problem somewhere but now im less worried after i saw they are not actually 7 complete clojure versions i was just hoping to clean it up somehow...

richiardiandrea18:04:32

@micha silly question, resolve always looks at the current pod right?

richiardiandrea18:04:12

so now I am confused on what this check is trying to detect

micha18:04:56

it's detecting when a task overrides an existing task

micha18:04:07

because things are refered into boot.user

micha18:04:12

that's the main use

micha18:04:40

but in this case i thnk you're actually somehow redefining tasks outside the pod

micha18:04:48

like maybe in runtests

richiardiandrea18:04:39

this is my deftesttask, just invokes deftask with a different body

richiardiandrea18:04:56

in a new pod like you were saying yesterday...

richiardiandrea18:04:08

anyways I am on it

micha18:04:49

maybe fetching namespaces causes them to be compiled

richiardiandrea18:04:55

deftesttask is executed on the require in build boot...

micha18:04:06

i dunno, just guessing

richiardiandrea18:04:36

yeah there is definitely some subtle issue

richiardiandrea18:04:14

require eventually calls clojure.lang.RT/load

richiardiandrea18:04:41

which makes a class loader

micha18:04:36

but my multi-module-build demo doesn't show the issue

micha18:04:48

and that loads the same build.boot in different pods

micha18:04:34

strange business

richiardiandrea18:04:32

to answer you previous question, both deftask and deftesttask trigger the warning so we can exclude the latter macro

richiardiandrea18:04:39

in the pod *ns* is probably the problem

richiardiandrea18:04:55

which is set to the namespace it is evaluating in cause of the (ns ...) on the top of the required file

richiardiandrea19:04:28

so if I create two pods, and do (ns ...) in both, should I get two separate clojure.lang.Namespace objects?

micha19:04:58

what do you mean by "do (ns ...) in both"?

richiardiandrea19:04:29

I mean evaluate (ns boot.task.built-in-test)

micha19:04:46

are you evaling that?

micha19:04:53

or requiring the namespace from a file?

richiardiandrea19:04:01

no well require-ing

micha19:04:06

that should be fine

richiardiandrea19:04:50

but that line is called in boot.main/-main many times through runboot

richiardiandrea19:04:08

because we evaluate the script over and over now

micha19:04:16

requiring the namespace should't cause things to be redefined

richiardiandrea19:04:18

let me try something

micha19:04:00

each pod should have its own instance of clojure.core/*loaded-libs*

micha19:04:18

that's a var that contains the map of namespaces that have been compiled

micha19:04:27

when you call require it shoudl check that

micha19:04:50

it doesn't reload the namespace if it's in there, unless you have :reload in the require

richiardiandrea19:04:31

yes but I am creating a new pod each time, and I agree (and it is doing it) that my require triggers every time

richiardiandrea19:04:48

my concern is that the namespace objects are shared

micha19:04:51

we need to somehow bisect the problem

micha19:04:31

if that's true then we can reproduce this with a simpler setup, right?

micha19:04:49

i mean if namespace objects are shared

richiardiandrea19:04:08

yep, I am trying to do it, I will get back to you with a simple example

micha19:04:26

one data point is that i can see the warning when i do make install

micha19:04:40

deftask boot.parallel/runboot was overridden
deftask boot.parallel/await-done was overridden
deftask boot.parallel/parallel-start was overridden
deftask boot.parallel/runcommands was overridden

richiardiandrea19:04:52

ah I didn't notice that

micha19:04:55

so it must be something at the top level

micha19:04:02

that happens during AOT

richiardiandrea19:04:17

only those tasks?

micha19:04:39

yep that's all

micha19:04:55

so perhaps commenting things out

micha19:04:01

might be a way to bisect

richiardiandrea19:04:03

that's a normal namespace...

micha19:04:03

i did make clean before make install

micha19:04:08

(i always do that)

micha19:04:23

so i don't think it's stale class files on the classpath or anything

richiardiandrea19:04:06

I haven't touched anything else in the Makefile, so maybe we need something

micha19:04:01

we need to be able to find a manageable repro acse

micha19:04:29

interesting

micha19:04:53

building a snapshot is weird too

micha19:04:04

because they depend on each other

micha19:04:09

i've seen weird stuff happen there

micha19:04:32

i'm trying with 2.6.0

micha19:04:56

and deleting any previously built jars for that version in my .m2

richiardiandrea19:04:14

I confirm that the actually are the same namespace object:

*ns* {:ns #object[clojure.lang.Namespace 0x1e6d22bf "boot.test-test"]}
existing {:ns #object[clojure.lang.Namespace 0x1e6d22bf "boot.test-test"], :name vars}
....then here the usual warning

micha19:04:55

i still see warnings

micha19:04:05

yeah i mean that makes sense

micha19:04:32

the question is whether deftask is being called multiple times in a single pod

micha19:04:37

which i think must be the case

micha19:04:06

i can't really follow how everything is being constructed there

micha19:04:33

it happens if you set :threads to 1

richiardiandrea19:04:20

I guess, "execute build.boot is one deftask, runboot is another

richiardiandrea19:04:28

but they should be in separate pods

richiardiandrea19:04:31

and indeed the ten -main calls are creating ten namespaces:

print-ns! *ns* {:ns #object[clojure.lang.Namespace 0x76d5985f "clojure.core"]}
print-ns! *ns* {:ns #object[clojure.lang.Namespace 0x4f9d4366 "clojure.core"]}
print-ns! *ns* {:ns #object[clojure.lang.Namespace 0x29cae75 "clojure.core"]}
print-ns! *ns* {:ns #object[clojure.lang.Namespace 0x43ee2557 "clojure.core"]}
print-ns! *ns* {:ns #object[clojure.lang.Namespace 0x3b81cd42 "clojure.core"]}
print-ns! *ns* {:ns #object[clojure.lang.Namespace 0x33c16375 "clojure.core"]}
print-ns! *ns* {:ns #object[clojure.lang.Namespace 0x38827b7b "clojure.core"]}
print-ns! *ns* {:ns #object[clojure.lang.Namespace 0x4961e445 "clojure.core"]}
print-ns! *ns* {:ns #object[clojure.lang.Namespace 0x60101bc1 "clojure.core"]}
print-ns! *ns* {:ns #object[clojure.lang.Namespace 0x22f894f3 "clojure.core"]}

adamkowalski19:04:44

@micha: hey so I am still struggling with getting my basic boot build to work haha

adamkowalski19:04:12

if you have a sec would you mind if I showed you me isolated repo that demonstrates what I am struggling with

adamkowalski19:04:55

I even removed compojure btw so that way I could try to isolate it completely

adamkowalski19:04:19

all i have is one handler that should just return a hiccup page, and then import a main.js file which lives in target/main.js

micha20:04:28

to run it i do boot dev in one terminal

micha20:04:41

then boot repl -c in another one to attach to the repl with a new client

micha20:04:53

i do (server/start) in that repl client

adamkowalski20:04:47

and that worked for you?

micha20:04:28

well i modified the server namespace

micha20:04:37

the version that works is in that gist there

micha20:04:56

i just added a simple routing thing

adamkowalski20:04:08

yeah just a catch all route

micha20:04:22

yeah you can get more fancy later for sure

adamkowalski20:04:24

so what is the “/“ route doing different then mine?

micha20:04:32

nothing, that's the same

micha20:04:40

but it needs to load all the various js files

micha20:04:55

that's the second arm of the conditional there

micha20:04:10

it's serving everything else from the classpath

micha20:04:27

i also got rid of the "target/main.js" in favor of "main.js"

micha20:04:38

because the target dir isn't something the server should really be looking at

micha20:04:46

it's in an undefined state while boot is running

micha20:04:56

but the classpath is well defined

adamkowalski20:04:23

it worked! your a genius, thank you!

adamkowalski20:04:41

so does boot repl vs boot repl -c make a big difference too?

micha20:04:52

boot repl will start another repl server

micha20:04:00

the cljs-repl task alraedy starts one

micha20:04:10

so that might be an issue, i dunno

micha20:04:25

boot repl -c just starts the repl client

micha20:04:35

it will attach to the already running server automatically

adamkowalski20:04:41

so you just use cider-connect rather then cider-jack-in

adamkowalski20:04:11

and connect to the nrepl over whatever port it spits out

richiardiandrea20:04:11

sorry guys don't want to interrupt 😄

micha20:04:21

adamkowalski: i would think so (not an emacs expert)

adamkowalski20:04:44

ok yeah i just tried it and it seems to work simple_smile

adamkowalski20:04:58

but I am still confused about the else clause for your routing

micha20:04:29

the else clause is doing this:

adamkowalski20:04:33

so essentially when it makes a request for anything besides root it will return js?

adamkowalski20:04:15

should I make a route for /main.js as the if clause and anything else should actually return index.html?

micha20:04:14

(-> (subs uri 1) ;; chop off the first character of the uri string, like "/foo" -> "foo"
    io/resource  ;; resolve the resulting uri from the classpath
    slurp)       ;; read the file from the classpath into a string

micha20:04:50

i took a shortcut because i know allthe things other than / are js files

micha20:04:57

so i hardcode the headers

micha20:04:01

you can get fancy later

adamkowalski20:04:48

yeah I might check out bidi like what we were talking about yesterday. looks intereseting

adamkowalski20:04:54

anyway thanks again for all your help simple_smile

micha20:04:13

the first thing is probably to get comfortable with ring and stuff

micha20:04:23

like doing it manually without libraries at first

adamkowalski20:04:43

yeah thats not a bad idea, and write my own simple router so i understand how exactly it works

micha20:04:46

because it's really simple and it's easy to make it more complicated with libraries

micha20:04:58

yep exactly

micha20:04:16

you can make your own thing very straightforwardly and it will do like 90% of all you will ever need

micha20:04:47

the last 10% is the really tricky stuff, and something like bidi has a super well engineered abstraction for that

adamkowalski20:04:05

like getting params out of the uri right

adamkowalski20:04:22

but thats the thing I don’t really wan’t to make a rest api, I would rather just use websockets to talk back and forth

micha20:04:31

yeah and organize routes in an optimal way for a large, complex application

adamkowalski20:04:49

then the client should just look at the url and dispatch the correct view according to the specified route

adamkowalski20:04:23

it seems like i dont need a compilicating routing solution for that

micha20:04:10

yeah i mean in your case you don't need hiccup either, i would say

micha20:04:26

just have static files and a general purpose server for development

micha20:04:33

just serve everything from the classpath

adamkowalski20:04:44

yeah i know i just prefer how hiccup looks to how html does haha 😛

micha20:04:48

then you can deploy to s3 and cloudfront

micha20:04:04

well you can make a boot task to generate the html file from a hiccup file

micha20:04:18

but you don't need to serve it from a servlet like that is what i mean

adamkowalski20:04:27

hmm interesting

adamkowalski20:04:06

so I would need a specific directory for hiccup files then have a boot task than takes every file there and renders to a string and into a html file in the target/ directory?

micha20:04:08

your files can all be on a CDN for production

micha20:04:35

yeah the task woud transform .html.hiccup files to .html files in the fileset

micha20:04:14

what you were describing is a single page app

micha20:04:20

so the html file will be only a shell

micha20:04:31

the actual contents are craeted in the client

adamkowalski20:04:10

cool, but what if I actually need a backend anyway? I was hoping to use datomic for storage and then everytime something there is changed publish the changes over a websocket. then use datascript on the front end and immediately transact every change into the client db

adamkowalski20:04:37

so clients can essentially have a mini database containing all the datoms they care about

richiardiandrea20:04:04

@micha it looks like resolve actually proactively resolves a symbol even if def has not been call on it, still need to prove this sentence, which might be totally off the target 😄

richiardiandrea21:04:19

so this is what happens before everything, and makes me think the check with resolve is not a good one and we should use maybe getMapping on the ns:

after (ns boot.task.built-in-test ...) {:ns "boot.task.built-in-test"], :pod 1}
before deftask with-meta-tests -- mapping of with-meta-tests null
before deftask with-meta-tests -- mapping of sift-with-meta-tests null
### evaluating (deftask with-meta-tests ...) *ns* {:ns "boot.task.built-in-test"], :pod 1}
maybeResolveIn sym with-meta-tests --- null
before resolve sym with-meta-tests -- .getMapping of with-meta-tests null
before resolve sym with-meta-tests -- .getMapping of sift-with-meta-tests null
after deftask with-meta-tests -- mapping of with-meta-tests #'boot.task.built-in-test/with-meta-tests
after deftask with-meta-tests -- mapping of sift-with-meta-tests null
### evaluating (deftesttask sift-with-meta-tests ...) *ns* {:ns "boot.task.built-in-test"], :pod 1}
after update-body *ns* {:ns "boot.task.built-in-test"], :pod 1}
### evaluating (deftask sift-with-meta-tests ...) *ns* {:ns "boot.task.built-in-test"], :pod 1}
##### maybeResolveIn sym sift-with-meta-tests --- #'boot.task.built-in-test/sift-with-meta-tests ##### this is wrong
before resolve sym sift-with-meta-tests -- .getMapping of with-meta-tests null
before resolve sym sift-with-meta-tests -- .getMapping of sift-with-meta-tests null
after resolve sym sift-with-meta-tests -- .getMapping of with-meta-tests null
after resolve sym sift-with-meta-tests -- .getMapping of sift-with-meta-tests null
existing-deftask# {:ns "boot.task.built-in-test"], :name sift-with-meta-tests}
deftask before warning {:ns"boot.task.built-in-test"], :pod 1}
deftask boot.task.built-in-test/sift-with-meta-tests was overridden
--- custom build.boot loaded {:ns "boot.user"], :pod 1} ---

richiardiandrea21:04:18

note that no deftask for boot.task.built-in-test is evaluated before this excerpt

richiardiandrea21:04:14

and this is all before build.boot is loaded, as the last line shows

richiardiandrea21:04:32

resolve ends up calling clojure.lang.Compiler/maybeResolveIn which I am still digging up to see how it resolve symbols

richiardiandrea21:04:58

for some reason (findInternedVar *ns* sift-with-meta-tests) --> #'boot.task.built-in-test/sift-with-meta-tests

richiardiandrea21:04:14

but the def has not been called yet

micha21:04:45

i see the namespace is being compiled twice

micha21:04:51

once because boot.test depends on it

micha21:04:08

and then once again by AOT

micha21:04:22

that's weird

richiardiandrea21:04:22

which namespace? boot.parallel?

micha21:04:34

i put like

micha21:04:53

(prn :got-here (ns-resolve *ns* 'runboot))

micha21:04:01

just before the (deftask runboot

micha21:04:22

(cd boot/core && lein install)
Compiling boot.cli
Compiling boot.task-helpers.notify
Compiling boot.core
Compiling boot.test
:got-here nil
Compiling boot.main
Compiling boot.git
Compiling boot.parallel
:got-here #'boot.parallel/runboot
:ns1 boot.parallel :ns2 boot.parallel
deftask boot.parallel/runboot was overridden

micha21:04:29

so it's actually correct, the warning

richiardiandrea21:04:41

yes in that case yes

richiardiandrea21:04:32

i am not an expert of AOT, shouldn't it compile it in the correct order?

micha21:04:47

yeah i would think so

micha21:04:56

i disabled all AOT for that stuff though

micha21:04:05

and the tests still get the warnings when i run them

micha21:04:11

so there must be something else going on

richiardiandrea21:04:23

here i see that boot.test boot.parallel and boot.task.built-in are triggering boot.util/warn, the rest of the namespaces don't

richiardiandrea21:04:37

which again is weird to me ..

micha21:04:53

boot.task.built-in shouldn't trigger anything

micha21:04:08

at least it never has in the past

richiardiandrea21:04:34

the trace at the top is coming from boot.test

richiardiandrea21:04:46

then boot.cli muted

richiardiandrea21:04:54

then boot.task.built-in

micha21:04:17

if something is requireing boot.task.built-in now then i can see how it would cause the warnings

richiardiandrea21:04:28

I think we have two problem there, I will try to solve the one above first ...

richiardiandrea23:04:21

Hey micha, I found the docstring of ns-resolve a bit disturbing:

Returns the var or Class to which a symbol will be resolved in the
namespace (unless found in the environment), else nil.  Note that
if the symbol is fully qualified, the var/Class to which it resolves
need not be present in the namespace.

richiardiandrea23:04:47

unless found in the environment?

micha23:04:20

i thikn that' sjust poorly worded

micha23:04:38

like in that namespace if you have this

micha23:04:56

(def foo 100)
(let [foo "asdf"] ...)

micha23:04:19

i don't think "resolve" is a good word to use for things in the environment

micha23:04:26

because you don't resolve those really

micha23:04:04

they're not vars, they're immutable values

micha23:04:26

the compiler could just put in the values instead of the symbols in some cases even

micha23:04:57

like in the (let [foo "100"] ...) case

richiardiandrea23:04:37

I feel like in the Shroedinger cat, I can't observe something because I am changing it while observing

richiardiandrea23:04:17

@micha a question for you, maybe you know better

richiardiandrea23:04:38

isn't it interning the var there?

richiardiandrea23:04:00

of course it is then found, maybe I don't read it properly

richiardiandrea23:04:40

ah ok sorry, intern actually does not do anything

micha23:04:41

but it's not found the first time the namespace is evaluataed

micha23:04:14

like if you put that prn line right about (deftask runtests

micha23:04:28

you can see the first time it's all good, and no warnings

micha23:04:37

but then the prn runs again, which should not happen

richiardiandrea23:04:42

there's a moment where things change and i cannot narrow down when

micha23:04:18

also boot.task.built-in namespace has an orphan :require

richiardiandrea23:04:18

### evaluating (deftesttask sift-with-meta-tests ...) *ns* {:ns #object[clojure.lang.Namespace 0x56078cea "boot.task.built-in-test"], :pod 1}
after deftesttask update-body (sift-with-meta-tests [] (boot.test/test-task (comp (sift :add-jar {(quote org.clojure/tools.reader) #".*"}) (sift :add-meta {#".clj$" :boot-test-tag}) (sift :with-meta #{:boot-test-tag}) (with-meta-tests))))
after deftesttask update-body *ns* {:ns #object[clojure.lang.Namespace 0x56078cea "boot.task.built-in-test"], :pod 1}
after deftesttask update-body sift-with-meta-tests -- .getMapping of with-meta-tests #'boot.task.built-in-test/with-meta-tests
after deftesttask update-body sift-with-meta-tests -- .getMapping of sift-with-meta-tests null
### evaluating (deftask sift-with-meta-tests ...) *ns* {:ns #object[clojure.lang.Namespace 0x56078cea "boot.task.built-in-test"], :pod 1}
maybeResolveIn sym sift-with-meta-tests --- #'boot.task.built-in-test/sift-with-meta-tests
(findInternedVar *ns* sift-with-meta-tests) --- #'boot.task.built-in-test/sift-with-meta-tests
before resolve sym sift-with-meta-tests -- .getMapping of with-meta-tests null
before resolve sym sift-with-meta-tests -- .getMapping of sift-with-meta-tests null
after resolve sym sift-with-meta-tests *ns* {:ns #object[clojure.lang.Namespace 0x56078cea "boot.task.built-in-test"], :pod 1}
after resolve sym sift-with-meta-tests -- .getMapping of with-meta-tests null
after resolve sym sift-with-meta-tests -- .getMapping of sift-with-meta-tests null

micha23:04:24

reefrencing boot.test

micha23:04:40

i don't think it makes any difference though

richiardiandrea23:04:16

thanks for spotting it

richiardiandrea23:04:59

you see above the .getMapping is fine until the second ### evaluating ...

richiardiandrea23:04:18

after deftesttask update-body sift-with-meta-tests -- .getMapping of sift-with-meta-tests null
### evaluating (deftask sift-with-meta-tests ...) *ns* {:ns #object[clojure.lang.Namespace 0x56078cea "boot.task.built-in-test"], :pod 1}
maybeResolveIn sym sift-with-meta-tests --- #'boot.task.built-in-test/sift-with-meta-tests

richiardiandrea23:04:35

these three lines don't make any sense if the *ns* is the same

micha23:04:05

it should be impossible to see those two "evaluating" lines

richiardiandrea23:04:12

the first is fine, the second still has not deffed anything

richiardiandrea23:04:24

the third should return null

micha23:04:27

to see what i mean make a namespace that just has the following line:

micha23:04:41

(ns foop)
(println "evaluating!")

micha23:04:57

you can then do (require 'foop) as many times as you want in one pod

micha23:04:03

and you won't see it print but one time

micha23:04:23

what you pasted above shows it printing multiple times

richiardiandrea23:04:19

yep it should be like you say for each and every isolated pod

micha23:04:41

like above pod 1 shows it defining the same task more than one time

richiardiandrea23:04:53

my lines are inside the same pod, requiring boot.task.built-in-test

richiardiandrea23:04:16

no well sorry the output is messy, but it is evaluating once

micha23:04:16

yeah something is not right there

micha23:04:41

i see this

richiardiandrea23:04:44

but ok, I still need to dig

micha23:04:45

### evaluating (deftesttask sift-with-meta-tests ...) *ns* {:ns #object[clojure.lang.Namespace 0x56078cea "boot.task.built-in-test"], :pod 1}

micha23:04:01

two times

richiardiandrea23:04:44

(ns boot.task.built-in-test ...

(deftask ^:private with-meta-tests []

(deftesttask sift-with-meta-tests [] <- this one then calls deftask

micha23:04:09

this reminds me of c++ template programming

richiardiandrea23:04:13

just to explain, the problem are these three lines:

after deftesttask update-body sift-with-meta-tests -- .getMapping of sift-with-meta-tests null
### evaluating (deftask sift-with-meta-tests ...) *ns* {:ns #object[clojure.lang.Namespace 0x56078cea "boot.task.built-in-test"], :pod 1}
maybeResolveIn sym sift-with-meta-tests --- #'boot.task.built-in-test/sift-with-meta-tests

richiardiandrea23:04:25

because I am still before defclifn

richiardiandrea23:04:01

no def has been triggered but the var is resolved

richiardiandrea23:04:09

by maybeResolveIn

micha23:04:32

maybe macroexpand the deftesttask expression

micha23:04:41

like macroexpand-all

richiardiandrea23:04:32

every thing is good, except I noticed that there was a particular way to apply alter-meta!

richiardiandrea23:04:59

at the bottom, probably of defclifn:

(let*
         [G__1664 var__210__auto__]
       (clojure.core/alter-meta!
        G__1664
        (clojure.core/fnil clojure.core/merge {})
        meta__212__auto__)
       G__1664)

richiardiandrea23:04:18

so I commented out my alter-meta!

richiardiandrea23:04:21

also, no tests executed because I need to set meta for my boot.test

richiardiandrea23:04:28

so still need to investigate

richiardiandrea23:04:35

but there might be something there

richiardiandrea23:04:41

I am putting meta on the return of def.........................................maybe not good

micha23:04:15

(do (deftask ...) (alter-meta! (var '~(first new forms)) assoc ::test-task ::test-me))

micha23:04:39

yesh that splits it up

micha23:04:49

i think i had to do that with defclifn also btw

micha23:04:29

i don't understand what's going on there

richiardiandrea23:04:36

sorry about that 😉 I am trying if it fixes

richiardiandrea23:04:59

hard to say, but alter-meta! has a big ! so...

micha23:04:23

in defclifn i had to do like this

micha23:04:48

(let [var# (def ...)]
  (doto var# (alter-meta! ...)))

richiardiandrea23:04:10

yes that's exactly my snipped above, I noticed that

richiardiandrea23:04:35

that's why I decided to comment it out

micha23:04:53

i never really knew why that makes a difference though

richiardiandrea23:04:11

I don't know, but not I see the description says "Atomically"

richiardiandrea23:04:25

which is always a big warning

richiardiandrea23:04:06

it is strange because def returns the var

micha23:04:28

maybe the var isn't created atomically

micha23:04:42

or someting

richiardiandrea23:04:23

yeah...don't really have a clue about that actually, it would be sage to open a JIRA issue probably

richiardiandrea23:04:50

wow i see no warnings!