@micha Something broke when I did the release of notify.

When I do a local build, it works. But when notify is downloaded from clojars, it does not.

what fails?

I did just boot build-jar push-release during release

yeah that sounds ok

can't find the notify rpc file

--gotta recreate, one sec

notify/
notify/notification_api.clj
notify/notification_rpc.cljs.hl
META-INF/
META-INF/maven/
META-INF/maven/hoplon/
META-INF/maven/hoplon/notify/
META-INF/maven/hoplon/notify/pom.properties
META-INF/maven/hoplon/notify/pom.xml
META-INF/MANIFEST.MF

deleted local repository

those are the contents of the jar in clojars

then ran boot dev and it downloads the notify jar and fails with this:

C:\Users\Bill\Documents\GitHub\demos\castra-simple2>boot dev Retrieving notify-0.0.1.jar from https://clojars.org/repo/ Starting reload server on <ws://localhost:59568> Writing boot_reload.cljs... Starting file watcher (CTRL-C to quit)... Compiling Hoplon pages... ò simple_notifications\index.cljs.hl Adding :require adzerk.boot-reload to index.html.cljs.edn... Compiling ClojureScript... ò index.html.js adzerk.boot_cljs.util.proxy$clojure.lang.ExceptionInfo$ff19274a: No such namespace: notify.notification-rpc, could not locate notify/notification_rpc.cljs, notify/notification_rpc.cljc, or Closure namespace "notify.notification-rpc" data: {:tag :cljs/analysis-error, :from :boot-cljs} clojure.core/ex-info core.clj: 4593 adzerk.boot-cljs.impl/handle-ex impl.clj: 60 adzerk.boot-cljs.impl/compile-cljs impl.clj: 97 ... clojure.core/apply core.clj: 630 boot.pod/eval-fn-call pod.clj: 223 boot.pod/call-in* pod.clj: 230 ... boot.pod/call-in* pod.clj: 233 adzerk.boot-cljs/compile boot_cljs.clj: 71 adzerk.boot-cljs/compile-1/fn boot_cljs.clj: 125 clojure.core/binding-conveyor-fn/fn core.clj: 1916 ... Elapsed time: 9.791 sec

I'll rebuild locally and look at the working jar...

you didn't use the --manifest option to hoplon task when you built the jar, maybe?

?

there is no hoplon/manifest.edn file in that jar

ok

look at the contents of the jar

which?

there is the notify/notification_rpc.cljs.hl entry

in the notify jar from clojars

ok, still have that

but that needs to be in the hoplon manifest

because hoplon needs to compile it

so when you build the jar you should use the --manifest option to the hoplon task

i don't understand why it would work locally though

that's puzzling

locally I have a bunch of things defined in bot.build

https://github.com/hoplon/notify/blob/master/build.boot

both the pom task options and dev

again, I did just boot build-jar push-release during release

so the processes must have differed somehow?

some magic in dev that is not captured in build-jar push-release?

@micha are you saying I should have done a boot build-jar --manifest push-release instead?

what do you do locally?

boot dev

ah

ok yes

I should have done a boot build-jar --manifest push-release?

what you need to do is boot hoplon -m build-jar push-release

OK!

release 0.0.2 coming up!

when you put the .hl files in the jar, you need to have the hoplon compiler write the manifest, which is what the hoplon compiler will use to figure out which files from dependency jars will need to be compiled when you use the jar from another project

can I define this release in my boot.build?

you can make a deploy task if you like

deftask deploy-release, yes

(deftask deploy
  []
  (comp
    (hoplon :manifest true)
    (build-jar)
    (push-release)))

ah i see now

the dev task has the manifest business in there already

that's why it was working locally

many thanks. again!

--going to release and then test it!

ttfn

@micha I did not get too far:

clojure.lang.ExceptionInfo: java.lang.AssertionError: Assert failed: project repo is not clean (or (not ensure-clean) clean?) data: {:file "C:\\Users\\Bill\\AppData\\Local\\Temp\\boot.user4061139093341627544.clj", :line 29} java.util.concurrent.ExecutionException: java.lang.AssertionError: Assert failed: project repo is not clean (or (not ensure-clean) clean?) java.lang.AssertionError: Assert failed: project repo is not clean (or (not ensure-clean) clean?) boot.task.built-in/fn/fn/fn/fn built_in.clj: 784 adzerk.bootlaces/eval769/fn/fn/fn bootlaces.clj: 54 adzerk.bootlaces/eval808/fn/fn/fn bootlaces.clj: 62 boot.task.built-in/fn/fn/fn/fn built_in.clj: 716 boot.task.built-in/fn/fn/fn/fn built_in.clj: 618 boot.task.built-in/fn/fn/fn/fn built_in.clj: 342 hoplon.boot-hoplon/eval565/fn/fn/fn boot_hoplon.clj: 119 boot.core/run-tasks core.clj: 791 boot.core/boot/fn core.clj: 801 clojure.core/binding-conveyor-fn/fn core.clj: 1916 ... C:\Users\Bill\Documents\GitHub\notify>

@micha it looks like (hoplon :manifest true) might be putting something in the repository, like

_maven.repositories

notify-0.0.2.jar and

notify-0.0.2.pom.

and then the next step checks to see that there is nothing in the repository for that release.

I tried deleting the 0.0.2 folder, but the same thing keeps happening.

that's just warning you that you're trying to deploy a release without committing your changes in git

if the changes aren't committed then the git commit info in the pom.xml won't be correct

if you commit your changes you should be able to push the release

it'll also tag the commit for you, so you can push the tags to github too

oh

manifest.edn now contains ["notify/notification_rpc.cljs.hl"]

@micha It works! 0.0.2 is a good release.

So by dropping the v from the version, github's own release mechanism is disabled. Fine by me!

How many folks are storing dependencies in EDN files and reading them into build.boot files?

@seancorfield: if you are looking to create a task to slurp in that vector I was just looking for that 😛

Oh, that was easy... I was just curious as to whether it was common practice...

@micha: i thought I had the checkout task working, however upon further inspection im not sure the hoplon compiler is picking up the changed hl file still

Now that we're not tied to declarative project.clj files, we can use dynamic Clojure!

@flyboarder: i will merge here and test

might want to keep it out of master for now

We have pinned versions of several libraries now (in xyz.version files) and we map over the dependencies and fix the versions of those libraries to the pinned versions. It's really nice.

Moving to Boot has been a huge win all round for us, for flexibility.

eliminate the build tool, and just program (tm)

@flyboarder: for a truly seamless experience we may need to move the checkout task into boot itself

so the dependency loading machinery is aware of checkout dependencies

@micha: so even with my fix if the symbols are in deps it fails to build with the new files

i removed those lines so they are only in checkout and it works

im missing something

:thinking_face:

ah i see something

https://github.com/hoplon/boot-hoplon/blob/master/src/hoplon/boot_hoplon.clj#L94

should be something more like (->> "hoplon/manifest.edn" pod/resources (filter #(= "jar" (.getProtocol %))))

i fix

@flyboarder: 0.1.12 is in clojars now

but you still need to remove the checkout dep from your :dependencies

but at least it works then

@micha: still had to remove the symbol manually from the deps list, is it possible they are already loaded by the time checkout runs?

yeah

there's no way around that unless the underlying dependency stuff in boot is changed

i modified checkout to print before and after that line and the deps are being removed, but if they were there prior to being removed new file are not compiled

yes

that's the problem

it's able to see the old things on the classpath, too

but why does that prevent the hoplon compiler from picking up the changed files? I’m confused a bit

it's picking up the one with the same path in the jar

cljs picks up changes fine?

ah ok

the one that isn't changing

actually i should make sure

but isnt it?

well the one in the jar isn't changing

but the jar itself is changing?

and hoplon compiler sees the one in the jar and emits a cljs file

isnt that what the checkout task is watching for

yes the checkout task is dumping the .hl file into the fileset

but the classpath can have only one thing with that path

now there are two things with the same path in the classpath

the .hl file in the jar and also the .hl file that was extracted to the fileset

which one will it see?

it can only see one of them

it must be seeing the one in the jar instead of the one in the fileset

so that one will not change

so it’s only seeing the one from the first time it saw the file, even tho the jar gets updated and the new file pulled out

hm i think i was slightly incorrect there

1. dependency jar is loaded, containing manifest file

2. checkout task unzips dependency jar into temp dir, including the .hl file

3. hoplon task sees manifest file in jar and extracts the .hl file to the fileset

(now there are two .hl files with the same path, in different tempdirs)

4. the hoplon compiler compiles the .hl file

question: which .hl file will it compile?

well it’s obviously the first file is what we are seeing

the hoplon task overwrote the one from the checkout task, perhaps

it's puzzling

very

seems like the checkout task should be clobering the other things

which is what we want

aha!

https://github.com/hoplon/boot-hoplon/blob/master/src/hoplon/boot_hoplon.clj#L117

that's clobbering the file from the checkout

with the one hoplon extracted from the jar itself

but shouldnt that file be new everytime the jar is recreated?

yes it is

then hoplon replaces it with the old file again

oh

it stashes the .hl file from the jar in the tmp-hl dir

each time it comes around in the pipeline, the first thing hoplon does is add those tmp-hl files to the fileset

right and then adds that folder to source

yep

tricky

ok im following with what your saying, how is the extracted folder clobbering the new file? doesnt the extracted folder contain the new file? That line makes sense to me???

oh i see what you mean!!!

wow that is hard to spot

yeah pretty sneaky ofit

*of it

(you know you can edit a previous message, right?)

oh sorry

irc for too long

lol

well i have a fix

there is a :mergers option to add-source that we can use here to resolve conflicts

i can use the boot.pod/first-wins-merger

should be a simple fix

really though i think the optimal solution involves moving the checkout machinery deeper into boot

for it to really work transparently

im not sure i see how that helps, just so it knows more about dependencies?

it would know to not load that dependency from the jar in the first place

and unzip the jar instead

ah ok

it would maybe be cleaner to do what you originally suggested, like have boot treat all dependencies as checkout dependencies when the flag is set

it could be quite slow though if it needs to unzip a lot of jars

yeah i see that being a slow process for a project which already takes minutes to build

and that would require the new (possibly development) version being in your build file

i wonder what happens if i modify a jar in place, when it's loaded into a classloader already

probably jvm crashes

yeah i would imagine it would error on the next call

i think it might just be easier to write a task which loads the deps from a file, this allows me to process the prod and dev deps vectors for conflicts before they ever actually get added to the environment

ah, it's fixed

@seancorfield: I don't know about "many", but I do it - read-dependencies! and update-deps task here - https://gitlab.com/jaen/clj-cljs-presentation/blob/master/build.boot. Works pretty nice; as a bonus you get updating of dependencies without killing boot for free (most of the time that is, if there's a conflict you still have to restart it, since you can't unload jars).

I've been asking here whether it would be possible to run the whole thing inside a pod, so even restarting on conflicts could be avoided, but so far this doesn't seem to work exactly as I want it

(for example the boot-cljs compile task errors out complaining that files don't exist, which it doesn't do without running things inside a pod)

is this statement true? https://groups.google.com/d/msg/clojure/UyHXJtSaLC0/LpY99X4_CAAJ

@magomimmo: iirc the problem on win7 is that open files can't be deleted, which we must do to manage the filesets. In win10 MS relaxed this restriction

@alandipert: thanks. I'll forward it to the mlist

@alandipert: Good to know, maybe you could document that please in the windows section of boots README?

@sveri: have you tried with windows 10 btw?

@micha: No, I don't have one at hand. But @seancorfield reported that he tried it on W10 without problems

@jaen: exactly what I was looking for!!

@flyboarder: re: dependencies? Nice.

Though keep in mind it doesn't solve all cases, like I described

If you get the yellow warning about deps conflict, you have to restart anyway.

yeah, but looking at your example will update-deps run every time a resource file is changed?

Yes, I am aware of that, I think I've fixed that in another project, but I don't have it in a public repo

(defn- modified-files? [before-fileset after-fileset files]
  (->> (fileset-diff @before-fileset after-fileset)
       input-files
       (by-name files)
       not-empty))

(deftask lein-generate
  []

  (let [fs-prev-state (atom nil)]
    (with-pre-wrap fileset
      (when true ; (modified-files? fs-prev-state fileset #{"resources/dependencies.edn"})
        (util/info "Regenerating project clj...\n")
        (generate-lein-project-file! :keep-project true)
        (reset! fs-prev-state fileset))
      fileset)))

(deftask update-deps
  []

  (let [fs-prev-state (atom nil)]
    (with-pre-wrap fileset
      (when true ; (modified-files? fs-prev-state fileset #{"resources/dependencies.edn"})
        (util/info "Dependencies changed, updating...\n")
        (set-env! :dependencies (fn [deps] (read-dependencies!)))
        (reset! fs-prev-state fileset))
      fileset)))

This should be it

I think.

Though I'm unsure why I have modified-files? calls commented out '

hahah cool, thanks for that I’ll play around with it!

But in general it should work, I've lifted modified-files? from system.boot where it works.

Let me know if it does.

@jaen: it works! I made a slight modification so i can keep it out of my boot file.

Cool!