Fork me on GitHub
#boot
<
2016-10-25
>
ag00:10:40

hey guys… I’m trying to get into cljs-repl (start-repl) is yelling at me

Can't change/establish root binding of: *cljs-repl-env* with set
. What does this mean? I haven’t seen this last week

hlship16:10:22

So here’s a question w/ my prototype. I’m building up a Dockerfile, with Boot tasks each “supplying” content. My intermediate format is a simple EDN file. But that means each task has to a) read the EDN file b) update it in memory c) create temp directory d) write Dockerfile.edn to it and c) add the temp directory as a resource:

(defn edit
  "Edits an EDN Dockerfile using function f to transform it.

  The EDN file is read if it exists, and parsed. The function
  is applied. The result is written to a` new temp file, and
  added to the fileset as a resource."
  [fs f & args]
  (let [existing (read-edn fs)
        updated (apply f existing args)
        out-dir (tmp-dir!)
        out-tmp (io/file out-dir file-name)]
    (->> updated
         prn-str
         (spit out-tmp))
    (-> fs
        (add-resource out-dir)
        commit!)))

hlship16:10:58

The final step is to convert the Dockerfile.edn to a Dockerfile before building the image:

(deftask build-image
  "Runs Docker in a shell to build an image from the fileset."
  [i image-name NAME str "Name of the image to build."
   v version VERSION str "Version (appended to the image name)"]
  (let [work-dir (tmp-dir!)]
    (with-pre-wrap [fs]
      (let [fs' (commit! fs)
            df (-> (df/read-edn fs')
                   (df/command :files :add "Dockerfile" (str "Dockerfile-" image-name)))]
        (apply sync! work-dir (output-dirs fs'))
        (df/write df work-dir)
        (util/info "Building Docker image %s:%s in folder %s\n"
                   image-name version (.getPath work-dir))
        (util/dosh "docker" "build"
                   "-t" (str image-name ":" version)
                   (.getPath work-dir))
        fs'))))

hlship16:10:25

So … my question is … would it be outside the orthodoxy of boot to build up the EDN Dockerfile contents in a dynamic var or atom instead, rather than awkwardly using the file system to store the intermediate steps?

micha16:10:50

the value of using the filesystem is that the fileset is immutable

micha16:10:22

this allows boot to provide certain guarantees with respect to how state changes over time

micha16:10:45

for example, one important guarantee is that no task will see the effects of tasks that succeed it in the pipeline

micha16:10:00

this makes tasks idempotent as far as the pipeline is concerned

micha16:10:19

so you can use tasks like the watch task that call the next task multiple times

micha16:10:56

if, on the other hand, a task that comes after this task can mutate the global environment you now introduce all kind of issues with state

micha16:10:14

the transformation isn't from state A to state B anymore

micha16:10:21

it's from A to B the first time it runs

micha16:10:32

then from A+X to C the next time

micha16:10:45

where X is mutation of the global env by subsequent tasks

micha16:10:55

and Cis some new state that arises

alandipert16:10:22

it is Right and Proper to use the fileset to communicate between tasks

hlship16:10:41

This is true … but I have to wonder if there might be room to extend the fileset concept to include some state in-memory; I’d just as soon make changes to the fileset and not go through the motions of committing these files to the file system.

hlship16:10:12

Yes, this makes it complected.

micha16:10:14

there is nothing preventing you from mutating global env in your tasks, you can use alter-var-root or swap atoms etc

alandipert16:10:34

metadata on a file is another option

alandipert16:10:06

personally i would factor my hydrate/dehydrate code into a function or macro to make it less cumbersome to grab/mutate/emit the file in the task

micha16:10:09

mutating global env just interferes with composition of tasks

hlship16:10:29

That’s what edit above exactly is.

micha16:10:39

if you don't need to be able to have independent composable tasks then there is no reason why you can't swap atoms or whatever

alandipert16:10:58

i see. looks excellent :thumbsup:

hlship16:10:24

But I’ll look at the meta data option as well; I could create an empty Dockerfile and attach and edit meta on it instead.

hlship16:10:36

That would actually help with some other things down the road.

alandipert16:10:41

i guess one question is, do you need the dockerfile to be viable in between your modifications?

micha16:10:05

having the Dockerfile itself be the thing you pass has one advantage

hlship16:10:19

Right now, the Dockerfile doesn’t exist until the final step; Dockerfile.edn gets create on first step and modified repeatedly.

micha16:10:32

it makes it possible to compose other tasks that do things to dockerfiles with your tasks then

alandipert16:10:04

maybe it would be awesomer to instaparse the real dockerfile?

hlship16:10:22

But, eventually, I’ll be using some of this code for like 15 or 20 Docker images and doing more in parallel would be cool.

micha16:10:31

or invent a new docker image descriptor file, like a pom.xml for docker images

micha16:10:39

could be edn

alandipert16:10:50

seems like what his edn already is?

micha16:10:51

that's what we did with cljs applications with the .cljs.edn file

hlship16:10:56

@micha Dockerfile.edn is pretty much that. It’s a Dockerfile broken up into stages.

micha16:10:21

that could be a good standard to promote

hlship16:10:29

{:preamble [[:from "anapsix/alpine-java:8"]], :setup [[:maintainer ""] [:run "mkdir -p /user/local/java-agents"]], :files [[:add "launch.sh" "/"]], :postamble [[:run "chmod a+x lau\
nch.sh"]]}

micha16:10:39

there must be something that people can use to compose with other docker tasks is what i mean

micha16:10:59

the Dockerfile itself is a standard established by docker, so it's universal

micha16:10:15

a dockerfile.edn is a new thing, but you could establish it as a standard

hlship16:10:13

Ah, here’s the delight of working on a specific solution: I don’t have to go generalize everything! Perhaps later, we’ll be ready to clean it up and open source it, but I’ll do that only after I have a fully working solution that includes not just the Docker parts, but the Kubernetes parts.

micha16:10:41

sure, boot lets you be as general as you like 🙂

alandipert16:10:07

not sure if it's good, but the name and dog picture are top notch

micha16:10:17

that's where atoms come in, if you really don't need generality now you can make a more monolithic stack and use atoms or whatever

micha16:10:46

you might run into issues with the watch task for instance

micha16:10:50

but you don't need to use that

hlship16:10:54

“You don’t understand boot!” “No YOU don’t understand boot!"

alandipert16:10:05

that's the problem with this channel and boot in genera, we're always coercing people to abstract things

alandipert16:10:09

we have a plan to do better exception notification and it's based on the idea of tucking an atom into the fileset

alandipert16:10:52

i can imagine using the same idea to fuse together a bunch of dependent tasks to make a monolithic thing

micha16:10:18

yeah i'd just make something that does the job first i guess

micha16:10:37

and then make it idiomatic and more general

micha16:10:51

i mean i definitely would do that lol

hlship16:10:12

On another subject … is there anything in the task options to mark an option as required, to do declarative validation, or to supply an explicit default?

micha16:10:31

you add that in as assertions and set the defaults yourself

micha17:10:20

(deftask foo
  [b bar bool "a flag"
   z baz COUNT int "how many bazes (required)"
   c car NAME str "name of the car (default 'baby')"]
  (assert baz "you must provide the number of bazes")
  (let [car (or car "baby")]
    ...

micha17:10:20

the DSL is already hairy enough, i couldn't figure out how to accomodate those features without making things more insane

micha17:10:46

and the assertions etc are very straightforward and easy to reason about when they're explicit

micha17:10:17

the thing the DSL really helps with is the unification of cli opts and kwargs

micha17:10:21

and documentation

micha17:10:33

that would be painful to do with tools.cli or whatever

micha17:10:49

i mean it was painful 🙂 that's what the DSL compiles to

alandipert17:10:25

looks like there's still plenty of room to innovate with macros there too

hlship17:10:36

Back to meta; feels like the meta support is really there just to support the by-meta filter; there isn’t a sanctioned way to get the meta for a particular file back out. What would be the repercussions of just storing what I want as a namespaced key on the fileset itself?

micha17:10:05

the meta is just keys assoced onto the TmpFile records

micha17:10:19

so the way you get the meta for a file is just clojure.core/get

micha17:10:37

and the way you set meta is clojure.core/assoc

micha17:10:00

so what you describe is the current thing, basically

hlship17:10:23

I’m feeling like the namespaced key directly on the fileset is the way to go; it seems to work quite nicely and simplified my code quite a bit.

micha17:10:08

that will also work with the boot fileset meta stuff too

micha17:10:17

since that's how boot does fileset meta also

hlship18:10:37

So far, I’ve been launching boot from the command line. I’m trying to work w/ REPL inside Cursive now.

hlship18:10:55

(defproject gleam "0.0.1"
    ; A placeholer to allow Cursive to edit the project
  :dependencies [[org.clojure/clojure "1.8.0"]
                 [boot/core "2.6.0" :scope "compile"]])

hlship18:10:48

What am I missing here? boot.App doesn’t seem to be anywhere. https://github.com/boot-clj/boot/wiki/For-Cursive-Users wasn’t helpful here.

micha18:10:22

boot.App is on the classpath if your application entry point is boot.sh

micha18:10:37

boot.App is part of boot.jar

micha18:10:48

which is downloaded and dynamically loaded into the classloader by boot.sh

micha18:10:59

(and cached locally of course)

hlship18:10:14

Well, I’m trying to run inside Cursive so that is a bit of a challenge. I haven’t played with Cursive connecting to a REPL it didn’t start.

micha18:10:33

i'm not familiar with the ins and outs of cursive at the moment

micha18:10:44

but i know @onetom uses it with boot all the time

micha18:10:24

the idea i believe is to use the lein-generate task to generate a project.clj for cursive just so cursive has the classpath and whatnot

micha18:10:35

but to clarify about boot.App, boot relies on having all clojure runtimes created via boot.App.newPod() or whatever

micha18:10:05

and clojure cannot be on the classpath when boot.App is loaded

micha18:10:21

otherwise there is no way to achieve pod isolation

hlship19:10:21

Ok, so running on the REPL is out then ‘cause Cursive wants to do its thing as well.

micha19:10:27

i'm pretty sure onetom uses the repl

micha19:10:15

i use vim with vim-fireplace, and what i do is just start boot repl in a tmux tab separately from my vim tmux screen

micha19:10:36

vim-fireplace won't start its own repl if there is already a repl server that's running

micha19:10:04

so that way the editor doesn't need to be very clever, it just connects to the repl port that's in the .nrepl-port file

micha19:10:32

and i can restart the repl whenever i like simply by switching to the repl screen in tmux and ctrl-d

arohner20:10:55

my cljs repl task is:

(comp (watch)
        (cljs-repl/cljs-repl-env)
        (cljs :ids #{”public/js/dev”}))
. I notice that when I add a new .cljc file to src/and load it using Cider, the cljs process doesn’t pick it up. What do I need to do to fix that?