Fork me on GitHub
#boot
<
2016-08-02
>
samedhi04:08:07

I am attempting to run boot in Docker. I can create my image, and then run my development task. I -p to expose the port that nREPL is running on. However, when I try to connect using cider-connect to my docker container, I get a

[nREPL] Connection closed unexpectedly (connection broken by remote peer)
progn: Sync nREPL request timed out (op clone id 1)

samedhi04:08:41

Does anyone have success connecting to NREPL running in a Docker container from emacs?

micha04:08:29

@samedhi: did you try something like nc or telnet to see if the port mapping is correct?

samedhi04:08:44

@micha: no, I did not. Note sure how to proceed there. Working down that line though, I am noticing that I get the same message regardless of whether or not I have the repl running. However, if I actually close the container, I get the usual

[nREPL] Establishing direct connection to localhost:6800 ...
: [nREPL] Direct connection failed

micha04:08:17

i think you want to take emacs out of the system to debug this

timothypratley04:08:34

are you using lein-ring ?

micha04:08:37

first establish whether the nrepl server is started and is accessible from the host

timothypratley04:08:37

if so what version?

samedhi04:08:39

Which probably indicates that somehow docker has opened the port, but the result I am getting does not necessary have anything to do with the repl.

micha04:08:19

you can use telnet to see if the port is open in the host, like telnet localhost 12345

samedhi04:08:54

@micha: does this seem reasonable?

Stephens-MacBook-Pro:~ stephen$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                                                              NAMES
3dd83b017b67        firengine:0.1       "/docker-entrypoint.b"   3 minutes ago       Up 3 minutes        0.0.0.0:6800->6800/tcp, 0.0.0.0:7800->7800/tcp, 0.0.0.0:8080->8080/tcp, 8000/tcp   cranky_turing
Stephens-MacBook-Pro:~ stephen$ docker exec -it cranky_turing /bin/bash
root@3dd83b017b67:/app# telnet localhost 6800
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]’

micha04:08:22

i mean do the telnet thing from the host

micha04:08:36

not from inside the container

micha04:08:46

or are you running emacs inside the container?

samedhi04:08:54

no, emacs outside, clojure inside.

micha04:08:16

there's a lot of stuff that can go wrong probably in the emacs part

samedhi04:08:31

Stephens-MacBook-Pro:~ stephen$ telnet localhost 6800
Trying ::1...
Connected to localhost.
Escape character is '^]'.
Connection closed by foreign host.

micha04:08:35

so testing whether the container is making the nrepl port available would be a help

micha04:08:45

ok so that's good

micha04:08:55

do you see the .nrepl-port file?

samedhi04:08:44

root@3dd83b017b67:/app# more .nrepl-port
6800

samedhi04:08:51

yes, it is there

micha04:08:03

can you do boot repl -c in the host?

micha04:08:14

that should connect you to the nrepl server in the container

samedhi04:08:20

Ah, I was reporting the .nrepl-port in the docker container again. Sorry. There is also a matching file in my local filesystem left over from when I have run on my local machine, I suppose it will use that one.

micha04:08:47

ah i think that could be the problem

samedhi04:08:22

So, running from my Mac.

Stephens-MacBook-Pro:~ stephen$ boot repl -c
lojure.lang.ExceptionInfo: Assert failed: host and/or port not specified for REPL client
                            (and h p)
    data: {:file
           "/var/folders/f8/brrq5_n902jdb7m3bb8vsvsh0000gn/T/boot.user3392206988170510241.clj",
           :line 17}
  java.lang.AssertionError: Assert failed: host and/or port not specified for REPL client
                            (and h p)
            boot.repl-client/client  repl_client.clj:   10
                                ...
    clojure.core/apply/invokeStatic         core.clj:  646
                 clojure.core/apply         core.clj:  641
              boot.pod/eval-fn-call          pod.clj:  328
                  boot.pod/call-in*          pod.clj:  379
                                ...
                  boot.pod/call-in*          pod.clj:  382
        boot.task.built-in/fn/fn/fn     built_in.clj:  393
                                ...
    clojure.core/deref/invokeStatic         core.clj: 2228
                 clojure.core/deref         core.clj: 2214
     boot.task.built-in/fn/fn/fn/fn     built_in.clj:  397
     boot.task.built-in/fn/fn/fn/fn     built_in.clj:  394
                boot.core/run-tasks         core.clj:  938
                  boot.core/boot/fn         core.clj:  948
clojure.core/binding-conveyor-fn/fn         core.clj: 1938
                                ...
Stephens-MacBook-Pro:~ stephen$

samedhi04:08:49

Running from within the container.

root@3dd83b017b67:/app# boot repl -c

ConnectException Connection refused
	java.net.PlainSocketImpl.socketConnect (PlainSocketImpl.java:-2)
	java.net.AbstractPlainSocketImpl.doConnect (AbstractPlainSocketImpl.java:350)
	java.net.AbstractPlainSocketImpl.connectToAddress (AbstractPlainSocketImpl.java:204)
	java.net.AbstractPlainSocketImpl.connect (AbstractPlainSocketImpl.java:188)
	java.net.SocksSocketImpl.connect (SocksSocketImpl.java:392)
	java.net.Socket.connect (Socket.java:589)
	java.net.Socket.connect (Socket.java:538)
	java.net.Socket.<init> (Socket.java:434)
	java.net.Socket.<init> (Socket.java:211)
	clojure.tools.nrepl/connect (nrepl.clj:184)
	clojure.core/apply (core.clj:646)
	clojure.core/apply (core.clj:641)
Bye for now!

micha04:08:01

looks like something wrong with the nrepl-port file

micha04:08:10

maybe volumes need to be mapped or something

micha04:08:20

also osx has issues with mapping volumes correctly

samedhi04:08:16

Ok, I will try volume sharing the nrepl port file.

micha04:08:25

the fix involves using rsync to synchronize the volumes

micha04:08:39

you could try a socket repl

micha04:08:49

i dunno what kind of support cider has for that

micha04:08:59

but you could use inferior lisp with a socket repl

samedhi04:08:39

Stephens-MacBook-Pro:firengine stephen$ docker build -t firengine:0.1 .; docker run -it --rm -p 8080:8080 -p 7800:7800 -p 6800:6800 -e FIRENGINE_URL=$FIRENGINE_URL -v /Users/stephen/firengine:/app -v /Users/stephen/.m2:/root/.m2 firengine:0.1 /bin/bash
Sending build context to Docker daemon 18.35 MB
Step 1 : FROM 
 ---> f0882ff41177
Step 2 : EXPOSE 6800
 ---> Using cache
 ---> 953c3d126660
Step 3 : EXPOSE 7800
 ---> Using cache
 ---> 66683f71eead
Step 4 : EXPOSE 8000
 ---> Using cache
 ---> 4a4f72233f1f
Step 5 : EXPOSE 8080
 ---> Using cache
 ---> a756c09a3cf5
Step 6 : RUN apt-get update
 ---> Using cache
 ---> 368ac22c49ea
Step 7 : RUN apt-get install curl -y
 ---> Using cache
 ---> 21a2123c727b
Step 8 : RUN cd /usr/local/bin && curl -fsSLo boot  && chmod 755 boot
 ---> Using cache
 ---> 1485779636f3
Step 9 : WORKDIR /app
 ---> Using cache
 ---> 5360a2ad0943
Step 10 : ADD app.yaml app.yaml
 ---> Using cache
 ---> d895e454e121
Step 11 : ADD boot.properties boot.properties
 ---> Using cache
 ---> b50a44188ccf
Step 12 : ADD build.boot build.boot
 ---> Using cache
 ---> 6487476d0976
Step 13 : ADD html html
 ---> Using cache
 ---> 1bc2016e691d
Step 14 : ADD service_account.json service_account.json
 ---> Using cache
 ---> a3f3ebf7773c
Step 15 : ADD src src
 ---> Using cache
 ---> 36985ae79c30
Step 16 : ENV BOOT_AS_ROOT yes
 ---> Using cache
 ---> 20359ce4a54c
Step 17 : RUN boot deps
 ---> Using cache
 ---> a59c1a3123ac
Step 18 : CMD boot production
 ---> Using cache
 ---> 718a690aabd0
Successfully built 718a690aabd0
root@3ea72558eb2b:/app# boot development
Starting reload server on 
Writing adzerk/boot_reload/init982.cljs to connect to ...
Writing boot_cljs_repl.cljs...
Compiling 1/1 firengine.datastore...
Started HTTP Kit on 

Starting file watcher (CTRL-C to quit)...

Adding :require adzerk.boot-reload.init982 to main.cljs.edn...
nREPL server started on port 6800 on host 127.0.0.1 - 
Adding :require adzerk.boot-cljs-repl to main.cljs.edn...
Compiling ClojureScript...
? js/main.js
Writing target dir(s)...
Elapsed time: 13.318 sec

samedhi04:08:45

And then from a different tab

samedhi04:08:50

Stephens-MacBook-Pro:firengine stephen$ boot repl -c
SocketException The transport's socket appears to have lost its connection to the nREPL server
	clojure.tools.nrepl.transport/bencode/fn--1615/fn--1616 (transport.clj:95)
	clojure.tools.nrepl.transport/bencode/fn--1615 (transport.clj:95)
	clojure.tools.nrepl.transport/fn-transport/fn--1587 (transport.clj:42)
	clojure.core/binding-conveyor-fn/fn--4676 (core.clj:1938)
	java.util.concurrent.FutureTask.run (FutureTask.java:266)
	java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1142)
	java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:617)
	java.lang.Thread.run (Thread.java:745)
Bye for now!

samedhi04:08:00

… Yeah, it seems like things might be “seeing” each other ok, don’t really have any clear ideas on how to proceed beyond this.

samedhi04:08:07

@micha: I appreciate your help though. Getting late, but maybe I will try to figure out how I can start boot nrepl on 0.0.0.0 instead of 127.0.0.1, I have had odd issues where that seems to fix things with Docker in the past.

samedhi04:08:49

I have tried to create a minimal reproducing case here. https://github.com/samedhi/boot-nrepl-docker

micha04:08:38

@samedhi: you can use the --bind option like boot repl --bind 0.0.0.0

micha04:08:25

if nrepl is listening on 127.0.0.1 that would definitely cause a problem with exposing the service

micha04:08:04

you can get around that by connecting to docker's interface address from the host, but binding to 0.0.0.0 seems easier

torgeir07:08:11

I just did the same thing, a boot repl running within docker, using the following Dockerfile. Built with docker build -t torgeir-boot-repl ., ran with docker run --name torgeir-boot-repl -d -p 9000:9000 torgeir-boot-repl -i, connected from emacs with cider, works like charm.

FROM adzerk/boot-clj:latest
ENV BOOT_VERSION=2.6.0
ENV BOOT_CLOJURE_VERSION=1.8.0
RUN boot repl
ENTRYPOINT ["/usr/bin/boot", "repl", "--bind", "0.0.0.0", "--port", “9000"]

borkdude13:08:59

the repl task's -e option only accepts one form, correct? if I want to do multiple, I'd have to wrap in do?

borkdude13:08:10

Example: boot -d com.rpl/specter repl -e "(do (use 'com.rpl.specter) (use 'com.rpl.specter.macros))"

anmonteiro14:08:01

I’m trying to config clojure.tools.logging with Boot, but I don’t think Boot is picking my log4j.properties ?

anmonteiro14:08:19

is there any way I can check if the file is being correctly added to the classpath?

alandipert14:08:07

(require [ :as io]) (slurp (io/resource "log4j.properties"))

anmonteiro14:08:24

hrm. so it’s there

anmonteiro14:08:27

I can’t see any logs though 😞

anmonteiro14:08:25

hah! datomic bundles the no-op logger 🙂

tianshu16:08:19

Hi, if I have a dependencies with native library and I want to aot compile my program. where should I load the native library(I mean clojure.lang.RT/loadLibrary)? The aot compile always throws exception for this library has already loaded. I

tianshu17:08:10

Thanks, I'm going to have a look.

micha17:08:58

it's not really something that's related to boot, it's just how the JVM handles native library loading

micha17:08:35

i just avoid AOT as much as possible

tianshu17:08:37

I just want to AOT the entry namespace, for making the executable jar.

micha17:08:49

yeah i make a shim namepsace for that

micha17:08:00

that doesn't :require any other clojure namespaces

micha17:08:45

(ns app.main-shim
  (:gen-class))

(defn -main [& args]
  (require 'app.main)
  (apply (resolve 'app.main/-main) args))

micha17:08:15

then I AOT the app.main-shim namespace and set that as my Main in the jar manifest

micha17:08:46

all the code for the entrypoint is in app.main/-main

micha17:08:03

but you only need to AOT app.main-shim, which won't cause you any trouble

tianshu17:08:13

using require in main avoid AOT the app.main ?

tianshu17:08:05

I'll have a try

micha17:08:36

we have a boot task somewhere here that creates the shim etc. somewhere here at work

tianshu17:08:06

By the way, what does the word shim mean?😂

micha17:08:20

like if you have a 4 legged table

micha17:08:25

and one of the legs is a little short

micha17:08:32

you can stick a shim under it

micha17:08:37

to prop it up

micha17:08:09

a thing that goes between other things to make them fit properly

micha17:08:56

"shim without shame" is a saying in carpentry

micha17:08:06

which also applies to computers i think lol

tianshu17:08:40

I also found an issue with discussing java.library.path that is different from the one in lein.

micha17:08:02

leiningen configures everything statically

micha17:08:08

boot does everything dynamically

micha17:08:43

boot would need to modify that at runtime

tianshu17:08:54

understand

micha17:08:54

which requires hacks and stuff

tianshu17:08:24

I think provide a task may help. The hack is not short.

micha17:08:06

@doglooksgood: yes although we need to understand the situation better

samedhi17:08:09

@micha & @torgeir: Thanks for your help last night. I used :bind to set my docker host ip address to “0.0.0.0” Seems to work now. I should say that I receive a warning about nrepl incompatibilities when I cider-connect to my docker boot repl instance that I do not get when I connect to the same boot repl instance natively. I don’t see any side effect of this though, and I think it may be false as I am seeing CIDER and nREPL both have version numbers attached to them.

samedhi17:08:38

Connected to nREPL server - 
;; CIDER 0.12.0 (Seattle), nREPL 0.2.12
;; Clojure 1.8.0, Java 1.8.0_72-internal
<elided for brevity>
;; `cider-repl-display-help-banner' to nil.
;; ======================================================================
WARNING: CIDER's version (0.12.0) does not match cider-nrepl's version (nil). Things will break!

samedhi17:08:27

Maybe the warning is always checking against localhost?

samedhi17:08:02

Also, here is a project that demonstrates this warning difference. https://github.com/samedhi/boot-nrepl-docker

samedhi17:08:23

Anyway, thanks again, really appreciated.

seancorfield17:08:27

@samedhi: Sounds like the REPL instance running in Docker isn’t loading the CIDER middleware?

tianshu17:08:44

@micha With app.main-shim, should app.main be compiled into a .class file in the uberjar?

micha17:08:18

clojure's compiler is available at runtime, which is what's happening there

micha17:08:46

the idea is that you only compile that one class

micha17:08:28

clojure itself doesn't need to be compiled

tianshu17:08:36

When I execute jar with java -jar project.jar I meat

Exception in thread "main" java.io.FileNotFoundException: Could not locate opencv_clj/main__init.class or opencv_clj/main.clj on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.
How can I include this clj file? I only found the class of shim in jar.

tianshu17:08:48

meet. 😂

micha17:08:17

you will want something like (set-env! :resource-paths #{"src"} ..

micha17:08:33

so the source files will be included in the jar

micha17:08:54

you probably have (set-env! :source-paths #{"src"} ...

tianshu17:08:01

understand, the file in resource-paths will be copied and file in source-paths will be compiled

tianshu17:08:07

@micha It works, thank you very much!

tianshu17:08:32

I'll have a look

micha17:08:09

that explains the logic behid "resource" paths, "source" paths etc

oahner22:08:34

getting a weird behavior on windows; (System/getProperty "fake.class.path") contains a lot of paths that start with /

typedeph23:08:27

how does one setup cider to prefer boot over leiningen on boot projects?

typedeph23:08:09

everytime I try to cider-jack-in to my boot project I get lein.bat being used instead of boot.exe

anmonteiro23:08:10

@typedeph: (setq cider-preferred-build-tool “boot")

micha23:08:16

@oahner: is it causing a problem?

micha23:08:30

(.getPath (URL. %))

typedeph23:08:54

unfortunately that setq wasn't enough

typedeph23:08:03

still preferring lein.bat

typedeph23:08:35

I have BOOT_HOME pointing to .boot

typedeph23:08:48

I have a boot folder in my path that has boot.exe in the path

typedeph23:08:08

it is higher on the precedence list than leiningen

micha23:08:36

what happens if you temporarily move or rename lein.bat?

micha23:08:44

does it choose boot then?

typedeph23:08:03

if I recall it fails to launch, but let me try it now

micha23:08:28

and you can do like boot repl in the project dir without issues, right?

typedeph23:08:03

yes, but I will test it out again

typedeph23:08:49

Wrong argument type: arrayp, nil

typedeph23:08:53

on lein.bat rename

anmonteiro23:08:39

@typedeph: ah I forgot to say you need to be on CIDER 0.13

typedeph23:08:44

unfortunately I'm there to, 0.13.0 on melpa-stable

typedeph23:08:34

as for boot repl

typedeph23:08:38

what should I be trying

typedeph23:08:49

I have boot repled on the top level of the project

micha23:08:03

ah ok i was just bisecting the problem

typedeph23:08:13

I am unable to type (load "search"), my clj src file under src

typedeph23:08:35

Here is the project structure in case that helps

typedeph23:08:04

ai
-> boot.build
-> src
     -> search.clj

typedeph23:08:07

files are small

typedeph23:08:14

as I haven't done anything

micha23:08:21

i think you need (load "search.clj"), no?

typedeph23:08:38

; boot.build
(set-env!
 :source-paths #{"src"}
 :dependencies '[[org.clojure/data.json "0.2.6"]])

typedeph23:08:40

tried that also

typedeph23:08:14

FileNotFoundException Could not locate boot/search.clj__init.class or boot/search.clj.clj on classpath. clojure.lang.RT.load (RT.java:456)

typedeph23:08:40

file is

(ns search
  (:require [clojure.data.json :as json]))

(defn bfs
  []
  nil)

(defn dfs
  []
  nil)

micha23:08:27

if you do this it will work:

micha23:08:56

(load-string (slurp ( "search.clj")))

micha23:08:15

which indicates something in clojure.core/load that's not doing what we think it does

typedeph23:08:33

thought it should work also

typedeph23:08:49

unfortunately we get this: IllegalArgumentException Cannot open <nil> as a Reader. (io.clj:288)

micha23:08:01

(load "/search")

micha23:08:28

(defn load
  "Loads Clojure code from resources in classpath. A path is interpreted as
  classpath-relative if it begins with a slash or relative to the root
  directory for the current namespace otherwise."

typedeph23:08:35

same as above with slight token changes: FileNotFoundException Could not locate search__init.class or search.clj on classpath. clojure.lang.RT.load (RT.java:456)

micha23:08:54

ok try this final test

micha23:08:04

boot -BP -s src repl

micha23:08:08

then in the repl do

micha23:08:20

boot.user=> (load "/search")

typedeph23:08:27

seems I was in the wrong directory for (load "/search") anyways

typedeph23:08:32

let me try that first then this

typedeph23:08:55

ok, nope, now this

micha23:08:01

you need to run boot from the project root, usually

micha23:08:24

relative paths are relative to the working directory of the JVM

micha23:08:33

meaning the directory you're in when you start boot

typedeph23:08:39

progress I think? CompilerException java.io.FileNotFoundException: Could not locate clojure/data/json__init.class or clojure/data/json.clj on classpath., compiling:(search.clj:1:1)

micha23:08:50

yep it found the file

typedeph23:08:07

now the dependency fails to localize

micha23:08:56

boot -BP -d org.clojure/data.json -s src repl

micha23:08:35

or (set-env! :dependencies #(conj % '[org.clojure/data.json "0.2.6"])) in the repl

typedeph23:08:02

tried the first one, not sure why clojure 1.4 was retrieved in the process:

Retrieving maven-metadata.xml from  (1k)
Retrieving clojure-1.4.0.jar from  (3341k)

micha23:08:14

that's fine

typedeph23:08:26

I'll try the other alternative as well

micha23:08:48

it looks like stuff is working, though

micha23:08:09

you might simplify and just put like (println "hello world") in search.clj at first

micha23:08:26

that will be compiled and run correctly

typedeph23:08:34

globally or in a function?

micha23:08:43

it just compiles and evaluates it

micha23:08:59

it's unusual to use load

micha23:08:05

in clojure

typedeph23:08:41

any suggestions for how I can go about fixing cider to boot now?

micha23:08:55

normally you would make a namespace like

(ns search)

(defn f1 [x] ...)

typedeph23:08:08

sorry, continue please

micha23:08:21

then you'd compile that with boot.user=> (require 'search)

micha23:08:38

then you can do boot.user=> (search/f1 100)

micha23:08:55

load is a lower-level thing that's used internally by require

micha23:08:32

require will compile the clj source file and set up the namespace in the global namespace registry and stuff

typedeph23:08:47

require isn't working from just boot repl

micha23:08:55

and it also has other options, like (require '[search :as s :refer [f1]])

micha23:08:22

if you're seeing a repl then require isn't broken, because clojure is working

micha23:08:34

so there must be something amiss in your setup

typedeph23:08:52

I tried (require 'search) but I get a FileNotFound

micha23:08:53

perhaps starting with a blank slate, and empty directory

micha23:08:07

then you can do this:

micha23:08:19

make a directory named "src" like before

micha23:08:38

and in that directory a file "src/search.clj" with contents like this

micha23:08:02

(ns search)

(defn doit [x] (inc x))

(defn doit2 [x] (dec x))

micha23:08:11

then you can do this:

micha23:08:18

boot -s src repl

micha23:08:33

and when you have a repl going you can do:

micha23:08:52

boot.user=> (require '[search :as s :refer [doit]])

micha23:08:01

that should return nil if everything went ok

micha23:08:16

if something went wrong you'll see an exception thrown

typedeph23:08:18

be right back

typedeph23:08:28

I think I have a couple runaway jvm processes

typedeph23:08:55

also, this sounds like a dumb question (sorry I am a newbie) but is there anyway to kill a cider repl without buffer kill

micha23:08:09

i don't know much about emacs unfortunately

typedeph23:08:18

ok i'll be right back

micha23:08:37

but if you're new to clojure and boot and everything i'd probably recommend doing some stuff at a standalone repl first

micha23:08:47

just to lower the blood pressure

micha23:08:41

you can get a lot of stuff done learning clojure with a minimum of tooling, just with the boot repl