Clojurians Log v2

well it wasn't docker settings because it's fixed without changing them, but unfortunately I can't break it again to work out what I did to fix it. I think it was caused by lsetxattr com.apple.lastuseddate#PS operation not supported errors during the tar process which were hard to read due to interleaving of warnings in the REPL, once I did xattr "-c" for all files before taring them it worked I think but now I can't reproduce it

anyway it works now and the code is the same, so it's not Contajners or Docker

Interesting. Well, lemme know if you can repro the phantom bug again. It did feel it’s something to do with the env.

ok I tested again by adding new files to the image build which hadn't previously been xattrd and it failed with Broken pipe again, so it is a MacOS specific problem. I don't know the actual cause but xattr -c before tar ing works

Possibly it comes from the difference of users owning things on the Mac and the docker vm. Tarring preserves these things and the vm possibly isn’t happy with the Mac things unless you xattr them

hmmm ok

The docker vm running Linux is likely the source of difference here

update: add --no-mac-metadata --no-xattrs to tar args

nice find, would you like to add a PR to the docs with this example for mac users? i think its useful

do you have a small repro for this?

the code that you wrote for this would help in debuggung

yes https://gist.github.com/Hendekagon/e747a13d91b6dde0578a1b4368e4d1ef I'm creating an image with files on the classpath for a project, which with all the dependencies ends up being a tar file around 500mb

cant see anything that seems to stand out. anything useful in the docker daemon logs maybe?

going by the error, the connection to the socket seems to be interrupted. also if you can send the full stacktrace, would help more

yep

java.net.SocketException: Broken pipe
 at org.newsclub.net.unix.NativeUnixSocket.write (NativeUnixSocket.java:-2)
    org.newsclub.net.unix.AFSocketImpl$AFOutputStreamImpl.write (AFSocketImpl.java:602)
    okio.OutputStreamSink.write (JvmOkio.kt:56)
    okio.AsyncTimeout$sink$1.write (AsyncTimeout.kt:102)
    okio.RealBufferedSink.emitCompleteSegments (RealBufferedSink.kt:256)
    okio.RealBufferedSink.write (RealBufferedSink.kt:147)
    okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSink.write (Http1ExchangeCodec.kt:311)
    okio.ForwardingSink.write (ForwardingSink.kt:29)
    okhttp3.internal.connection.Exchange$RequestBodySink.write (Exchange.kt:223)
    okio.RealBufferedSink.emitCompleteSegments (RealBufferedSink.kt:256)
    okio.RealBufferedSink.writeAll (RealBufferedSink.kt:195)
    unixsocket_http.impl.StreamingBody$_writeTo.invokeStatic (StreamingBody.clj:52)
    unixsocket_http.impl.StreamingBody$_writeTo.invoke (StreamingBody.clj:49)
    unixsocket_http.impl.StreamingBody.writeTo (:-1)
    okhttp3.internal.http.CallServerInterceptor.intercept (CallServerInterceptor.kt:62)
    okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.kt:109)
    okhttp3.internal.connection.ConnectInterceptor.intercept (ConnectInterceptor.kt:34)
    okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.kt:109)
    okhttp3.internal.cache.CacheInterceptor.intercept (CacheInterceptor.kt:95)
    okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.kt:109)
    okhttp3.internal.http.BridgeInterceptor.intercept (BridgeInterceptor.kt:83)
    okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.kt:109)
    okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept (RetryAndFollowUpInterceptor.kt:76)
    okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.kt:109)
    okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp (RealCall.kt:201)
    okhttp3.internal.connection.RealCall.execute (RealCall.kt:154)
    unixsocket_http.core$request.invokeStatic (core.clj:104)
    unixsocket_http.core$request.invoke (core.clj:85)
    contajners.jvm_runtime$request.invokeStatic (jvm_runtime.clj:59)
    contajners.jvm_runtime$request.invoke (jvm_runtime.clj:44)
    contajners.core$invoke.invokeStatic (core.cljc:96)
    contajners.core$invoke.invoke (core.cljc:67)
    hendekagon.docker$build_cmd_BANG_.invokeStatic (docker.clj:80)
    hendekagon.docker$build_cmd_BANG_.invoke (docker.clj:78)
    hendekagon.docker$build_BANG_.invokeStatic (docker.clj:102)
    hendekagon.docker$build_BANG_.invoke (docker.clj:101)
    hendekagon.docker$make_image_BANG_.invokeStatic (docker.clj:131)
    hendekagon.docker$make_image_BANG_.invoke (docker.clj:121)
    hendekagon.test.docker$eval6994.invokeStatic (docker.clj:289)
    hendekagon.test.docker$eval6994.invoke (docker.clj:289)
    clojure.lang.Compiler.eval (Compiler.java:7570)
    clojure.lang.Compiler.eval (Compiler.java:7525)
    clojure.core$eval.invokeStatic (core.clj:3229)
    clojure.core$eval.invoke (core.clj:3225)
    nrepl.middleware.interruptible_eval$evaluate$fn__968$fn__969.invoke (interruptible_eval.clj:87)
    clojure.lang.AFn.applyToHelper (AFn.java:152)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.core$with_bindings_STAR_.invokeStatic (core.clj:1990)
    clojure.core$with_bindings_STAR_.doInvoke (core.clj:1990)
    clojure.lang.RestFn.invoke (RestFn.java:428)
    nrepl.middleware.interruptible_eval$evaluate$fn__968.invoke (interruptible_eval.clj:87)
    clojure.main$repl$read_eval_print__9216$fn__9219.invoke (main.clj:437)
    clojure.main$repl$read_eval_print__9216.invoke (main.clj:437)
    clojure.main$repl$fn__9225.invoke (main.clj:459)
    clojure.main$repl.invokeStatic (main.clj:459)
    clojure.main$repl.doInvoke (main.clj:368)
    clojure.lang.RestFn.invoke (RestFn.java:1526)
    nrepl.middleware.interruptible_eval$evaluate.invokeStatic (interruptible_eval.clj:84)
    nrepl.middleware.interruptible_eval$evaluate.invoke (interruptible_eval.clj:56)
    nrepl.middleware.interruptible_eval$interruptible_eval$fn__999$fn__1003.invoke (interruptible_eval.clj:152)
    clojure.lang.AFn.run (AFn.java:22)
    nrepl.middleware.session$session_exec$main_loop__1067$fn__1071.invoke (session.clj:202)
    nrepl.middleware.session$session_exec$main_loop__1067.invoke (session.clj:201)
    clojure.lang.AFn.run (AFn.java:22)
    java.lang.Thread.run (Thread.java:1583)

feels like the docker daemon doesnt like this request for whatever reason. i think the daemon logs should reveal more. does this happen immediately or after sometime?

checking the log now....

it happens near the end

this looks relevant:

[2024-07-23T16:15:45.535293000Z][com.docker.backend.apiproxy][I] proxy << HEAD /_ping (6.770042ms)
[2024-07-23T16:15:45.536305000Z][com.docker.backend.apiproxy][I] proxy >> POST /grpc
[2024-07-23T16:15:45.536660000Z][com.docker.backend.apiproxy][I] proxy >> POST /grpc
[2024-07-23T16:15:45.541925000Z][com.docker.backend.apiproxy][I] Upgrading to raw stream
[2024-07-23T16:15:45.542349000Z][com.docker.backend.apiproxy][I] Upgrading to raw stream
[2024-07-23T16:15:45.546949000Z][com.docker.backend.apiproxy][W] Error forwarding raw stream from dockerd: write unix <HOME>/.docker/run/docker.sock->: write: broken pipe
[2024-07-23T16:15:45.547173000Z][com.docker.backend.apiproxy][W] Error closing raw stream to client: close unix <HOME>/.docker/run/docker.sock->: shutdown: socket is not connected
[2024-07-23T16:15:45.547245000Z][com.docker.backend.apiproxy][W] Error forwarding raw stream from dockerd: write unix <HOME>/.docker/run/docker.sock->: write: broken pipe
[2024-07-23T16:15:45.547283000Z][com.docker.backend.apiproxy][W] Error closing raw stream to client: close unix <HOME>/.docker/run/docker.sock->: shutdown: socket is not connected
[2024-07-23T16:15:45.547481000Z][com.docker.backend.apiproxy][I] proxy << POST /grpc (10.824208ms)
[2024-07-23T16:15:45.547713000Z][com.docker.backend.apiproxy][I] proxy << POST /grpc (11.4485ms)
[2024-07-23T16:15:45.551443000Z][com.docker.backend.apiproxy][I] proxy >> HEAD /_ping
[2024-07-23T16:15:45.551546000Z][com.docker.backend.apiproxy][I] proxy >> HEAD /_ping
[20

I'll continue with a fresh mind in the morning

ah i think it could be http timeouts, lemme look something up

also im assuming this is docker running in a vm? if so can you check if that has enough memory? when that goes OOM this happens too

this could be the most likely cause. try increasing that limit and see if that helps