Fork me on GitHub
#core-async
<
2018-10-02
>
hlolli18:10:11

what's the troubleshoot when go or go-loop just freeze (do nothing) in clj?

hlolli18:10:56

I suspect the use of <!! elsewhere for stdout logs of a subprocess, but I'm not sure

noisesmith19:10:13

typically multiple go blocks with blocking IO or long running CPU intensive code

noisesmith19:10:37

the go thread pool is small, and doesn't expand, and you shouldn't be putting blocking operations in it

noisesmith19:10:18

to actually see what's stuck, the output of jstack or Control-\ (both are ways of dumping all vm stack traces) are usually informative

ghadi19:10:10

<!! can definitely cause thread starvation in the go-block pool

noisesmith19:10:06

I wasn't sure if @hlolli meant <!! inside go blocks or not, but yeah

hlolli19:10:27

reading now (no tag no notice 😛 )

hlolli20:10:59

ok, I believe that core.async bindings over java process builder was causing it, it was putting the stdout/stderr logs into a chan. The interface, both from https://stackoverflow.com/a/45293277/3714556 and https://github.com/rksm/subprocess suggest <!! from outside go macro.

hlolli20:10:34

and I actually don't want to read the logs, just spawn a process and not being blocked.

hlolli20:10:23

So I homebrew something like this

(defn silent-subprocess [proc-name arg-string]
  (let [ps (atom nil)]
    (async/thread
      (let [pbuilder (ProcessBuilder. (into-array String [proc-name arg-string]))
            process  (.start pbuilder)]
        (reset! ps process)))
    ps))
the atom so I can stop it, but it aint pretty.

noisesmith20:10:35

async/thread already doesn't block

noisesmith20:10:07

it returns a channel that eventually has the return value

hlolli20:10:44

yes, so I'm going to use that, but if I'd use the original implementation

(defn run-proc-async
  [proc-name arg-string callback]
  (let [ch (async/chan 1000 (map callback))]
    (async/thread
      (let [pbuilder (ProcessBuilder. (into-array String [proc-name arg-string]))
            process (.start pbuilder)]
        (with-open [reader ( (.getInputStream process))]
          (loop []
            (when-let [line (.readLine ^java.io.BufferedReader reader)]
              (async/>!! ch line)
              (recur))))))
    ch))
I guess it would jam if I don't take from ch

noisesmith20:10:26

the async/thread would block if you don't read from ch, but that won't effect the go thread pool

hlolli20:10:17

ok, can't have it blocking 🙂 I got myself too comfortable with node.js subprocesses, async, plug and play.

hlolli20:10:20

thanks for this Justin!