Clojurians Log v2

@(p/process {:continue true
               :in "{\"foo\": \"bar\"}"
               :out :string
               :err :string}
              "bash -c" "echo $1")

I know the :in goes to the bash command which is not what we want, any suggestions how to make it goes to the echo one

I'm missing something because even without bash -c it prints $1\n , like it's escpaing the arg

this seems to be almost there:

(p/sh {:continue true
         :in "{\"foo\": \"bar\"}"}
        "bash" "-c" "echo $1")

well, I asked eca and it worked 😅

;; Send input via STDIN using :in.
  ;; Note: in bash, $1 is a positional argument; it does NOT read STDIN.
  ;; To read STDIN, use `read` (single line) or a loop.
  (p/sh {:continue true
         :in "{\"foo\": \"bar\"}\n"}
        "bash" "-c" "read -r x; echo \"$x\"")

  ;; If you specifically want to use $1, pass the value as an argument (not via :in):
  (p/sh {:continue true}
        "bash" "-c" "echo \"$1\"" "--" "{\"foo\": \"bar\"}")

good answer ;)

it's working great so far, thanks duckie

The problem is this:

(pb "tee" in-sha1sum)

so maybe it should be this:

(with-open [in-sha1sum (-> (process {:out "file.sha1"} "sha1sum")
                           :in io/writer)]
  (pipeline
   (pb {:in (io/file "file.gz")}
       "gunzip")
   (pb {:err in-sha1sum} "tee")
   (pb {:out "file.xz"}
       "xz")))

nice trick, thanks, let me think about that

I'd recommend adding run! check so you don't close the writer before the process has ended.

(with-open [in-sha1sum (-> (process {:out "file.sha1"} "sha1sum")
                           :in io/writer)]
  (run! check
        (pipeline
         (pb {:in (io/file "file.gz")}
             "gunzip")
         (pb {:err in-sha1sum} "tee")
         (pb {:out "file.xz"}
             "xz"))))

the trick is that tee writes both to stderr and stdout, right?

If I understand it correct — yes. For now it just an excercise how to force a program to use another program's input instead of a file

like https://www.gnu.org/software/bash/manual/html_node/Process-Substitution.html

This also works:

(-> (process "echo" "hello")
    (shell "cat"))

But that's different: it connects stdout of echo to stdin of cat. While process substitution allows to substitute stdin of a process2 so process1 will think it's a file.

to make process1 think it's a file, use io/file explicitly

oh I see what you mean, yeah that doesn't work here unless you write to a temporary file yourself

which you can do with (fs/file (fs/temp-dir) "1") or so

nice, thanks. I'll try it.

I don't think this is going to work out here though

in the bash example, unless you block the tee call to wait for it to finish writing the tmp file

Thanks for youy help, it's less mystical for me now. I'll continue to work on it tomorrow.

how check works in this case: in which case?

check should wait a process'es finish isn't it? So, we have proc1 → tee ./file → proc2. Let's assume tee writes into the ./file slowly than proc2 consumes it's input. So proc2 already finished, but tee still not.

proc2 won't finish until it consumes all input

So, does proc2 finished means all the pipeline finished.

yes

proc2 can consumes input faster than tee writes a data into a disk. That's my concern. But I think I'll do an experiment and test it tomorrow.

Again, thanks for your help.

that might be an issue. you could just wait for tee to finish before kicking off sha1sum on the tmpfile

but how it works with stderr is also fine I guess