This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-03-02
Channels
- # aleph (6)
- # beginners (57)
- # boot (1)
- # cider (27)
- # clara (23)
- # cljs-dev (166)
- # clojure (287)
- # clojure-dev (23)
- # clojure-greece (1)
- # clojure-italy (2)
- # clojure-russia (13)
- # clojure-spec (34)
- # clojure-uk (36)
- # clojurescript (68)
- # core-async (63)
- # core-logic (1)
- # cursive (1)
- # data-science (1)
- # datomic (26)
- # duct (1)
- # emacs (10)
- # figwheel (8)
- # fulcro (2)
- # garden (16)
- # graphql (8)
- # hoplon (20)
- # jobs (2)
- # leiningen (10)
- # off-topic (16)
- # onyx (2)
- # portkey (5)
- # quil (1)
- # re-frame (63)
- # reagent (95)
- # reitit (6)
- # remote-jobs (1)
- # ring (6)
- # rum (1)
- # shadow-cljs (76)
- # spacemacs (26)
- # specter (11)
- # sql (7)
- # unrepl (68)
- # vim (2)
- # yada (2)
anything unholy about using core.async for stuff like this:
(defn trace-stream [prefix stream c]
(let [lines (line-seq (BufferedReader. (InputStreamReader. stream)))]
(go
(doseq [line lines]
(>! c (str prefix line))))))
(defn trace-process [p]
(let [c (chan 1000 (keep println))]
(trace-stream "out> " (.getInputStream p) c)
(trace-stream "err> " (.getErrorStream p) c)))
(<!! (trace-process (.start (ProcessBuilder. ["ls" "-la"])))) ; to out>
(<!! (trace-process (.start (ProcessBuilder. ["ls" "-0bogus"])))) ; to err>
?
Not sure if there are better ways of doing this in clojure but I wanted a) custom handling of out/err for the process b) no intra-line interleaving of output between out/errin that line-seq is lazy, you do io to realize it, and you are realizing it in a go block
imagine there is an event loop like javascript has, and go blocks are all callbacks being executed there
ok thanks for pointing me in the right direction - had a feeling there was something rotten there
you know there is a method on processbuilder to have it redirect ls from the process to the jvms io streams
@mbjarland: also that doseq can be replaced by onto-chan
@hiredman, yes I know but I want to be able to prefix lines to know which process and stream this is coming from
@mbjarland: also, even if you don't switch the doseq out for onto-chan, closing the channel when done tends to be a good pattern
@noisesmith thanks!
I think you would be better off spinning on a thread per stream and having it read and print out
I tried it, wasn’t printed atomically in my somewhat convoluted sample code at least…I can re-check
@hiredman as for consuming, doesn’t the (keep println)
consume everything fairly fast?
as a transducer, it doesnt' drive consumption
hmm…so somebody would still have to pull on the channel even though nothing will actually get let through?
@hiredman: won't a transducer that drops inputs keep the channel from backing up though?
especially considering all it takes to be correct is try to take a single value off the channel
(defn trace-stream [prefix stream]
(async/thread
(with-open [rdr (BufferedReader. (InputStreamReader. stream))]
(doseq [line (line-seq rdr)]
(println prefix line)))))
(defn trace-process [p]
(async/into [] [(trace-stream "out> " (.getInputStream p))
(trace-stream "err> " (.getErrorStream p))]))
(<!! (trace-process (.start (ProcessBuilder. ["ls" "-la"])))) ; to out>
(<!! (trace-process (.start (ProcessBuilder. ["ls" "-0bogus"])))) ; to err>
@hiredman but that still leaves us with interleaving right? (assuming that is actually a problem)
(defn trace-stream [prefix stream]
(async/thread
(with-open [rdr (BufferedReader. (InputStreamReader. stream))]
(doseq [line (line-seq rdr)]
(locking #'*out*
(println prefix line))))))
(defn trace-process [p]
(async/into [] [(trace-stream "out> " (.getInputStream p))
(trace-stream "err> " (.getErrorStream p))]))
(<!! (trace-process (.start (ProcessBuilder. ["ls" "-la"])))) ; to out>
(<!! (trace-process (.start (ProcessBuilder. ["ls" "-0bogus"])))) ; to err>
I dunno, it has been a while, but I think if you called (println (str prefix " " line))
it would also take care of interleaving
(future
(while true
(println "foo")))
(future
(while true
(println "bar")))
produces:
foo
barfoo
foo
foo
foo
foo
foo
ok, so the lock would be a better solution than using channels for this then. In the interest of learning something of the core.async fu, would you care to elaborate why?
(defn trace-stream [prefix stream out]
(async/thread
(with-open [rdr (BufferedReader. (InputStreamReader. stream))]
(doseq [line (line-seq rdr)]
(async/>!! out (str prefix " " line))))))
(defn trace-process [p]
(let [c (async/chan)]
(async/into [] [(trace-stream "out> " (.getInputStream p) c)
(trace-stream "err> " (.getErrorStream p) c)
(async/thread
(when-let [line (async/<!! c)]
(println line)))])))
(<!! (trace-process (.start (ProcessBuilder. ["ls" "-la"])))) ; to out>
(<!! (trace-process (.start (ProcessBuilder. ["ls" "-0bogus"])))) ; to err>
well if nothing else I found this example instructive for playing around with core.async
@hiredman: any reason to use doseq instead of onto-chan and a transducer there?
will onto-chan
close the channel when the seq runs out…i.e. we have two seqs and the first to finish might kill it
mbarbieri: it's optional
that would be a good case for a transducer, transforming the data as it passes through
@hiredman: cool - you know enough core.async stuff that I halfway suspected you knew some downside I was missing