Fork me on GitHub
#clojure
<
2023-04-05
>
henrik09:04:51

What’s state of the art for piping an InputStream to a core.async channel? My current stab looks like this:

(with-open [rdr ( (get res :body))]
  (doseq [item (line-seq rdr)]
    (put! ch item))
  (close! ch))
As always with InputStream, I’m worried that I’m missing subtleties regarding buffering and so on. I’m expecting 500-1000 lines in rapid succession, so ch is buffered to 1000. onto-chan! seems to stall after the first message.

rolt09:04:45

you'd need to wait for all the items to be processed before closing the file (with-open [... ] (<!! (onto-chan!! ch (line-seq rdr))))

p-himik09:04:50

If it's in a go block, the code in the OP should be fine.

henrik09:04:48

Ah, @U2FRKM4TW, I think that’s it. I forgot a bang/go-block.

p-himik09:04:03

Correction - you also have to replace put! with >!.

henrik09:04:53

Yep, for the go variant.

p-himik09:04:47

There's also onto-chan! - might be more appropriate. Will close the channel for you by default.

p-himik09:04:52

Note the comment about blocking in its docstring though. Depending on the nature of that buffer, might make more sense to use onto-chan!!.

rolt09:04:55

i wasn't clear: if you go the onto-chan! road you need to make sure you wait for the items to to be processed by onto-chan! before closing the file

p-himik09:04:49

By the time the with-open block is done, all the data will have been processed, so it's fine. (assuming later comments)

rolt09:04:31

no i'm pretty sure onto-chan! and onto-chan!! returns immediately

☝️ 4
henrik09:04:52

Ah, I see. The doseq route seems more straightforward then. No further logic wrt waiting for the reader/input stream to be done.

p-himik09:04:01

Ah, yeah, right. I misread the "I wasn't clear" part as if you were talking about the initial scenario.

rolt09:04:14

that's why i do (<!! (onto-chan!! .. )) in my first answer

henrik09:04:34

Won’t that consume messages from the channel?

rolt09:04:50

no onto chan retruns an other channel, just to check for completion

henrik09:04:13

Ah, I see.

rolt09:04:14

(just like a go block returns a channel, thread returns a channel, etc

henrik09:04:11

Yes, this works fine. Thanks!

emilaasa09:04:46

I'm moving over to slingshot in my project, and I'm curious on how you would incrementally introduce it. Currently I have a top level try and catch Exception statement with cleanup functions that I'd like to keep for the time being, and then start working on the "leaf" exceptions where throw+ can help me with the handling. Currently the handling of my throw+ exceptions is only going to consist of more specific log statements. Is it reasonable to log and then re- throw from the new try+ error handlers that still need that cleanup logic, and in that case what would you throw? Or am I thinking about this in the wrong way?

seancorfield11:04:46

I'm curious what your use case is here? Slingshot hasn't been touched in over 8 years and the only contributor still active in the community is @U0NCTKEV8 who hasn't suggested using Slingshot at work... I'd be suspicious of code that relies so heavily on exceptions that it "needs" Slingshot I think...?

emilaasa12:04:40

Interesting! Slingshot is already used heavily in this project, so perhaps it's more accurate to say that I want to limit the variation in how exceptions are handled, and I'm not particularly attached to either the vanilla way or slingshot.

mpenet12:04:45

if I can suggest an alternative : https://github.com/exoscale/ex (I am slightly biased but...)

stopa19:04:50

Bit of a longshot, but does there happen to be a kind of "data visualizer" I could use in Clojure? Mainly, I want to convert a tree-style data structure, like so:

{:data "foo"
 :child-nodes [{:data "bar", :child-nodes []}]}
Into an image that shows a tree, with arrows

phronmophobic19:04:14

There's a bunch of ways to do this. Do you have an example or a sketch of what you want the output to be? The first thing that comes to mind are the various graphviz wrappers.

❤️ 2
lukasz19:04:42

it's built-in into the core:

(require 'clojure.inspector)
(def i (future (clojure.inspector/inspect-tree your-data))

😮 2
jamesleonis20:04:05

+1 for GraphViz, but if your looking for a REPL experience check out #C0185BFLLSE.

stopa15:04:38

Soo cool -- thank you team! I'll play with these.