This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-02-05
Channels
- # adventofcode (8)
- # aleph (42)
- # announcements (4)
- # beginners (157)
- # boot (4)
- # calva (1)
- # cider (6)
- # cljdoc (8)
- # cljs-dev (1)
- # cljsrn (3)
- # clojure (50)
- # clojure-europe (9)
- # clojure-italy (16)
- # clojure-nl (2)
- # clojure-russia (6)
- # clojure-spec (59)
- # clojure-uk (25)
- # clojurescript (20)
- # core-async (41)
- # cursive (30)
- # data-science (9)
- # datomic (20)
- # fulcro (28)
- # kaocha (9)
- # nrepl (1)
- # off-topic (3)
- # om (3)
- # pathom (10)
- # re-frame (8)
- # reagent (2)
- # ring-swagger (38)
- # rum (9)
- # shadow-cljs (209)
- # spacemacs (7)
- # tools-deps (11)
- # vim (6)
- # yada (2)
Has anyone ever implemented a ManyToManyChannel
to ByteBuf
conversion? Especially regarding to byte-streams/def-conversion
?
I have never implemented these conversions and have already spent way too much time into this
@niklas.collin Do you mean “to read until closed and convert to ByteBuf whatever was there”?
I have actually another problem too but even though that is related it is not strictly the same
You can try implement this by extending ByteSource
protocol, let me find you an example
Maybe there’s another approach for tackling the same problem? Or you’re okay with blocking version?
@niklas.collin as a matter of fact, i've done exactly this, but it's much simpler
i basically use manifold.stream/stream->seq
, and then reduce the whole sequence into a single bytebuf 🙂
(defn ^ByteBuffer concat-bytebufs
"Takes a collection of bytebuffers and returns a single large concatenated bytebuffer."
[xs]
(let [size (->> xs
(map #(.remaining %))
(reduce +))
bb (ByteBuffer/allocate size)
;; Puts a bytebuffer on the large bytebuffer
put-on-bb (fn [^ByteBuffer big ^ByteBuffer small]
(assert (>= (.remaining big) (.remaining small)))
(.put big (.slice small)))]
(run! (partial put-on-bb bb) xs)
(.flip bb)))
@lmergen bytes-streams
actually has conversions defined for (seq-of ByteBuffer)
. Meaning… concat-bytebuf
should work out of the box, no?
@niklas.collin Are we talking about ByteBuf
from Netty or ByteBuffer
from NIO?
@lmergen https://github.com/ztellman/byte-streams/blob/3e0822b58172b8d351b366213eda6090892e32b3/src/byte_streams.clj#L352
stream-of ByteBuffer
to InputStream means that everything else should go automatically
The only one conversion defined for netty’s ByteBuf
is https://github.com/ztellman/aleph/blob/d35d5bbbb090c31f3f53997ee1e0d68fc7b03044/src/aleph/netty.clj#L138-L142
So, that requires some work, but @niklas.collin you can use the same conversion as an example: https://github.com/ztellman/byte-streams/blob/3e0822b58172b8d351b366213eda6090892e32b3/src/byte_streams.clj#L352
But anyway. Basically what I would like to do is return a core async channel, then let it be automatically serialized as data comes in an async manner
If you use aleph
for handling websocket connection, everything you need to do is to “connect” your core.async channel to manifold.stream that represent the connection
Aleph handles serialization to ByteBuf
from pretty much anything byte-streams
can understand