aleph

p-himik 2022-12-21T20:13:53.113409Z

Just tried upgrading Aleph from 0.4.7-alpha5 to 0.6.0 (using Yada on top of it). It started giving this error:

java.lang.IllegalArgumentException: Don't know how to convert class io.netty.buffer.PooledSlicedByteBuf into class [B
	at byte_streams$convert.invokeStatic(byte_streams.clj:212)
	at byte_streams$convert.invoke(byte_streams.clj:173)
	at byte_streams$to_byte_array.invokeStatic(byte_streams.clj:797)
	at byte_streams$to_byte_array.invoke(byte_streams.clj:789)
	at byte_streams$to_byte_array.invokeStatic(byte_streams.clj:792)
	at byte_streams$to_byte_array.invoke(byte_streams.clj:789)
	at manifold.stream$map$fn__31455.invoke(stream.clj:621)
    [...]
It happens when Yada tries to convert a request's body into a byte array. Debugged for a bit, got it down to this code:
(do (require '[byte-streams.graph :as g])
    (import io.netty.buffer.PooledSlicedByteBuf)
    (require 'byte-streams)
    (require 'aleph.netty)
    (g/conversion-fn @byte-streams/conversions (g/type PooledSlicedByteBuf) (g/type (class (byte-array 0)))))
Running it on the old version gives me some conversion function. Running it on the new version gives me nil. What would be the right way to fix this?

✅ 1
p-himik 2022-12-21T20:29:20.310689Z

Created an MRE: https://github.com/p-himik/aleph-bytebuf-conversion-issue-test

p-himik 2022-12-21T20:34:52.614129Z

Seems like the difference has appeared in 0.5.0-rc1.

p-himik 2022-12-21T20:48:57.271599Z

Oh, crap... byte-streams is now split in two. Well that's a bloody nightmare. If I have any library that requires byte-streams, it will not be compatible with the new version of Aleph.

➕ 1
Matthew Davidson 2022-12-22T05:45:21.988669Z

See https://github.com/clj-commons/byte-streams/pull/66

p-himik 2022-12-22T06:39:09.887479Z

Thanks! Will try a bit later today.

iku000888 2023-06-03T08:41:40.094349Z

Yeah, reading between the lines of https://gist.github.com/malcolmsparks/bcfdcd9ae51e69aa3018c04d48f8749b it seems like there is something on the horizon 🤔

iku000888 2023-06-03T08:43:58.139289Z

esp. > I have since arrived at the opinion that what yada attempts is too ambitious.

iku000888 2023-06-02T11:33:20.897399Z

Hello, sorry for the blast from the past... I googled the following and brought me here

java.lang.IllegalArgumentException: Don't know how to convert class io.netty.buffer.PooledSlicedByteBuf into class [B
                                                                                                                    at byte_streams$convert.invokeStatic (byte_streams.clj:196)
                                                                                                                    byte_streams$convert.invoke (byte_streams.clj:162)
                                                                                                                    byte_streams.graph$seq_conversion_fn$fn__27758$fn__27765.invoke (graph.clj:314)
                                                                                                                    clojure.core$map$fn__5884.invoke (core.clj:2759)
                                                                                                                    clojure.lang.LazySeq.sval (LazySeq.java:42)
                                                                                                                    clojure.lang.LazySeq.seq (LazySeq.java:51)
                                                                                                                    clojure.lang.RT.seq (RT.java:535)
                                                                                                                    clojure.core$seq__5419.invokeStatic (core.clj:139)
                                                                                                                    clojure.core$map$fn__5884.invoke (core.clj:2750)
                                                                                                                    clojure.lang.LazySeq.sval (LazySeq.java:42)
                                                                                                                    clojure.lang.LazySeq.seq (LazySeq.java:51)
                                                                                                                    clojure.lang.RT.seq (RT.java:535)
                                                                                                                    clojure.core$seq__5419.invokeStatic (core.clj:139)
                                                                                                                    clojure.core$empty_QMARK_.invokeStatic (core.clj:6195)
                                                                                                                    clojure.core$empty_QMARK_.invoke (core.clj:6195)
                                                                                                                    manifold.stream.seq.SeqSource.take (seq.clj:42)
                                                                                                                    manifold.stream.graph$sync_connect$f__2887__auto____4902.invoke (graph.clj:272)
                                                                                                                    clojure.lang.AFn.run (AFn.java:22)
                                                                                                                    io.aleph.dirigiste.Executor$3.run (Executor.java:320)
                                                                                                                    io.aleph.dirigiste.Executor$Worker$1.run (Executor.java:62)
                                                                                                                    manifold.executor$thread_factory$reify__2329$f__2330.invoke (executor.clj:70)
                                                                                                                    clojure.lang.AFn.run (AFn.java:22)
                                                                                                                    java.lang.Thread.run (Thread.java:833)
I gathered from the links the issues were resolved in latest versions, bumped libs to the following and still seem to see the error.
[yada "1.2.15"]

                 ;; 
                 [aleph "0.6.2"]
                 [org.clj-commons/byte-streams "0.3.2"]
                 [manifold "0.4.1"]
And thanks @kingmob for heroically maintaining all these libraries!

iku000888 2023-06-02T11:51:28.083889Z

I downgraded the dependencies like this and the problem is gone (i.e. request body id properly consumed)

;; 
                 [aleph "0.4.7-alpha5"]#_[aleph "0.6.2"]
                 #_[org.clj-commons/byte-streams "0.3.2"]
                 [manifold "0.1.9-alpha3"]#_[manifold "0.4.1"]

Matthew Davidson 2023-06-02T11:54:16.721789Z

Hmmm.

Matthew Davidson 2023-06-02T11:55:07.482129Z

byte-streams is synced with the latest Aleph, but I've been busy with adding HTTP/2 and haven't made sure the latest manifold is synced up

Matthew Davidson 2023-06-02T12:01:22.964259Z

@iku000888 BTW, do you have a minimal example I could look at, with yada in the mix?

Matthew Davidson 2023-06-02T12:06:47.452819Z

Also, does moving yada after aleph/bs/manifold fix the issues?

iku000888 2023-06-02T19:48:56.503589Z

Thanks @kingmob! I did eventually figured out that yada's byte-streams was being picked up and excluding it from yada resolved the issue! https://clojurians.slack.com/archives/C0G922PCH/p1685706615663399 The project is not minimal so this is the best I can share at the moment 🙏

Matthew Davidson 2023-06-03T06:24:43.578159Z

Glad it worked out

Matthew Davidson 2023-06-03T06:25:53.231919Z

Unfortunately, it doesn't look like anyone's updated yada in 3 years, or I'd submit a PR