Fork me on GitHub
#aleph
<
2023-05-07
>
p-himik15:05:50

Trying to migrate from Yada+Bidi to Ring+naked Aleph+Reitit. Hitting an issue with form parameters. The Aleph's README states: > Aleph follows the Ring spec fully, and can be a drop-in replacement for any existing Ring-compliant server. And maybe I'm doing something wrong but so far it doesn't seem to be the case when ring.middleware.params/wrap_params is used. ๐Ÿงต

โœ… 2
p-himik15:05:53

Getting this exception:

java.lang.IllegalArgumentException: Cannot open <<< stream: {:pending-puts 0, :drained? false, :buffer-size 87, :permanent? false, :type "netty", :sink? true, :closed? true, :pending-takes 0, :buffer-capacity 16384, :connection {:local-address "/127.0.0.1:8000", :remote-address "/127.0.0.1:44438", :writable? true, :readable? true, :closed? false, :direction :inbound}, :source? true} >>> as an InputStream.
	at $fn__11595.invokeStatic(io.clj:167)
	at $fn__11595.invoke(io.clj:167)
	at $fn__11569$G__11523__11576.invoke(io.clj:69)
	at $fn__11591.invokeStatic(io.clj:165)
	at $fn__11591.invoke(io.clj:165)
	at $fn__11530$G__11519__11537.invoke(io.clj:69)
	at $reader.invokeStatic(io.clj:102)
	at $reader.doInvoke(io.clj:86)
	at clojure.lang.RestFn.applyTo(RestFn.java:139)
	at clojure.core$apply.invokeStatic(core.clj:669)
	at clojure.core$slurp.invokeStatic(core.clj:7009)
	at clojure.core$slurp.doInvoke(core.clj:7009)
	at clojure.lang.RestFn.invoke(RestFn.java:439)
	at ring.middleware.params$assoc_form_params.invokeStatic(params.clj:33)
	at ring.middleware.params$assoc_form_params.invoke(params.clj:27)
	at ring.middleware.params$params_request.invokeStatic(params.clj:51)
	at ring.middleware.params$params_request.invoke(params.clj:39)
	at ring.middleware.params$wrap_params$fn__37411.invoke(params.clj:75)
	at reitit.ring$ring_handler$fn__63815.invoke(ring.cljc:328)
	at clojure.lang.AFn.applyToHelper(AFn.java:154)
	at clojure.lang.AFn.applyTo(AFn.java:144)
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at aleph.http.server$handle_request$fn__44146$f__29594__auto____44147.invoke(server.clj:177)
	at clojure.lang.AFn.run(AFn.java:22)
	at io.aleph.dirigiste.Executor$Worker$1.run(Executor.java:62)
	at manifold.executor$thread_factory$reify__29472$f__29473.invoke(executor.clj:70)
	at clojure.lang.AFn.run(AFn.java:22)
	at java.base/java.lang.Thread.run(Thread.java:833)

p-himik15:05:12

The relevant part of the middleware:

(defn assoc-form-params
  "Parse and assoc parameters from the request body with the request."
  {:added "1.2"}
  [request encoding]
  (let [params (if-let [body (and (req/urlencoded-form? request)
                                  (:body request))]
                 (parse-params (slurp body :encoding encoding) encoding)
                 {})]
    (-> request
        (assoc-param-map :form-params params)
        (assoc-param-map :params params))))

p-himik15:05:40

As you can see, it tries to slurp the body to parse the params. But the body is (I think) a Manifold stream and slurp doesn't know how to read it.

p-himik18:05:04

So far, worked around it by adding this middleware:

(defn convert-x-www-form-urlencoded-body-to-input-stream [handler]
  (letfn [(convert [req]
            (cond-> req
              (= "application/x-www-form-urlencoded" (req/content-type req))
              (update :body (fn [body]
                              (if (stream/stream? body)
                                (bs/to-input-stream
                                  (stream/map bs/to-byte-array (bs/to-byte-buffers body)))
                                body)))))]
    (fn
      ([request]
       (handler (convert request)))
      ([request respond raise]
       (handler (convert request) respond raise)))))

jeroenvandijk19:05:15

Not sure but maybe this is related to the :raw-stream? setting? https://github.com/juxt/yada/blob/master/README.md#troubleshooting-faq to be set to true and maybe this is causing a stream instead of an inputstream

p-himik19:05:49

Oh, good catch. Let me check...

p-himik19:05:38

That seems to be it! Thanks!

Matthew Davidson (kingmob)09:05:37

The one major place where Aleph differs from Ring is in supporting Manifold streams for the body. If you get a Manifold stream and want something else, check out the clj-commons.byte-streams functions. They're used by Aleph, and designed to transform Manifold streams into things like strings, byte arrays, InputStreams, etc

p-himik09:05:22

Thanks! Yeah, that's what I did in that middleware above. No clue whether it was the best way to do it since I just coped the Yada code almost directly. But doesn't matter as now I have simply removed the :raw-stream? parameter.

Matthew Davidson (kingmob)10:05:11

For future reference, byte-streams maintains a conversion graph, and can work out intermediate transformations, so it should suffice to just specify the desired end result, i.e., to-input-stream . In this case, there's no intermediate transformations needed. See line 354 of the main byte-streams file: def-conversion ^{:cost 0} [(stream-of ByteBuffer) InputStream]

๐Ÿ‘ 2