This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-02-24
Channels
- # aleph (19)
- # announcements (59)
- # asami (34)
- # aws (1)
- # babashka (17)
- # beginners (174)
- # bitcoin (11)
- # calva (16)
- # chlorine-clover (5)
- # cider (5)
- # clj-kondo (14)
- # cljsrn (13)
- # clojars (25)
- # clojure (124)
- # clojure-australia (1)
- # clojure-europe (48)
- # clojure-nl (4)
- # clojure-spec (1)
- # clojure-uk (40)
- # conjure (6)
- # core-async (25)
- # cursive (30)
- # data-oriented-programming (3)
- # datomic (14)
- # depstar (14)
- # emacs (3)
- # graalvm (27)
- # helix (1)
- # honeysql (25)
- # hoplon (3)
- # jobs-discuss (6)
- # kaocha (3)
- # lsp (109)
- # lumo (1)
- # malli (5)
- # meander (21)
- # music (1)
- # pathom (1)
- # re-frame (4)
- # reitit (1)
- # remote-jobs (1)
- # reveal (11)
- # rewrite-clj (3)
- # shadow-cljs (42)
- # spacemacs (15)
- # sql (13)
- # startup-in-a-month (4)
- # tools-deps (45)
- # vim (16)
- # xtdb (23)
- # yada (1)
Hello, I have a question. I am trying to set up a server that can receive and process multipart messages. My attempt is with aleph with yada resources. But I have difficulties in processing the manifold.stream in the response. Do you have some tips and tricks?
This is the code I have:
(yada/listener
["/some-route"
(yada/resource {:methods
{:post {:consumes "multipart/form-data"
:produces "application/json"
:response (fn [ctx]
(let [boundary (last (clojure.string/split (get-in ctx [:request :headers "content-type"]) #"="))]
(try (->> (get-in ctx [:request :body])
(yada.multipart/parse-multipart boundary 10000 1000)
(manifold.stream/transform (xf-bytes->content))
manifold.stream/stream->seq)
(catch Exception e
(println (.getMessage e)))))
(clojure.data.json/write-str {:success true}))}}})]
{:port 1234
:raw-stream? true
:executor (flow/utilization-executor 0.9 64)})
what do you want to do with the stream @eelke? note that stream->seq
should probably come with a warning - it's a blocking function
Hey @mccraigmccraig, thanks for the response. I would like to process what comes through the stream. I can be that the client sends large multipart requests, containing tens of images or videos, and the server needs to handle this
@(d/chain (get-in ctx [:request :body])
#(manifold.stream/map bs/to-byte-array %)
#(manifold.stream/reduce conj [] %)
bs/to-string)
This does result in a string but it does not always contain everything that was send, I think
ha, i'm recalling a very bad evening spent debugging...
question - do you have {:raw-stream? true}
specified in your aleph.http/start-server
opts ?
if you don't, flaky multipart parsing ensues
doh, yeah
i've not been using the stream-based multipart interface - what does yada.multipart/parse-multipart create a stream of ?
we've been using the PartConsumer interface ... but iirc that's because that came first and we've been using it since the beginning - i would have preferred a stream-based interface
i'd have a look around for an example of what the canonical yada resource should do now... PartConsumer wasn't the easiest to get working