Clojurians Log v2
aleph 2022-10-04
Hi, I have an app which uses aleph (0.4.5-alpha) + manifold (0.1.7-alpha6) streams to serve SSE. I noticed that response that is served is not being compressed, and given there is a lot of data it goes pretty slow. So I wonder if there is an easy way to enable the compression or I need to do it manually, and if the latter, what is the best way to do it. The code looks something like this -
(defn sse-handler
[req]
(let [event-stream (s/stream)
keep-alive-stream (s/periodically timeout (fn [] ":keepalive\n"))]
(s/connect keep-alive-stream event-stream {:timeout timeout})
(s/put! event-stream (str "data: " (write-transit (export-data)}) "\n\n"))
{:status 200
:headers {"Content-Type" "text/event-stream; charset=utf-8"
"Access-Control-Allow-Origin" "*"
"Cache-Control" "no-cache"}
:body event-stream})):compression? true in aleph http server and it looks like the usual responses are being indeed compressed but not SSE.
Thanks in advance.
:compression? true should apply here, too. Perhaps your SSE client request doesn't include the proper Accept-Encoding header?
Nice idea with the keep-alive-stream btw 🙂 Although it's a bit suboptimal in that it also sends keepalive messages even when regular events are being sent during the timeout period. I wonder if there's a straightforward way to account for that, too 🤔
As far as I know client request is set up properly, but I will recheck it, thanks
Hm looks correct indeed
Which aleph version are you on btw?
and can you post the full set of request headers? 🙏
aleph version is 0.4.5-alpha3, a bit old, but this is kinda legacy app I'm maintaining (so that keep-alive-stream idea was not mine, still thanks though 🙂 )
Ha!
Then you're missing 061ee43090fcf793942a6b24945af3e7d43b37b1 which might be the fix you're looking for
Try updating to 0.5.0 if possible
oh cool thanks a lot! this totally looks like it should help my case
🤞
Hmm but the PR that includes that commit doesn't seem too relevant: https://github.com/clj-commons/aleph/pull/357 But worth a shot regardless
anyway I'll try to upgrade aleph later in the day or tomorrow and will report tomorrow
The more relevant PR is probably https://github.com/clj-commons/aleph/pull/360, which wasn’t released until 0.4.5-alpha6
Don't think it is - that one affects websockets while @spariev is using SSE via the regular HTTP server
Ahh, right, right.
hi guys, I have an update
I've tried to update aleph to 0.5.0 (had to bump up manifold and netty along the way) but no luck so far - still no Content-Encoding header
What place in the aleph source code you would suggest to look into ? Maybe I can come up with the PR to fix this.
Truth to be told it is not my biggest problem right now though, but I might have time later to dig into it.
Thanks! At least we know that we're not chasing an obsolete bug here then 🙂 The deflater is added to the Netty pipeline https://github.com/clj-commons/aleph/blob/master/src/aleph/http/server.clj#L559-L562.
I don't see why it shouldn't work. I also already checked Netty's codebase for whether they have any special handling for text/event-stream but doesn't seem to be the case.
Thanks for the pointers, will look into it. I also was thinking it should just work, but somehow it doesn't for me.