aleph 2022-10-04 | Slack Archive

spariev10:10:50

Hi, I have an app which uses aleph (0.4.5-alpha) + manifold (0.1.7-alpha6) streams to serve SSE. I noticed that response that is served is not being compressed, and given there is a lot of data it goes pretty slow. So I wonder if there is an easy way to enable the compression or I need to do it manually, and if the latter, what is the best way to do it. The code looks something like this -

(defn sse-handler
  [req]
  (let [event-stream (s/stream)
        keep-alive-stream (s/periodically timeout (fn [] ":keepalive\n"))]
    (s/connect keep-alive-stream event-stream {:timeout timeout})
    (s/put! event-stream (str "data: " (write-transit (export-data)}) "\n\n"))
    {:status 200
     :headers {"Content-Type" "text/event-stream; charset=utf-8"
               "Access-Control-Allow-Origin" "*"
               "Cache-Control" "no-cache"}
     :body event-stream}))

I've enabled :compression? true in aleph http server and it looks like the usual responses are being indeed compressed but not SSE. Thanks in advance.

dergutemoritz11:10:09

:compression? true should apply here, too. Perhaps your SSE client request doesn't include the proper Accept-Encoding header?

dergutemoritz11:10:21

Nice idea with the keep-alive-stream btw 🙂 Although it's a bit suboptimal in that it also sends keepalive messages even when regular events are being sent during the timeout period. I wonder if there's a straightforward way to account for that, too :thinking_face:

spariev11:10:29

As far as I know client request is set up properly, but I will recheck it, thanks

dergutemoritz11:10:01

Hm looks correct indeed

dergutemoritz11:10:28

Which aleph version are you on btw?

dergutemoritz11:10:47

and can you post the full set of request headers? 🙏

spariev11:10:21

aleph version is 0.4.5-alpha3, a bit old, but this is kinda legacy app I'm maintaining (so that keep-alive-stream idea was not mine, still thanks though 🙂 )

dergutemoritz11:10:32

Ha!

dergutemoritz11:10:47

Then you're missing 061ee43090fcf793942a6b24945af3e7d43b37b1 which might be the fix you're looking for

dergutemoritz11:10:12

Try updating to 0.5.0 if possible

spariev11:10:34

oh cool thanks a lot! this totally looks like it should help my case

dergutemoritz11:10:53

🤞

dergutemoritz11:10:04

Hmm but the PR that includes that commit doesn't seem too relevant: https://github.com/clj-commons/aleph/pull/357 But worth a shot regardless

spariev11:10:19

anyway I'll try to upgrade aleph later in the day or tomorrow and will report tomorrow

👍 1

Matthew Davidson (kingmob)08:10:16

The more relevant PR is probably https://github.com/clj-commons/aleph/pull/360, which wasn’t released until 0.4.5-alpha6

dergutemoritz09:10:59

Don't think it is - that one affects websockets while @U07M31NA0 is using SSE via the regular HTTP server

Matthew Davidson (kingmob)09:10:49

Ahh, right, right.

spariev11:10:15

hi guys, I have an update I've tried to update aleph to 0.5.0 (had to bump up manifold and netty along the way) but no luck so far - still no Content-Encoding header What place in the aleph source code you would suggest to look into ? Maybe I can come up with the PR to fix this. Truth to be told it is not my biggest problem right now though, but I might have time later to dig into it.

dergutemoritz12:10:15

Thanks! At least we know that we're not chasing an obsolete bug here then 🙂 The deflater is added to the Netty pipeline https://github.com/clj-commons/aleph/blob/master/src/aleph/http/server.clj#L559-L562.

dergutemoritz12:10:06

I don't see why it shouldn't work. I also already checked Netty's codebase for whether they have any special handling for text/event-stream but doesn't seem to be the case.

spariev12:10:25

Thanks for the pointers, will look into it. I also was thinking it should just work, but somehow it doesn't for me.

dergutemoritz13:10:47

Keep us posted on your investigation!

👍 1

2022-10-04

Channels