This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-11-04
Channels
- # announcements (7)
- # aws (5)
- # babashka (72)
- # beginners (43)
- # calva (12)
- # cider (9)
- # clara (3)
- # clj-kondo (12)
- # cljdoc (32)
- # cljs-dev (10)
- # cljsrn (1)
- # clojure (78)
- # clojure-dev (50)
- # clojure-europe (17)
- # clojure-gamedev (8)
- # clojure-nl (1)
- # clojure-spec (30)
- # clojure-uk (3)
- # clojurescript (52)
- # core-async (1)
- # cursive (5)
- # datomic (8)
- # emacs (58)
- # events (2)
- # fulcro (5)
- # graalvm (7)
- # holy-lambda (37)
- # honeysql (9)
- # jobs (5)
- # leiningen (3)
- # lsp (7)
- # lumo (2)
- # malli (3)
- # meander (13)
- # membrane-term (64)
- # missionary (19)
- # music (3)
- # nextjournal (8)
- # off-topic (29)
- # pathom (16)
- # polylith (14)
- # portal (16)
- # re-frame (2)
- # reagent (5)
- # sci (14)
- # shadow-cljs (20)
- # spacemacs (6)
- # sql (1)
- # tools-deps (58)
- # vim (14)
I’m not sure if this is a missionary specific question but I have a sequence of chunks from the function above like
["thing 1\n\n"
"thing2: "
"also thing2\n\n"
"thing3\n\nthing4\n\n"]
These come in “one by one”, but the downstream consumer can only deal with “complete” items, e.g. it will fail when it receives
["thing 1\n\n"
"thing2: "]
(complete meaning ending with \n\n
)
So kind of I want to hold back the "thing2: "
string until I get the one that completes the segment. I got some help in #clojure using mapcat
and partition-by
but these approaches don’t really allow “holding back” an incomplete item.(defn sse-chunk-xform
"Returns a transducer that retains incomplete events (i.e. no trailing \n\n)"
[]
(fn [rf]
(let [incomplete (volatile! nil)]
(fn
([] (rf))
([result] (rf result))
([result el]
(cond
(and (nil? @incomplete)
(.endsWith el "\n\n"))
(rf result el)
(and @incomplete (.endsWith el "\n\n"))
(do (rf result (str @incomplete el))
(vreset! incomplete nil))
:else
(vswap! incomplete str el)))))))
@U050TNB9F in the front end or back end?
frontend
I started with https://github.com/oliyh/oxbow but I think the author may have only used it with small events where the chunks are usually complete
don't take it the wrong way, why torture yourself? the SSE specification is annoying. Let someone else solve the problem for you
you may get several messages before you emit, so you need to handle all the transitions
Fair comments, I originally just used EventSource
which was working fine but I need HTTP header auth and so I have to go the js/fetch
route. It’s a good idea to use a library for the parsing, not sure I really looked for one to do just that but I’ll take a look
Kind of a shame that the EventSource
API is so tied to the networking layer when most of the complexity seems to be in the stream parsing
I mean after reading the spec it doesn’t seem that painful