This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2024-04-03
Channels
- # announcements (5)
- # babashka (8)
- # beginners (98)
- # biff (2)
- # calva (20)
- # cider (16)
- # clerk (2)
- # clj-kondo (20)
- # cljdoc (19)
- # clojure (90)
- # clojure-art (3)
- # clojure-boston (1)
- # clojure-europe (7)
- # clojure-nl (2)
- # clojure-norway (47)
- # clojure-uk (3)
- # clojurescript (10)
- # cursive (10)
- # data-science (1)
- # datalevin (1)
- # defnpodcast (1)
- # events (2)
- # fulcro (11)
- # gratitude (2)
- # honeysql (18)
- # hyperfiddle (11)
- # introduce-yourself (1)
- # jobs (2)
- # lambdaisland (4)
- # lsp (6)
- # malli (4)
- # membrane (3)
- # off-topic (58)
- # polylith (14)
- # portal (2)
- # releases (2)
- # ring-swagger (4)
- # tools-deps (8)
- # xtdb (8)
so in theory if one is gonna use electric to have a chat bot / ai friend, chat with / dialogue with you, how to make the results "stream" from the api endpoint?
i asked a similar question 6 months ago and it resulted in this discussion https://clojurians.slack.com/archives/C7Q9GSHFV/p1695368808283179
for the API side, you can use or reference this lib that supports streaming https://github.com/wkok/openai-clojure
Very wonderful of you to share so readily the successful result ty @U066TMAKS
We were advised to accumulate into an atom. Basically, an m/observe
which concats the result into a server-side atom, which is what we ended up doing.
The downside is that since you stream the accumulated result of "This",
"This is"
, "This is an"
, "This is an AI"
, "This is an AI respon"
, "This is an AI response"
and so on as the response is built up, the accumulated payloads ends up being the size of some power of the final length of the AI response. From what I measured, a 1.5kb response ends up costing 270kb on the wire. It works OK UX-wise, but keep an eye on your outgoing bandwidth costs if your thing gains traction.
here's a full implementation for reference: https://github.com/groundedsage/clj-chatbot-ui (ht @U05095F2K). it accumulates into an atom as well
Suppose on the server we had a reactive value chunks
containing a vector which gets extended as new values come in: ["This", " is ", " an", " AI", " respon", ...]
. If we tried (e/server (e/for [chunk chunks] (e/client (dom/text chunk))))
, I wonder what that would look like on the wire?
Gentlepeople who fiddle for a living, this article about a violin maker is really inspiring (found on HN) https://archive.ph/ymimZ
Here's an invalid program:
(defn not-a-continuous-flow [<f]
(m/ap
(m/amb
"prefix"
(m/?> <f))))
(e/defn Main [ring-request]
(e/client
(let [!a (atom 1)
a (e/watch !a)
<a (e/fn [] a)
>b (not-a-continuous-flow <a)
b (new >b)]
(prn b))))
#object[Error Error: Unable to build frame - not an object.]
Easily fixed of course with m/relieve. I'm curious, how does Electric even notice that this isn't a continuous flow? It seems like some discrete flows such as m/seed can be fed to Electric. Is the broken assumption here that the output flow won't be producing more values than the input flow?