biff 2022-09-15 | Slack Archive

In 15 minutes: https://clojurians.slack.com/archives/C013Y4VG20J/p1663010085022369

Random thing I've been thinking about lately: it would be super nice to have a job queue library for XT, like https://github.com/ivarref/yoltq for datomic. I'm at a point with one of my projects where that would be a bit handy, albeit not hugely necessary yet. Maybe coming to a future Biff release!

♥️ 2

refset17:09:11

Count me excited :)

🙌 1

Jacob O'Bryant17:09:43

secretly hoping someone else will see this message and say "wow that sounds like a fun way to spend a weekend", but I will probably get to it myself eventually 🙂

🧠 1

Steven Deobald21:09:59

I will admit it's definitely crossed my mind (I need one too) ... but across many weekends now, so. 😉

Jacob O'Bryant21:09:37

Do it Steven, think of all the glory and honor you'd have!

Steven Deobald21:09:57

Ah, but you misplace me, Jacob. I only do thankless and dishonourable work. I'll have to wait for a less important plugin.

👑 1

Jacob O'Bryant17:09:20

Speaking of XT (I'll bump biff's XT version at some point soonish) https://clojurians.slack.com/archives/CG3AM2F7V/p1663243773601199

macrobartfast18:09:49

I’m beginning to make third party API calls as a result of user actions on the front end. Some of those will both return results to the user (after some processing) as well as populate the DB a bit more. Logging the event may be part of it too. Feel free to send me to #beginners, absolutely, but I thought I’d check here if there may be some examples of how this should be done asynchronously to allow the backend to keep moving. There seem to be a lot of notions in my mind that may be part of this (some of which may not apply here): core.async, channels, promises, messaging (kafka), async over redis (someone just mentioned that to me), etc. I thought I should check with the crew here before piggybacking approaches onto my backend that may not flow nicely with Biff. I’m guessing my vague worries about making things wait on the backend may apply to db interactions, too. Again, forgive my swirly of noob and Biff questions.

Jacob O'Bryant19:09:20

This actually relates to https://clojurians.slack.com/archives/C013Y4VG20J/p1663262911101589! i.e. the most reliable way is to use a job queue. your backend ring handler puts a job on the queue, and then a worker thread consumes the job and does the api call. However you can also get away with just wrapping the api calls in a future:

(defn some-handler [req]
  (future
   (do-api-calls-and-stuff req))
  {:status 200
   ...})

I'd recommend doing the future thing since it's good enough. When you're scaling up and need to improve reliability then you can add a queue 🙂

Jacob O'Bryant19:09:15

if/when biff comes with a job queue, it'll be easy to use that. but for now adding a queue is most likely more trouble than it's worth

Jacob O'Bryant19:09:37

and if you want some-handler 's response to include some data from the api call, you can just omit the future

macrobartfast19:09:35

sounds good! it doesn’t sound like, then, that I’m going to get into the weeds not rolling these things in for now on a simple low traffic site.

Jacob O'Bryant19:09:06

:thumbsup: nope, not going into the weeds at all

🙂 1

macrobartfast18:09:05

😳

macrobartfast19:09:50

I did try to root around to find a good approach in the cljaverse but there are, umm, seemingly a lot!

2022-09-15

Channels