core-async

2024-04-04T14:16:15.611779Z

dropping buffer based on a condition? Hey team, is there a way you would recommend someone to build a 'custom dropping buffer' of sorts? I am something like this:

(def ch (a/chan (my-dropping-buffer (fn [el-to-add, els-in-buffer] (contains? els-in-buffer el-to-add))))
Here's the usecase I am thinking about: I have a system that sends commands to a channel that looks like: [:refresh id] If a refresh for a specific id hasn't been processed, there's no need to really add another refresh command.

2024-04-04T14:35:00.351259Z

Could you make an intermediate go loop to do the buffering? That seems sort of a common/intended way to do similar things. Is there a particular reason you think the functionality should be in a buffer? Alternatively maybe use https://clojure.github.io/core.async/#clojure.core.async/pub ?

Alex Miller (Clojure team) 2024-04-04T16:41:41.058689Z

If you care about subsequent ones, the dedupe transducer does this now

Alex Miller (Clojure team) 2024-04-04T16:41:57.455079Z

You could just add that to your channel

Alex Miller (Clojure team) 2024-04-04T16:42:50.079249Z

If you want some windowed version, the cgrand/xforms library has some tools for that

Alex Miller (Clojure team) 2024-04-04T16:43:47.107859Z

If you really are talking about the buffer, you could make your own by looking at the existing dropping-buffer - its protocol based (probably not actually what you want though)

2024-04-04T17:03:15.257559Z

I had thought of https://weavejester.github.io/medley/medley.core.html#var-dedupe-by but it wouldn't really be able to do "dedupe the thing but only while you're already processing the thing"

Jan K 2024-04-04T17:21:00.932389Z

I think I would make a go-loop that would aggregate requests into a priority-map (key=request, val=timestamp), which would act as the buffer. Then another loop to consume values from the map and do the work. You'd probably need an extra channel with a (dropping-buffer 1) and put some value in it for each new added request to wake up the worker loop if it went idle when the map was emptied.

Jan K 2024-04-04T17:34:41.967929Z

I suppose you could also make it into a custom buffer implementation based on the builtin FixedBuffer, just backed by a priority-map instead of a linked list

2024-04-04T19:51:33.805519Z

love being a clojurian Thanks for the support team! These are great ideas; I'll dive deeper and experiment