dropping buffer based on a condition? Hey team, is there a way you would recommend someone to build a 'custom dropping buffer' of sorts? I am something like this:
(def ch (a/chan (my-dropping-buffer (fn [el-to-add, els-in-buffer] (contains? els-in-buffer el-to-add))))
Here's the usecase I am thinking about:
I have a system that sends commands to a channel that looks like: [:refresh id]
If a refresh for a specific id hasn't been processed, there's no need to really add another refresh command.Could you make an intermediate go loop to do the buffering? That seems sort of a common/intended way to do similar things. Is there a particular reason you think the functionality should be in a buffer? Alternatively maybe use https://clojure.github.io/core.async/#clojure.core.async/pub ?
If you care about subsequent ones, the dedupe transducer does this now
You could just add that to your channel
If you want some windowed version, the cgrand/xforms library has some tools for that
If you really are talking about the buffer, you could make your own by looking at the existing dropping-buffer - its protocol based (probably not actually what you want though)
I had thought of https://weavejester.github.io/medley/medley.core.html#var-dedupe-by but it wouldn't really be able to do "dedupe the thing but only while you're already processing the thing"
I think I would make a go-loop that would aggregate requests into a priority-map (key=request, val=timestamp), which would act as the buffer. Then another loop to consume values from the map and do the work. You'd probably need an extra channel with a (dropping-buffer 1) and put some value in it for each new added request to wake up the worker loop if it went idle when the map was emptied.
I suppose you could also make it into a custom buffer implementation based on the builtin FixedBuffer, just backed by a priority-map instead of a linked list
love being a clojurian Thanks for the support team! These are great ideas; I'll dive deeper and experiment