Fork me on GitHub
#meander
<
2021-03-08
>
jjttjj14:03:02

Hi I'm trying to implement an advanced "pub/sub" for core.async. I have an incoming stream of data. I'd like to be able to subscribe to "queries". Basically, for each message that comes in, check which of the registered subscriptions matches. But I'd like it to be done efficiently, so if multiple subscriptions share a match condition, this would only be applied once to the incoming message. I have an almost working example here: https://gist.github.com/jjttjj/ad9c6355d8b6741b43f23e90c7c77442 But I'm wondering if I could use meander here, though I haven't fully wrapped my head around how that would look.

(sub pub
  {:venue ?venue
   :type (me/or :done :open)
   :price (me/pred #(< % 100000) ?price)}
  ch)
if I have something like this, is there a way to compile that data such that i just get a set of all bindings and match conditions in a way that they can be appended to a larger set of match conditions? And then apply the whole set of conditions to some incoming data, and then get back a set of matching conditions that could be compared for equality to the conditions in the "subscription" match query?

jjttjj16:03:24

After learning a bit more meander, I think a more direct question might just be, is it possible to m/search based from a map of pattern->clauses?

(def pattern->clause (atom {}))
(swap! pattern->clause assoc '[?x 1] :x)
(swap! pattern->clause assoc '{:a ?a} '?a)
Then do something like:
(let [x (mapcat identity @pattern->id)]
  (eval `(me/search
           {:a 1}
           [email protected])))
which currently doesn't seem to be working edit: ok this does work sometimes, maybe not with dynamic clauses?

jimmy17:03:10

So I didn’t get to process all of this yet. For dynamic things we just started exposing an interpreter. But It is definitely not something we have focused on for performance. And it is no where near as battle tested as the non-interpreted version.

jimmy17:03:25

In fact, I just pushed up a fix for pred that someone discovered. If you are looking for efficiency as your primary attribute, I’m not sure it if is the right answer for you. But also, if you are looking for efficiency, you might want to think about how dynamic you want to be.

jjttjj17:03:59

Thanks! I'll look into the interpreter stuff a bit. And I still have a bit more thinking about my problem to do, and learning how meander works.