Fork me on GitHub

Hi all 👋, I'm pleased to announce an early alpha of relic is now available. 🎉 relic is an in memory database and data processing library with a closer-to-sql query dsl, constraints and incremental materialized views. I do not know if its a good, ok, or terrible idea - but I wanted something like this to exist so I could better find out. github: channel: #relic Questions, discussion (and contributions!) massively appreciated.

🎉 22
❤️ 5
Lukas Domagala11:01:41

This looks really interesting, great work! I know its still in alpha, so performance is probably still way worse than it could be. Disclaimers aside, how would the materialized views compare to something like re-frames subscribers? I know that replacing it with datascript is a lot slower, but this seems like more performant alternative.


I haven't published any benchmarks yet, but perf should be competitive with the competition, YMMV! Main things to be aware of would likely be memory usage and write performance if you materialize a lot of queries. Certainly a design goal is to be able to use relic with react and invalidate hooks / ratoms directly using tracked-transact. Relic is just a fancy signal graph itself so its quite natural to do this.


The upper ceiling on perf tuning for something like this is basically 'write a compiler' so lots of room at the top.

❤️ 1
Lukas Domagala11:01:14

thank you for the quick reply. I’ll defiantly play around with it, looks very interesting.


@U0GE2S1NH the usage with React is also something I’d be interested in, please keep sharing how that turns out 🙂


if you want to go down this rabbit hole happy to talk in #relic


I have a hacky demo of its use with reagent - I'd love to see somebody build a glue library that makes it a little less hacky. I don't currently spend a lot of time working on cljs ui's.

Lukas Domagala13:01:24

BTW, you mentioned that recursive queries aren’t implemented yet. Did you just not get around to it yet or is it going to be a really hard problem because of the signal graph cycles that would exist? just curious


I didn't get around to it. Its certainly tricky, but I think something can be done in the future.


i love that folks don't consider this to be a solved problem, and keep coming up with new and interesting approaches to solving it. nice work @U0GE2S1NH!


the curse of lisp I guess 🙂


you say curse, i say creative playground -sung to the tune of tomato, tomahto-

😄 2
☝️ 1

Nice work @U0GE2S1NH! Excited to see this!

❤️ 1

@U02EMBDU2JU > ...recursive queries aren’t implemented yet... is it going to be a really hard problem The math does indeed get quite a bit trickier for this. The typical trick used in for resolving recursive queries (recursively evaluating a rule till you reach a fixed point) is more or less the same trick used to perform incremental view maintenance, so you have to generalize the iteration from a well-ordered set of indices to a partial order (in other words, you don't just iterate over either data-update or recursive-rule-update, you have to iterate over both at the same time). The solution is differential view maintenance (differential dataflow/datalog/computation, etc). Frank McSherry et al have worked this out:

👍 1

This one is better at explaining some of the background theory:


Very interesting stuff, but also requires quite a bit of additional machinery.


I read some of those papers, my brain is a bit too smooth for all the math but I believe it can be done. In my head I see a specialised data flow node that closes over the fixed point recursion and intermediate indexes. You won't just be able to recur anywhere like a madman, I imagine some kind of CTE-ish syntax


Thanks for the links again @U05100J3V, will review when it's time. Would be super happy to take notes / ideas in an issue too!

👍 1

Yeah; The background is super heady stuff. This post may be helpful, since it's a bit more pragmatic, and assumes the underlying diff-dataflow machinery:

❤️ 1

It would be amazing if we had something akin to the timely-dataflow library; Then this wouldn't be too much work at all. But that's actually a tall order.


I'd recommend chatting with @U8YCZLZH6, who built clj-3df ( They left all the dataflow stuff to Rust/timely-dataflow, and wrapped as a clj api.

👍 1

Thanks so much! I hope once the dust has settled and I figure out how good an idea it actually is, to have another pass at the internals and align it with the best-of-breed approach from timely