2024-10-05 biff | Clojure Slack Archive

biff

2024-10-05T18:31:35.289489Z

It's that time again: https://biffweb.com/p/rocksdb-indexes-yakread/

🤘 1

🤘🏼 1

kpassapk 2024-10-10T16:35:53.862779Z

thanks!

👌 1

roklenarcic 2024-10-06T08:47:33.953809Z

You wrote:

But I will likely write some documentation about using Biff for "serious"/commercial projects that are expected to grow large.

In my experience, the biggest problem with scaling is Pathom. If you take a look over at pathom channel you will see a lot of conversations about how to optimize when number of resolvers gets over 100 and the result sets get a bit larger. Pathom is a neat idea, but it is obviously affected by N+1 query problem, trying to fit resolvers to data takes time (so number of resolvers worsens performance), paging is kinda annoying to implement, and most people have hard time wrapping their head around not having context when loading entities.

2024-10-06T09:07:14.311179Z

We've got well over 100 pathom resolvers at work and performance has been fine 🤷

roklenarcic 2024-10-06T09:34:43.811019Z

Great to hear. I’ve been putting off making a bigger site for a while now.

👍 1

kpassapk 2024-10-09T16:39:15.065669Z

How much of the indexing logic is relevant in XTDB v2?

2024-10-09T16:52:28.166349Z

a lot of it actually, now that indexes are implemented with rocksdb. the xtdb-specific parts are mainly: • take an xt transaction, transform it a bit, and pass it to the custom indexer functions • make a db object that combines an xt snapshot with a rocksdb snapshot at the moment v2 doesn't have an equivalent to xtdb.api/listen and xtdb.api/open-tx-log. After those get those get added (https://github.com/xtdb/xtdb/issues/2454, see also the last bit of https://discuss.xtdb.com/t/deployment-recommendations-for-small-apps/483/3?u=jacobobryant) it shouldn't be a lot of work to make this work with v2.

Nik 2024-10-08T01:31:00.479149Z

Looking forward to ‘next questions’ mentioned in post

2024-10-08T02:01:47.841839Z

me too 😉

Clojurians Log v2

biff