Fork me on GitHub
#xtdb
<
2023-02-10
>
jmerrifield21:02:35

With traditional DBs, I'm used to minimizing round trips and data transfer, so with XTDB I've found myself working to craft a single, complete, yet minimal, pull query to traverse all the relationships I need to fulfill an operation (in particular, analyzing a GraphQL request to see which relationships and fields are being asked for and constructing exactly the right pull query to serve it). Given the local-index model though, I'm curious if that effort is warranted, and if it would be much different performance-wise to just lazily call (xt/entity) at the time I need to traverse each relationship?

phill02:02:22

Should a query's follow-up queries vanish into the woodwork, silently and unobtrusively furnishing whatever you need? A work-of-art along such lines was Datomic's Entity API, wherein an entity's member accessors retrieved linked entities, upon the first request for them. The Pull API came later to Datomic.

tatut14:02:53

I don’t really go out of my way to minimize “round trips”, I like being able to break things into functions that make sense and reuse them

tatut14:02:05

typically my query functions take a db as the first parameter, so they will have the same basis and xt/open-db should speed up multiple requests as per docs, but I haven’t done any measurements

refset22:02:44

I don't think it's a bad idea to aim for constructing big monolithic queries, however the performance can be harder to reason about and retain granular control over. That said, all database software should inherently attempt to gradually improve query planning (e.g. using data statistics) and execution (e.g. using data parallelism) over time, such that big monolithic queries could eventually run more efficiently than even the most carefully hand-optimised Clojure - we're certainly moving the XT engine in that direction over time. The advice to use open-db (per thread) is definitely the best way to regain the caching benefits of doing everything in a monolithic query.