Fork me on GitHub
#xtdb
<
2019-05-26
>
anovick12:05:43

Hi! Does Crux offer multi-tenancy of graph stores? i.e. multiple graphs can be hosted side-by-side on a single machine host, partitioned separately?

refset12:05:23

Hello 🙂 ! There is no "native" multi-tenancy in Crux currently but certainly you can run multiple independent nodes on a single machine host quite easily. I am personally very interested in multi-tenancy and ways we could easily enable it. For instance Apache Pulsar has far better multi-tenancy properties than Kafka. However if your data isn't huge then simply enforcing a :graph/id attribute on all your documents and queries would get you a partial solution.

refset12:05:02

Is the use-case SaaS?

anovick12:05:06

@U899JBRPF Thanks so much for replying! Yes the use-case for multiple graph stores is for a SaaS system that delivers something like a Graph filesystem (not described by myself, but my intentions are similar to the authors in here: https://www.iit.cnr.it/sites/default/files/gfs.pdf). Essentially every user has a dedicated graph store comprised of some custom ontology of their design. Regarding size of data: currently a ballpark of 10K nodes, 10K edges is my initial goal, I think that would serve as a good measure for how the more generalized system should perform (wherein nodes and edges size could be bigger). Having 100's if not 1000's of graphs hosted on a single machine should be unexpensive in performance, especially in a configuration where most of the data just sits in storage and maybe 5/1000 users are actively engaging their respective graphs at the same time, loading those graph data into memory when reading / writing. Do you think this is feasible with Crux right now with :graph/id? Crux would be a great system since it has unique properties that make it especially suitable for it, only the performance characteristics have to match, which I'm not sure about.

refset18:05:48

Cool, this sounds very intriguing! Based on your description I would say that Crux is definitely fast enough to handle partitioning of the graphs via some kind of graph/id

refset18:05:19

In regards "loading those graph data into memory when reading / writing", the way Crux works makes this very cheap since indexes are already lazily retrieved from the KV store

🔥 8
refset18:05:11

We are launching our hosted benchmarks publicly very soon, which will give you a sense of the size of data sets and queries we've been testing Crux against. Much of this can already been inferred by looking at the /test directory

anovick21:05:47

@U899JBRPF Sounds awesome! gonna give Crux a try one of these weekends 🙂 thanks for listening.

😎 4