This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-05-26
Channels
- # announcements (2)
- # beginners (46)
- # calva (16)
- # cider (5)
- # clj-kondo (1)
- # cljdoc (11)
- # cljsrn (4)
- # clojure (42)
- # clojure-dev (2)
- # clojure-spec (6)
- # clojure-uk (1)
- # clojurescript (18)
- # cursive (7)
- # datomic (18)
- # duct (1)
- # fulcro (11)
- # graalvm (1)
- # hoplon (9)
- # leiningen (1)
- # off-topic (8)
- # shadow-cljs (16)
- # spacemacs (9)
- # specter (3)
- # sql (33)
- # vim (3)
- # xtdb (8)
Hi! Does Crux offer multi-tenancy of graph stores? i.e. multiple graphs can be hosted side-by-side on a single machine host, partitioned separately?
Hello 🙂 ! There is no "native" multi-tenancy in Crux currently but certainly you can run multiple independent nodes on a single machine host quite easily. I am personally very interested in multi-tenancy and ways we could easily enable it. For instance Apache Pulsar has far better multi-tenancy properties than Kafka. However if your data isn't huge then simply enforcing a :graph/id
attribute on all your documents and queries would get you a partial solution.
@U899JBRPF Thanks so much for replying!
Yes the use-case for multiple graph stores is for a SaaS system that delivers something like a Graph filesystem (not described by myself, but my intentions are similar to the authors in here: https://www.iit.cnr.it/sites/default/files/gfs.pdf).
Essentially every user has a dedicated graph store comprised of some custom ontology of their design.
Regarding size of data: currently a ballpark of 10K nodes, 10K edges is my initial goal, I think that would serve as a good measure for how the more generalized system should perform (wherein nodes and edges size could be bigger).
Having 100's if not 1000's of graphs hosted on a single machine should be unexpensive in performance, especially in a configuration where most of the data just sits in storage and maybe 5/1000 users are actively engaging their respective graphs at the same time, loading those graph data into memory when reading / writing.
Do you think this is feasible with Crux right now with :graph/id
? Crux would be a great system since it has unique properties that make it especially suitable for it, only the performance characteristics have to match, which I'm not sure about.
Cool, this sounds very intriguing! Based on your description I would say that Crux is definitely fast enough to handle partitioning of the graphs via some kind of graph/id
In regards "loading those graph data into memory when reading / writing", the way Crux works makes this very cheap since indexes are already lazily retrieved from the KV store
We are launching our hosted benchmarks publicly very soon, which will give you a sense of the size of data sets and queries we've been testing Crux against. Much of this can already been inferred by looking at the /test
directory
@U899JBRPF Sounds awesome! gonna give Crux a try one of these weekends 🙂 thanks for listening.