Fork me on GitHub
#xtdb
<
2020-06-01
>
martinklepsch03:06:49

Could there be a crux implementation on the client that act’s as a node so to say?

jarohen09:06:24

Hey @U050TNB9F 🙂 Not entirely sure what you're looking for here - we've got a few options that might be suitable: * the remote API client looks entirely like a node but defers transactions and queries to a remote node * our standalone setup runs a full node completely within the client JVM * more generally, all Crux nodes could be considered individual 'clients' of the golden transaction/document stores, with a fully replicated set of indices - bit tenuous, potentially 🙂 * we don't currently have support for (eg) a Crux node within the browser, short of using the HTTP API In short, depends on what you'd expect the client to be responsible for/have knowledge of, and how much it'd defer to something more central?

martinklepsch11:06:14

Maybe what I’m asking goes more into the direction of having an offline-first Crux Node in the browser and connecting it to a regular online node on demand - does that make sense?

jarohen11:06:32

Ah, it does, yes - although it's not something we've yet considered. Definitely interesting, though - I wonder how we'd go about that :thinking_face:

martinklepsch11:06:08

That’s where firebase and others really shine in a way. And I just thought: you have a storage agnostic multi node database written in a language that can be compiled to JS 😄

martinklepsch11:06:02

Partly inspired by this awesome publication https://www.inkandswitch.com/local-first.html

💯 4
Jorin11:06:44

I think I saw some projects around syncing datascript databases with datomic. Like this one https://github.com/metasoarous/datsync Since Crux works a bit different to datomic and datascript and Crux is so flexible, it would be definitely super interesting how much of the core can made platform agnostic so that the already flexible protocols could also be implemented in ClojureScript 🙂 One of the more famous DBs that does this syncing well is also CouchDB https://pouchdb.com/

ordnungswidrig12:06:17

With offline comes the question of conflict resolution.

martinklepsch12:06:01

Right, but doesn’t that also apply to distributed databases in general? Or is that where the transactor node comes in?

refset18:06:55

Personally I've been very interested in replication & CouchDB in particular since 2012 :) @U050CTFRT and I also have a mutual fascination with Lotus Notes which is the original pioneer and still a >$1B offline-first app platform 30+ years later. Notes was actually what inspired Damien to write Couch in the first place (he worked at IBM beforehand on the Notes formula engine) In terms of Crux in a web client, I believe the Index Store protocol work going on right now will go a long way to enabling cljc distribution. No promises. As things stand today there's too much off-heap Java byte buffer interop stuff in the heart of the query engine to make it approachable. Also related, I have been reflecting recently on how valid time + vector clocks could be a pretty powerful combo, see https://github.com/juxt/crux/issues/895 and the blog link in my comment there

ordnungswidrig18:06:55

I’ve been looking at operation transformations and related technology for a while now. It would be interesting to see if this could be applied to the rather generic tx model that crux implements. (Anybody remembering google wave?)

Jacob O'Bryant22:06:34

@U050TNB9F you might be interested in this: https://github.com/jacobobryant/biff It's inspired heavily by firebase. It handles syncing between a client-side atom and a crux db. it has its own tx & query formats which are translated to crux's. You can submit txes and queries directly from the client and they'll be authorized/rejected by application specific rules which you define. I'm using it in production, though "production" for me is quite small :). It doesn't have any offline-mode/local-first features at the moment, though I am super interested in the article you linked and the space it discusses. I've been focusing on a different approach to some of the same problems; roughly the same approach that the Solid project takes (https://solid.mit.edu/), though idk if that's active any more. The idea is to still use the server-centric model, but just give each user their own server. that gives you criteria 2, 4-7 (from the article, for anyone else). I wrote briefly about it here: https://findka.com/blog/migrating-to-biff/#multitenancy

Jacob O'Bryant23:06:03

That approach is also described in https://knightcolumbia.org/content/protocols-not-platforms-a-technological-approach-to-free-speech, under the "Protecting User Data and Privacy" heading.