However, a pool of secondaries is only eventually consistent in PG, correct?

@val_waeselynck, any given secondary is internally consistent with some prior point-in-time that the master was at. That's effectively equivalent with what datomic gives you, if your PG schema is built to allow point-in-time queries.

(which is, of course, rare and expensive).

@U2V9F98N8 not sure I understand, let me ask specifically: if you write to the master then read to the secondary, are you guaranteed to read your writes ?

You're guaranteed a read that accurately reflects a single point in time (in senses that aren't true with some "eventually consistent" databases), but not an up-to-date one.

@U0DJ4T5U1

in my usecase i will only get cache-misses

because i am running a relatively shot-lived batch-process where every entity is touched only once

This is something I'm really interested in too

I have developed a 'variant' of GraphQL for my application, with essentially the same reads semantics as GraphQL. While directly converting GraphQL queries to pull patterns is appealing, as soon as you need either nontrivial authorization logic or derived fields or parameterized fields, Datomic pull is no longer powerful enough

However, developing a basic GraphQL interpreter (as a recursive function) on top of the Datomic Entity API is rather straightforward, to the point you could maybe do it without Lacinia. But be aware that this basic interpreter may be too naive an algorithm - you may get performance issues as soon as some fields require a network call, and maybe even a Datalog query (Datalog queries have much more overhead than Entity lookups).

My point being that most production-ready GraphQL interpreters need some way of leaving rooms for optimizations. In the NodeJS world, this is done by making Field Resolvers asynchronous, which also lets you do batch queries with some wizardry; In Lacinia, this is done via sub-query previews. My strategy has been to give up on Field Resolvers (synchronous functions that compute a single field of a single entity) and adopt the more general 'Asynchronous Tabular Resolvers' (asynchronous functions that compute several fields of a selection of entities).

You should also have a look on the work done for Om Next / Fulcro - the querying semantics are similar.

I may end up open-sourcing the query engine I made one of the days - let me know if you're interested.

@val_waeselynck I also (started)developed one, but it never finishes (my "main" app will not use it. Just future plans for now)

I am working on one right now that we plan to open-source once it's stable

Hopefully soon

It has two components, one is a library to that hooks up to the entity API and resolves stuff for you, and the other is a program that you point at your db and it extracts a lacinia schema definition from it

There's also the umlaut project, which can go take a graphql schema and produce a datomic schema from it

https://github.com/workco/umlaut

@val_waeselynck appreciate the reply, I'd definitely be interested in seeing that code. For this specific project, I may be concluding that datomic is just not the right tool at present...I really just need a straightforward way to expose my database to a GraphQL client, and unfortunately this doesn't feel "straightforward" enough given my project's extremely tight timeline. We may end up moving over to JS on the backend 😕 I've tried convincing my front-end guys of potentially embracing the Datomic API completely, and letting go of Graph, but they are hell-bent on keeping their familiar tools...such is life.

@U6GFE9HS7 I haven't used Lacinia yet, but I do think it's still very straightforward with it 🙂 - if that' s enough to tip the balance, you should try and sell the workflow aspects of Datomic to the clientside guys

We are doing something very similar to this at our company. No GraphQL though. The frontend subscribes to pull patterns by sending a HTTP request to the backend with the pull pattern and eid. The backend responds with the result of that pull pattern. Additionally, the frontend is connected via SSE so the backend sends updates to the frontend any time any datoms that match the pull patterns the client has subscribed to have been updated in the DB.

@U083D6HK9 that's exactly what I've been spiking out recently on our latest project. Remove GraphQL as a middle-man, and just speak Datomic over the wire with real-time updates via the datom comparison you described. I'm unfamiliar with SSE though, and have been using WebSockets. SSE seems much more appropriate for this context, I'll have to look into it.

Yep it’s pretty clean. You’ll likely need permissions, which are easy to implement using Datomic filters.

Probably yes, maybe you make something clever using the Log API to touch some of the last used index segments... However, you should definitely try the Memcached approach first as it requires zero effort.

Side note about memcached: it's also very useful on dev machines, especially since the same memcached cache can be shared for dev (on your machine) and production (remote) databases

thanks!

Are there any potential issues to consider from running the same memcached instance for staging and production environments of the the same peer application?

nevermind. obviously they wouldn’t be sharing anything 🙂

They could be sharing most of their data if the staging data is obtained via restoring the production data; I cannot think of any issues except for the additional load