datomic

mkvlr 2025-04-10T09:07:26.166239Z

do folks here have guidance on implementing application level retry for transact so it works with an HA setup?

2025-04-10T13:59:18.297809Z

It’s not datomic specific, but you can easily use fusebox for this https://github.com/potetm/fusebox?tab=readme-ov-file#retry

helios 2025-04-14T13:15:00.691309Z

How would that work with the following statement in https://docs.datomic.com/operation/deployment.html#peer-failover: > Not all transactions are idempotent, and transactions may have succeeded, even if the communication back to the peer failed. It seems that relying on application logic that uses return value of transact isn't enough to know whether something succeeded or not, but one has to use datomic/sync, although I'm not sure how

2025-04-14T21:43:57.964529Z

If a transaction might have failed, d/transact will throw. If your transaction is idempotent, you can just retry on exception. If the transaction isn't idempotent, you can either make a transaction function that makes it idempotent, or you can get the latest db on the client-side and recalculate your transaction. d/sync is used for when two different nodes are interacting and need to be sync'd to some version of the db. I don't think it's helpful here.

2025-04-10T13:07:28.930369Z

What are you all doing for working with your databases in production? Are there any good utilities out there to be aware of? For viewing the data, the Datomic Console is perfect: https://blog.datomic.com/2013/10/datomic-console.html. I can run this on my prod server, set it up behind some password protection, and I'm good to go for examining my DB. But for making changes (retracting accidental transactions, fixing customer data in emergency circumstances), I'm scratching my head for the best approach. I can imagine two options: - Provisioning a port with an nREPL connection to my application so I can jack in to the application and invoke database commands just like during local development. The most flexible option, but comes with obvious risks. You can secure that port behind some kind of auth mechanism, but it still introduces a direct line into the heart of your whole application. - Running the REST server, putting that behind an auth mechanism, and sending transactions over HTTP: https://docs.datomic.com/reference/rest.html. It is deprecated, but it strikes me as ... easier? Less flexible than a full REPL session, but with an interface and the opportunity to log requests.

favila 2025-04-10T13:26:57.047069Z

Peer server

favila 2025-04-10T13:29:06.475059Z

I'm surprised you are having good luck with console, because I have not. Once you scratch the surface of its abilities I run into all sorts of problems.

favila 2025-04-10T13:29:15.693659Z

e.g. its query parsing

favila 2025-04-10T13:29:26.179229Z

these are things I have seen:

favila 2025-04-10T13:31:30.154999Z

• command to start an instance inside your vpc that can start a peer process with a remote repl with whatever classpath you want. Tunnel with ssh • a web-based admin application, with or without a generic query textarea. Buttons and pages and stuff for what you need. (This is really great for adding domain-specific queries, views and ops) • "backoffice" rest endpoints with handers that do what you want. Ad-hoc is disallowed, you must write a handler when you want to do something new. (I see this in regulated environments where no one is allowed open connections to production) • Run a peer-server, ssh-tunnel the client connection from a local repl with a client. Works really well for ad-hoc querying and manual tx fixups, but a drag if your app uses the peer api because a lot of code can't be shared.

favila 2025-04-10T13:36:58.870289Z

I haven't seen it, but leaning into datafy/nav via morse may be cool too. Include operation thunks on datafy maps, and you have a very object-graph-like experience without the overhead of full webapp (html, rest handlers, http verbs, etc)

ghadi 2025-04-10T13:56:27.672909Z

I usually deploy an administrative endpoint that is tuned for the particular scenario

cch1 2025-04-10T14:05:09.248289Z

For completness: with Datomic Cloud, this is not an issue since you can start a remote cloud client in your REPL.

2025-04-10T14:48:32.481919Z

The ability to visually navigate my data and click around inside of references is huge for me, and the console shines at this out of the box. Datafy/nav would be cool to investigate though if it can arbitrarily navigate around such things in a console-like way

2025-04-10T14:49:25.222639Z

The console can be a bit annoying when you try to write a query that would work in your application, but doesn't work in the console. I can live with it 😅

2025-04-10T14:50:33.974819Z

SSH tunnels into a peer process I think will be a good starting point, maybe with bespoke admin endpoints sprouting up as time goes on and common production data operations or needs become obvious

favila 2025-04-10T14:52:33.956339Z

FWIW, I find biting the bullet and writing a web-based admin seems to pay dividends in the end, especially if non-devs are involved in the application's care and feeding

✔️ 2
2025-04-10T14:53:09.907149Z

Indeed they will be

2025-04-10T14:53:34.540569Z

Having me on call to be the code monkey for this kind of stuff will get increasingly annoying over time

2025-04-10T14:54:22.074279Z

I don't suppose there's an example you're aware of out there for a Datomic-specific web admin interface? Something to reference or start with?

favila 2025-04-10T14:54:48.778679Z

nothing I can show you, but there's nothing special about datomic

favila 2025-04-10T14:55:38.692419Z

if anything it reduces the friction immensely--you can grab any web framework, doesn't need an orm

favila 2025-04-10T14:58:44.841309Z

don't worry about perf, don't worry about frontend js/SPA. Use a big box css framework, out of box ring middleware, a routing library, render hiccup, sprinkle in some some htmx if you really don't want to reload the page

➕ 1
cch1 2025-04-10T14:59:22.650469Z

We've done just that: written an Admin UI using HTMX around datomic. Cheap, effective.

cch1 2025-04-10T14:59:41.442549Z

(but devs still tend to use the Datomic Cloud client from the REPL for complex stuff)

2025-04-10T14:59:46.195239Z

Retool could be an interesting option there: https://retool.com I've had a lot of success with Retool for SQL-based apps, maybe I'll give it a try.

2025-04-10T15:00:39.451629Z

@cch1 we're already building the app in HTMX! So we're pros at Datomic + HTTP forms + HTMX already 😆 It's been a fabulous combination. So much faster to build an application than anything else I've ever used.

🥳 1
cch1 2025-04-10T15:01:09.824939Z

I think if @dustingetz were here, he would probably mention electric as a candidate. More ambitious than a simple HTMX server, but a much higher ceiling. The demo is has a really powerful example of navigating Datomic.

cch1 2025-04-10T15:01:51.741879Z

We too are finding htmx (actually, unpoly) to be very effective for the html-based internal admin tooling.

Dustin Getz (Hyperfiddle) 2025-04-10T15:08:25.645109Z

i have been summoned, here is the https://electric.hyperfiddle.net/dustingetz.object-browser-demo3!ObjectBrowserDemo3/(dustingetz.object-browser-demo3!clojure-all-ns), we're actively developing it, we have not yet released. Click around! There's a datomic example. DM if you want to try it

🙇 2
Dustin Getz (Hyperfiddle) 2025-04-10T15:09:10.419939Z

and yes HTMX is much lighter than Electric for low ceiling apps, you should use it if you can