This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-01-19
Channels
- # announcements (19)
- # asami (9)
- # babashka (26)
- # beginners (87)
- # biff (23)
- # calva (6)
- # clerk (7)
- # clj-kondo (3)
- # cljsrn (3)
- # clojure (115)
- # clojure-belgium (1)
- # clojure-berlin (1)
- # clojure-europe (31)
- # clojure-gamedev (5)
- # clojure-nl (2)
- # clojure-norway (8)
- # clojure-uk (2)
- # clojurescript (43)
- # clr (23)
- # datalevin (1)
- # datomic (14)
- # dev-tooling (23)
- # fulcro (38)
- # graphql (1)
- # gratitude (1)
- # jobs (1)
- # lsp (30)
- # off-topic (7)
- # pathom (25)
- # portal (21)
- # quil (6)
- # releases (5)
- # remote-jobs (1)
- # shadow-cljs (34)
- # sql (5)
- # tools-deps (6)
- # xtdb (13)
I have some operations questions about Asami. I have a prod app using it, with storage on a shared disk. I assume it is not safe/possible to bring up a new version of the app accessing the same dB files as the old one, right? So I need to accept some downtime on each deployment. Q2: the cloud takes daily snapshot of the disk. If I restore it, will the DB be fine? Or is it possible that the snapshot would be taken at an unfortunate moment when the files are in some inconsistent state and thus unusable after a restore? 🙏🙏🙏
A1: I think it is. It only allows a single writer at a time, and reading is immutable. The main question would be if I remembered to handle the blank node counter correctly. If not, it’s still safe so long as only one process writes. Q2: this situation was explicitly designed for. It’s possible to snapshot during a write. In that case the final transaction will be corrupt. But a subsequent process opening it should see that and truncate that transaction. All indexes are fully forced to disk before the transaction record is written. (For this reason breaking up large transaction into multiple small transactions actually slows Asami down)
The more I think about A1, the more I’m thinking I allow multiple readers, but only 1 writer. The issue is the allocation of blank nodes. It’s based on a counter, and I don’t know that I refresh this after getting the file lock
Oh wow! It looks like I did remember this. Thank you past Paula! https://github.com/quoll/asami/blob/0434e590dba60c5c561e5e62cbe6ea0b5e051f4b/src/asami/durable/store.cljc#L203
Does that help @U0522TWDA ?
awesome, thanks a lot!
I should say that there is limited testing of this. It’s a bit tricky to test that 2 competing processes won’t cause problems! As for the snapshots during a write, the file design copies Mulgara, and we actually used to test that by pulling the power to the box during transactions, then booting up and reloading the database. Again, these are hard tests to do