Clojurians Log v2

I need to route the versioning API through the transactor proper, it predates this and I think this is why sometimes it can fail. I hope to get to this in the next days. I worked through all other bindings and the pod first, but the versioning API is next on my agenda, in particular a standardization between proximum, datahike, git, zfs etc. such that you can snapshot larger systems consistently with the same API. Feedback for what you would like to have is helpful at the current point.

My current need is to isolate database from main one for request process. Fork at the beginning and merge at the end. To keep whole the flow as function of request input and database state on the moment of input. As applied thing I want incapsulated merge! function that automatically (and maybe somehow optimized!) calculate diffs

Thank you!

You can do this, I think, but there might be a modification to the tiered store needed. I can take a look in a bit. It should not be hard to understand the tiered store code though and maybe you could use it to think through the memory model you would like to have (it literally just calls frontend and backend konserve stores, so not very heavy).

Hm.. i will take a look!

These are the different strategies, but it really depends on what your memory model and assumptions are: https://github.com/replikativ/konserve/blob/main/src/konserve/tiered.cljc#L244

Ha. looks like there no strategy I want. Maybe it is against some consistency rules? I need synchronous write to frontend to work with this data immidialtely, but async in background to backend to store data more durable

No, describe what you want. These were just some defaults to give people options. konserve does not enforce complicated consistencies, the goal is rather to make it easy to build different data solutions on top. For instance datahike/pss ensure that the writes are consistent and properly done, not konserve. It just makes sure that individual values you write to it are properly handled. Having said that being consistent when writing can be tricky. If you write async in your groundtruth storage (cockroachdb) and it fails you might have read values from your frontend that are not consistent. But there are many workloads where this can be acceptable to reduce latency.

Like this

{:write-backuped
   (let [frontend-result (<?- (-assoc-in frontend-store key-vec meta-up-fn val opts))]
     (go (try
           (<?- (-assoc-in backend-store key-vec meta-up-fn val opts))
           (catch #?(:clj Exception :cljs js/Error) e
             (warn "Backend assoc failed in write-backuped" {:key key-vec :error e}))))
     frontend-result)}

In 0.9.344 :write-behind should now do what you want.

https://github.com/replikativ/konserve/commit/50a7f4bf2ad18b67ba4f9323c3f1d6bb6c0615ec

Yes, I also just ran into this; investigating.

Which DB backend was this with?

I tried on :memory and :jdbc . Both of them hang

Thx

Did your store config have the :id with a UUID set?

When I put it correctly it works fine. But if something wrong - it hangs

Yes, found the issue, validating the fix.

In 0.6.* version it threw exception like connect, but in 0.7.* no

Should be fixed in the latest release now thanks to @uppfinnarjonas.