Fork me on GitHub
#xtdb
<
2022-06-12
>
pinkfrog04:06:29

Is there something like in browser xtdb, or do I have to stay with datascript for the moment?

zeitstein05:06:56

There is not. If you want histories, check out Datahike or Asami. Note this https://clojurelog.github.io/.

zeitstein14:06:35

As far as I was able to find, there seems to be no way to get the :xt/id of a doc that was ::xt/deleted after that tx has been submitted (i.e. either open-tx-log or listen) – it is encoded to #xtdb/id. Correct? So, what are the options? Use an impure tx function?

refset21:06:10

Have you tried the with-ops? option for open-tx-log or with-tx-ops? for listen?

zeitstein04:06:20

Yup. The issue is that deletes have`:xt/id`s encoded:

([:xtdb.api/put #:xt{:id :put}]
 [:xtdb.api/delete #xtdb/id "9485989ff514b5106b7738850fd73c23e8c1e3f7"])
I want to notify clients that a doc has been deleted, so need their :xt/id. From a related https://juxt-oss.zulipchat.com/#narrow/stream/194466-xtdb-users/topic/.E2.9C.94.20Can.20I.20trust.20that.20tx-log's.20.23xtdb.2Fid.20is.20derived.20from.20.60.3Axt.2E.2E.2E/near/269114309: > the one-way hashing is used so that eviction can fully remove documents without even revealing their IDs To me, this makes sense for eviction. But why does it seem to apply for deletion too?

refset12:06:47

I think there is a valid argument here to do something more friendly, like replacing the #xtdb/id with the :xt/id wherever possible in these APIs :thinking_face: but essentially it's just surfacing the raw information from the tx-log, which is always hashed, because the log is truly immutable (unlike documents...which may be evicted and replaced with tombstones)

refset12:06:20

You can actually pass the #xtdb/id into entity to find out the :xt/id but that requires the db to intersect with a window when the entity is visible:

(with-open [n (xt/start-node {})]
    (xt/submit-tx n [[::xt/put {:xt/id :foo}]])
    (xt/submit-tx n [[::xt/delete :foo]])
    (xt/sync n)
    #_(with-open [t (xt/open-tx-log n -1 {:with-ops? true})]
      (into [] (iterator-seq t)))
    (:xt/id (xt/entity (xt/db n {::xt/tx-id 0}) (xtdb.codec/new-id :foo)))) ;;=> :foo

refset12:06:04

xtdb.codec/new-id is what is used internally to generate the hash

refset12:06:09

I just raised an issue https://github.com/xtdb/xtdb/issues/1769 🙂 feel free to chime in there!

zeitstein12:06:26

Nice, thank you! I guess just decrementing the last tx-id may still place one out of the window (so to speak :))? Well, okay, you can just keep decrementing until entity returns something.

zeitstein13:06:31

> eid is an object which can be coerced into an entity id. "entity id" here is the hash, correct?

1
zeitstein13:06:20

Eeek, I see ::xt/fn ops are also conformed to the hash.

refset13:06:37

> I guess just decrementing the last tx-id may still place one out of the window ah yes, for sure, I just did that did in the example code to get something working 😅

refset13:06:11

> Well, okay, you can just keep decrementing until entity returns something to unblock you for now during development (tiny database sizes) that is probably not an absolutely terrible idea 🙂 better still, I can whip up some code to resolve #xtdb/id => :xt/id via internal APIs, but not until much later today/tomorrow

zeitstein14:06:19

Fantastic, thanks.

refset23:06:42

Hey again, having now looked at the internals a bit and reflected, I think the function in this updated code is actually a pretty acceptable userspace solution for now, via open-entity-history (avoiding valid time window confusion entirely, not sure why I didn't think of that earlier facepalm): https://github.com/xtdb/xtdb/issues/1769 It's not going to be exactly as efficient as a native solution, but is definitely not far off in comparison!

zeitstein03:06:31

Thank you!

🙏 1