Fork me on GitHub
#xtdb
<
2020-05-29
>
ordnungswidrig10:05:37

Where did txc/tx-event->doc-hashes go? :thinking_face: I was using it in a custom tx log: content-hashes (->> (set (map c/new-id (mapcat txc/tx-event->doc-hashes tx-events))))

jarohen10:05:41

hey @U054UD60U - looks like it's now txc/tx-events->docs

ordnungswidrig10:05:14

is it the same?

jarohen10:05:01

also, the usual heads-up that these namespaces aren't part of the Crux API and can change between releases 😉 (disclaimer disclaimer) that said, happy to help out with what to use instead

ordnungswidrig10:05:25

Yeah, I picked this piece of code up from somehwere else I guess.

ordnungswidrig10:05:44

What is then recommende to be used for a custom tx log implementation?

jarohen10:05:00

it now returns the docs from the doc store as well, although I don't know whether this is what you'd want in this case

ordnungswidrig10:05:30

it’s for the implementation of submit-tx so I only need ti hashes I guess

jarohen10:05:45

are you able to share more of the implementation? my first thought would be that the Crux tx-consumer should deal with most of that for you anyway

ordnungswidrig10:05:55

It’s experimental but was kind of working 😉

jarohen10:05:13

DynamoDB - niice :star-struck:

jarohen10:05:10

my guess (without seeing query-tx-table) is that crux.tx/docs is copied from our own previous Kafka implementation, which included metadata about which docs the transaction referred to in the tx-log event in Kafka, at least, that's no longer required, because the general tx-consumer figures out what docs are required from each transaction - so we've been able to remove it from the Kafka tx-log implementation

ordnungswidrig10:05:05

I saw that commit and was surprised 🙂

ordnungswidrig10:05:18

Would be nice to learn about what is actually needed to implement a tx log

ordnungswidrig10:05:29

So only the events must be stored?

ordnungswidrig10:05:12

Actually query-tx-table doesn’t return the value in crux.tx/docs either 😉

jarohen10:05:49

yes, only the tx-events - we decompose transaction operations (that the user supplies - excuse the internal naming here, it's not always 100% clear) into tx-events and documents. the tx-events go on the tx-log, the documents go into the document-store reason for this is that if we later want to irretrievably evict those documents, we only need mutate the document-store (replacing the documents with tombstones) - the tx-log remains unaltered

ordnungswidrig11:05:23

Thanks, I will explore this later.