datomic 2021-02-24 | Slack Archive

pinkfrog15:02:07

I am going to store blockchain block informations into datomic. From time to time, I would query the total transaction count in the most recent H blocks. So this is querying against a window of H blocks. How can I perform that with datomic *efficiently ?

jjttjj15:02:11

is there a reason not to just store a transaction count value with the block entity?

jjttjj15:02:39

Or are you asking how to do the moving window aggregation (rolling sum of last H blocks)

pinkfrog15:02:18

Yes. Sliding window as normally done in Flink. Wonder how to best achieve that w/ datomic.

bmaddy20:02:04

I'd like to monitor the time spent in d/q (specifically, to report it to New Relic). I could just put a wrapper function around d/q and hope developers remember to call that instead of calling d/q directly, but is there a better way? Ideally, I'd do something when setting up the connection to ensure all queries get monitored. Is there a protocol I could extend or something?

Lennart Buit20:02:41

On client at least, the datomic api is just a set of functions invoking protocol functions on your connection/db value. So what you can do is kinda proxy them: create a deftype for say a db value that forwards all calls to an ‘underlying’ datomic db value. In these wrappers you can then add logic like this

Lennart Buit20:02:51

The benefit is that you only have to control where the db or connection value comes from, the functions that get it are oblivious to that these values are ‘decorated’ like this

bmaddy20:02:16

Yeah, that's the initial route I was looking down, but I couldn't figure out how d/q used the db value. Specifically, I didn't see a q method or similar on there which suggested to me that member functions would be called many times on a single query, so I wasn't able to figure out how I'd know when a query starts and ends. Have you tried doing this?

bmaddy20:02:53

I guess I could wrap a db value and record which methods are called to maybe guess how it's used.

kenny20:02:57

@U067Q76EP take a look at https://github.com/ComputeSoftware/datomic-client-memdb/blob/master/src/compute/datomic_client_memdb/core.clj

👍 3

kenny20:02:11

Specifically LocalDb

souenzzo20:02:04

@U067Q76EP I did a small lib to ensure that every transact get audited (some extra tx-data on every transaction in a conn). You can do somethign like that, but to ensure that every (d/db) in your conn will return the "db-with-spy-methods" https://github.com/molequedeideias/conncat

kenny20:02:06

Be careful with that. Depending on your system, it can create lots of spurious transactions if you’re not checking for empty first.

bmaddy20:02:56

@U2J4FRT2T, yeah, exactly what I was thinking in terms of wrapping the connection. Nice to see that it works. Thanks! @U083D6HK9 Nice, it looks like client-impl/Queryable has a q function right there on it that you can implement. Does anyone konw if there's a similar interface for a peer (I should have mentioned in my question that we're using a peer)?

2021-02-24

Channels