Fork me on GitHub
#datomic
<
2017-07-03
>
djjolicoeur15:07:37

is d/transact-async bound by datomic.txTimeoutMsec as d/transact is?

potetm15:07:26

transact-async has no timeout.

hmaurer18:07:51

Hi! Quick question on datomic data modelling: how would you store a large sequence of items (e.g. a news feed) in such a way that the N latest items can be retrieved efficiently?

jeff.terrell18:07:33

@hmaurer - I think this kind of thing might not be easy in Datomic. (I hope somebody more knowledgeable about Datomic corrects me though.)

jeff.terrell18:07:19

Or, wait…unless you mean the N most-recently-transacted items. In which case maybe the Log API would be what you need?

jeff.terrell18:07:33

Disclaimer: speaking from semi-ignorance here…

hmaurer18:07:51

@jeff.terrell haha no problem, that’s what I do most of the time. The N most-recently transacted items could work but I was looking for a more general solution. From what I understand about Datomic I think you are right though, it might not be possible without building an additional service…

jeff.terrell18:07:40

One possibility would be to subscribe to the stream of transactions (I think this is possible in Datomic), then maintain a data structure with the information you need.

robert-stuttaford18:07:38

@hmaurer @jeff.terrell the disadvantage with walking the log backwards is you have to keep reading chunks of N txes until you’ve filled however many items you need. in a database with lots of other stuff, this could mean traversing a lot of non-news-item txes. you could use the linked-list approach, where the latest item points to the next-latest item which points to the next-next-latest item which …, which then gives you a pretty straightforward path to discovery.

jeff.terrell18:07:23

robert-stuttaford: Ah, yeah…that's familiar, now that you say it. Thanks for clarifying that. simple_smile

robert-stuttaford18:07:03

i believe Datomic’s feature request thing has a request for traversing indexes in reverse, which would give you a very clear and direct path: traverse the “primary key” attribute in reverse.

jeff.terrell18:07:05

Here's a link to that feature request (might have to log in through http://my.datomic.com first, I dunno): https://receptive.io/app/#/case/17927

hmaurer19:07:26

@robert-stuttaford hi and thank you! Is there an efficient Datalog query to get the top N items from a linked list?

hmaurer19:07:08

@robert-stuttaford Also, is there a way for a service to follow the transaction log while ensuring that it doesn’t “miss” any transaction? You mentioned using Onyx so you might know about this