Fork me on GitHub
#xtdb
<
2021-11-09
>
tatut05:11:56

what's the fast way to get the largest value of a given attribute? the naive approac of [:find (max n) :where [_ :num n]] goes thru all of the values... should I just keep a counter document with the current max or is there an index trick

xlfe08:11:58

my guess - the best option would be lookup the av index in reverse order - but I don't think the index api exposes a way to do that @U899JBRPF ? I've got a couple of use cases where being able to navigate the indexes in reverse order would be useful actually

refset10:11:10

> the best option would be lookup the av index in reverse order - but I don't think the index api exposes a way to do that that's correct, yep. There is a "prev" method on the KV protocol (though Rocks in particular is not optimised for reverse seeks), so theoretically someone could add an av-r here and corresponding step-fn-r (which calls prev instead of next) https://github.com/xtdb/xtdb/blob/e2f51ed99fc2716faa8ad254c0b18166c937b134/core/src/xtdb/kv/index_store.clj#L674-L681 - we've not done this ourselves because exposing it through the Datalog is non-trivial, and we generally don't want people dipping into the non-public functions & protocols One option is to invert the bits on the values you want to iterate in reverse, and then effectively store the values twice, like this: https://gist.github.com/refset/f737151c2956093bbe01d7b81a743328 (see this discussion for context https://github.com/xtdb/xtdb/discussions/1514) Another option, which is applicable to more general kinds of aggregations, is to use a transaction function to store the data manually. A final option would be to create a secondary index, along the same lines as the Lucene module https://github.com/xtdb/xtdb/blob/ea5db95eb7996e1c71d8a72806a9bb82af0b347c/modules/lucene/src/xtdb/lucene.clj#L382-L395

tatut11:11:39

this particular case I just want (in a tx fn) to create a sequence number, so I think it's easiest to just store {:xt/id :some-unique-counter-id :counter-value 123} and just get that and update it

👌 1
tatut11:11:08

normally we don't of course use sequence numbers but here we need one for an external system

👍 2
xlfe08:11:49

@U899JBRPF

that's correct, yep. There is a "prev" method on the KV protocol (though Rocks in particular is not optimised for reverse seeks), so theoretically someone could add an av-r here and corresponding step-fn-r (which calls prev instead of next)
 - we've not done this ourselves because exposing it through the Datalog is non-trivial, and we generally don't want people dipping into the non-public functions & protocols
would you consider a PR that did this? as I mentioned there's a couple of nice things it would enable for my use of xtdb

refset12:11:13

it's hard to say for certain now, as it depends on a lot more than just my own opinion, but I'm sure we could entertain an Issue or very rough demonstration PR soon, if you have the appetite 🙂

👍 1