This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-11-09
Channels
- # announcements (3)
- # asami (1)
- # babashka (19)
- # beginners (84)
- # calva (3)
- # cider (5)
- # clj-commons (22)
- # clj-kondo (31)
- # cljdoc (4)
- # cljs-dev (5)
- # clojure (65)
- # clojure-australia (1)
- # clojure-europe (44)
- # clojure-nl (2)
- # clojure-uk (2)
- # clojurescript (18)
- # code-reviews (12)
- # conjure (2)
- # core-async (12)
- # data-science (1)
- # datomic (47)
- # deps-new (1)
- # emacs (2)
- # events (4)
- # fulcro (35)
- # integrant (1)
- # jobs (5)
- # jobs-discuss (10)
- # london-clojurians (1)
- # lsp (13)
- # music (1)
- # nextjournal (1)
- # off-topic (11)
- # parinfer (3)
- # pathom (6)
- # polylith (11)
- # portal (41)
- # re-frame (4)
- # reagent (13)
- # reitit (8)
- # remote-jobs (3)
- # sci (18)
- # shadow-cljs (34)
- # spacemacs (3)
- # tools-build (12)
- # tools-deps (6)
- # vim (2)
- # xtdb (7)
what's the fast way to get the largest value of a given attribute? the naive approac of [:find (max n) :where [_ :num n]]
goes thru all of the values... should I just keep a counter document with the current max or is there an index trick
my guess - the best option would be lookup the av index in reverse order - but I don't think the index api exposes a way to do that @U899JBRPF ? I've got a couple of use cases where being able to navigate the indexes in reverse order would be useful actually
> the best option would be lookup the av index in reverse order - but I don't think the index api exposes a way to do that
that's correct, yep. There is a "prev" method on the KV protocol (though Rocks in particular is not optimised for reverse seeks), so theoretically someone could add an av-r
here and corresponding step-fn-r
(which calls prev
instead of next
)
https://github.com/xtdb/xtdb/blob/e2f51ed99fc2716faa8ad254c0b18166c937b134/core/src/xtdb/kv/index_store.clj#L674-L681 - we've not done this ourselves because exposing it through the Datalog is non-trivial, and we generally don't want people dipping into the non-public functions & protocols
One option is to invert the bits on the values you want to iterate in reverse, and then effectively store the values twice, like this: https://gist.github.com/refset/f737151c2956093bbe01d7b81a743328 (see this discussion for context https://github.com/xtdb/xtdb/discussions/1514)
Another option, which is applicable to more general kinds of aggregations, is to use a transaction function to store the data manually.
A final option would be to create a secondary index, along the same lines as the Lucene module https://github.com/xtdb/xtdb/blob/ea5db95eb7996e1c71d8a72806a9bb82af0b347c/modules/lucene/src/xtdb/lucene.clj#L382-L395
this particular case I just want (in a tx fn) to create a sequence number, so I think it's easiest to just store {:xt/id :some-unique-counter-id :counter-value 123}
and just get that and update it
normally we don't of course use sequence numbers but here we need one for an external system
that's correct, yep. There is a "prev" method on the KV protocol (though Rocks in particular is not optimised for reverse seeks), so theoretically someone could add an av-r here and corresponding step-fn-r (which calls prev instead of next)
- we've not done this ourselves because exposing it through the Datalog is non-trivial, and we generally don't want people dipping into the non-public functions & protocols
would you consider a PR that did this? as I mentioned there's a couple of nice things it would enable for my use of xtdb