Fork me on GitHub
#xtdb
<
2021-07-30
>
lgessler05:07:21

say I have a relationship between two entities foo and bar --I can express it with a join :foo/bar-join or :bar/foo-join. If I know that the majority of the time I will be querying for bars given some foo instead of vice versa, would it be much more efficient to have :foo/bar-join instead of :bar/foo-join? My best guess is that the performance of this query for :foo/bar-join has to be the most performant one, since the query engine would just have to look at what values are in the join attribute on the entity whose ID we already know, but I'm not sure just how much worse I can expect a :bar/foo-join to be

jarohen08:07:16

Hey @U49U72C4V 👋 I don't think Crux will care that much either way - we strip documents down into EAV triples into the indices, and then the query planner will be able to navigate the join in either direction depending on which variable it thinks is smallest.

jarohen08:07:57

i.e. if your query has a good filter on foo somewhere, it'll pick that first and then navigate to bar, and vice versa

jarohen08:07:01

We also keep indices both ways - AEV and AVE, so if you have E in hand we'll use AEV; V in hand we'll use AVE (we always have A)

jarohen08:07:46

In short, I'd probably prioritise modelling based on the cardinality of the relationship instead - i.e. if it's a parent:children relationship, a field on each child referencing its parent

👍 4
lgessler14:07:14

thank you, appreciate the answer! that's a relief--my case is a parent-child relationship, so i'll happily be putting the join on children then

richiardiandrea14:07:06

this is interesting, I though one reference would be fine and the reverse join /_foo feature would take care of the rest...would it be possible to do it that way? Is there a performance hit in using the reverse join?

jarohen14:07:10

@U0C8489U6 - not sure I quite understand your question I'm afraid - but yep, one reference from child to parent is fine. By reverse join, do you mean :where [[?child :has-parent ?parent]] (if so, yep, no performance hit, as outlined above) or the reverse joins in the pull API (if so, also there shouldn't be a performance hit - this also uses the AVE indices)? maybe an example would help?

richiardiandrea14:07:09

Yeah sorry - I meant the reverse join in the pull child/_has-parent basically

richiardiandrea14:07:03

and just was wondering what is the preferred way if any - I am very happy how terse the pull syntax is if no major drawbacks to it thanks!

jarohen16:07:05

Evening - 1.18.0 is out 🚀 * Significant improvements to the Lucene module - it now supports checkpointing, in-memory Lucene, and Crux will now ensure the Lucene index is up-to-date on startup. Requires a Lucene re-index for folks who have Lucene indices. * Ability to add 'secondary indices' (of which Lucene is the first) - custom indices that follow the Crux tx-log alongside the main indices. Can be used (for example) to create materialized views of your Crux transactions in other formats/stores - we're intrigued to see how folks use this 🙂 * First alpha release of crux-corda - adds the Corda blockchain (https://corda.net) as a possible source of Crux transactions, allowing you to query both the current and historical state of your Corda nodes using Crux's temporal queries. Keep your eyes peeled for some blogs over the coming weeks! * All the usual bugfixes 🐛 For more details, see the release notes at https://github.com/juxt/crux/releases/tag/1.18.0. As always, a big thanks to everyone contributing to this release by raising/fixing issues, and helping us with repros!

🎉 16
sheluchin19:07:10

Congrats on the release. I'm curious about the corda support. May I ask how that particular blockchain ecosystem was chosen and if there are plans to add others?

Steven Deobald19:07:48

The Corda blockchain was already in use on an existing client project and the adapter was built at their request. We could very well build more adapters in the future, but doing so either depends on customer demand or community contributions. It's not a specific goal of Crux to have a lot of blockchain adapters — we're just happy to build them if companies find them useful. 🙂

👍 2
plins20:07:01

hey Im trying to delete all crux entries in between tests, whats the best way of doing it? reading all entries and evicting one by one in a tx?

lgessler20:07:01

if it's all of them, perhaps you could just make a new in-memory crux node?

➕ 6
R.A. Porter21:07:38

☝️ Create the node in a fixture with a type of :each.

👍 2
plins21:07:21

thank you folks

jarrod21:07:21

I need some help understanding entity-history

;; When submitting a single tx for valid range 2001 to 2002 on an in-memory node
(crux.api/submit-tx 
                   crux
                   [[:crux.tx/put {:crux.db/id "test"} #inst "2001" #inst "2002"]]

;; I expect no history when specifying a later valid time range,
;;  but get the tx with a valid time of 2001-2002
(crux.api/entity-history 
                   (crux.api/db crux) "test"
                   :asc 
                   {:with-docs? true
                    :start {:crux.db/valid-time #inst "2020"}
                    :end {:crux.db/valid-time #inst "2021"}})
                            
;; Result                   
[{:crux.tx/tx-time #inst "2021-07-30T21:43:43.644-00:00",
  :crux.tx/tx-id 0,
  :crux.db/valid-time #inst "2001-01-01T00:00:00.000-00:00",
  :crux.db/content-hash
  #crux/id "5357154b961bd22dc5ddbc7fec9580419964976c",
  :crux.db/doc #:crux.db{:id "test"}}
 {:crux.tx/tx-time #inst "2021-07-30T21:43:43.644-00:00",
  :crux.tx/tx-id 0,
  :crux.db/valid-time #inst "2002-01-01T00:00:00.000-00:00",
  :crux.db/content-hash
  #crux/id "0000000000000000000000000000000000000000",
  :crux.db/doc nil}]

refset00:07:20

Start and end bounds aren't specified using maps like that (anymore), see https://opencrux.com/reference/1.18.0/clojure-api.html#_entity_history

refset00:07:57

Where were you referring to?

jarrod12:07:52

Hmm , thanks @U899JBRPF. Did that just change in 1.18.0 ? I had been using 1.17.1 and referencing https://opencrux.com/reference/1.18.0/queries.html#history-document-history-range.

refset13:07:01

Ah, yeah the docs fix only just got published 😅. Feels like an age since I fixed it already though: https://github.com/juxt/crux/commit/a8000e312a2788c7093111b4debd488bb4c5a52e

refset14:08:08

oops, sorry, I just clicked your link and realise my fix failed to cover that example as well :man-facepalming: One for tomorrow morning! Thanks for the report

jarrod14:08:20

Sure, thanks for the quick response.

🙏 2