Fork me on GitHub
#xtdb
<
2021-05-28
>
refset18:05:36

📢 Crux 1.17 is out! https://github.com/juxt/crux/releases/tag/21.05-1.17.0 ✓ HyperLogLog statistical index for join order heuristics ✓ JDK 16 support ✓ Bug fixes ✓ Lots more detail in the linked release notes 🙂 To save you the trouble of clicking through, here's the important snippet about what HyperLogLog is doing: > We've added https://en.wikipedia.org/wiki/HyperLogLog statistics, which can efficiently approximate the amount of distinct values in a large set. We then use these statistics to make better choices when your queries have filters on multiple attributes. Given a choice, we choose to filter first by attributes with a higher proportion of distinct elements (or higher 'selectivity') so that we can reduce the size of intermediate query results as much as possible as early as possible. In practical terms, this means Crux will be much more intelligent about calculating an optimal join order if you have something like a : attribute on every single document (and reference it in lots of your queries). Particularly when it only ever has a handful of possible values! Big thanks to everyone who has contributed, reported bugs etc. 🙏

🎉 24
👏 9
lispyclouds19:05:18

Thanks a lot for this! 😄 Just to clarify, I still need to add the --add-opens java.base/java.util.concurrent=ALL-UNNAMED for usage in java 16+ right? Crux is starting up with the warning now but seems to not work when querying till I add the flag back in.

jarohen10:05:42

Folks with JDBC tx-logs/document stores will also want to check the release notes for details of a DDL migration

jarohen10:05:45

@U7ERLH6JX sorry for the trouble here - the idea is that it should now cope without it but it won't be as performant. will have a look and see what's happening

lispyclouds10:05:07

Thanks a lot @U050V1N74! Here is a small code base in which the tests fail if i run it without the flags: https://github.com/bob-cd/bob/tree/main/entities hope this helps in some way? I can help debug this too! 😄 As always, thanks a lot for Crux 🙏:skin-tone-3:

jarohen10:06:35

Hey @U7ERLH6JX - have deployed a dev-SNAPSHOT version to Clojars with a fix for this, will try check it against that repo

lispyclouds10:06:25

Awesome! Thanks a lot 🙏:skin-tone-3:

jarohen10:06:27

looks good to me, running bb test. I've also tried to remove the --add-opens to verify that it works without that flag, can't see the log message but wondering whether that's because of logging setup. will keep digging

jarohen10:06:47

got it 🙂 looking good

lispyclouds10:06:21

just saw this! did it work fine?

lispyclouds10:06:10

without the flag it usually returns a nil for some of the queries failing the tests

jarohen10:06:15

yep, I temporarily disabled kaocha's output capturing and saw the Crux log message, tests passed with and without the flag 🙂

lispyclouds10:06:01

yay! awesome bananadance

lispyclouds10:06:07

thanks a a lot for following up!

🙏 3
🙂 3