Fork me on GitHub
#xtdb
<
2021-05-20
>
henrik09:05:20

Hi, I'm trying to understand the data locality/cache aspect of Crux. Is the idea that you run your app on the same machine/same process as RocksDB/the query indices storage, or is the DB "over there" so to speak?

henrik09:05:29

From Datomic I'm used to working with lots of pulls which each take something like 0.3ms (or less) to complete. I'm wondering whether this pattern is possible with Crux, or if you have to use more monilithic queries to not be eaten alive by latency.

jarohen09:05:49

Hey @U06B8J0AJ πŸ™‚ yes, assuming you're using in-process Crux (as opposed to the HTTP server/client) all of the query indices are local to the node, so the pattern of many small queries is perfectly reasonable. pull is unfortunately an exception - it currently needs to grab documents from your document store. This can be mitigated by using a local doc-store cache (something that I've just noticed we don't have documentation for, I can provide an example), and I'm also working on a change as we speak to remove this constraint

πŸ‘ 10
henrik10:05:57

I found one of the most transformative aspects of Datomic to be that I can ask for data where I need it, to match the logic where it is used, without thinking too much about it. I'm hoping to be able to do the same in Crux (rather than having to co-locate lots of queries that have nothing to do with each other, only to navigate performance concerns). A lean pull would be very nice to have with regards to this. In many situations, a q can be used though, it's just a bit more verbose for the use case of gathering data given a specfic ID.

rodolfo22:05:17

Hi all, when trying to use range operators (`<`, >) with java.time.ZonedDateTime values, we get different results when querying the same dataset a few times in a loop, depending on how we pass the value as the query arg. If we pass the same ZonedDateTime instance in all the queries, the results are consistent. But if we generate a value on the spot (such as ZonedDateTime/now), we get results only some times. I have a hunch that this might be caused by the lack of support for a ZonedDateTime codec, and internally a hashCode of the instance (or even its string representation) might be used to compare. Is that correct? Comparing with some other java.time values that have codecs, we have consistent returns on all of them, which supports this hypothesis, but wanted to confirm if this is what it’s indeed taking place. Here is a minimal reproducible example: https://gist.github.com/rodolfo42/f2ac54883e42ffc3bbae204662288a00

βœ… 5
jarohen09:05:31

Hi @U9V0ZDFB7 πŸ™‚ Yep, this is correct - we'd recommend keeping to either Date s or Instant s for now.

rodolfo12:05:53

Thanks @U050V1N74!

πŸ™ 2