Hey @U028ART884X what is the wider example here? [(identity v) v1] forces an evaluation order that is otherwise ambiguous if you were to use [(== v v1)]

I got it from you originally: https://clojurians.slack.com/archives/CG3AM2F7V/p1648633699555599 So if I could use == there why didn't you favor that instead?

Also what does it mean to force the order? As I understand the call to [(identity x) y] would evaluate the first argument. But then what would the expression [x y] mean inside or-join?

Oh I see 😅 So in the context of that kind or-join I think either approach will work the same. Like it's essentially a performing a union of two sets either way. As for your question about [x y] ...are you referring to the first argument of the or-join? If so, well that is simply acting as an kind of variable scoping boundary as values get passed from outside to inside the or-join and back out again, i.e. it is not a regular Datalog clause

Maybe I'm misunderstanding something though 🙂

In general, they are (at least if you're using xtdb embedded with Clojure). It's possible I'm misunderstanding your question, though?

I might also be misunderstanding how the libraries work

The embedded API is the default and queries are run in the calling thread

so the HTTP api is the only one where queries are run remote?

what is causing a deal of the confusion is this line in the docs

> Datomic peers lazily read from the global index and therefore automatically cache their dynamic working sets. XTDB does not use a global index and currently does not offer any node-level sharding either so each node must contain the full database. In other words, each XTDB node is like an unpartitioned replica of the entire database,

So datomic will lazily stream in data, but xtdb will store all the data in memory?

Hm. So, by my understanding, these are two separate issues. One issue is where the data lives relative to your application, relative to the query boundary. In both xt and Datomic, if you are using the Clojure libs the data it operates on is local to the calling thread, as Jeremy mentioned above. Datomic doesn't provide a remote API (that I'm aware of, anyway) and XTDB provides the HTTP and http client which make the node "remote" relative to your application — not only out of thread, but out of process. That's the first issue, which doesn't relate to that comment in the docs. This comment in the docs is referring to how the database itself stores data. Datomic can store data remotely in a sharded configuration. When you query Datomic in this setup, you are still querying a node (I'm not sure if that's the right terminology, mind you — I haven't looked at Datomic in ~8 years) local to your app but that node can go grab data it lacks in its local cache from remote shards. The sharding will be transparent to you, I think. As of 1.21, xtdb doesn't have sharding or other separation of storage from compute features. All data is local to your node, but on disk (not in memory). If you want a second node, that node will also have all the data on disk. I'm not sure if that clears things up?

Just the indexes, though, not all all, right?

That's true, yes, indexes only store small top level field values, anything large will be retrieved from the doc store as needed (with an in-memory cache in front)

it shouldn't 🙂 what args are you passing? what's the full error / stack trace?

No, actually wondering which exceptions I have to handle

If you mean 'are there other kinds of errors that could be thrown', then I think the answer to that would be 'yes', like if tx-log or doc-store rejects the requests or throw timeouts

Snap 😅 I guess another example would be if the disk was full. I don't think we've comprehensively analyzed all the ways that it can fail before. XT doesn't attempt to handle or wrap any of those examples currently

Got it thanks

Yeah, the surprises in exceptions can be nasty, and no static typing to help with that either 😞 Especially when some of them are recoverable with retries