Well, there's no single answer. We run tx and doc store on Cloud SQL (Postgres) with the XTDB nodes running inside the app, in kubernetes pods

Hey @U033V0AJFU4 - in the first instance, whilst prototyping, you could try deploying a pre-built Docker image (e.g. to ECS) and access XT over the HTTP interface: https://hub.docker.com/layers/xtdb-standalone-rocksdb/juxt/xtdb-standalone-rocksdb/1.20.0/images/sha256-ba74a6862281cb26769fa7a07c80ada248300bf0d73750d0b379255ccea1c7f9?context=explore or perhaps attempt to build an application server using something like https://biffweb.com/ However, the real answer is that you probably want to treat XT as an embedded db in your app, and therefore you can think of XT as just another Clojure library (except one that will require the instance to have a lot of RAM and disk!). In which case just do whatever you would otherwise plan to do for setting up a Clojure app server on GCP / AWS. When you do need persistence (beyond prototyping), you can config the XT library to connect to an RDS JDBC backend. The only caveat is that you likely will want to be able to scale the database layer independently, if not now then eventually, so adding a network API boundary of some sort is generally a good idea. Otherwise with a full monolith you have to deploy everything in tandem. There was some discussion on Zulip about this recently: https://juxt-oss.zulipchat.com/#narrow/stream/194466-xtdb-users/topic/HTTP.20API.20question/near/277669039

@U899JBRPF thank you for the detailed answer! I've been reading the excellent documentation on xtdb website ad getting hyped tbh! To ensure i understand what i read correctly: Let's say i have a GCP storage bucket, i can use it as the storage (or disk) for the general database, and then in the clojure backend code i can start a node which uses this bucket to retrieve info and can do queries against. Is that correct?

If with a bucket you mean a blob store, then it can be the document store, but not the transaction store

Check https://docs.xtdb.com/administration/configuring/#_storage

Thank you @U8ZQ1J1RR I actually just finished reading the page u just sent. But i'm having a hard time understanding these terms in xtdb. If you were to describe to a 5 years old the following terms: • Transaction Log • Document Store • Index Store How would u go about doing that? I know what a document store is it's kinda similar to how dynamodb is. But the other 2 i have no clue tbh

The document is the immutable data. Transactions when that data appeared, when was it deleted, when was it changed etc; the temporal aspect of it. Index is what makes it fast, and can be recreated from the other two.

I'm pretty sure that my six year old wouldn't get that, though…

@U8ZQ1J1RR I'll try to explain this to my nephew later in the evening 😂 Thank you so much! Very helpful answers!

:args is probably going to stick around a long time yet, at least until all the tests have been migrated to :in 🙂 that said it would be bad idea to start using it now I think. How come you are curious about it? Do you find it conceptually simpler than :in?

I have some code that uses that. I was wondering if I'd need to refactor it. That code has args already in the query moved to the :args section.

Well, it's probably a good idea to refactor such usage to :in at some point, if only to keep your codebase consistent and more easily digestible for new team members, but there's no urgency, and keeping :args working is pretty low effort for us at this point 😅

Hey! So when you say "side effect" ...what do you mean exactly? 🙂 A crucial idea of transaction functions is that they are pure and therefore also idempotent, so it's best to not produce any side effects (and certainly not a good idea to have the functions depend on any outside state, e.g. including (Date.), otherwise determinism across XT nodes can easily break). However, if you are just writing out to logs then maybe that's okay.

So i am using external API to fetch some data about sports teams. I already have some data about those teams so i wanted to check if external API updated those teams and 1. if they did - to update mine. 2. If they added some new - to store new one. 3. If i have some that have similar names - put them in atom (this is side effect) so that i can inspect manually and choose what to do. I also write to logs.

Basically the flow is (roughly) (xt/submit-tx node (create-tx (get-teams))) where (create-tx returns

[[::xt/fn :put-or-que-to-manual team1]
[::xt/fn :put-or-que-to-manual team2]
....]

to confirm the side-effect (API fetch) is inside :put-or-que-to-manual? or are you trying to access a shared atom between the :put-or-que-to-manual calls?

in either case, it sounds like may need another layer of transaction function that produce these slightly more primitive transaction function calls (which in turn produce the actual primitive ops)

API fetch is before as i wrote above (get-teams) Side effect that is in transaction function is putting to atom (point 3. above) So

are you trying to access a shared atom between the :put-or-que-to-manual calls?

Alright i've solved it. Debugging is difficult with transactional functions i can see why you recommend adding new one instead of changing existing one (a while back i searched channel history in order to try to find answer for my question) Conclusion: you can definitively add more transaction functions and they will be executed just like any other operation. Thank you for your time

😄 glad past me could help 🙏

> Debugging is difficult with transactional functions i can see why you recommend adding new one instead of changing existing one sounds like it's worth me taking a moment to add this tip to the docs! :thumbsup:

Hmh, perhaps I should encode the type in the id, would make things simpler

You can definitely do that, as long as you're pretty confident it won't cause you issues later. 🙂 This is the generic advice, which looks like your first approach: https://docs.xtdb.com/resources/faq/#_modeling_types I'm not sure if xt/entity or the regular query would have better perf, if you go that route, but I'm guessing someone might.

@U8ZQ1J1RR Feel free to post usage questions on http://discuss.xtdb.com if you like, btw. Since it's searchable (and edit-able) you'll be helping all future users of xt by doing so. 🙂 You can always point to a d.x.c post from here and/or summarize Slack conversations into a d.x.c post if you think they'll be helpful for someone down the road.

I posted at https://discuss.xtdb.com/t/limiting-results-to-a-given-type/89 with a bit more thought on the topic