Fork me on GitHub
#datomic
<
2020-04-24
>
tatut05:04:24

I'm trying to access datomic via codebuild for db tests, but I can't create endpoint for vpc https://docs.datomic.com/cloud/operation/client-applications.html#create-endpoint (is LoadBalancerName not available in solo topology?)

vlaaad12:04:47

Lets suppose i have an entity and a bunch of txs that touch that entity. What would be an efficient query to pull a bunch of data from this entity at these timepoints too see how it looked throughout its life? Or is (map #(d/pull (d/as-of db %) '[*] e) txs) the only way?

vlaaad12:04:23

I’m using cloud by the way

vlaaad12:04:48

so I would guess every d/pull is a separate request?

marshall13:04:52

@vlaaad you could query for everything about the entity from a history db

vlaaad13:04:47

but that’s another thing, I’m not interested in changes, I want full state at point in time

marshall13:04:17

full state at a point in time definitely sounds like as-of

vlaaad13:04:19

Yeah, and I wonder if there is a way to query for state of an entity at different time-points

vlaaad13:04:06

like [:find (pull $ ?e [*]) :in [[$ ?e] ...]]

vlaaad13:04:38

so it is a single request to server instead of multiple requests

vlaaad13:04:55

if it is multiple requests? hard to tell without source code…

marshall13:04:06

cloud or on-prem?

vlaaad13:04:20

I tried with more entites, and using multiple d/pull + d/as-of IS a N+1 problem: it gets more and more slow, so I guess it performs multiple requests

donyorm16:04:06

Is there a reference anywhere to what permissions a user/role needs in order to push and deploy for ions?

donyorm17:04:32

That seems to be more related to accessing the database itself, rather than just pushing ions. I don't need to give this role access to the database, it just needs to deploy ions. Does that still require being a datomic administrator?

hadils16:04:43

Cloud question: when a new EC2 instance is started, is a new transactor created? Or is there one transactor for the whole system, regardless of the number of EC2 instances? I am curious if I need to track machine ids and so forth to figure out who is writing to parts of the database. I am probably overthinking this.

marshall16:04:02

@hadilsabbagh18 you’re definitely overthinking it 🙂 there is no single transactor in Cloud All nodes of the primary compute group can perform writes

marshall16:04:38

traffic is all routed through the load balancer

marshall16:04:49

and uses consistent hashing and sticky sessions

marshall16:04:09

to route requests from the same client and/or about specific DBs to particular nodes

marshall16:04:16

but that is strictly a performance optimization

marshall16:04:30

any node in the group is capable of handling writes to any db

hadils16:04:27

Thanks @marshall. When a new EC2 instance is started, doesn't my code start on it as well? Isn't there a potential for two servers to work on the same datoms in the database? My code is multi-threaded so there are processes that may replicate work on different EC2 isntances if they are running. If that is the case, then I need to track who is doing what, right?

marshall16:04:04

no. individual transactions are still serialized via coordination with storage

marshall16:04:30

you may need to consider that separate nodes may try to do things “at the same time”

marshall16:04:50

but that is no different than multithreaded db access in any system

hadils16:04:06

Ah. Thanks! I can handle this case without machine ids, etc. Thanks a lot @marshall!

marshall16:04:10

i.e. use compare-and-set, optimistic concurrency, etc

hadils17:04:58

@marshall another question. I know that the lambda functions are actually proxies. Do they scale out and spawn separate processes within the EC2 pool when load becomes high?