Fork me on GitHub
#datomic
<
2017-11-20
>
fabrao04:11:20

@chris_johnson the ring Server use mysql database pool to serve 30-40 clients. So how is this works for datomic?

Empperi06:11:13

@fabrao the way Datomic works is like this: 1) You have data in whatever storage (mysql in your case) 2) You have an application with Datomic Peer within it (yes, within, it’s part of your app) 3) You query the Peer for data. This querying can happen within your application without any network I/O (so far!) or it can be a separate application. 4) Peer checks if it already has the needed segments of the database cached in memory. If the answer is NO, then it proceeds to download those segments from mysql into memory. 5) After the segments have been received (or if they already were there) Peer performs the actual query against the dataset that is within the memory of Peer (which is usually your application) and returns the data

Empperi06:11:37

so basically the limitation of having only 2 peers for free version means that you can have 2 applications at the same time talking to storage backend. These 2 applications can be either the same app duplicated for performance reasons or they can be completely separate apps

Empperi06:11:05

the way datomic free does this limitation is that when a Peer starts it does the following:

Empperi06:11:57

1) Peer connects to the Storage Backend 2) It retrieves several configurations from the Storage Backend, among those is the location of the Transactor 3) Peer connects to the Transactor and informs it is now available. With Datomic Free, if this is the third Peer informing about it’s availability to the Transactor this will fail. With Datomic Pro it succeeds

Empperi06:11:01

and why does Peer need this connection to the Transactor, after all it just reads from the storage backend and Transactor writes there? The reason is of course the Peer caches. When Transactor writes something to the storage backend it needs to inform all Peers about this new data so that Peers can either invalidate or update the caches accordingly

nickik10:11:04

Is this Java typed Java DSL released by now?

djjolicoeur20:11:42

is there an operator for pull for excluding an attribute? lets say I have one attribute I don't want the client to see....are my options in this programmatic removal and explicitly writing the pull for every attribute I do want to include, or is there a way to"blacklist" an attribute?

stijn09:11:52

@U06DX8UJY you can use a filtered database for this http://docs.datomic.com/filters.html

djjolicoeur12:11:08

@stijn I wonder what the performance tradeoff is, there. I hadn't thought of blacklisting attributes at the db level. you could add a :db/pii? attribute on the attribute schemas then filter out attributes where that is true.

jfntn22:11:58

We have a transactor deployed to an AWS vpc and configured through IAM roles, everything works fine on that front, but we now want to add a peer that’s not on AWS and I’m not sure it’s even possible?

favila22:11:27

It's an IAM role and AWS network config issue. Datomic doesn't care about the network boundaries. The peer just needs to be able to ip-route to storage and the transactor.

favila22:11:47

E.g. I have done it with ssh tunnels

jfntn23:11:31

@U09R86PA4 so we’d have to tunnel our external peer onto one of the aws peers?

jfntn23:11:00

Is there any config required?

favila23:11:28

You just have to get in the same network. You don't need to hit any machine in particular.

favila23:11:10

You could set up a VPN. You could whitelist IPs in IAM. You could ssh into a bastion machine, whatever.

favila23:11:24

The requirements are: 1) The datomic connection string you give to peer must be able to contact the storage; 2) The host= or alt-host= hostname on the transactor must be resolvable by the peer to the transactor

favila23:11:14

There are so many different ways to do that

djjolicoeur12:11:08

@stijn I wonder what the performance tradeoff is, there. I hadn't thought of blacklisting attributes at the db level. you could add a :db/pii? attribute on the attribute schemas then filter out attributes where that is true.