Fork me on GitHub
#datomic
<
2018-09-13
>
Oliver George07:09:18

I'm puzzling over how layer on some data integrity constraints. I see that using custom transactor functions are a way to add logic but I haven't seen any conventions around using it to ensure data integrity robustly. Simple example might be: all People entities must have a :person/name. This is easily enforced when creating but the other case to consider is a different piece of code retracting the :person/name attribute on an entity. Clearly the first issue is that entities don’t have a type. I wondered about giving entities an optional :constraint/spec keyword attribute and using a generic transactor helper fn to ensure all entities affected are still valid data using s/is-valid?

Oliver George07:09:51

How do you ensure data integrity?

dominicm07:09:34

@olivergeorge if all people must have a person/name, maybe you can't find a person if they don't have a person/name?

greywolve10:09:16

Question about datomic cloud. So if i'm reading things correctly, the only way to run your client applications, is in the same VPC as datomic cloud? Ie there's no way to have say, an app running in Heroku access datomic cloud via the client lib?

greywolve10:09:59

and if that's the case, how are people that run the transactor on aws, and their peers on heroku, managing to properly secure the transactor?

steveb8n10:09:57

@olivergeorge I use spec but in the client instead of in a transaction fn

stijn10:09:51

@greywolve there's a section in the docs that describes how to run clients in another vpc https://docs.datomic.com/cloud/operation/client-applications.html#separate-vpc

greywolve10:09:40

Thanks, do you know if it's possible to get this right if you're hosting on Heroku in the same aws region?

stijn11:09:40

sorry, i have no experience with heroku...

jeff.terrell11:09:05

I'm facing this same question myself. I'm thinking it would theoretically be possible either to have a proxy host within AWS that could mediate traffic between Heroku and Datomic Cloud, or else open up a relevant security group to allow traffic from Heroku's network.

jeff.terrell11:09:25

But I'm seeing a couple of concerns. First, it really wouldn't be a great idea to have the Heroku instances across the country (or the world) from the Datomic instances. That'd make the site really slow I would think.

jeff.terrell11:09:57

Second, maybe this kind of approach isn't allowed for Solo? > VPC Endpoints are only supported in a Production Topology.

jeff.terrell11:09:03

So I'm weighing the costs of exploring a solution to bridge Heroku and Datomic Cloud against the costs of reinventing some Heroku functionality (especially review apps) in AWS with the code integration tools like Code Pipeline. Leaning towards the latter, to be honest, but I expect it'll take some work.

greywolve14:09:33

How do you secure your proxy host though?

greywolve14:09:40

From my limited research, it doesn't seem like a good idea to whitelist ips, since the heroku ips are dynamic, and the range is too big. The other option is to fork out for one of the static ip addons, but they seem pricey

jeff.terrell14:09:41

Yeah, that's a good point about Heroku IPs being dynamic. This further inclines me to try to learn the Code* tools in AWS.

greywolve14:09:43

Yeah I'm starting to feel like the only sane approach is probably just to move over to AWS if we want to use Datomic

greywolve14:09:07

It's a pity Heroku private spaces cost so much ($1000 / month)

jeff.terrell14:09:22

I feel the same way. It's a little discouraging considering how easy it is to deploy a Clojure app to Heroku, plus all the nice features Heroku provides, but oh well.

greywolve14:09:04

Yeah 😞 I didn't expect it to be this challenging to setup Datomic there

greywolve14:09:13

I wonder if setting up the VPC endpoint, and connecting via Heroku on the same region would work, seems a bit fragile though

idiomancy15:09:12

hey, I know this must be a FAQ, but... okay, I'm just getting my feet wet with Datomic via datascript in the browser, and even though I can rip out and reschema my entire system with one version update, I am terrified by the notion of "correct" Data modeling with Datomic.. I don't know what should be namespaced, how to namespace them. There's so much flexibility that I have a serious "blank canvas" problem. Can anyone point me towards where to start in modeling your domain from a blank canvas with Datomic?

idiomancy16:09:01

okay, great, I'll take a look!

val_waeselynck19:09:17

@U250T6MFA table/column or class/method is a good starting point, you'll figure the paterns soon enough, and Datomic gives you enough flexibility to recover from your mistakes

Dustin Getz14:09:53

@U250T6MFA +1 for table/column. You will be able to figure this out as you go. Just get started.

eraad18:09:00

In addition to table/column, I like to think about the schema as an attribute catalogue that I can use to create entities. So don´t worry about having ALL possible attributes upfront. Just add the minimal attributes your domain needs and follow the best practices https://docs.datomic.com/cloud/best.html#datomic-schema

idiomancy15:09:25

IF things work out, moving to datomic proper might not be inconceivable so...

linus_gv20:09:34

Is there like a Merkle hash for a database?

okocim21:09:19

what is the proper way to read the body on an api-gateway ionized fn? I’m getting nil in the body on a POST, but the json is appearing on a :datomic.ion.edn.api-gateway/json key

okocim21:09:43

the ion-example doesn’t ionize any fns that actually read the POST body from what I can tell

kenny21:09:47

@okocim I am using Ions in a production environment and the body is available under the :body key in my Ring handler that I pass to ionize.

okocim21:09:36

thanks @kenny. Turns out I was passing a bad parameter on my test requests :face_palm: