Fork me on GitHub
#datomic
<
2018-04-13
>
marshall13:04:46

@cjsauer you don’t need to use vpc peering if your lambda is running in the datomic cloud VPC

Chris Bidler14:04:13

@cjsauer When you say “keeps timing out” do you mean you are running it again immediately after it times out and getting the same result? It might be that your initial run of the lambda is super slow because unless things have changed since I learned this, the first invocation of a VPC-hosted lambda has to wait on an ENI being provisioned and attached to the VPC

cjsauer16:04:47

@U07HA15PY ah ok! Thanks for the reply. So I've set the Lambda timeout well above 2 minutes, and noticed the same issue. Could it be that there is a timeout somewhere in Datomic that is the culprit? Maybe I just need to bump that up...

cjsauer16:04:47

It's definitely the call to d/connect that is failing. d/client seems to execute just fine.

Chris Bidler16:04:18

The only other thing I’d recommend checking is security group settings - you will absolutely get a timeout if you try to access a port with no security group rule allowing ingress; this is a security feature in the same way that S3 returns a 403 for anything it can’t find, rather than a 403 for things that exist and you don’t have permissions for and a 404 for things that don’t exist

cjsauer17:04:21

That makes sense, I'll keep digging. I have indeed added my Lambda function to the "Apps" security group mentioned in the docs: >Inside this VPC, the stack also creates an applications security group named $(SystemName)-apps that you can use for you client applications. The security group that the Datomic system instances run in allows access from the applications security group. Curious though if maybe I need to do additional custom tweaks to the security group? It seems though like things are configured to be "plug-and-play", at least that's my interpretation of the docs.

Chris Bidler14:04:24

between that and say JVM startup, you could easily hit a timeout. 🙂

davidw14:04:01

Is there any way to ensure uniqueness using multiple attributes in datomic cloud? With on prem I believe I could use a transaction function but that's not an option with cloud. Is there another option?

eraserhd15:04:51

Do you mean have multiple fields of an entity determine uniqueness, or do you mean the same value (like a uuid) can't be used for two different attributes?

davidw16:04:15

I mean have multiple fields of an entity determine uniqueness. In my case they're both strings and it would be safe to concatenate them to produce a third field which I can make unique, but that feels a little clunky.

favila16:04:31

Always writing all three fields (two source plus the composite unique index field) at the same time is the only way without a transaction function. We do use this technique (composite index field) with the peer lib. It's a little clunky but not terrible

eraserhd16:04:33

I don't know anything about Cloud. I think the transaction function thing is the only way to enforce this sort of thing (scroll up) to April 11, but that doesn't work on cloud.

eraserhd16:04:41

I had this kind of scenario once, and IIRC, when I figured out some modeling problems it went away. So... not saying your problem will go away... but I am curious about the specifics.

cjsauer20:04:05

Still having issues trying to connect a Lambda function with Datomic Cloud via the client API...I may be misunderstanding this passage in the docs: >Inside this VPC, the stack also creates an applications security group named $(SystemName)-apps that you can use for you client applications. The security group that the Datomic system instances run in allows access from the applications security group. I'm taking this to mean that if I add my Lambda function to the Datomic VPC, with access to all 3 subnets, and I have set the Lambda function to use the "AppsSecurityGroup" security group, should this be sufficient to connect to Datomic? I'm still seeing timeout errors as above, but have tried everything I can think of...including giving complete admin access to my Lambda functions in a last ditch effort. What else could I be missing? Is there some extra security group magic that I need to tweak?

cjsauer20:04:33

I'm attempting to use the endpoint: http://entry.%s.%s.datomic.net:8182, as it looks like the Route 53 hosted zone for Datomic is attached to both the default VPC, and the Datomic VPC, so I'm assuming that this address should resolve properly...

cjsauer20:04:14

Just trying to rule out all possibilities...I've tried giving the function explicit permission to one specific Datomic db as the docs describe, but still no luck.

cjsauer21:04:34

Might be helpful to reiterate that this is the error I'm seeing:

Unable to execute HTTP request: Connect to 
...
connect timed out
Specifically, it's the http://s3.amazonaws.com request on port 443 that's failing. Sorry to be flooding the channel, I'm just a bit out of my element here with these networking issues 😓

marshall13:04:55

@cjsauer What is in your connection config map?

marshall13:04:23

I just saw that you got it sorted. I’ll look into adding the information about public NAT access to the docs.