datomic 2023-07-26 | Slack Archive

Thomas Steffes18:07:14

hey all - we're running datomic cloud ions in our AWS account. In the past few days we've had several deploys fail at the deploy step (not push ) with an error message that indicates a memory issue. Here's the exact error message I see in the the codedeploy log:

Cannot allocate memory - rm -rf /opt/codedeploy-agent/deployment-root/13f5525c-d97a-48d8-919a-66598fdefe7e/d-JVVQ2DCQ0 2>&1

The first time we ran into this issue we terminated the EC2 instance, which was then replaced by the autoscaling group, and were able to deploy after that. Today the issue happened again - curious if anyone else has run into this issue or has some guidance. For a bit more context, we're running a t3.small instance, which I know is pretty limited but I'd like to dig into the issue a little bit before just bumping up to a larger instance type. Appreciate the assistance.

jaret11:07:08

Its definitely memory on a t3, but if you would like to dig in you can do the following to give you more information. 1. At what step does "allocate memory" throw? Before Install, sync libs ? 2. You can and should be able to monitor the ev2 available mem during a deploy on an instance from cloudwatch. 3. test locally at the jvm settings for t3: https://docs.datomic.com/cloud/ions/ions-reference.html#jvm-settings

2023-07-26

Channels