datomic 2019-12-12 | Slack Archive

Seeing memory allocation errors on the BeforeInstall step during deploy to a solo Datomic cloud instance.

[stderr]OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000faa00000, 72351744, 0) failed; error='Cannot allocate memory' (errno=12)

grzm17:12:46

If I recall in the past when this happens, just deploying again fixes it. Doesn't seem to be the case this time. No new deps or big code changes. Version 535-8812 (solo)

grzm17:12:40

The rollback after the failed deploy also is failing.

grzm18:12:27

Ended up turning down the Autoscaling from 1 instance to 0, and back to 1 to bounce the solo instance. Deploy then started working again.

steveb8n20:12:47

I have the same problem. Roughly every 3rd ci build. To fix I just kill the EC2 instance and the alb brings up a new one automatically

steveb8n20:12:28

The deploy works again. It has to be memory

grzm20:12:14

That sounds painful. Terminating the instance is probably a faster way than modifying auto-scaling each time (twice!). Thanks for the tip!

steveb8n23:12:42

glad to help. just make sure you don’t terminate the bastion 🙂

steveb8n22:12:44

Q: I’m about (tomorrow) to create a production cloud topology. I’d like to verify my thinking before I do this. Is there someone at Cognitect I can talk to about this?

2019-12-12

Channels