architecture 2020-02-28 | Slack Archive

jlmr07:02:14

@adam622 thanks for the tips! All these tips have been very helpful! Do you know about PaaS-services on AWS (like Beanstalk) that you can use for long-running processes? As far as I know Beanstalk is specifically for web servers, but I could be wrong.

👍 4

adamfeldman16:02:55

My pleasure! What sort of long-running processes? Are you referring to a stateful system, like Datomic?

jlmr19:02:40

More like a worker that periodically does some proocessing

jlmr19:02:02

For storage we will probably just use s3 and a managed db

adamfeldman19:02:29

Depending on how periodic and what the load is, you may want to explore using Lambda for the worker. Fargate would probably also be straightforward to use

orestis10:02:32

Beanstalk has a "web" mode and a "worker" mode. Essentially worker mode isn't exposed to the internet, but is meant to process events from an SQS queue.

orestis10:02:36

They will POST events from a selected SQS queue to a web endpoint (so you don't have to deal with the SQS queue yourself). I'm not sure what happens if you get no events at all from the queue, but I think the servers just stays up and does what you want it to do.

jlmr19:02:58

Ok, good to know. I will have to compare this option to using Fargate to figure out what would be the best option for our use case

lukasz19:02:48

I'm looking into Fargate right now and you can definitely run long-running processes without a http server

2020-02-28

Channels