aws 2021-06-02 | Slack Archive

zackteo04:06:09

Hello, a more generic AWS question here, but is I have a python script that I want to offload to AWS, because it needs way too much RAM and runs for many hours to generate a model (similar to ML but for Optimisation). Is AWS Lambda or Fargate what I need?

zackteo04:06:04

The alternative to have run the script on a server instead

Daniel Craig05:06:24

Lambdas cannot run for longer than 15 minutes as a hard limit, so that might eliminate Lambda for your use case

Daniel Craig05:06:30

EC2 would be the most akin to running your script the way it’s already running; EC2 just gives you a virtual machine to run your program on, you can install python and run your script

Daniel Craig05:06:11

I don’t think you need fargate

zackteo07:06:49

Right, I just looked at https://greenm.io/aws-lambda-or-aws-fargate/ and seems like fargate might fit quite well. Or do you still recommend EC2? I agree that EC2 is most akin to doing so. But would serverless be a better way since it is like a one time use thing?

zackteo08:06:03

Maybe Ill just go with EC2 after all, there is no option to use fargate with aws educate it seems

raspasov10:06:53

I would say go with EC2 to start. It’s the “simplest” in a way, and things like Lambda, etc are more akin to cost optimization strategies.

zackteo11:06:29

Right, that makes sense! Will go with that! Think that workflow might work out better after all

dabrazhe13:06:41

The most flexible approach would be to containerise your app. you can start with running it on EC2 and when comfortable migrate to Fargate. you can also consider Sagemaker that is geared towards ML, but it's more expensive

2021-06-02

Channels