Fork me on GitHub
#aws
<
2015-08-28
>
robert-stuttaford07:08:07

we’re using circleci + aws codedeploy

robert-stuttaford07:08:17

and cloudformation

robert-stuttaford07:08:26

and a metric shit-ton of coffee

alandipert12:08:08

@robert-stuttaford: wuold be curious to hear more about your experience with codedeploy

statonjr12:08:20

@robert-stuttaford: We’re using TravisCI with CodeDeploy and CloudFormation.

alandipert15:08:10

is anyone using Spark in EMR by any chance?

alandipert15:08:58

experiencing what looks like JVM OOM death ("ExecutorLostFailure"), despite a strategic RDD persist and StorageLevel.MEMORY_AND_DISK_SER memory/spill setting

alandipert15:08:40

vanilla yarn settings and only setting executor counts/memory max in spark config

lowl4tency20:08:31

AWS channel, awesome

lowl4tency21:08:47

statonjr: how do you release production? simple_smile

lowl4tency21:08:47

I mean workflow

statonjr21:08:19

@lowl4tency: Production releases are deployed manually using AWS Console or aws-cli.

lowl4tency21:08:04

statonjr: what about git workflow? do you have separate branch for production?

lowl4tency21:08:29

statonjr: yes, manually running deploy makes sense

lowl4tency21:08:43

alandipert: you can ask me about codedeploy simple_smile

statonjr21:08:45

Git workflow: GitHub -> Travis -> AWS Staging environment. If all looks good, then manual push. No separate production branch.

lowl4tency21:08:34

statonjr: so, do you run new build for production? or use exist bundle?

statonjr21:08:18

We use the existing bundle.

lowl4tency21:08:49

statonjr: ow do you manage environment variables for hosts?:)

lowl4tency21:08:23

statonjr: I'm asking because it's interesting how other people solve same tasks simple_smile

statonjr21:08:05

We have some hacks for environment variables, but we’re transitioning to Consul.

lowl4tency21:08:39

As far as I know, Consul is not best solution for this 😞

lowl4tency21:08:54

For managing variables for multiple hosts

statonjr21:08:31

We’re happy with Hashicorp tools so far. I’ll report back any issues.

lowl4tency21:08:26

statonjr: interestin to take a look on the solution