@alandipert: Ping

here

I haven’t looked at Kinesis yet

it's pretty neat

basically kafka on AWS, if you are familiar

Only vaguely.

So would the best thing be to put data right into Kinesis from the API endpoint, and then pull off and only mark as processed when it’s successfully written?

the cool thing is the various consumers of the kinesis stream you may have maintain their own idea of where in the stream they are

and if your consumers use the kinesis consumer library to do their receiving, this is handled automatically by the library, which stores things in dynamo

So if that stores in dynamo, do I actually gain any robustness by using kinesis?

well, the robustness of aws things is a factor of scale

considering the scale it can go, it's very robust

but maybe not as robust as a smaller thing with fewer AWS dependencies

Yeah, I see.

It’s hard to tell whether it’s more or less robust than a single server with dodgy monitoring set up by a monkey (me)

Looks like I can’t shovel data from an API endpoint right into Kinesis, sadly

If I have to worry about the data consumer going down anyway, I think I’d just put it right in Dynamo

And then on any error just send a detailed email via something non-AWS (mailgun)

that sounds legit

Much as I would like to think that thousands of people will be purchasing Cursive per second, I suspect I’m unlikely to have a legitimate big data problem

hehe

maybe immutant is in your ballpark?

i've looked at it for medium projects, seems like a nice way to arrange things

that also gives you a nice layer between you and the linode os

My main goal isn’t gigantic scale, it’s not having to maintain and monitor things.

I’m still fairly tempted by something bare-bones with just an embedded k/v store on my server, but Lambda seems like a pretty easy option.

And in theory, it should just stay working once I have it set up.

one of my favorite things about lambda is it automatically does what most bad programs need done

on some basis

which is, be restarted

Doesn’t it just get its dodgy state serialised?

heh, true

especially if it's doing something useful and writing to dynamo

> My main goal isn’t gigantic scale, it’s not having to maintain and monitor things What about something like GAE?

(I’ve never actually used GAE)