Fork me on GitHub
#aws-lambda
<
2017-10-21
>
valtteri06:10:48

Read/write units really are confusing. By default the UI proposes 5 units for both but for dev and testing you can happily drop them to minimum (1) to save costs.

valtteri06:10:30

Another confusing thing is indexes. You need those if you want to query efficiently using any other attribute than the primary key.

valtteri07:10:27

Best thing with DynamoDB is that it’s fully managed (like S3). If you can live with the limitations, it’s a very care free solution.

qqq07:10:16

I read that DynamoDB shards tables in 10GB chunks

qqq07:10:23

and then splits the read/write units over the chunks.

qqq07:10:53

So if you have an 100 GB table, and 10 WRU -- what happens is that there are 10 shards and each shard gets only 1 WRU. Is that true? If so, it sounds really limiting.

valtteri08:10:41

That’s true. If your dataset is really huge you also need to worry about the underlying partitioning. You need to choose your hash keys so that your data gets evenly distributed between the shards.

valtteri08:10:46

Also if you need to access data by several keys (indexes), the indexes consume their own capacity and not the table’s capacity.. Which practically means that you need to pay extra for each index.. Or to be precise, there are two kinds of secondary indexes, ‘local’ and ‘global’. Global indexes consume their own capacity and local consume table’s capacity.

valtteri09:10:13

My experience is that if you have complex data access patterns you need to plan really carefully how you’re going to query the data considering all DynamoDB’s limitations. If you can’t predict how you need to access the data, then it might be simpler to go with SQL and RDS. Another common solution is to use a search engine (i.e. ElasticSearch or CloudSearch) with DynamoDB. You can invoke Lambda functions for each operation on your table (DynamoDB Streams) that can do the indexing to the search engine. CloudSearch also provides some kind of OOB solution to setup indexing automagically from DynamoDB.. But at least a while ago it could deal only with flat data structures.

valtteri09:10:37

Oh, one more DynamoDB limitation which I should mention, is the lack of support for server side encryption. Java SDK supports client side encryption though.

valtteri09:10:45

Does anyone know if there’s some util/tool/framework to write CLJS, output it to lambda-compatible JS and package&deploy to AWS using Serverless Framework or AWS SAM? Or would it be to trivial to create such setup myself.. Or does it sound like a bad idea?