About me

DynamoDB

DynamoDB Backups

It is possible to have our requests throttled, even if the table’s provisioned capacity / consumed capacity appears healthy like this:

This has stumped many users of DynamoDB, so let me explain.

When creating a table in DynamoDB, you provision capacity / throughput for a table. Being a distributed database (made up of partitions), DynamoDB under the covers, evenly distributes its provisioned throughput capacity, evenly across all partitions.

Best practice for DynamoDB recommends that we do our best to have uniform access patterns across items within a table, in turn, evenly distributed the load across the partitions.

Don’t worry too much about being strict about uniform access, I’ve rarely seem perfectly distributed data in a table, you just need it distributed enough.

Unfortunately, DynamoDB does not enable us to see:

1. The partitions for each table

2. The throughput capacity allocated to each partition

3. Which partition each item is allocated to

This makes it very difficult to predict throttling caused by an individual “hot partition”.

A hot partition is a partition that receives more requests (write or read) than the rest of the partitions.

To add to the complexity, the AWS SDKs tries its best to handle transient errors for you. Things like retries are done seamlessly, so at times, your code isn’t even notified of throttling, as the SDK will try to take care of this for you.

This is great, but at times, it can be very useful to know when this happens.

The first step you need to focus on is creating visibility into your throttling, and more importantly, which Partition Keys are throttling.

You can do this by hooking into the AWS SDK, on retries or errors.

The AWS SDK has some nice hooks to enable you to know when the request you’ve performed is retrying or has received an error.

Keep in mind, an error means the request is returned to your application, where as a retry means the SDK is going to retry again.

Below is a snippet of code to demonstrate how to hook into the SDK.

Once you can log your throttling and partition key, you can detect which Partition Keys are causing the issues and take action from there.

I found this to be very useful, and a must have in the general plumbing for any application using DynamoDB.

HTH

Click to share on Twitter (Opens in new window)

Click to share on LinkedIn (Opens in new window)

Click to share on Reddit (Opens in new window)

Click to share on Google+ (Opens in new window)

Click to share on WhatsApp (Opens in new window)

Click to share on Skype (Opens in new window)

Click to share on Facebook (Opens in new window)

Click to email this to a friend (Opens in new window)

AWS Specialist, passionate about DynamoDB and the Serverless movement.

Currently focusing on helping SaaS products leverage technology to innovate, scale and be market leaders.

Introduction to CloudFormation

Getting Started with DynamoDB

DynamoDB in Production

Intro to CloudFormation Course

Using DynamoDB in Production – New Course

DynamoDB: Monitoring Capacity and Throttling

Pluralsight Course: Getting Started with DynamoDB

Partition Throttling: How to detect hot Partitions / Keys

AWS

Lambda

Pluralsight

Presentations

Rants

Uncategorized

Type and press Enter to search