Question 71792 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

What decides the number of partitions in a DynamoDB table? – Amazon web services

I’m a beginner to DynamoDB, and my online constructor doesn’t answer his Q/A lol, and i’ve been confused about this.
I know that the partition key decides the partition in which the item will be placed.
I also know that the number of partitions is calculated based on throughput or storage using the famous formulas

So let’s say a table has user_id as its partition Key, with 200 user_ids. Does that automatically mean that we have 200 partitions? If so, why didn’t we calculate the no. of partitions based on the famous formulas?
Thanks

Tags:

2

Answers


  1. 0

    The partition key value is hashed to determine the actual partition to place the data item into.

    Thus the number of distinct partition key values has zero affect on the number of physical partitions.

    The only things that affect the physical number of partitions are RCUs/WCUs (throughput) and the amount of data stored.

    • Nbr Partions Pt = RCU/3000 + WCU/1000
    • Nbr Partions Ps = GB/10

    Unless one of the above is more than 1.0, there will likely only be a single partition. But I’m sure the split happens as you approach the limits, when exactly is something only AWS knows.

    Login or Signup to reply.
  2. 0

    Let’s establish 2 things.

    1. A DynamoDB partition can support 3000 read operations and 1000 write operations. It keeps a divider between read and write ops so they do not interfere with each other. If you had a table that was configured to support 18000 reads and 6000 writes, you’d have at least 12 partition, but probably a few more for some head room.
    2. A provisioned capacity table has 1 partition by default, but an on-demand partition has 4 partitions by default.

    So, to answer your question directly. Just because you have 200 items, does not mean you have 200 partitions. It is very possible for those 200 items to be in just one partition if your table was in provisioned capacity mode. If the configuration of the table changes or it takes on more traffic, those items might move around to new partitions.

    There are a few distinct times where DynamoDB will add partitions.

    1. When partitions grow in storage size larger than 10GB. DynamoDB might see that you are taking on data and try to do this proactively, but 10GB is the cut off.
    2. When your table needs to support more operations per second that it is currently doing. This can happen manually because you configured your table to support 20,000 reads/sec where before I only supported 2000. DynamoDB would have to add partitions and move data to be able to handle that 20,000 reads/sec. Or is can happen automatically to add partitions because you configured floor and ceiling values in DynamoDB auto-scaling and DynamoDB senses your ops/sec is climbing and will therefore adjust the number of partitions in response to capacity exceptions.
    3. Your table is in on-demand capacity mode and DynamoDB attempts to automatically keep 2x your previous high water mark of capacity. For example, say your table just reached 10,000 RCU for the first time. DynamoDB would see that is past your previous high water mark and start adding more partitions as it tries to keep 2x the capacity at the ready in case you peak up again like you just did.
    4. DynamoDB is actively monitoring your table and if it sees one or more items are particularly being hit hard (hot keys), are in the same partition and this might create a hot partition. If that is happening, DynamoDB might split the table to help isolate those items and prevent or fix a hot partition situation.

    There are one or two other more rare edge cases, but you’d likely be talking to AWS Support if you encountered this.

    Note: Once DynamoDB creates partitions, the number of partitions never shrinks and this is ok. Throughput dilution is no longer a thing in DynamoDB.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search