Question 71892 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

C# : How to AWS S3 Bucket Size – Amazon web services

I want to get bucket size of AWS S3 by using .net.

Want to calculated allocated space, used space for the bucket.

  1. If my bucket name is XYZ.
  2. Inside my bucket there are n numbers of subfolders.
  3. I want complete size of XYZ bucket.

for this access your have to give permission to bucket for public access through S3 browser or AWS console. Will get output like below

<Key>File Name</Key>
<LastModified>Last Modified Date</LastModified>
<ETag>Some Text</ETag>
<Size>173266</Size>

Size : in Bytes

Now I have calculate one by one file size and then to do calculations.

If any one can suggest good idea to get complete bucket size in one method that would be very good for me.

Tags:

2

Answers


  1. 0

    There is no API call available to obtain the size of an Amazon S3 bucket.

    You can either:

    • Call ListObjects() to obtain a listing of all objects and total their sizes (note: Only 1000 objects are returned per call, so it would need to loop until all objects are returned), or
    • Use Amazon S3 Inventory, which can provide a daily or weekly CSV file listing all objects. You can then calculate the total bucket size from that file.
    • Use Amazon CloudWatch Metrics that list the size of every bucket. From Monitoring metrics with Amazon CloudWatch – Amazon Simple Storage Service: "These storage metrics for Amazon S3 are reported once per day and are provided to all customers at no additional cost.

    Another option (I haven’t tried it) might be to use Amazon S3 Storage Lens, which can track metrics in S3.

    See also:

    Login or Signup to reply.
  2. 0

    Cloud storage has no folders. Whether it’s AWS S3 or Azure Blob storage, a bucket is a flat list of files. Folders are emulated by treating a character in the name as a separator. A bucket isn’t a disk either, so it has no allocated vs used space statistics. Objects are stored in blob storage arrays that use compression and deduplication.

    If you want to find out what storage you’re actually billed for, it’s the total object size, not the actual physical storage. If you upload the same 1MB file 100 times you’ll be billed for 100MB even if just 1MB are actually stored.

    Calculating the total size of a bucket is too expensive, which is why it’s not updated automatically. If calculating the size of a local folder is slow, imagine how much slower it is to calculate the size of a bucket with 1M objects that are actively being updated.

    The only way to get the used space is to list the objects and sum their sizes. This operation is billed, no matter what tool is used to perform it.

    The article Find out the size of your Amazon S3 buckets shows which tools and services can be used to monitor the actual size.

    • Amazon S3 console
    • S3 Storage Lens
    • Amazon CloudWatch
    • Amazon S3 Inventory

    These services calculate and cache the size periodically, to avoid delays and extra billing.

    The other option is to list and sum all objects either through the AWS CLI or code. You can use the code in Listing object keys programmatically to list all objects in a bucket one page at a time and sum the sizes:

    public static async Task<long> GetBucketSizeAsync(IAmazonS3 client, string bucketName)
{
    long total=0;
    var request = new ListObjectsV2Request
    {
        BucketName = bucketName,
    };

    var response = new ListObjectsV2Response();

    do
    {
        response = await client.ListObjectsV2Async(request);

        total += response.S3Objects.Sum(obj =>obj.Size);

        request.ContinuationToken = response.NextContinuationToken;
    }
    while (response.IsTruncated);

    return total;
}

    Remember, ListObjectsV2Async is billed. You should cache the results instead of calling this all the time.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search