To enable cross-account S3 data sharing we decided to implement use of customer-managed KMS keys. Before we used AWS default KMS key (SSE-S3) and things worked great. We expected the switch to customer-managed keys (SSE-KMS) to be seamless, as in both cases the process is transparent for a client: keys are managed at AWS S3 API backend side somehow.
We use S3 bucket keys to allow more effective caching and lower KMS costs.
After re-ecrypting all our data we noticed hundreds of dollars KMS costs per day. Why is this happening?
2
Answers
It was quite hard to understand why this happens, the answer does not lay on the surface.
We had to ask AWS Support and after some time received a confirmation that is a normal behavior for SSE-KMS and there is nothing we can do about it. We decided to switch back to SSE-S3.
The following is AWS Support response:
With reading pattern for data in hundreds of thousands of files read daily, costs of KMS can be like you observe.
Check KMS pricing model.
It’s $1 per key and $0.03 per 10k requests to use this key.
For cross account acces you have to use KMS key as you have to adjust key policy to allow access to encryption key from different accounts.
There is a feature called Storage Leans in S3 service. You can use it to investigate operations patterns. In most cases expected usage is far away from reality.
I’d start from investigation of access patterns and apply optimization when possible.