I have several .NET WebApi microservices hosted in AWS via ECS. These services publish Open Telemetry metrics to an AWS-specific OpenTelemetry Collector (ADOT).
Each developer has a personal AWS account we use as a sandbox for testing during development. Then there are dev/prod accounts which only our CI/CD pipelines deploy to. To keep costs down, I’d like to have metrics not sent to CloudWatch for only the personal sandbox accounts. However, in dev/prod accounts they should still be published. Furthermore, I want to avoid nasty conditional logic in my CDK projects for this if possible.
One thought I had was maybe using a processor in the OTel Collector configuration YAML to filter out (drop) metrics that had a label attached to them and match by account ID. But by default, at least with the OTel .NET SDK, the AWS Account ID is not amongst the attributes sent.
Is my approach for this right? If not, what would be the most effective (least impact, least friction, most maintainable) approach? If I am on the right track, I need to figure out how to utilize the .NET SDK to grab the account ID and stuff that into metrics that get published. That could get complicated so I just want to make sure my head is in the right place before I continue down this path.
2
Answers
You can use
AWS Distro for Open Telemetry (ADOT)for filtering out the CloudWatch metrics as documented hereThe way I’d attack this is at the SDK level inside your app.
Add a setting to your
appsettingsthat isenableMetrics(or something like that, maybe you already have a section where this would make sense). Default it to false, then setup the CI/CD pipelines to set that settings to true.Then you can either, wrap the
.AddOtlpExporter()part specifically, or wrap the whole TracerProviderBuilder. I’d advise just removing the exporter since then you’re still generating the telemetry internally which does have an effect on logic.