Amazon Web Services Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – In AWS CloudWatch, is it possible to graph a sum of different metrics per minute?

There are multiple servers doing data stream processing, and each of them is sending two custom metrics to AWS CloudWatch every minute:

  • RecordCount
  • RecordCount-{server_id}

The metric RecordCount is the number of records processed in the last minute.

For example, for 2 servers and 2 minutes, the metrics sent to CloudWatch could be:

timestamp: 2024-11-06T10:00:12 RecordCount: 42
timestamp: 2024-11-06T10:00:48 RecordCount-1: 42
timestamp: 2024-11-06T10:00:32 RecordCount: 58
timestamp: 2024-11-06T10:00:33 RecordCount-2: 58
timestamp: 2024-11-06T10:01:08 RecordCount: 88
timestamp: 2024-11-06T10:01:09 RecordCount-1: 88
timestamp: 2024-11-06T10:01:56 RecordCount: 15
timestamp: 2024-11-06T10:01:57 RecordCount-2: 15

I’m trying to create a couple of graphs with essentially the same logic: the sum of all of the metrics with the same name, per minute. CloudWatch doesn’t seem to support that concept. If I use a SUM(RecordCount) as a metric to graph, it will sum an arbitrary number of metrics needed to create the graph, i.e. the number of data points it will sum to make the graph doesn’t align with the 1-minute intervals I have, so graphs are pretty much meaningless.

From the above example data, I’d like to have a graph with the RecordCount label with two data points, with these X,Y coordinates:

X: 2024-11-06T10:00:00 Y: 100
Y: 2024-11-06T10:01:00 Y: 103

Note that the timestamps aren’t aligned to the second. So whatever bucketing is to be done, should sum all records from the same minute.

Is this even possible?

Tags:

2

Answers


  1. 0

    For this i would think to do it with two different solutions:

    First using cloudwatch log insight to aggregate them and then use that query to create graphs:

    fields @timestamp, @message 
| parse @message "RecordCount-*: *" as serverId, count 
| stats sum(count) as RecordCountTotal by bin(1m)

    or create a Lambda function to create a new metric for me from the aggregate RecordCount-* and name it RecordCountTotal for example

    1. Use the GetMetricData API in the Lambda function to retrieve metrics.
    2. Aggregate them per minute.
    3. Use the PutMetricData API to publish a new aggregated metric (e.g., RecordCountTotal)

    Then, you can graph RecordCountTotal in cloudwatch.

    Login or Signup to reply.
  2. 0

    After selecting your metrics in the "Browse" section from the CW Dashboard or CW Metrics, you can go to the "Graphed metrics" tab and set the following 2 options:

    • Statistic: Sum
    • Period: 1 minute

    Also, for the line chart display, I’m not a big fan of the default one as there is no line when there is no data… But you could fix that with following:

    1. Add math > Common > Fill
    2. If you have multiple metrics, replace METRICS() with the ID of your expression (the ID column will appear only once you will add a custom expression)
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search