skip to Main Content

I found articles how to send alerts from Amazon Prometheus to Microsoft Teams with AWS SNS & Lambda. I configured everything according to articles. When I send test payload in Lambda, everything is ok, Lambda sends a message. But when I send the same test payload with SNS, I receive an error in Lambda (look below). What is the problem with test payload or Lambda or SNS? SNS Subscription configured to Lambda correctly.

[ERROR] KeyError: 'alertname'
Traceback (most recent call last):
  File "/var/task/teams-alerting-lambda.py", line 18, in lambda_handler
    alertname = message['alertname']

I added ‘print(event)’ command. When sending payload in SNS i receive in Lambda logs:

{'Records': [{'EventSource': 'aws:sns', 'EventVersion': '1.0', 'EventSubscriptionArn': 'ARN', 
'Sns': {'Type': 'Notification', 'MessageId': '', 'TopicArn': 'ARN', 'Subject': None, 
'Message': '{n  "Records": [n    {n      
"Sns": {n        
"Message": "{\"alertname\": \"CPU_ALERT_333\", \"status\": \"firing\", \"host_name\": \"host1\", \"cloud_account_id\": \"1234567890\", \"cloud_region\": \"eu-west-1\", \"summary\": \"High CPU Load\", \"severity\": \"warning\"}"n      }n    }n  ]n}', 
'Timestamp': 'TIME', 'SignatureVersion': '1', 'Signature': '...', 'SigningCertUrl': 'URL', 'UnsubscribeUrl': 'URL', 'MessageAttributes': {}}}]}

Payload:

{
  "Records": [
    {
      "Sns": {
        "Message": "{"alertname": "CPU_Alert", "status": "firing", "host_name": "host1", "cloud_account_id": "1234567890", "cloud_region": "eu-west-1", "summary": "High CPU Load", "severity": "warning"}"
      }
    }
  ]
}

Part of Python code that is responsible for handling payload:

13 def lambda_handler(event, context):
14    logger.info("Event: " + str(event))
15    message = json.loads(event['Records'][0]['Sns']['Message'])
16    logger.info("Message: " + str(message))
17
18    alertname = message['alertname']
19    status = message['status']

2

Answers


  1. The Message is a string like this:

    "{"alertname": "CPU_Alert", "status": "firing", "host_name": "host1", "cloud_account_id": "1234567890", "cloud_region": "eu-west-1", "summary": "High CPU Load", "severity": "warning"}"
    

    You need to decode the string into a dict.

    I.E. json.loads(thestring)

    > foo = "{"alertname": "CPU_Alert", "status": "firing", "host_name": "host1", "cloud_account_id": "1234567890", "cloud_region": "eu-west-1", "summary": "High CPU Load", "severity": "warning"}"
    > foo
    '{"alertname": "CPU_Alert", "status": "firing", "host_name": "host1", "cloud_account_id": "1234567890", "cloud_region": "eu-west-1", "summary": "High CPU Load", "severity": "warning"}'
    > import json
    > bar = json.loads(foo)
    > bar['alertname']
    'CPU_Alert'
    

    Catch the KeyError and see what you got:

    foo = "{"notalertname": "surprise"}"
    try:
        message = json.loads(foo)
        alertname = message['alertname']
    except KeyError:
        print(f"KeyError: {message.items()}")
        
    KeyError: dict_items([('notalertname', 'surprise')])
    
    Login or Signup to reply.
  2. You have to use ast python package on the event first to get the valid message, and then you can use json to get your alertname:

    import ast
    
    def lambda_handler(event, context):
        logger.info("Event: " + str(event))
    
        records = ast.literal_eval(event['Records'][0]['Sns']['Message'])
        message = json.loads(records['Records'][0]['Sns']['Message'])
        logger.info("Message: " + str(message))
    
        alertname = message['alertname']
        status = message['status']
    

    p.s.
    May need to implement iteration over all event['Records'] in case you get more then one record from sns.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search