Question 102288 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – Automate CW logs forwarding from stacks as they're added

I’d like to forward CloudWatch logs from various SAM stacks/lambdas (probably to Kinesis and then) to Graylog EC2 instances. The point is that I’d also like to automatically add logs from stacks I deploy later.

How do I automate the process of adding new CW lambda log groups to a stream ending up in Graylog? Or — how can I automatically deploy subscriptions to the log groups once they appear in CloudWatch?

Thank you.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I was building on Marcin's response. The final solution looks quite complicated but it just works once it's done.

    Several CloudFormation resources need to be provisioned here:

    • A Kinesis stream used by Graylog as an input.
    • A role allowing CloudWatch to put logs into the Kinesis stream.
    • A role and an instance profile allowing Graylog EC2 instances access to the Kinesis stream.
    • A DynamoDB table used by Graylog as a metadata storage.
    • A lambda to take care of CloudWatch log group discovery and subscription management.
    • The discovery lambda log group.
    • EventBridge rules:
      • Hourly log group discovery lambda trigger.
      • A trigger invoking the log group discovery lambda upon CloudFront stacks changes.
    • The log group discovery lambda invocation permissions for each of the rules.

    There were also some manual steps that I needed to perform manually (our environment is not CloudFormation based):

    1. Add the generated instance profile as a role to each of the Graylog EC2 instances.
    2. Install the graylog-integrations-plugins package on all Graylog EC2 instances and restart their respective graylog-server services.
    3. Create a new AWS Kinesis/CloudWatch Graylog input and point it to the Kinesis stream created by this stack.
    4. Create VPC endpoints for the following services so that Graylog doesn't need to call AWS APIs via the Internet (which is costly):
      • A DynamoDB gateway
      • A CloudWatch logs interface
      • A CloudWatch monitoring interface
      • A Kinesis streams interface

  2. 0

    You need custom solution for that. One way would be to setup CloudWatch Event rule for CloudFormation events which responds to CREATE_COMPLETE.

    The event would trigger a lambda function, which you have to program, to create the needed CloudWatch groups with subscriptions.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search