Question 259056 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – Choosing Between CloudWatch and a Separate RDS Instance for Event Data Storage in AWS Analytics

I’m currently working on data analytics for our organization using AWS services, specifically AWS QuickSight for data visualization. We have successfully connected a PostgreSQL database as a data source to QuickSight.

For event tracking within our system, we’ve been using AWS CloudWatch. However, the process of connecting QuickSight to CloudWatch involves setting up a Lambda function to copy events to an S3 bucket, adding AWS Glue, AWS Glue Data Catalog, Athena, and finally connecting it all to QuickSight. This seems quite complex and involves multiple steps.

I’m now considering whether it makes sense to switch to using a separate RDS instance for storing event data. It would certainly simplify the process of connecting to our analytics tools.

My question is, why would one choose to use CloudWatch for event tracking if extracting and using the data from it is such a hassle? Are there specific advantages or scenarios where CloudWatch is a more suitable choice over using a dedicated RDS instance for event data storage in an analytics context?

Tags:

2

Answers


  1. 0

    My question is, why would one choose to use CloudWatch for event tracking if extracting and using the data from it is such a hassle?

    Some advantages of using CloudWatch are:

    • It is a "serverless" managed service. You don’t have to stand up any servers or maintain any servers.
    • It is billed based on usage, where your RDS server will be billed 24/7 regardless of usage.
    • Many (most) people don’t need to pull CloudWatch data into QuickSight, so they aren’t running into the issues that you are.
    • If you just need to be able to access the data via the AWS console and see charts of your data, and/or have some automatic alerts go out based on certain event data/metrics, then CloudWatch works fine.

    In my experience, CloudWatch is great for sending event metrics from your application that you simply need to monitor and trigger some alerts on. As long as you don’t care about extracting that data into some other tool, then why not use CloudWatch?

    You appear to have a specific use case where CloudWatch is not a great fit. If you primarily want your event data to be accessible in some business intelligence tool like QuickSight, then by all means, send that event data to a database instead. Amazon’s Redshift analytics database might be a better fit than RDS.

    I would also note that your current solution of CloudWatch -> Lambda -> S3 -> Glue -> Glue Data Catalog -> Athena could be modified to CloudWatch -> Lambda -> RDS if you are considering storing the data in RDS.

    Login or Signup to reply.
  2. 0

    I’ll try to address what I think is more or less objectively answerable part:

    Are there specific advantages or scenarios where CloudWatch is a more suitable choice over using a dedicated RDS instance for event data storage in an analytics context?

    It’s already there

    For most AWS services, CloudWatch just works. When you create a Lambda, or an API gateway, or just about other native AWS service that emits lots of log data, CloudWatch either works out of the box, or you can enable it with a single checkbox.

    Money

    As long as your data is mostly just sitting there (which is the way most people use it), CloudWatch is way cheaper than RDS.

    CloudWatch bills by volume, RDS bills by volume and uptime (and the RDS bill is much higher for the same volume of data).

    Granted, it’s not "analytic usage" you were asking about, but people mostly use CloudWatch logs for troubleshooting, not data analytics.

    Convenience

    If you run something in AWS which does not integrate natively with CloudWatch, it’s much more easier to set up CloudWatch integration than other solutions. Even RDS has non-zero level of setup and maintenance. CloudWatch has neither, if you’re already in AWS.

    Built-in analytics

    CloudWatch has some rudimentary analytic capabilities through LogInsights. For many people, they are more than enough.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search