Question 295179 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – Moving data from Dynamo to Redshift in real-time

I want to move data from our dynamo DB to redshift, but I want to do this is real time as we are currently under a migration and still using the dynamo DB, any suggestions would be greatly appreciated.

I did a bit of research and found the following possible solutions

  • AWS Data pipelines
  • AWS Glue – heard it is not the greatest at times
  • Amazon Kinesis
  • AWS Lambda
  • Amazon DynamoDB Streams
  • AWS DMS

I don’t have to much experience in some of those services, so any help of suggestions would be appreciated greatly

Tags:

2

Answers


  1. 0

    I would look into using DynamoDB Streams. There are two options:

    1. Kinesis Data Streams (recommended by AWS, probably best for high velocity data changes)
    2. Lambda Triggers (easier, good enough for low velocity changes)

    When changes occur, the changes are added to the stream which either Kinesis or a triggered lambda function can access. In the case of lambda, the events are stored in the triggering event object. You can then use the lamdba function to move your data to RedShift.

    Login or Signup to reply.
  2. 0

    I have a tool that easily lets you integrate data from anywhere and provision usable data to any data system using any of the integration styles: ETL, ELT, API Integration, Data as a Service, or API Proxy.

    Lmk if you want more info!

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search