Is there any service offering from AWS, that does real time replication of data from Oracle which is on premises to Oracle instance running on EC2 in AWS ? But the catch is, the table structures are not same between the environments.
Looking at AWS documentation, I did not find any service that meets this requirement. But wanted to check if there is any service offering from AWS, which does real time replication, with transformation.
Alternatively, straight replication can be done either using AWS services or Golden gate replication and but may add event notification, and using lambdas to do some transformation and add to new table. But not sure, how real time this replication can be ?
Are there any other approaches that can offer real time/ near real time replication ?
Thanks
2
Answers
The most obvious solution which is open source would be starting with Debezium. This is an open source replication suite which allows you near real-time replication between various data sources. It also allows transformations.
If your source is on premise Oracle database you have many options for the source connector native LogMiner, XStream API or using OpenLogReplicator. More details in Debezium documentation.
If you want to pay you can go with Oracle GoldenGate or third party payed solutions like Qlik, Fivetran, Informatica, etc.
AWS Kinesis it one of the real time streaming functionality in AWS. You can use the service and migrate the data on real time. Since your source is structure is not always same, you have to design a customized code in AWS Lambda/ AWS Glue and call AWS Kinesis resource to perform real time straming.
This course helps you to understand more on Kinesis,
https://www.udemy.com/course/aws-kinesis/?referralCode=A5C18EAECFE9723B5A4B