Just wondering if there are any native AWS options to Orchestrate database scripts which are stored in AWS aurora serverless(postgresql).
ex: lets say if we have below three database scripts(functions) that we want to execute in sequence
-
select get_data();
-
select transform_data();
-
select load_data();
Is there any AWS native services that we can use to orchestrate above scripts..
2
Answers
Probably the easiest way would be using a lambda function to execute those three statements one after the other. If for some reason you want to have them executed on different lambda functions, you could always use step functions to orchestrate the next lambda to start only after successfully executing the previous one.
If you need to schedule automatic execution of your lambda function, you can achieve cron-like functionality using cloudwatch events with lambda
You could use AWS Batch to run jobs efficiently. AWS Batch is a managed service that abstracts the complexities of provisioning, managing, monitoring, and scaling your computing jobs, and enables you to easily and efficiently run jobs on AWS. Additionally, AWS Batch enables you to build jobs using the language of your choice and deploy it as a Docker container.
Now, for orchestrate your AWS Batch job you can use AWS Step Functions. Step Functions is a serverless function orchestrator that makes it easy to sequence calls to AWS services into business-critical applications.
You take take as reference following article how use AWS Batch and AWS Step Functions
Replace self-managed database scheduler batch jobs using AWS native solutions
Another simple solution is use AWS Batch and AWS EventBridge Scheduler
How to schedule and execute Amazon RDS jobs using AWS Batch and CloudWatch rules