AWS Batch documentation says it is based on ECS.
So why shouldn’t we just use ECS? What pressing benefits does AWS Batch offer that are not available in ECS?
Both offer autoscaling as well. Is queuing and priority of jbos the only advantages of AWS Batch?
3
Answers
AWS Batch is a management/abstraction layer on top of ECS and EC2 (and some other AWS resources). It does some things for you, like cost optimization, that can be difficult to do yourself. Think of it like Elastic Beanstalk for batch operations. It provides a management layer on top of lower-level AWS resources, but if you are comfortable managing those lower level resources yourself and want more control over them it is certainly an option to use those lower-level resources directly.
The major feature benefit AWS Batch would provide you is dependent job scheduling. For example, lets say you have 3 jobs: JobA, JobB and JobC. You want JobC to run only after JobA and JobB have finished. It is very trivial to achieve this with Batch, but on plain ECS you have to build this logic yourself. AWS Batch also provides automatic retries and some other goodies that are valuable in batch processing (including the ones you mentioned like queuing and scheduling policies. Also if you use the AWS Batch managed compute environments, then the resource scaling up/down is handled automatically by AWS. It creates the required autoscaling groups and scaling policies behind the scenes and manages it completely.
As already said, AWS Batch let’s you focus on developing the actual business logic without the need to wire all the job management system functions yourself.
The only thing I miss at the moment is the support for ARM64 architecture for jobs running on AWS Fargate (see here).