Our service has a memory leak issue, which obviously should ideally be corrected, but as a temporary solution we could reboot our Fargate tasks once a day, preferably at night.
I’m aware of ScheduledFargateTask, which allows triggering Fargate tasks at certain moments, but that’s not quite what is needed. Rather, we need a continuous task, which gets stopped and therefore Fargate launches a new one in its place.
Is there a straightforward way to tell Fargate to restart tasks?
2
Answers
It sounds like you have an ECS Service, deployed to Fargate, and you need to trigger an ECS task replacement in that Service once a day.
The easiest way to do this, would be to create a scheduled Lambda function, that uses the AWS SDK to trigger ECS to replace the running task. The actual API call you need to make for this would either be ECS Update Service with
forceNewDeploymentenabled which will cause the ECS service to do a rolling redployment of the ECS Task. Or, if a few minutes of down-time is acceptable, you could simply call Stop Task on the currently running task, and the ECS service will replace it with a new task after a minute or two.If I get you right, I’d create an EventBridge Scheduler to trigger Lambda function at night. The Lambda will send a termination command (SIGKILL) to the Task and ECS will restart the task automatically according to desired number of tasks in the service. In this case, you’ll be able to gracefully terminate the task without any side effects by implementing some wrapper to handle such cases.
This is a rough suggestion. It might be adjusted slightly in your case.
More here: https://aws.amazon.com/blogs/containers/graceful-shutdowns-with-ecs/