I have set up an AWS CodePipeline stack using the CDK Pipelines module. When the code is pushed to the backing CodeCommit repo, the pipeline starts executing and does the following:
- Pulls the latest source
- Synthesizes the stack
- Updates the pipeline (self mutating)
- Builds docker image
- Pushes docker image to ECR repo (with ‘latest’ tag)
- Creates Fargate Task Definition and Container (using ECR image with ‘latest’ tag)
- Creates Fargate Service
- Adds Fargate Service to a target group of an ALB
All of this is working fine.
But every time I push to the repo, I expect the Fargate Service to be updated with the latest docker image. This is not happening. The Fargate Service is not updated and the tasks within the service continue to run using the docker image that was latest at the time of creating the Fargate Service.
How do I make the Fargate Service update i.e. start new tasks with the latest image and shut down existing tasks with the old image?
If I run the following command manually from the command line then the service updates i.e. start new tasks with the latest image and shut down existing tasks with the old image.
aws ecs update-service --cluster CLUSTER_NAME --service SERVICE_NAME --force-new-deployment
2
Answers
You would do the same thing you do manually, but in the pipeline. Add a CodeBuildStep that executes the command after the deployment.
Alternatively, you could use assets and let the pipeline handle the building and ECR uploading.
For ECS, you’d use
ContainerImage.fromDockerImageAsset().This will update the Task definition with the new tag, which will force redeployment.
To understand what is going on you have to understand how those Tasks are working. Basically, when you register a task it will be created in S3. So when you start this task in ECS it’s asking S3 for task details.
If you didn’t release a new version of the task it will try to pull your image NOT by image tag, but image ID (which is different for every image, even if tag is the same).
So even if you pushed a new image it’s still pulling an old one.
To fix this you just need to release a new version of the task after pushing the image.
I don’t know how your CI/CD works, but you can do it using API:
https://docs.aws.amazon.com/cli/latest/reference/ecs/update-task-set.html
I had a similar issue and I resolved it by fetching the task definition, removing the version, and then pushing it using the update task set. It was releasing a new version of the Task definition in which was the ID of a new image.