I’m trying to use AWS ECS on EC2, but my deployments are always stuck "pending" and when I ssh into the machines, docker images
shows it didn’t download the image (nginx:latest
), and as far as I can tell from logs it didn’t even try. I can manually docker pull
and docker run
after which it runs as expected. I’ve also tried changing the network mode between awsvpc, bridge, and host with no difference. Security group allows all traffic in/out, and I’m using the default public VPC. In case it matters I’m using t2.micro instances on Amazon Linux 2023, and the tasks and container are given 1vCPU and 1GB of mem.
For testing I’ve given the ecsInstanceRole and ecsTaskExecution role both AmazonECS_FullAccess and AmazonSSMFullAccess policies and there is still no trace that the instance even attempts to download the container image. Grepping /var/log/
recursively for nginx or my account id produces nothing.
2
Answers
This turned out to be two separate issues. Firstly the task was given too much memory, and second the "pending" status only goes away if you define a health check.
The main issue turned out to be the "Task memory" parameter. I was testing with a t3.micro instance, which has 1 GB of Memory, and I had set the task memory to 1 GB. Since the OS and docker take up some memory the full 1 GB is not available, and trying to allocate 1 GB just for the task was presumably failing.
Lowering the task memory to .5 GB fixed the issue and allowed the task to run successfully. I haven't tested how high you could go before the problem arises, perhaps .95 GB would be OK, I don't know.
Even after this though, with the Apache server up and serving traffic just fine, it still showed "pending". This was fixed by defining a health check (specifically
CMD-SHELL,curl -sf http://localhost/ || exit 1
).I think you don’t have Docker Hub credentials in EC2 instance, so you can run command to login the AWS Private Registry with your AWS credentials:
If you are using third party Docker Hub or GitHub registry like ghcr.io, the same goes for that case.