I want to run my Nextflow pipeline using an AWS Batch executor. However, I pushed a private (not public) image to Dockerhub. When executing the Nextflow pipeline locally, I can access the local image.
However, when executing my Nextflow pipeline using the AWS Batch executor, the Docker image can’t be accessed because it’s private. Is there a way to provide a Dockerhub username and access token to my Nextflow pipeline so it can pull the private Docker image from Dockerhub?
2
Answers
You can use the Wave service to authenticate the access to the private container repositories. See the nextflow docs for more details
You might also be able to use AWS Secrets Manager to store the Dockerhub username and access token:
https://aws.amazon.com/blogs/compute/how-to-authenticate-private-container-registries-using-aws-batch/