I am working with the following Dockerfile
.
FROM ubuntu
RUN apt-get update
RUN apt-get install -y
python3.9
python3-pip
COPY . /app
WORKDIR /app
RUN pip install -r requirements.txt
EXPOSE 8501
ENTRYPOINT [ "streamlit", "run" ]
CMD ["app.py"]
Whenever, I rebuild the image, the docker uses the previously cached version of image and builds it quickly, except when it reaches the RUN pip install -r requirements.txt
part, which it runs every time I rebuild the image (does not cache). The problem is that one of the requirement in the requirements.txt
is streamlit==1.9.0
, which has a large number of dependencies, so it slows the down the rebuild process. Long story short, the docker is cacheing RUN apt-get install foo
but not the RUN pip install bar
, which, I guess, is expected. How would you find a work-around it, to speed-up the image rebuild process when you have a long list in requirements.txt
file?
2
Answers
It can’t cache it because your files in app are changing constantly (I guess). The way its usually done is to copy requirements.txt seperately first, then install, then copy everything else. This way, docker can cache the install as requirements.txt didn’t change.
Docker BuildKit has recently introduced mounting during build time.
See https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/syntax.md
For particular scenario for PIP, check out https://dev.doroshev.com/docker-mount-type-cache/