skip to Main Content

I am working with the following Dockerfile.

FROM ubuntu

RUN apt-get update
RUN apt-get install -y 
    python3.9 
    python3-pip

COPY . /app
WORKDIR /app

RUN pip install -r requirements.txt

EXPOSE 8501

ENTRYPOINT [ "streamlit", "run" ]
CMD ["app.py"]

Whenever, I rebuild the image, the docker uses the previously cached version of image and builds it quickly, except when it reaches the RUN pip install -r requirements.txt part, which it runs every time I rebuild the image (does not cache). The problem is that one of the requirement in the requirements.txt is streamlit==1.9.0, which has a large number of dependencies, so it slows the down the rebuild process. Long story short, the docker is cacheing RUN apt-get install foo but not the RUN pip install bar, which, I guess, is expected. How would you find a work-around it, to speed-up the image rebuild process when you have a long list in requirements.txt file?

2

Answers


  1. It can’t cache it because your files in app are changing constantly (I guess). The way its usually done is to copy requirements.txt seperately first, then install, then copy everything else. This way, docker can cache the install as requirements.txt didn’t change.

    FROM ubuntu
    
    RUN apt-get update
    RUN apt-get install -y 
        python3.9 
        python3-pip
    
    WORKDIR /app
    COPY requirements.txt . # Copy ONLY requirements. This layer is cached if this file doesn't change
    
    RUN pip install -r requirements.txt # This uses the cached layer if the previous layer was cached aswell
    
    COPY . .
    
    EXPOSE 8501
    
    ENTRYPOINT [ "streamlit", "run" ]
    CMD ["app.py"]
    
    Login or Signup to reply.
  2. Docker BuildKit has recently introduced mounting during build time.

    See https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/syntax.md

    For particular scenario for PIP, check out https://dev.doroshev.com/docker-mount-type-cache/

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search