Memcached Question posted in
A decent usage manual can be found here.

Memcached – What's the best way to cache downloads done by docker while building containers?

While testing new Docker builds (modifying Dockerfile) it can take quite some time for the image to rebuild due to huge download size (either direct by wget, or indirect using apt, pip, etc)

One way around this that I personally use often is to just split commands I plan to modify into their own RUN variable. This avoids re-downloading some parts because previous layers are cached. This, however, doesn’t cut it if the command that requires “tuning” is early on in the Dockerfile.

Another solution is to use an image that already contains most of the required packages so that it would just be pulled once and cached, but this can come with unnecessary “baggage”.

So is there a straight forward way to cache all downloads done by Docker while building/running? I’m thinking of having Memcached on the host machine but it seems kind of an overkill. Any suggestions?

I’m also aware that I can test in an interactive shell first but sometimes you need to test the Dockerfile and make sure it’s production-ready (including arguments and defaults) especially if the only way you will ever see what’s going on after that point is ELK or cluster crash logs

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    This here: https://superuser.com/questions/303621/cache-for-apt-packages-in-local-network

    Is the same question but regarding a local network instead of the same machine. However, the answer can be used in this scenario, it's actually a simpler scenario than a network with multiple machines.

    If you install Squid locally you can use it to cache all your downloads including your host-side downloads.

    But more specifically, there's also a Squid Docker image!

    Headsup: If you use a squid service in a docker-compose file, don't forget to use the squid service name instead of docker's subnet gateway 172.17.0.1:3128 becomes squid:3128


  2. 0

    the way i did this was

    • used the new --mount=type=cache,target=/home_folder/.cache/curl
    • wrote a script which looks into the cache before calling curl (wrapper over curl with cache)
    • called the script in the Dockerfile during build
    • it is more a hack, works
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search