Debian Question posted in
The oficial documentation can be found here.

Rebuilding containers when needed – Debian

An application container has generally three parts:

  • The base image that, in Linux case, contains the core POSIX tools and system libraries (e.g. FROM debian:stable)
  • The runtime libraries or helper tools usually installed from some kind of package (e.g. RUN apt-get update && apt-get python3-numpy && …—for base python or java etc. there is usually a ready base container to use, but the application may still have additional dependencies).
    • The same issue applies to the dependencies linked into the application (e.g. external dependencies pulled in by maven or nuget etc.). It would be nice if there was a general solution that handled this too, but otherwise another tool-specific solution will be sought.
  • The application itself (e.g. COPY …/application/ /usr/local/python3.8/dist-packages/application/ and set entrypoint).

The container that will go to the production environment is normally built on a build server. Now rebuilding it when changes are made to the application is obvious, and what CI servers are designed to do.

But the first two components may, and regularly do, have security advisories files against them, and when they do, the images should also be rebuilt and pushed to integration tests and then either the production environment should be updated too or the administrator notified to trigger it as soon as practical.

So is there some way or tool to help us set up suitable trigger to rebuild the container when there are updates to the base container or other dependencies? Docker default is Debian base images, but we would use another distribution if it had better support for this.

Note that just running docker build --pull --no-cache nightly is not a solution, because that will produce a new container even if the dependencies did not, in fact, change due to different timestamps of the installed files. And running just docker build --pull does not work either because either the cache is there, but then it only checks whether the Dockerfile has changed and does not actually check the package list, or the cache isn’t there, because it is running on different build agent or the build agent was cleaned up as it generally should to get reliable builds.

It can use any container builder, not necessarily docker.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    The answer is, as for many other docker-related issues, to ditch docker and switch to buildah. Both the buildah bud, which is drop-in replacement for docker build, and the buildah commit, which is for scripting the build by other means, have a --timestamp option that forces both the timestamp written to the manifest and the timestamps of files in the new layers to specified value. That seems to be the only nondeterminism from the tool itself; standard deterministic build techniques still need to be applied to the build of the application itself, but that's obviously out of buildah scope.


  2. 0

    Running docker build --pull --no-cache once a week or so is a reasonable compromise. It’s highly likely there will be some fix in the OS-level packages in that time frame, so you’re going to be restarting the container with a new image to get security updates, which is reasonable. Depending on how often you deploy to production, “on every production deploy” may or may not be a good time to do this as well.

    If consistency across environments is important to you, consider using a date-stamped version of the debian image (FROM debian:stable-20200422), or building your own base image that you can store in a registry. You can then use a Dockerfile ARG to specify the date stamp, and if you do that, you never need --no-cache. (But, you will have to manually discover the current version.)

    # Build with
#   docker build --build-arg DATE_STAMP=-20200422
# This must have a leading hyphen
ARG DATE_STAMP

FROM debian:stable${DATE_STAMP:-}

    For language packages, also consider that most package managers have a lock file that specifies an exact version of packages to use (NPM package-lock.json, yarn.lock, Ruby Bundler Gemfile.lock, Python requirements.txt or Pipfile.lock). In these cases you have to run some sort of “update” operation to update the lock file; doing that generates a commit, which triggers the CI system, and a file change, which will invalidate the Docker build cache.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search