Question 84371 in Docker Question posted in
The official documentation can be found here.

How to stop Docker from clearing logs for dead containers?

I use Dokku to run my app, and for some reason, the container is dying every few hours and recreates itself.

In order to investigate the issue, I am willing to read the error logs to this container and understand why it’s crashing. Since Docker clears logs of dead containers, this is impossible.

I turned on docker events and it shows many events (like container update, container kill, container die, etc.) But no sign of what triggered this kill.

How can I investigate the issue?

Versions:

  • Docker version 19.03.13, build 4484c46d9d
  • dokku version 0.25.1
Tags:

2

Answers


  1. 0

    Logs are deleted when the container is deleted. If you want the logs to persist, then you need to avoid deleting the container. Make sure you aren’t running the container with an option like --rm that automatically deletes it on exit. And check for the obvious issues like running out of disk space.

    Login or Signup to reply.
  2. 0

    There are several things you can do to investigate the issue:

    1. You can run the container in the foreground and allow it to log to your console.

      If you were previously starting the container in the background with docker run -d (or docker-compose up -d), just remove the -d from the command line and allow the container to log to your terminal. When it crashes, you’ll be able to see the most recent logs and scroll back to the limits of your terminal’s history buffer.

      You can even capture this output to a file using e.g. the script tool:

      script -c 'docker run ...`

      This will dump all the output to a file named typescript, although you can of course provide a different output name on the command line.

    2. You can change the log driver.

      You can configure your container to use a different logging driver. If you select something like syslog or journald, your container logs will be sent to the corrresponding service, and will continue to be available even after the container has been deleted.

      I like use the journald logging driver because this allows searching for output by container id. For example, if I start a container like this:

      docker run --log-driver journald --name web -p 8080:8080 -d docker.io/alpinelinux/darkhttpd

      I can see logs from that container by running:

      $ journalctl CONTAINER_NAME=web
Feb 25 20:50:04 docker 0bff1aec9b65[660]: darkhttpd/1.13, copyright (c) 2003-2021 Emil Mikulic.

    These logs will persist even after the container exits.

    (You can also search by container id instead of name by using CONTAINER_ID_FULL (the full id) or CONTAINER_ID (the short id), or even by image name with IMAGE_NAME.)

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search