Here’s the scenario:
On a Debian GNU/Linux 9 (stretch) VM I have two containers running. The day before yesterday I got a warning from the monitoring that the memory usage is relatively high. After looking at the VM it could be determined that not the containers but Docker daemon needs them. htop
After a restart of the service I noticed a new increase of memory demand after two days. See graphic.
Is there a known memory leak for this version?
Memory development (container) after 2 days:
Container 1 is unchanged
Container 2 increased from 21.02MiB to 55MiB
Memory development (VM) after 2 days:
The MEM increased on the machine from 273M (after reboot) to 501M
dockerd
– after restart 1.3% MEM%
– 2 days later 6.0% of MEM%
2
Answers
Monitor your containers to see if their memory usage changes over time:
> docker stats
We saw a similar issue and it seems to have been related to the
gcplogs
logging driver. We saw the problem on docker 19.03.6 and 19.03.9 (the most up-to-date that we can easily use).Switching back to using a log forwarding container (e.g. logspout) resolved the issue for us.