skip to Main Content

Currently we have multiple docker containers(host A).
We send the logs from each docker container to logger(which is runs on docker container on another server).
Here is my daemon.json:

{                                                                                                                                                                                   
   "log-driver":"gelf",                                                                                                                                                             
   "log-opts":{                                                                                                                                                                     
      "gelf-address":"tcp://10.*.*.*:12201"                                                                                                                                    
   },                                                                                                                                                                               
   "dns":[                                                                                                                                                                          
      "10.*.*.*"                                                                                                                                                                                                                                                                                                                             
   ],                                                                                                                                                                               
   "icc":false                                                                                                                                                                      
}

The problem is that if logger docker is not running and i restarting host A, they not starting because cannot connect to logger.
Is there any way to configure docker containers to start even if they cannot connect to logger configured in daemon.json?
Thank you.

2

Answers


  1. With this you are not configuring docker containers, but the daemon itself. If you restart you host, you restart the daemon and on startup it reads the config. If the config is invalid or parts of it are not working, it doesn’t start up. You can manually start up the docker daemon with a manual configuration like

     dockerd --debug 
      --tls=true 
      --tlscert=/var/docker/server.pem 
      --tlskey=/var/docker/serverkey.pem 
      --host tcp://192.168.59.3:2376
    

    see: Docker daemon documentation

    Keep in mind, that it will keep running with those options, until it’s restarted.

    Login or Signup to reply.
  2. The logging settings in daemon.json are defaults for newly created containers. Changing this file will not change existing containers being restarted.

    You may want to reconsider your logging design. One option is to swap out the logging driver for a logging forwarder, leaving the logs in the default json driver, and having another process monitor those and forward the logs to the remote server. This avoids blocking at the cost of missing some logs written just as the container is deleted (or very short lived containers). The other option is to improve the redundancy of your logging system since it is a single point of failure that blocks your workloads from running.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search