Question 301045 in Docker Question posted in
The official documentation can be found here.

Docker volume to keep Loki config and data files

I’m beginner in docker, so I probably don’t understand how it should work.
I have loki container – here is part of docker-compose.yml:

loki:
  image: grafana/loki:latest
  ports:
    - "3100:3100"
  volumes:
    - /loki:/docker_data/loki_data
    - /etc/loki:/docker_data/loki_config
  networks:
    - monitoring
  (...)
networks:
  monitoring:

volumes:
  docker_data:
    external: true

Functionallity works well, but it doesn’t use my external volume (I created it manually):

$ docker volume inspect docker_data
    [
        {
            "CreatedAt": "2023-12-08T15:26:05Z",
            "Driver": "local",
            "Labels": null,
            "Mountpoint": "/var/lib/docker/volumes/docker_data/_data",
            "Name": "docker_data",
            "Options": null,
            "Scope": "local"
        }
    ]

I wanted to keep things from /loki and /etc/loki in my volume, but volume is still not in use, and I found that inside container have been created 2 directories -> docker_data/loki_data and docker_data/loki_config (but they are empty, nothing has been stored there).

The question is what’s wrong? How to do it in the best way, if is it possible?

The most answer using it in second direction, to get directories from outside to image, but I have error: failed parsing config: /etc/loki/local-config.yaml does not exist, set config.file for custom config path, so I understood that there should be ready configuration to start up new container with previous configuration. It will be helpful when I would want to create image based on my files, but firstly I want to get them in outside volume or my device storage, just to keep back up with all logs, when container would be removed or something like that.

Thank you in advance for any hints.

EDIT after answers:
Okay, thank you, it makes sense now (when you use word ‘mount’). I thought it’s possible in both directions. So do you know some practice to keep config files/data in my storage? Or I have to copy it manually/by script during certain periods of time? My purpose is keeping data outside docker image – to have backup of storaged data (because Loki has kept a lot of logs, which are very important).

Tags:

3

Answers


  1. 0 
      volumes:
    - /loki:/docker_data/loki_data
    - /etc/loki:/docker_data/loki_config

    It doesn’t – it uses /loki and /etc/loki (?). Mount the volume if you want to:

     volumes:
    - docker_data:/where/to/put/docker_data/volume/inside/loki/container
    #              ^^^^^^ path inside the container
    # ^^^^^^^^^^^ - volume
    #^ - no slash

    I do not think it is possible to mount docker volume subdirectory in docker, but maybe now it’s possible. Also, because docker volumes are maybe hard to manage, it’s just simpler to volumes: - ./etc_loki:/etc/loki mount relative to docker-compose working directory.

    Login or Signup to reply.
  2. 0

    The syntax is always "what is mounted to where". So in this case, which volume is mounted to which place in the container. Meaning you want to mount docker_data to some dir inside the container.

    So your yamls needs to be adjusted as follows:

    loki:
  image: grafana/loki:latest
  ports:
    - "3100:3100"
  volumes:
    - docker_data:/loki
    - docker_data:/etc/loki_config
  networks:
    - monitoring
  (...)
networks:
  monitoring:

volumes:
  docker_data:
    external: true

    Keep in mind though, that docker-compose is just a better wrapper around regular docker instructions and you have to obey those rules, so you can just mount a volume to a dir, but not a dir in a volume to a dir

    https://docs.docker.com/compose/compose-file/compose-file-v3/#volume-configuration-reference
    https://docs.docker.com/storage/volumes/

    Login or Signup to reply.
  3. 0

    Other answers have clarified the syntactic issue – the first part of volumes: entries names a host path or named volume, and the second part a container path. It seems like you have a couple of additional questions around configuration vs. data.

    Standard Docker has two types of storage. A named volume is generally inaccessible from the host, but it can be substantially faster using Docker Desktop, and it’s often a good choice for storing opaque application data. A bind mount injects part of the host system into a container, so it’s better for files that are accessed infrequently but need to be directly read or written by the user; that is, your configuration file. You have two different concerns here and I’d store the two things in two different places.

    There’s one further detail about volume mounts, of both types: whatever’s in the mounted content completely hides what was in the image originally. In your example you’re trying to mount the configuration over all of /etc/loki, and that hides the base configuration provided in the image. You can copy that out, but you can also inject only specific files using a bind mount (it gets tricky if the application tries to write the file though).

    Looking at the grafana/loki Docker Hub image page, the application data (usually a named volume) should be stored in /loki, and an additional configuration file (injected via a bind mount) can be supplied at /etc/loki/local-config.yaml. Those two paths can appear on the right-hand side of volumes: entries:

    version: '3.8'
services:
loki:
  image: grafana/loki:latest
  ports:
    - "3100:3100"
  volumes:
    - loki_data:/loki
    # ^^^^^^^^^ references named volume
    - ./loki-config.yaml:/etc/loki/local-config.yaml
    # ^^^^^^^^^^^^^^^^^^ references host path
    #                    relative to docker-compose.yml file
volumes:
  docker_data:

    If you are specifically on a native-Linux host not using Docker Desktop, there is not a performance cost to using a bind mount. You discuss backups as a use case; it is possible but indirect to back up a named volume and a host directory might be easier to manage here. However, you also need to manage the directory permissions and the container’s user so that the process in the container has permission to write to the host directory.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search