skip to Main Content

I am trying to setup a Local Persistent volume using local storage using WSL. But the pod STATUS stops at Pending.

The kubectl describe pod <pod-name> gives below error.

Warning FailedMount 21s (x7 over 53s) kubelet MountVolume.NewMounter initialization failed for volume "pv1" : path "/mnt/data" does not exist

The path /mnt/data has been created and exists on the local machine but cannot be accessed by the container.

and the pod and Persistent volume configuration as below.

apiVersion : v1
kind : PersistentVolume
metadata :
   name : pv1
spec :
  capacity :
    storage : 2Gi
  accessModes :
    - ReadWriteOnce
  persistentVolumeReclaimPolicy : Retain
  storageClassName : local-storage
  local : 
    fsType : ext4
    path : /mnt/data
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - node1

---

apiVersion : v1
kind : PersistentVolumeClaim
metadata : 
  name : pvc1
spec :
  resources :
    requests :
      storage : 1Gi
  accessModes :
    - ReadWriteOnce
  storageClassName : local-storage

---

apiVersion : v1
kind : Pod
metadata :
  name : pod1
spec :
  containers:
  - name: www
    image: nginx:alpine
    ports:
      - containerPort: 80
        name: www
    volumeMounts:
      - name: www-store
        mountPath: /usr/share/nginx/html
  volumes :
    - name : www-store
      persistentVolumeClaim :
        claimName : pvc1

Any help would be appreciated.

4

Answers


  1. You are using nodeSelector for the pv, telling it to use node1 for the volume , chances are 1. node1 does not have /mnt/data directory present, which is hostPath for the volume.
    OR 2. node1 may be having /mnt/data, but the pod is getting scheduled on some other node which does not have /mnt/data directory:

    apiVersion : v1
    kind : PersistentVolume
    metadata :
       name : pv1
    spec :
      capacity :
        storage : 2Gi
      accessModes :
        - ReadWriteOnce
      persistentVolumeReclaimPolicy : Retain
      storageClassName : local-storage
      local : 
        fsType : ext4
        path : /mnt/data
      nodeAffinity:
        required:
          nodeSelectorTerms:
          - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
              - node1
    

    Solution:
    Make sure /mnt/data is present on all schedulable nodes

    OR

    Try modifying your file to add, nodeName or nodeSelector or nodeAffinity to force it to get scheduled on the same node which have proper hostPath. In below example, it is assumed that node1 have /mnt/data directory present.

    apiVersion : v1
    kind : Pod
    metadata :
      name : pod1
    spec :
      nodeName: node1 #<------------this
      containers:
      - name: www
        image: nginx:alpine
        ports:
          - containerPort: 80
            name: www
        volumeMounts:
          - name: www-store
            mountPath: /usr/share/nginx/html
      volumes :
        - name : www-store
          persistentVolumeClaim :
            claimName : pvc1
    
    Login or Signup to reply.
  2. If running on a Rancher Kubernetes Engine (RKE) cluster, this problem can arise from the fact that each kubelet also runs as a container. Thus, it does not see the filesystem of the node it runs on.

    The solution is to add extra bind mounts (for kubelet service) when configuring the cluster in cluster.yml. For example, to have /data-1 on the node to be mounted as /data-1 on the kubelet:

    services:
      ...
      kubelet: 
        extra_binds:
        - "/data-1:/data-1"
    
    Login or Signup to reply.
  3. On Talos which similarly to the Rancher Kubernetes Engine (RKE) from the other answer also runs the kubelet containerized, one must configure
    extraMounts in the MachineConfig.kubelet, e.g. like this:

    extraMounts:
      - destination: /var/mnt/nvme1
        type: bind
        source: /var/mnt/nvme1
        options:
          - bind
          - rshared
          - rw
    
    Login or Signup to reply.
  4. If you are using Rancher-Desktop(Windows) and facing the same issue
    i.e. Warning FailedMount 21s (x7 over 53s) kubelet MountVolume.NewMounter initialization failed for volume "pv1": path "/mnt/data" does not exist

    As per the below document, ‘For Windows, all files are automatically shared via WSL2.’
    https://docs.rancherdesktop.io/faq/#q-does-file-sharing-work-similarly-to-docker-desktop-do-i-have-to-do-any-additional-configuration-to-mount-volumes-to-vms

    So you can create the directory inside any of the drives eg. C: or D:
    and during specifying the path in YAML you should prefix with /mnt

    for example: suppose you created a directory inside D: drive let’s say, D:data

    Now path will be like this – /mnt/d/data

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search