skip to Main Content

I’m running a mongoDB (5.0.12) instance as a kubernetes pod. Suddenly the pod is failing and I need some help to understand the logs:

{"t":{"$date":"2022-09-13T18:39:51.104+00:00"},"s":"E",  "c":"STORAGE",  "id":22435,   "ctx":"AuthorizationManager-1","msg":"WiredTiger error","attr":{"error":1,"message":"[1663094391:104664][1:0x7fc5224cc700], file:index-9--3195476868760592993.wt, WT_SESSION.open_cursor: __posix_open_file, 808: /data/db/index-9--3195476868760592993.wt: handle-open: open: Operation not permitted"}}
{"t":{"$date":"2022-09-13T18:39:51.104+00:00"},"s":"F",  "c":"STORAGE",  "id":50882,   "ctx":"AuthorizationManager-1","msg":"Failed to open WiredTiger cursor. This may be due to data corruption","attr":{"uri":"table:index-9--3195476868760592993","config":"overwrite=false","error":{"code":8,"codeName":"UnknownError","errmsg":"1: Operation not permitted"},"message":"Please read the documentation for starting MongoDB with --repair here: http://dochub.mongodb.org/core/repair"}}
{"t":{"$date":"2022-09-13T18:39:51.104+00:00"},"s":"F",  "c":"-",        "id":23091,   "ctx":"AuthorizationManager-1","msg":"Fatal assertion","attr":{"msgid":50882,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_session_cache.cpp","line":109}}
{"t":{"$date":"2022-09-13T18:39:51.104+00:00"},"s":"F",  "c":"-",        "id":23092,   "ctx":"AuthorizationManager-1","msg":"nn***aborting after fassert() failurenn"}

So why is there operation is not permitted? I already run mongod --repair, but the error still occurs.

This is how the pod is deployed:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongodb
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongodb
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: mongodb
    spec:
      hostname: mongodb
      # securityContext:
      #   runAsUser: 999
      #   runAsGroup: 3000
      #   fsGroup: 2000
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: data
      containers:
        - name: mongodb
          image: mongo:5.0.12
          args: ["--auth", "--dbpath", "/data/db"]
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 27017
          volumeMounts:
            - mountPath: /data/db
              name: data
          # securityContext:
          #   allowPrivilegeEscalation: false

Update

The PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi

2

Answers


  1. You should have a look at setting the umask on the directory:

    http://www.cyberciti.biz/tips/understanding-linux-unix-umask-value-usage.html

    That will ensure new files in the directory are created with the specified permissions/ownerships.

    Login or Signup to reply.
  2. You can try checking the permissions for that file before execution:

    ls -l
    

    then using chmod you can try changing the permission and then try executing it.

    OR

    You can refer here, this might help you:

    https://kubernetes.io/docs/tasks/configure-pod-container/security-context/

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search