skip to Main Content

I have setup Percona MongoDB Exporter and when I attempt to scrape metrics for my MongDB instance I am getting an error :

An error has occurred while connecting to MongoDB:
cannot connect to MongoDB: server selection error: server selection timeout, current topology: { Type: Unknown, Servers: [{ Addr: mongodb-headless-service.labs.svc.cluster.local:27017, Type: Unknown, Last error: connection() error occured during connection handshake: dial tcp: lookup mongodb-headless-service.svc.cluster.local on 159.XX.XX.XX:XX: no such host }, ] }

Both MongoDB and Percona-MongoDB Exporter are deployed in the same namespace on Kubernetes and I have created a user (at admin level ) to extract the metrics :

db.createUser({ user: "promt", pwd: "abc123", roles: [ { role: "clusterMonitor", db: "admin" },{ role: "read", db: "local" } ], mechanisms:["SCRAM-SHA-1"]})

To verify if the exporter is able to scrape any metrics from the MongoDB instance I am using:

kubectl -n labs port-forward service/mongodb-exporter 9216

and I can access the Exporter at http://localhost:9216

In the Percona MongoDB-Exporter deployment I have specified Mongo-URI as below :

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongodb-exporter
  namespace: labs
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongodb-exporter
  template:
    metadata:
      labels:
        app: mongodb-exporter
    spec:
      containers:
      - name: mongodb-exporter
        image: percona/mongodb_exporter:0.30
        imagePullPolicy: "IfNotPresent"
        args:
        - "--mongodb.direct-connect=false"
        - "--mongodb.uri=mongodb://promt:[email protected]/admin"
        ports:
        - name: metrics
          containerPort: 9216
        resources:
          requests:
            memory: 128Mi
            cpu: 250m

I have tried a lot of options but they all result in the same error:

  1. Adding k8s namespace to the mongo-uri

    –mongodb.uri=mongodb://promt:abc123@mongodb-headless-service.labs.svc.cluster.local/admin

  2. Adding option for SSL
    –mongodb.uri=mongodb://promt:[email protected]/admin?ssl=false

  3. Switching from using a k8s headless service to a ClusterIP

–mongodb.uri=mongodb://promt:[email protected]/admin

  1. Enabling direct-connect option
  • "–mongodb.direct-connect=true"
    – "–mongodb.uri=mongodb://promt:[email protected]/admin"

For all these options I also tried variations of adding the ssl option and namespace but that is failing with same error in all cases.

My MongoDB instance is a standalone deployment with the following definition :

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongodb-deployment
  namespace: labs
  labels:
    app: mongodb
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongodb
  template:
    metadata:
      labels:
        app: mongodb
    spec:
      containers:
      - name: mongodb
        image: docker.io/bitnami/mongodb:4.4.6-debian-10-r0
        imagePullPolicy: "IfNotPresent"
        securityContext:
            runAsNonRoot: true
            runAsUser: 1001
        ports:
          - containerPort: 27017
        env:
            - name: MONGO_INITDB_ROOT_USERNAME
              valueFrom:
                secretKeyRef:
                  name: mongodb-secret-amended
                  key: mongo-root-username
            - name: MONGO_INITDB_ROOT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mongodb-secret-amended
                  key: mongo-root-password
        volumeMounts: 
            - mountPath: /data/db
              name: mongodb-vol
      volumes:
      - name: mongodb-vol
        persistentVolumeClaim:
          claimName: mongodb-claim
---
apiVersion: v1
kind: Service
metadata:
  name: mongodb-service
  namespace: labs
spec:
  type: NodePort
  selector:
    app: mongodb
  ports:
    - protocol: TCP
      port: 27017
      targetPort: 27017 
---
apiVersion: v1
kind: Service
metadata:
  name: mongodb-headless-service
  namespace: labs
spec:
  selector:
    app: mongodb
  ports:
    - protocol: TCP
      port: 27017
      targetPort: 27017 
---
apiVersion: v1
kind: Service
metadata:
  name: mongo-single-clusterip
  namespace: labs
spec:
  type: ClusterIP
  selector:
    app: mongodb
  ports:
    - protocol: TCP
      port: 27017
      targetPort: 27017

For context I am following the example given here and here.

What am I missing?

2

Answers


  1. The reason is stated in the error message you got back:

    error occured during connection handshake: dial tcp: lookup mongodb-headless-service.svc.cluster.local on 159.XX.XX.XX:XX: no such host

    DNS lookup for this FQDN: mongodb-headless-service.svc.cluster.local fails.

    You can go through the steps here to confirm your DNS is set up correctly: https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/

    Login or Signup to reply.
  2. You need to change your mongoDB URI to mongodb-headless-service.labs.svc.cluster.local in the configuration & it should start working well.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search