skip to Main Content

I have a deployment with multiple replica pods behind a service. Sometimes, I want to know which of these pods is serving a request, i.e. to view the logs when there is some strange behaviour. Is there a way to add an HTTP header like X-Pod: my-app-5954c566c7-48s97 to each request served by an ingress using the NGINX Ingress Controller, for example through annotations?

3

Answers


  1. If you deploy a ingress-nginx controller, it will create a default ConfigMap with the name of your ingress controller. This is the one you want to modify.

    There would be something like this:

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: {name-of-the-helm-chart}-nginx-ingress-controller
      namespace: {namespace-where-the-nginx-ingress-is-deployed}
    date:
       ...
    

    The config of log-format-upstream in the data should be added:

    data:
      allow-snippet-annotations: "true"
      enable-real-ip: "true"
      log-format-upstream: "[$time_local] $remote_addr - $remote_user - $server_name $host to: $upstream_addr: $request $status upstream_response_time $upstream_response_time msec $msec request_time $request_time"
    

    Note using $upstream_addr that would be the IP address of pod.
    Another example would be:

    log-format-upstream: '$remote_addr $host $remote_user [$time_local] $request $status $body_bytes_sent $http_referer $http_user_agent $request_length $request_time [$proxy_upstream_name] [$proxy_alternative_upstream_name] $upstream_addr $upstream_response_length $upstream_response_time $upstream_status $req_id
    

    But if you want your ingress controller to log the pod name by X-My-Pod-Name value, you can set this HEADER in your service. Like for example having the manifest of service updated to have env var:

    env:
    - name: POD_NAME
      valueFrom:
         fieldRef:
           fieldPath: metadata.name
    

    then for example in python, something like:

    resp.headers['X-My-Pod-Name'] = os.environ.get('POD_NAME')
    

    and finally you can show it in logs by $http_x_my_pod_name:

      log-format-upstream: $remote_addr - $remote_user [$time_local] "$request" $status
        $body_bytes_sent "$http_referer" "$http_user_agent" "$http_x_my_pod_name"
    
    Login or Signup to reply.
  2. Yes, you can achieve this by using annotation and snippet. $HOSTNAME will contain the pod name.

    kind: Ingress
    metadata:
      name: some-ingress
      annotations:
        nginx.ingress.kubernetes.io/server-snippet: |
          proxy_set_header X-Pod $HOSTNAME;
    
    Login or Signup to reply.
  3. Yes, custom HTTP headers can be added to requests handled by the NGINX Ingress Controller via annotations. To include a custom header, add the nginx.ingress.kubernetes.io/configuration-snippet annotation.

    Here’s how you can accomplish it:

    • Add the annotation to the Ingress resource. Here’s an example of how your Ingress definition could look:

    apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: my-ingress annotations: nginx.ingress.kubernetes.io/configuration-snippet: | more_set_headers "X-Pod: $pod_name"; spec: rules: – host: myapp.example.com http: paths: – path: / pathType: Prefix backend: service: name: my-service port: number: 80

    • Ensure the pod’s name is easily accessible. You can utilize the NGINX variable $pod_name, which represents the name of the pod that is handling the request. However, in order to use this variable, make sure that your NGINX Ingress Controller is set to provide pod information.

    • Deploy the Ingress. Apply the YAML configuration to your cluster: bash

      kubectl apply -f your-ingress-file.yaml

    • Check the requests. After implementing the setup, when requests pass through the Ingress, the X-Pod header is included in the requests forwarded to your service. You can review your application’s logs to see which pod is handling the requests.

    This allows you to easily identify which pod is delivering a specific request based on the X-Pod header in the logs.

    Refer to the Kubernetes Community Forums for more information.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search