Custom response for Ingress-Nginx External Authentication

Here’s a solution that worked for me. It allows the auth service to return a custom error message for each request.

The caveat is that because nginx can’t access auth response body, the pth-auth service needs to put the data in Pth-Auth-Error header (base64-encoded).

This example handles 401, 500, and a special case when pth-auth service is unavailable.

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-service
  annotations:
    kubernetes.io/ingress.class: 'nginx'
    nginx.ingress.kubernetes.io/use-regex: 'true'
    nginx.ingress.kubernetes.io/rewrite-target: /$1
    nginx.ingress.kubernetes.io/auth-response-headers: item_id
    nginx.ingress.kubernetes.io/auth-method: POST
    nginx.ingress.kubernetes.io/auth-url: http://pth-auth.default.svc.cluster.local:8000/item/1
    # UPDATE THIS LINE ABOVE
    nginx.ingress.kubernetes.io/configuration-snippet: |
      # Redirect auth errors to custom named locations
      error_page 401 = @ingress_service_custom_error_401;
      error_page 500 = @ingress_service_custom_error_500;

      # Grab data from auth error response
      auth_request_set $pth_auth_error $upstream_http_pth_auth_error;
      auth_request_set $pth_auth_error_content_type $upstream_http_content_type;
      auth_request_set $pth_auth_status $upstream_status;

    nginx.ingress.kubernetes.io/server-snippet: |
      location @ingress_service_custom_error_401 {
        internal;

        # Decode auth response header
        set_decode_base64 $pth_auth_error_decoded $pth_auth_error;

        # Return the error from pth-auth service if any
        if ($pth_auth_error_decoded != ""){
          add_header Content-Type $pth_auth_error_content_type always;
          return 401 $pth_auth_error_decoded;
        }

        # Fall back to default nginx response
        return 401;
      }

      location @ingress_service_custom_error_500 {
        internal;

        # Decode auth response header
        set_decode_base64 $pth_auth_error_decoded $pth_auth_error;

        # Return the error from pth-auth service if any
        if ($pth_auth_error_decoded != ""){
          add_header Content-Type $pth_auth_error_content_type always;
          return 500 $pth_auth_error_decoded;
        }

        # Return a hardcoded error in case no pth-auth pods are available
        if ($pth_auth_status = 503){
          add_header Content-Type application/json always;
          return 503 "{"msg":"pth-auth service is unavailable"}";
        }

        # Fall back to default nginx response
        return 500;
      }
spec:
  rules:
    - http:
        paths:
          - path: /?(.*)
            # UPDATE THIS LINE ABOVE
            backend:
              serviceName: client-cluster-ip-service
              servicePort: 3000
          - path: /api/?(.*)
            # UPDATE THIS LINE ABOVE
            backend:
              serviceName: server-cluster-ip-service
              servicePort: 5000
          - path: /pth-auth/?(.*)
            # UPDATE THIS LINE ABOVE
            backend:
              serviceName: pth-auth
              servicePort: 8000

Inspired by: https://stackoverflow.com/a/31485557/99237

Troubleshooting tips:

• Here’s the template nginx ingress uses when transforming the ingress annotations into nginx config file.
• Connect to the ingress controller pod and look at /etc/nginx/nginx.conf to view the generated nginx config.