Nginx Question posted in
The official documentation can be found here.

Kubernetes ingress to pod running on same host? – Nginx

We are just getting started with k8s (bare metal on Ubuntu 20.04). Is it possible for ingress traffic arriving at a host for a load balanced service to go to a pod running on that host (if one is available)?

We have some apps that use client side consistent hashing (using customer ID) to select a service instance to call. The service instances are stateless but maintain in memory ML models for each customer. So it is useful (but not essential) to have repeated requests for a given customer go to the same service. Then we can just use antiAffinity to have one pod per host.

Our existing service discovery mechanism lets the clients find all the instances of the service and the nodes they are running on. All our k8s nodes are running the Nginx ingress controller.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I finally got this figured out. This was way harder than it should be IMO! Update: It's not working. Traffic frequently goes to the wrong pod.

    The service needs externalTrafficPolicy: Local (see docs).

    apiVersion: v1
kind: Service
metadata:
  name: starterservice
spec:
  type: LoadBalancer
  selector:
    app: starterservice
  ports:
    - port: 8168
  externalTrafficPolicy: Local

    The Ingress needs nginx.ingress.kubernetes.io/service-upstream: "true" (service-upstream docs).

    The nginx.ingress.kubernetes.io/server-alias: "~^starterservice-[a-z0-9]+\.example\.com" bit is because our service discovery updates DNS so each instance of the service includes the name of the host it is running on in its DNS name.

    apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: starterservice
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/server-alias: "~^starterservice-[a-z0-9]+\.example\.com"
    nginx.ingress.kubernetes.io/service-upstream: "true"
spec:
  rules:
    - host:  starterservice.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: starterservice
                port:
                  number: 8168

    So now a call https://starterservice-foo.example.com will go to the instance running on k8s host foo.


  2. 0

    I believe Sticky Sessions is what you are looking for. Ingress does not communicate directly with pods, but with services. Sticky sessions try to bind requests from the same client to the same pod by setting an affinity cookie.

    This is used for example with SignalR sessions, where the negotiation request has to be on the same host as the following websocket connection.

    Example:

    apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minimal-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/affinity: cookie
    nginx.ingress.kubernetes.io/affinity-mode: persistent
spec:
  rules:
  - http:
      paths:
      - path: /testpath
        pathType: Prefix
        backend:
          service:
            name: test
            port:
              number: 80

    Affinity mode "balanced" is the default. If your pod count does not change, part of your clients will lose the session. Use "persistent" to have users connect to the same pod always (unless it dies of course). Further reading: https://github.com/kubernetes/ingress-nginx/issues/5944

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search