Nginx Question posted in
The official documentation can be found here.

Customize nginx behavior differently for each path in the Nginx Ingress http rule

I am using the nginx ingress (community edition) to serve different services within my Kubernetes cluster while trying to optimize costs.

I encountered issues when trying to add a websocket rule that although it’s supported out of the box from the controller it actually requires (if I understood correctly) setting with annotations:

nginx.ingress.kubernetes.io/proxy-read-timeout: 3600
nginx.ingress.kubernetes.io/proxy-send-timeout: 3600

Now I want to use this setting only for the http path exploited by the websocket and not for the others. How can I accomplish that?

— Extra Info —

I tried to look in the documentation, but I didn’t find any way to fine tune this and, changing the nginx.conf manually doesn’t seem an option here. Moreover, the nginx inc controller has "mergeable" ingresses with master and minions that I guess can get different configurations easily through annotations, but it’s not supported in the community edition (and I don’t wanna switch to the limited free nginx inc one).

As an extra detail the nginx inc version lets you specify the websockets services with an annotation nginx.org/websocket-services: "ws-svc" that it’s not available in the community edition (and I am not sure what it does).

I will paste here for the sake of completeness a similar structure to my ingress without including everything unrelated

apiVersion: networking.k8s.io/v1
    kind:Ingress
    metadata:
      name: nlb-ingress
      namespace: test
      annotations:
        nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
        nginx.ingress.kubernetes.io/rewrite-target: "/$2"
    spec:
      ingressClassName: "nginx"
      rules
      - host = cafe.example.com
http:
  paths:
  - path: /banana(/|$)(?:.*)
    pathType: Prefix
    backend:
      service:
        name: banana-svc
        port:
          number: 5678
  - path: /webapp(/|$)(.*)
    pathType: Prefix
    backend:
      service:
        name: webapp
        port:
          number: 80
  - path: /ws
    pathType: Prefix
    backend:
      service:
        name: ws-svc
        port:
          number: 8008

Moreover, being able to customize differently the paths within an ingress resource would also help me for other things/parameters like having better rewrite targets, so I guess that whatever solution applies here it will apply also to other configurations through annotations.

Tags:

2

Answers


  1. 0

    You’re right that this behaviour is possible with NGINX Ingress Controller via Mergeable Ingresses. Mergeable Ingresses are supported in both OSS and Plus NGINX Ingress Controllers.

    If you want to use the proxy_send_timeout or proxy_read_timeout directives in NGINX Ingress Controller you can use these annotations

    • nginx.org/proxy-send-timeout
    • nginx.org/proxy-read-timeout

    You can see the full list of supported annotations in the docs.

    changing the nginx.conf manually doesn’t seem an option here

    This is an option, and can be achieved in two ways, one is with snippets, the other is with templating (if its the nginx.conf you want to change, this is the main-template). Snippets is the easier one to maintain so always consider that first. And you can target the http or stream blocks using a config map, or the server and location blocks using either config maps or annotations (annotations if you want them to be only on a specific ingress). I don’t think you will need snippets or templating for the behaviour you are asking about however.

    As an extra detail the nginx inc version lets you specify the websockets services with an annotation nginx.org/websocket-services: "ws-svc" that it’s not available in the community edition (and I am not sure what it does).

    What this nginx.org/websocket-services annotation does is inform NGINX Ingress Controller which services are websocket services. Under the hood what happens is these two directives are added to the location block which uses the websocket service.

    proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;

    You can see an example of using this directive here. I have copied this below. As you can see, the service which the /ws path uses, ws-svc is referenced in the nginx.org/websocket-services annotation, telling NGINX Ingress Controller to insert the two proxy_set_header directives, in location /ws in the generated config.

    apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: cafe-ingress
  annotations:
    nginx.org/websocket-services: "ws-svc"
spec:
  rules:
  - host: cafe.example.com
    http:
      paths:
      - path: /tea
        pathType: Prefix
        backend:
          service:
            name: tea-svc
            port:
              number: 80
      - path: /coffee
        pathType: Prefix
        backend:
          service:
            name: coffee-svc
            port:
              number: 80
      - path: /ws
        pathType: Prefix
        backend:
          service:
            name: ws-svc
            port:
              number: 8008

    Lastly, to put these two annotations below which you mentioned on a specific route, you will need to use Mergeable Ingresses.

    • nginx.org/proxy-send-timeout
    • nginx.org/proxy-read-timeout

    You can use the Mergeable Ingresses example for this. Modify the master and minion(s) to get the behaviour you want. It seems you will have a minion on the path /ws with the nginx.org/proxy-send-timeout, nginx.org/proxy-read-timeout and nginx.org/websocket-services annotations set.

    Login or Signup to reply.
  2. 0 
       You can have a multiple paths like this
`    location /coffee {
  proxy_http_version 1.1;
  proxy_connect_timeout 60s;
  proxy_read_timeout 60s;
  client_max_body_size 1m;
  proxy_set_header Host $host;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Host $host;
  proxy_set_header X-Forwarded-Port $server_port;
  proxy_set_header X-Forwarded-Proto $scheme;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection $connection_upgrade;
  proxy_buffering on;
  proxy_pass http://default-cafe-ingress-coffee-minion-cafe.example.com-coffee-svc;
 }
  # *Minion*, configured in Ingress Resource: default-cafe-ingress-tea-minion
 location /tea {
  proxy_http_version 1.1;
  proxy_connect_timeout 60s;
  proxy_read_timeout 60s;
  client_max_body_size 1m;
  proxy_set_header Host $host;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Host $host;
  proxy_set_header X-Forwarded-Port $server_port;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection $connection_upgrade;
  proxy_set_header X-Forwarded-Proto $scheme;
  proxy_buffering on;
  proxy_pass http://default-cafe-ingress-tea-minion-cafe.example.com-tea-svc;
 }    `
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search