Too many redirects on ingress-controller - Nginx

Displayname
September 28, 2021
139 views
0 votes
2 Answers

I am trying to setup an Ingress Controller based upon:
https://kubernetes.github.io/ingress-nginx/deploy/#aws
It works fine for ELB, but for some reason, if I set the following in NLB:

nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"

then I am getting a Too many redirects error.
If I set the above to false then I can access both HTTP and HTTPS separately but there is no redirection.

In my Service annotations for NLB I have:

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true'
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: '3600'
    service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:eu-west-1:12345:certificate/xyz
    service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-FS-1-2-2019-08
...
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
      appProtocol: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: http
      appProtocol: https

for ELB where it works ok I have:

    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: '60'
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true'
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:eu-west-1:12345:certificate/xyz
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
    service.beta.kubernetes.io/aws-load-balancer-type: elb
...
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: tohttps
      appProtocol: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: http
      appProtocol: https

I’ve tried many combinations but I can’t get NLB to act in the same way like with ELB.

Tags: kubernetes kubernetes-ingress

Answers

Chosen as BEST ANSWER
- Displayname
- September 29, 2021 at 1:01 pm
- 0 votes
0
If backend protocol set to "ssl" everything works fine, except the fact that we're doing double TLS offloading for no reason (on NLB first, then on ingress). If backend protocol set to "tcp", we'll get "Plain HTTP request sent to TLS port" error. If we map https to http port to address the above then HTTP -> HTTPS redirects stop working.

So to make it working with NLB I needed set the backend protocol to ssl: service.beta.kubernetes.io/aws-load-balancer-backend-protocol: ssl and then:
```
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: https
```

(Edit)

Try with removing the appProtocol: https and offload SSL at the LB level

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true'
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: '3600'
    service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:eu-west-1:12345:certificate/xyz
    service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-FS-1-2-2019-08
...
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: HTTP

You can check config at : https://aws.amazon.com/blogs/opensource/network-load-balancer-nginx-ingress-controller-eks/

Also, check from AWS console LB having 80 and TLS 443 Listeners.

SSL offloading & terinating : https://aws.amazon.com/premiumsupport/knowledge-center/terminate-https-traffic-eks-acm/