Oauth2-proxy and Traefik: too many redirect – Nginx

I have been trying for days and countless hours to make this thing work with Traefik, however not matter what I have tried nothing has been working so far.

I have tried first with Nginx ingress controller and managed to make it work, so I am sure that my provider (Cognito) and the oauth2-proxy pod is working correctly.

So, I am using an EKS 1.20 cluster and we have Traefik installed via helm chart

replicas: 1

rbac:
  enabled: true

accessLogs:
  enabled: false

service:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "my-certs"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"

externalTrafficPolicy: Local

ssl:
  enabled: true
  enforced: true
  upstream: true

helm upgrade -i traefik traefik/traefik -f traefik-values.yaml

This is the config of my oauth proxy, which works with Nginx.
In the Oauth-proxy there is no mention about Traefik so I guess this should work as well:

config:
  clientID: "xxxxx"
  clientSecret: "xxxxxx"
  cookieSecret: "xxxx"
  configFile: |-
    email_domains = [ "*" ]
    upstreams = [ "file:///dev/null" ]

extraArgs:
  oidc-issuer-url: "https://cognito-idp.<region>.amazonaws.com/<pool_id>"
  oidc-jwks-url: "https://cognito-idp.<region>.amazonaws.com/<pool_id>/.well-known/jwks.json"
  provider: oidc
  provider-display-name: "Cognito SSO"
  cookie-secure: false
  cookie-name: "_oauth2_proxy"
  skip-provider-button: true
  scope: openid
  reverse-proxy: true
  real-client-ip-header: X-Forwarded-For
  whitelist-domain: mycompany.com
  cookie-domain: mycompany.com
  set-authorization-header: true
  

Those is the configuration suggested from their documentation transformed to Kubernetes: https://oauth2-proxy.github.io/oauth2-proxy/docs/configuration/overview#configuring-for-use-with-the-traefik-v2-forwardauth-middleware

---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: auth-headers
spec:
  headers:
    sslRedirect: true
    stsSeconds: 315360000
    browserXssFilter: true
    contentTypeNosniff: true
    forceSTSHeader: true
    stsIncludeSubdomains: true
    stsPreload: true
    frameDeny: true

---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: oauth-auth
spec:
  forwardAuth:
    address: https://mycompany.com/oauth2/auth
    trustForwardHeader: true

---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: oauth-errors
spec:
  errors:
    status:
      - "401-403"
    service:
      name: oauth2-proxy
      port: 80
    query: "/oauth2/sign_in"

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: oauth2-proxy
spec:
  entryPoints:
    - websecure
  routes:
    - match: "Host(`mycompany.com`) && PathPrefix(`/oauth2/`)"
      kind: Rule
      services:
        - kind: Service
          name: oauth2-proxy
          port: 80
      middlewares:
        - name: auth-headers

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: linkerd-dashboard-ingress
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  entryPoints:
    - websecure
  routes:
    - match: "Host(`mycompany.com`)"
      kind: Rule
      services:
        - name: nginx-service
          kind: Service
          port: 8080
      priority: 1
      middlewares:
        - name: oauth-auth
        - name: oauth-errors

With this configuration now it will get redirected, however it goes in loop and I have ERR_TOO_MANY_REDIRECTS

If I follow with curl it will go in loop:

HTTP/1.1 301 Moved Permanently
Content-Type: text/html; charset=utf-8
Location: https://mycompany.com/oauth2/auth
Date: Wed, 08 Sep 2021 10:12:04 GMT
Content-Length: 84

HTTP/1.1 301 Moved Permanently
Content-Type: text/html; charset=utf-8
Location: https://mycompany.com/oauth2/auth
Date: Wed, 08 Sep 2021 10:12:04 GMT
Content-Length: 84

...