skip to Main Content

I am using a Kubernetes deployment model inside Azure, having an OAuth2 proxy(https://github.com/oauth2-proxy/oauth2-proxy) which is protecting the cluster resources by enabling SSO login through various clients. That is ok from the end user perspective who can easily login with his SSO client. The problem appears when the APIs exposed by the services behind the OAuth2 proxy, need to be consumed by external applications via REST calls.

The configuration is the default one, having a dedicated Kubernetes service for OAuth2 and the following rules inside the Ingress file.

nginx.ingress.kubernetes.io/auth-signin: 'https://$host/oauth2/start?rd=$request_uri'
nginx.ingress.kubernetes.io/auth-url: 'https://$host/oauth2/auth'

From what I checked, currently those services can only by consumed via REST calls from an external application(Postman for example) only if I add a cookie parameter(_oauth2_proxy) which is generated after a successful login using the UI and the SSO client provider. If I do not add this cookie parameter an error such as cookie _oauth_proxy is not present.

Is there any option which I can add to the proxy configuration in order to permit token based authentication and authorization/programmatic generation of some identifier in order to access the resources behind the OAuth2 proxy for a technical user(external application) through REST calls? I can generate an access token based on the existing configuration(client id, secret, application scope, etc).

The OAuth2 proxy deployment YAML looks like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: oauth2-proxy
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: oauth2-proxy
  template:
    metadata:
      labels:
        app: oauth2-proxy
    spec:
      containers:
      - env:
          - name: OAUTH2_PROXY_PROVIDER
            value: azure
          - name: OAUTH2_PROXY_AZURE_TENANT
            value: <REPLACE_WITH_DIRECTORY_ID>
          - name: OAUTH2_PROXY_CLIENT_ID
            value: <REPLACE_WITH_APPLICATION_ID>
          - name: OAUTH2_PROXY_CLIENT_SECRET
            value: <REPLACE_WITH_SECRET_KEY>
          - name: OAUTH2_PROXY_COOKIE_SECRET
            value: <REPLACE_WITH_VALUE_OF python -c 'import os,base64; print base64.b64encode(os.urandom(16))'>
          - name: OAUTH2_PROXY_HTTP_ADDRESS
            value: "0.0.0.0:4180"
          - name: OAUTH2_PROXY_UPSTREAM
            value: "<AZURE KUBERNETES CLUSTER HOST e.g.  >"
        image: bitnami/oauth2-proxy:latest
        imagePullPolicy: IfNotPresent
        name: oauth2-proxy
        ports:
        - containerPort: 4180
          protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: oauth2-proxy
  name: oauth2-proxy
  namespace: default
spec:
  ports:
  - name: http
    port: 4180
    protocol: TCP
    targetPort: 4180
  selector:
    app: oauth2-proxy 

EDIT:

I was finally able to use the generated token through the AD OAuth2 token endpoint in order to call my APIs behind the proxy. In order to achieve that, I changed the docker image from machinedata/oauth2_proxy to bitnami/oauth2-proxy.
Beside that I added the following arguments to the container:

 args:
 - '--provider=azure'
 - '--azure-tenant=TENANT_ID'
 - '--skip-jwt-bearer-tokens=true'
 - >-
     --oidc-issuer-url=https://sts.windows.net/TENANT_ID/
 - >-
    --extra-jwt-issuers=https://login.microsoftonline.com/TENANT_ID/v2.0=APP_ID
 - '--request-logging=true'
 - '--auth-logging=true'
 - '--standard-logging=true'

Also, I had to do some changes at the app registration manifest from Azure AD in order for the token to be validated by the OAuth2 proxy against the correct version.

"accessTokenAcceptedVersion": 2

I found some useful explanations in here as well: https://github.com/oauth2-proxy/oauth2-proxy/issues/502 .

Now I can use the token endpoint provided by Azure to generate a Bearer token for my API calls. The only issue which is still remaining is an error which I have when I try to access the application UI.

The error/warning received in the pod logs is:
WARNING: Multiple cookies are required for this session as it exceeds the 4kb cookie limit. Please use server side session storage (eg. Redis) instead.

The error received in the browser is 502 Bad Gateway

EDIT #2:

I was able to bypass this new error by increasing buffer size at my ingress level for OAuth2 proxy. More details can be found in the below URLs.
https://oauth2-proxy.github.io/oauth2-proxy/docs/configuration/oauth_provider/#azure-auth-provider
https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/

2

Answers


  1. Chosen as BEST ANSWER

    I finally made it work using the following configuration:

    deployment.yaml for OAuth2 proxy:

    kind: Deployment
    apiVersion: apps/v1
    metadata:
      name: oauth2-proxy
      namespace: default
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: oauth2-proxy
      template:
        metadata:
          creationTimestamp: null
          labels:
            app: oauth2-proxy
        spec:
          containers:
            - name: oauth2-proxy
              image: 'bitnami/oauth2-proxy:latest'
              args:
                - '--provider=azure'
                - '--azure-tenant=TENANT_ID'
                - '--skip-jwt-bearer-tokens=true'
                - >-
                  --oidc-issuer-url=https://sts.windows.net/TENANT_ID/
                - >-
                  --extra-jwt-issuers=https://login.microsoftonline.com/TENANT_ID/v2.0=CLIENT_ID
                - '--request-logging=true'
                - '--auth-logging=true'
                - '--standard-logging=true'
              ports:
                - containerPort: 4180
                  protocol: TCP
              env:
                - name: OAUTH2_PROXY_AZURE_TENANT
                  value: TENANT_ID
                - name: OAUTH2_PROXY_CLIENT_ID
                  value: CLIENT_ID
                - name: OAUTH2_PROXY_CLIENT_SECRET
                  value: CLIENT_SECRET
                - name: OAUTH2_PROXY_COOKIE_SECRET
                  value: COOKIE_SECRET
                - name: OAUTH2_PROXY_HTTP_ADDRESS
                  value: '0.0.0.0:4180'
                - name: OAUTH2_PROXY_UPSTREAM
                  value: 'http://your-host'
                - name: OAUTH2_PROXY_EMAIL_DOMAINS
                  value: '*'
    
    

    ingress.yaml for OAuth2 proxy:

    kind: Ingress
    apiVersion: networking.k8s.io/v1beta1
    metadata:
      name: oauth2-proxy
      namespace: default
      labels:
        app: oauth2-proxy
      annotations:
        kubernetes.io/ingress.class: addon-http-application-routing
        # in my case the generated cookie was too big so I had to add the below parameters
        nginx.ingress.kubernetes.io/proxy-buffer-size: 8k
        nginx.ingress.kubernetes.io/proxy-buffers-number: '4'
    spec:
      tls:
        - hosts:
            - YOUR_HOST
      rules:
        - host: YOUR_HOST
          http:
            paths:
              - path: /oauth2
                backend:
                  serviceName: oauth2-proxy
                  servicePort: 4180
    

    In addition to those configuration files, I also had to change the value for accessTokenAcceptedVersion in the Azure application registration manifest. By default this value is setup to null which means it will go for V1 tokens instead of V2 as specified in the extra-jwt-issuers argument.

    "accessTokenAcceptedVersion": 2
    

    After those changes were in place, I was able to use the generated token through Azure token endpoint in order to go through the OAuth2 proxy and reach my application exposed APIs:

    HTTP POST to https://login.microsoftonline.com/TENANT_ID/oauth2/v2.0/token
    Content-Type: application/x-www-form-urlencoded
    Body:
      - client_id: YOUR_CLIENT_ID
      - grant_type: client_credentials
      - client_secret: YOUR_CLIENT_SECRET
      - scope: api://YOUR_CLIENT_ID/.default - this was generated by me, but it should work with MS Graph as well
    

  2. I am using a Kubernetes deployment model inside Azure, having an OAuth2 proxy which is protecting the cluster resources by enabling SSO login.
    I have the Oauth service running successfully, I also have the application ingress and the oauth ingress deployed. But when I access the application URL, I got 500 Internal error. If I access the Oauth ingress url,I got the login window. I can provide the details on the oauth deployment yaml and the application ingress and oauth ingress yaml.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search