How to allow IP dynamically using ingress controller - Nginx

FarkhodSadykov
March 28, 2022
269 views
0 votes
2 Answers

My structure

Kubernetes cluster on GKE
Ingress controller deployed using helm
An application which will return list of IP ranges note: it will get updated periodically

curl https://allowed.domain.com
172.30.1.210/32,172.30.2.60/32

Secured application which is not working

What I am trying to do?

Have my clients IPs in my API endpoint which is done

curl https://allowed.domain.com
172.30.1.210/32,172.30.2.60/32

Deploy my example app with ingress so it can pull from the https://allowed.domain.com and allow people to access to the app

What I tried and didn’t work?

Deploy the application with include feature of nginx

nginx.ingress.kubernetes.io/configuration-snippet: |
      include /tmp/allowed-ips.conf;
      deny all;

yes its working but the problem is when /tmp/allowed-ips.conf gets updated the ingress config doesn’t

I tried to use if condition to pull the IPs from the endpoint and deny if user is not in the list

    nginx.ingress.kubernetes.io/configuration-snippet: |
      set $deny_access off;
      if ($remote_addr !~ (https://2ce8-73-56-131-204.ngrok.io)) {
        set $deny_access on;
      }

I am using nginx.ingress.kubernetes.io/whitelist-source-range annotation but that is not what I am looking for

None of the options are working for me.

Answers

- mozello
- April 1, 2022 at 12:30 pm
- 0 votes
0
From the official docs of ingress-nginx controller:

The goal of this Ingress controller is the assembly of a configuration file (nginx.conf). The main implication of this requirement is the need to reload NGINX after any change in the configuration file. Though it is important to note that we don’t reload Nginx on changes that impact only an upstream configuration (i.e Endpoints change when you deploy your app)

After the nginx ingress resource was initially created, the ingress controller assembles the nginx.conf file and uses it for routing traffic. Nginx web server does not auto-reload its configuration if the nginx.conf and other config files were changed.

So, you can work around this problem in several ways:
- update the k8s ingress resource with new IP addresses and then apply changes to the Kubernetes cluster (kubectl apply / kubectl patch / smth else) / for your options 2 and 3.
- run nginx -s reload inside an ingress Pod to reload nginx configuration / for your option 1 with include the allowed list file.
```
$ kubectl exec ingress-nginx-controller-xxx-xxx -n ingress-nginx -- nginx -s reload
```
- try to write a Lua script (there is a good example for Nginx+Lua+Redis here and here). You should have a good understanding of nginx and lua to estimate if it is worth trying.
Login or Signup to reply.

- snowpeak
- October 6, 2022 at 2:55 pm
- 0 votes
0
Sharing what I implemented at my workplace. We had a managed monitoring tool called Site24x7. The tool pings our server from their VMs with dynamic IPs and we had to automate the whitelisting of the IPs at GKE.

nginx.ingress.kubernetes.io/configuration-snippet allows you to set arbitrary Nginx configurations.
1. Set up a K8s CronJob resource on the specific namespace.
2. The CronJob runs a shell script, which
- fetches the list of IPs to be allowed (curl, getent, etc.)
- generates a set of NGINX configurations (= the value for nginx.ingress.kubernetes.io/configuration-snippet)
- runs a kubectl command which overwrites the annotation of the target ingresses.
Example shell/bash script:
```
#!/bin/bash

  site24x7_ip_lookup_url="site24x7.enduserexp.com"

  site247_ips=$(getent ahosts $site24x7_ip_lookup_url | awk '{print "allow "$1";"}' | sort -u)
ip_whitelist=$(cat <<-EOT

  # ---------- Default whitelist (Static IPs) ----------
  # Office
  allow vv.xx.yyy.zzz;
  # VPN
  allow aa.bbb.ccc.ddd;

  # ---------- Custom whitelist (Dynamic IPs) ----------

  $site247_ips                            # Here! 

  deny all;

  EOT
  )

  for target_ingress in $TARGET_INGRESS_NAMES; do
    kubectl -n $NAMESPACE annotate ingress/$target_ingress 
      --overwrite 
      nginx.ingress.kubernetes.io/satisfy="any" 
      nginx.ingress.kubernetes.io/configuration-snippet="$ip_whitelist" 
      description="*** $(date '+%Y/%m/%d %H:%M:%S') NGINX annotation 'configuration-snippet' updated by cronjob $CRONJOB_NAME ***"
  done
```
The shell/bash script can be stored as ConfigMap to be mounted on the CronJob resource.
Login or Signup to reply.