i have been testing this situation, where i pick an IP address from the nodepool subnet and using it as a external ip of an load-balancer service. it works, but i wonder if this is a correct thing to do?
i would like to know how AKS Azure CNI keeps track of what are the IPS have been assigned to pods? can we control this , so that we can reserve some ips from this subnet range?
as a summery is it possible to ‘reserve’ an ip(s) from aks subnet so they wont get auto assigned to pods?
thank you,
asitha
looking for recommendations
2
Answers
For anyone else having the same question. I also cross posted question on Microsoft.
depending on the default number of node and pod allocations, we can predict what are the free node sub-net ips at the start of the cluster, and we can use these ips's statically on the configuration. ( in other words, if an ip is available at the point of allocation, its all good, the service will be allocated the specific ip, and the underlying azure vnet will be updated, as @Alan Cheng mentioned, so when ever a new ip is requested by AKS vnet will make sure that it will not issue an which is already in use )
Link to Ms question and answer for more details
https://learn.microsoft.com/en-us/answers/questions/1074378/aks-azure-cni-reserving-a-static-ip-from-aks-node.html
For setting external IP for LB type service, I assume you’re referring to services of internal LB type. Yes, you can pick an unused IP from nodepool subnet and assign it to a service of internal LB type, but better to let Azure pick the IP instead of you picking it manually.
As for how Azure CNI manages IP addresses, when you create a node, that node will get assigned an IP from the nodepool subnet, but also another 30 (default max pod per node, can be changed) IPs are pre-allocated to that instance too, and these IPs will no longer available in the nodepool subnet until the node is removed, even when there are no pods running on that node (yet).
You can check the pre-allocation of IPs by going to the virtual network of your aks cluster, under the "connected devices", you can all IPs allocated to a single cluster node (or vmss instance, in another term), as below:
I don’t think there is a way to reserve IP from the nodepool subnet, but Azure does allow you to allocate IPs from a different subnet for services of internal LB: https://learn.microsoft.com/en-us/azure/aks/internal-lb#specify-a-different-subnet
Also, there is another preview feature to allocate IP dynamically instead of pre-allocate IPs to nodes:
https://learn.microsoft.com/en-us/azure/aks/configure-azure-cni#dynamic-allocation-of-ips-and-enhanced-subnet-support-preview