I’m currently learning about Istio’s Gateway, and there’s something I don’t quite understand.
So when we install istio, we get the Deployment istio-ingressgateway
:
$ kubectl get deploy -n istio-system
NAME READY UP-TO-DATE AVAILABLE AGE
istio-egressgateway 1/1 1 1 3h9m
istio-ingressgateway 1/1 1 1 3h9m
istiod 1/1 1 1 3h9m
Now, this istio-ingressgateway
is exposed via the Service:
$ kubectl get svc -n istio-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-egressgateway ClusterIP 172.20.33.252 <none> 80/TCP,443/TCP 3h10m
istio-ingressgateway LoadBalancer 172.20.19.8 abc-xyz.us-east-1.elb.amazonaws.com 15021:30308/TCP,80:32116/TCP,443:30034/TCP,31400:30430/TCP,15443:30260/TCP 3h10m
istiod ClusterIP 172.20.27.14 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 3h10m
Now, the proxy (istio-ingressgateway deployment’s pods) listens on port 8080
, which is the targetPort
for port 80
in the istio-ingressgateway
Service.
Now let’s say I have the following Gateway configuration:
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: my-ingress
spec:
selector:
app: my-ingressgateway
servers:
- port:
number: 80
name: http2
protocol: HTTP2
hosts:
- "www.example.com"
What I don’t understand is, why is the server’s port.number
configuration is important, and on what it affects?
I’ll explain: when I provision a EKS cluster for example and install istio on it, so I get a AWS Load Balancer for that istio-ingressgateway
Service. One of the ports of the load balancer is port 80, which forwards the traffic to a nodePort, which is than routed to port 8080 in the proxy container:
...
- name: http2
nodePort: 32116
port: 80
protocol: TCP
targetPort: 8080
...
So basically, when I access <load-balancer-domain>:80
, I’m routed to one of the cluster nodes to port 32116, and from there to the appropriate pod, so the pod doesn’t even know what port I as a client used to reach it.
My question is then, what does the Gateway configuration "care" about what port I use to reach the proxy container?
EDIT
I’ll try to rephrase the question:
The istio-ingressgateway
pod listens on port 8080
. The istio-ingressgateway
Service routes traffic to this port. If so, what is the meaning of the .servers[].port
configuration in the Ingress
resource?
The documentation about Gateway says:
While Istio will configure the proxy to listen on these ports, it is the responsibility of the user to ensure that external traffic to these ports are allowed into the mesh.
And I ask: What do they mean by "Istio will configure the proxy to listen on these ports"? The proxy listens on the ports configured in the pod’s specification, for example 8080 for HTTP, and the Service is the one that specifies which port will forward traffic to these ports in the pod, so what is the meaning of this port number in the Ingress
resource ?!?!
Hope it’s more clear now
2
Answers
Ok I think I understand how it works.
Let's set the stage: we have the
istio-ingressgateway
Deployment, which runs the Envoy proxy pod, which is our Gateway. This pod listens on a set of ports, such as 9443, 8080, and few other. Additionally, there is a Service that serves this Deployment, which exposes the above-mentioned ports by ports of the Service. For example, port 8080 of the pod is exposed by port 80 of the Service. Now, we create aGateway
resource which defines aserver
that listens on port80
.Now ask yourself: The Envoy pod listens on port 8080, not 80. But the
server
configuration inGateway
specifies port 80. How does it know that traffic received on port 8080 should be handled as HTTP, as specified in the Gateway?Apparently, Istio (I guess it's
istiod
) reads theGateway
configuration, and if it sees aserver
with a port that corresponds to a port in the Envoy's Service (80), than it configures the corresponding port of the Envoy (8080) to listen for the specified protocol and settings.This explains the following behaviors:
Gateway
configuration, the application is unreachable immediately. This is because there's no matching between the port in the Service and the port in the Gateway, so istio says: ok port 8080 should not apply theseserver
's settings anymore.Service
configuration, you will still reach the application for a while (about 10-20 minutes). After this it stops. This is apparently because istio polls the Service's configuration every once in a while, and doesn't get notified immediately like it does with theGateway
. If you now change the port in theGateway
to the modified port in the Service, you will get traffic.Hope that's clear
The configuration of
Gateway
(and alsoVirtualService
andDestinationRule
) are abstractions for envoy. They don’t configure kubernetes but the envoys that run in theistio-ingressgateway
(and pod sidecar) containers.With your
my-ingress
gateway manifest you simple tell istio: Configure the istio-ingressgateway that runs in a pod matching the selectorapp: my-ingressgateway
to listen to requests for hostwww.example.com
on port80
.You can check out the envoy config with the
istioctl proxy-config
command, likeistioctl proxy-config all my-ingress
(add-o json
for a more detailed output (but in json)). See also docs about that command.So it’s actually:
Request
<load-balancer-domain>:80
->
<nodeIP>:32116
-> istio-ingressgateway pod
-> envoy config determines how to handle the request for
<load-balancer-domain>:80
(=Gateway
+VirtualService
for routing +DestinationRule
,EnvoyFilter
, etc)-> Forward to pod
-> istio sidecar handles request, based on envoy config (
EnvoyFilter
,Sidecar
,RequestAuthentication
, etc)-> Forward to main application in the pod
If you have access to the kubernetes node you can run
iptables -t nat -L KUBE-NODEPORTS | grep istio-ingressgateway
to see and follow the actual routing.