I have a simple web app that uses volume
/persistent volume claim
to serve static data from there. Pods got scheduled only on the first worker node where volume resides.
How to deal with the shared
volume between nodes in case I want to scale pods and have them allocated across all the available nodes?
- name: app-nginx
image: nginx:1.20.0
command: ["/usr/sbin/nginx", "-g", "daemon off;"]
volumeMounts:
- name: static-volume
mountPath: "/opt/static"
volumes:
- name: static-volume
persistentVolumeClaim:
claimName: pvc-static
One option is to use NFS
and not physically allocate volume on an EC2 instance.
Another way is to duplicate static files for each pod (populate them into proper directory with init container) but that requires extra time and is not feasible for a lot a static data.
What’s the proper way to deal with such a case within Kubernetes? Is there a way to declare deployment which will be using logically same volume but physically different instances located on the different nodes?
2
Answers
What you are looking for is a volume provider that supports the
ReadOnlyMany
or
ReadWriteMany
Access Mode.Follow the documentation link to get a list of the officially supported ones.
If you are on
AWS
than probably usingEFS
through theNFS
plugin will be the easiest solution, but please take into account the fact it is an NFS-based solution and you might hit a performance penalty.As a side note, what you are trying to do smeels like an anti-pattern to me.
Docker images for an application should be self contained. In your case, having a container serving a static website should contain all the static files it needs to in order to be fully portable. This would remove the need to have an external volume with the data completely.
Yes, you are right one option is to use the NFS. You have to implement the ReadWriteMany or ReadOnlyMany : https://stackoverflow.com/a/57798369/5525824
If you have a scenario of ReadOnlyMany you can create the PVC in GCP with GKE. https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/readonlymany-disks
However, if you are looking forward to doing a write operation also the available option is to use FileSystem or NFS.
You can also checkout implementing the Minio if not want to use any managed service and following cloud-agnostic : https://min.io/
NAS : https://github.com/minio/charts#nas-gateway
Just for FYI : https://github.com/ctrox/csi-s3 performance might be not good.