I am trying to configure a redis statefulset on a specific node_pool, but when I apply the manifest, I get the following error.
0/4 nodes are available: 1 node(s) had volume node affinity conflict, 3 node(s) didn't match Pod's node affinity/selector.
My GKE is Regional and I create a nodepool with one replica in one zone, because it is sufficient for the redis work.
The node_pool is in a single zone. My idea is that the 3 replicas live inside that particular node.
How can I configure that the 3 PVCs are in the same node_pool?
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis
namespace: redis
spec:
serviceName: redis
replicas: 3
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
initContainers:
- name: config
image: redis:6.2.3-alpine
command: [ "sh", "-c" ]
args:
- |
cp /tmp/redis/redis.conf /etc/redis/redis.conf
echo "finding master..."
MASTER_FDQN=`hostname -f | sed -e 's/redis-[0-9]./redis-0./'`
if [ "$(redis-cli -h sentinel -p 5000 ping)" != "PONG" ]; then
echo "master not found, defaulting to redis-0"
if [ "$(hostname)" == "redis-0" ]; then
echo "this is redis-0, not updating config..."
else
echo "updating redis.conf..."
echo "slaveof $MASTER_FDQN 6379" >> /etc/redis/redis.conf
fi
else
echo "sentinel found, finding master"
MASTER="$(redis-cli -h sentinel -p 5000 sentinel get-master-addr-by-name mymaster | grep -E '(^redis-d{1,})|([0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3})')"
echo "master found : $MASTER, updating redis.conf"
echo "slaveof $MASTER 6379" >> /etc/redis/redis.conf
fi
volumeMounts:
- name: redis-config
mountPath: /etc/redis/
- name: config
mountPath: /tmp/redis/
containers:
- name: redis
image: redis:6.2.3-alpine
command: ["redis-server"]
args: ["/etc/redis/redis.conf"]
ports:
- containerPort: 6379
name: redis
volumeMounts:
- name: data
mountPath: /data
- name: redis-config
mountPath: /etc/redis/
volumes:
- name: redis-config
emptyDir: {}
- name: config
configMap:
name: redis-config
nodeSelector:
cloud.google.com/gke-nodepool: redis-pool
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "standard"
resources:
requests:
storage: 50Mi
---
apiVersion: v1
kind: Service
metadata:
name: redis
namespace: redis
spec:
clusterIP: None
ports:
- port: 6379
targetPort: 6379
name: redis
selector:
app: redis
2
Answers
UPDATE: I found a solution, i create a StorageClass with a "matchLabelExpressions" matching a specific zone, so I define to use that StorageClass on my StafefulSet manifest matching the StorageClass name to "data"
Statefulset
StorageClass
Can you ensure that the nodes in your node_pool have the label
redis-pool
? From your requirements, you should only have 1 node with this label, which will allow all 3 replicas to be scheduled to this node.And it is not technically correct to say
PVCs in the same node_pool
. You are using thestorageClass: default
in your PVC template, this will cause yourPersistentVolumeClaims
to be backed byGoogle Persistent Disks (GPDs)
. Which are separate from your node_pools (compute resource) and should not be considered part of the compute infrastructure.They belong to the storage part of your GCloud. The GKE however handles the connection between the storage and compute transparently to you, so it doesn’t feel like you’re using the GPDs, but it is important to understand this distinction.