I am connecting to a small Redis cluster hosted by AWS which is not sharded, it has just one master and 3 slaves. I have been having difficulty with replication, and this is a simple reproduction of what I tried.
My clusters:
172.28.52.18:6379> CLUSTER NODES
cc378ecb71d02c2495e219ce7043ea343eb91c1f 172.28.52.18:6379@1122 myself,slave 6ce7214224036cc42ba272486d9e8fe5d1b11875 0 1610811475000 2 connected
1e17d7794741c7db491a888dc2bca76590b52e64 172.28.53.83:6379@1122 slave 6ce7214224036cc42ba272486d9e8fe5d1b11875 0 1610811476195 2 connected
6ce7214224036cc42ba272486d9e8fe5d1b11875 172.28.53.213:6379@1122 master - 0 1610811474182 2 connected 0-16383
d780dfbb4d33275e50c098032d1cceb1cd368a65 172.28.52.180:6379@1122 slave 6ce7214224036cc42ba272486d9e8fe5d1b11875 0 1610811475189 2 connected
Connect directly to master (172.28.53.213 in the above output) and set some data:
$ redis-cli -h 172.28.53.213
172.28.53.213:6379> SET key1 value1
OK
Then connect to one of the slaves:
$ redis-cli -h 172.28.52.180
172.28.52.180:6379> GET key1 value1
(error) MOVED 9189 172.28.53.213:6379
There are no shards and as I see it:
- All slots (0-16383) are owned by master
- All of the slaves are connected to master
- Everything is in sync
So I don’t understand why I would be redirected to master. Shouldn’t the slave have a copy?
I am aware that I can connect with -c
so that redirects are followed automatically, but the whole point of replication should be that redirects aren’t necessary!
2
Answers
You can read about it here.
If you want to read from a replica you have to execute READONLY command on it. And to revert to old behaviour there is a command READWRITE.
As a Redis newbee, I encountered a similar issue and came to this thread. But the answer doesn’t tell why. Finally, I figured it out after reading the redis server log (specified by
logfile
in the config) of the slave node, and I hope my answer helps.In my case, the root cause is that the slave config file does not enable
masterauth <password>
while the master node enablesrequirepass <password>
. It is commented out. In the log, this error leads me to the solution,In the cluster mode, when a failover occurs, a slave node is promoted to a master node. Later when the failed original master node comes back, it becomes a slave node. So, if the original slave node enables
requirepass <password>
, remember to enablemasterauth <password>
in the original master node config too. Otherwise, after the failover, the replication from the new master to the new slave still fails.And one more thing I learned is that I need to restart the redis node after editing the config file.