I have a single redis server running in a docker container on my server.
I use the defaults for everything.
I populate it with some key values and call save.
Every day though, it gets wiped out. The logs look like so
Before turning into a replica, using my own master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
REPLICAOF 46.12.32.122:8886 enabled (user request from 'id=66 addr=82.112.107.100:34932 fd=14 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=46 qbuf-free=32722 obl=0 oll=0 omem=0 events=r cmd=slaveof user=default')
Connecting to MASTER 46.12.32.122:8886
MASTER <-> REPLICA sync started
Non blocking connect for SYNC fired the event.
Master replied to PING, replication can continue...
Trying a partial resynchronization (request 05e89fe9fc1391690bdeed6ce650cfd4eb511553:1).
Full resync from master: ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ:1
Discarding previously cached master state.
MASTER <-> REPLICA sync: receiving 55664 bytes from master to disk
MASTER <-> REPLICA sync: Flushing old data
MASTER <-> REPLICA sync: Loading DB in memory
Wrong signature trying to load DB from file
Failed trying to load the MASTER synchronization DB from disk
Connecting to MASTER 46.12.32.122:8886
MASTER <-> REPLICA sync started
Non blocking connect for SYNC fired the event.
Error reply to PING from master: '-Reading from master: Operation now in progress'
Connecting to MASTER 46.12.32.122:8886
MASTER <-> REPLICA sync started
Non blocking connect for SYNC fired the event.
Error reply to PING from master: '-Reading from master: Connection reset by peer'
Connecting to MASTER 46.12.32.122:8886
MASTER <-> REPLICA sync started
Non blocking connect for SYNC fired the event.
Error reply to PING from master: '-Reading from master: Connection reset by peer'
Connecting to MASTER 46.12.32.122:8886
MASTER <-> REPLICA sync started
Non blocking connect for SYNC fired the event.
Error reply to PING from master: '-Reading from master: Connection reset by peer'
Connecting to MASTER 46.12.32.122:8886
MASTER <-> REPLICA sync started
Non blocking connect for SYNC fired the event.
Error reply to PING from master: '-Reading from master: Operation now in progress'
Connecting to MASTER 46.12.32.122:8886
MASTER <-> REPLICA sync started
Non blocking connect for SYNC fired the event.
Error reply to PING from master: '-Reading from master: Connection reset by peer'
Module ./red2.so failed to load: It does not have execute permissions.
Setting secondary replication ID to 05e89fe9fc1391690bdeed6ce650cfd4eb511553, valid up to offset: 1. New replication ID is e6492767f48bc9203cda8c66520d29701364391d
MASTER MODE enabled (user request from 'id=66 addr=82.112.107.100:34932 fd=14 name= age=7 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=34 qbuf-free=32734 obl=0 oll=0 omem=0 events=r cmd=slaveof user=default')
I suppose this is related to the issue, but I am extremely confused as to why this happens (both the actual replication and the failure)
2
Answers
Issue: redis port was open to the internet.
Solution was to close it.
This is clearly the abuse of exposed Redis as explained in the following references.
Conclusion: we need to block all incoming requests to Redis from the Internet
https://www.linkedin.com/pulse/evening-redis-kinsing-malware-jasim-alam/
https://www.trendmicro.com/en_in/research/20/d/exposed-redis-instances-abused-for-remote-code-execution-cryptocurrency-mining.html
https://nbailey.ca/post/redis-hack/