go-redis does not recover after being idle and I am not sure how to configure it to automatically reconnect and retry. When my service tries to execute a redis command after being idle, it fails the first time. However, subsequent attempts do work as expected.
Here is the error:
dial tcp: lookup z.ec2.redns.redis-cloud.com: i/o timeout
Here is the line of code that fails:
r.SIsMember(context.TODO(), "x", x).Result()
And here is how I’m connecting:
opt, err := redis.ParseURL("redis://x:[email protected]:12719?dial_timeout=5&read_timeout=6s&max_retries=2")
if err != nil {
log.Fatal().Err(err).Send()
}
opt.DialTimeout = 5 * time.Second
opt.ReadTimeout = 3 * time.Second
opt.WriteTimeout = 3 * time.Second
opt.PoolTimeout = 6 * time.Second
opt.MinRetryBackoff = 1 * time.Second
opt.MaxRetryBackoff = 5 * time.Second
opt.MaxRetries = 5
How can I recover from idle connections on the first try?
2
Answers
It looks like after your service becomes idle, all your Redis client pool connections are discarded by
ConnMaxIdleTime
(defaults to 30min), and when the pool tries to create a new connection it timeouts (DialTimeout
).Timeout errors when creating a new connection are not retried by the Redis Go client, so you get the error directly, and probably on the next try the connection succeeds faster.
You can try to increment your
DialTimeout
so the pool has a little bit more margin when trying to create a new connection.I don’t remember there’s a configuration for go to retry connection automatically, but you could set up a manual loop to check for
i/o timeout
error and if it encounters the error, it will attempt to create a new client to retry the redis operation until it reachesn
number of attempts or the operation succeeds, whichever comes first