skip to Main Content

go-redis does not recover after being idle and I am not sure how to configure it to automatically reconnect and retry. When my service tries to execute a redis command after being idle, it fails the first time. However, subsequent attempts do work as expected.

Here is the error:

dial tcp: lookup z.ec2.redns.redis-cloud.com: i/o timeout

Here is the line of code that fails:

r.SIsMember(context.TODO(), "x", x).Result()

And here is how I’m connecting:

    opt, err := redis.ParseURL("redis://x:[email protected]:12719?dial_timeout=5&read_timeout=6s&max_retries=2")
    if err != nil {
        log.Fatal().Err(err).Send()
    }

    opt.DialTimeout = 5 * time.Second
    opt.ReadTimeout = 3 * time.Second
    opt.WriteTimeout = 3 * time.Second
    opt.PoolTimeout = 6 * time.Second
    opt.MinRetryBackoff = 1 * time.Second
    opt.MaxRetryBackoff = 5 * time.Second
    opt.MaxRetries = 5

How can I recover from idle connections on the first try?

2

Answers


  1. It looks like after your service becomes idle, all your Redis client pool connections are discarded by ConnMaxIdleTime (defaults to 30min), and when the pool tries to create a new connection it timeouts (DialTimeout).

    Timeout errors when creating a new connection are not retried by the Redis Go client, so you get the error directly, and probably on the next try the connection succeeds faster.

    You can try to increment your DialTimeout so the pool has a little bit more margin when trying to create a new connection.

    Login or Signup to reply.
  2. I don’t remember there’s a configuration for go to retry connection automatically, but you could set up a manual loop to check for i/o timeout error and if it encounters the error, it will attempt to create a new client to retry the redis operation until it reaches n number of attempts or the operation succeeds, whichever comes first

    success := false
    attempts := 0
    const MaxAttempts = 5
    r := redis.NewClient(opt)
    
    for success != true && attempts < MaxAttempts {
        val, err := r.SIsMember(context.TODO(), "x", "x").Result()
    
        if err == nil {
            success = true
            fmt.Printf("SIsMember %t", val) 
            // Do something with val result
        } else if strings.Contains(fmt.Sprint(err), "i/o timeout") {
            attempts += 1
            r = redis.NewClient(opt)
        }
    }
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search