skip to Main Content

We normally use command dbsize to count elements, using expiration. For instance, to count how many events of type X happened in the last 10 seconds, we dedicate a DB to count those elements, inserting them with expiration set to 10 seconds. The value of command dbsize gives the number of elements.

I know that expired keys are not reflected in dbsize, and that is why we set a flag in that database, with expiration, for instance, 1 sec. Every time we go to check dbsize, we check first if that flag has expired. If expired, we run “keys *” to refresh status of keys, and set the flag again. That way we refresh the status of expired elements only every second, to not put too much load on Redis. It’s not many elements, a few hundreds or thousands.

However, now it’s not working. Maybe something changed in the last implementation. Even after executing “keys *”, the value of dbsize is still not updated. Look:

I set three values, with expiration 10 seconds.

127.0.0.1:6379[20]> set kk1 1 ex 10
OK
127.0.0.1:6379[20]> set kk2 1 ex 10
OK
127.0.0.1:6379[20]> set kk3 1 ex 10
OK

Check all keys:

127.0.0.1:6379[20]> keys *
1) "kk1"
2) "kk2"
3) "kk3"

dbsize returns 3:

127.0.0.1:6379[20]> dbsize
(integer) 3

Keys start to expire:

127.0.0.1:6379[20]> keys *
1) "kk2"
2) "kk3"
127.0.0.1:6379[20]> dbsize
(integer) 3

Now, here is the problem:

127.0.0.1:6379[20]> keys *
(empty list or set)

“keys *” already reflects that all keys expired, so there are no keys in the database. However, dbsize was not updated.

127.0.0.1:6379[20]> dbsize
(integer) 3

It takes a loooong time after “keys *” indicated empty database for dbsize to really update its value.

It broke the functionality of many working scripts we have.

I am testing it on Fedora 30 : redis-5.0.7-1.fc30.x86_64

This worked before. What changed?

Any ideas?

2

Answers


  1. Chosen as BEST ANSWER

    Well, I could not find a way to update dbsize.

    So, I am counting. Instead of using the result of dbsize I use:

    local allKeys = redis.call( "keys", "*" )
    totalKeys = #allKeys - 1 -- -1 is for not counting totalKeys itself
    

    And insert totalKeys back into Redis, to use that value until next refresh.


  2. You can try debug object kk1 to see if the key is still in memory, even if not returned by keys *.

    I can only think of one scenario where what you described would happen:

    1. You have set the config lazyfree-lazy-expire yes
    2. You have done debug SET-ACTIVE-EXPIRE 0

    To check lazyfree-lazy-expire, try config get lazyfree-lazy-expire.

    The second one cannot be consulted, but you do debug SET-ACTIVE-EXPIRE 1 to ensure normal mode.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search