We normally use command dbsize to count elements, using expiration. For instance, to count how many events of type X happened in the last 10 seconds, we dedicate a DB to count those elements, inserting them with expiration set to 10 seconds. The value of command dbsize gives the number of elements.
I know that expired keys are not reflected in dbsize, and that is why we set a flag in that database, with expiration, for instance, 1 sec. Every time we go to check dbsize, we check first if that flag has expired. If expired, we run “keys *” to refresh status of keys, and set the flag again. That way we refresh the status of expired elements only every second, to not put too much load on Redis. It’s not many elements, a few hundreds or thousands.
However, now it’s not working. Maybe something changed in the last implementation. Even after executing “keys *”, the value of dbsize is still not updated. Look:
I set three values, with expiration 10 seconds.
127.0.0.1:6379[20]> set kk1 1 ex 10
OK
127.0.0.1:6379[20]> set kk2 1 ex 10
OK
127.0.0.1:6379[20]> set kk3 1 ex 10
OK
Check all keys:
127.0.0.1:6379[20]> keys *
1) "kk1"
2) "kk2"
3) "kk3"
dbsize returns 3:
127.0.0.1:6379[20]> dbsize
(integer) 3
Keys start to expire:
127.0.0.1:6379[20]> keys *
1) "kk2"
2) "kk3"
127.0.0.1:6379[20]> dbsize
(integer) 3
Now, here is the problem:
127.0.0.1:6379[20]> keys *
(empty list or set)
“keys *” already reflects that all keys expired, so there are no keys in the database. However, dbsize was not updated.
127.0.0.1:6379[20]> dbsize
(integer) 3
It takes a loooong time after “keys *” indicated empty database for dbsize to really update its value.
It broke the functionality of many working scripts we have.
I am testing it on Fedora 30 : redis-5.0.7-1.fc30.x86_64
This worked before. What changed?
Any ideas?
2
Answers
Well, I could not find a way to update dbsize.
So, I am counting. Instead of using the result of dbsize I use:
And insert totalKeys back into Redis, to use that value until next refresh.
You can try
debug object kk1
to see if the key is still in memory, even if not returned bykeys *
.I can only think of one scenario where what you described would happen:
lazyfree-lazy-expire yes
debug SET-ACTIVE-EXPIRE 0
To check
lazyfree-lazy-expire
, tryconfig get lazyfree-lazy-expire
.The second one cannot be consulted, but you do
debug SET-ACTIVE-EXPIRE 1
to ensure normal mode.