Question posted in
The official documentation can be found here.

Redis: How do I count the elements in a stream in a certain range?

Bussiness Objective

I’m creating a dashboard that will depend on some time-series and I’ll use Redis to implement it. I’m new to using Redis and I’m trying to use Redis-Streams to count the elements in a stream.

XADD conversation:9:chat_messages * id 2583 user_type Bot
XADD conversation:9:chat_messages * id 732016 user_type User
XADD conversation:9:chat_messages * id 732017 user_type Staff
XRANGE conversation:9:chat_messages - +

I’m aware that I can get the total count of the elements using the XLEN command like this:

XLEN conversation:9:chat_messages

but I want to also know the elements in a period, for example:

XLEN conversation:9:chat_messages 1579551316273 1579551321872

I know I can use LUA to count those elements but I want some REALLY fast way to achieve this and I know that using Redis markup will be the fastest way.

Is there any way to achieve this with a straight forward Redis command? Or do I have to write a Lua script to do this?

Additional information

I’m limited by AWS’ ElastiCache to use the only Redis 5.0.6, I cannot install other modules such as the RedisTimeSeries module. I’d like to use that module but it’s not possible at the moment.

Tags:

2

Answers


  1. 0

    While the Redis Stream data structure doesn’t support this, you can use a Sorted Set alongside it for keeping track of message ranges.

    Basically, for each message ID you get from XADD – e.g. “1579551316273-0” – you need to do a ZADD conversation:9:ids 0 1579551316273-0. Then, you can use ZLEXCOUNT to get the “length” of a range.

    Login or Signup to reply.
  2. 0

    Sorry, there is no commands-way to achieve this.

    Your best option with Redis Streams would be to use a Lua script. You will get O(N) with N being the number of elements being counted, instead of O(log N) if a command existed.

    local T = redis.call('XRANGE', KEYS[1], ARGV[1], ARGV[2])
local count = 0
for _ in pairs(T) do count = count + 1 end
return count

    Note the difference between O(N) and O(log(N)) is significant for a large N, but for a chat application, if tracked by conversation, this won’t make that big of a difference if chats have hundreds or even thousands of entries, once you account total command time including Round Trip Time which takes most of the time. The Lua script above removes network-payload and client-processing time.

    You can switch to sorted sets if you really want O(log N) and you don’t need consumer groups and other stream features. See How to store in Redis sorted set with server-side timestamp as score? if you want to use Redis server timestamp atomically.

    Then you can use ZCOUNT which is O(log(N)).

    If you do need Stream features, then you would need to keep the sorted set as a secondary index.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search