Question posted in
The official documentation can be found here.

How do I get records in a Redis stream by position instead of ID?

With XREAD, I can get count from a specific ID (or first or last). With XRANGE, I can get a range from key to key (or from first - or to last +). Same in reverse with XREVRANGE.

How do I get by position? E.g. "give me the last 10 records in order"? XREVRANGE can do it with XREVRANGE stream + - COUNT 10, although it will be backwards in order, and XRANGE will give me the first 10 with XRANGE stream - + COUNT 10, but how do I:

  • get the last X in order?
  • get an arbitrary X in the middle, e.g. the 10 from position 100-109 (inclusive), when the stream has 5000 records in it?
Tags:

2

Answers


  1. 0

    Redis Streams currently (v6.0) do not provide an API for accessing their records via position/offset. The Stream’s main data structure (a radix tree) can’t provide this type of functionality efficiently.

    This is doable with an additional index, for example a Sorted Set of the ids. A similar discussion is at https://groups.google.com/g/redis-db/c/bT00XGMq4DM/m/lWDzFa3zBQAJ

    The real question is, as @sonus21 had asked, what’s the use case.

    Login or Signup to reply.
  2. 0

    It is an existing API that allows users to retrieve records from index
    x to index y. It has a memory backend (just an array of records) and a
    file one, want to put Redis in it as well.

    We can just use Redis List to get the items randomly and at any offset using LRANGE

    Attaching consumers to Redis LIST is easy but it does not come with an ack mechanism like a stream, you can build an ack mechanism in your code but that could be tricky. There could be many ways to build an ack mechanism in Redis, see one of them here Rqueue.

    Ack mechanism is not always required but if you just want to consume from the LIST, you can use BLPOP/LPOP commands to consume elements from the list, that could be your simple consumer, but using BLPOP/LPOP commands would remove entries from the LIST so your offset would be dynamic, it will not always start from the 0. Instead of using BLPOP/LPOP, if you use LRANGE and track the offset in a simple key like my-consumer-offset, using this offset you can build your consumer that would always get the next element from the list based on the current offset.

    Using LIST and STREAM to get random offset and streaming feature. Most important part would be you should use Consumer group so that streams are not trimmed. During push operation you should add element to STREAM as well as LIST. Once elements are inserted your consumer can work without any issue in a consumer group, for random offset you use LIST to get elements. Stream can grow as it’d be in append only mode, so you would need to periodically trim stream. See my other answer for deleting older entries

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search