Javascript Question posted in
A very good W3school tutorial can be found here.

Javascript – Redis relational data cache invalidation problem

I’m facing a cache invalidation problem in Redis, particularly with relational data.

For example, I have a User object:

{
    "id": 1,
    "name": "John"
}

And a UserClass object:

{
    "user_id": 1,
    "class_id": 3
}

I have an API that retrieves all users by class_id, and I store this data in Redis using the key users:class:3, which contains an array of users. I also have an API that retrieves a user by ID using the key user:1.

The problem arises when someone updates, inserts, or deletes user data. I need to remove all corresponding user data from the cache. Removing the user by the ID key is straightforward, but to invalidate the cache for the user by class, I currently have to query the database to get all classes the user is associated with and then remove the corresponding keys in Redis one by one using class_id.

This approach doesn’t seem optimal, especially if the user has other relationships beyond classes. Each time, I would need to query the database again to find the corresponding keys, leading to a lot of operations. Do you have any suggestions on how to solve this problem more efficiently?

Tags:

2

Answers


  1. 0

    There are various options i can think of:

    Option-1:
    Instead of querying the database to find all related keys, you can maintain sets in Redis that track these relationships. For example:

    • When you store a user by class_id, also store the key users:class:3 in a set like user:1:classes.
    • This set will have all the corresponding class keys if the user is associated with multiple classes.
    • On updates or deletes, you can quickly retrieve all related class keys from this set and invalidate them in a single operation without hitting the database.

    Option-2

    In addition to storing the data, maintain a reverse mapping for cache invalidation. For example:

    • When you cache users:class:3, also create a reverse mapping user:1:relatedKeys, which keeps track of all cache keys that contain data related to this user.
    • When the user data changes, you can quickly look up user:1:relatedKeys and invalidate all associated keys without querying the database.

    There are other options, too, depending on the application and environment. For example, the SCAN command can scan with a pattern match(users:class:*) to find and delete related keys; however, this would be highly non-performant. We can also use Redis pipelines or multi-exec commands to invalidate multiple keys in one go.

    Login or Signup to reply.
  2. 0

    Not sure if you are using Redis client-side caching or not when referring to invalidation, but the data modeling problem can be approached as follows.

    You want the ability to classify users by class_id, and retrieve data by class_id or using additional relations.

    Your approach is manual secondary indexing, which is inefficient due to the effort to maintain relationships, make scans, serialize and deserialize data in the string.

    I suggest using the Redis Stack capabilities (which will also be available in the standard Redis Community Edition 8).

    Then you would create an index as follows:

    FT.CREATE user_idx ON HASH PREFIX 1 user: SCHEMA class_id AS class_id TAG attributes AS attributes TAG SEPARATOR ","

    Insert data with:

    HSET user:1 id 1 class_id 3 name "John" attributes "class1,class2"
HSET user:2 id 2 class_id 7 name "Mary" attributes "class1,class25"
HSET user:3 id 3 class_id 7 name "Ted" attributes "class2"
HSET user:4 id 4 class_id 7 name "Shawn" attributes "class1,class32"
HSET user:5 id 5 class_id 3 name "Mike" attributes "class12,class13"

    And filter by class or attribute

    127.0.0.1:6379> FT.SEARCH user_idx '@class_id:{3}' RETURN 1 name
1) (integer) 2
2) "user:1"
3) 1) "name"
    2) "John"
4) "user:5"
5) 1) "name"
    2) "Mike"

127.0.0.1:6379> FT.SEARCH user_idx '@class_id:{7}' RETURN 1 name
1) (integer) 3
2) "user:2"
3) 1) "name"
    2) "Mary"
4) "user:3"
5) 1) "name"
    2) "Ted"
6) "user:4"
7) 1) "name"
    2) "Shawn"

127.0.0.1:6379> FT.SEARCH user_idx '@attributes:{class2}' RETURN 1 name
1) (integer) 2
2) "user:1"
3) 1) "name"
    2) "John"
4) "user:3"
5) 1) "name"
    2) "Ted"

    These commands are available also in the Redis supported client libraries. You can read more about the search and query capability here. If you’d rather user JSON to model your data, you can do that as well.

    This should help you model and search your data without any overhead or maintenance effort.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search