We are looking to implement a redis based cache for read heavy data for fronting our database as a read through cache. I would like to implement a better invalidation mechanism than just TTL or LRU based eviction to prevent stale reads as much as possible.
Several databases provide notification mechanism for database objects such as tables. For example oracle has Change Notifications and Postgresql has NOTIFY for this purpose. Is there any existing open source project/component that listens to these notifications and uses them to invalidate out of process caches like redis or memcached? I have seen several projects for doing this to in-process caches but none so far for out of process (either clustered/unclustered) caches.
2
Answers
debezium is a component that implements the whole pipeline from using CDC from the database to publishing those changes in a format you prefer.
Redis Labs announced their new "RedisCDC" solution at RedisConf 2021 which seamlessly migrates data from heterogeneous data sources to Redis and Redis Modules. Its configurable and extendable, so you can easily create a custom stage that invalidates Redis keys when there is an update or delete on the source side.