Question 234971 in Redis Question posted in
The official documentation can be found here.

How to migrate Redis RDB file created with a module to a deployment without that module?

When I try to load a Redis database with Redis Stack 7.2 from an RDB file which has been created with a prior version of RS, it does not load, printing this error:

* Loading RDB produced by version 6.2.13
* RDB age 848231 seconds
* RDB memory usage when created 1.50 Mb
# The RDB file contains AUX module data I can't load: no matching module 'graphdata'

This is likely due to Redis phasing out RedisGraph, the article about that mentions RG "is no longer a part of Redis Stack from version 7.2".

I was assuming it’s all fine, as I have never used any graph related functionality. However, the RDB file seemingly references the RedisGraph module despite not having any entries related to it, which I see no logical reason for.

What is the intended way for users to keep their Redis Stack version up-to-date, if they need RDB persistence? Seems like version 7.2 will simply not accept any RDB file from any previous version.

Is there a simple way to migrate it and keep using this file?

Tags:

2

Answers


  1. 0

    This got me too. In particular, I’ve been using redis-stack for a little project locally and the Homebrew cask updated from redis-stack-server 6.2.6-v9 to 7.2.0 a week ago (commit).

    The reason we can’t load our RDBs, even though we never used RedisGraph, is that loading the module saved metadata to the RDB in AUX fields (custom data types?), which the new version of Redis can’t read without the module. This also disallows you from unloading the module at runtime with MODULE UNLOAD.

    This is wack and they’re talking through a fix in pull requests #11056 and now #11374.

    Anyway, a basic strategy for rescuing your data is to load it up in the previous working configuration, export it as a plaintext AOF log in a way that doesn’t include the offending references to the old module, upgrade to 7.2.0 again—having deleted the offending RDB file—and load in the AOF we generated. SAVE and you’ve got a fresh RDB.

    I’m working locally with about 5 MB of data and there’s no production instance to keep available. This approach assumes there are no clients connecting and trying to modify data except for you. So here’s a basic outline, but modify for your situation.

    1. Make a backup of your RDB file before you do anything else, so you have a safety net if these steps don’t work precisely on the first try. For me, with Homebrew, I just did e.g.
      cp /opt/homebrew/var/db/redis-stack/dump.rdb ~/Desktop/dump.orig.rdb

    2. Downgrade back to 6.2.x. Again, this will depend on your OS and configuration. For Homebrew, you can attempt to downgrade a cask by downloading the version of the formula you want and installing it from disk. I downloaded the previous version of the cask formula for redis-stack-server.rb, then
      brew install --cask ~/Downloads/redis-stack-server.rb

    3. Try starting redis-stack-server. You should be back.

    4. We can see those problematic references to the module by inspecting the RDB file, btw:

      $ redis-check-rdb /opt/homebrew/var/db/redis-stack/dump.rdb
[offset 0] Checking RDB file dump.rdb
[offset 27] AUX FIELD redis-ver = '6.2.13'
[offset 41] AUX FIELD redis-bits = '64'
[offset 53] AUX FIELD ctime = '1692437664'
[offset 68] AUX FIELD used-mem = '5479248'
[offset 84] AUX FIELD aof-preamble = '0'
[offset 96] MODULE AUX for: graphdata
[offset 111] MODULE AUX for: scdtype00
[offset 124] MODULE AUX for: ft_index0
[offset 129] Selecting DB ID 2
[offset 2666081] MODULE AUX for: graphdata
[offset 2666096] MODULE AUX for: scdtype00
[offset 2666108] Checksum OK
[offset 2666108] o/ RDB looks OK! o/
[info] 4713 keys read
[info] 0 expires
[info] 0 already expired

    5. Export your data.

      $ redis-cli
127.0.0.1:6379> CONFIG SET aof-use-rdb-preamble no
OK
127.0.0.1:6379> CONFIG SET appendonly yes
OK
127.0.0.1:6379> BGREWRITEAOF
Background append only file rewriting started

      Check the server log for that to finish, then disconnect the client and stop the server.

    6. Delete the active RDB file.
      rm /opt/homebrew/var/db/redis-stack/dump.rdb
      There should be a larger appendonly.aof alongside it now. You can move that somewhere safe.
      mv /opt/homebrew/var/db/redis-stack/appendonly.aof ~/Desktop/appendonly.6.2.aof

    7. Upgrade back to 7.2.0. For me, using Homebrew, that’s just
      brew upgrade redis-stack-server.

    8. Finally, start 7.2.0 without any data (since we deleted/moved it all away)
      redis-stack-server
      And replay the log:
      redis-cli --pipe < ~/Desktop/appendonly.6.2.aof

    You can now issue a SAVE or BGSAVE to generate a new RDB file.

    I hope this helps! I haven’t been in charge of a production Redis instance in a decade or so, and I just chose Redis Stack for the JSON support, because I’m caching entire responses from a JSON API and thought it might be handy. But this issue with module interoperability makes me wonder whether I should just stick with vanilla Redis for now.

    By the way, Redis 7.2.0 says this on startup for me:

    71950:M 21 Aug 2023 04:07:05.933 * <redisgears_2> Created new data type 'GearsType'
71950:M 21 Aug 2023 04:07:05.934 * <redisgears_2> Detected redis oss
71950:M 21 Aug 2023 04:07:05.934 # <redisgears_2> could not initialize RedisAI_InitError

71950:M 21 Aug 2023 04:07:05.934 * <redisgears_2> Failed loading RedisAI API.
71950:M 21 Aug 2023 04:07:05.934 * <redisgears_2> RedisGears v2.0.11, sha='0aa55951836750ceabd9733decb200f8a5e7bac3', build_type='release', built_for='Macos-mac os12.6.3.arm64v8'.
71950:M 21 Aug 2023 04:07:05.935 * <redisgears_2> Registered backend: js.
71950:M 21 Aug 2023 04:07:05.935 * Module 'redisgears_2' loaded from /opt/homebrew/Caskroom/redis-stack-server/7.2.0-v0/lib/redisgears.so

    So… I assume that’s fine. 🙂

    Login or Signup to reply.
  2. 0

    I’m from Redis. We are sorry for this. We will fix it soon (the target date is set for next week).

    We plan to release a new dummy module – RedisGraphStub. This module will be minimalistic. It will support no commands and will ignore graph data upon loading. If there is an empty graph AUX field – the module will ignore it and continue. If there are graph keys – the module will fail (users with graph keys would have to delete them manually before they can upgrade to Redis Stack 7.x. This will ensure graph data won’t be deleted unintentionally).

    We will also release a patch for Redis Stack 7.2 that will include the RedisGraphStub module.

    If you are using Redis with RedisGraph and would like to remove RedisGraph because of the end-of-life but still want to keep all your other data, you would be able to add the RedisGraphSub module manually.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search