Question 121086 in Redis Question posted in
The official documentation can be found here.

How to send vectors to Redis cache for vector similarity?

Im trying to implement vector search similarity in c#. I have to use the HSET command to send vectors to my index in Redis. The hset command is something like: HSET item:3 "x00x00x00x00". When I test this, I find out that the string I add which is "x00x00x00x00" gets added as ""\x00\x00\x00\x00". Redis does not remove the extra backslash and thus my searching fails later on.

   public static async Task CreateIndexAsync()
    {
        try
        {
            await mux.GetDatabase().ExecuteAsync("FT.CREATE", "embeddings", "ON", "HASH", "PREFIX", "1", "item:", "SCHEMA", "vector", "VECTOR", "FLAT", "6", "TYPE", "FLOAT32", "DIM", "2", "DISTANCE_METRIC", "COSINE");
        }
        catch (Exception)
        {
            //swallow exception if index exists
        }
    }`

    public static async Task AddAsync(string docId, string prefix, float[] vector)
    {
        string hex = toHexString(vector);
         mux.GetDatabase().Execute("Hset", $"{prefix}{docId}", "vector", hex);
    }`

I dont know how to solve this issue, it seems that Redis does not have good c# support

Tags:

2

Answers


  1. 0

    you are more than welcome to use NRedisStack, A .NET client (builds on StackExchange.Redis) and supports Redis Stack commands.
    NRedisStack has support for vector search similarity commands.

    Your code will look like that with NRedisStack:

    public static async Task CreateIndexAsync()
{
    try
    {
         await ft.CreateAsync("embeddings",
         new FTCreateParams()
         .On(IndexDataType.Hash)
         .Prefix("item:"),
         new Schema()
         .AddVectorField("vector",
         VectorField.VectorAlgo.FLAT,
         new Dictionary<string, object>()
         {
             ["TYPE"] = "FLOAT32",
             ["DIM"] = "2",
             ["DISTANCE_METRIC"] = "COSINE"
         }));
       }
       catch (Exception)
       {
           //swallow exception if index exists
       }
   }

public static async Task AddAsync(string docId, string prefix, float[] vector)
{
    db.HashSet($"{prefix}{docId}", "vector", toHexString(vector));
}

    I tried to reproduce the problem you described, while using NRedisStack, and this line worked fine:

    var res = db.HashSet("item", "vector", "x00x00x00x00");

    this is the monitor output:

    ~"FT.CREATE" "embeddings" "PREFIX" "1" "item:" "SCHEMA" "vector" "VECTOR" "FLAT" "6" "TYPE" "FLOAT32" "DIM" "2" "DISTANCE_METRIC" "COSINE"
~"HSET" "item:3" "vector" "x00x00x00x00"

    Links:
    NRedisStack – link to github |
    NRedisStack – link to NuGet

    Login or Signup to reply.
  2. 0

    Just to make it clear from the answer of Shachar Pashchur. You don’t have to translate it to an escaped hexadecimal sequence.

    You can just pass a flattened byte array of float or double values into the hashset.

    public static async Task CreateIndexAsync()
{
    try
    {
         await ft.CreateAsync("embeddings",
         new FTCreateParams()
         .On(IndexDataType.Hash)
         .Prefix("item:"),
         new Schema()
         .AddVectorField("vector",
         VectorField.VectorAlgo.FLAT,
         new Dictionary<string, object>()
         {
             ["TYPE"] = "FLOAT32",
             ["DIM"] = "2",
             ["DISTANCE_METRIC"] = "COSINE"
         }));
       }
       catch (Exception)
       {
           //swallow exception if index exists
       }
   }

public static async Task AddAsync(string docId, string prefix, float[] vector)
{
    db.HashSet($"{prefix}{docId}", "vector", vector.SelectMany(BitConverter.GetBytes).ToArray());
}

    If you are going to pass in an array of double values, remember to change the VectorField TYPE to FLOAT64

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search