Question posted in
The official documentation can be found here.

Redis-py hset() mapping – TypeError when setting multiple item values

I am building some very large lookup tables in Redis. I have some straightforward code that works as intended when looping through a dict to set a single value in my Redis hash (via a pipeline) for each item using hset():

foo = {"1234": "5678", "abcd": "efgh", ... }

with self.db.pipeline() as pipe:
    for foo in bar:
        pipe.hset("lookup_table", foo["key"], foo["value"])
pipe.execute()

This is slow with a large dict. To speed it up, I want to be able to set multiple items as a mapping into the pipeline without having to loop over it. With hmset() now deprecated, it seems that hset() can accept a mapping via a keyword arg. I have attempted to do the following:

with self.db.pipeline() as pipe:    
    pipe.hset("lookup_table", mapping=foo)
pipe.execute()

but this yields the error TypeError: hset() got an unexpected keyword argument 'mapping'. Am I using hset() incorrectly? Or am I mistaken in thinking that hset() can accept multiple items in this way?

I’m using py-redis 3.4.1 with Python 3.7.5.

Tags:

3

Answers


  1. 0

    This appears to be a known issue as shown here –> https://github.com/andymccurdy/redis-py/issues/1310#issuecomment-603081122.

    As you can see in that image linked, the source code in PyPi has hset with a function signature that does not include the keyword mapping. You should verify in your installation of py-redis that the same issue is present and follow that ticket as well. To work around it you can clone straight from master branch in order to use that feature.

    Login or Signup to reply.
  2. 0

    Updating with

    pip install -U redis

    solved the issue for me.

    Login or Signup to reply.
  3. 0

    For anyone coming later to find a better answer and example.

    # test mapping to add a full dict
foo = {"1234": "5678", "abcd": "efgh" }
queueLib.redis.hset("queues_data2", mapping=foo)
# works

import json
foo2 = {"1234": "5678", "abcd": {} }
queueLib.redis.hset("queues_data2", mapping=foo2)
# fails with
# redis.exceptions.DataError: Invalid input of type: 'dict'. Convert to a bytes, string, int or float first.

# if you try via serialization
pdict_string = json.dumps(foo2)
queueLib.redis.hset("queues_data2", mapping=pdict_string)
# fails with 
# AttributeError: 'str' object has no attribute 'items'

# now try 
update_nested_dict = json.dumps(foo2['abcd'])
foo2['abcd'] = update_nested_dict
queueLib.redis.hset("queues_data2", mapping=foo2)

# you can also loop through each value of the dict and serialize it with something like:

foo3 = {"1234": {}, "abcd": {} }
{k: json.dumps(v) for k,v in foo3.items()}
# {'1234': '{}', 'abcd': '{}'}

    Also covered more on handling dict with redis here

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search