Question 121022 in Redis Question posted in
The official documentation can be found here.

Issue, iteration cross KV (NoSql/Redis) Target

I would like to iterated cross KV (NoSql,Redis) Target in MLRun without specification of primary key(s) also, but I saw only ability to get specific KV item(s) based on specific key(s). I have to use only unique items, not duplicit items.

I chose NoSql/Redis Target because key operations in-memory are critical for quick responce (based on specific key), but from time to time I have to iterate cross whole collection of keys (and it is case of this question).

You can see part of my code for getting values based on keys:

import mlrun 
import mlrun.feature_store as fs
...

svc=fs.get_online_feature_service(fs.FeatureVector("myVector", ["client.*"]))
resp = svc.get([{"cuid": 1926, "key1": 100023}])

Do you know, how to iterate cross items in KV (NoSql, Redis) target in MLRun CE/Paid editions (version 1.2.1)?

Tags:

2

Answers


  1. 0

    The NoSqlTarget is using a KV store which is inherently for retrieving an individual value based on a given key. If you are looking to retrieve the entirety of your feature vector, you should use the offline store instead. More info in the MLRun docs here

    Alternatively, you could use the offline store to get a list of keys and then query the online store using those:

    import mlrun.feature_store as fstore

df = fstore.get_offline_features("my-vec").to_dataframe()
keys = list(resp.to_dataframe().index.unique())

svc = fstore.get_online_feature_service("my-vec")
svc.get([{"my-key" : key} for key in keys[:5]])
    Login or Signup to reply.
  2. 0

    I am using these solutions for iteration cross keys in KV storage:

    1. NoSqlTarget

    Iteration via v3io (it is not pure part of MLRun API, but it is part of MLRun distribution packages). More information about v3io python SDK see and iteration cross KV items (cursor usage) see sample

    import v3io.dataplane

v3io_client = v3io.dataplane.Client(endpoint='https://v3io-webapi:8081', access_key='some_access_key')

# create a query, and use an items cursor to iterate the results
items_cursor = v3io_client.kv.new_cursor(container='users',
                                         table_path='/user-profile',
                                         attribute_names=['income'],
                                         filter_expression='income > 150')

# print the output
for item in items_cursor.all():
    print(item)

    BTW: NoSqlTarget is available only for MLRun Enterprise edition

    2. RedisTarget

    You can use easy iteration cross KV items, it is part of Redis API

    import redis

r = redis.StrictRedis(host='localhost', port=6379, db=0)
for key in r.keys('*'):
    r.delete(key)

    It is possible to use commandline also via redis-cli see sample:

    redis-cli keys users*

    or remove from redis specific keys based on list of keys:

    redis-cli keys users* | xargs redis-cli del

    BTW: RedisTarget is available for MLRun CE and Enterprise editions

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search