skip to Main Content

I have a timeseries I want to compute a rolling sum of. My idea (which is probably suboptimal already) is to have a script to compute each data point then run it from python in a pipeline.

I struggle with the script.

return 
redis.call('TS.ADD',
           KEYS[1],
            ARGV[1],
            redis.call('TS.RANGE',
                       'ts:1s',
                        ARGV[2],
                        ARGV[3],
                        'AGGREGATION',
                        'sum',
                        120000)[1][2])

When I EVAL this on redis-cli, with args 1 ts:rolling:sum 1609011855000 1609013655000

it returns:
(error) ERR Error running script (call to f_e57a3287fc55b792d33ba7c21e3aae715c5ee3e5): @user_script:1: @user_script: 1: Lua redis() command arguments must be strings or integers

the first call

eval "return redis.call('TS.RANGE',
            'ts:1s',
             ARGV[2],
             ARGV[3],
             'AGGREGATION',
             'sum',
             120000)" 0 1609011855000 1609013655000

gives me

1) 1) (integer) 1609012800000
   2) 2057

Strange thing is, I can sum the timestamps but not the values.

I have never touched lua before, so I may be missing something totally obvious here

I’m not sure I’m being clear so here’s in python, with pandas, what I’d like to achieve:

import pandas as pd
df = pd.DataFrame(my_data_granular_to_1s, columns=['date','val']).set_index(date)
df.val.resample('1s').sum().rolling(120).sum() # <= that

2

Answers


  1. Chosen as BEST ANSWER

    It turns out

    1) 1) (integer) 1609012800000
       2) 2057
    

    2057 in accessible in lua via v[1][2]["ok"]

    From my "I just touched lua for the first time today" perspective, looks pretty much like a bug. I'll familiarize a bit more to be sure I don't miss something obvious before opening an issue on github to inform the team


  2. Why

    2057 in accessible in lua via v[1][2]["ok"]

    is quite clear, if you are using built-in debugger.

    Here’s an example:

    -- test.lua
    
    redis.pcall("TS.ADD", KEYS[1], "*", 1, "ON_DUPLICATE", "SUM")
    
    local reply = redis.call("TS.RANGE", KEYS[1], "-", "+")
    redis.debug(reply)
    
    redis.debug(reply[1][2]["ok"])
    
    redis-cli -h your-host -p your-port --ldb --eval ./test.lua test:key 
    
    -> 2   redis.pcall("TS.ADD", KEYS[1], "*", 1, "ON_DUPLICATE", "SUM")
    lua debugger> s
    <redis> TS.ADD test:key * 1 ON_DUPLICATE SUM
    <reply> 1655798188752
    * Stopped at 4, stop reason = step over
    -> 4   local reply = redis.call("TS.RANGE", KEYS[1], "-", "+")
    lua debugger> s
    <redis> TS.RANGE test:key - +
    <reply> [[1655798188752,"+1"]]
    * Stopped at 5, stop reason = step over
    -> 5   redis.debug(reply)
    lua debugger> s
    <debug> line 5: {{1.6558e+12; {["ok"]="1"}}}
    * Stopped at 8, stop reason = step over
    -> 8   redis.debug(reply[1][2]["ok"])
    lua debugger> s
    <debug> line 8: "1"
    * Stopped at 9, stop reason = step over
    -> 9   <out of range source code line>
    lua debugger> s
    
    (nil)
    
    (Lua debugging session ended -- dataset changes rolled back)
    

    As you can see TS.RANGE returns a nested table:

    {{1.6558e+12; {["ok"]="1"}}}
    

    So to get its result I have to use

    reply[1][2]["ok"]
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search