Trying to write a query from the following table that calculates a count of records for every day date(timestamp) and the prior 4 days. So basically a rolling count of records for every previous 5 days. Every time i do the calculation returns slightly off.
Symbol | timestamp | high | number | date(timestamp) | date(timestamp – interval 1 day) | date(timestamp – interval 2 day) | date(timestamp – interval 3 day) | date(timestamp – interval 4 day) |
---|---|---|---|---|---|---|---|---|
SPY | 2021-04-26 04:00:00+00:00 | 416.97 | 1 | 2021-04-26 | 2021-04-25 | 2021-04-24 | 2021-04-23 | 2021-04-22 |
SPY | 2021-04-26 06:20:00+00:00 | 416.91 | 2 | 2021-04-26 | 2021-04-25 | 2021-04-24 | 2021-04-23 | 2021-04-22 |
SPY | 2021-04-26 08:00:00+00:00 | 416.84 | 3 | 2021-04-26 | 2021-04-25 | 2021-04-24 | 2021-04-23 | 2021-04-22 |
SPY | 2021-04-26 08:05:00+00:00 | 416.8 | 4 | 2021-04-26 | 2021-04-25 | 2021-04-24 | 2021-04-23 | 2021-04-22 |
SPY | 2021-04-26 08:10:00+00:00 | 416.81 | 5 | 2021-04-26 | 2021-04-25 | 2021-04-24 | 2021-04-23 | 2021-04-22 |
SPY | 2021-04-26 08:15:00+00:00 | 416.78 | 6 | 2021-04-26 | 2021-04-25 | 2021-04-24 | 2021-04-23 | 2021-04-22 |
SPY | 2021-04-26 08:20:00+00:00 | 416.75 | 7 | 2021-04-26 | 2021-04-25 | 2021-04-24 | 2021-04-23 | 2021-04-22 |
SPY | 2021-04-26 08:25:00+00:00 | 416.54 | 8 | 2021-04-26 | 2021-04-25 | 2021-04-24 | 2021-04-23 | 2021-04-22 |
SPY | 2021-04-26 08:30:00+00:00 | 416.51 | 9 | 2021-04-26 | 2021-04-25 | 2021-04-24 | 2021-04-23 | 2021-04-22 |
SPY | 2021-04-26 08:35:00+00:00 | 416.34 | 10 | 2021-04-26 | 2021-04-25 | 2021-04-24 | 2021-04-23 | 2021-04-22 |
SPY | 2021-04-26 08:40:00+00:00 | 416.33 | 11 | 2021-04-26 | 2021-04-25 | 2021-04-24 | 2021-04-23 | 2021-04-22 |
The following query returns the counts but they are slightly off. For example of 2021-4-30 the count should match the max(number), the enumerated rows but it does not. Please help
select date(t.timestamp),date(t.timestamp - interval 4 day),
(count(t.timestamp)+count(t.timestamp - interval 4 day)+count(t.timestamp - interval 3 day)+count(t.timestamp - interval 2 day)+count(t.timestamp - interval 1 day)) as cntrecords,
max(number), min(number)
from
(SELECT Symbol,
timestamp,
high,
ROW_NUMBER() OVER (ORDER BY timestamp) AS number,
date(timestamp),
date(timestamp - interval 4 day)
#BETWEEN DATE_SUB(date(timestamp), INTERVAL 4 DAY) AND date(timestamp)
FROM test.rawdata
#WHERE date(timestamp) BETWEEN DATE_SUB('2021-04-30', INTERVAL 4 DAY) AND '2021-04-30'
#group by date(timestamp)
order by timestamp) as t
group by date(t.timestamp);
2
Answers
We will start by getting the counts per symbol per day with a simple GROUPed query. If you want the results for the week commencing 26th April 2021 you need to remember to include the four days prior to the start of the desired date range –
Now we can use SUM as a window function with a 5 day frame to get your counts –
The above query will return the leading four rows which we can remove by adding another level of nesting (we cannot use a HAVING clause as the filter would be applied before the SELECT list is evaluated) –
As an alternative approach, you could use a recursive cte to build a list of ranges to join and group by –
This was not stated in the question, but I’m guessing that columns like
number
anddate(timestamp - interval 3 day)
were added to try and help solve this problem, or are results of an intermediate query. Either way, they are unnecessary.Let’s solve this problem for a table which only has
symbol
,timestamp
, andhigh
, using a window function with frame specification:Note that I’m using
RANGE
instead ofROW
because if there are days for which there are no data, that would throw off the counts.Also note that this will not work in MariaDb which does not yet support RANGE expressions with TIME like fields and intervals.
You can play around with this example in this DB Fiddle:
https://www.db-fiddle.com/f/ctN927ouAMHrWK1QJQD6Z2/0
(Note I only used 1 day in the range there so I didn’t have to generate as much test data.)