I have some webscraped data. I’m using redis as my server and fetching data from nodejs and displaying it in react.
This is what I have:
2 redis keys both ~ 200kb (so far, this will become ~10-20 mb in the future)
I fetch data from redis by getting both keys, parsing the data and then sending it. Just to note I’m storing data as one big string in the keys.
express:
try {
// Fetch redis data
const data1 = await GET_ASYNC('data1');
const data2 = await GET_ASYNC('data2');
// Parse data
const parse1 = JSON.parse(data1);
const parse2 = JSON.parse(data2);
const result = [...data2, ...data1];
res.send(filterResults);
} catch (err) {
//
}
The problem is that nodejs takes about ~2000ms to fetch and send the data, for my situation I need to lower this to 500ms or less. How could I do this? Is my problem in nodejs or how I’m storing data in redis? Thanks.
2
Answers
My suggestion would be to go for smaller keys instead of just 2 large keys. This will have few advantages i.e.
My suggestions: