skip to Main Content

My Question/ Expectation is to find a possible point or statment that could cause such problems. If you have experienced something similar in JS, where it does go through the whole Function but only calculations one part, and have a solution that would still help?

I’m currently fixing some stuff in my code of my web worker and broke something and its not throwing an error, what’s weird is that it only does a bit of the calculation (only the stuff for CandleSeriesArray). I have no clue what to do since logging the data that’s send is correct which is also confirmed by the CandleSeriesArray:

First some information on how the code works the part with the calculation is further down:
In Worker file – Worker receive and send

let testCounter = 0;
let counter2 = 0;
let preparationCounter = 0;
//worker.postMessage({chunk: chunk, index: i, preparationChunk: preparationChunk, changer: changer});
self.onmessage = function (e) {
    const { chunk, index, preparationChunk, changer } = e.data;

    preparationChunk.forEach(line => {
        //var message = line;//JSON.parse(line);
        if (line.m === false) {
            processCandleData(line.E, line, false, changer)
        };
        preparationCounter++;
    });
    // Iterate over each line and parse as JSON
    chunk.forEach(line => {
        //var message = line;//JSON.parse(line);
        if (line.m === false) {
            calculateKline = processCandleData(line.E, line, true, changer)
            testCounter++;
        };
        counter2++;
    });

    // Send the processed result back to the main script
    self.postMessage({ index: index, result: calculateKline, counter: counter2, preparationCounter: preparationCounter , testCounter: testCounter });
};

First the data received:

worker.postMessage({chunk: chunk, index: i, preparationChunk: preparationChunk, changer: changer});

chunk is an array that consist of many lines in this format, which is the same for the preparationChunk. This is then send to the function processCandleData(line.E, line, false, changer), line is the placeholder for both data Chunks.

{m: false, E: 1716823048658, k: {…}}
1. E: 1716823048658
2. k:
  1. T: 1716823079999
  2. c: "69968"
  3. i: "1m"
  4. o: 69968
  5. s: "BTCFUSDT"
  6. t: 1716823020000
3. m: false

index is the number of the worker, changer has the value 5 and is needed for the calculation of the SMA value in the processCandleData function under the comment: //Update the data for the SMA series

the following part:

self.postMessage({ index: index, result: calculateKline, counter: counter2, preparationCounter: preparationCounter , testCounter: testCounter

sends it back to the main script result: calculateKline looks in the console like this
(result after two worker were done):

{CandleSeriesArray: Array(26034), SmaSeriesArray: Array(0), CloseValueSeriesArray: Array(0)}
{CandleSeriesArray: Array(26842), SmaSeriesArray: Array(0), CloseValueSeriesArray: Array(1)}

This shows me that the arrays for SmaSeriesArray and CloseValueSeriesArray are missing. What it should actual look like is something like this: (numbers are made up)

{CandleSeriesArray: Array(26034), SmaSeriesArray: Array(10304), CloseValueSeriesArray: Array(234)}
{CandleSeriesArray: Array(26842), SmaSeriesArray: Array(10604), CloseValueSeriesArray: Array(236)}

this is done by the processCandleData return which is then but in the result value of the send back message of the worker

First I wanted to check one of the 2 call function that don’t work. The first part works fine:

if (ChunkProcessing === true) { // === true not needed but it's there for clarity
    const CandleSeries = createCandleSeries(candlestick);
    CandleSeriesArray.push(CandleSeries);
}

But this part if (PreviousCandlestick && PreviousCandlestick.T < candlestick.t) (located at around line 39) doesn’t work but it must be true since PreviousCandlestick is defined and T is always a full minute timestamp when t is higher then that minute then the statement is true. My data has enough for multiple hours so that means that it must at one point be higher then a previous full minute.

combinedMessageArrayLength: (5) 

[26034, 37019, 36698, 41044, 37896]

Calculation

// I do not know if those are all arrays in the function, but in my code all arrays are defined
let CandleSeriesArray = []; 
let SmaSeriesArray = []; 
let CloseValueSeriesArray = []; 
let MessageArray = []; 
let LastCandlestickArray = []; 
let closeValuesArray = [];


//-- CALCULATION --//
function createCandleSeries(candlestick) {
        return {
            time: candlestick.t / 1000,
            open: candlestick.o,
            high: candlestick.h,
            low: candlestick.l,
            close: candlestick.c
    };};
    
    function createCloseValueSeries(LastCandlestick, CloseValue) {
        return {
            time: LastCandlestick.t / 1000,
            value: CloseValue
    };};
    
    function createSmaSeries(LastCandlestick, SMA, colorSMA) {
        return {
            time: LastCandlestick.t / 1000,
            value: SMA,
            color: colorSMA
    };};
    
    
    function processCandleData(ActualTimestamp, message, ChunkProcessing, changer) {
        var candlestick = message.k
        if (ChunkProcessing === true) { // === true not needed but it's there for clarity
            const CandleSeries = createCandleSeries(candlestick);
            CandleSeriesArray.push(CandleSeries);
        }
        //return processedCandleData = processCandleData(klineData)
    
    
        //CLOSEVALUE OF CANDLESTICK
        MessageArray.push(candlestick)
        let PreviousCandlestick = undefined;
        if (MessageArray.length >= 2) {
            PreviousCandlestick = MessageArray[MessageArray.length - 2]
        };
        if (PreviousCandlestick && PreviousCandlestick.T < candlestick.t) {
            LastCandlestickArray.push(PreviousCandlestick)
            LastCandlestick = LastCandlestickArray[LastCandlestickArray.length - 1];
            CloseValue = LastCandlestick.c;
            closeValuesArray.push(CloseValue);
            if (ChunkProcessing === true) { // === true not needed but it's there for clarity
                const CloseValueSeries = createCloseValueSeries(LastCandlestick, CloseValue);
                CloseValueSeriesArray.push(CloseValueSeries);
            }
        };
    
        //Update the data for the SMA series
        if (closeValuesArray.length >= changer) {
            // CALCULATION ON EVERY CLOSE //
                if (closeValuesArray.length >= changer && !cal_on_every_tick) {
                lastChangerValues = closeValuesArray.slice(-changer);
                sumClose = lastChangerValues.reduce((a, b) => +a + +b);
                SMA = sumClose / changer;
                colorSMA = (CloseValue >= SMA) ? '#0000FF' : '#FFA500';
                if (ChunkProcessing === true) { // === true not needed but it's there for clarity
                    const SmaSeries = createSmaSeries(LastCandlestick, SMA, colorSMA);
                    SmaSeriesArray.push(SmaSeries);
                };
            };
            };
        return {
            CandleSeriesArray,
            SmaSeriesArray,
            CloseValueSeriesArray,
        };
    }

2

Answers


  1. Chosen as BEST ANSWER

    So I figured it out! I didn't realize when I checked the data since I only saw a minute change but didn't pay closer attention to it.

    E: 1716823048658, T: 1716823079999, t: 1716823020000 
    E: 1716823079998, T: **1716823139999**, t: 1716823020000 
    E: 1716823080001, T: 1716823139999, t: **1716823080000** 
    

    The part you mentioned checks if the previous T value is lower than the current t which shows a new minute has started (in one line T is always > than t). since the data creation code made the T value of the previous candle (second one) the same as the T value of the actual minute change (third one), so 60000 higher than it should actual be. The t value could never be higher than T. This resulted in the third function also not getting data since it is based on the second function. this is how the data should look like:

    E: 1716823048658, T: 1716823079999, t: 1716823020000 
    E: 1716823079998, T: **171682079999,** t: 1716823020000 
    E: 1716823080001, T: 1716823139999, t: 1716823080000
    

    Conclusion: always check data thoroughly! And thx for the help so far


  2. This Code part in particular:

    //CLOSEVALUE OF CANDLESTICK
    MessageArray.push(candlestick)
    let PreviousCandlestick = undefined;
    if (MessageArray.length >= 2) {
       PreviousCandlestick = MessageArray[MessageArray.length - 2]
    };
    if (PreviousCandlestick && PreviousCandlestick.T < candlestick.t) {
    

    PreviousCandlestick is undefined, you’re checking if MessageArray has more than 2 elements. Somewhere above, we have:

    var candlestick = message.k
    

    This one element was pushed into MessageArray, so maybe the array has only one element? And hence fails > 2 test, and PreviousCandlestick fails it’s if test?

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search