skip to Main Content

I have this piece of code. I make an writeRecords request and then i do it again.
The next request i put in an array and use an Promise.all()

My question is:

What if requests.push(writeClient.writeRecords(timestreamParams).promise()) has an error, will it be an unhadled Promise error, or will the try / catch catch the error?
Because its in an loop it can be that the first element in the array has already sent back an response while the other data is still in the loop so i am not sure where the error will occur?

let requests = []
  const chunkSize = 100
  for (let i = 0; i < Records.length; i += chunkSize) {
    const chunk = Records.slice(i, i + chunkSize)
    const timestreamParams = {
      DatabaseName: db_name,
      TableName: TimeSpanRawE[table_name],
      Records: chunk,
    }
    await writeClient
      .writeRecords(timestreamParams)
      .promise()
      .catch(() => {}) // Silent error

    // Successfully retry same writeRecordsRequest with same records and versions, because writeRecords API is idempotent.

    requests.push(writeClient.writeRecords(timestreamParams).promise())
  }

  try {
    return Promise.all(requests).then((res) => ok(res))
  } catch (error) {
    return err({ error, params })
  }

2

Answers


  1. Yes, your code can lead to unhandled promise rejections, if one of the promises in the requests array rejects before the Promise.all is reached (due to the loop being suspended on the await). It cannot happen if the loop were run synchronously.

    You should be using either the concurrent version

    try {
      const requests = []
      const chunkSize = 100
      for (let i = 0; i < Records.length; i += chunkSize) {
        const chunk = Records.slice(i, i + chunkSize)
        const timestreamParams = {
          DatabaseName: db_name,
          TableName: TimeSpanRawE[table_name],
          Records: chunk,
        }
        requests.push(writeClient.writeRecords(timestreamParams).promise())
      }
    
      const res = await Promise.all(requests)
      ok(res)
    } catch (error) {
      return err({ error, params })
    }
    

    or the sequential version

    try {
      const res = []
      const chunkSize = 100
      for (let i = 0; i < Records.length; i += chunkSize) {
        const chunk = Records.slice(i, i + chunkSize)
        const timestreamParams = {
          DatabaseName: db_name,
          TableName: TimeSpanRawE[table_name],
          Records: chunk,
        }
        res.push(await writeClient.writeRecords(timestreamParams).promise())
      }
    
      ok(res)
    } catch (error) {
      return err({ error, params })
    }
    

    but not the mix between them. If you really needed to, the .catch(() => {}) // Silence error needs to go on the promise that you are not awaiting immediately (i.e. the ones you put in the array).

    Login or Signup to reply.
  2. I would put the retry logic in a separate function to make some separation of concerns. My usual way is to add to the Promise: Promise.retry.

    Also your requests are independent from each other so you could execute them in parallel. We use Array::map() for that to get an array of promices

    async function retry(func, numberOfRetries = 1) {
        let error;
        do {
            try {
                return await func();
            } catch (e) {
                // some error logic would be nice
                error = e;
            }
        } while (--numberOfRetries);
        throw error;
    }
    
    const chunkSize = 100;
    
    const requests = Records.reduce((chunks, item, index) =>
        ((chunks[Math.floor(index / chunkSize)] ??= []).push(item)) && chunks
        , [])
        .map(chunk => retry(writeRecords({
            DatabaseName: db_name,
            TableName: TimeSpanRawE[table_name],
            Records: chunk,
        }).promise()));
    
    try {
        return Promise.all(requests).then((res) => ok(res))
    } catch (error) {
        return err({ error, params })
    }
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search