Question 243200 in Javascript Question posted in
A very good W3school tutorial can be found here.

how to make an async function complete before moving to the next task in typescript/javascript

I have a function (nodejs on aws Lambda) that reads all the files in a directory on s3, converting each, and merging their contents before returning to the front end.

The main function is as follows:

export async function handleFilesRequest(foundry: string, type: string): Promise<APIGatewayProxyResult> {


  let accumulatingMap: Map<string, any> = new Map<string, any>();
  try {
    accumulatingMap = await getFileList(foundry, type)
      .then((list) => {
        return list.map(async key => {
          readFile(key)
            .then(content => {
              return transformFile(content)
            })
            .then((currMap) => {
              accumulatingMap = new Map<string, any>([...accumulatingMap.entries(), ...currMap.entries()])
              return accumulatingMap;

            })
            .then((acc) => {
              // a console.log here shows that this has the complete set of values (but not in time, I guess)
              return accumulatingMap;
            })
        })

      }).catch(e => {
        return new Map<string, any>();
      }) as Map<string, any>;

    return OK.format(JSON.stringify(accumulatingMap));

  } catch (e) {
    return ServerError.format(e);
  }
}

What happens is that the variable accumulatingMap has not had time to get its values before it is returned in the line:

return OK.format(JSON.stringify(accumulatingMap));

how do I ensure that accumulatingMap has the time to complete, or is completed, before that return line is called?

I have tried variations of Promise.all and Promise.resolve, but these just seem to wrap things in further Promise’s and don’t actually halt the process in time for the Values to complete. Or, at least, it didn’t seem to work for me.

how do I make this work?

BTW: the statement from MDN web docs (The await operator is used to wait for a Promise and get its fulfillment value.) does not seem to to work, so a sub-question is: what is await really for?

The functions that this function delegates to all appear to work (doing as I expect them to do), so I have listed them here as a reference:

const readFile = async (key: string): Promise<string> => {

  if (key?.endsWith('.json')) {

    const command = new GetObjectCommand({
      Bucket: S3_BUCKET_NAME,
      Key: key,
    });
    return s3Client.send(command).then(b => {
      return b.Body?.transformToString() as Promise<string>;
    });

  }
  return ''
}

const transformFile = (content: string): Map<string, any> => {
  let objFileContents = JSON.parse(content);
  let fileResult: Map<string, any> = new Map<string, any>();

  if (objFileContents) {
    (Object.keys(objFileContents)).forEach(k => {
      fileResult.set(objFileContents[k].name, objFileContents[k])
      return fileResult;
    })

  }
  return fileResult;
}

const getFileList = async (foundry: string, type: string): Promise<string[]> => {
  const command = new ListObjectsCommand({
    Bucket: S3_BUCKET_NAME,
    Delimiter: '/',
    Prefix: `resources/${foundry}/`
  });

  const listedObjects = [];
  const data: ListObjectsCommandOutput = await s3Client.send(command);
  if (data['Contents'] && data['Contents'].length > 0) {
    for (let index = 0; index < data['Contents'].length; index++) {

      let currentKey = data['Contents'][index]['Key'];

      if (currentKey?.includes(type)) {
        listedObjects.push(currentKey);
      }
    }
  }

  return listedObjects;

}
Tags:

2

Answers


  1. 0

    The problem is that you don’t wait for promises in Array::map, map returns an array, not a promise. The second issue is that you don’t return readFile(key) promise from the map.

    try to update your code with this:

    accumulatingMap = await getFileList(foundry, type)
  .then((list) => {
    return Promise.all(list.map(async key => {
      return readFile(key)
        .then(content => {
          return transformFile(content)
        })
        .then((currMap) => {
          accumulatingMap = new Map<string, any>([...accumulatingMap.entries(), ...currMap.entries()])
          return accumulatingMap;

        })
        .then((acc) => {
          // a console.log here shows that this has the complete set of values (but not in time, I guess)
          return accumulatingMap;
        })
    })
    Login or Signup to reply.
  2. 0

    The mixed usage of await and .then is absolutely an anti-pattern, and almost definitely causing the code to behave differently to how you expect.

    I’ve rewritten the code to just use await, and to have a more direct structure:

    export async function handleFilesRequest(foundry: string, type: string): Promise<APIGatewayProxyResult> {
  try {
    const fileList = await getFileList(foundry, type);
  
    const fileMaps = await Promise.all(fileList.map(async (key) => {
      try {
        const content = await readFile(key);
        return transformFile(content);
      }
      catch {
        return new Map<string, any>();
      }
    }));

    const entries = fileMaps.flatMap((item) => [...item.entries()]);
    const accumulatingMap = new Map<string, any>(entries);

    return OK.format(JSON.stringify(accumulatingMap));
  }
  catch (e) {
    return ServerError.format(e);
  }
}

    In general, there’s a few issues with your original code:

    • You call .then on things that don’t return promises (e.g. transformFile)
    • You call .then on something you’ve already awaited, which doesn’t work the way you expect
    • You don’t await the promises in your map

    I’d highly recommend reading more about the difference between async/await and the traditional .then promise API; as the control flow is quite different between the two of them.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search