Javascript Question posted in
A very good W3school tutorial can be found here.

Javascript – Request to Express endpoint won't stop loading, even if I set response status (Multer and Express issues)

I’m on the process of making a filter based off evaluating a file’s signature obtained via a file’s buffer provided by Multer. I know Multer provides the MIME type, but I also have to check wether the file is actually of that MIME type via some other ways of validation.

Because of that, my steps are:

  1. Temporarily store the request’s files in Multer’s memoryStorage() to get the buffer, then
  2. Filter the valid files
  3. Append those valid files and other information I need for a newly made FormData
  4. Make a POST request to the other endpoint("/upload") with the new FormData
  5. Store files in the cloud with Multer

PROBLEM IS: Once I force some error on the second endpoint, the process ends up on a loop. The error is detected, however I cannot return any HTTP code as a response to the first endpoint!

First endpoint:

/* NOTE: storeInMemory.array("files") stores in memory the files' buffer information, which 
/* are needed in order to work with validating the files! */
app.post("/filesUpload", storeInMemory.array("files"), async (req, res) => {
    const files = req.files;

    /* this array stores the items i have to upload */
    let uploadArray = [];

    /* filter only the items i actually need to upload */
    for (const item of files) {
        /*-- some filter logic --*/
        uploadArray.push(item);
    }

    /* create new FormData to store the actual files i have to upload -
    /* as multer works with http a new formdata has to be created, so i can make a call to another endpoint */
    let form = new FormData();
    form.append("someData", req.body["someData"]);

    /* append the files to the form i'll use to upload later */
    uploadArray.forEach((item) => {
        form.append("files", item.buffer, item.originalname);
    });

    try {
        const postUrl = process.env.MY_URL + "/upload";
        const myHeaders = { headers: { 'Content-Type': `multipart/form-data; boundary=${form._boundary}` } };
        /* Request to other endpoint */
        let uploadResult = await axios.post(postUrl, form, myHeaders);

        if (uploadResult.status == 200) return res.send(uploadResult).status(200);
        /* PROBLEM HERE: this is the point the program never gets to and i need it to */
        else {
            return res.send(uploadResult).status(500);
        }
    }
    catch (error) {
        console.error("Unpredicted error occurred: ", error);
        return res.send(error)
    }

});

Second endpoint:

app.post("/upload", async (req, res) => {
    try {
        await new Promise((resolve, reject) => {
            /* upload the files to cloud */
            uploadOnCloud.array("files")(req, res, (err) => {
                if (err) {
                    console.error("Error happened successfully!", err)
                    return reject(err);
                }
                else
                    return resolve();
            });
        });
        /* If no err detected, ends with 200 */
        res.sendStatus(200);
    }
    catch (error) {
        /* PROBLEM: can neither set status, nor anything else */
        res.status(500).send(error.message);
    }
});

What could be wrong here?

Tags:

2

Answers


  1. 0

    When your second endpoint sends a status 500, your first endpoint assumes that an error occurred and directly goes to your catch. You need to specify your status there to solve your problem :

    catch (error) {
    res.status(500).send(error.message);
}
    Login or Signup to reply.
  2. 0

    I found an error while reproducing the original code you wrote. The line form.append("files", item.buffer, item.originalname) was incorrect. It should be as shown below. Additionally, there’s no need to set the Content-Type explicitly.

    form.append("files", 
   new File([file_buffer], file_name, {
        type: file_mimetype,
   })
);

    Also, I wasn’t able to reproduce the stuck loading issue. I think it might be due to Multer waiting for data on the request body, but there wasn’t any since you attached the request body with no files!

    And the Solution

    Variables

    const cloudStorage = multer.memoryStorage();
const localStorage = multer.memoryStorage();

const cloudUploader = multer({
  limits: {
    fileSize: 1024 ** 3, // 1GB
  },
  storage: cloudStorage,
});

const localUploader = multer({
  limits: {
    fileSize: 100 * 1024 ** 2, // 100MB
  },
  storage: localStorage,
});

    Upload files route & its controller

    app.post("/upload-files", localUploader.array("files"), async (req, res) => {
  /** @type {Express.Multer.File[]} */ const files = req.files;

  const form = new FormData();
  form.append("someData", req.body["someData"]);

  // ? You can perform your filter logic here on files

  // Applying append function directly on files 
  files.forEach((file) => {
    form.append(
      "files",
      new File([file.buffer], file.originalname, {
        type: file.mimetype,
      })
    );
  });

  try {
    const postUrl = `${process.env.ORIGIN}/upload`;
    
    // Using built in node-fetch & no need to set Content-Type headers manually as form-data handles it
    const uploadResult = await fetch(postUrl, {
      method: "POST",
      body: form,
    });

    if (uploadResult.ok) { // uploadResult.status is in range 200-299 ( success )
      return res.status(200).json({
        error: false,
        message: "FILES UPLOADED SUCCESSFULLY!"
      });
    } else {
      return res.status(500).json({
        error: true,
        message: "Something went wrong",
      });
    }
  } catch (error) {
    console.error(error);
    return res.status(500).json(error);
  }
});

    Cloud file upload route & its controller

    app.post("/upload", async (req, res) => {
  try {
    await new Promise((resolve, reject) => {
      /* upload the files to cloud */
      cloudUploader.array("files")(req, res, (err) => {
        //? TO SEE EVERYTHING WORKS SAVING FILES TO LOCAL DISK
        /** @type  {Express.Multer.File[]} */ const files = req.files;
        files.forEach((file) => {
          fs.writeFileSync(`public/uploads/${file.originalname}`, file.buffer, {
            encoding: "utf-8",
          });
        });

        if (err) {
          console.error("Error happened successfully!", err);
          return reject(err);
        } else return resolve();
      });
    });

    return res.json({ error: false, message: "Files uploaded" });
  } catch (e) {
    console.error(e);
    return res.json({ error: true, message: "Something went wrong" });
  }
});
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search