Question 184365 in Javascript Question posted in
A very good W3school tutorial can be found here.

Javascript – Repeatedly Send Request if no Response

How can I keep fetching the webpage while it didn’t get response being in a loop?

for (el of NodeList) {
  const url = el.getAttribute('href')
  const res = await fetch(url) //if no response, below code won't execute
  const html = await response.text()
  const parser = new DOMParser()
  const doc = parser.parseFromString(html, 'text/html')
  alert('parsed successfully')
}

So as it works in a loop and sends very many requests at a time, even a small and short issue with network can spoil the whole process, what can I do?

Tags:

2

Answers


  1. 0

    Don’t loop

    Try something like this (not tested)

    let nList = [...NodeList].map(({href}) => ({ href,done:false,doc:"" }));
const getData = () => {
  if (nList.length === 0) {
    processText(); // here you can loop over the doc in the object
    return;
  }
  const current = nList[0];
  const res = fetch(url)
  .then(response => response.text())
  .then(text => {
    const parser = new DOMParser()
    current.doc = parser.parseFromString(html, 'text/html')
  })
  .done(() => {
    current.done = true;
    console.log(current.href,'parsed successfully')
    nList = nList.filter(({done}) => !done);
  })
  .always(() => setTimeout(getData,2000)); // try next (or again)
}
getData();
    Login or Signup to reply.
  2. 0

    I guess you should handle network errors. Try to do this with retry mechanism that tries to fetch the webpage again if no response is received.

    You can make function fetchWithRetry that takes two arguments: the URL to fetch and the maximum number of retries (to avoid infinite loop).

    async function fetchWithRetry(url, numberOfRetries) {
  try {
    const response = await fetch(url);
    const html = await response.text();
    const parser = new DOMParser();
    const doc = parser.parseFromString(html, 'text/html');
    alert('parsed successfully');
    return doc;
  } catch (error) {
    if (numberOfRetries > 0) {
      console.error('Error fetching webpage. Retrying...', error);
      return fetchWithRetry(url, numberOfRetries - 1);
    } else {
      console.error('Error fetching webpage. Maximum retries exceeded.', error);
      throw error;
    }
  }
}

    Then you can modify your code to use the fetchWithRetry function instead of the fetch function.

    for (el of NodeList) {
  const url = el.getAttribute('href');
  const doc = await fetchWithRetry(url, 3);
  // Do something with the parsed document
}
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search