Question 51458 in Javascript Question posted in
A very good W3school tutorial can be found here.

How can I check the status of 100 sites in node – Javascript

I have an array of 100 sites and I need to see if they are available or not. If they are available I need to check the response body for certain content, I also need to check the MX records.

I can do this pretty easily synchronously but it’s takes a long time.

I can’t work out how I would approach this with promises. I’m sure I’m missing something. Can someone point me in the right direction.

Thanks for your help.

All the tutorials I look at run for 1 url not 100s

Tags:

2

Answers


  1. 0

    You can do this in Node by executing async functions within an async loop. Any HTTP requests made in the async function won’t wait for responses before proceeding.

    Of course, you’ll still need to solve for all the CORS access and such.

    Here’s a simplified example:

    const urls = [
    "https://www.typescriptlang.org/",
    "https://www.w3.org/",
    "https://developer.mozilla.org/",
];

async function checkWebsite(url) {
    const response = await fetch(url);
    const status = await response.status;
    return status
}

urls.forEach(async url => {
    const statusCode = await checkWebsite(url);
    console.log(statusCode)
});
    Login or Signup to reply.
  2. 0

    If its NodeJS and not Javascript, you could use Axios and push the results to an array, like this:

    const axios = require('axios');
const websites = [/* Array of website URL's go here */];
var statuses = [];
for (let i = 0; i < websites.length; i++) {
  asynchronousProcess(function(){
    var response = await axios.get(websites[i]);
    var status = await response.status;
    statuses.push(status)
  });
}
function getstatus(url) {
  let index = websites.indexOf(url);
  let result = statuses[index]
  return result
}

    Or, If you don’t want a promise-based approach, you could remove the asynchronous function calls:

    const axios = require('axios');
const websites = [/* Array of website URL's go here */];
var statuses = [];
for (let i = 0; i < websites.length; i++) {
  var response = axios.get(websites[i]);
  var status = response.status;
  statuses.push(status)
}
function getstatus(url) {
  let index = websites.indexOf(url);
  let result = statuses[index]
  return result
}

    Though be aware: using the non-promised based approach may occasionally not work, because you are not waiting for a promise change from the axios.get function

    Feel free to let me know if something doesn’t work and I will be glad to check!

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search