seo Question posted in
Some basic information to get you started with this can be found here.

Creating 60k+ pages slows down page performance – SEO

I work on a fairly large e-commerce site that was built using gatsby. Up till now we have been dynamically loading all our product data at runtime so we only had 1 product page for the whole site and a total of about 3k pages.

However, for SEO reason we are trying to statically build all our product pages and that adds about 60k pages to the website.

After we did this our app.[hash].js file has increased to 10mb and general page performance as tanked. Hovering links causes page to hang for a half a second.

We are creating the product pages by passing the product data as page context when calling create pages:

  const productsRaw = await fetch(productsUrl)
  const products = await productsRaw.json()

  /* Build a page for each product */
  if (products) {
    products.forEach(product => {
      const productPage = {
        path: `/product${product.route}`,
        component: path.resolve(`./src/templates/product.jsx`),
        context: product,
      }
      createPage(productPage)
    })
  }

We believe the bad page performance is related to the app.js file jumping to about 10mb. Is there any optimization we can make to reduce the size of this file or is this just unavoidable when increasing the page count?

Update

After analyzing the webpack I found that the issue is that each product page is added to the match-path.json file. This caused the file to exceed 10mb. Is it necessary that all these paths get added to the file and if not is there a way to correct that when I create the page?

enter image description here

Update 2

I ended up writing a script to remove all non-regex paths from match-path.json before webpack runs and the site seems to work great and perform very well. Any way I can prevent them from getting added in the first place?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I figured this out. So we actually do create another dynamic product page that contains a matchPath. This page is used just in case we dont properly build out all the products and a user tries to access a product thats not statically generated. In this case, it would hit a match path that would dynamically load product data. This page also uses the same template as the other product page. When I remove this page from getting created all the other products get removed from match-path.json but when it is included all the other products get added to the match-path.json. Here is the code:

    /* Build a page for each product */
  if (products) {
    products.forEach(product => {
      const productPage = {
        path: `/product${product.route}`,
        component: path.resolve(`./src/templates/product.jsx`),
        context: { product, banners: PDPBannerData ? PDPBannerData.contentfulProductAdditionalContent : null },
      }
      createPage(productPage)
    })
  }

  const productPage = {
    path: '/product',
    component: path.resolve(`./src/templates/product.jsx`),
    context: { banners: PDPBannerData ? PDPBannerData.contentfulProductAdditionalContent : null },
    matchPath: `/furniture/product/*`,
  }
  createPage(productPage)

    I'm assuming it does this for a reason, any thoughts on why this is necessary? We were talking about removing this dynamic page so its probably not a big deal for us to just deleted it and move on.


  2. 0

    While 60k is a fair number of pages to generate at build-time, it’s not extreme. The issue you’re having is that you are passing product data in via context. That is intended to be a small amount of data that allows you to query for the data in your page template.

    1. In gatsby-node.js, call createPage for each page that should be exported
    2. Include the minimum amount of information needed to find the data for that page in the context passed to createPage (e.g. an ID or slug)
    3. In your page template, use the context to filter the data returned by the GraphQL query (each property of context is available as an argument to a named GraphQL query)
    4. Fetch only the data you need for the page in your GraphQL query; any data that remains constant across all instances of the component can use a static query instead

    If you follow the above, each page query data will be written out as an individual page-data.json file that will be rather small (it’s literally just the query data). That should be the only data that is prefetched when you hover over a link.

    Note: if, after doing this, your app.<hash>.js file is still large, chances are you have some data you’re not expecting in there. Check it out using webpack-bundle-analyzer to see where the weight is coming from.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search