seo Question posted in
Some basic information to get you started with this can be found here.

Angular 8 sitemap and robots.txt – SEO

I created an angular application using angular 8. I am working on referencing my website on Google. I have added a sitemap.xml and robot.txt at the route of the project, however when I try to access the file in my browser by doing so:

https://blablawebsite.fr/sitemap.xml

The routing module is picking up the route and can’t find the page. How can I make sure sitemap.xml and robots.txt are not picked by the routing module?

Tags:

4

Answers


  1. 0

    You need to first look into converting your project to Angular Universal. Google and other search engine bots can’t and won’t navigate through your app because all of that happens after-the-fact.

    Start Here: https://angular.io/guide/universal

    A lot of people misunderstand how all of this works and get very far into their project before realizing the difference between a website and an SPA. No biggie, you can still get your angular app to rank with Server Side Rendering

    Login or Signup to reply.
  2. 0

    Using Angular Universal with Cloud Functions for Firebase to host and I’m putting sitemap.xml with robots.txt inside src. Then in angular.json within "assets" adding it like so:

    ...
"assets": [
  "APP_NAME/src/robots.txt",
  "APP_NAME/src/sitemap.xml",
],
...

    Works great 🙂

    Login or Signup to reply.
  3. 0

    Contrary to how I understand the accepted answer, tt is not neccessary to add angular universal to your project just for adding a sitemap.xml or robots.txt.

    As @David said more succinctly in his comment,all you need to do is:

    1. Add put the files into your project (next to the favicon.ico) in /src
    2. Add them to projects>architect>build>options>assets in your angular.json like so:
     "assets": [
                  "src/favicon.ico",
                  "src/assets",
                  "src/robots.txt",
                  "src/sitemap.xml"
                ],

    It is explained in a bit more detail here.

    Add. remarks:

    • Make sure you do not accidentally catch the assets of "test" only in your angular.json, they come into view earlier when skimming through the file
    • You can verify Google now recognizes your robots.txt if the warning that your robots.txt file is invalid no longer appears underneath the SEO section of the Lighthouse report in Chrome Dev Tools
    • It is true however, that even with a robots.txt, search engines other than Google are not known to be able to deal well with Angular SPAs without server side rendering (e.g. through Angular Universal). And even Google’s capabilities here are debated. This guy explains it quite well.
    • My answer holds for Angular 10 and 11, other versions I do not know
    Login or Signup to reply.
  4. 0

    While other answers provide a solution for adding single robots.txt and sitemap.xml files, usually you would want to have different versions of these files based on the instance you are deploying.

    For example, in production you would want to allow search engines to crawl and index your pages, but not on staging or development instances.

    If you want to have different version of files based on the environment, you can do it like this:

    Step 1

    Create one folder called robots, and 3 subfolders inside called development, staging and production (or whatever environments you want). Then, in each of subfolders create environment specific robots.txt and sitemap.xml files.

    Step 2

    In angular.json file, specify assets separately for each environment:

    {
  "architect": {
    "build": {
      "configurations": {
        "configurations": {
          "production": {
            "assets": [
              "src/favicon.ico",
              "src/assets",
              "src/manifest.webmanifest",
              {
                "glob": "robots.txt",
                "input": "src/robots/production/",
                "output": "./"
              },
              {
                "glob": "sitemap.xml",
                "input": "src/robots/production/",
                "output": "./"
              }
            ],
            ...
          },
          "staging": {
            "assets": [
              "src/favicon.ico",
              "src/assets",
              "src/manifest.webmanifest",
              {
                "glob": "robots.txt",
                "input": "src/robots/staging/",
                "output": "./"
              },
              {
                "glob": "sitemap.xml",
                "input": "src/robots/staging/",
                "output": "./"
              }
            ],
            ...
          },
          "development": {
            "assets": [
              "src/favicon.ico",
              "src/assets",
              "src/manifest.webmanifest",
              {
                "glob": "robots.txt",
                "input": "src/robots/development/",
                "output": "./"
              },
              {
                "glob": "sitemap.xml",
                "input": "src/robots/development/",
                "output": "./"
              }
            ],
            ...
          }
        }
      }
    }
  }
}
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search