skip to Main Content

Here are two lines in robots.txt file:

Disallow: /messages
Disallow: /qanda/edit/

What paths is this /messages point to? And what about /qanda/edit/? Is the / in the end of the second path useless? Or it has a specific meaning?


Or let me ask this way: what’s the exact meaning of theses?

Disallow: /messages
Disallow: /messages/

2

Answers


  1. if you want to disallow all in messages directory I think you have to let the slash at the end

    Disallow: /messages/
    

    if you dont put the “/” at the end, it could be /messages.html for exemple

    Disallow : /messages
    

    it is like

    Disallow : /message*
    
    Login or Signup to reply.
  2. Disallow values represent the beginning of the URL path.

    Disallow: /messages

    would block URLs like these:

    https://example.com/messages
    https://example.com/messages.html
    https://example.com/messages/
    https://example.com/messages/foo
    

    Disallow: /messages/

    would block URLs like these:

    https://example.com/messages/
    https://example.com/messages/foo
    

    but allow URLs like these:

    https://example.com/messages
    https://example.com/messages.html
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search