Here are two lines in robots.txt
file:
Disallow: /messages
Disallow: /qanda/edit/
What paths is this /messages
point to? And what about /qanda/edit/
? Is the /
in the end of the second path useless? Or it has a specific meaning?
Or let me ask this way: what’s the exact meaning of theses?
Disallow: /messages
Disallow: /messages/
2
Answers
if you want to disallow all in messages directory I think you have to let the slash at the end
if you dont put the “/” at the end, it could be /messages.html for exemple
it is like
Disallow
values represent the beginning of the URL path.Disallow: /messages
would block URLs like these:
Disallow: /messages/
would block URLs like these:
but allow URLs like these: