We are in a process of redesign, and would like to test a way google sees our website, but we would not like it to be indexed it at this point.
We have a separate subdomain for development, but if we disallow scraping in robots.txt, we can’t do any tesing, because google crawlers will not scrape any testing pages.
Is there a way to run some tests via google webmasters console without allowing site to be indexed publicly?
2
Answers
I didn’t try to do it in that way, but I think it should work for you. You can close your domain with HTTP Basic Authentication. And add your login and passwords to URL when you adding your domain to Google Console. Smth like that https://gooduser:[email protected]/
Use the
noindex
tag.The
noindex
value of an HTML robots meta tag requests that automated Internet bots avoid indexing a web page. Source