seo Question posted in
Some basic information to get you started with this can be found here.

Robots.txt and sub-folders – SEO

Several domains are configured as add-ons to my primary hosting account (shared hosting).

The directory structure looks like this (primary domain is example.com):

  • public_html (example.com)
  • _sub
    • ex1 –> displayed as example-realtor.com
    • ex2 –> displayed as example-author.com
    • ex3 –> displayed as example-blogger.com

(the SO requirement to use example as the domain makes explanation more difficult – for example, sub ex1 might point to plutorealty and ex2 might point to amazon, or some other business sub-hosting with me. The point is that each ex# is a different company’s website, so mentally substitute something normal and different for each “example”)

Because these domains (ex1, ex2, etc) are add-on domains, they are accessible in two ways (ideally, the 2nd method is known only to me):

(1) http://example1.com
(2) http://example.com/_sub/ex1/index.php

Again, example1.com is a totally unrelated website/domain name from example.com

QUESTIONS:

(a) How will the site be indexed on search engines? Will both (1) and (2) show up in search results? It is undesireable for method 2 to show up in google)

(b) Should I put a robots.txt in public_html that disallows each folder in the _sub folder? Eg:

 User-agent: *
 Disallow: /_sub/
 Disallow: /_sub/ex1/
 Disallow: /_sub/ex2/
 Disallow: /_sub/ex3/

(c) Is there a more common way to configure add-on domains?

Tags:

2

Answers


  1. 0

    Do not Use Multi site features with Google. Google Ranking effect on Main domain also. If Black hat and also Spam generate sub directory sites.

    My Suggestion If you need important site on Sub Categories then Put all Sub Domain noindex .

    Robot.txt

    User-agent: *
 Disallow: /_sub/
 Disallow: /_sub/ex1/
 Disallow: /_sub/ex2/
 Disallow: /_sub/ex3/
    Login or Signup to reply.
  2. 0

    This robots.txt would be sufficient, you don’t have to list anything that comes after /_sub/:

    User-agent: *
Disallow: /_sub/

    This would disallow bots (who honor the robots.txt) to crawl any URL whose path starts with /_sub/. But that doesn’t necessarily stop these bots to index your URL itself (e.g., list them in their search results).

    Ideally you would redirect from http://example.com/_sub/ex1/ to http://example1.com/ with HTTP status code 301. It depends on your server how that works (for Apache, you could use a .htaccess). Then everyone ends up on the canonical URL for your site.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search