Websites become listed on search engine results if, and only if, the search engine spiders are indexing them. If they pass right by your pages, and don’t even know they exist, then you’re simply not going to get listed.
Do this simple check. Go to http://www.google.com and type in site:yourdomain.com. You should see a list of the pages from your web domain that Google has in their index. If there are no pages listed you have either made a mistake typing your domain or Google does not have ANY pages off your web domain in their index. If there are less pages indexed than you had expected here are some reasons why.
You May be Inadvertently Blocking Google
1. We regularly help customers who have either a robots.txt file saying disallow robots or some other block on the search engines.
2. Your pages are ‘hidden’ in an HTML frameset.
You Don’t Want Certain Pages Indexed
Some pages are just not meant to be indexed. This may be as you plan it however it’s useful to understand what spiders do and don’t index. Search engine spiders don’t index:
1. Pages only accessible by using a search form i.e. there are no links to the page from other pages on your website
2. Pages that require a log in
3. Pages that require visitors to submit a form
4. Pages that redirect to a different URL before showing content
While some pages are not meant to be indexed, other pages are and might be missed. Pages that spiders often ignore include:
5. Pages with too many outgoing links – try and keep below 100 (see Google’s Webmaster Guidelines)
6. Pages with complex URLs – these often give spiders an error result
7. Pages which are more than three clicks from the home page, often described as “deep pages”
Other factors may prevent your web pages from being indexed by search engines. These factors include:
8. Broken links from your site
9. A webpage that exceeds 105K
10. A slow loading time or a down server.
Finally, if your page is a flash page, search engine spiders struggle to index it as it may have limited content they can scan. As you’re optimizing your website and specific web pages, pay attention to these factors. The goal, of course is to be indexed and to achieve first page search engine results.
Finally Google recommend submitting a Sitemap using Google Webmaster Tools. Google uses your Sitemap to learn about the structure of your site and to increase our coverage of your webpages.