SEO Consult Blog
Tell the Search Engines what you want them to see
Filed under: SEO Strategy by Nick on June 11th, 2008 @ 4:36 pm
The key to a successful and focused SEO campaign is undoubtedly making the search engines aware of what you don’t want them to crawl on your website. Whether it be entire directories or external links there are ways and means of keeping the power circulating, this is done by robots.txt files, htaccess files, nofollow links and noindex functions.
A robots.txt file is a simple notepad file on the root of your domain, this is also known as an exclusion file and is the place to locate any directories that do not require spidering. Admin, image, pdf and various other folders can be huge in terms of size, and therefore will waste valuable crawling time. This will ensure the pages with potential are noticed and indexed more thoroughly.
The htaccess file is a file that directs one page to another page, a good example of this is when you have a homepage named www.seoconsult.co.uk/ and another called www.seoconsult.co.uk/index.htm, by using the htaccess effectively you can direct one to the other, thus creating one page of combined power.
Nofollow and noindex are functions that can be placed within the a tag. The nofollow will state that any page authority that the link is situated on will not be passed on, improving the strength of the other links that have not been nofollowed. This is useful when linking to external links, keeping the strength within your own site. Noindex is essentially an onpage exclusion function, stating to the search engines which links have pages that are not meant for indexing.
Related posts:
- Duplicate content
From an SEO and a Search Engine point of view, content is king and duplicate content is very, very bad. Just like in the real... - URL Rewrites and Redirects
Rewriting your websites URLs is an important and proven method for successful SEO. There are four core reasons for this, they maybe staring you in... - Using Redirects On Your Website
A site redesign is often well complemented with good redirects. A redirect not only redirects human visitors but also the search engine spiders. It isn’t... - The Characteristics Of A Good SEO Link
Link building is not only an important part of search engine optimization, but can drive traffic directly your web pages. Every link should be judged... - SEO - Why don’t web developers use Re-directs?
After working on several websites this week for clients-I am posing the question why don’t web developers and designers utilise permanent redirects. It appears that...
This entry was posted on Wednesday, June 11th, 2008 at 4:36 pm. You can follow any responses to this entry through the RSS feed.
Contact SEO Consult About Search Engine Optimisation
Responses to “Tell the Search Engines what you want them to see”
Leave a Reply
Search Blogs
Highest Rated Articles




(5 out of 5)Multiple Domains for SEO?




(5 out of 5)Why One Keyword Won’t Do




(5 out of 5)Site Structure Will Affect Your Website Optimization Campaign




(5 out of 5)SEO - Copywriting Services - Getting Half The Package




(5 out of 5)Tag Cloud
WP Cumulus Flash tag cloud by Roy Tanck requires Flash Player 9 or better.
Recent Posts
SEO Tools 3 - “What Google Sees” Tool
SEO Tools 2 - Link Popularity Checker
Search Engine Optimization Tools 1 - Keyword Tools
Basic ABC of SEO
Blog Categories
Browser Software and Search Engine Optimisation
CMS and Admin Systems and SEO
eCommerce and Search Engine Optimisation
Ethical Search Engine Optimisation
General SEO News
Geo Targeting Search Engine Optimisation
Google and Search Engine Optimisation
Google PageRank
Google PR Update
Internet Marketing
Keyword Research and Search Engine Optimisation
Latest News
Link Campaigns and Search Engine Optimisation
Pay Per Click & SEO
Press Releases
Search Engine Optimisation
Search Engine Optimisation Advice
Search Engine Optimisation Analysis
Search Engine Optimization
Search Engine Optimization Keywords
Search Engine Optimization Marketing
SEO Analysis for Results
SEO Companies
SEO Consult
SEO Copywriting
SEO Experiment
SEO Management
SEO Strategy
SEO Submissions
SEO Tips
SEO Tools
SEO vs PPC
Site Analysis and Search Engine Optimisation
Social Networking and Search Engine Optimization
Top SEO Tips
W3C and Search Engine Optimisation
Website Optimization
Yahoo! and Search Engine Optimisation
View All Categories
Blog Archives
| December 2008 | November 2008 |
| October 2008 | September 2008 |
| August 2008 | July 2008 |
| June 2008 | May 2008 |
| April 2008 | March 2008 |
| February 2008 | January 2008 |
| December 2007 | November 2007 |









[...] put it plainly yet confusing the hell out of - search engines use ’spiders’ or ‘Robots‘ (actually programs) that ‘crawl’ their way through the websites of the world [...]