WWW Robots…

A Web Crawler or WWW Robot is simply the easiest way to find better search results from more of the Web. A crawler is a program that visits Web sites and reads their pages and other information in order to create entries for a search engine index. The major search engines on the Web all have such a program, which is also known as a “spider” or a “bot.” Crawlers are typically programmed to visit sites that have been submitted by their owners as new or updated. Entire sites or specific pages can be selectively visited and indexed. Crawlers apparently gained the name because they crawl through a site a page at a time, following the links to other pages on the site until all pages have been read. 

I have done a fair research on this and created presentation[ personal_robots-exclusion-standard.ppt  ], which is attached with this post.

Please feel free to shoot comments.

 Thanks & Regards,Babulal K.R.

Welcome ….

Hi,

In this area you will be finding all those which is related to me … Mostly techie stuffs.. As I am in to IT. Plus my personal articles, Documents, and presentations..All suggestions are welcome….

Thanks & Regards,

Babulal K.R.

“………Every Point has 3-sides; Your Side, My Side & the Right Side. To understand either the Other’s Side or the Right Side, one needs to leave His Side…….. . ”.