Thursday, November 25, 2010

Official Google Webmaster Central Blog

Official Google Webmaster Central Blog

Link to Google Webmaster Central Blog

Controlling crawling and indexing now documented on code.google.com

Posted: 24 Nov 2010 03:14 PM PST

Webmaster level: All

Do you know how Google's crawler, Googlebot, handles conflicting directives in your robots.txt file? Do you know how to prevent a PDF file from being indexed? Do you know Googlebot's favorite song? The answers to these questions (except for the last one :)), along with lots of other information about controlling the crawling and indexing of your site, are now available on code.google.com:

Controlling crawling and indexing



Now site owners have a comprehensive resource where they can learn about robots.txt files, robots meta tags, and X-Robots-Tag HTTP header directives. Please share your comments, and if you have questions you can post them in our Webmaster Help Forum.

No comments:

Post a Comment