Source: Live Search Webmaster Blog
Live Search team is likely to test a new crawler - msnbot/2.01b. However the new crawler will follow the current robots.txt protocol set for MSN. Here goes the post from the Live Search Webmaster Blog....
Live Search is committed to continuing to improve how we discover and crawl your content. In the coming weeks, we will be testing an update to MSNBot, which may show up as a new crawler name in your referrer logs. The new crawler user agent string will appear as:
msnbot/2.0b (+http://search.msn.com/msnbot.htm)
You’ll still see the current version msnbot/1.1 (+http://search.msn.com/msnbot.htm) as Live Search’s primary user agent. We intend to ensure that any robots exclusion protocol you are using is respected. As such, you don’t need to update your Robots.txt file. We plan on crawling at a slow speed during the tests with the updated version.
When the new crawler replaces our current crawler, we will keep you up-to-date here on the Webmaster Center blog. If you have any questions about the Live Search web crawlers, you can post them in our forums.
No comments:
Post a Comment