Robots.txt file restricting msnbot at http://lists.kde.org/robots.txt

[Amy Wilcox (Murphy & Associates)]

Hi,

I am contacting you from the Microsoft Corporation and its Internet search engine Bing (http://www.bing.com) in regards to your robots.txt file located at http://lists.kde.org/robots.txt  Our customers have alerted us that your site was not visible in our results and we have discovered that you are hiding your site to our search engine crawler, named msnbot, via your robots.txt file.

We would be pleased if you edit your robots.txt file to allow our search engine to index your content which additionally will increase traffic to your site via our search results. The only step needed on your part is to add the following lines for msnbot within the robots.txt file.

User-agent: msnbot
Disallow:

If you feel that msnbot is crawling your site too frequently, you can use the crawl delay directive in robots.txt http://www.bing.com/community/blogs/webmaster/archive/2008/04/18/ramping-up-msnbot.aspx

For more information about robots.txt http://www.bing.com/community/blogs/webmaster/archive/2008/06/03/robots-exclusion-protocol-joining-together-to-provide-better-documentation.aspx Also our Webmaster Center website (http://www.bing.com/webmaster/) can assist you further in obtaining the best results possible for your business or website.  If you have further questions please let me know.

Best regards,

Amy Wilcox
Web Analyst, Bing from Microsoft
v-amwilc at microsoft.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.kde.org/mailman/private/kde-www/attachments/20090718/bb4d9637/attachment.html>