robots.txt in quickgit.kde.org

[Lydia Pintscher]

On Sun, Dec 27, 2015 at 12:08 PM, Ben Cooksley <bcooksley at kde.org> wrote:
> On Sun, Dec 27, 2015 at 11:53 PM, Ashish Bansal
> <bansal.ashish096 at gmail.com> wrote:
>> Hi everyone,
>
> Hi Ashish,
>
>>
>> "quickgit.kde.org" contains robots.txt[0] which is disallowing search
>> engines to fetch the project repos. I just wanted to know if this is
>> intentional or not?
>>
>> If I recall correctly, mirror of kde repositories on github was created just
>> because it wasn't being indexed by the search engines.
>
> This is intentional, and is done to reduce the server load created by
> indexers such as Google on the system hosting quickgit.kde.org.
> (Generation of the pages, including the main index is substantially
> more expensive than it appears due to the disk access required by
> Git/SVN to return the needed information).

Is there some place where search engines can easily index our source
code or are we shooting ourselves in the foot here?


Cheers
Lydia

-- 
Lydia Pintscher - http://about.me/lydia.pintscher
KDE e.V. Board of Directors / KDE Community Working Group
http://kde.org - http://open-advice.org