robots.txt in quickgit.kde.org

[Ben Cooksley]

On Sun, Dec 27, 2015 at 11:53 PM, Ashish Bansal
<bansal.ashish096 at gmail.com> wrote:
> Hi everyone,

Hi Ashish,

>
> "quickgit.kde.org" contains robots.txt[0] which is disallowing search
> engines to fetch the project repos. I just wanted to know if this is
> intentional or not?
>
> If I recall correctly, mirror of kde repositories on github was created just
> because it wasn't being indexed by the search engines.

This is intentional, and is done to reduce the server load created by
indexers such as Google on the system hosting quickgit.kde.org.
(Generation of the pages, including the main index is substantially
more expensive than it appears due to the disk access required by
Git/SVN to return the needed information).

>
> [0] https://quickgit.kde.org/robots.txt
>
> --
>
> Regards,
> Ashish Bansal
> http://ashish-bansal.in

Regards,
Ben Cooksley
KDE Sysadmin

>
>
>>> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
>>> <<
>