Changes to our Git infrastructure

[Jeff Mitchell]

On 29 Dec 2014, at 11:36, Jan Kundrát wrote:
>> As I see it, scratch repos are the first stage in a project's life 
>> cycle.
>> Before playground, you might fiddle with something, drop it in a 
>> scratch
>> repo and share the link on IRC. Deleting it is painless when you 
>> discover
>> that your idea is terrible, or already exists elsewhere.
>
> I agree with scratch repos being useful as a first step.

Except nobody deletes it. That's a large problem. Scratch is nice in 
concept but it's a sysadmin nightmare.

Out of the 846 repos I currently count in scratch/, nearly all of them 
haven't seen a commit in years. Meanwhile that's an extra 846 repos that 
have to be hosted, distributed to anongits, and backed up. That's not 
just a lot; that's the largest piece of our Git hosting. There are more 
scratch repos than normal repos. If clone repos were excluded, scratch 
repos would actually be a majority of all repos -- and again, most of 
them haven't been touched in years. People complain about propagation 
delay of repos to anongits -- syncing all these totally outdated repos 
is a major reason.

I understand the reason people like them -- as a place to test out early 
concepts before making them official projects -- but I think that a 
combination of social changes and technical changes can address losing 
them.

One social change would be divesting ourselves of the idea that getting 
a Git repo from sysadmins is onerous. You can start a git repo locally 
and push it up later; the extra few hours or even a day it might take to 
get a hosted version should really not be the reason that your project 
simply cannot see the light of day.

Another social change, in combination with a technical change, would be 
to get interested people who would like to see this kind of thing happen 
faster and have them take on specific sysadmin tasks -- for instance, 
creating repos on request as soon as possible, spreading the workload 
out so that any given request is likely to be handled faster (your 
standard task queue). The reason it's semi-onerous now (but we should 
stop thinking that it is, see above) is due to lack of manpower. But 
volunteers can be given rights to handle specific ticket types without 
having to become full sysadmins. Like anything else, if people think it 
takes too long, get involved.

Clones (336 of them) are also nice in concept but with better code 
review and/or auditing capabilities (especially pre-commit 
capabilities), I think they will become less important, and remarks on 
this thread seem to back that idea up. On GitHub they're mostly used to 
prepare branches for merge requests because random user X can't commit 
to a branch in repo Y, but KDE's open-to-all-by-default policy, and 
generally decent project communication (IRC, mailing lists, and other 
means), makes that less necessary. Approved developers discussing 
improvements with developers are more likely to get approved to have a 
branch in the main repo; tiny fixes can be approved via a pre-commit 
audit. IOW, I think losing clones can be dealt with by some reasonable 
workflow rejiggering and social engineering.

Remember, we don't have the resources of GitHub or Bitbucket. No 
solution will be 100%. We're trying to decide what solution will, on 
balance, be closest to that 100%. But this also needs to factor in 
actual sysadmin time constraints/availability.

Sorry to be the sad-sack voice in this conversation. Ben and I have been 
having many conversations about alternate hosting mechanisms over the 
past few days, weeks, months, and (yes, really) years, and he's 
definitely Good Cop and I'm definitely Bad Cop when it comes to meeting 
every single desire of every single community member. We'd both love to 
do it (again, really), but I'm trying to be pragmatic, not the least 
because my availability is very hit and miss now and so even more burden 
falls on Ben. I don't want the minimal sysadmin staff we have getting 
crushed by the weight of requirements. Again -- I'd like to see us get 
as close to 100% as possible, and I think that some features that a few 
people find very useful -- but that can fairly effectively be addressed 
through other means -- and that cause amazing amounts of sysadmin and 
system burden should be up on the chopping block unless there are truly 
compelling arguments against.

--Jeff