Sysadmin report on the modernization of our infrastructure

[Jan Kundrát]

On Wednesday, 21 January 2015 15:54:52 CEST, Milian Wolff wrote:
>> 6) The discussion focuses in highlighting Phabricator's benefits, which is
>> understandable from your point of view. However, much of the same things
>> can be said about Gerrit as well, especially its backing by a well-known
>> player, adoption by multiple companies and multinational corporations
>> (SonyErisccon, SAP, KitWare,...) as well as by many FLOSS projects (Qt,
>> Andorid, LibreOffice, Eclipse, OpenStack, WikiMedia, Typo3, Cyanogenmod,
>> Tizen, and a ton of others). Its scaleability has also been verified by
>> real-world use over many years (for far longer than Phabricator).
>
> Imo, quite the contrary. It concentrates on the issues the admins have with 
> their current setup, and then shows how Phabricator could help with that.

Right, I should have said "the proposal to use Phabricator and the reasons 
for using that particular tool focus on highlighting Phabricator benefits 
...". Sorry for not being clear, I appreciate the value of describing the 
pain points of the legacy setup (and I woud appreciate even more details to 
be able to offer a better alternative).
 
>> 9) I do not know what effort for integration with KDE Indentity you're
>> referring to. It's a simple LDAP server, and the entire "integration" was
>> trivial. It's really just a matter of configuring Gerrit to talk to LDAP,
>> which I expect to be also the case with Phabricator.
>
> I don't see where this is mentioned in regard to Gerrit. I can 
> only find LDAP 
> being mentioned when talking about the status quo for KDE, which does not 
> include Gerrit.

The part which I was commenting on is the following paragraph:

> As a result of this it would be necessary to combine several different 
> components to produce a complete Git solution. This would require further 
> effort to integrate them with both each other and parts of KDE 
> infrastructure such as Identity. Even after such effort is completed a 
> certain degree of synchronisation between the tools will need to be 
> maintained, such as registering repositories in both the code hosting 
> tool and Gerrit.

There was no extra work to integrate Gerrit with KDE's Identity, and I 
expect most of other tools which we might need to use to have LDAP support 
out of box because that's an industry standard for identity management.

"KDE Identity" == "LDAP", in this context.

The paragraph above also assumes that Gerrit would not be used as a primary 
code hosting place. There's no reason for that, so the raised conclusions 
("this will require syncing") are not true.

>> 4) You have indicated some (pretty important to me) limitations of
>> Phabricator with a remark that "it's a fast moving software, we might get
>> there eventually". I think that SW should be evaluated based on
>> functionality present right now in the development version and 
>> not based on
>> opened feature requests. We've been promising e.g. support for multiple
>> accounts in Trojita for many years already, and it didn't make it happen.
>
> This belongs to the point below, imo. Or what are you referring 
> to? 

When I read the proposal, I see some enthusiasm for Phabricator's swift 
development pace. That's good, but at the same time it isn't an answer to a 
lack of features. Here are some of the relevant bits:

- CI integration
- scratch repos
- clustering for scaleability
- preserving author metadata
- direct git access to patches under review
- patch upload via git

What I'm saying is that an opened feature request and a rapid speed of 
development are not something to gurantee that these will be ready in a 
month, or even in a couple of years.

> To my knowledge, here are some things that Gerrit does not provide, but 
> Phabricator potentially provides:

Yes, Gerrit doesn't include wiki pages, issue tracker, Kanban planner and 
various calendars. I don't necessarily see that as a drawback. Do we want 
to migrate wikis and Bugzilla? If yes, I can understand that Phabricator 
might be a compeling tool, but so far the proposal was limited to just 
revamping the git hosting and code review.

> - a project overview with a code browser, project meta data etc. pp. and a 
> list of commits inside a repository. Qt still uses gitorious for this, afaik

Right, we will have to pick one (or multiple) of the WWW git browsers out 
there. If I was designing such a service, I would let it run from a 
dedicated VM and have Gerrit replicate stuff to these servers. Scaling up a 
stateless, read-only service such as cgit/gitweb/... is very easy.

> - the ability to get notified about new commits in a project. (this is 
> different from new reviews)

Gerrit has hooks for triggering this ("ref-updated"), and it's easy to send 
e-mails from that context. There are scripts for that in git's contrib/.

@sysadmins, how is that tackled by Phabricator?

Also, if a proper code review was enforced, there would be no need for 
this.

> - Apparently the anongit problem, but Ben would need to fill in 
> more details here.

We are already using a stock upstream plugin called "replication" which can 
push refs around, create and even delete remote repos based on local 
events. Our Gerrit instance is using it for propagation of changes to 
git.kde.org.

I do not understand where the problem could possibly be; Gerrit is 100% 
ready to replicate to as many anongit nodes and/or GitHub repos as needed. 
Many people are doing this, and many people are even using various Gerrit 
instances together for even better scaleability. I seriously doubt that KDE 
will need *that* level of scaling in the next five years, though.

> - post-commit review

One can comment on closed review requests, and the result is available as 
any other review, e-mailed out to the review participants, and accessible 
to search engines.

It's true that there's no git browser where you could attach notes to a 
particular line and "open a ticket" for that. Do we need such a feature, 
and if so, how do we intend to use it?

> - project management, todo / kanban board

"Project management" can mean multiple things, so I assume that you're 
referring to release planning, issue prioritization etc here. Gerrit indeed 
doesn't doesn't have that. The plan will have to offer some tool for this. 
What are your (community's) favorite tools for TODOs etc?

> - issue tracker (only if we ever port the bugzilla content over)
> - paste (only if that would replace paste.kde.org)
> - wiki (only if that could be used to replace the existing wiki structure)
> - various tools which would help with release management, e.g. calendars / 
> countdown timers / release branches

See above; I'm afraid that we have to draw a line somewhere on what we 
would like to replace and what to preserve. A Gerrit proposal should 
probably say "we're happy with curent wikis because they work well enough", 
sure.

I just don't see Phabricator's feature set as a huge benefit here.

> - overall, the app framework phabricator provides seems to 
> indicate that its 
> easy to extend, contrary to what I heard from gerrit

My understanding is that other tools are expected to interact with Gerrit 
through its rich set of APIs. There are REST APIs for everything, there's 
an SSH-based notification channel and we already have an IRC bot notifying 
us about reviews which uses this. There are patches under review upstream 
which add support for a real message bus (AMQP IIRC). I've deployed a full 
CI system which talks to Gerrit through these APIs and provides feedback 
about changes succeeding/failing to build on multiple platforms. Other 
people are tying into these events to automatically create and upload 
release tarballs. I'm sure that this demonstrates that Gerrit is quite 
extensible.

Cheers,
Jan

-- 
Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/