Sysadmin report on the modernization of our infrastructure

[Ben Cooksley]

On Thu, Jan 22, 2015 at 11:39 AM, Thomas Lübking
<thomas.luebking at gmail.com> wrote:
> On Mittwoch, 21. Januar 2015 22:47:20 CEST, Ben Cooksley wrote:
>
>>> - CI integration
>>
>>
>> We've addressed that. It can be done with little fuss.
>
> Do you have any hard data on the "little fuss"?
>
> I googled for CI and saw quite some "we need such" comments over the last
> two years and also some homebrew script to integrate travis, but it didn't
> sound like "put this 10 LOC sippet here and be happy" or so and apparently
> is not a trivial issue.

Reviewboard does not make it trivial to create such an integration at
this time. Please read the report where this is explained in full.

Phabricator does make it easier, please see the links i've posted to
the mailing list.

> Would Scarletts efforts (you're pointing them?) be upstreamed to
> Phabricator?

Scarlett's efforts are KDE specific and relate exclusively to Jenkins.
We would provide information to the Phabricator community on how we
achieved our integration of course.

>
>>> - clustering for scaleability
>>
>>
>> Not sure what you mean here.
>
> I'd say
> https://secure.phabricator.com/book/phabricator/article/cluster/
>
> That looks scary.
> On how many machines do we run atm? (combined, ie. git, RB, and bugzilla, in
> case)

Git runs on the physical host Kater (i7-2600 w/ 32gb RAM). It shares
this with Identity, CiviCRM and Jenkins, as well as our IRC services.
The machine isn't heavily utilised except at certain times - but that
is caused by scaling issues in the anongit network. At all other times
the machine is fairly idle.

Reviewboard runs on the physical host Tesla (i7-950 w/ 8gb RAM). It
shares this with reports.kde.org, an Owncloud instance and some other
minor services.
Once again, the machine isn't heavily utilised except when someone
makes a large number of commits or sends a stack of emails, which
generates a bit of load as reports.kde.org processes them (design is
slightly suboptimal here).

Bugzilla runs on the physical host Katze (i7-3770 w/ 32gb RAM). It
shares this with the Dot, Forum, Wikis as well as all our other CMS
based sites including krita.org, calligra.org, kdevelop.org and
kdenlive.org. It also hosts todo.kde.org and paste.kde.org. This
machine gets a little busy during the EU daytime due to these other
sites - Bugzilla doesn't present a load issue (it is a memory hog due
to the Perl and MySQL caching needed to keep it performant when
queried, but is itself a very low hit rate site).

Quickgit runs on a donated container, on the anongit node Serenity. It
shares this with nothing else. There are no load issues here.

Chiliproject is extremely suboptimal, and a massive resource hog. It
runs on a separate machine again.

I don't expect there to be any issues requiring us to cluster or move
beyond one server hosting Phabricator.

>
>
>> I appreciate this is important to you, but it isn't as critical to others.
>
> It has it's charme to not have to understand and use a second D/VCS and
> switch around between git and arc logics.
> The report seemed to suggest this was "in production"?

What was in production? The hack allowing for reviews to be posted
using Git itself? It isn't at this time...

>
>
> Cheers,
> Thomas

Thanks,
Ben