Another proposal for modernization of our infrastructure

[Ben Cooksley]

On Fri, Jan 30, 2015 at 10:34 AM, Thomas Lübking
<thomas.luebking at gmail.com> wrote:
> On Donnerstag, 29. Januar 2015 21:03:32 CET, Eike Hein wrote:
>
>> I think it's a real concern, and I'm wary of "we can patch
>> it away" because carrying a huge custom patch delta for UI
>> mods is what kept us from upgrading Bugzilla for multiple
>> years.
>
>
> Afaiu Jan's proposal, the default gerrit webui is "just yet another frontend
> on a REST API" (on a sidenote: looking at that CLI thing, I wonder whether
> and how hard it would be to have this integrated into kdevelop) - so I'm not
> *that* concerned here (assuming the API will remain rather stable across
> version iterations)

Please examine Phabricator's Conduit UI - it has quite a bit of
functionality as well.

>
> Given the multiple concerns on the gerrit webfrontend (not only in this kcd
> thread) I however also assume that it should be not too hard to get a
> serious improvement upstream.

Given that upstream has had multiple attempts now at an improved
interface, I would question whether they would be willing to accept a
user interface which is suitable for our needs. It appears that they
are quite comfortable with an interface many find unintuitive or
difficult to use. If they weren't, considering the number of backers
it has - one would think someone would have sponsored such an
interface.

> That includes "If we endup w/ a -hypothetical- decision between 'powerful,
> but the webfrontend sucks' and 'pretty ui, but the backend seriously falls
> short', i'd be happily willing to help on an improvement here".

Define seriously falls short (listing of features, etc).
Please bear in mind that what one person deems a major issue may be
minor or irrelevant to others because other functionality is more
important to them.

>
> (At least from my POV, it should be simpler to fix some GUI than to get a
> well scaling replication and CI backend - by the order of some magnitudes
> ;-)

Well designed user interfaces are easier said than done.
Note that we already have plans for how a well scaling anongit
replication system would be built.

As for the CI backend, please mention what is wrong with Jenkins - if
it would be integrated to check code review submissions.

>
>> cluster - do we have one? Where are we going to get that horsepower? Can
>> we keep it?
>
> That indeed is a severe issue.
> Having CI on precombined patches is a *really* great feature and, walking
> around in a candy shop, I'd immediately drop a whatsoever shiny glossy
> button UI to get one (and regardless on the actual tool - I would seriously
> want that with phabricator, gerrit, reviewboard, ...), but if the feature is
> not affordable, it's not a feature at all :-(

Correct. With Reviewboard it is too hard to implement.
Phabricator is more than capable of doing it with little issue.

>
> @Ben, Since this is mostly a peak performance limitation:
> are there any statistics on the arrival of review requests? (It doesn't help
> if we've the power during the week, but on Sunday afternoon, when everyone
> wants to submit patches, you'll have to wait until Tuesday for the results)
> - we might require some EC2 here to get this?
>

I'm afraid that information isn't really available. Having a count on
the arrival of review requests wouldnt be helpful anyway, as different
projects have different resource costs.
One review for Calligra/Krita would take more time than 10 for
kcoreaddons for instance. Sort by time taken in Jenkins to get an idea
of resource cost. Most projects are relatively lightweight thanks to
the splitting.

Also be aware that the power difference for the build nodes varies
significantly - one of them has many times more power (CPU, memory and
disk speed wise) compared to the others. This means the above numbers
can fluctuate as well depending on the node used to perform the build.

In my experience, the system is busiest during EU daytime on weekdays.

>
> Cheers,
> Thomas

Regards,
Ben