Pre-merge CI for Gerrit

[Jan Kundrát]

On Tuesday, 2 December 2014 19:46:18 CEST, Albert Astals Cid wrote:
> Dependencies are the hard part. Any reason you didn't piggy-back on 
> build.kde.org for it?

That's right.

The reason for not using Jenkins was that the existing KDE's instance was 
not up to that task without significant changes which are underway (and I 
originally estimated that this project would be completed much sooner 
anyway) -- there was really no support of feeding it my own git ref and 
letting it test that without extra code.

Also, "using just Jenkins" would not give me what I'm ultimately looking 
for, i.e. support for gating the trunk against regressions, using 
cross-project integration etc. Another missing feature is that the Gerrit 
Trigger plugin by default tests just the change, and not how it would look 
after a merge (this is git, so a patch could be based on something else but 
a tip of the target branch). So that led me to a decision to use something 
smarter than just plain old Gerrit/Jenkins/Gerrit-Trigger combo.

The Zuul tool does exactly what I'd like to achieve. The OpenStack infra 
project uses it to launch jobs through Jenkins (in fact, through 8 Jenkins 
masters), but at that point the Jenkins' role is essentially degraded to 
"just a dumb job launcher". When doing pre-merge CI, the nice features of 
Jenkins such as plotting the trends of build warnings, extraction of test 
results into HTML tables and what not are not that relevant, IMHO. What 
matters is whether the build passes, whether the tests run, and all that in 
a well-defined manner and using some sets of policies of what sorts of 
mishaps to allow. An example -- if I want my project warning-free, I build 
with -Werror, and I don't care whether a patch introduces 50 of them or 
just 1; either it passes my policy decision of not allowing warnings, or it 
fails. If I have flaky tests, I mark them as such so that `ctest` or `make 
test` or whatever doesn't result in a failure, etc. Anyway, my impression 
was that the OpenStack infra team and Wikimedia staffers both seem to 
struggle with Jenkins; they have scripts which produce XMLs to be fed to 
Jenkins and just let it act as a form of RPC. So I can say that I wanted to 
try if I can avoid Jenkins if possible because it doesn't provide any added 
value in this particular use case, AFAIK.

One of the things missing or up-for-overhaul within KDE's CI was preparing 
job definitions in some DSL, specifying the dependencies in a mass scale, 
etc. I've heard that it's under consideration to use JJB, the Jenkins Job 
Builder, a tool from OpenStack which can equally well create both config 
for Jenkins as well as one for Zuul. This would be great, of course, and 
being aligned with an extremely vibrant development community (and, in 
fact, a family of communities) is a huge benefit IMHO.

So in the end, this Gerrit/Zuul/Turbo-Hipster eventually ends up executing 
a shell script. This shell script can be job-specific, and it should 
probably do stuff like fetching the already-built project dependencies, and 
launching the build and test cycle. I just hacked together a very quick .sh 
one for building Trojita, but I'm of course sure that the only reasonable 
path is to have a single script to be used both by KDE's Jenkins and this 
Gerrit thingy.

Cheers,
Jan

-- 
Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/