Setting up a Quality Team within KDE

[Andreas Pakulat]

On 09.04.12 14:29:27, Michael Jansen wrote:
> On Monday, April 09, 2012 02:05:28 PM Martin Gräßlin wrote:
> > On Monday 09 April 2012 13:15:26 Andreas Pakulat wrote:
> > > > There is not only Jenkins, but also CDash, which integrates nicely with
> > > > our
> > > > cmake-based build system.
> > > > 
> > > > I was trying to set up nightly builds for the KDE modules two years ago
> > > > or
> > > > so (hosted on http://my.cdash.org, some of our projects are actually
> > > > using it), gave a talk at Akademy in Tampere and tried really hard, but
> > > > there was not much response to it.
> > > 
> > > Well, I'd say jenkins has a lot more to offer than cdash. Its also a lot
> > > simpler to use, setup and understand for newcomers in my opinion. With
> > > jenkins I can have a shell-script job which runs cmake && make && make
> > > test and be done. Setting up a build for submission to cdash takes a lot
> > > more effort - at least it did when I tried to do this maybe things got
> > > easier meanwhile.
> > 
> > I quite agree. Jenkins offers in my opinion much, much more. I have been
> > using my own custom Jenkins installation for at least half a year now and
> > that allows me to easily build KWin with various compile flags, etc. etc.
> > Also the community around Jenkins is very active and there are lots of
> > plugins available to improve the testing (e.g. cppcheck which got recently
> > added to our build.kde.org instance)
> 
> As far as i understand things here (which probably is not much), the two 
> complement it other.
> 
> CDash is in no way a substitute or alternative to jenkins. But another way to 
> congregate and present the unit test results from different builds, machines, 
> configurations side by side.
> 
> Jenkins as far is i know only presents the results of one build. But i know it 
> can congregate the test results of more than one build. At least it can for 
> java. But i am not sure it can do it for builds running on different jenkins 
> instances.

You usually have only 1 jenkins instance and a so-called jenkins-slave
on each machine that actually runs builds. The build.kde.org server is
its own slave atm, presumably because there's no further hardware
available.

In jenkins each configuration would be 1 job, each job has a number of
past builds and associated test results as well as output from various
other plugins (if configured, like static code-checkers). The website
can aggregate the information and generate graphs (or rather the various
plugins do this) to see things like test-trends etc. It doesn't matter
on which of the slaves the build actually runs.

If you don't depend on volunteers submitting their own local builds to
cdash, then this is similar to cdash, except that cdash only does the
aggregation of results. So CDash leaves setting up build infrastructure,
automatic VCS-check etc. up to you to do. Relying on volunteers to
submit their own local builds has proven to be not working well for us,
there are few if any builds regularly submitted.

> CDash can. So each jenkins build could run the test and give them to cdash.
 
Except, this would have absolutely no benefit compared to just
displaying the data in Jenkins itself :)

Andreas