Setting up a Quality Team within KDE

[Andreas Pakulat]

On 09.04.12 13:28:31, Anne-Marie Mahfouf wrote:
> On 04/09/2012 01:15 PM, Andreas Pakulat wrote:
> >On 09.04.12 12:58:02, Alexander Neundorf wrote:
> >>On Friday 06 April 2012, Anne-Marie Mahfouf wrote:
> >>>On 04/06/2012 02:23 PM, Aleix Pol wrote:
> >>>>On Thu, Apr 5, 2012 at 1:42 PM, Anne-Marie Mahfouf
> >>>>
> >>>><annemarie.mahfouf at free.fr>   wrote:
> >>>>>Hi,
> >>>>>
> >>>>>We would like to setup a Quality Testing Team within KDE in order to
> >>>>>better test our next KDE SC and make the beta process more efficient.
> >>>>>Attached is the first draft of the ideas I brainstormed. Source .odt of
> >>>>>this document is available at
> >>>>>http://quickgit.kde.org/index.php?p=scratch%2Fannma%2Fquality-testing-do
> >>>>>c.git
> >>>>>
> >>>>>(We can make it a public Google doc if it is more convenient or an
> >>>>>etherpad).
> >>>>>
> >>>>>The document roughly describes 2 areas of action:
> >>>>>- reinforcement (labelled "Reinforce") of existing structures (mainly
> >>>>>targetted to developers and mainly relevant before the beta stage of the
> >>>>>release). This could
> >>>>>
> >>>>>be used as guidelines for new developers. Of course it won't be newbies
> >>>>>which will develop Unit Tests or check the code quality. But some
> >>>>>guidelines can
> >>>>>be useful for libs outside Frameworks.
> >>>>>An area of relevant reinforcement that can be currently done is the
> >>>>>Review process for new integration.
> >>>>>- new infra-structures: this is labelled "New" in the doc: this is what
> >>>>>I would see to happen for the 4.9 release.
> >>>>>
> >>>>>
> >>>>>A new mailing list has been set up in order to discuss all this: please
> >>>>>subscribe to it if you would like to be part of this
> >>>>>https://mail.kde.org/mailman/listinfo/kde-testing
> >>>>>An IRC channel also was created on Freenode:
> >>>>>#kde-quality
> >>>>>
> >>>>>Please join the mailing list and the IRC channel so we can setup a plan
> >>>>>to start putting all this in gear!
> >>>>Hi!
> >>>>I think this project is a very interesting idea and definitely
> >>>>something we need. Said that, I'd like to propose some small changes
> >>>>on how this should be done.
> >>>>
> >>>>In the document there are some actions to be taken that sound a bit
> >>>>too technical, for example it specifies reinforcing the unit testing.
> >>>>This is something that we should have been doing already and I don't
> >>>>see someone from the Quality Team going to a development team and
> >>>>improving that.
> >>>Making unit tests is the developer task (and the document mentions it)
> >>>and we do have solit Unit Tests in kdelibs and Frameworks and
> >>>kdepimlibs. However there are other libs within KDE where maybe unit
> >>>tests are not as present as they should (I did not research that
> >>>though). Also, about unit tests, it's not only writing them, it's also
> >>>running them. This is not done in a regular basis and needs to be
> >>>automated in the future and the fails need to be fixed.
> >>>No newbie will ever touch to any Unit Tests of course. And any action
> >>>will be discussed with the gurus in this field (/me does not point to
> >>>David).
> >>>
> >>>We have many tools for developers which are not fully used: latest great
> >>>tool is Jenkins which I was aware of only recently. My label "Reinforce"
> >>>is to take full advantage of those existing tools.
> >>>
> >>>(It would be cool if at Akademy there are some talks focusing on using
> >>>these tools.)
> >>There is not only Jenkins, but also CDash, which integrates nicely with our
> >>cmake-based build system.
> >>
> >>I was trying to set up nightly builds for the KDE modules two years ago or so
> >>(hosted on http://my.cdash.org, some of our projects are actually using it),
> >>gave a talk at Akademy in Tampere and tried really hard, but there was not
> >>much response to it.
> >Well, I'd say jenkins has a lot more to offer than cdash. Its also a lot
> >simpler to use, setup and understand for newcomers in my opinion. With
> >jenkins I can have a shell-script job which runs cmake&&  make&&  make
> >test and be done. Setting up a build for submission to cdash takes a lot
> >more effort - at least it did when I tried to do this maybe things got
> >easier meanwhile.
> >
> What's a bit sad is that nobody talks very much about Jenkins

There's been several blogposts in the past weeks/months about
build.kde.org, but indeed its more of a "Developer-Resource" in that
it builds KDE modules and runs their unit-tests to inform developers
asap when something broke.

> Andreas, your script should be public.

Uhm I'd use this as a start:

cd build && cmake ../ && make && make test

But I don't "own" any of the jobs at build.kde.org, so you should
probably talk to the admins of the jenkins server to find out how the
builds are actually run.

Andreas