Setting up a Quality Team within KDE

[Andreas Pakulat]

On 09.04.12 12:58:02, Alexander Neundorf wrote:
> On Friday 06 April 2012, Anne-Marie Mahfouf wrote:
> > On 04/06/2012 02:23 PM, Aleix Pol wrote:
> > > On Thu, Apr 5, 2012 at 1:42 PM, Anne-Marie Mahfouf
> > > 
> > > <annemarie.mahfouf at free.fr>  wrote:
> > >> Hi,
> > >> 
> > >> We would like to setup a Quality Testing Team within KDE in order to
> > >> better test our next KDE SC and make the beta process more efficient.
> > >> Attached is the first draft of the ideas I brainstormed. Source .odt of
> > >> this document is available at
> > >> http://quickgit.kde.org/index.php?p=scratch%2Fannma%2Fquality-testing-do
> > >> c.git
> > >> 
> > >> (We can make it a public Google doc if it is more convenient or an
> > >> etherpad).
> > >> 
> > >> The document roughly describes 2 areas of action:
> > >> - reinforcement (labelled "Reinforce") of existing structures (mainly
> > >> targetted to developers and mainly relevant before the beta stage of the
> > >> release). This could
> > >> 
> > >> be used as guidelines for new developers. Of course it won't be newbies
> > >> which will develop Unit Tests or check the code quality. But some
> > >> guidelines can
> > >> be useful for libs outside Frameworks.
> > >> An area of relevant reinforcement that can be currently done is the
> > >> Review process for new integration.
> > >> - new infra-structures: this is labelled "New" in the doc: this is what
> > >> I would see to happen for the 4.9 release.
> > >> 
> > >> 
> > >> A new mailing list has been set up in order to discuss all this: please
> > >> subscribe to it if you would like to be part of this
> > >> https://mail.kde.org/mailman/listinfo/kde-testing
> > >> An IRC channel also was created on Freenode:
> > >> #kde-quality
> > >> 
> > >> Please join the mailing list and the IRC channel so we can setup a plan
> > >> to start putting all this in gear!
> > > Hi!
> > > I think this project is a very interesting idea and definitely
> > > something we need. Said that, I'd like to propose some small changes
> > > on how this should be done.
> > > 
> > > In the document there are some actions to be taken that sound a bit
> > > too technical, for example it specifies reinforcing the unit testing.
> > > This is something that we should have been doing already and I don't
> > > see someone from the Quality Team going to a development team and
> > > improving that.
> > 
> > Making unit tests is the developer task (and the document mentions it)
> > and we do have solit Unit Tests in kdelibs and Frameworks and
> > kdepimlibs. However there are other libs within KDE where maybe unit
> > tests are not as present as they should (I did not research that
> > though). Also, about unit tests, it's not only writing them, it's also
> > running them. This is not done in a regular basis and needs to be
> > automated in the future and the fails need to be fixed.
> > No newbie will ever touch to any Unit Tests of course. And any action
> > will be discussed with the gurus in this field (/me does not point to
> > David).
> > 
> > We have many tools for developers which are not fully used: latest great
> > tool is Jenkins which I was aware of only recently. My label "Reinforce"
> > is to take full advantage of those existing tools.
> > 
> > (It would be cool if at Akademy there are some talks focusing on using
> > these tools.)
> 
> There is not only Jenkins, but also CDash, which integrates nicely with our 
> cmake-based build system.
> 
> I was trying to set up nightly builds for the KDE modules two years ago or so 
> (hosted on http://my.cdash.org, some of our projects are actually using it), 
> gave a talk at Akademy in Tampere and tried really hard, but there was not 
> much response to it.

Well, I'd say jenkins has a lot more to offer than cdash. Its also a lot
simpler to use, setup and understand for newcomers in my opinion. With
jenkins I can have a shell-script job which runs cmake && make && make
test and be done. Setting up a build for submission to cdash takes a lot
more effort - at least it did when I tried to do this maybe things got
easier meanwhile.

Andreas

PS: Personally I find the Web-GUI also a bit nicer, but thats really just a
personal preference.