[Kde-pim] Unittest policy?

[Volker Krause]

On Monday 28 September 2009 18:03:40 Guy Maurel wrote:
> Hello!
>
> On Saturday 26 September 2009 21:20:57 Thomas McGuire wrote:
> > Hi,
> >
> > On Saturday 26 September 2009 17:45:55 you wrote:
> > > I decided to run the entire kdepimlibs test suite...
> > >
> > > This resulted in that after >20 hours I terminated the suite (almost
> > > all time spent waiting for akonadi tests to timeout) .
> > >
> > > I'm wondering if there is a policy about tests in kdepim[libs]?
> > > Or if there isn't should there be?
> >
> > There should be. And the policy should say that the tests always have to
> >  pass. Otherwise these tests aren't useful anymore to detect actual
> >  regressions, if we have to always remember which tests pass and which
> >  don't.
> >
> > If a test doesn't pass because of a bug and the bug can't be fixed right
> >  now, use QEXCEPT_FAIL to mark that as a expected bug, allowing the unit
> >  tests to pass again.
> >
> > > After my test run I think that any tests should always pass and any
> > > commit that introduces a failure is a bad commit.
> >
> > Agreed 100%.
> >
> > > Further if a test requires any
> > > special setup it is that tests responsibility to ensure that this setup
> > > is in place (in my case the akonadi db tests).
> >
> > Normally the Akonadi tests do exactly that, they set up an environment
> > that includes a MySQL server and the Akonadi server running in an
> > confined environment. If that doesn't work for you, there's a bug
> > somewhere. BTW, it looks like the KCal problem you have is local; here,
> > all KCal tests pass.
> >
> > Volker now has set up a dashboard that runs those tests nightly, for the
> > Akonadi server, for kdepimlibs and for kdepim. You can find the results
> > at http://dashboard.akonadi-project.org/CDash/index.php
> >
> > Let's have a look at which tests in kdepimlibs currently fail.
> > Maintainers of those libraries, please fix your tests!
> >
> > For Constantin:
> > akonadikmime-racetest
> >
> > For Volker:
> > akonadi-mysql-fs-servermanagertest
> > akonadi-mysql-db-servermanagertest
> >
> > For Frank:
> > Atom-/home/ctester/k/kde4/src/KDE/kdepimlibs/syndication/tests/atom/atom1
> >0_ entry_content.xml
> > 
> > Atom-/home/ctester/k/kde4/src/KDE/kdepimlibs/syndication/tests/atom/blogs
> >p ot.com_heimwege.xml
> > 
> > Atom-/home/ctester/k/kde4/src/KDE/kdepimlibs/syndication/tests/atom/heise
> >f oto.xml
> > 
> > Rss2-/home/ctester/k/kde4/src/KDE/kdepimlibs/syndication/tests/rss2/chaos
> >r adiopodcast.xml
> > 
> > Rss2-/home/ctester/k/kde4/src/KDE/kdepimlibs/syndication/tests/rss2/hilfe
> >_ us.xml
> >
> > For Christian (still around?)/Mike:
> > kblog-testblogger1
> > kblog-testwordpressbuggy
> >
> > For Allen:
> > kholidays-testlunar
> >
> > For Kevin:
> > kimap-logoutjobtest
> >
> > For Tobias:
> > kldap-testkldap
> >
> >
> > Fixing those issues would mean that all tests in kdepimlibs pass. The
> > advantage is that we can detect regressions more easily and faster. It
> > will make the tests much more useful, in combination with Volker's CDash
> > runs.
> >
> > Regards,
> > Thomas
>
> Could one transform these works to some junior-jobs?

Absolutely! The task is easy: Look at dashboad.akonadi-project.org and make it 
all green ;-)

Fixing the failing tests is one task there, but most of them seem to be rather 
random hangs/setup issues and therefore might not be the optimal starting 
point.

You'll also see lots of compiler warnings there. While the "unused 
variable/parameter" ones seem to be especially tempting, keep in mind that 
Q_UNUSED() is rarely the right fix for them, they more often point to 
unfinished code. However, the deprecation warnings most of the time have a 
rather obvious fix, namely porting to the non-deprecated equivalent, which is 
typically mentioned in the corresponding API docs.

There are also a considerable amount of valgrind warnings listed, but most of 
them are actually coming from code below Qt. So, finding out how to supress 
those would be the first step here.

Oh, and finally, there is the coverage value. To get that into the green range 
you need to write new unit tests for the uncovered code ;-)

regards
Volker
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20090929/cf12c8f8/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/