[Kde-pim] Unittest policy?

[Thomas McGuire]

Hi,

On Saturday 26 September 2009 17:45:55 you wrote:
> I decided to run the entire kdepimlibs test suite...
> 
> This resulted in that after >20 hours I terminated the suite (almost all
>  time spent waiting for akonadi tests to timeout) .
> 
> I'm wondering if there is a policy about tests in kdepim[libs]?
> Or if there isn't should there be?

There should be. And the policy should say that the tests always have to pass. 
Otherwise these tests aren't useful anymore to detect actual regressions, if 
we have to always remember which tests pass and which don't.

If a test doesn't pass because of a bug and the bug can't be fixed right now, 
use QEXCEPT_FAIL to mark that as a expected bug, allowing the unit tests to 
pass again.

> After my test run I think that any tests should always pass and any commit
> that introduces a failure is a bad commit. 

Agreed 100%.

> Further if a test requires any
> special setup it is that tests responsibility to ensure that this setup is
>  in place (in my case the akonadi db tests).

Normally the Akonadi tests do exactly that, they set up an environment that 
includes a MySQL server and the Akonadi server running in an confined 
environment. If that doesn't work for you, there's a bug somewhere.
BTW, it looks like the KCal problem you have is local; here, all KCal tests 
pass.

Volker now has set up a dashboard that runs those tests nightly, for the 
Akonadi server, for kdepimlibs and for kdepim. You can find the results at 
http://dashboard.akonadi-project.org/CDash/index.php

Let's have a look at which tests in kdepimlibs currently fail. Maintainers of 
those libraries, please fix your tests!

For Constantin:
akonadikmime-racetest

For Volker:
akonadi-mysql-fs-servermanagertest
akonadi-mysql-db-servermanagertest

For Frank:
Atom-/home/ctester/k/kde4/src/KDE/kdepimlibs/syndication/tests/atom/atom10_entry_content.xml
Atom-/home/ctester/k/kde4/src/KDE/kdepimlibs/syndication/tests/atom/blogspot.com_heimwege.xml
Atom-/home/ctester/k/kde4/src/KDE/kdepimlibs/syndication/tests/atom/heisefoto.xml
Rss2-/home/ctester/k/kde4/src/KDE/kdepimlibs/syndication/tests/rss2/chaosradiopodcast.xml
Rss2-/home/ctester/k/kde4/src/KDE/kdepimlibs/syndication/tests/rss2/hilfe_us.xml

For Christian (still around?)/Mike:
kblog-testblogger1
kblog-testwordpressbuggy

For Allen:
kholidays-testlunar

For Kevin:
kimap-logoutjobtest

For Tobias:
kldap-testkldap


Fixing those issues would mean that all tests in kdepimlibs pass. The 
advantage is that we can detect regressions more easily and faster. It will 
make the tests much more useful, in combination with Volker's CDash runs.

Regards,
Thomas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20090926/16e06c50/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/