The state of the unit tests

[Niko Sams]

On Sun, May 30, 2010 at 18:29, Esben Mose Hansen <kde at mosehansen.dk> wrote:
> On Sunday 30 May 2010 13:49:53 Andreas Pakulat wrote:
>> On 30.05.10 11:31:45, Esben Mose Hansen wrote:
>> > In my humble opinion, failing unit tests and especially completely broken
>> > unit tests are worse than no unit test at all, since it discourages
>> > further testing.
>>
>> FACK.
>
> I don't think I know that word, and wikipedia doesn't either :o)
http://de.wikipedia.org/wiki/Liste_von_Abk%C3%BCrzungen_%28Netzjargon%29 :D

>
>
>> > 1. disabling and/or expect-fail failing tests
>>
>> I don't see how that helps, the tests are still failing/broken.
>
> It would help in this way: If I run make test and something fails, I probably
> screwed up something. That is a big help. If a lot of tests are failing before
> I begin, you'll have to dig to see if I broke the test, or if it was already
> broken.
I agree. All tests should always pass. If one test fails there are tree options:
- fix it
- mark as expect-fail and fix it later
- if it won't be fixed delete it

>>
>> > 2. some sort of automatic fingering of which checkin breaks previously
>> > passing tests.
>>
>> That requires writing a tool which updates kdevplatform and kdevelop
>> clones to roughly the same revision. Also
>
> wouldn't git pull do that? :) I was more worried about the getting the test
> result up on the web bit.
the difficulty is to know what kdevplatform revision use for what
kdevelop revision.
And publishing the result could be done using a mail sent to kdevelop-devel.
(Actually Andreas had set up that some time ago)

>> > 3. some effort to make tests that depends on externalities (I'm guessing
>> > at least vcs + gdb depends heavily on such) to check for such
>> > externalities and replaces the test with an expect fail.
>>
>> Gdb depends on a gdb, vcs doesn't depend on anything, svn-tests depend
>> on the svn libs. This are all dependencies we also have for the full app
>> so are not a problem. The one depending on user-interaction has a fix in
>> a pending merge-request.
>
> That is good, then :) And now we are down to 2 failing tests in kdevelop, gdb
> and qtprinters.
do you have gdb >= 7.0 installed? qtprinters passes for me.
gdb not, it's showing some MessageBoxes, I'll look into this issue.

Niko