Getting to 100 % succedding tests (for 2.9), or, simply dropping them all?

[Dmitry Kazakov]

> > There is one important point: *all* the test sets should be compiled when
> > KDE_BUILD_TESTS==on
>
> Where compiled!=run? Makes sense, I agree. The whole idea of CI is to
> cover as
> much as possible, so we do not need to do things manually :)
>

Yes. Build all, but be able to choose what to run. Ideally I should be able
to run two subsets of tests: 1) for integration testing; 2) more
complicated for yearly semi-manual regression-testing.

 > 5) It would also be nice to be able to choose different subtests of one

> > executable to be in different subsets. Though I am not sure whether it is
> > doable.
>
> Could you give an example what you mean here? "executable" is not clear, as
> well as "to be in different subsets"?
>

See e.g. KisNodeTest.

It has two parts. The first consists of "integration" tests, which can be
run very fast:

    void testCreation();
    void testOrdering();
    void testSetDirty();
    void testChildNodes();
    void testDirtyRegion();

The second consists of stress tests which are brute-force checking thread
safety. They might take up to several minutes to execute and preferably
should be run in semi-manual mode yearly or something like that:

    void propertiesStressTest();
    void graphStressTest();


>> 2) Consequence of 1): from time to time we should manually check what
>> exactly is wrong with the test. Is it a pixel drift or a real problem.

> Do you happen to have a list of the tests where such external deps
influence
> the result? Or could the list be created from checking for
> checkQImageExternal()?

Everything that uses TestUtil::checkQImageImpl(). It includes almost all
tests written for last couple of years.


-- 
Dmitry Kazakov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/calligra-devel/attachments/20150205/92041033/attachment.htm>