Test Detection [was: Re: Kdevelop4 gsoc]

[Manuel Breugelmans]

On Saturday 10 May 2008 22:15:47 Andreas Pakulat wrote:
> On 06.05.08 21:41:12, Manuel Breugelmans wrote:
> > 1. How will the different test(cases) be discovered by the framework?
>
> What exactly does this mean? So far I only know QtTest, JUnit and Squish
> as testing frameworks and the first one do "discovering" by using a
> special base class, the latter by having a config file describing the
> testcase scripts. Running of tests is always done by executing a binary
> (eventually with some special parameters).

QtTests just inherit QObject, there is no root testcase. The only clue that 
you are dealing with a testcase is a call to QtTest::qExec freestanding 
function which takes a QObject pointer. Typically wrapped in a main-macro 
such as QTEST_MAIN, KDEV_TESTMAIN etc. I'm not sure whether Kdevelop can peek 
inside macros? Or optionally just their arguments, these are typically not 
numerous so could be made configurable.

> > a/ let this be the responsability of the user, ie require him/her to
> > write a seperate piece of code that collects all the tests
> > b/ re-use the framework specific registration method, eg for CppUnit this
> > is basicly built-in
> > c/ present selectable list of known classes, deduce testcases with the
> > help of duchain?
> >
> > The first option is obviously very flexible but not user friendly. The
> > second is easy for certain frameworks eg it's builtin in CppUnit, however
> > QTest has no such facilities. The last seems do-able as well, it will
> > need some adaption for the C framework, Check (no classes).
>
> Well, b+c should both be done, eventually moving c) to the end of the
> Soc, depending on your plans and time. Wether a) is needed mostly
> depends on the answer to the above and wether there are cases where you
> want full control over "discovering testcases".
>

In retrospect maybe a) should just be left out. Strong testcase detection in 
the framework specific plugins combined with some graphical selection method 
should be enough. Who will want to go and write extra code just to get a 
graphical test runner going? I know I wouldnt :) Tops some configuration but 
that gets annoying real fast as well. The main goal is to make test running 
fun and graphically attractive (alike the junit eclipse runner which does a 
great job.)

Manuel