Test Execution [was: Re: Kdevelop4 gsoc]

[Manuel Breugelmans]

On Sunday 11 May 2008 09:38:58 Andreas Pakulat wrote:
> On 11.05.08 03:52:42, Aleix wrote:
> > On 5/10/08, Andreas Pakulat <apaku at gmx.de> wrote:
> > > On 06.05.08 21:41:12, Manuel Breugelmans wrote:
> > >  > 3. How to determine the entry points and actually execute the tests?
> > >
> > > I'm not sure what you mean with entry points.

The point where test(command/case) execution starts, be it a path to an 
executable or object code in a library.

> > >  > a/ run seperate executables for the different tests, not optimal at
> > >  > all ... slow
> > >  > b/ use dynamic linking to inject the freshly compiled test objects
> > >  > eg with QLibrary/QPluginLoader
> > >  >
> > >  > Option b is preferable for sure, easy for CppUnit. Much faster both
> > >  > while compiling and linking + when actually executing. Trouble with
> > >  > QtTest since it only supports building executable(s). Might get
> > >  > funny with PyUnit ... maybe it's better to wait for Kross support?
> > >
> > > Executing tests should IMHO be done the same way its done without
> > >  kdevelop, we just want to make it easy to do so.
> > >  I don't see people using KDevelop to run all the tests in larger
> > >  projects - thats something that nightly test runs should do. However I
> > >  think people will want to run those tests that test the components
> > > they just changed.
> > >

Well, the only thing that keeps you from running a larger set is performance. 
Tests only deliver a boost in development time when they are run often and 
thus detect issues asap, but no-one likes to wait. I was taught that 'pure' 
unit tests shouldnt even touch the filesystem for exactly this reason. Anyway 
I suppose I can ditch that principle and just use the exe's. [and maybe try 
this dlopen+no-xml with a qttestlib fork approach later :p]

One issue is the notion of running testcases (a class) versus testcommands 
(typically a method, or an instantiation with data). Right now only the 
second is supported by QxRunner, but you'll want to run a testcase in one go 
to preserve the shared fixture (eg through initTestCase). Might be a little 
tricky with the limited interface of exe's as opposed to full blown objects.

An interesting extra feature is auto-backtrace information on hard errors. 
Since Kdevelop has strong gdb support this is doable?


Manuel