Kdevelop4 gsoc

[Manuel Breugelmans]

First task done :) The QtTest suite for QxRunner is finished and comitted 
under kdevelop/xtest. A coverage report can be found at  
http://fenix.cmi.ua.ac.be/~p035120/qxcov/qxrunner [this also happens to be 
the functionality I want to implement in part 3]. Not quite the 90% I was 
aiming for but then the remaining classes with low coverage arent worth the 
extra effort, imo.

Next milestone is the integration of qxrunner in kdevelop, just graphically 
and able to run test primitivly. This should be done by next tuesday, 13 may.

When thats done focus lays on making the runner plugin smooth, easy and fun to 
use. Ideally things should 'just work' without much configuration but still 
powerful enough. I identified 4 major design decisions that need answering 
accompanied with some solutions.

1. How will the different test(cases) be discovered by the framework?

a/ let this be the responsability of the user, ie require him/her to write a 
seperate piece of code that collects all the tests
b/ re-use the framework specific registration method, eg for CppUnit this is 
basicly built-in
c/ present selectable list of known classes, deduce testcases with the help of 
duchain?

The first option is obviously very flexible but not user friendly. The second 
is easy for certain frameworks eg it's builtin in CppUnit, however QTest has 
no such facilities. The last seems do-able as well, it will need some 
adaption for the C framework, Check (no classes).

2. How to compile the tests, what kind of user input is required here?

a/ just let the user wite his own build rules with system of choice
b/ automate it for cmake based projects? request only include dir's and 
libraries/objects to link to.

Option a is the easiest implementation wise but not so user friendly. If this 
route is taken 1. is a non-issue ... Maybe support this first and extend to 
b/ later.

3. How to determine the entry points and actually execute the tests?

a/ run seperate executables for the different tests, not optimal at all ... 
slow
b/ use dynamic linking to inject the freshly compiled test objects eg with 
QLibrary/QPluginLoader

Option b is preferable for sure, easy for CppUnit. Much faster both while 
compiling and linking + when actually executing. Trouble with QtTest since it 
only supports building executable(s). Might get funny with PyUnit ... maybe 
it's better to wait for Kross support?

4. How to fetch the test results and pipe them into qxrunner?

this depends on the route taken for 3, obviously.

a/ if using seperate executables, produce xml results and stream parse it back 
in, which will be slow (eg with QXmlStreamReader)
b/ if using dlsym/open and friends this is easy just use some TestResult-like 
class directly

When looking at these the main problem is QtTest's limited interface. All the 
internals are private, the only real link is qExec + the convention that 
commands are private slots. To bypass this either (i) a slow hacky route 
needs to be taken (compile into executable(s) -> run seperatly -> parse XML 
results), or (ii) duplicate (ie fork) QTestLib and make the internals public, 
possible tweak a bit here and there as well. Backwards compatibility with the 
original QTestLib will be crucial obviously. This will be easy though, 
exactly because of the extremly low coupling. (iii) motivate Trolltech to 
expose more of the internals, not going to happen.


pff a wee bit too long.


Manuel