Unit testing (once again)

[Milian Wolff]

On Tuesday 07 February 2012 18:00:36 Miha Čančula wrote:
> Hello!
> 
> I had a grand idea of finishing and integrating the unit testing plugin in
> KDevelop over the last summer. Unfortunately, I ended up working on
> something else all summer, so the testing idea got forgotten. However, now
> I have some free time on my hands again, and I started refactoring and
> partially rewriting the whole plugin.
> 
> Compared to the existing implementation, it now has a cleaner
> interface-implementation separation, a tree view similar to the one in the
> project view, and is smaller. I made a scratch repo for it at
> http://quickgit.kde.org/index.php?p=scratch%2Fmihac%2Fkdev-unit-test.git&a=s
> ummary. So far I mostly moved code around and only written a little of my
> own, but now I am at the point where new code would have to be written, and
> I turn to you for suggestions. Basically, I see three options.

W00t, you make me a happy camper :) Are you a student by any chance? I was 
thinking of mentoring this project for GSOC this year. You'd event get money 
for that then :)

> 1. Keep the whole thing in a plugin. This means a separate view for the
> test tree and their output, and it would only work with CTest. It is
> basically what's there now. For KDE/C++ it is enough, but other languages
> have their own test frameworks.

as was said: this is a no-go, the framework *must* be extensible to other 
languages and test-providers

> 3. Integrate it all the way into the Project framework, adding a new
> ITestController interface in projects. A possibility is also a custom
> ProjectBaseItem subclass for showing the tests inside the project view.

This sounds nice, but I don't think its that easy to implement cleanly. CTest 
would be "simple" since cmake knows about that. But for anything else, how 
should the project manager (generic, custom make, cmake ...) know that a 
target XYZ is actually a test?

Furthermore I think it is rather more important to have all tests, grouped by 
suite, in a separate toolview for easy execution and - most importantly - 
quickly grasping the results (what passed, what failed, ...).

> 2. Separate the general view code in one plugin, and the CTest code in
> another, with a single new interface in Kdevplatform. The view plugin
> basically loads the test runner plugin for each project. Not much changes,
> except the possibility of using other test frameworks. The interface
> doesn't have to be used from anywhere within KDevelop, only for plugin
> loading and dependency, but uses could be added later.

As such, I think this would be the option to go for.

> Personally, I am leaning towards 2, but it requires (minor) changes to
> Kdevplatform, so I better ask first. Otherwise, I'll go with 1 as it covers
> most KDE projects. The third option is there only if you feel the need to
> have unit testing integrated with other services, like automatic running of
> all tests before committing, and showing tests on the project tree. It also
> means more work, so I wouldn't do it without at least some help from the
> core team.

As I said, #2 it should be. But note that #2 also allows proper integration, 
if done correctly. But as I said, the tests should (imo) not be shown in the 
already crowded project tree, but rather in a separate toolview.

> If I get an "OK" for going the second route, I'll post a review request
> where it would be easier to discuss the new interfaces.

Yes, we definitely need to talk about proper integration and a clean interface 
I guess. I presume it would be best to develop the interface together with two 
different implementations, like ctest (very important) and something rather 
different, like some python testing framework or a php test framework.

> I am not a regular KDevelop developer, but I use it every day, so I thought
> this might be a good way of becoming one.

Welcome aboard :) I really appreciate that you are working on this!

Cheers

-- 
Milian Wolff
mail at milianw.de
http://milianw.de