Unit Testing: Interfaces VS Implementation

[Andreas Pakulat]

On 11.06.11 15:38:54, Milian Wolff wrote:
> Hey all,
> 
> I have stumbled upon some issues while setting up the unit test 
> infrastructure: The separation of interfaces and implementation is troublesome 
> when trying to unit test it.
> 
> So far, there are at least three unit tests in kdevplatform which - imo - 
> should be revised:
> 
> * VcsBlackBoxTest (disabled for now)

This one is still a good test (if it passes or shows errors in the
vcs-impls), its just not a simple unit-test. As the name says its a
blackbox test and yes it requires a bit more effort to set it up in a
way to test everything.

That said, I don't think you need to care about implementations outside
of the stable released code, in particular not right now while you get
the whole thing up and running.

> * ProjectModelTest (requires custom [senseless?] code in project model for the 
> case of items without project/projectmanager, i.e. imo not actually testing 
> the "real" thing)

No its not senseless, quite the contrary. A test for the project model
should only ever test exactly that. It shouldn't test the project class
itself (or all the rest of the kdevplatform api). So making the project
model easy to plug out of the complete infrastructure is good to write a
unit-test for it.

Using the 'real' project impl. and the real projectcontroller impl would
mean writing an integration test - to verify that both work properly
together.

One alternative would be to provide a mock implentation of project (and if
necessary projectcontroller), that does not have any logic itself and
simply provides static values for all its getters.

> and maybe (I have to investigate it yet)
> StandardOutputViewTest
> 
> These are basically trying to test the pure interface, which of course only 
> works with some kind of implementation. The big issue is the hen-egg problem. 

For the project model test, there's no hen/egg problem, it tests the
project model in kdevplatform. If a plugin subclasses the basic item
types, that plugin needs to have its own unit-tests for those types.

> Only installing KDevplatform and running the unit tests will either fail (no 
> implementation found) or only test the (few?) implementations in KDevplatform, 
> completely ignoring everything outside (kdevelop / playground / ...).

As I said above, testing 'everything' is completely out of the scope
here. And I think vcsBlackboxTest is the only one testing all
implementations (that are installed) of the vcs interfaces. The project
model test at least is specifically for the public API and only that.
Its not supposed to test the qmake, cmake or custom makefile item
subclasses.

> So what I propose is creating some kind of public TestXYZImplementation class. 
> Implementations can then create a unit test 
> instantiating/subclassing/implementing this class and run the contained tests. 
> 
> The advantages:
> - no henn-egg problem, we test when we have something that needs to be tested
> - it's clear which implementation fails
> - the TestXYZImplementation class in KDevplatform is still only checking the 
> basic interface without any implementation details
>
> The cons:
> - test class needs to be/have public api (probably?)
> - minimal code duplication

I'm not sure I understand what you're trying to achieve with that class.
Can you provide some pseudocode?

Andreas