Unit testing (once again)

[Andreas Pakulat]

On 09.02.12 15:48:54, Aleix Pol wrote:
> On Thursday 09 February 2012 14:39:38 Milian Wolff wrote:
> > On Wednesday 08 February 2012 21:22:26 Miha Čančula wrote:
> > > 2012/2/8 Milian Wolff <mail at milianw.de>
> > > 
> > > > Aleix Pol, 08.02.2012:
> > > > > On Wed, Feb 8, 2012 at 5:23 PM, Miha Čančula <miha at noughmad.eu> wrote:
> > > > > > 2012/2/8 Aleix Pol <aleixpol at kde.org>
> > > > 
> > > > <snip>
> > > > 
> > > > > >> To extract the information, the view plugin will load all the
> > > > > >> plugins
> > > > > >> that implement the UnitTestProvider interface and ask for the
> > > > > >> provided
> > > > > >> information. Here's some rough idea of what I think this interface
> > > > > >> might have looked like.
> > > > > >> 
> > > > > >> class Test {
> > > > > >> 
> > > > > >>   QString executable(); //maybe a launcher?
> > > > > >>   QMap<QString, QStringList> cases(); //test name with the
> > > > > >>   arguments
> > > > > >> 
> > > > > >> to pass to the execution
> > > > > >> };
> > > > > >> 
> > > > > >> class IUnitTestProvider {
> > > > > >> 
> > > > > >>    virtual QMap<KUrl, Test> retrieveTests() = 0;
> > > > > >> 
> > > > > >> };
> > > > > > 
> > > > > > This is basically what I had in mind, except for the tree structure
> > > > 
> > > > (and
> > > > 
> > > > > > asynchronous operation, and checking whether this particular
> > > > > > provider
> > > > 
> > > > is
> > > > 
> > > > > > the right one for a project). My question is where to put the Test
> > > > > > class. It's possible to avoid it altogether by replacing it with a
> > > > > > QPair<QString, QStringList> or something even more convoluted, but
> > > > > > that
> > > > > > looks ugly. On the other hand, an ugly typedef is still better than
> > > > > > a
> > > > > > whole separate library.
> > > > > 
> > > > > It has to be together with the interface, because the interface
> > > > > depends on it. I think it's fine if it's inside interfaces.
> > > > 
> > > > Just make it a pure abstract interface as well.
> > > > 
> > > > namespace KDevelop {
> > > > class ITestSuite
> > > > {
> > > > 
> > > >  ILaunchConfiguration* launcherForAllCases() const  = 0;
> > > >  ILaunchConfiguration* launcherForCase(const QString& case) const  = 0;
> > > >  ILaunchConfiguration* launcherForCases(const QStringList& cases) const 
> > > >  =
> > > > 
> > > > 0;
> > > > 
> > > >  QStringList cases() const = 0;
> > > >  
> > > >  KUrl url() const = 0;
> > > > 
> > > > };
> > > > }
> > > > Q_DECLARE_INTERFACE(KDevelop::ITest)
> > > 
> > > This looks clean, but I think at least the data members (list of cases,
> > > url, name) don't have to be virtual.
> > 
> > but then you need to link to a lib, with the above you don't need that. And
> > note that the above leaves it to the implementation on where to get the
> > cases from. For example it could be the keys of a hash map. If one would
> > need to pass the qstringlist directly to some setter function one would
> > actually waste space.
> > 
> > same for the url, maybe you associate it with an indexed topducontext or
> > something and return the url from that.
> > 
> > > > And personally, I would rather create a test controller where unit test
> > > > suites
> > > > can be added/removed. I.e.:
> > > > 
> > > > class ITestController
> > > > {
> > > > 
> > > >  void addTestSuite(ITestSuite* suite);
> > > >  void removeTestSuite(ITestSuite* suite);
> > > > 
> > > > };
> > > > 
> > > > This way plugins can easily add test suites by just implementing a
> > > > ITestSuite
> > > > class and registering it at the controller.
> > > 
> > > So you want global testproviders to be global plugins with no interface
> > > that float around, look for tests and report them? Sounds simple enough on
> > > the plugin side, but isn't that a slight waste of resources?
> > 
> > how so? what resources are wasted? note:
> > 
> > - cmake could directly, in the project manager code, report ctests
> > - all other plugins need some special logic anyways, for php e.g. I'd
> > probably put that logic into the php lang support and report testcases
> > whenever I find subclasses of PHPUnit_Framework_TestCase
> > 
> > > And the
> > > controller would have to be globally accessible, in shell perhaps?
> > 
> > yes, but it can just as easily live in a plugin. Query for the plugin that
> > offers that interface, and get the plugin's extension for that interface.
> > Just like IMakeBuilder is used.
> 
> Well... If you do that you'll always need to control the case of "there's no 
> plugin" which is odd...

Not really, since the declaration in the desktop file allows to say an
interface is required and if there's no implementation for that
interface the plugin requiring it will not be loaded at all. And if a
plugin does not work without another that implements a certain
interface, then this interface has to be a required one and not an
optional one.

The checks in the cmakebuilder and others are more or less safe-guards
against bugs in the plugin-loading code only.

So the only thing one might need to handle is the case of having more
than one implementation for a required interface.

Andreas