Unit testing (once again)

Wed Feb 8 17:21:36 UTC 2012

On Wed, Feb 8, 2012 at 5:23 PM, Miha Čančula <miha at noughmad.eu> wrote:
> 2012/2/8 Aleix Pol <aleixpol at kde.org>
>>
>> On Wed, Feb 8, 2012 at 3:18 PM, Miha Čančula <miha at noughmad.eu> wrote:
>> > Thanks Millian, yes I am a student and I even applied for this thing at
>> > GSOC
>> > last year, but as I said I ended up working for Orange instead (which
>> > I'm
>> > still a bit sorry for). So I'm trying to make up now :)
>> >
>> >
>> > 2012/2/8 Milian Wolff <mail at milianw.de>
>> >>
>> >> I think this should be done using our plugin architecture, you can
>> >> "provide"
>> >> interfaces in a plugin and other plugins can then "require" interfaces.
>> >> That
>> >> makes it possible to do what you want I think. Take a look at e.g.
>> >> IMakeBuilder and such in kdevelop/projectbuilders/makebuilder/*
>> >>
>> >> cheers
>> >
>> > (and what Aleix said):
>> >
>> > I'm not sure I understand completely, so let me explain my thinking
>> > better.
>> > The way I envisioned the structure is:
>> >
>> >  A) One plugin (let's call it TestViewPlugin) to show all available
>> > test.
>> >  B) An interface (ITestProvider, maybe a different name) that
>> > specialized
>> > plugins (what I called "generator" inappropriately, "provider" is
>> > probably
>> > closer), which find tests, and run them. Since the tests can be nicely
>> > arranged in a tree, I think a tree structure for tests is appropriate.
>> > For
>> > CTest with Qt, this is a three-level structure (Project -> executable ->
>> > function). So the Provider(B) would generate the Test tree and send it
>> > to
>> > View(A) for display.
>> >
>> > Now plugins aren't supposed to link to eachother, so B-plugins shouldn't
>> > link to A or vice versa. However, they should be able to share data (B
>> > must
>> > tell A of all the tests it found, and A should tell B which ones to
>> > run). So
>> > I would add a shared data class (call it UnitTest or TestCase) for
>> > holding
>> > [name, executable, arguments, parent, children]. Alternatives that I can
>> > think of is having the B-plugins provide a QAbstractItemModel (which
>> > also
>> > make it hard to display tests for different projects in the same
>> > treeview),
>> > having the TestCase class completely abstract, or simply storing
>> > everything
>> > in the A-plugin. However, all of these complicate the B-plugins with
>> > code
>> > that could be shared. There are probably other options, so I'd be happy
>> > for
>> > suggestions.
>> >
>> > Millian, if I understand you correctly, you want to define interfaces in
>> > a
>> > plugin, like IMakeBuilder (and now that I did look around,
>> > IExecutePlugin
>> > and IExecuteScript) does. But that requires cross-including between
>> > makebuilder and its user cmakebuilder. However, in my case, I would like
>> > to
>> > share a concrete class, along with its implementation.
>> >
>> >
>> >>
>> >> --
>> >> Milian Wolff
>> >> mail at milianw.de
>> >> http://milianw.de
>> >
>> >
>> >
>> > --
>> > KDevelop-devel mailing list
>> > KDevelop-devel at kdevelop.org
>> > https://barney.cs.uni-potsdam.de/mailman/listinfo/kdevelop-devel
>> >
>>
>> Hi Miha,
>> I think you're messing yourself a little bit.
>
> :)
>>
>>
>> From one side the provider should list all the test and execute them
>> individually. On the other side, the view will list the tests and let
>> the user select which ones must be run at a time.
>>
>> We should decide then UI wise how do we want to run the unit tests.
>> Maybe we want to only run the tests relevant to the file that is being
>> edited, or by a subfolder or whatever. This kind of information should
>> be exposed in the interface that the provider plugins will implement.
>
> Yes, I did not think of that. So instead of having a tree structure, you
> propose
> indexing by URL, and relying on the view to display it in some way.
> I suppose because I started with the Veritas library which creates such a
> tree, I didn't
> even consider any different indexing. But it does make more sense to me.
Well, note that a path is a tree structure.
I used URL to simplify the communication. Nothing further.

>>
>>
>> To extract the information, the view plugin will load all the plugins
>> that implement the UnitTestProvider interface and ask for the provided
>> information. Here's some rough idea of what I think this interface
>> might have looked like.
>>
>> class Test {
>>   QString executable(); //maybe a launcher?
>>   QMap<QString, QStringList> cases(); //test name with the arguments
>> to pass to the execution
>> };
>>
>> class IUnitTestProvider {
>>    virtual QMap<KUrl, Test> retrieveTests() = 0;
>> };
>
> This is basically what I had in mind, except for the tree structure (and
> asynchronous operation, and checking whether this particular provider is the
> right one for a project). My question is where to put the Test class. It's
> possible to avoid it altogether by replacing it with a QPair<QString,
> QStringList> or something even more convoluted, but that looks ugly. On the
> other hand, an ugly typedef is still better than a whole separate library.
It has to be together with the interface, because the interface
depends on it. I think it's fine if it's inside interfaces.

>>
>>
>> Now what we would need is to figure out is what's the best way to pass
>> this data.
>>
>> Unit testing usability depends a lot on how people use them, in the
>> end it's just regular programs that have some semantics that we want
>> to adapt to. So we must find a good middle spot where you don't have
>> to configure how it all works and also you get to configure how you
>> want to use it, somehow.
>
> In my limited experience, it's either "run them all", "run all failing", or
> run one specific test.
> But still I think it's best if the UI is as separated from the provider as
> possible.
Well, personally I have different approaches. For example what I
usually do when working on KAlgebra is to keep running the specific
test case while coding. When it works i run the executable and if it
didn't break anything I then run the GUI tests too.

In KDevelop, on the other hand, running all tests can take quite a
while and you rarely want to run the C++ tests when working on the
CMake plugin.

>>
>>
>> Hope that helped!
>> Aleix
>
> Sure, thanks!
>>
>>
>> --
>> KDevelop-devel mailing list
>> KDevelop-devel at kdevelop.org
>> https://barney.cs.uni-potsdam.de/mailman/listinfo/kdevelop-devel
>
>
>
> --
> KDevelop-devel mailing list
> KDevelop-devel at kdevelop.org
> https://barney.cs.uni-potsdam.de/mailman/listinfo/kdevelop-devel
>