Kdevelop4 gsoc

[Andreas Pakulat]

On 11.05.08 03:52:42, Aleix wrote:
> On 5/10/08, Andreas Pakulat <apaku at gmx.de> wrote:
> > On 06.05.08 21:41:12, Manuel Breugelmans wrote:
> >  > 1. How will the different test(cases) be discovered by the framework?
> >
> >
> > What exactly does this mean? So far I only know QtTest, JUnit and Squish
> >  as testing frameworks and the first one do "discovering" by using a
> >  special base class, the latter by having a config file describing the
> >  testcase scripts. Running of tests is always done by executing a binary
> >  (eventually with some special parameters).
> >
> >
> >  > a/ let this be the responsability of the user, ie require him/her to write a
> >  > seperate piece of code that collects all the tests
> >  > b/ re-use the framework specific registration method, eg for CppUnit this is
> >  > basicly built-in
> >  > c/ present selectable list of known classes, deduce testcases with the help of
> >  > duchain?
> >  >
> >  > The first option is obviously very flexible but not user friendly. The second
> >  > is easy for certain frameworks eg it's builtin in CppUnit, however QTest has
> >  > no such facilities. The last seems do-able as well, it will need some
> >  > adaption for the C framework, Check (no classes).
> >
> >
> > Well, b+c should both be done, eventually moving c) to the end of the
> >  Soc, depending on your plans and time. Wether a) is needed mostly
> >  depends on the answer to the above and wether there are cases where you
> >  want full control over "discovering testcases".
> Well, maybe we can provide the test case executables from the buildtool.
> 
> CMake can do that (because add_test() is used) so if you want, I can
> expose it easily. QMake will do it easy also probably, I don't really
> know about others.

The above is not about the buildsystem-support of creating the tests,
but rather how to tell the testing framework that the files/code X,Y,Z
are testcases. Apparently this differs quite a bit between different
testing frameworks.

> >  > 2. How to compile the tests, what kind of user input is required here?
> >  >
> >  > a/ just let the user wite his own build rules with system of choice
> >  > b/ automate it for cmake based projects? request only include dir's and
> >  > libraries/objects to link to.
> >  >
> >  > Option a is the easiest implementation wise but not so user friendly. If this
> >  > route is taken 1. is a non-issue ... Maybe support this first and extend to
> >  > b/ later.
> >
> >
> > Both are needed IMHO. a) can be easy in larger projects such as KDE4
> >  which already have unit-tests and thus its basically just a one-liner to
> >  add a new test. However if you're not in such a project it would be nice
> >  to use the buildtools. Using the Buildsystem API you can even do it
> >  easily for all buildsystems that kdevelop supports, because you can just
> >  tell the buildsystem to add a new target item with a couple of sources
> >  attached.
> Well, you already know how I feel about editing user scripts, I don't
> really think that messing with it is a good idea...

Well, thats why I said both need to be there. People that generally use
the GUI for their buildsystem will use b), others that know their
buildsystem will use option a) anyway :)

> But anyway, my point is that you shouldn't do it buildtool dependent,

Exactly my point, use the API we already have:
BuildSystemManager::createTargetItem()

> >  One thing I just thought of: Are different testing-tools/API's going to
> >  be different plugins or do you want to encapsulate all in one plugin?
> >  IMHO the former is better, using common widgets and stuff available
> >  from a library (maybe even more than just that), because it allows to
> >  more easily add support later for other testing tools - specifically
> >  using scripting languages. I guess I should really try to look into the
> >  code, but my time is really limited right now (hopefully not anymore so
> >  in a week or two).
> >
> >
> >  > 3. How to determine the entry points and actually execute the tests?
> >
> >
> > I'm not sure what you mean with entry points.
> >
> >
> >  > a/ run seperate executables for the different tests, not optimal at all ...
> >  > slow
> >  > b/ use dynamic linking to inject the freshly compiled test objects eg with
> >  > QLibrary/QPluginLoader
> >  >
> >  > Option b is preferable for sure, easy for CppUnit. Much faster both while
> >  > compiling and linking + when actually executing. Trouble with QtTest since it
> >  > only supports building executable(s). Might get funny with PyUnit ... maybe
> >  > it's better to wait for Kross support?
> >
> >
> > Executing tests should IMHO be done the same way its done without
> >  kdevelop, we just want to make it easy to do so - i.e. right-click a
> >  testcase in the GUI and say "execute", eventually even doing that for
> >  just one of the slots in a QtTest (the executable test does support
> >  this). Then we should display the results in a nice way.
> >
> >  I don't see people using KDevelop to run all the tests in larger
> >  projects - thats something that nightly test runs should do. However I
> >  think people will want to run those tests that test the components they
> >  just changed.
> >
> >  Also I'm not sure what you gain from linking all testcases into one
> >  executable, except for the case where you want to run all/nearly all of
> >  the tests.
> 
> I think we don't have to force the user to compile the tests in any
> strange way, KDevelop just would have to understand which one is the
> one that the user wants to run in each case.

Not compiling, execution. Compiling needs to be done automatically
before execution. And execution can be easy too, either we present
test-targets specially in the project manager or we have some other gui
which presents test. Then the user can select those that should be
executed...

> >  > or (ii) duplicate (ie fork) QTestLib and make the internals public,
> >
> >
> > No, definetly no. Thats not an Option.
> >
> >
> >  > (iii) motivate Trolltech to expose more of the internals, not going to
> >  > happen.
> >
> >
> > Not before Qt5, as that would most probably be binary incompatible.
> >
> 
> Why don't just let KDevelop adapt to the environment? We have good
> tools around, just benefit from them all in a productive way!

Not sure what you mean here? Adapt to what environment - how QtTest
works?

Andreas

-- 
You will be surprised by a loud noise.