Setting up a Quality Team within KDE

[Andras Mantia]

On Thursday, April 12, 2012 10:49:10 PM Alexander Neundorf wrote:
> Yes, how good squish works for you depends on at least two things:

We also use Squish, and it found bugs and regressions in our code. 
Still, there is a big problem with it: the test needs to be maintained 
constantly. If they are not, thing will quickly escalade to the wrong 
direction, ie. you will end up with test cases that just fail and are 
hard to adapt.
The difference between maintaining unit tests and squish tests is that 
squish tests the code through the UI and it is very easy to change the 
UI in a way it breaks the tests, unless the test case is well set up 
(which means it is not just "record and save"). 

> * how well you set up your squish scripts, object map, etc. If done ad
> hoc,  you end up in an unmaintainable mess. If done carefully, it
> works.

Exactly. :) 

> * and of course, what you actually test. We don't really test that
> dialogs  etc. still look the same, we use squish to drive our
> software, see that it doesn't crash while doing that, that the
> expected actions work properly, and the produced output data is
> correct. We actually try to do things directly in many cases to stay
> independent from GUI details, i.e. we call slots and check Qt
> properties directly from the python scripts.

This is something we might not agree, as the point of squish is to test 
the app through the UI. If you test through slots, in most cases you 
might write unit tests as well.

I thing Squish tests would improve KDE's code, but this will need a 
dedicated team (which we want to set up, i know :) ). The point is, that 
Squish is not magic, you will need to think, design, write code, just 
not in C++, but in Python for example.

Andras
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-core-devel/attachments/20120416/d2c11090/attachment.sig>