Results of the QA meeting - Automated HIG tests

[Ellen Reitmayr]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Hi all,

as mentioned before I spent this weekend with some guys interested in
KDE QA. We talked about the possibilities of automated testing of
usability/accessibility criteria, and I presented the ideas collected on
http://test.openusability.org/wiki_ou/index.php/Automated

Well, they found that the first ideas described there (menu structure
etc) are fairly easy to test, the latter ones (Feedback, Interaction,
Help) are harder (basically what I expected).

- From a technical point of view, they decided to use Squish, a GUI
testing framework by froglogic -> froglogic, our host for the meeting,
already gave us a free license :)   The advantage of that framework
compared to checking UI files is that you can test *any* Qt application,
not only the ones using UI files.

Reggie already implemented the first criteria of the Menus section
(check that items in menubar consist of a single word only, max 15 items
in a toplevel menu, grouping correct (max 4 in a group), number of menu
levels max 2, menu sizes correct (min 3)). It was great to see how easy
checking for HIG compliance can be! I think that it will much increase
the acceptance of the HIG, and its actual application -> the developers
won't have to read the whole 1000 pages book, but just have a look at
the checker results for simple criteria like menus, toolbars, possibly
layout. And maybe we can even extend the criteria, once the HIG is complete.

But there was one question we spent quite some time thinking about: How
do we make the developers have a look at the results? Frerich and
Adriaan suggested to integrate the HIG results with
www.englishbreakfastnetwork.org . Currently, you'll find the API
documentation sanitizer there, but other tests such as code checks will
be added. We plan to also integrate links to the bug database, and
calculate an overall score for each application (e.g. only 10 bugs in
the database, 3 errors in the code, 2 HIG violations, 1 documentation
error will make quite a high score), and communicate the extraordinarily
positive values (a bit like 'app of the month'). The checks will be
performed daily.

All in all, I'm very happy with the results of the session :) Of course
we'll have to fine-tune the criteria, and add more easy-to-check issues
step by step, but it is a good start.


Greetings,
/el





- --

______________________________________________________

Ellen Reitmayr          email: reitmayr at relevantive.de
Usability Engineer      mobil: +49.177.3325867
relevantive AG          fon:   +49.30.23455630
Saarbruecker Str. 38    fax:   +49.30.23455639
10405 Berlin            web:   www.relevantive.de
_______________________________________________________

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDnUZ92mIgQiX+cvcRAumkAJ0Xwr3Coo1gGaJsJ+UxyupAs6OC/gCgm2b/
x+t963Zhp/r3mwVZXzxzziI=
=TtVb
-----END PGP SIGNATURE-----