Massive Konqueror Regression

[Aaron J. Seigo]

On Thursday 18 August 2005 12:20, Luke-Jr wrote:
> On Thursday 18 August 2005 00:03, Aaron J. Seigo wrote:
> > a) a lot of those items are GUI related and require a human sitting there
> > to watch for things (unless you have a GUI regression tester suite
> > available for use ;)
>
> How about rendering to an image and comparing it with the last
> known-acceptable version? If it's different, mark it as "requires user
> check" and have a simple PHP site to allow at least 5 people to validate
> the changed render.

yes, this is what the khtml render regression tests do. but with kicker that 
doesn't work so well as it often involved things like animation (so you'd 
have to capture then compare video), order of async events from the window 
manager or response to user input. and it's not all image based either, 
imagine the test to test that scrolling wheeling through the taskbar works 
properly. it is certainly doable, but to get 100% coverage would be an 
insanely huge task. i'd also need a testing-only system where the background, 
wm style, etc never change. even just running the tests to generate the 
images would take a long time because you'd have to:

	o change settings for kicker
	o change window manager behaviours
	o change desktop configurations
	o change kiosk (lock down) settings
	o change settings for each applet

and then run sets of tests at each point. the test matrix is friggin' huge for 
kicker. and given how it changes over time codewise.. ug... it's really much, 
much more sane to just distribute the testing amongst the user base.

-- 
Aaron J. Seigo
GPG Fingerprint: 8B8B 2209 0C6F 7C47 B1EA  EE75 D6B7 2EB1 A7F1 DB43

Full time KDE developer sponsored by Trolltech (http://www.trolltech.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.kde.org/pipermail/kde-quality/attachments/20050818/9ad77a67/attachment.pgp