Massive Konqueror Regression

[Geoffrey Huang]

On Thu, 2005-08-18 at 12:12 -0600, Aaron J. Seigo wrote:
> On Thursday 18 August 2005 11:22, Geoffrey Huang wrote:
> > I realize I'm jumping into the middle of the conversation here, but
> > quality is something I feel strongly about.  W.r.t. Aaron's call for
> > more human testers, I agree that that certainly would help.  However, I
> > don't believe you can guarantee quality just by asking people to use a
> > pre-release version. 
> 
> in the real world, when producing mass market software one can't guarantee 
> quality, per say. one can only create mechanisms that help ensure quality. 
> automated regression testing is one tool that covers some of the problem, and 
> for the parts it can cover it's invaluable.
> 
> > Quality -- that is to say, the prevention of 
> > regressions -- relies on repeatable tasks for sane results.  This means,
> > of course, that the human testers would need to follow a fairly rigorous
> > test plan.
> 
> this is true if we have just a few people doing it. if there were enough 
> people using KDE from SVN on a day to day basis, statistically we'd get 
> rather good coverage. but if we had a small number of very active testers 
> (e.g. a q/a department that was funded) then building task lists would be the 
> first order of business.
> 

I have to disagree about this.  If we assume that the distribution of
probabilities for code executed was uniform, then you'd be correct:
given a moderate number of testers, all code paths would get executed.
I'd be willing to bet, however, that the majority of KDE users will
execute the same areas of code.  Specifically, I'd bet that error cases
don't get executed enough, nor do advanced features.  Having a test plan
would ensure that all parts of the code get executed.  This also
supports the argument for automation of these tests, since it'll become
tedious to execute all corners of the code.

Still, if the case is that there aren't good (or readily accessible)
automation tools out there, the very least is to have a test plan.

-g

> > To be honest, it really makes more sense to me to automate regression
> > tests, and have humans concentrate on testing new features. 
> 
> it's not an either/or ... regression testing where that makes sense ought to 
> be done. i don't think anyone has an issue with that. those tests that are 
> missing still need to be written, of course. but i'd be happy to see you 
> manage to automate the regression testing of handheld devices with kpilot, or 
> automate the regression testing of much of kicker.
> 
> > I've done 
> > automated UI testing before, albeit on Windows.  Are there no UI testing
> > tools that KDE could use?
> 
> none that i'm aware of that are useful to the project (e.g. open source, work 
> with Qt/KDE classes, etc)
> 
> _______________________________________________
> kde-quality mailing list
> kde-quality at kde.org
> https://mail.kde.org/mailman/listinfo/kde-quality