Testing minor releases

[Martin Gräßlin]

On Wednesday 13 February 2013 12:21:14 David Edmundson wrote:
> Questions:
>   - How much is it actually a problem?
For me it's a huge problem. I'm quite afraid of pushing to branch at all 
because it's not tested (KWin is special in that regard we may never ever have 
regressions). In the end it means that users have to wait half a year for a 
(trivial) bug fix.
>   Is there a way to quantify how many
> regressions come in during minor releases?
with our current structure we cannot quantify it - bugzilla doesn't allow to 
search for something like that (keyword regression is too broad, if it's a bug 
for 4.10.1 does it mean it's a regression compared to 4.10.0 or to 4.9?)

I can give you one example of a really bad regression we had in a minor last 
year: BUG 306260

In category from 1 to 10 in how bad that was (10 being the worst one can think 
of) it is a 12 (to show that it is not a typo: twelve). If the user would have 
tested the stable branch we would have not released with the problem. The bug 
got reported about an hour after the release.
>   - How many user's do you think we could get?
Difficult to say, but we don't need many. If we can get the developers to 
provide information to the QA team about where changes have been done it 
should be easy to have few people running it.

In fact we could even just use bugzilla information (requires of course that 
devs only push for bugs in stable).
>   - Is it worth pushing for this as opposed to pushing for user's to run
> latest master? Assuming we can really only put our effort in one direction.
I think it's not mutual exclusive. There's probably a user base who would be 
willing to test, but doesn't want to run latest master (understandable).

I would suggest to set up a blog post and ask whether users would be 
interested in running it. If we get let's say at least 20 it would be worth 
the effort.
>   - Are there any other ways we can do this?
get also other distris to do that :-)

--
Martin Gräßlin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-testing/attachments/20130213/9d762548/attachment.sig>