Better testing of tagged tars

[Martin Gräßlin]

On Tuesday 12 February 2013 11:57:21 Anke Boersma wrote:
> As to point 2, having a much clearer set policy, that any distro can convey
> to their testers must surely result in less badly placed bug reports.
>  Testers who have to read documentation just to be able to use a certain
> repo, are far more likely to also read about reporting issues correctly.
> Any users that builds KDE from git, those don't result in often misplaced
> bug-reports?  Or any user who really has no idea what version they are
> running, just choose one for a bug report, isn't that more likely to happen
> then an educated testers filing an incorrect bug report?
The problem here is DrKonqui and the automatic version matching. For someone 
running master it does not matter. Most products have a version field called 
"git master" and while DrKonqui is not able to match it, the users normally 
tell so.

Now let's assume we would have the packages out in the wild for 4.11 prior to 
release. The version information reported by DrKonqui is 4.11.0 - no matter 
which tarball is currently running. At the same time there's still an RC out 
(4.10.98). Which means we cannot yet enable the version field 4.11.0 as that 
could result in incorrect data from bug reporters (we all know our users, it 
would end up in you have to ask each time whether it's really, really 4.11.0). 
No 4.11.0 in the versions means that the DrKonqui version matching magic 
doesn't work and the bug ends up as unspecified, but says in the initial 
comment 4.11.0. That creates additional work as all bugs reported like that 
needs to be re-triaged once 4.11.0 is released.

Now the problem of re-spin tarballs. Let's assume we have a crash in KWin 
(driver related) not present in the RC, but in the final tars. We fix it 
(based on general assumptions on what might have caused the crash - yeah 
driver bugs for which you don't have the hardware tend to end up like that) 
and cause the release team to do a respin. But we still get the crash reports. 
What does it mean now? Issue not fixed? User not yet updated the package? 
Looking at backtrace doesn't help - if it is driver related they look all 
different for each distribution.

Would a solution like introducing dedicated versions help here: maybe. It 
would require each developer working with such issues to track the release 
team mailing list to get the notification of a respin, the new version number 
and the matching git hash. Tricky and again with lots of work. Also 
problematic once the final version is created because the version information 
must be exactly the same otherwise Dr.Konqui magic doesn't work.

For me as the product maintainer at the point between last RC and final 
release it's extremely important to get correct information as if there is a 
crash introduced after last RC, I would have to run to the release team to 
call stop.

I can understand the need for distros to put out the packages for greater 
usage early. But from my developer perspective I must say that I would not 
want bugreports for this intermediate state. I also don't think that it in 
general would help much to have bug reports for this special stage. It should 
be only to find showstoppers and for that I doubt our bugzilla is suited at 
the moment anyway (c.f. the Plasma/Qt/Gentoo issue which went unnoticed on 
this list).

That said: it would be much more help if users would test the betas and RCs 
more - I think from our perspective that's more help.

--
Martin Gräßlin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/release-team/attachments/20130212/bd41e147/attachment.sig>