Better testing of tagged tars

[Anke Boersma]

But this is the exact point I'm trying to make.  Educate early testers to
report any issues they find within the distro.  We have done that for 3
years, and is well known and accepted by our testers (this includes testing
all beta and rc builds for Chakra).
It is just a 3-4 day period, many early test runs pick up issues that can
be distro specific build errors.  True bugs can be relayed to the release
team.  Once a version is final, then official bug reports can be filed
again.
But if any KDE dev here feels the negatives far outweigh any issue fixes
before a new release is final, then there will be no other way for us but
to stop building early tars, since there is just no way to hide our early
builds on the server.

On Tue, Feb 12, 2013 at 12:32 PM, Martin Gräßlin <mgraesslin at kde.org> wrote:

> On Tuesday 12 February 2013 11:57:21 Anke Boersma wrote:
> > As to point 2, having a much clearer set policy, that any distro can
> convey
> > to their testers must surely result in less badly placed bug reports.
> >  Testers who have to read documentation just to be able to use a certain
> > repo, are far more likely to also read about reporting issues correctly.
> > Any users that builds KDE from git, those don't result in often misplaced
> > bug-reports?  Or any user who really has no idea what version they are
> > running, just choose one for a bug report, isn't that more likely to
> happen
> > then an educated testers filing an incorrect bug report?
> The problem here is DrKonqui and the automatic version matching. For
> someone
> running master it does not matter. Most products have a version field
> called
> "git master" and while DrKonqui is not able to match it, the users normally
> tell so.
>
> Now let's assume we would have the packages out in the wild for 4.11 prior
> to
> release. The version information reported by DrKonqui is 4.11.0 - no matter
> which tarball is currently running. At the same time there's still an RC
> out
> (4.10.98). Which means we cannot yet enable the version field 4.11.0 as
> that
> could result in incorrect data from bug reporters (we all know our users,
> it
> would end up in you have to ask each time whether it's really, really
> 4.11.0).
> No 4.11.0 in the versions means that the DrKonqui version matching magic
> doesn't work and the bug ends up as unspecified, but says in the initial
> comment 4.11.0. That creates additional work as all bugs reported like that
> needs to be re-triaged once 4.11.0 is released.
>
> Now the problem of re-spin tarballs. Let's assume we have a crash in KWin
> (driver related) not present in the RC, but in the final tars. We fix it
> (based on general assumptions on what might have caused the crash - yeah
> driver bugs for which you don't have the hardware tend to end up like that)
> and cause the release team to do a respin. But we still get the crash
> reports.
> What does it mean now? Issue not fixed? User not yet updated the package?
> Looking at backtrace doesn't help - if it is driver related they look all
> different for each distribution.
>
> Would a solution like introducing dedicated versions help here: maybe. It
> would require each developer working with such issues to track the release
> team mailing list to get the notification of a respin, the new version
> number
> and the matching git hash. Tricky and again with lots of work. Also
> problematic once the final version is created because the version
> information
> must be exactly the same otherwise Dr.Konqui magic doesn't work.
>
> For me as the product maintainer at the point between last RC and final
> release it's extremely important to get correct information as if there is
> a
> crash introduced after last RC, I would have to run to the release team to
> call stop.
>
> I can understand the need for distros to put out the packages for greater
> usage early. But from my developer perspective I must say that I would not
> want bugreports for this intermediate state. I also don't think that it in
> general would help much to have bug reports for this special stage. It
> should
> be only to find showstoppers and for that I doubt our bugzilla is suited at
> the moment anyway (c.f. the Plasma/Qt/Gentoo issue which went unnoticed on
> this list).
>
> That said: it would be much more help if users would test the betas and RCs
> more - I think from our perspective that's more help.
>
> --
> Martin Gräßlin
> _______________________________________________
> release-team mailing list
> release-team at kde.org
> https://mail.kde.org/mailman/listinfo/release-team
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/release-team/attachments/20130212/b5cf90bc/attachment-0001.html>