KDE4Daily Post Mortem

[Simon St James]

Hi all, 

The KDE4Daily project ( http://dot.kde.org/1195829316/ ) came to an end 
yesterday with the release of KDE 4.0.0, and Julian Seward (of valgrind fame) 
was asking me how it fared in its goal of improving the quality of the 
KDE4.0.0 release.  Not being a KDE dev, I couldn't really answer that 
properly, so I thought I'd ask you (those who have been exposed to it via bug 
reports, etc) if you had any opinions.  

To re-cap, the KDE4Daily project had the following goals:

1) To provide low(-ish)-bandwidth (binary) daily updates (generally 20-50MB 
per day) of an SVN snapshot of a reasonably wide range of KDE modules ( 
http://etotheipiplusone.com/kde4daily/docs/kde4-modules.txt ) 

2) To be able to generate useful backtraces, such as this sample: 
http://etotheipiplusone.com/kde4daily/docs/konqueror-backtrace-sample.txt

(Note that inclusion of Qt debug info in backtraces was disabled by default 
due to high memory usage when loaded into GDB.)

3) To be self-contained and distro and operating system neutral - KDE4Daily 
was provided as a Qemu Virtual Image, and the updates were performed from 
within the image itself.

Basically, it aimed to be a way of lowering the barrier of entry to people who 
want to test KDE and file bugs and also to confirm that bugs claimed to be 
fixed in SVN were indeed fixed.  

Over its 7-week run, only ~5 daily updates were missed.  The homepage asked 
people to mention "kde4daily" in their bug reports, and a simple search 
reveals ~54 such bugs:

http://bugs.kde.org/simple_search.cgi?id=kde4daily

Some of these are just "this bug is still occuring in revision 7xxxxxx"; and, 
embarrassingly, were actually caused by KDE4Daily itself! The following 
weaknesses in the project were uncovered:

1) Backtrace generation was incredibly time-consuming and resource intensive 
due to the need to fetch and patch-in up-to-date debug info, which probably 
discouraged many testers from generating backtraces.
2) VMs do not provide 3D acceleration/ compositing, so these aspects were not 
tested.
3) The usage instructions neglected to detail how to enable sound, so sound 
(Phonon, etc) was not tested.
4) Generally, KDE4Daily is a very homogeneous test environment: every 
up-to-date install was effectively identical to all others, excepting the 
user's personal data/ config files.  Great for reproducing bugs; lousy for 
shaking out those little corner cases that only 1 in 100 people will ever 
stumble upon.
5) I think many end-users approached the project as a way to satisfy their 
curiosity about KDE4.0.0 and to watch it progressing, rather than as a means 
to help them file bug reports.

So basically, in light of the false bug reports and drawbacks listed above, 
does anyone think it helped/ hindered at all, and would anyone object to it 
making a return in the few weeks prior to the release of 4.1.0? An updated 
version would probably have a more efficient backtrace generation system (the 
current one is much more memory intensive than it needs to be), and will drop 
the silly partitioning scheme that has prevented many people from developing 
apps inside KDE4Daily due to lack of space - in fact, I might go the whole 
hog and make it easy to construct a full KDE4 build environment with 
updateable SVN snapshots, as the GNOME guys seem to have done.  I'll also 
ensure that a link to the homepage is built into the image this time, so that 
people have no excuse for ignoring the "Contact" section and pestering random 
people about KDE4Daily in #kde-devel :)

Best Wishes (and congrats on KDE4.0.0!),
Simon

PS I am subscribed to kde-core-devel, so no need to CC me :)