Massive Konqueror Regression

[Michael Pyne]

On Wednesday 17 August 2005 23:14, David van Hoose wrote:
> What isn't clear enough to me is why code isn't tested AT ALL before
> going into beta.

> Does KDE undergo alpha testing? Due to its opensource 
> nature, I will go out on a limb here and so no it isn't.

As I have already mentioned, KDE goes through multiple alpha releases, 
followed by multiple beta releases.  As do most opensource projects for that 
matter.

> Patches come in 
> untested. There needs to be someone that does alpha tests on each patch.

They don't get committed before they get tested.  Unfortunately usually it's 
the single developer who does the testing, but they hardly end up in 
Subversion untested.

> The best 
> regression testing is knowing what you're modifying, and checking to see
> if it breaks anything.

Believe it or not, we don't break stuff on purpose.  We may have things broken 
during a development period, intending to fix it, but if it doesn't get 
fixed, it gets reverted.

> I'll stop trying to make this clear now since if it isn't now, it never
> will be.

No one has been saying testing was a bad idea.  We'd love testing.  We'd love 
a good regression suite.  But it is hard to programmatically test a GUI for 
b0rkenness.

Here's the conversation so far as I see it:

<David>: Boy, you guys should really test your stuff.
<KDE Devs>: We do.
<David>: Well, like with regression tests too.  Especially that shitty KHTML.
<KDE Devs>: Uh, KHTML has a shit ton of regression tests, wtf are you on 
about?
<David>: Well, not KHTML, but the rest of KDE.  It needs them too.
<KDE Devs>: Well duh, are you volunteering to help?  We could use the help.
<David>: Boy, you guys should really test your stuff.
<KDE Devs>: ...

Or in other words, we *know* that stuff needs tested, but *you as a user* 
don't want just developers doing it.  Far too often actual bugs a developer 
experiences either get worked around or ignored without so much as a thought, 
whereas a *user* testing it would notice immediately.  How many developers 
have uncommitted patches on their hard drive to fix a little quibble?  How 
many developers simply ignore a module build error since it's happened 29/30 
days this month.  I'm just as guilty as the rest, but the reason is sheer 
human nature.

What's worse, a developer typically has only a few possible configurations he 
can test with.  Whereas the multitude of potential users out there has 
practically exponentially more combinations.

So what we need are *users* to test our software during the alpha and beta 
releases before they hit stable.  You *will* end up testing it sooner or 
later, why not make it sooner that way you don't have to wait for KDE 3.5.1?

Let's make it incredibly clear:

*     WE     *
*    WOULD   *
*    LOVE    *
* REGRESSION *
*  TESTING   *

But we (as a developer group) don't have time to do it all ourselves.  That 
is, after all, the whole point of open development and the many-eyes 
philosophy, is that users can chip in and test as well.

Regards,
 - Michael Pyne