Release Criteria

[Ian Reinhart Geiser]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thursday 14 November 2002 01:54 pm, George Staikos wrote:
> This is meant to be a general criteria for release of KDE 3.1.  It covers
> only major items.  It is not a substitute for other testing!  My proposal
> is that for KDE 3.1 to ship, each item here must be confirmed by 3-5 people
> (where possible).  This will help to avoid major problems in the release.
>
>
> KDE 3.1.0 Release Criteria
>   --------------------------
>
> - All modules compile on the most common platforms we port to:
>         - Linux x86
>         - Linux PPC
>         - FreeBSD
>         - Solaris
>         - Tru64
>
This might be a problem, just because there are different configs. If we state 
the config though that causes KDE to build, then I am all for this.

I know there are some issues with KDE on solaris 8 w/ gcc 2.95 that are gone 
with 3.2, but most installs still use gcc 2.95... so there may be some issues 
there.

> - Version numbers are set appropriately
>
Makes sense
> - Library versioning is set appropriately
>
Makes sense
> - Binary compatibility is preserved
>
Do we have an automated way of testing this?  It would be great to have a make 
bctest to run when we are doing changes here local.  

> - KDE 2.2.x -> KDE 3.1.x upgrade is possible and relatively bug free
>
Again, do we have an automated way to test this?
Also dists to special things that may complicate this test.

> - KDE 3.0.x -> KDE 3.1.x upgrade is possible and relatively bug free
>
See the above comment.

> - Konqueror works on various popular websites such as:
>         - www.kde.org
>         - webmail sites
>         - search engines
>         - news sites
This is kind of cumbersome.  Not that I disagree with it, but its slow and 
annoying testing that needs to be done and done in the same method every 
time.

>
> - KMail basic functionality is present
>         - Requires approval from the KMail team for release.
>
> - No processes are left lying around on exit
Makes sense, but again, there should be an automated way to test this.  Like a 
test script that will start up every KDE app and then exit it to see if its 
exited cleanly.

>
> - No broken signal/slot connections are present in application debug output
This again, should be automated like above.

>
> - No DCOP calls are incorrect and failing
Yes, this is important as scripting KDE takes off.

>
> - No background processes are crashing silently (kded, etc)
>
How would we test this?  Are there testcases that cause these processes to 
crash?

> - Each application installed should be tested for basic functionality.
>         - CVS module maintainer should approve that each app works at least
>           minimally.
This I think is key, id be willing to put up a simple php script that allowed 
each developer to sign off on their module before it was relelased.  This 
also should help us identify unmaintained code faster.

>
> - File permissions are correct
Again we should have a script to test this.

>
> - Headers are being installed if needed, not installed if not needed
Makes sense

>
>
> Note: KMail and Konqueror (especially with KHTML) are very important and
> this is why they have separate entries.  I think they must be the most
> commonly used apps of all (kicker is more a part of the desktop than an app
> in the eyes of the user).
This may be a can of worms though... I am sorry to say I dont have good ideas 
on how to test it though.

> I'm sure that there will be some documentation and i18n feedback too.  I
> don't know enough about those processes to insult the team members with an
> attempt at determining what makes their work "ready to ship".
I am a big fan of automated testing.  If we could automate a great deal of 
these tests or at least have a very verbose checklist for each test that 
would rock.  I always marvel at how I can miss stuff when I do it over and 
over again.  If we can automate or document it I think we can increase 
reliabilty of the testing.

Just my 2c... but i agree we need something like this, and am willing to help 
with input and code.

- -ian reinhart geiser

- -- 
========================================
May a Misguided Platypus lay its Eggs in your Jockey Shorts
========================================
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE90//QPy62TRm8dvgRAjDnAJ4yjW0SNhJT32XDVBxpAxGJtHsyAQCeIysm
wYvGEYf1VBaLdSscYiSh+zU=
=0VFD
-----END PGP SIGNATURE-----