Toward time-boxed development for KDE

[Kevin Ottens]

Hi all,

--------- Executive summary

In this mail I'll be advocating the way I think we should make our release 
planning from now on. Some parts will look similar to what we were doing, 
some won't. I think doing strict time-based release would be a mistake and I 
think we should do ourselves a favor by having something more flexible.

If you don't care about what led me to this proposal you can skip the Intro 
part freely.

--------- Intro

I'm taking a white sheet for my proposal regarding the release planning in 
KDE. I think we ought to learn from the past (both the KDE3 times, and the 
path we followed toward KDE 4.0) to get to our release planning right.

In my opinion several things worked for us so far:
 - In the KDE 3 times, the fact that our release cycle wasn't strictly time 
based, it was "roughly 9 months"
 - Late during the KDE 4 development, regular checkpoints to give a beat to 
the development (the BIC monday, and the nice period of betas each months)

There's probably more (being innovative, enough room to people to implement 
features, etc.), but I think it was two strong points. And well, points which 
I particularly noticed.

Now I also think we have a couple of things to fix:
 - Communicating to the outside when our release is getting ready. I'm not 
talking about marketing here, but release management of related projects 
(like amarok for instance) or downstream like distros. That said marketing 
would surely benefit from it too.
 - Understanding our own release cycle (basically communicating to the inside 
how we work), KDE3->KDE4 was totally different to what we were used to, and 
now we have to go back to a more "KDE3-ish" way to do things. New people have 
to adapt to that etc.

Once again there's probably more to fix (some lack of driving force during the 
work to get to the release, exceptions and more exceptions, etc.), but the 
other problems we have can basically be tracked down to the two ones I 
mentionned.

--------- Proposal: time-boxed development

What I'm proposing is to use time-boxed development. In short, we're issuing 
snapshots on a regular basis and we had to this criteria to determine which 
snapshots will be betas, rcs and releases.

If we keep the "criteria" part out for a second, that means we can better 
communicate to the outside and the inside already, because we have a clear 
schedule for the snapshots. It's predictible and will happen regularly. Also, 
it means we can get more feedback from people since they can grab the 
snapshots and test.

Let's be more precise from now. Giving assumptions and definitions before 
explaining the cycle.

I'll assume two things in the following:
 - The snapshot frequency will be one month (I think we could go to three 
weeks, but I find this a bit tight)
 - Every module has a release coordinator, that builds on the new release team 
which is a good thing IMO

Now we have two thing to manage: snapshot cycle and phases transition.

1) The snapshot cycle
That's the first thing I introduced, every month we tag the codebase to 
produce a snapshot and publish the tarballs. For that, it means we're 
freezing the codebase a couple of days every month to ensure that it fully 
builds and pass all the tests (ok, I'm probably dreaming on this one... but 
one day!).

2) The phases and their transition

a) Warmup
When we're opening the development toward a new release (like going toward 
4.1) we enter the warmup phase. During this phase, the development is free 
for all, and the release coordinators collects new features for their modules 
(which makes the feature lists grow). During this phase, the snapshots are 
not named.
Once we get the greenlight from all the coordinators that they got enough 
input for their feature lists (one to two iterations in most case) we get to 
the construction phase.
At that point, the coordinators determine the lists of features that should be 
in the release for each module.

b) Construction
During this phase, only small features or the ones already begun during Warmup 
are developed in trunk. Any other feature should be developed in a branch (or 
playground for new components) and merged when ready.
Each snapshot is called an alpha. After each alpha, the coordinators can 
decide to push back some of the features at a later release if there's 
already enough new features in the module and the other ones still need too 
much work.
Once we get the greenlight from all the coordinators that the features still 
selected to be in the release are done, we enter the stabilization phase.

Only exception: for KDE N->KDE N+1 transition, the platform will go to 
stabilization before the other modules.

c) Stabilization
From now, no new feature are developed in trunk. Only bugfixes, no new 
messages, translations and artwork modifications[*] are allowed.
Each snapshot is called a beta.
When we get the greenlight from the coordinators that all the major bugs are 
fixed we get to the releasing phase.

c) Releasing
From now, trunk is deep frozen. Only bugfixes and translations are allowed.
Each snapshot is called a release candidate.
When we get the greenlight from the coordinators that all the known bugs are 
fixed we get to the maintenance phase.

d) Maintenance
We got the main release out. The branch for the release is created. Only 
bugfixing and translations are allowed in this branch.
Each snapshot is called a point release.


So in short, the benefits I see in such a scheme:
 - we're more predictible
 - we keep flexibility for KDE5 without having to push for a completely 
different cycle (it's basically just a longer feature list, in particular for 
kdelibs, and having an earlier freeze for kdelibs)
 - we share the load of the release management
 - we get toward a release based on the current status of the codebase and 
feature lists, not some external factors
 - we're better at communicating what we're doing, and the status of the 
codebase

* What's the deal with time?
Well, it's not forcing 6 months, 7 months, 8 months or whatever, and that's 
perfectly intended. That said during the construction phase, it's obvious 
that the ability for the release coordinators to push back features to the 
next release is here to ensure we release as early as possible and as often 
as possible. That means, that with experience from the coordinators we can 
expect to be able to release at a regular pace while keeping the safety net 
of flexibility (we don't want to publish anemic releases do we?).

--------- Why I don't buy "time-based release"

Mainly for two reasons.

First as pointed out at several place (and in particular in Martin Michlmayr 
PhD), there's real concerns regarding innovation with such release planning. 
That means that you would have to use a radically different cycle for major 
versions, that's somehow what we did to get KDE 4.0 and we probably don't 
want that anymore. People are change resistant, you break habits when you 
change the type of release cycle. And we don't want to prevent innovation, do 
we?

Second, to me it really looks like getting the problem backward. You're 
forcing regular releases of a given period onto the project taking the risk 
that it would never fit (too long, too short...). My take on the subject is 
that if you have time-boxed development with the coordinators per module, 
you'll tend to obtain regular releases. But this time the frequency will be 
natural to the project, not forced to X months because you rolled a dice 
upfront. Regularity ought to be the consequence, not the goal.

--------- Potential flaws and objections to this proposal

So far I listed only two:

* The extensive use of branches during construction. That said there's an open 
door regarding the definition of "small" and it's up to the module 
coordinator discretion IMO.

* The extensive number of snapshots produced, one every month. I know that can 
be a tedious process, even if it's already partly automated. Now I think it 
could be the right opportunity to:
 - automate it even more, I'd bet there's room for improvement
 - attract some people during the production of those snapshots to fix build 
problems (that looks like a lot of junior jobs, and the bug squashing 
sessions we had during the KDE 4.0 cycle attracted people doing only that)

--------- The End

Ok, now it's late and I'm tired, just hoping I didn't write too much bullshit 
in there.
Comments are of course welcome. :-)

Regards.

[*] It means, no new artwork enter the release, but previously existing 
artwork can still be modified and tuned.
-- 
Kévin 'ervin' Ottens, http://ervin.ipsquad.net
"Ni le maître sans disciple, Ni le disciple sans maître,
Ne font reculer l'ignorance."
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part.
Url : http://mail.kde.org/pipermail/release-team/attachments/20080109/ed805eac/attachment.pgp