Releases in 3 months

Mon Jul 15 10:54:15 BST 2013

On Sunday, July 14, 2013 04:19:52 Inge Wallin wrote:
> Without having any scientific proof, I believe that there are 2 main
> categories of users:
>  1. Those generally satisfied who want stability
>  2. Those who long for new updates all the time.

Regardless of whether someone is a Category 1 or a Category 2 user, they want 
software with as few defects as possible.

Agreed?

> My feeling is that the first category is the silent majority and the second
> category is the loud minority.  Of course there is always a number of people
> who want just "this special new feature" to make it perfect but those are
> probably split in which feature they want and therefore still a minority.

This could well be true (I agree with you that it probably is), but it is also 
not salient to the topic. The goal is not to get features out to users faster; 
that would be a side-effect, if anything. The goal is to have a development 
cycle that allows us to fold features and bug fixes into the existing code 
bases in less disruptive ways.

That ought to please the Category 1 people a lot.

> And the users want these bugs
> fixed. If the statement above is indeed true, then the majority of users
> want to have the bugs fixed without having to suffer through other changes
> too.

I don’t see how that follows in the least.

You suggest that changes are “suffered through”; perhaps that’s a sign that 
we’re doing those changes wrong. For example:

* smaller changes introduced evenly over a period of time may be easier to 
adapt to than a large set of changes introduced all at  once

* smaller change sets may result in fewer bugs making to the user

Perhaps our longer cycles are introducing too many changes at once causing 
more problems for your Category 1 users than any # of bugs fixed in the minor 
update releases we do.

> Let's face it: upgrading your distribution is often a pain and always a
> risk. Configurations have to be redone, sometimes things stop working in
> mysterious ways, you have to redo any customizations, etc, etc. 

That’s sort of the thing we’re trying to improve on. Again, you’re arguing 
that what we’re doing now sucks and so lets keep doing it because if we change 
we might break it even further. Instead, let’s examine the actual ideas 
presented instead of posing abstract arguments against all possible 
alternative approaches.

> So what is the impact on the user of the proposal to make the release cycle
> 3 months?

Good question, the sort that we need to answer!

> Basically that there will be no more bugfixes for the end user. 

The sky will fall and the levies will break asunder!

Really the only way this statement could ever be true is if we never, ever 
made bug fixes anymore. A realistic phrasing of the possible problem might be:

 "It will take much longer for people to get the bug fixes on their system"

That sounds a lot less scary, I know, but  it also sticks to reality in a way 
that can be responded to.

> Here are some assumptions. Correct me if they are wrong:
>  - KDE developers support the last relesed and *maybe* the second to last
> release with bugfixes.
>  - Distributions have a release cycle of 6 months or longer.
>  - Distributions pick their contents 2-3 months in advance, at least

Under the current development and release system, yes.

We’re suggesting changing those systems, and that means the resulting effects 
could also change. In fact, that’s the  entire idea.

> So if a distribution picks (say) KDE 4.25 for their new relesae, then 4.26
> will come out around the same time as that distribution is released. But
> only the distro jumpers install a new release in the first few months, the
> people who want stability (the majority) will wait a couple of months
> before they do it to get the worst bugs out of the system first. But then
> KDE 4.27 is released 3 months after the distribution comes out which makes
> KDE 4.25 more or less unmaintained.

This already happens today. Judging just be reports on bugs.kde.org, and not 
relying at all on common sense whatsoever, a huge % of our user base is using 
versions of our software that is at least 1 release old and often much more 
than that. Seeing bug reports filed against versions we released a year ago is 
commonplace. So this is not a problem we will face, it is a problem we *do* 
face.

So  how can we make it better?

One way is to improve the quality of each release so that the severity and 
prevelance of bugs goes down. That’s what this proposal is about.

Yes, I put that twice, because that is such an important point. :)

But now back to the puzzle of how to get bug fixes to releases in distributions 
that people are actually using ...

We are experimenting with the idea of a “long term release” version with kde-
workspace. If 4.11 gets widely used, deployed and stabalized (as we hope it 
will) then blessing specific releases for extended #s of releases may be a good 
approach.

Right now we treat all releases as equal: every 6 month release is as 
important and as recommended as the one before it.

With Plasma Desktop 4.11,  that will no longer be the case for Plasma 
Workspaces. We are recommending that version above others, before before and 
after, for deployment purposes.

So if we do 3 month *development* cycles, but bless just one of the resulting 
*releases* each year as “this is the one you want to use if you are looking 
for stability”, perhaps then backporting efforts could focus on that one 
release.

Under that regimen, we might not even have bug fix releases except for that one 
version. It could work like this:

January: Release x.y is made, marked as an “annual” release, x.(y+1) starts
February: x.y.1 featuring backports from x.(y+1)
March: x.y.2 featuring backports from x.(y+1)
April: x.(y+1) and  x.y.3 released; x.(y+2) starts
Mayt: x.y.4 featuring backports from x.(y+2)
June: x.y.5 featuring backports from x.(y+2)
July: x.(y+2) and  x.y.6
.. etc.

These “long term release” versions could happen every 6 months, every 9 
months, every 12 months, every 24 months or whatever else strikes our fancy.

With this concept instead of caring about x.y and x.(y-1),  developers would  
instead care about the last long term release and the current devel cycle. 
This would mean developers would care about the same number of releases (e.g. 
2) but  for different time spans than they do now.

Bug fix releases for non-long-term-releases wouldn’t happen; you’d need to 
upgrade to the next release to get fixes or install the last long-term release.

Where this could fall down is if the long term release ends up sooooo outdated 
code-wise that  bug fixes simply stop. Reflecting on what happened with 3.5, 
which saw 10 releases, that may not be an issue.

It could also fall down if distributions insist on shipping the latest release 
instead of the long term releases. Some distributions certainly would do so 
because it would fit their “bleeding edge” motto, but that’s fine. More of a 
concern is that whatever time frame we pick would not work for a distro and 
they’d feel compelled to ship a release that won’t have many (or any?) 
subsequent minor releases. Or we could communicate this poorly to our 
downstreams and they’d accidentally ship the “wrong” release and we’d all look 
poorly.

Or the long  term releases could be so successful that nobody uses the more 
frequent releases, lowering testing and general involvement from the user 
community.

I list all these “could go wrongs” so that we may see if there are ways to 
address them. If there aren’t, then this idea probably won’t work  .. and if 
there are answers, then if we were to do such a thing we would then need to 
implement those answers! :)

Bottom line, however, is this: every plan has risks. So observing that there 
are risks at all is a non-sequitor. The question is whether the risks can be 
addressed to a reasonable level. In the case of “people won’t get bug fixes to 
them”, thisis a risk but  one that probably has realistic answers if we take 
that extra step beyond observing the possible failure mechanisms and ask 
ourselves if there are ways to address those risks.

> So a user that installs this distribution as above and finds a bug after 1
> month and reports it gets told "screw you, we're doing 4.28 now; we don't
> support that old shit anymore" (hopefully in nicer words though :) ).

No, the worst case scenario is they would get told “This is already fixed in 
4.28, please upgrade.” If it is also broken in 4.28 then it is relevant; if it 
is not broken in 4.28 (or can’t be replicated, which may or may not be the 
same thing :) then the worst case of “upgrade to 4.28” enters.

However, I think what I outlined above could address this concern.

> So therefore I am against the proposal. I think keeping 6 months is a good
> figure to ensure both reasonable turn-around *and* actual bugfixes of
> versions being used in the real world.

What makes 6 months a good figure?

Here are some things many of us have observed it results in:

* lots of feature dumping due to the rush to get into a release rather than 
wait another 8 months (2 months for freeze, plus another 6 month cycle).

* large numbers of features being included all at once; this is not only 
difficult for some people to adjust to when these chunks changes land, but it 
makes it harder to get a consistent level of quality in those features which 
often interact with each other

* fewer people testing at reasonable intervals; instead many people who do 
help with beta testing get a full 6 months of changes. they then report on the 
top N problems. if the # of problems is N+M, then M problems make it through. 
this is because many of the N problems overshadow, or even block getting to, 
the other M problems. by making M smaller, N approaches M and we get better 
testing feedback as measured by “the fraction of actual defects that exist 
that get reported”.

That last point, btw, is easily seen in action around beta releases of our 
current six month cycles. Beta 1 of a release will usually get lots of dupes  
of the same N problems. Beta 2 comes out with those fixed (or some  portion of 
them) and we end up with a new set of reports that are generally N in number 
as well. This can easily repeat until the number of reported problems by 
testers starts dropping below N. This does not mean there were N problems, but 
rather N + (proportion of N fixed in each beta)*(# of betas) bugs.

The entire idea that making large epochal releases works really only holds 
when test coverage is excellent and largely automated and the testing period 
is considerably long relative to the feature development period.  We don’t 
have extensive testing regimes in place (or the resources to do so) and 
spending 2 months on features and 4 months on stabilization would slowly kill 
projects due to the incentives for both developers and users and the 
competition they face. So we don’t have this .. and so our releases are not 
nearly as good as they could be .. and so some of us are asking for a re-
assessment of the processes with a mind to improving the exact issues you 
state as being important (quality) intead of repeating what we do now as if we 
are completely satisfied with the results.

-- 
Aaron J. Seigo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-core-devel/attachments/20130715/1bbf9a0e/attachment.sig>