Thoughts about a better Quality Management process for Plasma

[Aaron J. Seigo]

On Sunday, January 13, 2013 15:50:25 Martin Gräßlin wrote:
> I think the problem with our QM process is, that we don't have a tool to
> support it. Our bugtracker is (in it's current state) totally useless. Let
> me just show a few stats for Plasma:

the difference betwen Plasma and KWin components on bugzilla is simple to figure 
out: 2 developers are looking into it for KWin, which has half the (new) 
reports; the last half of 2013 we had ~0 developers looking at it for Plasma. 
i'm surprised the #s are even as good as they are.

the stats offered are also a bit oversimplified. 2x the reports is not 2x the 
work. plasma not only has many more components in it, the reports also cover 
many, many more dependencies. "Plasma" is often a place where kdelibs bugs are 
reported, for instance, since that's where people notice them. Plasma has more 
individual use cases than kwin.

so the idea that if it takes 2 people to handle kwin's bugs, 4 can handle 
plasma's is perhaps an underestimate. 6 people doing 5-7 reports per week is 
perhaps closer to reality. this matters because:

> What is an unconfirmed bug? It's a bug which has not been
> reproduced/verified that it is valid! A bug report should not stay in this
> status for a long time. Our users spend time to report it, so we should
> process them.

we need some people who use master, but aren't responsible for writing (much) 
code, doing this work. otherwise those writing the code will simply get 
nothing done.

> What is a bug in component general? A bug that applies to everything of
> Plasma? Impossible! There is no general. A bug in general means that it is
> unknown where it belongs to. It should be moved into the best fitting
> component. If it doesn't exist create a new component! But don't keep it in
> general - nobody is responsible for those bugs.

requires very active triage.
 
> Now the action items I would suggest to do:
> 
> *Re-evaluate all components for Plasma*

given that you suggest closing all reports later on, perhaps we should take 
this step when we do that.

> *Find a maintainer for each component*
> The current maintainer model of Aaron and Marco for everything doesn't
> scale. We have so many people who could take over maintainership for a
> small component.

agreed; this means keeping documentation of this current, of course. perhaps a 
role for the "super maintainers" you mention next.

> Whoever is maintainer of a component, should become the
> default assignee for bugs in that component. No longer a one address for
> all assignee.

i think the maintainer should be added as a default CC, but if each component 
goes to a single person's inbox, then it means we rely on them to watch it. it 
also means that when maintainers change, we have to go through and change 
ownership of tons of bugs .. meh.

i think it also discourages participation by people other than the maintainer: 
all the bugs are already assigned to someone, after all.

> *Assign some "super" maintainers*

+1. such person(s) would also be the ones responsible for ensuring cross-
component coordination happens. they don't have to DO it, but they need to 
ensure it gets done.

e.g. if everyone gets together spontaneously on IRC to discuss the goals for 
4.11, awesome! but the "super maintainer" is charged with noticing if that 
doesn't happen and either organizing such a thing or poking the right people 
who can.

> *Restrict the available version numbers*

+1

> *Close all existing bug reports*

for the first Workspaces 2 release, i agree this would be sensible. unpopular, 
but sensible :) so we'll need to do some serious communication before doing 
this.

> *Every commit should be referenced to a bug*
> What is the motivation for a commit? It's either a bug fix or it is a new
> feature/improvement. If it's a bug it's clear that there has to be a bug
> report for it (out of experience: there is always a bug, if not: create it).
> If it's a feature, it should also have a feature request in the tracker.
> Create it yourself if you need to. Sounds beaurocratic, but it comes quite
> handy as it allows to generate changelogs from it, allows people to easily
> test new features.

a few challenges to this:

* many features, and even some bug fixes, are large enough to warrant multiple 
commits. yes, we can CCBUG every single commit, but that seems really overkill 
(and is going to make my inbox bulge ;).

* it means adopting bugzilla as our One True Workflow tool. that doesn't just 
sound beaurocratic, it is beaurocratic. if someone appears with a patch fixing 
some bug or implementing some feature that isn't in bugzilla, do we first send 
them to create one before accepting it? ugh.

i think one of the main motivations behind this style of development is the 
practice of doing development in a shared branch (e.g. master).

if development instead is done in separate topical branches, the branches 
become the documentation. it also works with the regular development workflow 
without causing trips to bugzilla (or redmine, or whatever tool).

a merge commit can document exactly what it is merging, feature or bug fix, and 
that can also be used to generate a changelog.

> *Bug fixes should come with a unit test*
> Plasma so far does not have any unit tests.

only in kde-workspace. libplasma does indeed have some ... but certainly 
nowhere near a majority of code paths covered.

> Why? Let's not fool ourself with
> "it's UI, that's difficult to test". No it's not, because Plasma has a
> great separation between UI and logic. It would not be a problem to create
> a unit test for the data engines.

for the non-GUI bits, i agree. one wrinkle in this, however, is that much of 
this code is async, and this is only going to be more and more the case in 
future. so if this is going to be a goal, then we probably need to invest some 
time into making it dreadfully simple to write tests that require the event 
loop and waiting for responses.

-- 
Aaron J. Seigo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/plasma-devel/attachments/20130114/bacab203/attachment.sig>