[Owncloud] Releases and stuff

[Klaas Freitag]

On 18.07.2012 09:30, Christian Reiner wrote:
Hi,

> I'd like to add two cents to the valuable points Thomas&  Thomas made:
>
> On Wednesday 18 July 2012 02:00:27 Thomas Tanghus wrote:
>> [...] release [...] how this is to be handled [...]
>> [...] the 4.0 release gave us some bad cred [...]
>> [...] really have an overview of what's supposed to be in that release [...]
>
> On Wednesday 18 July 2012 02:00:27 Thomas Müller wrote:
>> [...] some kind of wiki/doc/blog [...]
>> [...] I would have liked to see some concept paper on that in advance [...]
>
> Also others have mentioned ideas going into the same directions.
>
> I partly followed the discussion about test principles and coder
> responsibilities and all here on the mailing list. Many valuable things have
> been said. However there is one thing that no one appears to think about,
> which looks a little odd to me:
>
> Those approaches to testing and quality enhancements I noticed all targeted at
> the existing team. That means either doubled work load (which won't fly) or
> slowed down progress (which is not wanted and fortunately not the case) or
> gaps in exactly those points all here regard as so important and required
> (which is a desaster).
>
> In my former position (I quit about a good year ago) I learned a few lessons
> if it comes to releases:
> - trusting in developers to reliable test their own code makes no sense
> ( no offense meant, I am a developer myself )
Absolutely. Coders can't test very well.

> - happily contributing until the day before release can't work out
> ( certainly all fixes with good intentions, but we all know how that is...)
> - direct and agile communication in favor over formal rules is silly
> ( formal rules and communication are completely separate )
> I learned that even in my former team of only 6 developers a formal schedule
> with things like feature freezes, milestone planning and a formal and
> documented test catalog may look bloated at first. However it clearly improves
> quality _if_ actually followed. And this is the crucial point.
> It was a hard lesson for me to learn, but at one point (one very important
> project) I brought myself to hire someone external to do the dirty work: the
> formal tracking, checking for test progress, deciding about last-minute-
> things, ... And the step turned out as one of the best I ever made.
This is the hard job of an release dude. And it feels natural to that 
position to set up a time line with milestones, translation freezes etc.

The release dude however is very much depending on testing. On the one 
hand, if it is really ruled out that devs are not longer allowed to 
commit after feature freeze, we could keep HEAD closed and suddenly 
there is a time window for doing testing. But I agree that is only the 
second best way.
It would be better to have a ownCloud community QA team who think about 
testcases, communicate with developers what is to test, make sure no 
regressions start etc. In SUSE, we once had a beta test team which 
consisted of many sysadmins with a large variety of test environments 
and a good overview what actually is important and really relevant in 
real live. That was very good. It would be a dream if we could start 
something similar here as well. In the ownCloud community we also have a 
wide variety of people with different environments and experiences.

I wonder if we have people around who were interested and what we would 
have to do to get that started.

>
> Form a small test team apart from the developers, have one single external
> person control and document the progress, including tests, implementations,
> tracking, schedule and milstone planning. And hand over formal decisions to
> that external person. This way things get more formal and may slow down a
> little. But you gain transparency, a better feeling in your stomach around
> those release dates and, last not least, quality.
Yes. But to make that work, release dude and QA team need the power of 
decision, because finally it's mostly the question what is a blocker 
bug. We kind of have the "release dude" with Frank, he also has the 
power, yet we are lacking a more intensive testing phase IMO.

regards,

klaas