Keeping CI green

[Kevin Ottens]

Hello,

On Thursday 22 May 2014 21:08:05 Alex Merry wrote:
> There are currently four frameworks that are yellow (for between 6 hours and
> 8 days). kdelibs4support has had 3 commits (not counting those by scripty)
> since it broke that have done nothing to fix the issue.
> 
> If we claim that our release schedule means more stability because of our
> review process and CI system, we need to enforce it. How are we going to do
> this? Because I don't think our current system is working very well. 8 days
> is a long time for breakage to stick around when we release every month.

Agreed. That said, what else would you think needs enforcing when we already 
review most changes before commit and get the changes through the CI?

I see room for improvement in what gets evaluated when (like ability to run a 
patch in jenkins as part of the review process), I'm just stuck on the term 
"enforcing" there, not sure what you have in mind.

> I don't think Jenkins currently sends emails directly to people who appear
> to have broken something - can we make it do so? Do we want to?

AFAIK we can't do that reliably. Now, what we could do is to always have the 
maintainer in CC of breakages, expecting said maintainer to react.

I'm not fully sold on that idea, as that's likely to create a situation where 
the maintainer hunt down and point finger to the person who pushed the commit 
which broke something, while ideally it should be treated as a team thing.

The problem is that we're too good at ignoring breakages... it's a cultural 
thing, it needs changing. I didn't find the right tool for that yet, if 
someone has an idea I'm all ears.

> Do we want to have a "fix it or have it reverted" policy? How would that
> interact with failures that can only be reproduced on Jenkins (which does
> have an unusual setup)?

That's something we could do, but that means we should do something about 
flaky tests. A flaky test is a mostly useless test, so either it should be 
fixed or it should be removed...

Regards.
-- 
Kévin Ottens, http://ervin.ipsquad.net

KDAB - proud supporter of KDE, http://www.kdab.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-frameworks-devel/attachments/20140603/27e9a135/attachment.sig>