Blacklisting of PIM from the CI system

[Ingo Klöcker]

On Samstag, 30. November 2019 22:48:49 CET Ben Cooksley wrote:
> On Sun, Dec 1, 2019 at 9:58 AM Ingo Klöcker <kloecker at kde.org> wrote:
> > [Intentionally top posting; CCing the CWG, but excluding kde-core-devel
> > and
> > kde-devel.]
> > 
> > Hi Ben,
> 
> Hi Ingo,
> 
> > I can understand that you are extremely annoyed by the nths disturbance of
> > the CI system by some PIM projects, but quite frankly I find your
> > reaction questionable. You seem to imply that the KDE PIM developers are
> > sabotaging the
> My reaction is based on several years of observed behaviour.

I know.

> A substantial proportion of the issues which are encountered with the
> CI system tend to be due to PIM.

I'm aware of the problems some PIM projects are causing on the CI system again 
and again.

> This includes jobs getting stuck due to tests not behaving properly,
> jobs having to be retried because new code was added to libraries and
> immediately put into use without delay (which because the CI system
> cannot sequence builds based on dependencies guarantees failure),

Okay. So there is a technical shortcoming of the CI system that makes the 
problems worse.

> and
> total failures to build in general which tend to require followup for
> them to be fixed (because the emails from the CI system are ignored)

Yes. That's apparently a social problem. It could be caused by the developers 
actively ignoring those emails. Or the emails are simply overlooked or 
postponed because the developers get too many emails and have too much on 
their plate.

> > CI system on purpose. Maybe they are careless and need to be disciplined,
> > but I'm not sure that publicly (by that I mean on mailing lists other
> > than kde- pim; of course, kde-pim is also public) accusing them of
> > intentional bad behavior ("over 2 days ago, there is no excuse for this
> > series of build failures") was called for. In my opinion, your
> > blacklisting of PIM and
> Given that projects are supposed to communicate via their mailing
> lists, and the CI system informs said mailing list that the build
> failed, with no evidence of attempts being made to fix the build, how
> else would you interpret this?

See above. I believe that the PIM developers are not ignoring those emails 
deliberately.

> > threatening of continued exclusion from CI is bordering at abuse of power.
> > You are acting as judge, jury, and executioner in one person which I find
> > disturbing.
> 
> There is no abuse of power here.
> This is the simple act of me declaring this continued state of affairs
> as unsustainable and unmaintainable and proposing two possible ways
> out of it.

I think this is a symptom of a much larger social problem, namely that for our 
CI system there are no clearly defined rules of behavior and no clearly 
defined sanctions in case people do not follow those rules. This is a common, 
recurring  theme in commons. I will propose a fix for this in a separate 
email.

Due to the lack of such rules and sanctions, you, as the person feeling 
responsible for the CI system, come up with ad-hoc rules and sanctions on your 
own. You are acting in good faith. The problem is that you are left alone by 
the KDE community in this.

> I'll note that you've not proposed any action plan here to ensure the
> problem does not occur again once it is fixed.

I'm not actively involved in the PIM development community. Therefore, I'm not 
in a position to propose an action plan how to fix the issue with PIM. As 
mentioned above, I will propose an action plan how to fix the underlying 
"tragedy of the commons"-style social problem.

Regards,
Ingo

> > On Samstag, 30. November 2019 19:14:38 CET Ben Cooksley wrote:
> > > Hi all,
> > > 
> > > This morning I went to look into provisioning a new Windows Builder
> > > node for the CI system, but hit a roadblock created by PIM currently
> > > failing to build from source.
> > > 
> > > As some background to this, we use Craft to provide various
> > > dependencies of our projects that aren't provided by Windows itself
> > > (such as Qt, gettext, poppler and so forth). From time to time this is
> > > updated to provide newer versions of software, but unless it is
> > > necessary for a project, the CI nodes themselves often aren't updated.
> > > 
> > > We use the same approach for both Linux and FreeBSD, except those
> > > dependencies are provided by the distribution for those two.
> > > 
> > > This means that when it comes time to provision a new node, all of the
> > > nodes need to be updated. As these changes essentially always break
> > > compatibility in some form or another this makes it necessary for us
> > > to rebuild all the KDE software as well.
> > > 
> > > It is therefore not possible to proceed with any of the above while
> > > something is failing to build.
> > > 
> > > Which is where the problem with PIM comes in - because it currently
> > > has many repositories failing to build from source on all platforms
> > > those builds are enabled for (including Linux and FreeBSD).
> > > 
> > > Given that the PIM project mailing list is emailed by the CI system,
> > > and that the changes in one case were pushed over 2 days ago, there is
> > > no excuse for this series of build failures.
> > > 
> > > In addition to all of the above, this round of updates was to lay the
> > > ground work for adding additional dependencies which are necessary for
> > > the builds of Digikam and SubtitleComposer to commence on the CI
> > > system for Windows. These failures by PIM have therefore indirectly
> > > harmed other KDE projects.
> > > 
> > > As this is not the first time that PIM has caused issues in this
> > > manner, I would therefore like to proceed with blacklisting PIM from
> > > the CI system. This would include prohibiting other projects from
> > > depending on any part of PIM, so those projects that have a required
> > > dependency on PIM would also have their builds removed by this.
> > > 
> > > Whilst I would prefer another solution to this, given that it is a
> > > recurring issue that makes maintenance of the CI system substantially
> > > harder, I see the removal of PIM from the equation as the only
> > > reasonable path forward.
> > > 
> > > Should the PIM developers wish to avoid this consequence for their
> > > actions, they will need to provide an action plan as to how this will
> > > be avoided in the future.
> > > 
> > > Fixing the current set of failures will not prevent this blacklisting
> > > action from being carried out - as a recurring issue it needs a
> > > permanent solution.
> > > 
> > > Regards,
> > > Ben Cooksley
> > > KDE Sysadmin