CI Requirements - Lessons Not Learnt?

[Adriaan de Groot]

I think there's a middle ground to be found between dog-piling on Martin and 
letting things slide; a middle ground between dependency-creep and sticking to 
a stale platform. So let's step back for a moment and ask "what went wrong?" 
(and try to answer that carefully!), "what is the impact?" and "how can we do 
our best to prevent this next time?" or "how can we do better?".

Probably the wording "went wrong" itself is wrong, because of the emotional 
load carried by wrong-doing.

Anyway, I'll carry on while wearing three hats: as a casual KDE contributor, 
as a casual CI contributor, and as a packager of KDE. Any "we" below 
ambiguously refers to one of those three roles or communities.

The software (code) we produce is consumed by a bunch of different parties as 
source code. The core developers are up to their elbows in it every day. Other 
contributors use the source casually. But there are also automated consumers 
of the source-code, and they're a lot less smart than the people: the CI 
systems which try to build stuff all the time (and developers, we do hope that 
you see there's a value in that). And packaging systems, which thrive on 
"here's a new version, do the same as with the last version" and don't like 
radical changes.

What constitutes a radical change is something that's up for discussion: I was 
recently annoyed by KDE software that shipped a tar.bz2 instead of a tar.xz .. 
not because that's difficult to deal with, but because it's an annoying and 
unnecessary bump in my packaging workflow.

Martin seems to find this thread an annoying and unnecessary bump in *his* 
development workflow, which I can understand too. I simply won't presume bad 
faith on his part, .. possibly a lack of regard for some of the consumers of 
the source, but he has a good reason, and I'll back him in making that choice.

> >> > On Thu, Jan 5, 2017 at 10:28 PM, Martin Gräßlin wrote:
> >> >> It should be rather obvious that we don't introduce new dependencies
> >> >> because we like to. There is a very important software reason to it.

At the same time, that choice right now is causing a problem (for one 
automated consumer of the source) and may cause a problem, or surprise and 
annoy, a bunch of other consumers of the source.

Can we agree that that's a reasonable answer to "what went wrong?": "A 
dependency was unexpectedly introduced / updated."

Note I'm trying to phrase this abstractly: that's because we have a concrete 
instance here, but it's an example of something that happens all the time 
(what's that you say about libinput? or KDE applications being released with 
dependencies on totally unreleased software that needs to be picked up from 
random GitHub commits). So please separate concrete-instance with general-
case.

So what's the impact?

Twofold: for CI it causes failing builds. We should consider that red-flag 
property of CI to be a *good* thing -- because it indicates something changed 
in our assumptions or in the code, and the source is no longer good (green), 
for some suitable definition of "good".

And for distro's, there's an upcoming problem, as indicated by Kevin Kofler. 
I'd flag the same thing: it's unpleasant when packaging <X> turns into 
packaging <X>, <X'>, <X''> .. because of dependencies the packagers didn't 
know about beforehand. 

On Thursday 05 January 2017 21:49:52 Martin Gräßlin wrote:
> .. I think that this dependency is way more important to overall
> KWin than a few people having to manually build xkbcommon. Which is
> fairly trivial as David showed in his reply to the thread ..

The impact is mostly because the automated systems don't know all that much 
about dependency bumps, and also really aren't ready for manually-building-
stuff. As Martin has pointed out, there *is* now a release to build against, so 
this concrete case is not in the realm of random-GitHub-commit.

It is, however, a burden to CI and packagers. CI -- or rather the people 
behind CI, which I guess are largely Ben and Scarlett -- does its darnedest to 
give us an accurate picture of the state of KDE software. Packagers are the 
people who get the software in the hands of users.

My packaging and CI hats say "hey, don't bump dependencies, because it gets in 
the way of my workflow." My developer hat says "I don't want to be beholden to 
your idea of what the software stack is."

I claim that's not unreasonable for either hat to say. Furthermore, I claim 
that that's a source of conflict, and that we should subsequently work on 
resolving that conflict, or finding a mechanism for coping.

In other words, let's try to do better in future.

Martin has identified a problem with the CMake output (on CI, but this applies 
generally). It says that a versioned dependency is not satisfied, but doesn't 
say what version is available. Having that knowledge -- what's there -- makes 
it easier to explain why the bump is needed.

CI and packagers have identified a problem with the bump itself: they didn't 
know it happened, and need to go about discovering what's wrong (why CI is 
red, why packaging fails once the next release arrives).

Sounds like two things that can be done better; not by one person, not by 
protocols and procedures (although those can help), but by us as a community.

I'll presume to suggest (at a high level) some solutions / ways to do better.

Technically, we can go and improve CMake modules that don't provide enough 
information when version checks fail.

Socially, we can try to keep CI (through sysadmin-tickets) and packagers 
(through the distributions@ list) informed of changes in dependencies, so that 
those groups can respond in a timely fashion.

That way, we can keep everyone happy and green (er .. KDE blue) with a small 
effort.

Hugses, [ade]

PS. CI and distro's are indeed different consumers: CI wants the latest, while 
distro's take releases (and can indeed use an LTS in some cases), so the scope 
and timeframe of the impact of a problem is different.