CI Requirements - Lessons Not Learnt?

[Martin Gräßlin]

Am 2017-01-05 11:20, schrieb Ben Cooksley:
> On Thu, Jan 5, 2017 at 10:28 PM, Martin Gräßlin
> <privat at martin-graesslin.com> wrote:
>> Am 2017-01-05 09:44, schrieb Ben Cooksley:
>>> 
>>> Hi all,
>>> 
>>> It seems that my previous vocal complaints about system level /
>>> serious impact dependency bumps on the CI system have gone completely
>>> unnoticed by (some) members of our Community.
>>> 
>>> This was demonstrated earlier this week when components of Plasma
>>> bumped their version requirements for XKBCommon and Appstream-Qt -
>>> without even a thought about notifying Sysadmin or checking which
>>> version the CI had, until their builds broke.
>>> 
>>> Neither of these is easy to fix at this stage, as the system base is
>>> now too old to receive updates such as these. Base upgrades require a
>>> full rebuild of everything on the CI system, and usually involve
>>> significant additional churn and is a process that must be done
>>> roughly twice a year, depending on dependency bump demands.
>>> 
>>> Does anyone have any suggestions as to how we may avoid this in the
>>> future?
>> 
>> 
>> I have a few questions here:
>> 
>> 1) Where is this requirement to check with sysadmins codified? So far 
>> I was
>> only aware of dependency freeze.
> 
> It's been codified since the PIM Qt 5.6 / WebEngine debacle, where
> Sysadmin had to rush delivery of Qt 5.6 to the CI system because the
> whole of PIM broke when they started using QtWebEngine. That was
> around March/April 2016, my mail archives can't seem to find the exact
> thread though.

I'm sorry Ben, but I fear "sending out a mail about an issue with PIM" 
doesn't
qualify as codifying it. Given what we have it looks like this did not 
reach the
target audience. And neither will this thread.

This needs to change the process, the way KDE develops software. It 
needs to be
listed in the release schedule (is not, I checked), maybe reviews need 
to be
acked by release managers, etc.

> 
>> 2) How can we easily check what build.kde.org has? Looking at cmake 
>> output
>> is not a sufficient way as it gives me wrong information
> 
> If CMake is outputting wrong information, then your CMakeLists.txt
> can't make the appropriate decisions as to whether the available
> version is suitable, so i'd say you've got bigger problems here that
> need to be addressed first.

Cmake feature summary says: "required version >= 0.5" and that's for all 
found
depeendencies. Unfortunately no information at all in the feature 
summary about
the actual version.

> 
> In any case, you can see the Docker log of the container being
> generated at https://build.kde.org/job/create_ubuntu_slave-ange/

and where do I find this information if I would not have asked in a 
mail?
This is very related to properly codifying and documenting such 
requirements.

You cannot tell people "don't introduce new dependencies", without 
telling them
how to check.

> 
>> 3) What should we do when build.kde.org does not have the requirement?
> 
> You have to file a Sysadmin ticket, also tagging the project
> 'build.kde.org' at the same time.

And then? What's the process then? How long do we have to expect this to 
go?
Would it allow to block a finished feature or an important bug fix? 
Would we be
forced to write ifdef hackery?

Sorry, but I'm not thrilled by this process.

What matters to me is getting out good software to our users. And for 
that I have
a hard requirement I have to hit: dependency freeze.

> 
>> 
>> It should be rather obvious that we don't introduce new dependencies 
>> because
>> we like to. There is a very important software reason to it.
>> That's the case for the xkbcommon dependency increase. Should I have 
>> let the
>> code broken as it was, expecting half a year of bug reports till
>> build.kde.org has the base upgraded to Ubuntu 16.04?
> 
> That's what #ifdef is for...

I see you volunteer to:
1. write the ifdef
2. adjust the unit test to skip
3. Inform distros that the reported minimum version is wrong, that in 
truth it
   requires a newer version than reported
4. handle all the bug reports related to it

If not, please don't suggest ifdef. We all know that it comes with a 
huge cost.
A cost I decided is too high in this case. After all I had many people 
complain
about it and you can imagine how annoyed I am about the broken build.

If it were as simple as an ifdef, we would have done it, wouldn't we?

> 
>> 
>> If I have to degrade the quality of the product for serving the CI, I 
>> and
>> all users have a problem. And this is currently the only alternative. 
>> The
>> quality of our product is highly at risk as our changes are no longer
>> compile tested. This is a huge problem for the release of Plasma 5.9. 
>> On the
>> other hand I cannot revert the dependency change as that would break 
>> tests
>> or introduce the broken code again. So actually we are caught between 
>> a hard
>> and a rock place.
>> 
>> When I increased the dependency I had the dependency freeze of Plasma 
>> 5.9 in
>> mind. That's the one target I have to hit from release process 
>> currently.
>> Also I had to consider a social aspect here. I asked xkbcommon devs to 
>> do
>> the release. I would have feeled ashamed if we asked for the release 
>> and
>> then don't use it. For me it was from a social point of view a very 
>> high
>> requirement to ship with the dependency in the next release after 
>> xkbcommon
>> release.
>> 
>> If we have to wait an arbitrary time till build.kde.org has upgraded 
>> the
>> base, maybe the choice of the base is not sufficient. E.g. I asked 
>> openSUSE
>> about this dependency weeks ago. Actually a few days after xkbcommon 
>> had the
>> release and it was already shipped in tumbleweed. Similar for Mesa 13 
>> which
>> I'm also eagerly waiting for build.kde.org to fetch it.
> 
> Mesa 13 is news to me.

Oh we talked about it months ago when I told you that Mesa 13 will have 
a new way
to get a surfaceless context. That was prior to the release of Mesa 13, 
though.

Cheers
Martin