CI talk (Was: re: Manner in which kde-gtk-config development is conducted)

[Ben Cooksley]

On Mon, Mar 23, 2020 at 12:49 AM Albert Astals Cid <aacid at kde.org> wrote:
>
> El diumenge, 22 de març de 2020, a les 3:19:57 CET, Ben Cooksley va escriure:
> > Note however that images based upon Fedora or anything that shares
> > it's lineage (including CentOS and it's derivates) is strictly
> > prohibited and won't be accepted for inclusion.
>
> I still find this highly annoying since Fedora seems to be the only distro providing an almost full stack of mingw packages so for example i could add easily a mingw/windows CI to QCA using fedora but i can't because, for some reason i forgot, you are very unhappy with them.
>

I'm unhappy with them due to how they handled the complete disaster
that was a significant version update to a core system library (libc I
think) which they did in a stable, released distribution. As one would
expect, trying to do such an update broke things, in particular it
broke the BDB libraries, which are used by amongst other things, their
package manager (DNF).

Once this became apparent (which didn't take long) not only did they
not withdraw the update to libc/glibc, they pressed ahead with a
series of hotfixes to bdb and the package manager. Unfortunately these
were reliant on the underlying file system supporting a particular set
of POSIX specified functionality, which overlayfs (the file system
used by Docker) didn't support. The end result of all of this is that
it was impossible for us to generate a new CI image.

While this is already bad, this was then amplified by the reaction on
the Fedora side, where the DNF developers ignored our (and others)
report of the issue they said was fixed still being broken. Two weeks
later, and still being ignored, I complained to the Fedora Engineering
Steering Committee (Fesco) concerning this, and after a few days
someone asked someone else to rebuild the base Fedora Docker image -
which pulled in some fixes that resolved the issue finally and let us
generate an image.

Following this my complaint was closed as "resolved", even though they
had not put in place any safeguards to prevent this from happening
again, nor had they addressed the failure of the maintainers of one of
the core components of their distribution to respond to users
regarding something that was totally broken.

Unsurprisingly, within a few days the DNF developers pushed yet
another update that broke the build of the image again. I returned to
Fesco, asking for the push privileges of the DNF developers to be
suspended as they had broken it yet again.

This finally resulted in the DNF developers responding (funny that!),
but not with an apology or anything along those lines. Instead, they
engaged in a ranting attack. In this they made various errornous
claims (including that we were using an out of date Fedora image,
which was not the case and I had provided logs earlier which
demonstrated this which it was obvious they had failed to read) and
were in general extremely hostile.

Following this there was some back and forth and the issue was
scheduled for discussion at the next Fesco online meeting (apparently
they only make decisions once a week at these meetings). This time for
this meeting came and went, with no comment made on the issue.
Following further investigation I found out the meeting had been
cancelled due to a lack of attendence, which had not been communicated
to us.

It was at this point that I had finally had enough of Fedora (having
previously had to deal with their internal politics over the packaging
of QtWebEngine, which meant we ended up having to use Qt 5.8 rather
than the Qt 5.9 which we had initially targeted as the 5.9 packaging
was blocked) and dumped them for SUSE, the images that we continue to
use today.

For those wondering, my complaint against the DNF developers was
dismissed and closed as "they were responding now" with a one line
message posted through an IRC Bot about a week later when they held
their next meeting.

Suffice to say, I don't want anything to do with Fedora ever again. It
was an extremely stressful 6 weeks or so in all, with developers
chasing up when we could get some additional packages installed and us
being unable to deliver because Fedora had broken things.

This is why Fedora and all associated derivatives are banned from our systems.

> Cheers,
>   Albert

With regards to MingW, the better way to do this would be by using
native MingW rather than cross compiling. Craft already uses MingW for
some dependencies, and Krita's binary factory builds use it as well so
most of the infrastructure for this is already in position.

Regards,
Ben

>
> >
> > >
> > > Regards,
> > >
> > > - Johan
> >
> > Cheers,
> > Ben
> >
>
>
>
>