Flatpak jobs on KDE CI vs. continuous integration on main/master/devel branches

[Volker Krause]

On Mittwoch, 7. Februar 2024 19:48:27 CET Friedrich W. H. Kossebau wrote:
> Am Sonntag, 4. Februar 2024, 19:22:28 CET schrieb Ben Cooksley:
> > On Mon, Feb 5, 2024 at 4:28 AM Friedrich W. H. Kossebau <kossebau at kde.org>
> > 
> > wrote:
> > > Hi,
> > > 
> > > ((cc:kde-frameworks-devel for heads-up, replies please only to
> > > kde-core-deve))
> > > 
> > > I hit the problem that when working on a repo which would like to use
> > > latest
> > > KF development state to integrate some new KF API just added in
> > > cooperation
> > > with that very repo wanting to use it, I cannot do so when someone had
> > > added a
> > > flatpak job on CI to that repo.
> > > 
> > > Because with such flatpak jobs it seems they are limiting the available
> > > KF
> > > version not to the current latest one, as expected for continuous
> > > integration,
> > > 
> > > but some older (anywhere documented?) snapshot:
> > >     "runtime-version": "6.6-kf6preview",
> > 
> > Please see https://invent.kde.org/packaging/flatpak-kde-runtime/-/tree/kf6
> > for what is in the KF6 preview.
> 
> Thanks. So documented by implementation :)
> 
> > > What can be done here to reestablish the old immediate continuous
> > > integration
> > > workflow? Where new APIs (also from KF) are instantly available?
> > 
> > With Flatpak new APIs were never instantly available - there has always
> > been a delay as the Flatpak Runtime uses the most recent released version
> > of our software.
> 
> I guess this was shadowed by feature development of KF having stalled during
> the Qt5/KF5->Qt6/KG6 porting phase as well as Qt6-based flatpaks jobs only
> activated recently.
> 
> > > Right now this is a new extra burden which makes working on new features
> > > with
> > > KF and apps more complicated. Thus less interesting, and one/I would
> > > rather
> > > duplicate code in apps to get things done.
> > > 
> > > Blocking latest KF API from usage also means less testing of that before
> > > the
> > > initial release.
> > > 
> > > 
> > > Besides all the resource costs to create flatpaks on master builds by
> > > default
> > > every time, when those are usually not used by anyone anyway.
> > 
> > Those applications that have a hard dependency on features being added to
> > Frameworks are not good candidates for making use of our Continuous
> > Delivery systems i'm afraid.
> > Both Flatpak and Craft based (Linux Appimages, Android APKs, Windows and
> > macOS) CD jobs are best optimised for those applications that rely on the
> > stable Frameworks releases.
> > 
> > There are ways (in .craft.ini) to make newer Frameworks available, but
> > that
> > requires that the system recompiles that Framework each time you trigger a
> > build and is therefore not recommended.
> > 
> > Allowing those systems to use the "latest" artifacts of Frameworks would
> > be
> > a non-trivial exercise.
> 
> So effectively this means:
> * KF - no CI on new API with non-KF repos, only KF-internal CI
> * KF - no CD, only released versions
> 
> Which makes KF a second class product, with lesser testing :(
> And less interesting to contribute to, given it gets worse CI/CD care.

CI builds can (as they always could on Gitlab and currently mostly do) use 
latest KF master. CD builds can't (and never could, binary factory had the 
same limitation). Nothing is getting worse here, the only thing you cannot 
have is unconditionally using latest KF and expect CD builds for that based on 
cached runtime/platform parts.

The most common way of working for apps seems to be supporting at least the 
last released version of KF, if needed with version ifdefs to already 
integrate newer code. In most cases that is very limited effort, and as soon as 
more than one contributor is involved that is usually highly desired anyway to 
not randomly force full rebuilds on others.

There are cases where this is not feasible, e.g. in case of very tight 
coupling or QML code that cannot easily be ifdef'ed. In that case you either 
wait for the next release, or you can only have CD builds in the release 
branch I'd say. Using expensive "full-stack" CD builds for that would need a 
very good justification given the cost IMHO.

I can't speak for KDE Games obviously, but for the stuff I am involved in I 
most definitely wouldn't want to miss CD builds anymore. There's a bunch of 
people daily-driving Itinerary from those for example, and that is such a big 
help for QA. Bug reports are practically always still relevant, the turnaround 
time for feedback is days or even hours rather than weeks or months, and a 
whole bunch of issues gets caught before even hitting the releases. On top of 
that the Flatpak build provides an extra safety net there for finding 
incompatibilities with two major external dependencies (poppler and zxing), by 
using the latest versions of those.

One thing I do agree on though is that there is no point in running jobs (CI 
nor CD) that really /nobody/ cares about anymore, and the occasional cleanup 
there is probably a good idea (per repo and per job type, like the recent 
retirement of the 32bit x86 Android builds). Running all this has a cost for 
the people working on "horizontal" topics as well (CI/CD, Craft, platform/OS 
support, etc), and that's a limited resource we shouldn't waste. That needs 
careful balancing though, it's also not nice to have your work thrown out when 
briefly not watching.

> For me this is an extra hurdle to work in KDE. And things are made worse for
> me given I am not convinced by those platforms (besides AppImage perhaps)
> which now are ruling contribution/development here.
> 
> I understand people fancy easy deployment on their favourite platforms, so
> do I. There was a time when also Debian package info was in some KDE 
> projects IIRC.
> But please think about those who are not into your respective platform, do
> not force them into (supporting) yours.

This goes both ways though. I might not care about the same platforms you care 
about. Yet, I'm sure you'd expect me to not break your platform and/or to fix 
whatever breakage I might have caused.

Yes, this means I have to also look after stuff I don't care about personally, 
but in return I get others doing the same for things I care about.

None of this means you are expected to be proficient in arcane Windows compiler 
issues etc, it merely means taking responsibility for getting breakage 
addressed. Most of the time that is just a matter of asking in chat or pasting 
the error into your favorite search engine.

And regarding the "extra hurdle" part, I've also seen the exact opposite, with 
contributors being especially motivated by the fast turn-around time and 
seeing the result of their work within minutes on their phone.

Regards,
Volker
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-core-devel/attachments/20240208/eb0942a1/attachment.sig>