CI Flatpak Builds

[Ben Cooksley]

On Sat, Jan 25, 2025 at 11:44 PM Volker Krause <vkrause at kde.org> wrote:

> On Freitag, 24. Januar 2025 18:57:43 Mitteleuropäische Normalzeit Ben
> Cooksley
> wrote:
> > On Fri, Jan 24, 2025 at 4:44 PM Justin Zobel <justin at 1707.io> wrote:
> > > On 24/1/25 02:55, Volker Krause wrote:
> > > > On Donnerstag, 23. Januar 2025 10:21:08 Mitteleuropäische Normalzeit
> Ben
> > > >
> > > > Cooksley wrote:
> > > >> On Thu, Jan 23, 2025 at 7:21 AM Volker Krause <vkrause at kde.org>
> wrote:
> > > >>> On Mittwoch, 22. Januar 2025 18:52:46 Mitteleuropäische Normalzeit
> Ben
> > > >>>
> > > >>> Cooksley wrote:
> > > >>>> On Thu, Jan 23, 2025 at 5:49 AM Volker Krause <vkrause at kde.org>
> > >
> > > wrote:
> > > >>>>> On Dienstag, 21. Januar 2025 23:11:56 Mitteleuropäische
> Normalzeit
> > > >>>
> > > >>> Albert
> > > >>>
> > > >>>>> Astals Cid wrote:
> > > >>>>>> neochat - NEW
> > > >>>>>>
> > > >>>>>>   * https://invent.kde.org/network/neochat/-/pipelines/869365
> > > >>>>>>
> > > >>>>>>    * flatpak build fails
> > > >>>>>
> > > >>>>> Same as
> https://invent.kde.org/network/neochat/-/merge_requests/2127
> > >
> > > I
> > >
> > > >>>>> guess,
> > > >>>>> looks like an API change in libQuotient with the corresponding
> > >
> > > version
> > >
> > > >>>>> change
> > > >>>>> still pending (ie. a similar problem we occasionally hit with
> > > >>>>> Poppler).
> > > >>>>>
> > > >>>>> The quick solution for this would probably be (temporarily)
> > > >>>>> switching
> > > >>>>> Flatpak
> > > >>>>> to the latest release of libQuotient. The IMHO proper solution
> would
> > > >>>
> > > >>> be to
> > > >>>
> > > >>>>> have version bumps at the beginning of a development cycle that
> > > >>>>> changes
> > > >>>>> API.
> > > >>>>
> > > >>>> This also raises another question though - should we be strongly
> > > >>>> discouraging use of heavily moving targets like upstream dev /
> master
> > > >>>> branches and instead favouring the use of stable branches?
> > > >>>> Not really ideal if upstream can essentially break our builds
> without
> > >
> > > us
> > >
> > > >>>> doing anything.
> > > >>>
> > > >>> I can't comment on NeoChat, but for Itinerary the builds against
> the
> > > >>> latest
> > > >>> versions of essential (external) dependencies with varying
> > > >>> intentions/success
> > > >>> in keeping backward compatibility (ZXing, Poppler, Quotient) are
> very
> > >
> > > much
> > >
> > > >>> deliberate, to catch breakage/regressions early (same as we are now
> > >
> > > doing
> > >
> > > >>> for
> > > >>> KF with Qt pre-releases).
> > > >>>
> > > >>> And this is actually working, NeoChat has been fixed to build with
> the
> > > >>> upcoming
> > > >>> Quotient version, same for Itinerary/Poppler. The only thing we are
> > > >>> missing
> > > >>> for those fixes to take effect is those dependencies bumping their
> > >
> > > version
> > >
> > > >>> numbers.
> > > >>>
> > > >>> We could work around that by doing configure-time detection using a
> > > >>> compile
> > > >>> test, but that's a lot of extra work, so we'd better try to address
> > >
> > > this
> > >
> > > >>> upstream first.
> > > >>>
> > > >>>
> > > >>> I'd very much suggest the use of release tarballs (or stable
> branches)
> > >
> > > for
> > >
> > > >>> apps that aren't continuously monitoring their dependencies this
> way
> > > >>> though,
> > > >>> but I'm not aware we have such a case.
> > > >>
> > > >> Doesn't this interfere somewhat with the long term eventual goal of
> us
> > > >> pushing Flatpak binaries directly to Flathub though?
> > > >> (not to mention make stable branch builds impossible to reproduce in
> > > >> the
> > > >> long run)
> > > >
> > > > For Itinerary at least the release branch is using release tarballs
> > >
> > > rather
> > >
> > > > than (latest) Git branches, matching what we have on Flathub. (This
> does
> > >
> > > imply
> > >
> > > > some manual work at branch time, but at least for me that's more than
> > >
> > > offset by
> > >
> > > > what this saves in catching issues early).
> > > >
> > > > But you are right of course that this is abusing Flatpak a bit as CI.
> > >
> > > That's
> > >
> > > > not ideal but it's just very tempting as it has minimal extra cost
> > >
> > > (Flatpak
> > >
> > > > has to build those dependencies anyway, and Flatpak builds run less
> > >
> > > often than
> > >
> > > > normal CI builds).
> > > >
> > > > Regards,
> > > > Volker
> > >
> > > I've moved this to a new email subject as it's no longer just about
> > > failing CI builds.
> > >
> > > My proposal would be have 3 files in the repository that are Flatpak
> > > related:
> > >
> > > - .flatpak-manifest.yml will continue to exist for git master builds
> and
> > > be deployed to https://cdn.kde.org/flatpak/
> > >
> > > - org.kde.appname.json which will be for Flathub stable builds will
> only
> > > run on release branches when a tag is made (I am hoping CI can be
> > > triggered by a tag).
> > >
> > > - Both will reference a file called org.kde.appname.dependencies.json
> > > which contains the the dependencies that get build for the app and will
> > > point to tarballs or tags for released versions.
> >
> > Not sure how that solves the issue?  Currently the issue is the version
> of
> > the dependencies - with us using master / equivalent branches in our
> > Flatpak jobs.
> > Having a shared file for that would ensure that impacts Flathub builds or
> > other stable builds we run as well.
> >
> > I had been thinking that having a separate file that would be used
> instead
> > of .flatpak-manifest.yml if present when doing a build on a stable branch
> > would be a good solution.
> > Under this you would have a .flatpak-manifest.yml and a
> > .flatpak-manifest-stable.yml file
> >
> > When building on master the .flatpak-manifest.yml file would be used, but
> > when on a release branch / tag the .flatpak-manifest-stable.yml file
> would
> > be used instead (if present).
> >
> > That would eliminate needing the release manager / developer to update
> > .flatpak-manifest.yml on branching a new stable release branch.
>
> I agree there are probably ways to avoid that manual step during
> branching, be
> it separate files or some form of automation.
>
> There is a much larger manual effort involved here though that we haven't
> mentioned yet, updating stable versions of dependencies. Flathub has
> automation for this, we currently don't. If we actually want to produce
> end-
> user/production ready Flatpak builds ourselves that is probably the most
> important thing to address.
>

>From what I understand the tool that Flathub runs should in theory be
deployable against our repositories without too much issue.
It just needs to be worked on (which is the painful bit) and we likely need
ARM builds as well (which is something I already have a ticket for).

I'd like to dig myself out from under the big pile of infrastructure debt
that needs to be paid down as well as move to VM based CI before we look
into those though.


>
> (Bonus points if that would also cover Craft, we have the same challenge
> there.)
>

Pretty sure we will have to write our own version of the tooling Flathub
wrote to do this alas.

That sort of task should be tackled at the same time we look into tackling
some of the other recurring issues around Craft though (like updates to one
package breaking some platforms and needing additional work to fix).
Still not 100% sure what is the best approach to that issue though as there
are no simple solutions that I can see at least - especially given Craft is
supporting 4 different OSes (Linux, Android, Windows and macOS) all of
which have their own quirks.


>
> Regards,
> Volker
>

Cheers,
Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-devel/attachments/20250126/8c42f382/attachment-0001.htm>