flatpak CI and stable builds - Re: KDE Gear projects with failing CI (release/23.08) (12 September 2023)

Thu Sep 21 08:54:59 BST 2023

On Thu, Sep 21, 2023 at 8:35 AM Albert Astals Cid <aacid at kde.org> wrote:

> El dimecres, 20 de setembre de 2023, a les 13:17:22 (CEST), Ben Cooksley
> va
> escriure:
> > On Wed, Sep 20, 2023 at 9:42 AM Albert Astals Cid <aacid at kde.org> wrote:
> > > El dimarts, 19 de setembre de 2023, a les 22:18:40 (CEST),
> > >
> > > christoph at cullmann.io va escriure:
> > > > On 2023-09-19 21:35, Albert Astals Cid wrote:
> > > > > Please work on fixing them, otherwise i will remove the failing CI
> > > > > jobs on their 4th failing week, it is very important that CI is
> > > > > passing
> > > > > for
> > > > > multiple reasons.
> > > > >
> > > > > Bad news: We have 2 new repositories failing :/
> > > > >
> > > > > = FLATPAK FAILING =
> > > > >
> > > > > kate:
> > > > >  * https://invent.kde.org/utilities/kate/-/pipelines/484147
> > > > >
> > > > >   * This highlights a design problem, it's building markdown part
> from
> > > > >
> > > > > master
> > > > > instead of from stable branch. We can manually change the branch,
> but
> > > > > i
> > > > > would
> > > > > prefer a solution that doesn't mean changing lots and lots of
> flatpak
> > > > > manifests when we branch.
> > > >
> > > > Hmm, yes that sounds not nice.
> > > >
> > > > But not sure how that would work without that, seems
> > >
> > >
> https://invent.kde.org/utilities/kate/-/blob/master/.flatpak-manifest.json
> > > ?r>
> > > > ef_type=heads
> > > >
> > > > hard codes what to fetch.
> > > >
> > > > Given one hard codes there the
> > > >
> > > >   "runtime-version": "5.15-22.08",
> > >
> > > That one is "fine", the 22.08 here it's related to the "flatpak kde/
> > > freedesktop sdk" not to Gear stuff.
> > >
> > > Yes, we will massively have to update them on master when we decide to
> > > depend
> > > on a new one, but it won't cause problems on the stable branches like
> the
> > > oner
> > > we're experiencing here.
> > >
> > > The problem here is
> > >
> > > {
> > >
> > >   "name": "markdownpart",
> > >   "buildsystem": "cmake-ninja",
> > >   "sources": [
> > >
> > >     {
> > >
> > >       "type": "git",
> > >       "url": "https://invent.kde.org/utilities/markdownpart.git"
> > >
> > >     }
> > >
> > >   ]
> > >
> > > }
> > >
> > > This unconditionally compiles the master branch of markdownpart repo
> > >
> > > As far as i can see there's three solutions:
> > >
> > > A) If this is just "to make sure it builds CI", we don't need
> markdownpart
> > > nor
> > > konsole on the flatpak since they are just runtime dependencies. This
> is a
> > > sub-optimal solution i'd say since it makes it so that we can't offer
> the
> > > package for testing in the future and makes the diff with the flathub
> > > manifest
> > > bigger than it needs to be
> >
> > The overall intention is for the bundles produced by the Flatpak jobs to
> be
> > useful for people to locally test a project build.
> > In the not too distant future we'll have them available from a Flatpak
> > repository for actual mainline/release branches as well.
> >
> > So the answer certainly isn't (a).
> >
> > > B) Depend on released versions, i.e. a tarball in "sources" instead of
> a
> > > git
> > > repo. This is probably suboptimal too in the sense that will require
> > > constant
> > > updating on master and if we offer the resulting flatpak as "nightly"
> in
> > > the
> > > future for testing it's not "nightly" as it could be.
> >
> > Guess it depends on the nature of the dependency, but in the case of
> > software released together you probably want to build against what you
> will
> > be shipping against yes.
> >
> > > C) Add a marker in the .json like branch: "kde-same-branch" and then
> have
> > > the
> > > code in
> > >
> https://invent.kde.org/sysadmin/ci-utilities/raw/master/gitlab-templates/f
> > > latpak.yml replace that "kde-same-branch" either by "master" or by
> > > the appropriate stable branch before actually compiling the flatpak. I
> > > think
> > > this would be the optimal solution but needs work.
> >
> > This is my preferred solution as well. it wouldn't be too difficult given
> > we have a Python script acting as a middle-man already.
> > In the past we did some rewriting of the .flatpak-manifest.yml already.
> >
> > Depending on our requirements it may be worth tying into the same
> > branch-rules.yml logic that the rest of the CI system uses but this is
> > probably best answered by someone who knows the various Flatpak manifests
> > we have better.
> >
> > In #flatpak:kde.org it was mentioned that this would mean that people
> > wouldn't be able to build it as easily themselves, but if we make it well
> > documented (comments in the .flatpak-manifest.yml, etc) then I think this
> > shouldn't be much of an issue.
>
> I had not thought of that and it's indeed not great.
>
> We could rename all the flatpak manifest that need this feature to .
> json.in to
> make it clear "it's not their final form" and that they need to be pre-
> processed.

> This does not help a lot to the "non CI user" if they want to actually use
> the
> manifest since they still need to pre-process it somehow :/
>

At least it would make it explicit that something is required before the
files can be utilised elsewhere.

Ultimately in the long term we are potentially looking at having release
builds we make be submitted into Flathub so this does need a solution in
some form or another.

Realistically I think we only have two choices here:
a) Have our release tooling include logic that bumps/updates the
.flatpak-manifest.yml files to have the right branch in them; or
b) Have a intermediary script like flatpak-build.py do that for us

For now the path of least resistance is probably (b) I think?

>
>
> Solution E)
>
> Since usually/mostly we have our projects be backwards API compatible it's
> usually/mostly not a problem to that kate stable uses markdown master
> (understanding the flatpaks generated by that CI are nightlies), we're
> only
> having this problem right now because of the KF6 transition.
>
> One potentially valid solution is to just disable flatpak CI in stable
> until
> this gets fixed, it's not great but it's just a few months.
>
> Cheers,
>   Albert
>

Cheers,
Ben

>
>
> >
> > > D) Something smarter I have not thought about.
> > >
> > > Cheers,
> > >
> > >   Albert
> >
> > Cheers,
> > Ben
> >
> > > > I assume one will need to hard code that, too, if one creates no own
> > > > scripting.
> > > >
> > > > But I might be wrong.
> > > >
> > > > Greetings
> > > > Christoph
> > > >
> > > > > = FAILING UNIT TESTS =
> > > > >
> > > > > konsole:
> > > > >  * https://invent.kde.org/utilities/konsole/-/pipelines/484148
> > > > >
> > > > >   * freebsd_qt515 tests are failing
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-devel/attachments/20230921/af19eaba/attachment-0001.htm>