flatpak CI and stable builds - Re: KDE Gear projects with failing CI (release/23.08) (12 September 2023)

[Albert Astals Cid]

El dimecres, 20 de setembre de 2023, a les 13:17:22 (CEST), Ben Cooksley va 
escriure:
> On Wed, Sep 20, 2023 at 9:42 AM Albert Astals Cid <aacid at kde.org> wrote:
> > El dimarts, 19 de setembre de 2023, a les 22:18:40 (CEST),
> > 
> > christoph at cullmann.io va escriure:
> > > On 2023-09-19 21:35, Albert Astals Cid wrote:
> > > > Please work on fixing them, otherwise i will remove the failing CI
> > > > jobs on their 4th failing week, it is very important that CI is
> > > > passing
> > > > for
> > > > multiple reasons.
> > > > 
> > > > Bad news: We have 2 new repositories failing :/
> > > > 
> > > > = FLATPAK FAILING =
> > > > 
> > > > kate:
> > > >  * https://invent.kde.org/utilities/kate/-/pipelines/484147
> > > >  
> > > >   * This highlights a design problem, it's building markdown part from
> > > > 
> > > > master
> > > > instead of from stable branch. We can manually change the branch, but
> > > > i
> > > > would
> > > > prefer a solution that doesn't mean changing lots and lots of flatpak
> > > > manifests when we branch.
> > > 
> > > Hmm, yes that sounds not nice.
> > > 
> > > But not sure how that would work without that, seems
> > 
> > https://invent.kde.org/utilities/kate/-/blob/master/.flatpak-manifest.json
> > ?r> 
> > > ef_type=heads
> > > 
> > > hard codes what to fetch.
> > > 
> > > Given one hard codes there the
> > > 
> > >   "runtime-version": "5.15-22.08",
> > 
> > That one is "fine", the 22.08 here it's related to the "flatpak kde/
> > freedesktop sdk" not to Gear stuff.
> > 
> > Yes, we will massively have to update them on master when we decide to
> > depend
> > on a new one, but it won't cause problems on the stable branches like the
> > oner
> > we're experiencing here.
> > 
> > The problem here is
> > 
> > {
> > 
> >   "name": "markdownpart",
> >   "buildsystem": "cmake-ninja",
> >   "sources": [
> >   
> >     {
> >     
> >       "type": "git",
> >       "url": "https://invent.kde.org/utilities/markdownpart.git"
> >     
> >     }
> >   
> >   ]
> > 
> > }
> > 
> > This unconditionally compiles the master branch of markdownpart repo
> > 
> > As far as i can see there's three solutions:
> > 
> > A) If this is just "to make sure it builds CI", we don't need markdownpart
> > nor
> > konsole on the flatpak since they are just runtime dependencies. This is a
> > sub-optimal solution i'd say since it makes it so that we can't offer the
> > package for testing in the future and makes the diff with the flathub
> > manifest
> > bigger than it needs to be
> 
> The overall intention is for the bundles produced by the Flatpak jobs to be
> useful for people to locally test a project build.
> In the not too distant future we'll have them available from a Flatpak
> repository for actual mainline/release branches as well.
> 
> So the answer certainly isn't (a).
> 
> > B) Depend on released versions, i.e. a tarball in "sources" instead of a
> > git
> > repo. This is probably suboptimal too in the sense that will require
> > constant
> > updating on master and if we offer the resulting flatpak as "nightly" in
> > the
> > future for testing it's not "nightly" as it could be.
> 
> Guess it depends on the nature of the dependency, but in the case of
> software released together you probably want to build against what you will
> be shipping against yes.
> 
> > C) Add a marker in the .json like branch: "kde-same-branch" and then have
> > the
> > code in
> > https://invent.kde.org/sysadmin/ci-utilities/raw/master/gitlab-templates/f
> > latpak.yml replace that "kde-same-branch" either by "master" or by
> > the appropriate stable branch before actually compiling the flatpak. I
> > think
> > this would be the optimal solution but needs work.
> 
> This is my preferred solution as well. it wouldn't be too difficult given
> we have a Python script acting as a middle-man already.
> In the past we did some rewriting of the .flatpak-manifest.yml already.
> 
> Depending on our requirements it may be worth tying into the same
> branch-rules.yml logic that the rest of the CI system uses but this is
> probably best answered by someone who knows the various Flatpak manifests
> we have better.
> 
> In #flatpak:kde.org it was mentioned that this would mean that people
> wouldn't be able to build it as easily themselves, but if we make it well
> documented (comments in the .flatpak-manifest.yml, etc) then I think this
> shouldn't be much of an issue.

I had not thought of that and it's indeed not great.

We could rename all the flatpak manifest that need this feature to .json.in to 
make it clear "it's not their final form" and that they need to be pre-
processed.

This does not help a lot to the "non CI user" if they want to actually use the 
manifest since they still need to pre-process it somehow :/


Solution E)

Since usually/mostly we have our projects be backwards API compatible it's 
usually/mostly not a problem to that kate stable uses markdown master 
(understanding the flatpaks generated by that CI are nightlies), we're only 
having this problem right now because of the KF6 transition.

One potentially valid solution is to just disable flatpak CI in stable until 
this gets fixed, it's not great but it's just a few months.

Cheers,
  Albert


> 
> > D) Something smarter I have not thought about.
> > 
> > Cheers,
> > 
> >   Albert
> 
> Cheers,
> Ben
> 
> > > I assume one will need to hard code that, too, if one creates no own
> > > scripting.
> > > 
> > > But I might be wrong.
> > > 
> > > Greetings
> > > Christoph
> > > 
> > > > = FAILING UNIT TESTS =
> > > > 
> > > > konsole:
> > > >  * https://invent.kde.org/utilities/konsole/-/pipelines/484148
> > > >  
> > > >   * freebsd_qt515 tests are failing