Post-MegaRelease projects

[Volker Krause]

On Saturday, 24 February 2024 04:31:49 CET Ben Cooksley wrote:
> On Fri, Feb 23, 2024 at 11:12 PM Sune Vuorela <nospam at vuorela.dk> wrote:
> > On 2024-02-22, Nate Graham <nate at kde.org> wrote:
> > > I've started pondering post-megarelease projects. We've spent so long on
> > > porting and bugfixing that I think it might be useful to shift gears to
> > > feature work, and I'd like to brainstorm potential large-scale projects
> > > and gauge the level of interest in putting resources into them soon.
> > 
> > A bit more from the devops end that I'd love to see people tackle:
> >  - Ensure frameworks and app unit tests interacting with windows can run
> >  
> >    on Windows.
> >    More details: The following fails on our windows CI
> >    https://invent.kde.org/sune/windows-test-thingie/-/blob/master/main.cpp
> >    I find it weird that we are spending resources on putting things in
> >    the windows store and making apps available on windows, but we can't
> >    actually have passing tests in our CI.
> 
> This unfortunately will not be easy to solve.

And that's fine, we are in dreaming territory here anyway :)

> One of the key things that we've learned out of doing CI, as has been
> showcased by FreeBSD in particular, is that the builders need to be
> ephemeral - that is only around for the build in question that is being run.
> We're currently accomplishing this by using containers - via Podman (for
> Linux/Android/FreeBSD) and Docker (for Windows).
> 
> Containers also offer us the advantage of allowing people to easily
> reproduce the CI environment on their local system without too much trouble.
> 
> For Windows however, Microsoft has limited the container stack to not allow
> anything GUI related to work. The underlying libraries may be there, but
> the equivalent display server components are not operational.
> 
> To complicate things further, on Windows certain permissions are restricted
> to the interactive console and are not possible to do as either a scheduled
> task or as a system service.
> Usage of existing Windows automation frameworks such as Powershell Remoting
> or SSH will therefore not work if we want things to perfectly replicate a
> end user environment - because those will run the command(s) as part of a
> non-interactive session (even if the user we connect as is the same one
> logged in on the desktop console).
> 
> Resolving this will therefore not only require running full sized Windows
> VMs (on an ephemeral basis to avoid the recently resolved issues that used
> to plague FreeBSD), but would also need the VM to automatically login and
> spawn an agent as part of the interactive desktop session that we would
> connect to in order to run the CI tests.
> 
> The spawning of a VM would require refactoring the setup of our poor CI
> workers (again - after they were only just recently completely rebuilt to
> allow the transition to Podman to fix flatpak-builder) to make use of
> something like:
> -
> https://forum.gitlab.com/t/custom-executor-into-windows-vm-using-linux-kvm-l
> ibvirt/72713/5 -
> https://docs.gitlab.com/runner/executors/custom_examples/libvirt.html
> 
> We would still have to write the agent that receives our commands
> (something like
> https://gist.github.com/cschwede/3e2c025408ab4af531651098331cce45 maybe)
> 
> We would still have to solve the issue of how to share disk images among
> our nodes so they were built (ideally would be able to use Gitlab's
> Container Registry for this, which is something Tart can do - Tart being a
> virtualisation management utility for ARM based Macs), as well as
> automating the installation of the various OSes we need to run this way.
> See
> https://learn.microsoft.com/en-us/windows-hardware/manufacture/desktop/autom
> ate-windows-setup?view=windows-11 for some notes on that for Windows.
> 
> The big downside to all of that of course is that the worker nodes would
> take longer to startup, and it would make sharing build artifacts much more
> difficult and/or less efficient.
> 
> >  - Find a way to run unit tests on android CI.
> 
> Likewise, this is very non-trivial - from my understanding our tooling
> currently for building Android software is centered around it all being
> cross compiled rather than being built on the native architecture it will
> be run on.
> Naturally tests cannot be run (unless you emulate, which I guess we could
> consider...) if the CPU architectures are not compatible.
> 
> Even if you emulate though, I imagine we would need to provide a full
> Android system setup (including all of it's relevant bits of system
> infrastructure, such as it's display server DisplayFlinger) in order for
> tests to truly be of use.
> The path of least resistance is probably by making use of Google's existing
> Android emulator - although i've no idea how you would tie that into CI.

Right, the Android emulator is probably the first thing to look at for this. 
That alone isn't enough though, we can't just copy over a unit test executable 
and run it there, this needs to be wrapped in an APK (and which potentially 
needs to be given permissions, as we don't want system UI dialogs popping up, 
etc).

The Qt CI seems to be doing something like this meanwhile, might be worth 
investigating how this is done there.

> We would need to have our build chains ready to build on a native system
> before we could think about this I think. Building Android x86/x64 wouldn't
> cover things off properly due as it won't reflect the actual CPUs being
> used on end-user devices (and ARM processors can expose issues that don't
> happen on x86/x64 based systems)

For Android I'd rate that less relevant actually, we'd get by far the most 
things covered with x86_64 as well. I'm only aware of one ARM-specific runtime 
issue recently, and that was ARM32 only even. The vast majority of problems 
are either related to APK packaging or interaction with (changing) Android 
platform infrastructure, all of which is platform-independent. And the 
majority of ARM-specific issues we actually encounter during building.

ARM64 native builds nevertheless belong on this wishlist, but IMHO more 
importantly for ARM Flatpaks for the mobile apps.

> >  - Make autotests guarding on all our CI's.
> >  
> >  - Clazy and clang-tidy and cppcheck on all our repositories in CI
> >  
> >  /Sune
> 
> Hopefully the above is food for thought as to why we don't have the above -
> don't mean to say it can't be done, but it certainly is not trivial to
> accomplish...

Sure, and likewise the wishlist isn't meant as criticism of what we have, on 
the contrary, IMHO our CI/CD system has made huge steps forward with the 
migration to Gitlab and to container runners as well as the new signing 
service, but now we are hooked for more ;-)

Yes, those are ambitious ideas, but they would significantly help application 
development I think, and assuming Nate's original question was also about 
where the e.V. might want to invest resources into going forward, our CI/CD 
infrastructure would IMHO be a good area to (continue to) invest in.

Regards,
Volker

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-community/attachments/20240224/5827ba65/attachment.sig>