Transition to VM based CI
Soumyadeep Ghosh
soumyadeepghosh2004 at zohomail.in
Mon Jul 7 20:03:45 BST 2025
Hello again,
So, the last time I checked the notary service, what it does is
basically sends the artifact to a remote server, where the artifact is
signed with a valid key that's recognisable as KDE's to validate the
artifact as KDE's artifact in various stores. Am I explaining it correctly?
If I am right, then, I don't think that's needed for snaps. To upload a
snap to the store, one basically needs the auth token of the KDE account
in the snap store and pass it when uploading the snap. IIRC, till now,
some of the vms were having this auth token embedded. Can't we replicate
something like this?
Thanks,
Soumyadeep
On 08/07/25 00:20, Ben Cooksley wrote:
> On Mon, Jul 7, 2025 at 11:55 PM Soumyadeep Ghosh
> <soumyadeepghosh2004 at zohomail.in> wrote:
>
> Hi Ben,
>
>
> Hi Soumyadeep,
>
>
> This is an awesome news! I just wanted to know what would be the
> problem in publishing those snaps built inside the new VM based CI.
>
>
> There is currently no Notary built that supports handling the
> publishing process at this time, which is why we're unable to publish.
>
>
> Thanks,
> Soumyadeep Ghosh
>
>
> Cheers,
> Ben
>
>
>
>
>
> ---- On Mon, 07 Jul 2025 17:19:14 +0530 *bcooksley at kde.org * wrote
> ----
>
> Hi all,
>
> I'm happy to announce that VM based CI is now in a state where
> it is ready to begin making the transition into general
> availability. As part of this i'm happy to advise that builds
> for Snaps (but not their publishing) are also becoming
> generally available, and FreeBSD will be updating to Qt 6.9 as
> well.
>
> This transition will also mark the general retirement of
> Docker based CI, with Docker functionality only being retained
> for website builds and linter jobs (such as xml-lint,
> cppcheck, etc).
>
> As part of this support for Qt 5 (except for general Linux) is
> also being retired. This includes all FreeBSD and Windows
> support, as well as all binary support (Appimages, Windows
> exes, etc)
>
> _Steps you need to take:_
>
> If you only inherit the templates from sysadmin/ci-utilities,
> and do not have custom jobs, then no action is needed on your
> part.
>
> If you have custom jobs which are based on the templates
> currently in sysadmin/ci-utilities then you should examine
> the diff in
> https://invent.kde.org/sysadmin/ci-utilities/-/merge_requests/507
> to ensure that no changes are needed to your job.
>
> Custom jobs not based on the sysadmin provided templates
> likely do not need to be changed.
>
> Projects that are Qt 5 based should remove includes for
> everything except Linux general (linux.yml) as those templates
> will be removed as part of this roll out.
>
> _Timeline for the transition:_
>
> Over the next week the first node (node6.ci.kde.org
> <http://node6.ci.kde.org>) will be taken out of rotation for
> running existing container based jobs to facilitate testing of
> the automated deployment. Once this has been validated, the
> conversion of the remaining nodes (node2, node3, node4, node5)
> will be scheduled - converting them all to be VM based CI runners.
>
> To allow for final legacy jobs to be completed, node1 will be
> left to run remaining legacy jobs for a period of time (to be
> determined). Once that finishes up, that node will also be
> converted.
>
> As part of this the dedicated VM runners currently supplied
> for building KDE Linux and Snaps will also be retired. Support
> for Yocto builds is out of scope for this transition at this
> time but that may be re-evaluated in the future.
>
> _What specs will the VMs be provided with?_
>
> By default builds will be provided with 8 vCPU cores, 16GB RAM
> and 200GB of disk space, although some of that disk space will
> be occupied by the VM OS, development tools, etc.
> Builds requiring additional resources should file a Sysadmin
> ticket.
>
> VMs will have a shared mount provided from the host at /mnt
> that allows for artifacts to be cached between builds on that
> node. This is mainly intended to be used for caching
> dependencies and other downloaded artifacts and should not be
> relied on to transfer results between jobs (as there is no
> guarantee contents won't go away, or that jobs will be
> allocated to the same build node).
>
> _What benefits will we get as part of this change?_
>
> For Linux based builds, as these are now running in a fully
> fledged system, anything that depends on system level services
> (like CUPS or systemd/logind) or which needs to undertake
> actions normally restricted in a Docker/Podman container (like
> manipulating FUSE mounts or interacting with hardware like a
> TPM) should now be able to be unit tested.
>
> For FreeBSD based builds, these images no longer have to be
> built manually by hand, and are now easily available to be run
> locally on developer systems for local testing. Reliability of
> FreeBSD builds should also improve with the elimination of the
> dependency on the Podman daemon being running on the build node.
>
> For Windows based builds, these should run faster due to the
> elimination of the overhead of the Docker overlay file system,
> which on Windows introduces significant overhead to IO
> operations. For Kate, regular Windows CI build times were
> reduced from 6 minutes (under Docker) to 3 minutes 40 seconds
> (in a VM based CI setup). This includes the overhead of
> provisioning the VM and waiting for Windows to start. We
> should also see improvements to builder availability, as the
> out of memory issues that have caused Windows build nodes to
> be killed by the OOM-killer are not possible in a VM based CI
> environment.
>
> For binary builds, Linux support is enhanced by it now being
> possible to build Snaps, as well as Flatpaks no longer
> requiring workarounds to be built, while Appimages are now
> able to be run if needed. These are actions that have all
> previously been restricted by operating in a container
> environment.
>
> The system will also generally benefit from being able to
> scale build capacity for different OSes on an as needed basis
> within our available hardware (so long build queues for
> Windows post major release events should become less of an issue).
>
> As an additional benefit, the system will require
> significantly less work to maintain. Currently each build
> node, along with the FreeBSD and Windows VM thereon, have to
> be maintained by hand and disk space allocated between them in
> a fixed fashion. This means that any cleanup from stale disk
> images, over-filled caches, etc. has to be done 3 times on
> each build node (being the Linux host as well as the FreeBSD
> and Windows guest VMs). Currently provisioning new nodes is
> significantly labour intensive as well (see
> https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/gitlab-templates/README.md
> for the instructions).
>
> This is essentially completely eliminated with the transition
> to VM based CI, with the majority of the deployment now being
> possible using Ansible with the only manual step being the
> registration with Gitlab - which is a fairly quick process
> taking less than 20 minutes per node. Maintenance is
> significantly reduced as each node only needs one set of
> cleanup - not three.
>
> Should there be any questions on the above please let me know.
>
> Many thanks,
> Ben
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/release-team/attachments/20250708/45150bc1/attachment.htm>
More information about the release-team
mailing list