Rollout of VM based CI

[Ben Cooksley]

On Mon, Jul 21, 2025 at 7:03 AM <pillowtrucker at proton.me> wrote:

> Hi Ben,
>

Hi Maciej,


>
> Thank you for your hard work on this!
> I have a pipeline stuck on a Windows CI job and when I retry the job I
> still get a docker runner, which doesn't have (and probably will never
> have) an update that I need. The pipeline is here
> https://invent.kde.org/pillowtrucker/crow-translate/-/pipelines/972407 and
> the related MR was here
>  https://invent.kde.org/sysadmin/ci-images/-/merge_requests/449
> <https://invent.kde.org/sysadmin/ci-images/-/merge_requests/449.>. Is
> there something I need to change in the ci configuration for the project?
> Bump it with a new commit?
>

I'd suggest you create a new pipeline, which will pull in the updated job
definitions and send your jobs to the VM based CI runners that are updated.
If you open your project, select Build > Pipelines, you should have an
option to create a "New Pipeline" in the top right - select this and follow
the prompts (selecting the appropriate branch) and you should be good to go.

This can be done at any time, and for any branch - the only place you can't
create new pipelines is on merge requests (which is why we have the old
builders for a week)

Thanks,
Ben


> Best,
> Maciej
> On Sunday, July 20th, 2025 at 8:45 PM, Ben Cooksley <bcooksley at kde.org>
> wrote:
>
> Hi all,
>
> Yesterday afternoon I actioned the rollout of VM based CI, which is now in
> production for all platforms.
>
> As part of this, Snap builds are now generally available and may be used
> freely. Publishing of Snaps still requires a notary to be built, so that
> component remains unavailable for now.
> The previous dedicated VM providing support for Snaps has also been
> retired.
>
> FreeBSD has also updated to Qt 6.9 as part of this changeover, and all
> other images (SUSE Qt 5.15, Qt 6.9 and Qt 6.10, Alpine Qt 6.8) have also
> been rebuilt and are fully updated as well.
>
> Builder wise, 4 of our previous 6 builders have been converted to be VM
> based. The remaining two are scheduled for retirement to allow for an ARM
> builder to be provided, and will be temporarily available for a further
> week or so to allow for old pipelines to be completed.
> The ARM builder has also been provisioned and connected as a VM based
> runner to Gitlab.
>
> Should your project be Qt 5 based still, you will find that all support
> for everything except Linux has been removed. This is in line with what was
> previously announced and you will need to remove those jobs from your CI
> configuration.
> All build artifacts relating to Qt 5 on FreeBSD and Windows have already
> been purged from the system, as support for those platforms has now ended.
>
> If your project has custom jobs, it would be advisable to check that those
> jobs are making use of VM based CI where possible, as build power on the
> Docker side following this conversion will be more limited (being primarily
> intended to support building websites and running linter checks).
>
> Should you be running custom workflows it is important to note that the
> location for caching artifacts has changed as part of this migration. It is
> no longer at /mnt/artifacts/$PLATFORM/ or /mnt/caches/$PLATFORM/ - instead
> things are now found at /mnt/$PLATFORM/artifacts/ and
> /mnt/$PLATFORM/caches/. It is imperative that this change is reflected in
> your jobs, otherwise you may encounter permission related failures due to
> different distributions having different user/group IDs.
>
> For those curious, the rollout procedure across all four machines took
> less than an hour to process (including the Hetzner re-imaging of the
> machines), and is largely fully automated. There is now also automation in
> place to clean up old images as well, which will hopefully reduce the risk
> that the builders run out of disk space.
>
> Please let me know if you have any questions.
>
> Many thanks,
> Ben
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-devel/attachments/20250721/24043a34/attachment-0001.htm>