General CI image freeze now in effect
Kristen McWilliam
kristen at kde.org
Mon Jun 30 16:19:30 BST 2025
This sounds great for being able to run CI locally! Very nice work! :)
Is this process intended to be documented, at https://develop.kde.org/docs/
maybe?
On Mon, Jun 30, 2025 at 7:17 AM Ben Cooksley <bcooksley at kde.org> wrote:
> On Mon, Jun 30, 2025 at 8:42 AM Albert Vaca Cintora <albertvaka at gmail.com>
> wrote:
>
>> What will be the procedure if we ever need to add a dependency now? With
>> Docker it was easy because we could just edit the Dockerfile. What will be
>> the equivalent?
>>
>
> This is still being managed through sysadmin/ci-images, which now contains
> everything needed to build the VM templates.
> Much like with the Docker setup, there is a folder still for each VM image
> which contains the logic for that particular image.
>
> If you look in each folder there you'll see various scripts which are used
> to perform the build - with all except for Windows having two common entry
> points of "build.sh" and "sysprep.sh".
>
> These two entry point scripts are executed by sudo / as root within the VM
> image - with build.sh essentially being the new "Dockerfile". If you need
> to bring files in like patches or other scripts, for all OSes (except
> Windows) they can be simply placed in that folder and will become available
> at /var/lib/ci-files/ within the VM. I'd suggest looking at flatpak-builder
> / alpine-qt68 as examples of how this is used.
>
> The CI jobs in the sysadmin/ci-images repository are still used to build
> and publish images in a fully automated fashion.
>
> For those wondering how this works, in sysadmin/ci-images you'll find a
> utility "vm-runner" which is used for fetching images as well as
> provisioning and tearing down VMs based on those images (or any other qcow2
> disk image you happen to have locally). It can be run on any system that
> has a functioning libvirt instance and KVM setup, which most distributions
> should provide - so you can run anything the CI system runs on your local
> system with minimal effort.
>
> For Linux and FreeBSD vm-runner works by leveraging cloud-init to
> configure the system, which allows it to set a hostname, ensure the disk is
> fully expanded, provision user accounts for access, etc. You can see what
> the default templates do by looking in vm-runner/templates/. Among other
> things this includes dropping the files from those folders in
> sysadmin/ci-images into the image (which is what the --cloudinit-resources
> flag to vm-runner run triggers).
>
> To build an image all we do (for Linux/FreeBSD at least) is:
> a) Download a cloud-init enabled image for the distribution in question
> b) Boot that image using vm-runner, passing in the resources from the
> sysadmin/ci-images folder for the image being built
> c) Via SSH, run the build.sh script within the VM (using sudo to give it
> root privileges)
> d) Via SSH, run the sysprep.sh script within the VM (once again using sudo)
> e) Once the VM has shutdown, upload it to storage.kde.org/vm-images/ for
> use by VM based CI builds / vm-runner clients run elsewhere.
>
> Developers are more than welcome to use vm-runner locally should they wish
> to. It's requirements are pretty light (ignoring the VM specs you specify)
> and i've been using it on my local system while developing this setup (and
> it actually makes some sysadmin tasks easier by letting me quickly spin up
> VMs to test things in). The images used by the CI system live at
> storage.kde.org/vm-images/ and should be specified without the .qcow2
> extension.
>
> If I want to run an Alpine CI image locally for instance i'd run:
> vm-runner run --cpu 4 --ram 8G --disk 100G --image
> storage.kde.org/vm-images/alpine-qt68 --pull --mount ~/shared/ --ssh-keys
> ~/.ssh/id_ed25519.pub myalpineci
>
> Once booted it will advise me of the IP address and I can login by doing a
> relatively standard ssh user@$IP to obtain a shell - with ~/shared/ from
> my host system mounted at /mnt in the VM. Note that shared mounts don't
> currently work for FreeBSD due to missing drivers on the FreeBSD side - I
> believe those are in review for FreeBSD 15.0 currently.
>
> Hope that illustrates things a bit.
>
> Cheers,
> Ben
>
>
>>
>> On Sat, Jun 28, 2025 at 1:23 PM Ben Cooksley <bcooksley at kde.org> wrote:
>>
>>> Hi all,
>>>
>>> As part of our transition to VM based CI, i'm about to start cutting
>>> over the build infrastructure for the underlying images used to support our
>>> CI system from the current Docker model to the new VM based model.
>>>
>>> This follows the successful on-infrastructure build of a Windows image
>>> completed this evening, which was the last milestone component remaining to
>>> be validated when it came to building images.
>>>
>>> Because the system will continue to rely on the Docker images for a
>>> period of time while we're transitioning over the next week or two, we will
>>> temporarily be unable to add, update or remove any system level
>>> dependencies (including Qt) during this time period.
>>>
>>> More information on the transition will be published in the coming week
>>> once the images have been ported over, all successfully built, and a
>>> variety of test builds have been successfully accomplished using them.
>>>
>>> I'm anticipating the completion of migration to VM based CI to be
>>> completed within the next 2 weeks, if not sooner.
>>>
>>> Thanks,
>>> Ben
>>>
>>
--
Cheers,
Kristen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-devel/attachments/20250630/31ddc142/attachment-0001.htm>
More information about the kde-devel
mailing list