CI Utilisation and system efficiency
Albert Astals Cid
aacid at kde.org
Sun Apr 20 10:06:48 BST 2025
El dissabte, 19 d’abril del 2025, a les 10:47:13 (Hora d’estiu d’Europa
central), Ben Cooksley va escriure:
> On Sat, Apr 19, 2025 at 7:25 AM Ben Cooksley <bcooksley at kde.org> wrote:
> > Hi all,
> >
> > Over the past week or two there have been a number of complaints regarding
> > CI builder availability which i've done some investigating into this
> > morning.
> >
> > Part of this is related to the Windows CI builders falling offline due to
> > OOM events, however the rest is simply due to a lack of builder time
> > availability (which is what this email is focused on).
> >
> > Given we have 6 Hetzner AX51 servers connected to Gitlab (each equipped
> > with a Ryzen 7 7700 CPU, 64GB RAM and NVMe storage) the issue is not
> > available build power - it is the number of builds and the length of those
> > builds that is at issue.
> >
> > This morning I ran a basic query to ascertain the top 20 projects for CI
> >
> > time utilisation on invent.kde.org which revealed the following:
> > full_path | time_used | job_count
> >
> > ------------------------------+------------------+-----------
> > plasma/kwin | 320:47:04.966412 | 2387
> > graphics/krita | 178:03:19.080763 | 423
> > multimedia/kdenlive | 174:08:09.876842 | 697
> > network/ruqola | 173:17:47.311305 | 555
> > plasma/plasma-workspace | 155:10:03.618929 | 660
> > network/neochat | 138:03:23.926652 | 1546
> > education/kstars | 129:49:17.74229 | 329
> > sysadmin/ci-management | 111:21:09.739792 | 154
> > plasma/plasma-desktop | 108:56:52.849433 | 776
> > kde-linux/kde-linux-packages | 81:00:10.001937 | 33
> > kdevelop/kdevelop | 59:40:51.54474 | 217
> > office/kmymoney | 54:32:00.24623 | 271
> > frameworks/kio | 53:54:19.046685 | 690
> > education/labplot | 52:36:30.343671 | 245
> > murveit/kstars | 52:32:56.882728 | 128
> > frameworks/kirigami | 47:07:19.172935 | 1627
> > system/dolphin | 46:09:58.02836 | 705
> > kde-linux/kde-linux | 39:25:54.052469 | 46
> > utilities/kate | 36:09:22.18958 | 356
> > wreissenberger/kstars | 35:58:14.120515 | 105
> >
> > If we look closely, KStars has three spots on this list (totalling 216
> > hours of time used, making it the biggest app user of CI time).
> >
> > Projects on the above list are asked to please review their jobs and how
> > they are conducting development to ensure CI time is used efficiently and
> > appropriately.
> >
> > Other projects should also please review their usage and optimise
> > accordingly even if they're not on this list as there is efficiencies to
> > be
> > found in all projects.
> >
> > When reviewing the list of CI builds projects have enabled, it is
> > important to consider to what degree your project benefits from having
> > various builds enabled. One common pattern i've seen is having Alpine,
> > SUSE
> > Qt 6.9 and SUSE Qt 6.10 all enabled.
> >
> > If you need to verify building on Alpine / MUSL type systems and wish to
> > monitor for Qt Next regressions then you probably shouldn't have a
> > conventional Linux Qt stable build as those two jobs between them already
> > cover that list of permutations.
>
> Thinking through this further...
>
> All projects that have enabled Linux Qt 6.10 (Linux-Next) which also have
> FreeBSD or Windows builds shouldn't really have Linux Qt 6.9 builds enabled
> unless that job has other specific benefits (ie. Alpine and you're
> monitoring buildability with another libc)
>
> My logic behind this is:
> - Linux Qt 6.10 monitors buildability on glibc + gcc + Qt 6.10
> - Linux Qt 6.9 monitors buildability on glibc + gcc + Qt 6.9
> - Alpine Qt 6.9 monitors buildability on musl + gcc + Qt 6.9
> - FreeBSD Qt 6.9 monitors buildability on BSD libc + Clang + Qt 6.9
> - Windows Qt 6.9 monitors buildability on MS CRT + MSVC + Qt 6.9
>
> In the above combination sets, a project with Linux Qt 6.10 may also have
> one or more of Alpine Qt 6.9, FreeBSD Qt 6.9 and Windows Qt 6.9.
>
> It must never have Linux Qt 6.9 unless it is a Linux only project that does
> not support musl - because coverage for glibc / gcc / Qt 6.9 has already
> been achieved by Linux Qt 6.10 (for the first two) and any of the other
> jobs (for Qt 6.9).
No.
Linux-next is a test tool that is broken to break when the Qt developers do
some mistake, we can't use it as a "you have this so you don't need that".
Also, we specifically got money for a new CI machine to cover linux-next, so
using it for "you need to remove some other CI if you are using it" excuse
seems uncalled for.
Cheers,
Albert
>
> KWin and Ruqola must therefore under the above logic drop Linux Qt 6.9 as
> they are using Linux Qt 6.10 and are making use of FreeBSD / Windows /
> Alpine builds.
>
> > I've taken a quick look at some of these and can suggest the following:
> >
> > KWin: it has two conventional Linux jobs (suse_qt69 and suse_qt610) plus a
> > custom reduced feature set job. It seems like one of these conventional
> > Linux jobs should be dropped.
> >
> > KStars: Appears to have a custom Linux job in addition to a conventional
> > Linux job. Choose one please.
> >
> > Ruqola: Appears to be conducting a development process whereby changes are
> > made in stable then immediately merged to master in a ever continuing
> > loop.
> > Please discontinue this behaviour and only periodically merge stable to
> > master.
> >
> > Also needs to drop one of it's Linux jobs as they're duplicating
> > functionality as noted above.
> >
> > Plasma Workspace/Desktop: At least in part this seems to be driven by
> > Appium tests. Please reduce the number of these and/or streamline the
> > process for running an Appium test. Consideration should be given to
> > enabling the CI option use-ccache as well.
> >
> > KDevelop: Please enable the CI option use-ccache.
> >
> > Labplot: Appears to have a strange customisation in place to the standard
> > jobs which shouldn't be necessary as flags in .kde-ci.yml should permit
> > that to be done.
> >
> > Thanks,
> > Ben
>
> Cheers,
> Ben
More information about the Kstars-devel
mailing list