Deploy stage manually on master, unit test build time technologies (was: Re: Excessive tests in Ruqola [...])

[Friedrich W. H. Kossebau]

Hi all,

Am Donnerstag, 16. Juli 2026, 13:35:05 Mitteleuropäische Sommerzeit schrieb 
Ben Cooksley:
> Over the past week or two Sysadmin has received a number of complaints
> regarding a lack of CI builder availability, primarily from those who work
> on Plasma. Investigation has revealed that Ruqola appears to use a
> disproportionately high amount of CI time compared to the size of the
> project and it's activity.
> 
> This appears to be driven by the fact that Ruqola has a large number of
> tests that do not take very long to execute (700 or so tests, yet done in 
6
> minutes - with many of these being less than 1 second to run). Given that
> every single one of these unit tests needs to have automoc / compiler /
> linker run on it i'd suggest that similar tests be merged into the same
> compilation unit (you can run the individual tests, but keep similar tests
> together in the same unit test). This would also explain why the build 
time
> on Windows is significantly longer, as NTFS does not perform well when
> compared to Linux file systems, even on NVMe storage.

DEPLOY STAGE "TESTS"

When it comes to "Excessive tests" could we please also consider to make for 
master/development/MR branches the "Deploy" stage on-demand, instead of 
always running it together with the "Build" stage? That would scale across 
all apps. And by my rough estimate by some samples I looked at save half of 
those projects' CI time. And most activity happens on master/development/MR 
branches, so this would scale even more.
For release branches always running the deploy stage seem reasonable, there 
one wants to properly check everything as early as possible for regressions.

From what I understand, those package builds (on master/development/MR 
branches) are just done to test if packaging still works. So a plain test 
run, just on package level. Taking roughly the full time of the normal 
build. Sometimes even (much) longer, despite not building also normal (unit) 
tests and running them.

I very doubt that the resulting package artifacts on those branches are 
actually used by anyone on a regular basis. There is no official CD also, is 
there? (Nothing related spotted on apps.kde.org or heard about). So it seems 
just a "does it package" test. With massive resource usage tag, outnumbering 
any normal unit tests.

So could we please reduce the test package/deploy builds on master?
Make them manually activateable only, and perhaps have them run 
automatically only as part of the weekly "Albert" run? Or as part of some 
nightly CD system at some point?

Again, not done the full numbers, but my gut feeling from those samples I 
saw is that this will have a noticeable effect on CI resource uages.


TESTS RUN ONLY ON ONE PLATFORM

Disabling the unit tests on some platforms runs the risk to not cover 
characteristics of that platform, BTW. Not ideal, is it. Ruining some 
purpose of a CI.


TECHNOLOGIES WANTED FOR REDUCING UNIT TEST BUILD TIME

Enabling some kind of unity build for tests might be something nice to have 
also IMHO. Because there cannot be enough tests usually, and Ruqola seems in 
a very healthy state here when it comes to coverage, at least by the number 
of unit tests. Just exposing that way some badly scaling costs.

Personally would be interested into any technologies/approaches to merge 
builds of unit tests, without affecting the actual test setups. By my pet 
example Okteta, building the unit tests takes rougly half or more of the 
time of building the actual product artifacts. A pain. And I would like to 
have more tests there actually :)

Those into the Qt development world, would you know any related approaches, 
when it comes to Qt testing frameworks? I'd be interested to explore this 
actively together with others.

Cheers
Friedrich