Plasma CI Health Check

[Nate Graham]

This is great, thanks so much for taking initiative to do it!

Nate



On 5/11/26 7:12 AM, Kristen McWilliam wrote:
> Hello, and welcome to the new Plasma CI Health Check!
> 
> You may be familiar with the "KDE Gear projects with failing CI" 
> project/emails that Albert has been doing for some time — I will be 
> doing similar for Plasma, to ensure that our CI is healthy and improve 
> our testing strategy.
> 
> We'll be looking at a few interesting metrics, for example whether:
> 
> • CI is passing or failing on master
> 
> If CI is failing on master its usually a bad sign, and something we want 
> to fix ASAP.
> 
> 
> • Tests are being automatically retried
> 
> Having the CI configured to auto retry tests is usually a sign of a 
> workaround for flaky tests, and flaky tests are bad tests; at best they 
> are frustrating and a waste of CI resources, at worst they give a false 
> sense of security that code is actually reliable. A test should either 
> pass or fail — if it doesn't do so reliably it should be fixed or removed.
> 
> In the .kde-ci.yml this will look something like:
> 
> ```yaml
> Options:
>   ctest-arguments: '--repeat until-pass:5'
> ```
> 
> We should prefer to remove this option and fix the tests.
> 
> 
> • Tests are configured as required passing for the repo
> 
> With this enabled, if tests fail in the CI the associated merge request 
> will show this as a nice obvious warning; if it isn't enabled, the MR 
> will report the pipeline is completely green — not good!
> 
> We definitely want it to be obvious if the tests failed, otherwise 
> people are likely to merge it without even being aware there is an issue.
> 
> In the .kde-ci.yml this will look like:
> 
> ```yaml
> Options:
>   require-passing-tests-on: ['@all']
> ```
> 
> Knowing if the tests are passing or failing is super important, so we 
> need to work towards this being enabled for all our repos.
> 
> 
> ---------------------------
> 
> 
> There will be additional things to track to assess our CI and test 
> health, but this is a good place to start.
> 
> Here are our initial metrics:
> 
> • 86 monitored repos
> • 10 failing CI in master
> • 3 configured for auto-repeating tests
> • 14% of repos configured to require passing tests
> 
> 
> Failing CI:
> 
> • plasma/qqc2-breeze-style
> • plasma/plasma-desktop
> • plasma/kmenuedit
> • plasma/discover
> • plasma/lightdm-kde-greeter
> • plasma/kde-gtk-config
> • plasma/plasma-sdk
> • plasma/plasma-systemmonitor
> • plasma/latte-dock
> • plasma-mobile/spacebar
> 
> 
> Auto-retrying tests:
> 
> • kwin-x11
> • kwin
> • plasma-desktop
> 
> 
> I hacked together a simple dashboard to help visualize the state of 
> things, which also includes links to the relevant CI run for each repo; 
> if interested in checking, it lives here for the moment:
> 
> https://ci-healthcheck-5e5c80.local-kde.org/
> 
> 
> Thank you for helping to improve the reliability of our software, and 
> reducing wasted CI resources.
>