Excessive tests in Ruqola / dependencies not built as part of the Craft caches

[Ben Cooksley]

On Fri, Jul 17, 2026 at 12:00 AM laurent Montel <montel at kde.org> wrote:

> Le jeudi 16 juillet 2026, 13:35:05 heure d’été d’Europe centrale Ben
> Cooksley a écrit :
> > Hi Laurent, all,
>
> Hi,
>
> >
> > Over the past week or two Sysadmin has received a number of complaints
> > regarding a lack of CI builder availability, primarily from those who
> work
> > on Plasma. Investigation has revealed that Ruqola appears to use a
> > disproportionately high amount of CI time compared to the size of the
> > project and it's activity.
> >
> > This appears to be driven by the fact that Ruqola has a large number of
> > tests that do not take very long to execute (700 or so tests, yet done
> in 6
> > minutes - with many of these being less than 1 second to run). Given that
> > every single one of these unit tests needs to have automoc / compiler /
> > linker run on it i'd suggest that similar tests be merged into the same
> > compilation unit (you can run the individual tests, but keep similar
> tests
> > together in the same unit test).
>
> In ruqola each autotest depends against a specific class.
> So merging them in one autotest seems not logical.
> It's the same idea that merging autotests from kio in one autotest.
>

Sorry but you have 700 unit tests and there should be a way of grouping
them up that still remains logical when many of them take less than 1
second to run.
You can test multiple things in the same unit test and there is no reason
why some degree of merging can't be applied.

A degree of pragmatism reflecting the fact that 700 tests that only take a
few minutes to run for a codebase the size of Ruqola's needs to be
exercised here.

We are spending a significant amount of time compiling, generating code
coverage results, etc. due to the substantial number of tests, which is bad
enough on Linux (where threads are cheap and processes are expensive to
launch) but becomes even worse on Windows where poor I/O becomes a
constraint.


>
> After that I am not against to execute autotests only on linux-qt6-next
> It's the main target. but I need to know how to told to CI to exclude
> other platforms.
>
>
That isn't really possible i'm afraid.


>
> for example for freebsd, I can't debug so I don't need to execute autotest
> on it.
>
>
> > This would also explain why the build time
> > on Windows is significantly longer, as NTFS does not perform well when
> > compared to Linux file systems, even on NVMe storage.
> >
> > FreeBSD also has 8 tests failing (timing out, with a 1 minute timeout)
> and
> > Windows has three tests failing with timeout.
> >
> > If the Ruqola developers could please merge their unit tests into a
> smaller
> > number of unit tests and look into the broken unit tests as a priority
> that
> > would help your fellow KDE Developers by reducing the wastage of CI time.
> >
> > On the Craft based builds, Ruqola also uses a stable version of a
> > dependency (ktextaddons) that is not in the Craft cache.
>
> I don't understand why it's not in craft cache as it's used by
> ruqola/pim*/kaichat.
>
> I will look at it.
>
> > Projects making
> > use of Craft CD builds should review their blueprints and ensure
> where-ever
> > possible that their dependencies are all in the list of projects built in
> > the shared cache. In the case of KTextAddons, adding it to the cache
> should
> > save ~10 minutes on Windows and ~3-4 minutes on Linux (Appimage) per
> > pipeline run.
> >
> > Details on the shared cache can be found at
> > https://invent.kde.org/sysadmin/craft-ci. A merge request to correct
> this
> > oversight as well as any other project it uses (Pulseaudio also appears
> to
> > be a candidate) would be appreciated .
> > Other projects using Craft CD builds should please review their
> Blueprints
> > as well for similar oversights.
> >
> > Thanks,
> > Ben
> >


Cheers,
Ben


>
>
> Regards.
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-devel/attachments/20260717/1ccefbf9/attachment-0001.htm>