CI Utilisation and system efficiency

[Ben Cooksley]

On Sun, Apr 20, 2025 at 3:05 AM Thomas Friedrichsmeier <
thomas.friedrichsmeier at kdemail.net> wrote:

> On Sat, 19 Apr 2025 07:25:36 +1200
> Ben Cooksley <bcooksley at kde.org> wrote:
> [...]
> > This morning I ran a basic query to ascertain the top 20 projects for
> > CI time utilisation on invent.kde.org which revealed the following:
>
> What (rough) percentage is that of the total? I.e. is the problem more
> one of a few projects hogging up most resources, or is it waste across
> the board? (Also keeping in mind that using a lot of resources may also
> be a sign of a lot of development activity).


The totals as at right now (time has moved on so have re-run the numbers)
are:

    time_used     | job_count
------------------+-----------
3536:44:31.03921 |     45860

          full_path           |    time_used     | job_count
------------------------------+------------------+-----------
plasma/kwin                  | 325:40:45.439134 |      2419
plasma/plasma-workspace      | 224:56:33.415042 |      1032
multimedia/kdenlive          | 195:29:20.224873 |       794
graphics/krita               | 189:03:12.118222 |       441
network/ruqola               | 180:30:09.57436  |       585
network/neochat              | 151:25:19.726593 |      1695
education/kstars             | 129:53:37.649402 |       333
sysadmin/ci-management       | 116:32:43.423853 |       163
plasma/plasma-desktop        | 116:07:05.90223  |       818
kde-linux/kde-linux-packages | 84:45:50.156624  |        35

So KWin represents 9.19% of all CI time utilised on invent.kde.org which
even if we assume it is one of the more active projects seems excessive.

When I started writing this query I fully expected it to be dominated by
applications, especially ones targeting Android (multiple architectures),
due to their both conventional CI jobs and CD jobs, with KWin featuring
somewhere on the list.
Yet it is two Plasma projects that take the top crowns for biggest CI
resource users, and by quite some margin.

Kdenlive is being negatively impacted right now by the MingW cache for
Craft being somewhat broken, which is being worked on.

Those totals are also diluted by the fact that kde-linux-packages runs
separately and does not utilise the resources of the main build cluster.
Some of those applications in the above will be utilising macOS builds as
well (which once again isn't part of the main build cluster).

RKWard is only 0.6% of total CI time so i'm not worried about that - that
is perfectly reasonable.


>
>
> For RKWard, I believe it's our deployment jobs that take up a
> disproportionate amount of time (and I have tossed out one of those,
> now). Personally, I would be quite happy, if those would run with some
> delay (possibly with an option to trigger them early, manually). No
> idea, whether a change like that would make a noticeable difference, if
> applied across the board.
>
> Regards
> Thomas
>

Thanks,
Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-devel/attachments/20250420/18ae4215/attachment-0001.htm>