Why is windows CI so variable?

[Ben Cooksley]

On Thu, Oct 13, 2022 at 2:11 PM Steven Robbins <steve at sumost.ca> wrote:

> Ben,
>

Hi Steven,


>
> Thanks very much again for all the information you provided.  It was very
> illuminating and answered all my questions to this point.
>
>
> On Wednesday, October 12, 2022 3:48:50 A.M. CDT Ben Cooksley wrote:
>
> > Additionally, Digikam is one of three projects that consume significant
> > amounts of CI resources, so i'd very much rather that didn't grow
> further.
>
> This brings a separate question to mind.  I've noticed that the windows
> build
> is hugely variable.  It can take anywhere from about 25 minutes to over an
> hour (at which point the job is killed).
>
> Have you any insight into why the windows machine is so variable compared
> to
> linux and bsd?
>

This will likely be due to the Linux and FreeBSD builds being able to make
use of ccache - which means the computationally expensive parts of the
build are for the most part accelerated/skipped over.
Windows unfortunately has no such benefit available to it, and therefore
needs to compile completely from scratch each time.

We currently have 3 workers connected to Gitlab - node3, node4 and node5 -
each one being a Hetzner AX51-NVME (
https://www.hetzner.com/dedicated-rootserver/ax51-nvme).
Each of these workers hosts two virtual machines, one for FreeBSD and
another for Windows (which are the nodeX-freebsd and nodeX-windows builders
you see in Gitlab) and also handle Linux Docker jobs (as nodeX).

As such what can happen sometime is that another CPU intensive job will
start on Linux or FreeBSD and slow down Windows builds. For the vast
majority of KDE projects this is a non-issue as they are fairly small and
only take a few minutes to complete under normal circumstances - and
therefore end up taking maybe 10-15 minutes under high load situations.

Digikam however is one of three projects (the other two being Krita and
KWin) that are high resource users of the CI system, due to the number of
builds you run and the amount of time your build and tests take to complete.

The best fix for this will likely be for us to deploy ccache on Windows
(see https://ccache.dev/releasenotes.html#_ccache_4_6).

Note that projects do not need to do anything specific to enable ccache
support by themselves, it will be enabled by a Sysadmin for your project if
it is suitable (it is enabled for all three high resource projects
mentioned above).
(Please don't attempt to do so yourself, it is just more likely you will
end up breaking things)


> THanks,
> -Steve
>

Thanks,
Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-devel/attachments/20221013/aec550e0/attachment.htm>