<div dir="ltr"><div dir="ltr">On Mon, Mar 14, 2022 at 4:40 AM David Faure <<a href="mailto:faure@kde.org">faure@kde.org</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">After the recent discussions on state of CI, I fixed the last unittest failures (kio, purpose... + apol fixed ECM) so that<br>
<a href="https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.15/" rel="noreferrer" target="_blank">https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.15/</a> <br>
is all green^H^Hblue again.<br>
Please keep it that way!<br></blockquote><div><br></div><div>Thanks for looking into and fixing all of these David.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Note however that<br>
<br>
* kwayland has a flaky test:<br>
<br>
<a href="https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.15/job/kwayland/job/kf5-qt5%20SUSEQt5.15/171/testReport/junit/projectroot.autotests/client/kwayland_testDataDevice/" rel="noreferrer" target="_blank">https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.15/job/kwayland/job/kf5-qt5%20SUSEQt5.15/171/testReport/junit/projectroot.autotests/client/kwayland_testDataDevice/</a><br>
<br>
FAIL! : TestDataDevice::testReplaceSource() Compared values are not the same<br>
Actual (selectionOfferedSpy.count()): 1<br>
Expected (2) : 2<br>
Loc: [autotests/client/test_datadevice.cpp(557)]<br>
<br>
Who can look at this one? git log mostly shows Martin Flöser <<a href="mailto:mgraesslin@kde.org" target="_blank">mgraesslin@kde.org</a>><br>
who I think isn't active anymore?<br></blockquote><div><br></div><div>Not sure if it applies to KWayland as well, but I know that KWin has load sensitive tests (which is why the Gitlab .kde-ci.yml files support the flag <span style="color:rgb(0,0,0);font-family:monospace">tests-load-sensitive</span>)</div><div>If this test appears to be flaky, then it is quite possible that it is load sensitive as well.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
* krunner has a flaky test [2] because it measures time spent and expects small values like 65ms<br>
(I changed that one to 100ms), 250ms, 300ms. With only 10% safety margins. On a busy CI system,<br>
this is bound to fail regularly, even with bigger safety margins. In my experience this kind of test<br>
is just not possible (we're not running on a real time OS), I vote for removing the test.<br>
CC'ing Eduardo.<br>
<br>
<a href="https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.15/job/krunner/job/kf5-qt5%20SUSEQt5.15/325/testReport/junit/projectroot/autotests/runnermanagertest/" rel="noreferrer" target="_blank">https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.15/job/krunner/job/kf5-qt5%20SUSEQt5.15/325/testReport/junit/projectroot/autotests/runnermanagertest/</a></blockquote><div><br></div><div>Yes, that will definitely fail more often than not - your only way to make sure tests like this pass on our CI system is to set tests-load-sensitive=True (in Gitlab CI)</div><div>Note however that option should be avoided where possible as it means your build will stop and wait for load to fall to low levels before proceeding with running tests - which blocks a CI worker slot from being used by another project.</div><div><br></div><div>I'd also be in favour of removing this test.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
<br>
-- <br>
David Faure, <a href="mailto:faure@kde.org" target="_blank">faure@kde.org</a>, <a href="http://www.davidfaure.fr" rel="noreferrer" target="_blank">http://www.davidfaure.fr</a><br>
Working on KDE Frameworks 5<br>
<br>
<br>
<br></blockquote><div><br></div><div>Cheers,</div><div>Ben </div></div></div>