<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi,</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Regarding the krunner timer-based test which I authored:</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
We could just delete the few lines that are enforcing those upper-limit timeouts and add some comments explaining that in real life those timeouts should have been met, I don't think we need to delete the entire test. I was explicitly asked to write a test
when I submitted the MR to enforce that this functionally wouldn't regress in the future.<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Also, I'm wondering... if we happen to have a compilation directive that's #defined in this CI environment only, or some framework function that returns weather we are in the CI environment or not, we could put some #ifndef (or if's) on those few upper-limit
timeout lines and avoid compiling/running just those lines in the CI environment. I'm not familiar with this environment so I don't know if there is such a thing, I'm just wondering...</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I'm available to submit a quick MR if you'd like me to, just give me some directions for what you'd like me to do.<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
[],</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Eduardo<br>
</div>
<div id="appendonsend"></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Ben Cooksley <bcooksley@kde.org><br>
<b>Sent:</b> Sunday, March 13, 2022 1:53 PM<br>
<b>To:</b> KDE Frameworks <kde-frameworks-devel@kde.org><br>
<b>Cc:</b> eduardo.cruz@kdemail.net <eduardo.cruz@kdemail.net><br>
<b>Subject:</b> Re: Unit tests all pass in Jenkins on Linux</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div dir="ltr">On Mon, Mar 14, 2022 at 4:40 AM David Faure <<a href="mailto:faure@kde.org">faure@kde.org</a>> wrote:<br>
</div>
<div class="x_gmail_quote">
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
After the recent discussions on state of CI, I fixed the last unittest failures (kio, purpose... + apol fixed ECM) so that<br>
<a href="https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.15/" rel="noreferrer" target="_blank">https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.15/</a>
<br>
is all green^H^Hblue again.<br>
Please keep it that way!<br>
</blockquote>
<div><br>
</div>
<div>Thanks for looking into and fixing all of these David.</div>
<div> </div>
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<br>
Note however that<br>
<br>
* kwayland has a flaky test:<br>
<br>
<a href="https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.15/job/kwayland/job/kf5-qt5%20SUSEQt5.15/171/testReport/junit/projectroot.autotests/client/kwayland_testDataDevice/" rel="noreferrer" target="_blank">https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.15/job/kwayland/job/kf5-qt5%20SUSEQt5.15/171/testReport/junit/projectroot.autotests/client/kwayland_testDataDevice/</a><br>
<br>
FAIL! : TestDataDevice::testReplaceSource() Compared values are not the same<br>
Actual (selectionOfferedSpy.count()): 1<br>
Expected (2) : 2<br>
Loc: [autotests/client/test_datadevice.cpp(557)]<br>
<br>
Who can look at this one? git log mostly shows Martin Flöser <<a href="mailto:mgraesslin@kde.org" target="_blank">mgraesslin@kde.org</a>><br>
who I think isn't active anymore?<br>
</blockquote>
<div><br>
</div>
<div>Not sure if it applies to KWayland as well, but I know that KWin has load sensitive tests (which is why the Gitlab .kde-ci.yml files support the flag <span style="color:rgb(0,0,0); font-family:monospace">tests-load-sensitive</span>)</div>
<div>If this test appears to be flaky, then it is quite possible that it is load sensitive as well.</div>
<div> </div>
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<br>
* krunner has a flaky test [2] because it measures time spent and expects small values like 65ms<br>
(I changed that one to 100ms), 250ms, 300ms. With only 10% safety margins. On a busy CI system,<br>
this is bound to fail regularly, even with bigger safety margins. In my experience this kind of test<br>
is just not possible (we're not running on a real time OS), I vote for removing the test.<br>
CC'ing Eduardo.<br>
<br>
<a href="https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.15/job/krunner/job/kf5-qt5%20SUSEQt5.15/325/testReport/junit/projectroot/autotests/runnermanagertest/" rel="noreferrer" target="_blank">https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.15/job/krunner/job/kf5-qt5%20SUSEQt5.15/325/testReport/junit/projectroot/autotests/runnermanagertest/</a></blockquote>
<div><br>
</div>
<div>Yes, that will definitely fail more often than not - your only way to make sure tests like this pass on our CI system is to set tests-load-sensitive=True (in Gitlab CI)</div>
<div>Note however that option should be avoided where possible as it means your build will stop and wait for load to fall to low levels before proceeding with running tests - which blocks a CI worker slot from being used by another project.</div>
<div><br>
</div>
<div>I'd also be in favour of removing this test.</div>
<div><br>
</div>
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<br>
<br>
-- <br>
David Faure, <a href="mailto:faure@kde.org" target="_blank">faure@kde.org</a>, <a href="http://www.davidfaure.fr" rel="noreferrer" target="_blank">
http://www.davidfaure.fr</a><br>
Working on KDE Frameworks 5<br>
<br>
<br>
<br>
</blockquote>
<div><br>
</div>
<div>Cheers,</div>
<div>Ben </div>
</div>
</div>
</div>
</body>
</html>