KF 5.91: 24 modules with failing unit tests (Re: Please fix failing unit tests with Windows platform)

[Ben Cooksley]

On Sun, Feb 6, 2022 at 10:40 PM Friedrich W. H. Kossebau <kossebau at kde.org>
wrote:

> Am Montag, 24. Januar 2022, 01:06:40 CET schrieb Friedrich W. H. Kossebau:
> > Hi,
> >
> > since a long time there are lots of failing unit tests across multiple
> > repositories. Could the Windows platform maintainers/stakeholders please
> > look soonish into either fixing those tests or properly marking them as
> > expected to fail, so the resources the KDE CI spends on running the tests
> > every hour, day and week make some sense again, as well as having
> something
> > usable to diff results again, to notice any new regressions?
> >
> > Please see
> >
> >
> https://build.kde.org/job/Frameworks/view/Platform%20-%20WindowsMSVCQt5.15/
> > (best sort by "S" build status to get a list what need
>
> And those who believe in the broken windows theory also would claim this
> slacking now resulted in the regressions in the openSUSE builds, where 5
> modules now have failing unit tests at time of release tagging, when it
> once
> was 0 thanks to hard work of David F. and others. :(
>
> Is it time to remove
> https://community.kde.org/Frameworks/
> Policies#Frameworks_CI_failures_are_treated_as_stop_the_line_events
> because seemingly this is just old dead pixels on a web page and not the
> spirit these days?
>

Not sure that is the ideal outcome here - preferrably our tests would
continue to all pass.

I know some tests on certain platforms have been flaky and switch between
failing/passing - are we sure that isn't the driver of people ignoring the
results?

Cheers,
Ben


>
> Friedrich
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-frameworks-devel/attachments/20220207/0cfcd0de/attachment.htm>