[sysadmin/ci-tooling] build-specs/Plasma: Disable execution of tests for plasma-integration.

[Albert Astals Cid]

El dissabte, 26 de gener de 2019, a les 9:13:45 CET, Ben Cooksley va escriure:
> On Sat, Jan 26, 2019 at 10:35 AM Albert Astals Cid <aacid at kde.org> wrote:
> >
> > El dilluns, 21 de gener de 2019, a les 6:46:32 CET, Ben Cooksley va escriure:
> > > Git commit f6c79ff4787148459aa91c17d683e4fd6a57c323 by Ben Cooksley.
> > > Committed on 21/01/2019 at 05:46.
> > > Pushed by bcooksley into branch 'master'.
> > >
> > > Disable execution of tests for plasma-integration.
> > > This is necessary to ensure CI nodes do not become blocked due to hanging tests withing plasma-integration.
> > >
> > > Currently, plasma-integration has several tests that make use of KIO slaves directly (skipping KLauncher).
> > > Unfortunately, they do not terminate the slaves prior to the conclusion of the test, resulting in the kioslave processes being left around afterwards.
> > > This is a condition that CTest will not tolerate, leading to it waiting indefinitely for these processes to exit - and in turn blocking all other builds on the CI node in question.
> > > While this is not a major issue in the case of Linux builds, it can quickly become a severe condition in the case of FreeBSD and Windows builds due to those builders being fixed rather than dynamically allocated.
> > >
> > > This class of issue (CTest waiting due to resident processes being left behind) has been a major issue as of late and is quickly leading to the CI system becoming unmaintainable due to the level of breakage.
> > > Should it be necessary to ensure the maintainability of the system, withdrawal of execution of tests for all projects is an option currently under consideration.
> >
> > Stopping running the tests is never the solution.
> >
> > We need to fix the tests.
> 
> That would be the ideal solution yes. Unfortunately in most cases I
> don't get a response from the developers involved with the code, at
> least not without some followup.
> Having to login to the various Linux container / FreeBSD VM / Windows
> VM machines every day to kill off hung processes isn't sustainable
> though.
> 
> >
> > And to fix the tests we need to be able to reproduce failures.
> 
> To my knowledge, with the exception of the Akonadi hangs, all of the
> other tests were either reproducible by the developers or I haven't
> heard back from them regarding the failures.
> 
> >
> > Is this error only happening on the CI? If so is there an easy way someone can reproduce the setup that happens in our CI?
> >
> > If not we should be working on that.
> 
> For Linux builds I can provide instructions for how to reproduce the
> CI environment - the majority of it comes from a Docker image so it
> should be fairly easy for people to get up and running.
> 
> You'll want a decent internet connection though as you'll need to
> download the KDE binaries built by the CI system, which are refreshed
> as part of the Dependency Builds each week and as the system performs
> builds - so these aren't something you can download once and keep
> reusing. To provide some context Frameworks alone represents about
> 1.7GB - and with applications having dependencies on other parts of
> KDE you'll need to download additional binaries on top of this.

Can we try to get this documented? 1.7GB isn't a lot for lots of people with fast internet (if the server serves fast enough).

Cheers,
  Albert

> 
> For FreeBSD it's somewhat more complicated as you'd need to setup a
> virtual machine that matches the CI environment (Tobias and Adriaan
> look after this so they'd know more about the specifics required
> here).
> 
> Windows is similar in needing you to setup a virtual machine matching
> the CI environment. Due to licensing restrictions we can't provide a
> disk image for Windows.
> 
> >
> > I understand your frustration when CI breaks, but I have seem that often the answer for such issues is "it works for me and i can't reproduce the CI problem".
> >
> > So if we had a simple "run docker with this parameter and then run these 3 scripts to reproduce the error" answer I'm sure more people would try to fix them. At least i might.
> 
> A few more steps than 3 is required, but for the most part that is the case yes.
> 
> >
> > Cheers,
> >   Albert
> 
> Cheers,
> Ben
> 
> >
> > >
> > > CCMAIL: plasma-devel at kde.org
> > > CCMAIL: kde-frameworks-devel at kde.org
> > > CCMAIL: release-team at kde.org
> > > CCMAIL: kdevelop-devel at kde.org
> > > CCMAIL: sysadmin at kde.org
> > >
> > > M  +2    -0    build-specs/Plasma/plasma-integration.yaml
> > >
> > > https://invent.kde.org/sysadmin/ci-tooling/commit/f6c79ff4787148459aa91c17d683e4fd6a57c323
> > >
> > > diff --git a/build-specs/Plasma/plasma-integration.yaml b/build-specs/Plasma/plasma-integration.yaml
> > > index 3d39455..159546c 100644
> > > --- a/build-specs/Plasma/plasma-integration.yaml
> > > +++ b/build-specs/Plasma/plasma-integration.yaml
> > > @@ -1,5 +1,7 @@
> > >  kf5-qt5:
> > > +  run-tests: False
> > >    force-inject-asan: True
> > >
> > >  stable-kf5-qt5:
> > > +  run-tests: False
> > >    force-inject-asan: True
> > >
> >
> >
> >
> >
>