Review Request 122206: [kio] Make tests optional

[Matthew Dawson]

> On Jan. 23, 2015, 8:43 a.m., Vishesh Handa wrote:
> > I'm not against this, but I am curious as to why this is being done. 
> > 
> > I would think that packagers should be building the tests and running them on their platform and make sure everything passes. We have a strict policy that all tests must always pass.
> 
> Matthew Dawson wrote:
>     This is mostly useful on source based distributions (specifically, this patch comes from Gentoo).  While in general running tests everywhere would be great, source distro users may not have the cpu time to compile/run tests.  Also, some test suites don't work and users may not care to figure out why (for instance, last time I tried enabling tests in Gentoo, binutils failed its suite).
>     
>     For binary distriubtions, I agree they should be running tests (especially since we work to keep them green).  But source based distros aren't so clear cut.
> 
> Andreas Sturmlechner wrote:
>     Exactly, packagers do build the tests of course, but that does not mean users of source packages should have a permanent dependency on Qt5Test.
> 
> Vishesh Handa wrote:
>     It is even more important for source based distros to be running tests. They generally have very different compile options and flags. What is the point of them running the software and possibly finding bugs, when it could have been caught by just running the tests.
>     
>     Actually, the more I think about this, the more I realize that everyone should be running the autotests. -2 from my side. But I'm not the maintainer of kio.
> 
> Vishesh Handa wrote:
>     > Exactly, packagers do build the tests of course, but that does not mean users of source packages should have a permanent dependency on Qt5Test.
>     
>     On a source based distribution you already have a dependnecy on cmake, the compiler, and many other things. These are only required during build time. Qt5Test is the same. Once the pacakge has been built + tests have been run, Qt5Test can be removed.
> 
> Matthew Dawson wrote:
>     At least on Gentoo, by default build time dependencies are not automatically removed (though you can remove them if you want).  Generally speaking that is the right choice, as you will need cmake/compiler/etc. later to build the package when a version is released.  Also, one of the benefits of Gentoo is that the entire development toolchain sticks around, allowing for easy development/bug triaging.
>     
>     Anyways, source based distros won't always run tests, because users won't always want to run them.  In a perfect world, I agree that is wrong.  In reality, I don't run any test suites across any of my Gentoo installs.  So forcing the tests to build just burns CPU time, and is easily patched out by downstreams.  I don't think trying to force this will get KDE anywhere.
> 
> Vishesh Handa wrote:
>     > Anyways, source based distros won't always run tests, because users won't always want to run them.  In a perfect world, I agree that is wrong.  In reality, I don't run any test suites across any of my Gentoo installs.  So forcing the tests to build just burns CPU time, and is easily patched out by downstreams.  I don't think trying to force this will get KDE anywhere.
>     
>     - If the user doesn't want to run them, I'm sure Gentoo can provide some options for that. Compiling them cannot be such a huge cost.
>     - They are already burning CPU time by using a source based distro. This way they might actually catch some bugs and possibly not waste developers time by filing bugs which may have been an issue with their system.
>     
>     I'm not sure if I will approve such patches on packages I maintain.

I think we've both stated our piece here, and we aren't going to get further towards a consensus.  May I suggest posting this to the general kde-frameworks (or kde-core-devel, I'm not sure what would be better) seeking to make a general policy wrt Frameworks?  That way Frameworks has a consistent approach to building tests, whatever way the community decides.

In the meantime we should probably merge this patch, as building autotests without finding Qt5Test is only going to break builds.  Then packages can be updated with the policy decision by removing the BUILD_TESTING option.

The Plasma community should also probably come to a consensus for its packages as well.


- Matthew


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://git.reviewboard.kde.org/r/122206/#review74602
-----------------------------------------------------------


On Jan. 22, 2015, 2:48 p.m., Andreas Sturmlechner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://git.reviewboard.kde.org/r/122206/
> -----------------------------------------------------------
> 
> (Updated Jan. 22, 2015, 2:48 p.m.)
> 
> 
> Review request for KDE Frameworks.
> 
> 
> Repository: kio
> 
> 
> Description
> -------
> 
> [kio] Make tests optional
> This is a small patch to CMakeLists.txt to only depend on Qt5Test if BUILD_TESTING.
> 
> 
> Diffs
> -----
> 
>   CMakeLists.txt 7fe0be5d4b2d7d9475a7844b4f8d93fc2f0a00c3 
> 
> Diff: https://git.reviewboard.kde.org/r/122206/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Andreas Sturmlechner
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-frameworks-devel/attachments/20150123/60309193/attachment.html>