KDE-buildsystem-basics package ?

[Alexander Neundorf]

Hi,

nice that we got the discussion going :-)

On Saturday 21 January 2012, Stephen Kelly wrote:
> Alexander Neundorf wrote:
> > Beside that, none of the stuff in
> > FindKDE4Internal.cmake/KDE4Macros.cmake/KDE4Defaults.cmake is actually
> > _necessary_ to build software.
> > It "only" does two things:
> > * it makes developing for "KDE" more convenient, e.g. it presets RPATH
> > settings, so developers have to care less
> > * it helps with consistency among stuff which uses them, like e.g. the
> > install directories
> 
> In that case I wonder if some of this stuff wouldn't make sense as
> something which belongs 'on top of' KDE frameworks, so that KDE
> application developers get the convenience, but that convenience is not
> entirely there (or necessary, maybe) for frameworks. Or maybe this stuff
> belongs 'somewhere' in KDE frameworks, but not necessarily in 'tier 0'
> which everything else depends on.
> 
> Could it even be optional and not used if not found?

I thought about that too. My result: not really.
E.g.
install(FILES foo.h DESTINATION ${INCLUDE_INSTALL_DIR} )

INCLUDE_INSTALL_DIR must be set somewhere, and making it optional whether it 
comes from some package or defining it in the project itself will make for 
really ugly and long, i.e. hard to maintain, CMakeLists.txt in all the 
framework libraries.

> > On Friday 20 January 2012, Stephen Kelly wrote:
> >> Alexander Neundorf wrote:
> >> > Hi,
> >> > 
> >> > any comments ?
> >> > IOW: this is about introducing a small package which everything in KDE
> >> > frameworks would depend on.
> >> 
> >> I see the problem, but I'm not a big fan of the proposed solution.
> >> 
> >> > On Saturday 14 January 2012, Alexander Neundorf wrote:
> >> >> Hi,
> >> >> 
> >> >> we have more or less the following groups of stuff in
> >> >> FindKDE4Internals.cmake, KDE4Macros.cmake and KDE4Defaults.cmake:
> >> >> 
> >> >> Settings/switches/options:
> >> >> * the install dirs [1]
> >> >> 
> >> >> > [1] I posted an email about our install dir issue to the cmake-
> >> 
> >> developers
> >> 
> >> > list:
> >> http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/2595
> >> 
> >> > Brad explained that GNUInstallDirs.cmake is explicitely implementing
> >> > the install dirs compatible to autotools, so it is meant for projects
> >> > coming from there. Since this is not what we have in KDE, we do not
> >> > necessarily have to use it. So basically he also recommends to have a
> >> > separate solution for KDE for this. I agree with him on this.
> >> 
> >> The GNU equivalent lives in CMake. Could the KDE equivalent live in ECM
> >> sensibly? Couldn't a 3rd party follow the KDE framework conventions
> >> while not depending on an KDE libraries (eg Grantlee, or anything in
> >> kdesupport).
> > 
> > Well, by following the KDE conventions it would depend on something
> > provided by KDE, not a library, but the package provided by KDE which
> > contains those settings. It would be a build-time dependency to KDE.
> > 
> >> >> * the cmake policy settings
> >> 
> >> Any reason this needs to be global for all frameworks?
> > 
> > E.g. so that all macros und functions provided right now in
> > KDE4Macros.cmake work as expected. E.g. the duplicate target names. Or to
> > silence warnings if new policies are introduced, because KDE developers
> > would think something has been broken. Or to enable the OLD behaviour, so
> > that stuff they did before stays the recommended way to do it.
> > The average developer doesn't know about CMake policies, so we should
> > preset them to what we consider sensible (or necessary in some cases).
> 
> Is KF5 a good time to drop the old policies and start fresh? And accept the
> policies defined by:
> 
> cmake_minimum_required(VERSION 2.8.8)

We have the chance now to break compatiblity, so for most things, yes.
Probably not for all.
The problem is the following: we define a minimum cmake version required to 
build KDE. Developer A uses this version.
Then a new version of cmake is released, and introduces a policy, which 
changes some behaviour.
Developer A updates to this version, and suddenly sees warnings, not errors, 
due to that new policy.
He fixes the warnings, and now the new cmake version is quiet.
But do the modified CMakeLists.txt still work properly also with the minimum 
required version of cmake, or did he just break the build for everybody else ?

This is uncertain.
Nevertheless we should go through the list of policies which are set and check 
whether we still want them this way.

E.g. setting CMP0005 to NEW may very well break the build in some places, or 
keep it building but produce not-correctly-working results.


> I'm not familiar with the duplicate target names issue you mentioned.

This is used somewhere for the test targets.
It would be nice if we find a way to get rid of that. There were some 
discussions about similar things on the cmake list last year. I have to check.
 
> >> >> * typical settings, like KDE4_BUILD_TESTS
> >> 
> >> Looks like something that could go into CMake.
> > 
> > I don't think so.
> > This is just a cmake option to be used to put around the directories
> > where the tests reside. All it does is providing a common name for this
> > option: option(FOO ...)
> > if(FOO)
> > 
> >   add_subdirectory(tests)
> > 
> > endif()
> 
> looks like it's already in CMake?
> 
> include(CTest)
> if (BUILD_TESTING)
>   add_subdirectory(tests)
> endif()

See below...

> >> >> , KDE4_ENABLE_FPIE
> >> 
> >> This isn't very different to other compiler and linker flags.
> >> 
> >> >> * RPATH settings
> >> 
> >> Also just a special case of linker flags? Or does this do something
> >> more?
> > 
> > Well, it sets the cmake RPATH related settings.
> > The default is empty install RPATH. Everything else is project-specific,
> > as our settings are KDE specific.
> 
> It's not enough to do this?:
> 
> set( CMAKE_INSTALL_NAME_DIR ${CMAKE_INSTALL_PREFIX}/${LIB_INSTALL_DIR})
> set( CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
 
Almost.
Even if it would be completely enough, we can rely on that the average 
developer will not set them correctly always from the start. He may set them 
correctly once his package has been released, and people started to report 
problems.

Additionally, you may have to add your own LIB_INSTALL_DIR to the RPATH (if 
you install libraries or executables which link against other shared libraries 
in the same project).
But you should add it only if your current LIB_INSTALL_DIR is not a system 
library dir.
It was discussed already on the cmake-developers list and I think there is 
also an entry in the cmake bug tracker to make this last condition 
automatically, but it was decided that this should not be automatic.
If the developer says "use dir A in the RPATH", then dir A will be in the 
RPATH.

> >> >> * compiler and linker flags
> >> 
> >> I'm not sure how to handle this. I think it can sensibly be in ECM for
> >> the same reason as the install dirs (conforming to the KDE norms without
> >> depending on anything specifically KDE). Some compiler flags (and Qt
> >> flags like QT_STRICT_ITERATORS)are just sensible, and if building
> >> sensible software based on Qt they should be used.
> >> 
> >> I know you disagree.
> > 
> > Well, I'd replace "... they should be used." with "...we think they
> > should be used."
> 
> Yes, where 'we' could mean 'the ecm developers, who through experience with
> building KDE have experience with these flags'.

Ok, I think at least we understand what we want to say :-)
 
> >> >> , kde4_add_unit_test, kde4_add_executable
> >> 
> >> I don't know. I'd be interested to see what they look like after
> >> removing the automoc stuff and seeing what's left and if refactoring is
> >> in order.
> > 
> > kde4_add_unit_test() does not depend on anything, it is mostly just
> > convenience.
> 
> Is the convenience that
> 
> add_executable(testname ${testname_SRCS})
> add_test(testname testname)
> 
> is reduced to one line? Is that something for ecm?

It does a lot more. Have a look at it, it's in KDE4Macros.cmake. And it is 
very KDE specific.
What I wrote before about KDE4_BUILD_TESTS was actually wrong.
The test executable targets are always part of the build, but if 
KDE4_BUILD_TESTS is not set, they are EXCLUDE_FROM_ALL, and a target 
"buildtests" is created, which all such test executables depend on.
This was discussed on the buildsystem list June 2007, mainly with apaku, 
David, Thiago and me.
Additionally, kde4_add_executable() creates wrapper shell/batch scripts, which 
set PATH/LD_LIBRARY_PATH/DYLD_LIBRARY_PATH so that the executables can also be 
executed uninstalled if RPATH is disabled or under Windows.

> >> I don't see why it would need to be in a directory. It can just be a few
> >> files. Maybe even just one.
> > 
> > Because the idea of e-c-m is to make stuff, which can be used independent
> > from KDE (before this meant "no kdelibs", what does it mean then ?)
> > available for non-KDE software.
> 
> It might be odd to consider that 'everything except KDE is in scope for
> ECM'. That is, we can add stuff for helping to build Qt or boost or any

CMake or e-c-m is not providing any required stuff for building Qt or any 
other project.
It contains stuff which help using Qt or boost. But it does not contain stuff 
decided upon by the Qt or boost communities, and then put into CMake.

> number of other packages, but adding stuff for KDE conventions is
> off-limits for ECM? I guess it could be understandable considering that
> KDE is capable of providing its own stuff. (unless it's not, which is the
> subject of this discussion).
> 
> > Putting the cmake files, which kind of make an application a KDE
> > application, there, turns e-c-m into a KDE package.
> 
> I think it depends on what KDE stuff is put in there exactly. Does having
> GNUInstallDirs in CMake make CMake a GNU package?

I think this is different.
With GNUInstallDirs.cmake, cmake is just providing something which follows a 
"standard". If e.g. KDEInstallDirs.cmake would be part of CMake, CMake would 
ship the file which define the standard, but the standard would not be decided 
by the CMake developers, but by the KDE developers.

> > The KDE stuff which would be in there would not be general purpose, it is
> > what we, the KDE developers, consider good for KDE applications and
> > libraries.
> > 
> > So at the very least, I'd put that into a separate directory.
> 
> I don't see the value in a separate directory, but if you do, then maybe
> that's one way to solve the problem at hand.
> 
> >> >> * an extra package kde-frameworks-buildsystem or something like this
> >> 
> >> Similar to above, that might mean adding a KDE dependency to everything
> >> in kdesupport (which have already decided not to have KDE dependencies).
> > 
> > No, not really.
> > What is a "KDE dependency" ? A dependency to anything hosted by KDE ?
> > Anything developed by KDE developers ?
> > Then e-c-m is also a "KDE dependency".
> > 
> > If e-c-m is not considered a "KDE dependency", then why should, let's
> > call it KDEFrameworksBuildsystem, be considered a "KDE dependency". It
> > would be very similar.
> 
> Personally I would not call ecm a KDE dependendency because it has a
> broader scope. It has find modules and macros (taking some at random:
> FindFlac, FindPCRE, CheckPointerMember) which have no relation to KDE and
> which can be used by depending on ecm.
> 
> If it's going to be possible for 3rd parties to create KDE frameworks tier1
> libraries (is that wanted?),

Here we are again in the gray area what makes a library a KDE library. Even 
more so a KDE tier 1 library.

> then ECM seems like a reasonable place to enable that to some extent.
> 
> > Another issue with putting all small things into different places, e.g.
> > some macros into e-c-m, some into kcore, some into kdocumentation, etc.,
> > is that the macros will all change their name.
> > Right now they all have the "KDE4_" prefix, because they come from
> > FindKDE4Internal.cmake. And only stuff from that file uses this prefix.
> 
> Either a script does
> 
> s/kde4_install_kcfg/kf5_install_kcfg/ig
> 
> or
> 
> s/kde4_install_kcfg/kcore_install_kcfg/ig
> 
> (or whatever macros)
> 
> It doesn't really matter to the script, but in kf5, the KDE4_ prefix
> doesn't make much sense.

Yes, keeping the KDE4_ prefix wouldn't make sense.
But using one common prefix ("KDE5", "KF5", "KDE", ...) would be easier to 
remember.

Now I actually figured out why I have a slightly bad feeling with distributing 
the macros to the packages to which they belong: maintainance.
Right now in kdelibs, they are all in one place. 
FindKDE4Internal.cmake+KDE4Macros.cmake+KDE4Defaults.cmake.
If I keep an eye on that, everything is mostly fine.
If it will be split, the macros etc. will be distributed over many modules.
I doubt that every KDE frameworks library maintainer will maintain the cmake 
macros properly. At least this didn't happen with kdelibs 4.

So, if the way how KDE is developed stays the same, e.g. that the build system 
maintainer cares more or less for the buildsystem of all of KDE SC, then this 
significantly increases (my) workload. I'll regularly will have to check 
multiple places.
IMO this is one of the things which make up KDE development: application 
developers can concentrate on the application, there will be e.g. translation 
maintainers who will care for the translations, buildsystem people will care 
for the buildsystem etc.
With the modularization the job of those "horizontal" people becomes harder, 
except if the "vertical" people start to care also for those things.

So keeping the stuff close together has also advantages.

> > If we distribute the macros, some will get the "ECM" prefix, some
> > "KCORE", some "KDOC", etc.
> > If we expect that the future KDE application developers are aware where
> > all the things come from exactly, then maybe it's no problem.
> 
> I would expect them to do what I did and copy working stuff from other
> files
> 
> :).
> 
> In saying that, there's nothing preventing us creating a 'Tier 4' umbrella
> package which depends on the libraries that used to make up 'kdelibs' and
> provides macros like:
> 
> macro(kde5_install_kcfg)
>   kcore_install_kcfg(${ARGN})
> endmacro()
> 
> For *application* developers, there is consistency.
> For *framework* developers and people using those frameworks there is
> modularity.
> 
> I'm just trying to stress the point that even if consistency is not
> introduced at the lowest levels where stuff is introduced, consistency can
> be re-introduced higher in the stack for 'KDE application developers'...

I thought about this too, but came to the conclusion that this is not a good 
idea.
IMO the KDE frameworks libraries must be basically of the highest quality, 
since they provide the base for everything building upon it.
Relying on copy'n paste in the buildsystem in those libraries in order to get 
them consistent would be a bad choice IMO.

> > Or should they all be aliased on a FindKDEFrameworks5.cmake file ? I
> > don't think I consider this a good solution.
> 
> ... Although I guess you don't consider that a good solution.
> 
> Why is that? You don't like the difference between KDE frameworks and KDE
> application consistency?

I don't like the idea of aliasing everything.

> > I'd like to come to a conclusion with this still in January.
> 
> I'm sure we can call that a decision deadline.
> 
> We can gather options, and if there's no concensus by the end of January,
> you pick one of them (someone must simply decide eventually) and we stick
> with it and not complain :).
> 
> > I think we agree that many/most of the various macros could go into the
> > respective library packages.
> 
> Ok, lets consider this agreed.

Mostly agreed.
It makes sense, but it also has its downsides (see above).
 
> > What is left are all the settings we provide, install dirs, options,
> > compiler flags and convenience/consistency macros like
> > kde4_add_unit_test(). I think they should go together into one place.
> 
> Options as I see them are:
> 
> 1) ECM provides them.
> Advantages:
> * KF5 Frameworks can mostly (entirely?) be expected to depend on ecm anyway
> * It provides the single dependency point
> * To some extent the stuff (sensible compiler flags, Qt flags) is also
> independent of KDE, but KDE has determined them to be a good thing.
> * 3rd parties can still use ECM without using the KDE stuff.
> Disadvantages:
> * It would mean ecm contains 'KDE stuff' which may be icky or unwanted.
> 
> 2) Provide them in a separate package and make all frameworks depend on
> them Advantages:
> * ECM is KDE-free.
> * Package would have KDE in the name

* the package which provides very basic KDE stuff is created and released by 
KDE for exactly that purpose

> Disadvantages:
> * It would be odd for all frameworks to have to depend on two 'build system
> enhancement' packages.

Yes.

> ** 3rd parties wishing to use a single KDE framework would need both.
> * Back to the situation of a single point of dependency convergence
> ** Temptation to put more 'global stuff' in there (dumping ground) would be
> high.

As written above, the "global stuff" has also advantages.

> 3) Put them in an 'umbrella' package on top of KF5
> Advantages:
> * Such an umbrella package will likely have to exist anyway to maintain the
> 'consistency' for application developers moving from KDE4 to 'KDE5'.
> * KDE frameworks are easier to use for non-KDE developers (fewer
> dependencies, using 'standard' add_executable instead of 'alien'
> kde4_add_executable).
> Disadvantages:
> * The frameworks themselves can't use the KDE buildsystem convenience
> stuff.
> * The frameworks might end up inconsistent with each other for
> example in what variables they choose to do the RPATH handling.

Good summary. As you can see, I would paint 2) a bit more positive, but beside 
that, I agree.

Now, my opinion: option 3) is no viable option, it would result in a lot 
copy'n paste.
Have a look at the current FindKDE4Internal.cmake. It's 1400 lines, consisting 
of quite a few empty lines, around maybe 400 or 500 lines of comments, which 
leaves let's say 500 lines of cmake code.
These lines contain no macros.
It's all just settings. Install dirs, compiler flags for the different 
compilers on the different operating systems, etc.

Option 3) means to copy these lines in every frameworks library.

(and this doesn't even consider the macros in KDE4Macros.cmake yet, which I 
did not write for the fun of it, but because people asked for the features).


Alex