KDE-buildsystem-basics package ?

[Stephen Kelly]

Alexander Neundorf wrote:
> On Saturday 21 January 2012, Stephen Kelly wrote:
>> Alexander Neundorf wrote:
>> > Beside that, none of the stuff in
>> > FindKDE4Internal.cmake/KDE4Macros.cmake/KDE4Defaults.cmake is actually
>> > _necessary_ to build software.
>> > It "only" does two things:
>> > * it makes developing for "KDE" more convenient, e.g. it presets RPATH
>> > settings, so developers have to care less
>> > * it helps with consistency among stuff which uses them, like e.g. the
>> > install directories
>> 
>> In that case I wonder if some of this stuff wouldn't make sense as
>> something which belongs 'on top of' KDE frameworks, so that KDE
>> application developers get the convenience, but that convenience is not
>> entirely there (or necessary, maybe) for frameworks. Or maybe this stuff
>> belongs 'somewhere' in KDE frameworks, but not necessarily in 'tier 0'
>> which everything else depends on.
>> 
>> Could it even be optional and not used if not found?
> 
> I thought about that too. My result: not really.
> E.g.
> install(FILES foo.h DESTINATION ${INCLUDE_INSTALL_DIR} )

Indeed variables can't really be optional. But something like 
kde4_add_executable which does 'kde magic which is only needed by kde' could 
be conditionally used.

I dislike kde4_add_executable anyway for various reasons.

Looking at what it actually does:

* some handling for when it is a auto or manual test
* set properties for gui executables on win and mac.
* automoc stuff (obsolete)
* create_final stuff (obsolete)
* uac manifest stuff (how is that KDE specific?)
* rpath stuff.

For the test stuff this might be possible:

# begin ECMMarkTest.cmake

if (NOT CMAKE_TESTING)
    get_property(GLOBAL _buildtestsAdded BUILDTESTS_ADDED)
    if(NOT _buildtestsAdded)
        add_custom_target(buildtests)
        set_property(GLOBAL PROPERTIES BUILDTESTS_ADDED TRUE)
    endif(NOT _buildtestsAdded)
    add_custom_target(buildtests)
endif()

macro(ecm_mark_test _target)
  set_target_properties(${_target} 
    PROPERTIES 
    COMPILE_FLAGS -DTEST_SOURCE_DIR="\\"${CMAKE_CURRENT_SOURCE_DIR}/\\""
  )

  if (NOT CMAKE_TESTING)
    set_target_properties(${_target} 
      PROPERTIES 
      EXCLUDE_FROM_ALL 1
    )
    add_dependencies(buildtests ${_target})
  endif ()
endmacro()

# end ECMMarkTest.cmake

Then it would be used like this:

add_executable(foo ...)
ecm_mark_test(foo) # Explicitly mark foo as a test.

Which I prefer to 

bar_add_executable(foo TEST ...)

because it doesn't pollute the API for adding executables with something 
orthogonal (testing related stuff).

Setting the WIN32 and MACOS_BUNDLE parameters is also not needed in the 
add_executable call, but can also be done with PROPERTIES. If we can add to 
CMake something like:

set(CMAKE_GUI_EXECUTABLES 1) # CMake sets the platform specific gui 
                             # properties on executable targets

Then we can do something like:

macro(mark_non_gui_executable target_)
  # TODO: Do platform check for mac/windows.
  set_target_property(${_target} 
    PROPERTIES 
    WIN32_EXECUTABLE 0 
  )
endmacro()

and use it in the (very rare, except in unit tests) cases where a kde non-
gui executable is needed. 

I don't know about the manifest or rpath stuff, but I still can't see what 
makes them kde specific.

> 
> INCLUDE_INSTALL_DIR must be set somewhere, and making it optional whether
> it comes from some package or defining it in the project itself will make
> for really ugly and long, i.e. hard to maintain, CMakeLists.txt in all the
> framework libraries.

I think INCLUDE_INSTALL_DIR is not a great example. It's very odd for it to 
be anything other than 'include' right? Can't it be set in ECM? 

>> Is KF5 a good time to drop the old policies and start fresh? And accept
>> the policies defined by:
>> 
>> cmake_minimum_required(VERSION 2.8.8)
> 
> We have the chance now to break compatiblity, so for most things, yes.
> Probably not for all.
> The problem is the following: we define a minimum cmake version required
> to build KDE. Developer A uses this version.
> Then a new version of cmake is released, and introduces a policy, which
> changes some behaviour.
> Developer A updates to this version, and suddenly sees warnings, not
> errors, due to that new policy.
> He fixes the warnings, and now the new cmake version is quiet.
> But do the modified CMakeLists.txt still work properly also with the
> minimum required version of cmake, or did he just break the build for
> everybody else ?

How is this different to the version of Qt depended on?

Even if that developer does break stuff for everyone else, it will become 
quickly apparent, just as it is for the Qt dependency or any other 
dependency. I don't think the CMake dependency has to be special cased (at 
least in frameworks - if the 'umbrella' location adds consistency for 'KDE 
applications', that's a good place to put something like this too)

> 
> This is uncertain.
> Nevertheless we should go through the list of policies which are set and
> check whether we still want them this way.

Yes.

> 
> E.g. setting CMP0005 to NEW may very well break the build in some places,
> or keep it building but produce not-correctly-working results.
> 

I would set it to new (by not setting it at all and using a sufficiently 
high cmake minimum version) and call any breakage the result of code that 
needs to be fixed.

No one will expect to port to KDE5 and then release with 0 testing. If the 
policy causes a significant breakage it will be found immediately during 
porting or quickly on first run. Otherwise it's not significant breakage and 
it's 'just a bug' in the CMake code which hasn't been maintained since the 
policy was introduced.

If it does break something the fix is simple. Either set the policy more 
locally to the breakage, or port to unbroken use.

>> > Well, it sets the cmake RPATH related settings.
>> > The default is empty install RPATH. Everything else is
>> > project-specific, as our settings are KDE specific.
>> 
>> It's not enough to do this?:
>> 
>> set( CMAKE_INSTALL_NAME_DIR ${CMAKE_INSTALL_PREFIX}/${LIB_INSTALL_DIR})
>> set( CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
>  
> Almost.
> Even if it would be completely enough, we can rely on that the average
> developer will not set them correctly always from the start. He may set
> them correctly once his package has been released, and people started to
> report problems.

A good reason to set those variables in ecm imo.
<snip>

>> >> >> , kde4_add_unit_test, kde4_add_executable
>> >> 
>> >> I don't know. I'd be interested to see what they look like after
>> >> removing the automoc stuff and seeing what's left and if refactoring
>> >> is in order.
>> > 
>> > kde4_add_unit_test() does not depend on anything, it is mostly just
>> > convenience.
>> 
>> Is the convenience that
>> 
>> add_executable(testname ${testname_SRCS})
>> add_test(testname testname)
>> 
>> is reduced to one line? Is that something for ecm?
> 
> It does a lot more. Have a look at it, it's in KDE4Macros.cmake. And it is
> very KDE specific.

What it does is:

* Wrap the testname argument to cmake command add_test()
* The NOGUI stuff because it wraps kde4_add_executable
* Set the output format of running Qt tests (should be in ecm or Qt?)
* Determine whether QtTestlib is used (I have never seen it not used - 
    probably a case that should be handled in a separate dedicated macro) so 
    that xml output from tests can be set.
* Add the buildtests target you mentioned before (does it really need to be 
    part of this macro?)

> What I wrote before about KDE4_BUILD_TESTS was actually wrong.
> The test executable targets are always part of the build, but if
> KDE4_BUILD_TESTS is not set, they are EXCLUDE_FROM_ALL, and a target
> "buildtests" is created, which all such test executables depend on.
> This was discussed on the buildsystem list June 2007, mainly with apaku,
> David, Thiago and me.

Yes, but I think there are solutions available which include de-coupling (As 
I wrote above with ecm_mark_test). We just need to think about the problems 
and solutions again. I doubt any of you were thinking about de-coupling in 
2007, but it is one of the goals today.

> Additionally, kde4_add_executable() 

I think you mean KDE4_ADD_UNIT_TEST ?

> creates wrapper shell/batch scripts,
> which set PATH/LD_LIBRARY_PATH/DYLD_LIBRARY_PATH so that the executables
> can also be executed uninstalled if RPATH is disabled or under Windows.

Aren't there other ways of executing the tests uninstalled without scripts? 
I can run Grantlee tests uninstalled, and I don't have any wrapper scripts.

> 
>> >> I don't see why it would need to be in a directory. It can just be a
>> >> few files. Maybe even just one.
>> > 
>> > Because the idea of e-c-m is to make stuff, which can be used
>> > independent from KDE (before this meant "no kdelibs", what does it mean
>> > then ?) available for non-KDE software.
>> 
>> It might be odd to consider that 'everything except KDE is in scope for
>> ECM'. That is, we can add stuff for helping to build Qt or boost or any
> 
> CMake or e-c-m is not providing any required stuff for building Qt or any
> other project.
> It contains stuff which help using Qt or boost. But it does not contain
> stuff decided upon by the Qt or boost communities, and then put into
> CMake.

Yes, that is a difference.

Still, all we'd have to do is define it to be Ok.

> 
>> number of other packages, but adding stuff for KDE conventions is
>> off-limits for ECM? I guess it could be understandable considering that
>> KDE is capable of providing its own stuff. (unless it's not, which is the
>> subject of this discussion).
>> 
>> > Putting the cmake files, which kind of make an application a KDE
>> > application, there, turns e-c-m into a KDE package.
>> 
>> I think it depends on what KDE stuff is put in there exactly. Does having
>> GNUInstallDirs in CMake make CMake a GNU package?
> 
> I think this is different.
> With GNUInstallDirs.cmake, cmake is just providing something which follows
> a "standard". If e.g. KDEInstallDirs.cmake would be part of CMake, CMake
> would ship the file which define the standard, but the standard would not
> be decided by the CMake developers, but by the KDE developers.
> 

Yes, but we could define it as something that's ok in ecm.

>> then ECM seems like a reasonable place to enable that to some extent.
>> 
>> > Another issue with putting all small things into different places, e.g.
>> > some macros into e-c-m, some into kcore, some into kdocumentation,
>> > etc., is that the macros will all change their name.
>> > Right now they all have the "KDE4_" prefix, because they come from
>> > FindKDE4Internal.cmake. And only stuff from that file uses this prefix.
>> 
>> Either a script does
>> 
>> s/kde4_install_kcfg/kf5_install_kcfg/ig
>> 
>> or
>> 
>> s/kde4_install_kcfg/kcore_install_kcfg/ig
>> 
>> (or whatever macros)
>> 
>> It doesn't really matter to the script, but in kf5, the KDE4_ prefix
>> doesn't make much sense.
> 
> Yes, keeping the KDE4_ prefix wouldn't make sense.
> But using one common prefix ("KDE5", "KF5", "KDE", ...) would be easier to
> remember.

Yes, for KDE application developers (which is why I'd propose a wrapper 
macro in cases where a framework provides its own macros with a its own 
prefix, if that is to happen). For framework developers, they would need to 
keep in mind that if they use a kcore macro it will have a kcore prefix. In 
that sense it is consistent (the macro name has the name of the framework it 
comes from built in).

> 
> Now I actually figured out why I have a slightly bad feeling with
> distributing the macros to the packages to which they belong:
> maintainance. Right now in kdelibs, they are all in one place.
> FindKDE4Internal.cmake+KDE4Macros.cmake+KDE4Defaults.cmake.
> If I keep an eye on that, everything is mostly fine.
> If it will be split, the macros etc. will be distributed over many
> modules. I doubt that every KDE frameworks library maintainer will
> maintain the cmake macros properly. At least this didn't happen with
> kdelibs 4.

Right. I have no solution to this other than if something is broken, it will 
have a 'broken effect' and therefore become a visible break.

> So, if the way how KDE is developed stays the same, e.g. that the build
> system maintainer cares more or less for the buildsystem of all of KDE SC,
> then this significantly increases (my) workload. I'll regularly will have
> to check multiple places.

Potentially, yes. There are already multiple places which you could consider 
checking (kdelibs, kdebase, all other modules). This could potentially add 
more.

There are other people who check in certain places though and have the CMake 
expertise (eg krop taking care of kdepim), and there is the policy of 
significant changes to CMake stuff going through the buildsystem list.

There is never going to be 0% breakage anyway, but yes of course ways to 
reduce breakage are good. But we can consider that it does have a cost to 
keep everything in one place and whether it's worth it. We can also consider 
that someone who breaks something and then is told that they broke it won't 
break it that way again.

> IMO this is one of the things which make up KDE development: application
> developers can concentrate on the application, there will be e.g.
> translation maintainers who will care for the translations, buildsystem
> people will care for the buildsystem etc.

Right, but we need to consider if KDE framework development is different to 
KDE application development. KDE applications will all get consistency, no 
matter what - whether it comes from ECM or the 'umbrella' layer. So 
application developers can still focus on the application. I don't see that 
changing with a move to KF5. The 'umbrella layer' would be the aggregator 
that looks something like 'kdelibs' in the eyes of application developers.

I don't know what might change for translation developers and translators. 
I'd hope close to nothing would change.

For the buildsystem, it might change some stuff on the library level. 
Frameworks maintainers might have to be trusted to not break the CMake 
stuff.

> With the modularization the job of those "horizontal" people becomes
> harder, except if the "vertical" people start to care also for those
> things.
> 

Yes, for horizontal people it can become harder. The vertical people (at 
least on a frameworks level) might have to take some responsibility for the 
cmake macros in their module to ensure that they don't break them. 

How much of a real issue that actually is remains to be seen until we see 
how many different places the macros would actually end up.

> So keeping the stuff close together has also advantages.

Yes. But also costs. The two need to be weighed together.

> 
>> > If we distribute the macros, some will get the "ECM" prefix, some
>> > "KCORE", some "KDOC", etc.
>> > If we expect that the future KDE application developers are aware where
>> > all the things come from exactly, then maybe it's no problem.
>> 
>> I would expect them to do what I did and copy working stuff from other
>> files
>> 
>> :).
>> 
>> In saying that, there's nothing preventing us creating a 'Tier 4'
>> umbrella package which depends on the libraries that used to make up
>> 'kdelibs' and provides macros like:
>> 
>> macro(kde5_install_kcfg)
>>   kcore_install_kcfg(${ARGN})
>> endmacro()
>> 
>> For *application* developers, there is consistency.
>> For *framework* developers and people using those frameworks there is
>> modularity.
>> 
>> I'm just trying to stress the point that even if consistency is not
>> introduced at the lowest levels where stuff is introduced, consistency
>> can be re-introduced higher in the stack for 'KDE application
>> developers'...
> 
> I thought about this too, but came to the conclusion that this is not a
> good idea.
> IMO the KDE frameworks libraries must be basically of the highest quality,
> since they provide the base for everything building upon it.

Right. The way to maintain that high quality would be for the maintainers to 
know what they are doing or ask for help if they change cmake macros. 
kdelibs development isn't exactly high speed anyway.

> Relying on copy'n paste in the buildsystem in those libraries in order to
> get them consistent would be a bad choice IMO.

I'm not sure what would need to be copy+pasted?

>> > Or should they all be aliased on a FindKDEFrameworks5.cmake file ? I
>> > don't think I consider this a good solution.
>> 
>> ... Although I guess you don't consider that a good solution.
>> 
>> Why is that? You don't like the difference between KDE frameworks and KDE
>> application consistency?
> 
> I don't like the idea of aliasing everything.

Ok. And why not? Because aliasing creates a difference between frameworks 
and applications as I wrote above? Or something else?

>> > I'd like to come to a conclusion with this still in January.
>> 
>> I'm sure we can call that a decision deadline.
>> 
>> We can gather options, and if there's no concensus by the end of January,
>> you pick one of them (someone must simply decide eventually) and we stick
>> with it and not complain :).
>> 
>> > I think we agree that many/most of the various macros could go into the
>> > respective library packages.
>> 
>> Ok, lets consider this agreed.
> 
> Mostly agreed.
> It makes sense, but it also has its downsides (see above).
>  
>> > What is left are all the settings we provide, install dirs, options,
>> > compiler flags and convenience/consistency macros like
>> > kde4_add_unit_test(). I think they should go together into one place.
>> 
>> Options as I see them are:
>> 
>> 1) ECM provides them.
>> Advantages:
>> * KF5 Frameworks can mostly (entirely?) be expected to depend on ecm
>> anyway * It provides the single dependency point
>> * To some extent the stuff (sensible compiler flags, Qt flags) is also
>> independent of KDE, but KDE has determined them to be a good thing.
>> * 3rd parties can still use ECM without using the KDE stuff.
>> Disadvantages:
>> * It would mean ecm contains 'KDE stuff' which may be icky or unwanted.
>> 
>> 2) Provide them in a separate package and make all frameworks depend on
>> them Advantages:
>> * ECM is KDE-free.
>> * Package would have KDE in the name
> 
> * the package which provides very basic KDE stuff is created and released
> by KDE for exactly that purpose
> 
>> Disadvantages:
>> * It would be odd for all frameworks to have to depend on two 'build
>> system enhancement' packages.
> 
> Yes.
> 
>> ** 3rd parties wishing to use a single KDE framework would need both.
>> * Back to the situation of a single point of dependency convergence
>> ** Temptation to put more 'global stuff' in there (dumping ground) would
>> be high.
> 
> As written above, the "global stuff" has also advantages.
> 
>> 3) Put them in an 'umbrella' package on top of KF5
>> Advantages:
>> * Such an umbrella package will likely have to exist anyway to maintain
>> the 'consistency' for application developers moving from KDE4 to 'KDE5'.
>> * KDE frameworks are easier to use for non-KDE developers (fewer
>> dependencies, using 'standard' add_executable instead of 'alien'
>> kde4_add_executable).
>> Disadvantages:
>> * The frameworks themselves can't use the KDE buildsystem convenience
>> stuff.
>> * The frameworks might end up inconsistent with each other for
>> example in what variables they choose to do the RPATH handling.
> 
> Good summary. As you can see, I would paint 2) a bit more positive, but
> beside that, I agree.
> 
> Now, my opinion: option 3) is no viable option, it would result in a lot
> copy'n paste.

What would be copy+pasted?

I think some combination of 1 and 3 could make sense. Anything which is a 
candidate for copy+paste could make sense in ecm.

> Have a look at the current FindKDE4Internal.cmake. It's 1400 lines,
> consisting of quite a few empty lines, around maybe 400 or 500 lines of
> comments, which leaves let's say 500 lines of cmake code.

I took out everything that I thought could be somewhere else (kcfg stuff 
with kcfg, standard kde install locations to ecm, sensible compiler flags to 
ecm) or is obsolete (automoc, phonon stuff - obsoleted because phonon is a 
module).

The result (attached) is a file full of stuff which I don't know why it 
exists (either because it is windows stuff or the rpath stuff).

I would like to get very specific about what needs to be 
central/global/whatever so that we can ask specifically why.

> Option 3) means to copy these lines in every frameworks library.

Or to say 'these are the sensible flags to use when creating Qt libraries' 
and put it in ecm.

> 
> (and this doesn't even consider the macros in KDE4Macros.cmake yet, which
> I did not write for the fun of it, but because people asked for the
> features).

Yes, and some of which could potentially be moved to other locations in a 
modularized world, right?

> 
> 
> Alex
-------------- next part --------------
A non-text attachment was scrubbed...
Name: FindKDE4Internal.cmake
Type: text/x-cmake
Size: 3977 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kde-buildsystem/attachments/20120121/1c9da7de/attachment-0001.bin>