automoc v2

[Alexander Neundorf]

Hi,

On Friday 13 July 2007 16:52, Matthias Kretz wrote:
> On Friday 13 July 2007, Alexander Neundorf wrote:
> > On Thursday 12 July 2007 05:18, Matthias Kretz wrote:
> > > There are two downsides of using a custom target:
> > > a) a lot of I/O at cmake time
> > > b) the automoc target is not remade on /fast
> > >
> > > The current setup still has one dependency problem though: make doesn't
> > > know that the .moc includes in the .cpp files are generated by creating
> > > the _automoc.cpp file and therefore does not wait for kde4automoc to
> > > finish
> >
> > Does this mean parallel builds are broken ? This would be a serious
> > problem.
>
> I've tried hard to see whether the problem exists for normal builds but
> have not been able to break it. It seems to break only for /fast builds.

AFAIK currently cmake generates all generated files before building the rest 
of the target (but I'm not completely sure about this).
So if xxx_automoc.cpp is generated before the rest of the target is built it 
should work. But the problem you mention above seems to say this is not the 
case. 
When cmake runs, it creates the dependencies it know about. It doesn't 
currently know that the cpp file depends on the moc i.e. the automoc.cpp file 
(not sure it did know before the change).
Then when make runs, cmake scans the dependencies of the C/C++ files, and adds 
them if it can find the included file. So you may get different results on 
the first run (the .moc file doesn't exist yet so the dependency isn't added) 
and a later run (where a .moc file already exists so that the dependency is 
recognized). So a parallel build from a fresh build tree should show if there 
are problems.

I think the correct solution is to use a custom target.

...
> > > I still think adding headers to the list of sources of a target is
> > > awkward and not at all self-explanatory. Adding headers to a macro that
> > > has moc in the name says so much more.
> >
> > You could also see it this way, that the KDE4_ADD_EXECUTABLE/LIBRARY etc.
> > macros just handle moc completely automatically, you just have to list
> > all the files you care about.
>
> Right, you don't only list all the implementation files but all files that
> belong to the target...
> Ok, I'm fine with removing KDE4_MOC_HEADERS then.

Cool :-)

> > How about that: we get rid of automoc completely, i.e. no need anymore to
> > put the "include foo.moc" in any file, but we just scan all interesting
> > files for "Q_OBJECT" and if found, moc them.
> > Ah, well, this would increase build time, since the moc files would be
> > compiled separately. I guess this wasn't a good idea.
>
> That's what kde4automoc already does. You can have e.g. two source files
> that include the .moc and those two sources are passed to

There have been no loud "oh my god, it recompiles all my (not-automoced) moc 
files again if I change only one of them" cries yet ?
Probably because most in KDE is (currently) automoced.

> kde4_add_executable. Compiles. Now you remove the .moc includes and
> recompile the target (even with /fast if you want to) and kde4automoc will
> #include the two moc files into <targetname>_automoc.cpp. This will get
> compiled and linked into the target. Since all not-included mocs are
> compiled in one go you don't lose much.
>
> > If the tool kde4automoc would be written without Qt, e.g. STL only, then
> > this could theoretically also be used to build Qt itself...
>
> I don't think you'll be able to easily substitute the STL for QtCore in
> kde4automoc.

There is some more stuff in kwsys:
http://www.cmake.org/cgi-bin/viewcvs.cgi/Source/kwsys/?root=CMake
no threads, but at least processes. This abstracts some cross platform issues, 
should be self-contained and is BSD-licensed.
(this is not a "we should do this now", it's just a "we should keep it in 
mind").

> > > > Third point: maybe we can write a cmake command plugin which does a
> > > > lot of the handling for all kde4 targets (i.e.
> > > > kde4_check_executable_params(), kde4_handle_automoc(),
> > > > kde4_create_final_files() ), so a lot of the string-vector
> > > > conversions would fall away.
> > >
> > > That might be interesting IMO.
> >
> > I've never done that and I'm not really sure what is required to build
> > such a command. I'll give it a try when I find time.
> >
> > > We should just provide one file that documents where to find a given
> > > macro...
> >
> > Huh ?
> > All macros starting with KDE4_ are in KDE4Macros.cmake, and all macros
> > being in KDE4Macros.cmake should start with KDE4.
> > This kind of namespacing should be used for all cmake module files.
>
> Yes, I meant that we need it when we start implementing macros as C++
> plugins to cmake.

Ah, ok.
IMO implementing something as a loadable command should be the last resort, 
since it adds complexity.
The plugin API is in C. The plugin will be loaded into the running cmake. This 
cmake is not necessarily built with the same compiler which is now used for 
building KDE (e.g. if you install the binary package for Linux or especially 
if you install the binary for Windows and then use mingw). I don't know if a 
MSVC compiled cmake could actually use a mingw compiled plugin. Probably only 
if it's using only C.

And I would say if we do it we should do only one command, not everything we 
could do.

But since this is "just" about speeding the cmake run up, it is probably 
better to either optimize our cmake files or optmize cmake. One thing I 
suspect to eat a lot of time is the conversion from string -> vector<string>. 
In cmake variables are represented as map<string, string>, which maps the 
name to the value. Whenever the value is needed as a list of strings, the 
string is converted. E.g. in kde4_add_executable I think this happens 8 times 
for the list of source files (the call to check_executable_params, 4 times in 
check_executable_params (once for each list() call), once in the loop in 
handle_automoc, once when calling  create_final_files(), once when calling 
add_executable(). If the vector<string> would be cached this might speed 
things up. But I should valgrind cmake again before doing something on this.

Alex