bksys / scons (Re: win32 port)

[Holger Schröder]

Hi,

On Monday 09 January 2006 18:33, Jaison Lee wrote:
> I hate to add fuel to a conversation that has no doubt gone back and
> forth already, but what *was* the primary reason that scons was
> chosen?  As a newcomer looking things over I have been baffled at how
> bksys is using scons. Most everyone here seems to think that the main
> scons tree has intolerable problems, and there are so many layers
> (especially once the "SConfigure" system gets made) that it barely
> resembles what a normal scons build system looks like.
>
> Don't get me wrong; I use scons at work and I really like it, but if
> it isn't capable of handling KDE then why are we trying to make it do
> so? Is it really worth keeping a KDE specific build system (which is
> what we are moving towards)?
>
> scons/bksys has been in development for months and yet it seems to be
> months away at best from even being serviceable. Perhaps it's time to
> cut our losses and run.
>

i think we have to look a little bit deeper at the current process, in which 
we are with scons/bksys:

- what do we want ? we do not only want to have a replacement for unsermake on 
unix, we want kde to run on linux, unix and mac os. and we want that with 
only one well-designed build system. this is quite a big task.

basically we want to "talk" to the build system on a higher level: when a 
program links against e.g. kparts, it should be built after (under linux) 
libkparts.so. no more hacks with COMPILE_FIRST = xxx etc.

we want the build system to be "single-pass", that means: when i add a source 
file to any SConscript file, a call to scons should do everything that is 
necessary: rebuild the target, when the target is a library, relink the 
programs, that link against it, if needed. no more create_makefile, and no 
more strange errors, because people forget to call it.

we want the syntax of the normal Makefile.am replacements (now SConscript 
files) to be platform-independent and simple. for example (from 
kdelibs/kabc/Makefile.am) :
 libkabc_la_LIBADD = vcardparser/libvcards.la $(LIB_KIO) \
                    $(top_builddir)/kresources/libkresources.la

in these two lines the beginning "lib"-prefix is redundant, the 
"$(top_builddir)" is redundant, and the ".la"/"_la_" is redundant, and i am 
not sure, if these lines tell unsermake, that libkresources.la should be 
built before libkabc.

so if we want to have a build system for kde 4, that is as simple (and simple 
to explain to new developers), as it can be, a lot of things have to be 
managed behind the scenes. in this example the build system should know, that 
the lib is of course in the build dir, and not in the source dir, it should 
know, that the prefix for a library is "lib" and the suffix is ".la", and it 
should work, when it is ".dll" under windows too.

we have to create replacements for any bash-snippets in Makefile.am´s, when it 
should become platform-independent. it is for example used to copy some files 
to a installation dir, to do globbing for files, etc. this will never work 
under windows, and if it would, it would be a maintenance nightmare. ( think 
about path separators, etc. ).

we have quite some advanced technology in kde, and we have to check how it can 
be made usable under non-linux platforms. how can a kpart be loaded under 
windows? can konsole be used to run cmd.exe in it? how can i compile/debug a 
program in kdevelop under winows? how can this all be done with only one 
platform-independent build system?


so the whole thing is quite complex, and we have to think about a lot of 
things. and i think we are quite successful at implementing a new build 
system when working on a moving target (kdelibs trunk). from the list above 
quite some things, that do already work under scons would have to be 
reimplemented, when we would want to try cmake to build kde under windows/mac 
os, for example detecting libraries under windows, or replacing embedded 
bash-scripts by platform-independent python scripts.

when i look at the scons/bksys system, as it is now, i see some areas, which 
should be worked on, of course. these are:

performance: on my 2 ghz pentium 4M notebook, a unsermake run in kdelibs lasts 
about 2 seconds, when there is nothing to do (only check that everything is 
up-to-date). when i do this in scons, it takes me more than a minute. with 
this round trip-time, it is quite slow to hack on the build system. but the 
scons time already "includes" the "make -f Makefile.cvs"-equivalent. and when 
ita´s experiments are really faster than unsermake (i.e. 2 secs for a 
complete scons up-to-date-check in kdelibs) i would be quite happy with this.

quite some SConscript files include python snippets to do some things. i would 
like if these snippets would be converted to "scons tools", snd the normal 
SConscript files would only be descriptions of things, that should be done:
the program with name A should be build from a.cpp and b.cpp instead of "gcc 
-o A a.cpp b.cpp". this for example for create_hash_table in khtml/ecma.

so i think we are already quite far with scons/bksys:
we can build some subset of kdelibs, without need for any changes in the 
source files. we do not interfere with the unsermake system, so we do not 
delay anybody, who wants to hack on kde and who does not want to fiddle 
around with the build system. the katetest program can be executed when built 
with scons, and you see the kate window, with a black background, the menu 
bar is missing the menu entries, the toolbars are missing, etc. but you can 
already enter text. that means loading of the kate part seems to work.

i have no deep knowledge about the configuration system of scons, but when the 
above-mentioned performance problems can be fixed, i don´t see any reason to 
switch away from bksys, as we are already 80% finished (yes, i know the 80/20 
rule ;-) ). i have not yet run into any problem when hacking on bksys, that 
is not solvable. the bksys sources are quite readable, but the scons sources 
are not as understandable.


i think we can bring bksys into a shape, which is not only usable for kdelibs 
and other kde or qt programs, when we attack the problem a little bit more 
coordinated. i am mostly hacking on it in the following way: do a svn up, 
compile with scons, try to understand things, try to fix problems. but when 
somebody wants to refactor bksys, i would be quite happy with it. but then we 
should add tests to bksys, so that we can prove, that refactoring did not 
break things.

-- 
regards,

Holger Schröder