isolating builds within a module

Thu Nov 13 01:14:30 GMT 2008

On Wednesday 12 November 2008, Alexander Neundorf wrote:
> On Thursday 13 November 2008, Aaron J. Seigo wrote:
> > On Wednesday 12 November 2008, Alexander Neundorf wrote:
>
> ...
>
> > > The time is not "wasted"...
> >
> > it absolutely is wasted. i spent 90+% of my build time yesterday waiting
> > for checks on non-libplasma. not one of those checks resulted in any
> > building of any other code that was being checked ..... because they
> > weren't changed!
>
> How should the buildsystem know without checking that ?
> The time is spent to ensure your build is correct.
>
> ...
>
> > unless those checks could be made an order of magnitude faster (in which
> > case it wouldn't mater),
>
> The algorithm was changed from O(n^2) to O(n) [n being the number of source
> files] IIRC last year, so this is in cmake 2.6.x.
> So, it checks what it has to check, I would suspect it cannot be made
> faster by magnitudes (at least the algorithm should be more or less optimal
> already).
>
> I measured it here on my system (quite old already), I guess on a recent
> system it's at most 50% of that:
> "make plasma" without any changes: 11 s
> "make plasma" with modified runnermanager.cpp: 19 s
> "make plasma/fast" without any changes: 0.5 s
> "make plasma/fast" with modified runnermanager.cpp: 7 s

that doesn't reflect my experience at all.. let's see:

aseigo at independence:~/kde4/build/KDE/kdelibs/plasma> time make -j4
[  3%] Built target threadweaver                                  
[ 16%] Built target solid                                         
[ 35%] Built target kdecore                                       
[ 64%] Built target kdeui                                         
[ 83%] Built target kio                                           
[ 87%] Built target knewstuff2                                    
[ 90%] Built target kfile                                         
[100%] Built target plasma                                        

real    0m8.894s
user    0m8.329s
sys     0m1.392s

aseigo at independence:~/kde4/build/KDE/kdelibs/plasma> time make plasma/fast -j4

real    0m0.445s
user    0m0.392s
sys     0m0.032s

that gives a sense of the overhead. but that's probably not realistic. let's 
modify a file:

aseigo at independence:~/kde4/build/KDE/kdelibs/plasma> time make -j4
[  3%] Built target threadweaver
[ 16%] Built target solid
[ 35%] Built target kdecore
[ 64%] Built target kdeui
[ 83%] Built target kio
[ 87%] Built target knewstuff2
[ 90%] Built target kfile
Scanning dependencies of target plasma
[ 90%] Building CXX object plasma/CMakeFiles/plasma.dir/framesvg.o
Linking CXX shared library ../lib/libplasma.so
[100%] Built target plasma

real    0m12.325s
user    0m10.941s
sys     0m2.116s

aseigo at independence:~/kde4/build/KDE/kdelibs/plasma> time make plasma/fast -j4
Building CXX object plasma/CMakeFiles/plasma.dir/framesvg.o
Linking CXX shared library ../lib/libplasma.so

real    0m2.488s
user    0m2.376s
sys     0m0.332s

as you can see when something is modified in libplasma, i save 10 whole seconds 
at build. that changes the dynamic from code, save, build, test as a smooth 
continuum to code, save, build, waitwait, test ... doing changes in libplasma, 
i will often build 10-30 times for significant changes, and not being able to 
see the effects in quick turnarounds is really annoying.

in 10 rounds through that cycles, it's less than half a minute of build versus 
two minutes. my coding time in that same period will be, say, 10 minutes. yes, 
that's an aggressive code/build/test cycle, but it's not uncommon, esp with 
the sorts of things plasma does =) but it means that my build time drops from 
20% of my time spent working on libplasma to just 5%. even 5% sucks in i-want-
a-pony-too terms, but i can deal with 5%. 20% is a non-starter.

thank goodness for make plasma/fast .. i just wish i knew about it a week ago 
=)

if you're wondering why my laptop isn't better than your old system, it's 
because disk IO on laptops still lives in the dark ages.

> We might check whether automoc4 introduces some delay which could be
> minimized.
> Ok, done that:
> "make plasma" without any changes and without automoc4: 8 s

woo! 27% chopped off the top... nice; down to ~15% of my dev cycle

> Maybe libplasma links to something it doesn't need ?

no. libplasma truly leverages the full synergy of the kde infrastructure.

.... web 2.0. *twitch* XML!

.... * marble

;-P

> How about making it possible to build plasma as a separate project ?

that's a possibility ... though more of a workaround than anything.

> > is there any way to tell cmake "always build this
> > component with /fast?" when i run cmake for the first time? then it
> > becomes
>
> You mean like a cmake switch to make the fast targets the default ones ?

yes.

> This doesn't exist currently.
> I'm also not sure the cmake developers would accept a patch, since this
> would mean some people who didn't remember that they are building without
> full dependency checking because they disabled it once will end up with
> broken builds from time to time, and this is something the cmake developers
> usually want to avoid. They basically always prefer reliably correct to
> fast.

that's completely understandable. perhaps look at it like a "hidden option"; 
heck, i didn't know about the /fast trick. i doubt many others would find out 
about a -DCMAKE_NO_DEP_CHECKS_BETWEEN_MODULES_AT_BUILD switch, or be so scared 
by it's length, scary soundingness and vagueness that only people who knew 
what it did would use it. ;)

-- 
Aaron J. Seigo
humru othro a kohnu se
GPG Fingerprint: 8B8B 2209 0C6F 7C47 B1EA  EE75 D6B7 2EB1 A7F1 DB43

KDE core developer sponsored by Qt Software

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-core-devel/attachments/20081112/c0bd5051/attachment.sig>