Lock contention analysis for 4 thread background parsing with kdevelop

[Milian Wolff]

On Thursday 03 December 2009 12:49:01 Hamish Rodda wrote:
> Hi,
> 
> I've just stumbled on what seems to be a great new tool for lock contention
> analysis: mutrace by Lennart Poettering, at
> http://0pointer.de/blog/projects/mutrace.html
> 
> After some initial problems building on kubuntu, I managed to get it
>  compiled. Kubuntu/ubuntu/debian users will probably have to do like I did,
>  so that libbdb is built with -fPIC: install binutils-dev and
>  binutils-source, extract the source, ./configure  --prefix=/usr
>  --with-pic, cd bdb, make && make install.

Interesting, I'll try to get it working with Archlinux over the weekend I 
think.

> Running kdevelop was easy once I removed the patchreview plugin, since that
> plugin throws exceptions (which are not currently compatible with mutrace).

You should try out duchainify 
($builddir/kdevplatform/utils/duchainify/duchainify)

> I haven't yet had time to really analyse each of the locks that is
>  represented here, but it certainly shows that we do massive amounts of
>  locking, and that there are very many instances of lock contention. 
>  Hopefully through this tool we can find some hot spots to optimise, such
>  that running with more than one background parsing thread will one day
>  make sense and be faster than a single thread.

Is that really true? I.e. that we are currently slower with more threads than 
with a single thread?

> You'll need to read the description at the program homepage to understand
>  the output. Here is the best output I could generate so far (full parse of
>  kdevplatform/kdevelop/java by kdevelop with clean .kdevduchain):
>
> hamish at Sleek:/opt/kde4/src/kdevplatform/plugins/patchreview$ mutrace
>  --hash- size=100000 --max=30 kdevelop
> mutrace: Application appears to be compiled without -rdynamic. It might be
>  a mutrace: good idea to recompile with -rdynamic enabled since this
>  produces more
> mutrace: useful stack traces.

How would one compile with -rdynamic? There is a cmake flag for custom linker 
options, right?
 
...
> mutrace: Showing 30 most contended mutexes:
> 
>  Mutex #   Locked  Changed    Cont. tot.Time[ms] avg.Time[ms] max.Time[ms]
> Flags
>     7702 10867922  1289097   602227     2045.941        0.000        3.661
> M-.--.
>     4543  1438294   493776   220689      507.489        0.000       21.206
...

What is imo interesting: avg Time of 0 ms. Imo the sheer number of lockings is 
what makes things slow, but imo it's not really easy/possible to reduce that 
number?

> mutrace: WARNING: 384 internal hash collisions detected. Results might not
>  be as reliable as they could be.
> mutrace:          Try to increase --hash-size=, which is currently at
>  100000.

How serious is that warning, anyone knows?

-- 
Milian Wolff
mail at milianw.de
http://milianw.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kdevelop-devel/attachments/20091203/2b0f24e9/attachment.sig>