lock contention - what can we do about it?

[David Nolden]

I'm just saying that the <100% is due to I/O. There is no other possible
explanation anyway, unless if some thread would take a lock and start
sleeping while holding it.

The fact that we don't use all cores is due to lock contention of course.
Using SpinLock instead of QMutex in the whole duchain would have many
advantages:
a) It can be debugged
b) It is less buggy (100 LOC instead of a few thousand provenly buggy LOC..)
c) It is probably faster, at least when the protected region is very small,
as is usually the case in the duchain
d) It would allow us to analyze lock contention using valgrind, by disabling
all sleeping in the spinlock and in the duchain lock

Anyway, personally I don't care so much about the lock contention. What I
see though is, that all the statistics, analyzes and hypotheses that have
been done regarding this issue simply are not useful to do anything about
it. If you want to fix something, then you need to know: Which exact lock is
responsible for most of the contention, and from where is it called? This
could be answered by valgrind with spin-locks.

There would be even more useful statistics, like "Who is holding the lock
while we're waiting for it, and what is he doing?", but that would be pretty
hard to answer.

Am 06.03.2011 22:58 schrieb "Milian Wolff" <mail at milianw.de>:

David Nolden, 06.03.2011:

> When doing "./duchainify --threads 10 -f all-declarations-and-uses
> my-project --force-update" I ...
Sorry David, but I cannot follow you here. With 10 threads you should
ideally
up to ten cores busy, but well - on the ubiquotos dual core machines of
today
(which I assume you have as well) at least 200%. How your number "proofs"
that
it's IO wait I have no idea.

Furthermore please take my other emails into account, i.e. wa / iowait time
as
reported by top or vmstat, or general reads reported by iotop or or or...


> That mutrace output does not really look useful.
Yep, I wonder what we can do about it. I wonder whether it's not maybe that
my
Qt lib is missing debug symbols, I'll retry that one.


> I just got an idea
> how we can very simply fight the lock contention: Simply replace all
> uses ...
Really? I'd rather use Qt code than custom code. I'm against this change.


> For profiling, we can then
> simply disable the QWaitCondition fallback in SpinLock, which means
...
Ah, you mean only for profiling? It would still mean tons of ifdefs and
stuff
like that, nothing I want to pollute our codebase with. But one could think
about maintaining a patch(-set) for that and see what output it produces...

Still, I cannot believe that the tools (drd, mutrace, ... ?) are really not
sufficient for our case. We are not the only ones with such problems after
all.

bye


-- 
Milian Wolff
mail at milianw.de
http://milianw.de

--
KDevelop-devel mailing list
KDevelop-devel at kdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinfo/kdevelop-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kdevelop-devel/attachments/20110307/8294942a/attachment.html>