Thread-safety issue in cmake support ?

[Milian Wolff]

David Nolden, 12.06.2010:
> The fact that there is a lot of lock contention during parsing is of
> course true, and it's mainly because there is a lot of central data
> repositories that are used all the time during the parsing. Millions
> of locks a second are acquired there. Also the design of that stuff is
> completely off-topic in this discussion.
> 
> If we design the parsing process right, we will need a few foreground
> locks for each file of the project. One before parsing where we ask
> for include-paths, defines, etc., and translate changed ranges, and
> one more to highlight the parse result after parsing, and that should
> be basically it. This is a completely different magnitude than the
> lock-contention which slows down the parsing.
> 
> With the current design of smart ranges it won't work of course,
> because whenever accessing smart ranges the lock would be required.
> Smart-ranges should be removed from the duchain guts, and the new
> sequence should basically be:
> 
> {
>  Lock foreground
>  Get document text
>  Get include paths etc.
> 
>  if(document is open in editor)
>    Translate duchain ranges using MovingRangeInterface
> }
> 
> Parse+update duchain
> 
> if(document is open in editor)
> {
>  Prepare declaration highlighting
>  Lock foreground
>  Highlight declarations using MovingRange
> }
> 
> This is basically equivalent to moving everything behind "Lock
> foreground" right into the foreground thread, with the only advantage
> that we can keep the code together.
> 
> The hard work, which is "Parse+update duchain" stays in the
> background, and that's also the only part within which the whole
> multi-threading thing makes sense at all, so that's fine, and at least
> in this design there's no additional lock contention to worry about.

Ah ok, we all misunderstood you. You actually propsose something very similar 
to what we said, with the only difference to use the foreground lock instead of 
events / ctor + dtor for doing the foreground stuff.

Actually I think we could still use the ctor + dtor of a parsejob for the 
foreground stuff, or do it in the background parser or similar, since that code 
is (imo) very generic and should be done for each parsejob. And since only the 
::run() method is done in the background, doing that stuff somewhere else would 
be fine, no?

> >> It wouldn't even be desirable to eliminate all instances of the lock, as
> >> the lock would achieve the same as a message-passing interface between
> >> the foreground and background which would be the alternative, just with
> >> much less code.
> > 
> > But with lot more lock contention. The GUI will hold the lock often, for
> > each event, ....
> 
> It's the same. If you send a message to the foreground, you also have
> to wait until the foreground is idle and processes the event, and at
> the same point you'd acquire the lock.
> 
> > If the accesses are that rare, it would be not much work and code to port
> > them to work on local stored stuff, or?
> 
> Anyway just take a look at the design depicted above. I anyway don't
> see much that you can copy and store locally.

well, what you show up there is just what we proposed, only slightly different. 
You copy the editor contents in the ::run() method, while we'd do it before 
its done, i.e. directly in the foreground thread.

> > Else, you contradict yourself, and they are not that seldom at all and
> > you will get your performance penalty from the locks.
> 
> If you wrap each access to a SmartRange with a foreground lock then
> yes, you may get additional lock contention, although not much more
> than with the smart-lock that we're using now (usually the foreground
> is most busy with the editor, which in turn holds the smart-lock).
> Actually doing that wrapping might be a viable first step to get rid
> of all the multi-threading smart-range crashes.
> 
> Anyway I think the _first_ thing we need to worry about is stability,
> as that's one of the main problems now. The second most important
> thing is "fluidity", eg. that the IDE is well-usable even while the
> background-parser is running. Only then comes "speed" of the parsing.
> Who cares whether the whole project is processed in 4 or 8 minutes, if
> you can work without issues while parsing? So the whole talking about
> lock contention feels kind of useless atm.
> 
> Greetings, David


-- 
Milian Wolff
mail at milianw.de
http://milianw.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kdevelop-devel/attachments/20100612/efc15fec/attachment.sig>