Thread-safety issue in cmake support ?

[David Nolden]

The fact that there is a lot of lock contention during parsing is of
course true, and it's mainly because there is a lot of central data
repositories that are used all the time during the parsing. Millions
of locks a second are acquired there. Also the design of that stuff is
completely off-topic in this discussion.

If we design the parsing process right, we will need a few foreground
locks for each file of the project. One before parsing where we ask
for include-paths, defines, etc., and translate changed ranges, and
one more to highlight the parse result after parsing, and that should
be basically it. This is a completely different magnitude than the
lock-contention which slows down the parsing.

With the current design of smart ranges it won't work of course,
because whenever accessing smart ranges the lock would be required.
Smart-ranges should be removed from the duchain guts, and the new
sequence should basically be:

{
 Lock foreground
 Get document text
 Get include paths etc.

 if(document is open in editor)
   Translate duchain ranges using MovingRangeInterface
}

Parse+update duchain

if(document is open in editor)
{
 Prepare declaration highlighting
 Lock foreground
 Highlight declarations using MovingRange
}

This is basically equivalent to moving everything behind "Lock
foreground" right into the foreground thread, with the only advantage
that we can keep the code together.

The hard work, which is "Parse+update duchain" stays in the
background, and that's also the only part within which the whole
multi-threading thing makes sense at all, so that's fine, and at least
in this design there's no additional lock contention to worry about.

>> It wouldn't even be desirable to eliminate all instances of the lock, as
>> the lock would achieve the same as a message-passing interface between the
>> foreground and background which would be the alternative, just with much
>> less code.
> But with lot more lock contention. The GUI will hold the lock often, for each
> event, ....
It's the same. If you send a message to the foreground, you also have
to wait until the foreground is idle and processes the event, and at
the same point you'd acquire the lock.

> If the accesses are that rare, it would be not much work and code to port them
> to work on local stored stuff, or?
Anyway just take a look at the design depicted above. I anyway don't
see much that you can copy and store locally.

> Else, you contradict yourself, and they are not that seldom at all and you
> will get your performance penalty from the locks.

If you wrap each access to a SmartRange with a foreground lock then
yes, you may get additional lock contention, although not much more
than with the smart-lock that we're using now (usually the foreground
is most busy with the editor, which in turn holds the smart-lock).
Actually doing that wrapping might be a viable first step to get rid
of all the multi-threading smart-range crashes.

Anyway I think the _first_ thing we need to worry about is stability,
as that's one of the main problems now. The second most important
thing is "fluidity", eg. that the IDE is well-usable even while the
background-parser is running. Only then comes "speed" of the parsing.
Who cares whether the whole project is processed in 4 or 8 minutes, if
you can work without issues while parsing? So the whole talking about
lock contention feels kind of useless atm.

Greetings, David