Patch: TODO highlighting in comments

[Dmitry Risenberg]

2010/11/13 David Nolden <zwabel at googlemail.com>:
> I think it was too early to apply this on master. There are some
> severe efficiency problems. First of all, there were some very heavy
> QString/QByteArray operations on every single comment encountered
> during parsing, which is evil, since we already before spent a lot of
> time processing comments. I have somewhat fixed these efficiency
> problems by adding an _efficient_ pre-check for the todo keywords
> (based on fast IndexedString token comparison).

For future information, is the correct way to deal with parsing tokens
to theat them all as IndexedStrings?

> The other big problem is (as Dmitry already noted) the way the
> problems are collected from the top-contexts when the scope goes over
> the whole project. There doesn't seem to be a proper updating scheme
> (see BackgroundParser::parseJobFinished for a possible entry point
> into updating), and the whole TopDUContexts of the whole project are
> processed whenever the items are collected.

I thought about updating the list on timer: start a timer for MIN time
once we call for a batch parse. When the timer fires, the list of
problems is updated. If parse job finishes before the timer fires, the
timer is reset to MIN again, but only if less than MAX time has passed
since the last update. This way we can be sure that we don't update
the problem list too often, but still can be responsive enough and
make the user wait for refresh. We also don't have to bother with
updating any out-of-duchain data when a job finishes. Of course this
does not guarantee to scale linearly with the size of the project,
because if a typical parse job takes more than MIN, we still have the
same behaviour as now, but if it already does take that long, then
adding some more time on updating the problem list is OK. I'll try
this solution first and if it doesn't work, try to store data from
parseJobs (below).

> For the slightly farrer future, it would be better to not use the
> duchain directly, for example to build the TODO lists, but to transfer
> the TODOs of the whole project into a separate ItemRepository, where
> they can be stored and accessed persistently and efficiently (this
> repository would need to be partially updated only after each
> parseJobFinished).

This does not behave very well with "Show Imports" feature, because we
will also have to track the list of imported modules for each module
and implement a traversal of imports when we build the problem list. I
had such an implemantation in the beginning, but decided to abandon it
because it became hard to track when the problem list should be
updated.

-- 
Dmitry Risenberg