Patch: TODO highlighting in comments

[David Nolden]

2010/11/14 Dmitry Risenberg <dmitry.risenberg at gmail.com>:
> 2010/11/13 David Nolden <zwabel at googlemail.com>:
>> I think it was too early to apply this on master. There are some
>> severe efficiency problems. First of all, there were some very heavy
>> QString/QByteArray operations on every single comment encountered
>> during parsing, which is evil, since we already before spent a lot of
>> time processing comments. I have somewhat fixed these efficiency
>> problems by adding an _efficient_ pre-check for the todo keywords
>> (based on fast IndexedString token comparison).
>
> For future information, is the correct way to deal with parsing tokens
> to theat them all as IndexedStrings?

Yes, that's what those integers mean. However, depending on the
context, it may also happen that each of then represents only a single
character.

Regarding the next message:
I've pushed a change that fixes a problem. The problem was that, in
the test, the string was converted to a pre-tokenized IndexedString
sequence using tokenizeFromByteArray, while during normal usage, it
was only "converted" using convertFromByteArray, where each character
gets its own token. The preprocessor later merges the tokens for
identifiers, but not within comments.

Greetings, David