Cpp Parser & multibyte chars (bug 274430)

[Milian Wolff]

Hey all

I've spent some time today and investigated bug 274430 [1], which shows that 
our C++ parser breaks on C-Strings containing wide chars.

Andreas tried to convince me in IRC that this is "broken code", since anything 
besides ASCII in C++ code is undefined.

I highly disagree, just because it's undefined doesn't mean one must not use 
it. Sure, if you are writing portable code one *should* not use it, but at 
least in my university and probably in science in general, people tend to like 
utf8 symbols in the output of their computation results. And since most of 
them are using UTF8 anyways, they will simply put UTF8 chars into their code.

So I'd like to fix this, but how? The big issue I see is that our parser 
operates on QByteArrays (why?) instead of QString, and as such looses all 
encoding information. Hence our lexer needs two steps to iterate over an "ä"-
char instead of one and thusly things it's two chars wide...

Any ideas on how to fix this without rewriting the whole parser to use 
QStrings?

[1]: https://bugs.kde.org/show_bug.cgi?id=274430

Bye
-- 
Milian Wolff
mail at milianw.de
http://milianw.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kdevelop-devel/attachments/20111118/c49f3c1d/attachment.sig>