Cpp Parser & multibyte chars (bug 274430)

[Andreas Pakulat]

On 20.11.11 17:43:51, Milian Wolff wrote:
> On Sunday 20 November 2011 17:28:19 Milian Wolff wrote:
> > On Sunday 20 November 2011 17:19:51 Andreas Pakulat wrote:
> > > On 20.11.11 11:13:02, Aleix Pol wrote:
> > > > On 11/20/2011 09:38 AM, Andreas Pakulat wrote:
> > > > >On 19.11.11 19:09:07, David Nolden wrote:
> > > > >>Using QString is not an option, because it not only affects
> > > > >>memory-usage,
> > > > >>but also efficiency.
> > > > >
> > > > >Whats the efficiency problem with QString here? After all indexing into
> > > > >it to get a QChar out should be O(1), right?
> > > > >
> > > > >>It shouldn't be too complicated though to map ranges regarding utf8
> > > > >>when
> > > > >>doing the highlighting in the editor.
> > > > >
> > > > >Then you break things for anybody not using utf-8 which is just as
> > > > >valid as a source file encoding.
> > > > 
> > > > Now it's broken for people using utf8 apparently, so I don't think
> > > > it's a valid concern.
> > > 
> > > I think you misunderstood. What I wanted to get across is that fixing
> > > just the utf8 case only means things get broken for a different set of
> > > users. Which is just as wrong as sticking with the current situation.
> > 
> > Imo all that is required is to copy Kate's behavior. Esp thanks to Kate's
> > modelines (e.g. in .kateconfig) we would have one central place for users to
> > configure their charset.

Hmm, copy sounds extremely ugly and still leaves the question what to do
when the user changes the encoding via kate's encoding menu.

> Another thing: IndexedString has no encoding knowledge, right? Are we not 
> always blindly assuming that incoming data is UTF-8 encoded? Anything else 
> would be very complicated I assume, or?

As long as all our code converting indexedstring from/to qstring uses
utf8 and there's no code using positions from an indexed string directly
with a converted qstring everything should be fine.

Andreas