Review Request 110043: Proposed fix/workaround for legacy encoded filename handling

[David Faure]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://git.reviewboard.kde.org/r/110043/
-----------------------------------------------------------

(Updated May 5, 2013, 8:13 a.m.)


Review request for kdelibs and Thiago Macieira.


Changes
-------

I very much like the idea of finally making it possible to deal with non-utf8 files, since this can always happen when plugging in a USB key coming from another OS, mounting a network partition, etc.

Ideally this should be handled in Qt rather than in KDE though. But let's see what Thiago has to say about either solution, it's his domain...

Triggering this with a special env var is probably too "shy". We probably want this to work out of the box without the need for users to change set an env var, provided that it doesn't create any regressions (ok, the performance issue would be a regression, which is another reason for doing this in Qt directly -- only for filenames of course).


Description
-------

This patch works around the problem of filenames that are not valid UTF8 strings:  in KLocalePrivate::initFileNameEncoding() KDE sets the QFile's encoding/decoding function, to to/fromUTF8() in QString, which in turn calls QUtf8's converter function (QUtf8 is not exported to developers, so I had to use an inefficient method, I think it would be better if we could use the state parameter for error detection). I replaced this with the said functions' copy/pasted version and changed it, so when it encounters an invalid UTF8 string, it will encode it byte by byte, mapping the lower 128 their normal unicode place and the upper 128 to U+18000-U+1807F, and of course the decoder reverses it. 
To make this actually work you have to define the KDE_UTF8_FILENAMES enviroment variable to a specific value ("broken_names").

To test it, do the following: .kde/env/KDE_UTF8_FILENAMES.sh with this content: 
export KDE_UTF8_FILENAMES=broken_names
logout, login, try dolphin on faulty files. (instead of the usual boxed "?" you'll see just boxes)


This addresses bug 165044.
    http://bugs.kde.org/show_bug.cgi?id=165044


Diffs
-----

  kdecore/localization/klocale_kde.cpp b010e74 

Diff: http://git.reviewboard.kde.org/r/110043/diff/


Testing
-------


Thanks,

Róbert Szókovács

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-core-devel/attachments/20130505/0bc8c9a0/attachment.htm>