[Konsole-devel] [Bug 59162] New: Fixed width font alignment problems for Japanese (and other languages I suspect)

[Ken Deeter]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
     
http://bugs.kde.org/show_bug.cgi?id=59162     
           Summary: Fixed width font alignment problems for Japanese (and
                    other languages I suspect)
           Product: konsole
           Version: unspecified
          Platform: Gentoo Packages
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: NOR
         Component: general
        AssignedTo: konsole-devel at kde.org
        ReportedBy: ktdeeter at alumni.princeton.edu


Version:            (using KDE KDE 3.1.2)
Installed from:    Gentoo Packages
Compiler:          gcc 3.2 
OS:          Linux

This may be a Qt problem, I've done a fair bit of fidgeting, and I can see a solution, but it would likely make the rendering of the terminal quite slow.

Here's the problem. Say I'm running in a japanese locale. In konsole, i can choose a custom font such as, say "Bitstream Vera Sans Mono". In the font preview in the font dialog, it displays the correct font for english characters, and then it also displays from Japanese characters, which are brought in from the font Kochi Gothic on my system. This is likely because the underlying fontconfig provides Kochi Gothic as an alternative, in case the requested font doesn't cover those certain characters. (I believe the problem is similar when Qt uses its own font replacement)

Up to this point, all is fine and dandy except for a few points. Because there is no relationship between the bitstream font and the kochi font, the typical relationship that is true with Japanese fixed width fonts (i.e. one half-width character is exactly half the width of one full width character) is not true.

On top of this QPainter::drawText() will draw the combined font (Bitstream + Kochi) without respecting the fact that a CJK character should occupy exactly twice
as much space as an alphanumeric one, even though konsole specifically asks for a fixed pitch font. Rather, it will give each character only the space the glyph takes up, as is done with proportional fonts.

The effects of this are as follows:

Say I have a line of text on my terminal that has some mixed english/japanese characters, with all the same graphical attributes. The way that konsole would render this, would be using a single drawText() function. Unfortunately, since the drawText() does not enforce the CJK = 2 * alphanumeric width rule (even if the font was explicity requested as fixed pitch), every character that occurs after the first CJK character will be out of place (the problem occurs with the first CJK character because it is the first "offending" one. I.e. it will not fit into the scheme that konsole uses to calculate it's terminal cell spacing). The degree to in which it is misaligned depends on the ratio of CJK to alphanumeric characters in the current mixed font.

The obvious solution, one might think, is to just use a Japanese font that is carefully designed so that width(CJK char) == 2 * width(alphanumeric). Unfortunately, (and I think this is a Qt bug) if I choose Kochi Gothic in the font dialog directly, then for some reason, every alphanumeric character gets twice as much space as it needs (so they looked spaced out). Because the current console also knows about full/half width characters, in this case, every half-width character gets one full space, and every full-width character gets TWO full spaces, making things just shoot of the side of the screen.

The real fix is probably to fix how Qt behaves. (and btw, QChar really needs a function that says whether a char is full-width or not, as this is defined in the Unicode standard annex, and quite essential to a lot of typesetting algorithms) However, it can also be fixed in konsole, by calling a drawText for each character individually, starting from each actual correct grid position. If this is done, then when it is the case that width(CJK char) < 2 * width(alphanumeric char) then, all CJK chars will be rendered with some extra space between them (but at least they will be aligned!). When the relationship is a ">" instead of "<" There will probably be some overlap, as this means the CJK chars are too wide, but I don't think this happens much in practice.. you would have to be using very "thin" western characters (super condensed)

Ideally, when you call drawText() with a font that is supposed to be fixed pitch, Qt should enforce the CJK = 2*alphanumeric rule, but I am skeptical that this will be fixed soon.

BTW, this problem also causes the screen to become garbled when only parts of it need to be redrawn (like when doing selection, or when an XIM client window disappears above it, etc etc). You get little fragments of characters that only go away with a full redraw. The real annoyance though, is that things that are supposed to line up across lines using the CJK = 2*alphanumeric rule do not, making column-aligned data hard to read.