[konsole] [Bug 339439] Konsole treats non-BMP (Basic Multilingual Plane) unicode characters inconsistently

[Kurt Hindenburg]

https://bugs.kde.org/show_bug.cgi?id=339439

Kurt Hindenburg <kurt.hindenburg at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Latest Commit|                            |https://commits.kde.org/kon
                   |                            |sole/e74cf6c36642247f3f7919
                   |                            |4da373d01a00645d36
         Resolution|---                         |FIXED
             Status|REPORTED                    |RESOLVED
   Version Fixed In|                            |18.12

--- Comment #4 from Kurt Hindenburg <kurt.hindenburg at gmail.com> ---
Git commit e74cf6c36642247f3f79194da373d01a00645d36 by Kurt Hindenburg, on
behalf of Mariusz Glebocki.
Committed on 03/10/2018 at 15:11.
Pushed by hindenburg into branch 'master'.

Use new character width code based on Unicode 11

Summary:
Adds a code for getting character width togeter with LUTs generated
using uni2characterwidth from Unicode 11 lists.

Skin tone, flags, gender, and other emoji with and modifer are not
joined (you will see e.g. a skin tone square + generic yellow emoji).
I think joining them would cause problems in most editors, command line
prompts, and other programs which use character width data, as the
characters would behave as combining or emoji depending on context (like
ligatures).

Examples:
* light thumb up: 👍🏻
* dark thumb up:  👍🏿
* Polish flag:    🇵🇱

This behavior is allowed:
* https://unicode.org/reports/tr51/#Emoji_Modifiers_Display
* https://unicode.org/reports/tr51/#Emoji_ZWJ_Sequences

It is possible to add support for sequences, but those would work
only for a string width functions.

Some characters which can be presented as emoji are narrow (e.g. ✖️, ©️).
Those characters are listed without "presentation" mode, which means
they should be rendered as text by default (real presentation depends on
renderer and/or font). Noto Sans Color Emoji renders them as wide,
DejaVu Sans as narrow. Vim, bash and zsh treat them as narrow, so I made
them narrow.

https://unicode.org/reports/tr51/#Presentation_Style
Related: bug 396435, bug 378124, bug 392171

FIXED-IN: 18.12

Depends on D15757

Test Plan:
* Look at emoji_test.txt - emojis should look "normal" (two characters
width).
* Look at GLASS.txt - characters width should look correct.
* CharacterWidthTest should pass.
* perl -XCSDL -e 'print map{chr($_), " "} 1..0xffff'

Reviewers: #konsole, #vdg, hindenburg

Reviewed By: #konsole, hindenburg

Subscribers: hindenburg, broulik, ngraham, konsole-devel

Tags: #konsole

Differential Revision: https://phabricator.kde.org/D15758

D  +0    -64   COPYING.Unicode
M  +1    -1    src/CMakeLists.txt
M  +2    -2    src/Character.h
A  +159  -0    src/CharacterWidth.cpp     [License: GENERATED FILE]  *
A  +8    -0    src/CharacterWidth.h     [License: UNKNOWN]  *
A  +102  -0    src/CharacterWidth.src.cpp     [License: GPL (v2+)]
M  +1    -1    src/Filter.cpp
M  +1    -1    src/TerminalCharacterDecoder.cpp
M  +1    -1    src/TerminalDisplay.cpp
M  +6    -2    src/autotests/CharacterWidthTest.cpp
D  +0    -238  src/konsole_wcwidth.cpp
D  +0    -16   src/konsole_wcwidth.h
A  +3    -0    tools/uni2characterwidth/overrides.txt

The files marked with a * at the end have a non valid license. Please read:
https://community.kde.org/Policies/Licensing_Policy and use the headers which
are listed at that page.


https://commits.kde.org/konsole/e74cf6c36642247f3f79194da373d01a00645d36

-- 
You are receiving this mail because:
You are the assignee for the bug.