[konsole] [Bug 373232] Horizontal lines with fractional HiDPI scaling

[Arthur Peters]

https://bugs.kde.org/show_bug.cgi?id=373232

--- Comment #150 from Arthur Peters <amp at singingwizard.org> ---
(In reply to lukebenes from comment #149)
> (In reply to Nate Graham from comment #140)
> > Because of the nature of floating point math and various Qt and X11 bugs,
> > limiting the
> > scale factor increment to 0.0625 (6.25% in percentage form) will improve 
> > the display in many apps.
> 
> Was  0.0625 found by trial and error? Any experts in floating point math on
> this list? Wondering if this is the right magic number or if the correct
> solution is to convert everything to integer math. If so, what would the
> right magic number/flick[1] be for this case.  
> 
> [1]
> https://mux.com/blog/in-defense-of-flicks-or-how-i-learned-to-stop-worrying-
> and-love-705600000/

0.0625 is exactly 1/16. This makes it representable in very few bits in binary
floating point. This is very different from 0.1 which must be approximated and
that approximation contains information in all of the mantissa bits. For
example, in 64-bit floating point 0.1 + 0.1 + 0.1 == 0.30000000000000004. This
kind of very slight difference is an issue since it tends to *change* as you
add or subtract from the number. This means that certain arithmetic is
"numerically unstable" meaning (roughly) that small errors due to floating
point compound through an algorithm. For example,

(0.1 - 1024 + 1024) / 0.1 = 1.0000000000002274
(0.1 - 1100 + 1100) / 0.1 - 0.9999999999990905

However if we use 1/16 this is much better,

>>> (0.0625 - 1024 + 1024) / 0.0625
1.0
>>> (0.0625 - 1100 + 1100) / 0.0625
1.0

This is where the "various Qt and X11 bugs" come in: display code should not do
much in floating point (as you recommend), or it should avoid accumulating
values and even be careful with multiplication. However, the code probably does
all these things (accumulating the height of lines to find where the next line
should be placed maybe be part of this specific bug). 

By limiting using scaling factors that are less susceptible to these issues
will help avoid the bugs. I don't know how 1/16 was chosen though, using 1/32
or 1/64 would have similar results but have less of a safety margin to resist
numerical instability. That said, I think 1/16 is plenty small for real use. I
doubt any one needs a scaling of 33/32 and will not be happy with 34/32 =
17/16.

-- 
You are receiving this mail because:
You are the assignee for the bug.