On Time and Events

Sat Dec 20 13:54:39 UTC 2014

Hello there, kaoron from IRC

I've been trying for the last few weeks to use my spare-spare time on krita
and help tackling a bug which cripples the velocity sensor.

I've been told that this bug was a non trivial one and had already been
looked by at least three developers without a simple solution to implement.
Make that four of us now, I might have an idea, but it's not simple and it
spans wider than just the velocity sensor(, and I'm not a hardcore C++/Qt
coder at all, and I don't have that much time to consistently work on it
:/).

Through my research, I've identified a few points which might need
consideration :

# Timestamps :

The gist of the speed sensor bug was twofold. First, the timestamp used to
calculate speed was implemented as integer, and couldn't be interpolated
correctly for each dab, this is fixed by commit
d07f0912242373641e167904ec17a48d9106878b
Second, event timestamp is not accurately measured, leading to blobby
strokes anyway (fig 1).

There's several proposed solutions to this second bug :

1/ speed smoothing : compute speed as a weighed mean of the N last measured
event speed. This is implemented in the freehand tool, but it has limits
with extremely f*cked up timestamps (several zero-timed events in a row).

2/ early time measurement : Dmitry's proposal is to get the timestamp for
tablet events from the window manager, and later from Qt5's
QInputEvent::timestamp.
I tried to implement a proof of concept for this (see attached patch), but
it has limitations too. First, the event pipeline is not implemented in a
way that makes such a modification easy to implement (see # Events).
Second, it's a non-answer to the problem, and just delegates event time
measurement to another software, which may as well have wrong timing (fig
2) or implement complete garbage (for example, OSX's cocoa implements event
timestamps as seconds, which Qt5 just multiplies by 1000 to fit their
milliseconds interface). Third, timestamps should also be available for all
pointer events.

3/ time interpolation : my original proposition was to fix garbage input at
the source. Timestamps are packed and zero-timed because they're cpu-bound,
okay : keep the most likely correct events, and interpolate the rest. the
"likely" function might have user-tunable parameters. The limit is that
false positives for likely/unlikely correct times may produce garbage where
the  original input was fine.

fig 1 : blobby speed stroke, result from time-packed/zero-timed events
fig 2 : speed stroke, with original window manager timestamp, still

not perfectly smooth

MyPaint seems to deal with time with a combination of the three above
tricks.

-----

# Events :

- Event timestamp is currently measured deep into the event handling. It's
actually implemented within the base freehand tool (which looks like the
only tool to make use of it).
I think this is way too late and that it should be handled as early as
possible in the event handling pipeline (see point 2/ and 3/ above).

- Original QInputEvents have inconsistent interfaces, which are abstracted
away by Calligra's KoPointerEvent. This abstrastion is good, but it happens
way too late within the canvas/tool proxy input pipeline.

This means that anything happening between the event input and its
conversion by KisToolProxy (attaching a timestamp for example) has to deal
with more than one type of pointer events and their inconsistencies. And
KisInputManager/KisToolProxy have their share of complexity.

This also means that only a canvas can have access to the unified interface
provided by KoPointerEvent, which is used to handle tool-device bindings.
This leads to another identified problem : tool-device binding happen only
within a canvas, if you flip the stylus to bind the easer end to another
tool while being outside of the canvas, and flip back to the stylus end
before hitting the canvas again, your binding will not be understood
correctly.

I think it would be better to leverage the global event filter to create a
unified pointer event as soon as possible, perhaps a subclass of
QMouseEvent or QTabletEvent for widget compatibility, and rely on that
interface.

It's barely a scratch on the surface, I'm not digging into
multiple/tool-device contexts on multiple documents yet, and who should
manage what for that matter (you quickly come to it when poking ToolProxy
and Canvas).

What do you think ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kimageshop/attachments/20141220/d4f26fcf/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: speed_stroke_blob.png
Type: image/png
Size: 11265 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kimageshop/attachments/20141220/d4f26fcf/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: speed_stroke.png
Type: image/png
Size: 8771 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kimageshop/attachments/20141220/d4f26fcf/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: event_timestamp.patch
Type: text/x-patch
Size: 17823 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kimageshop/attachments/20141220/d4f26fcf/attachment-0001.patch>