On Time and Events

Sat Dec 20 18:51:49 UTC 2014

2014-12-20 18:07 GMT+01:00 Dmitry Kazakov <dimula73 at gmail.com>:
> Hi, Damien!
>
>
>> 1/ speed smoothing : compute speed as a weighed mean of the N last
>> measured event speed. This is implemented in the freehand tool, but it has
>> limits with extremely f*cked up timestamps (several zero-timed events in a
>> row).
>
>
> Yes, agree.
>
>>
>> 2/ early time measurement : Dmitry's proposal is to get the timestamp for
>> tablet events from the window manager, and later from Qt5's
>> QInputEvent::timestamp.
>> I tried to implement a proof of concept for this (see attached patch), but
>> it has limitations too. First, the event pipeline is not implemented in a
>> way that makes such a modification easy to implement (see # Events). Second,
>> it's a non-answer to the problem, and just delegates event time measurement
>> to another software, which may as well have wrong timing (fig 2) or
>> implement complete garbage (for example, OSX's cocoa implements event
>> timestamps as seconds, which Qt5 just multiplies by 1000 to fit their
>> milliseconds interface). Third, timestamps should also be available for all
>> pointer events.
>
>
> Ok, if the proof of concept sais it doesn't work, then yes, we should find
> some other solution.
>
It does have a noticeable effect on X11, wild guess is that coming
from the window manager, the system timestamp is probably less
affected by the cpu load scheduled for krita, so there's less garbage
on input. But it doesn't achieve perfectly smooth speed. However, I
think it can still provide "better" input values  for any smoothing or
interpolation technique.

>>
>>
>> 3/ time interpolation : my original proposition was to fix garbage input
>> at the source. Timestamps are packed and zero-timed because they're
>> cpu-bound, okay : keep the most likely correct events, and interpolate the
>> rest. the "likely" function might have user-tunable parameters. The limit is
>> that false positives for likely/unlikely correct times may produce garbage
>> where the  original input was fine.
>
>
> Did you have some time to do some experiments with it? Any results?
>
Hmm, nope, I stashed that to experiment with system timestamps, and I
got a bit carried away with the refactoring ideas expressed under. And
I didn't have that much time since then. I'll try that approach on my
next attempts.

>> - Event timestamp is currently measured deep into the event handling. It's
>> actually implemented within the base freehand tool (which looks like the
>> only tool to make use of it).
>> I think this is way too late and that it should be handled as early as
>> possible in the event handling pipeline (see point 2/ and 3/ above).
>>
>> - Original QInputEvents have inconsistent interfaces, which are abstracted
>> away by Calligra's KoPointerEvent. This abstrastion is good, but it happens
>> way too late within the canvas/tool proxy input pipeline.
>>
>> This means that anything happening between the event input and its
>> conversion by KisToolProxy (attaching a timestamp for example) has to deal
>> with more than one type of pointer events and their inconsistencies. And
>> KisInputManager/KisToolProxy have their share of complexity.
>
>
>> I think it would be better to leverage the global event filter to create a
>> unified pointer event as soon as possible, perhaps a subclass of QMouseEvent
>> or QTabletEvent for widget compatibility, and rely on that interface.
>
>
> Well, if you say using the system timestamps doesn't make the situation much
> better, I wouldn't spend much time on that refactoring. Probably, we should
> just stick to your original approach with time value filtering and then
> ckeck whether using original timestamps instead of meausred with a QTimer
> make any noticeable difference :) If not, just keep it is as it is.
>

As I said above, system timestamps make the situation noticeably
better (see example images, I couldn't manage to get really awful
lines with it, so it's a better), but it's not a silver bullet and it
needs to be complemented with proper handling of slightly off values.

Besides, I don't have better arguments for this refactoring than :
- it would de-obfuscate KisInputManager/KisToolProxy
- having a unique consistent complete pointer event interface is
better done early and made available to all Krita components
- ...my guts are telling me to, and my guts never lie

So yeah, it's probably not critical beyond the ease of implementing
timestamps correctly, I just think it would help in reducing the
unneeded complexity in several parts of the code.

>>
>> This also means that only a canvas can have access to the unified
>> interface provided by KoPointerEvent, which is used to handle tool-device
>> bindings. This leads to another identified problem : tool-device binding
>> happen only within a canvas, if you flip the stylus to bind the easer end to
>> another tool while being outside of the canvas, and flip back to the stylus
>> end before hitting the canvas again, your binding will not be understood
>> correctly.
>
>
> Could you elaborate a bit? As far as I remember, the check of the tool id
> happens in KoInputManager, which compares the tool id and switches current
> canvas. And this switch happens on the first TabletMove event it gets, so
> the switch happens correctly.
>
AFAICT, there's no KoInputManager, I guess you meant KoToolManager,
where switchInputDevice is implemented. This is called by
KoToolProxy's fooEvent functions (which sometimes uses KoPointerEvent,
sometimes not... I may have oversimplified that in my first
description of the problem) which are called from
KisToolProxy::forwardEvent, which is triggered by the
InputManager/ShortcutMatcher/Actions pipeline.

The problem is that InputManager is an event filter for *Canvas
events*, so it only happens when your cursor interacts with the
canvas. Reproduce these sequences with a two ended stylus :
- Initial state : preset A
- move cursor with stylus end to the preset box
- flip to eraser without moving over the canvas
- choose preset B
--- Branch 1
- move to canvas, draw with B
- flip again to stylus and draw with... B ? <-- Oops, should be A
--- Branch 2 (repeat first steps)
- flip again to stylus without moving over the canvas
- choose preset C
- move to the canvas, draw with C
- flip to eraser and draw with.... C ? <-- Oops, should be B

It has bothered me a few times while painting. I guess it could be
fixed in place without refactoring everything, but it could also be
triggered in one early place.
And, well, KoPointerEvent probably isn't *needed* for that, but from
an architectural consistency standpoint it would make sense to get the
device id from it, to me at least.

> Though there was a bug with it recently, but I fixed it on Thursday.
>
> As a conclusion, I guess you should continue with your research with speed
> filtering that you were suggesting initially :)
>

Yes, I'll try to implement a hybrid solution, each part of the
solution has it's advantages and dead angles, a combination should
reach completeness.

Thanks for your time !

>
> --
> Dmitry Kazakov
>
> _______________________________________________
> Krita mailing list
> kimageshop at kde.org
> https://mail.kde.org/mailman/listinfo/kimageshop
>