Complex text input in Plasma

[Martin Gräßlin]

Am 2017-04-08 21:21, schrieb Weng Xuetian:
> On Saturday, 8 April 2017 09:46:17 PDT，Martin Gräßlin wrote：
>> Am 2017-04-08 17:26, schrieb Weng Xuetian:
>> > You're wrong about the QT_IM_MODULE stuff. To make application to use
>> > the
>> > wayland protocol to type (text-input), the implementation must be done
>> > with
>> > QT_IM_MODULE=wayland. I don't mind if it is set to certain application
>> > but set
>> > it in general won't work. Also to have real virtual keyboard , you need
>> > to let
>> > the input method daemon to provides a virtual keyboard implementation.
>> 
>> No you are wrong about that one :-) It might be that it used to be 
>> like
>> that, but Wayland is the default if no QT_IM_MODULE is specified. See
>> https://code.qt.io/cgit/qt/qtwayland.git/tree/src/client/qwaylandintegration
>> .cpp#n142
> 
> What I want to say is, if you set QT_IM_MODULE to qtvirtualkeyboard, 
> that app
> can't talk to im daemon via the wayland protocol. The automatic 
> selection
> procedure is not really a problem there.

And that's not the case. Our virtual keyboard is using the Wayland text 
input protocol and does not require setting the QT_IM_MODULE at all.

Please believe a little bit in what I'm writing. I coded this stuff.

> Also, running it inside kwin could still be problematic. It'll also 
> need to
> display UI. From what I heard about, to run a qml window inside kwin 
> could
> even be a problem.

I don't know what you have heard but all our UI in KWin is written in 
Qml nowadays.

> 
> The races you pointed about is not really a problem. Weston has already 
> done
> it well. Unless you want to reinvent a new protocol, using the current 
> wayland
> protocol essentially does what I said. And if such a thing would 
> interfere
> with kwin's rendering procedure, then probably kwin should refactor 
> about
> that.

Just because Weston doesn't think of this problem doesn't mean it 
doesn't exist. Weston is not perfect and I found several bugs in the way 
they implemented the protocols they developed and Weston is the 
reference for.

> 
> And let me give you some number here, a modern input method for Chinese 
> could
> takes about 30ms to handle one single key (it is THAT complex and we 
> are all
> limited by the complexity, though I measured it about 5 years ago). You 
> really
> want to have kwin block on that and wait for such a long time? And just
> because it is such a slow thing, the IPC latency is not even 
> comparable. If
> you want good prediction or handwriting, it will be slow. There's no im 
> use
> neural network on linux right now, but iOS's virtual keyboard is 
> already doing
> that. Which might be the future, but also mean the im processing time 
> might
> not slow down.

I don't want to base any decisions on 5 year old numbers. 30 msec would 
be bad, but if it is with modern hardware just 10 it wouldn't be that 
bad.

> 
> Also the "input method always on" idea is not saying that every single 
> key
> will be send to input method, it just mean whenever user need type 
> text, it
> will be forwarded to input method.
> 
> I don't know whether I could persuade you, but please look around the 
> world,
> iOS, android, sailfish (also use wayland), mac OS , (windows used to 
> run im
> directly inside application.. but now they also have a service for that 
> ), no
> one would run a im daemon inside the compositor. Are they all ignore 
> the ipc
> overhead?

I'm using sailfish and Android and both keyboards suck bad times. 
Sailfish is really bad, well it's maliit so no surprise there. With both 
systems I see problems in UI positioning because it's not in the 
compositor. Both don't feel perfect especially not in the "every frame 
perfect sense" we are aiming for on Wayland.

Cheers
Martin