a virtual keyboard plasmoid for touchscreens

[Björn Ruberg]

Hello plasmafolks,

this mail is about my idea (and implementation) of an virtual keyboard for 
plasma. It is splitted in three sections, so you can skip what you do not want 
to read.

1. Why I'm interested in a virtual keyboard for plasma and where my ideas come 
from
2. Why having such a virtual keyboard is important for kde
3. Some technical questions. I need your help

-------------------
Part 1:
Some weeks ago I owned a Gigabyte M704. That's an 7-inch touchscreen UMPC with 
a VIA C7 1,2 GHz CPU and 768 MiB of RAM. (I do not have it anymore, gave it 
back because it was not really useful) I had KDE 4.2 running on it and it 
worked suprisingly well. 
Most interesting part of the device was its touchscreen. (It makes fun to move 
plasmoids with your finger.)
Not really useable was it's keyboard. You can only type with one finger per 
hand, it's not comfortable to hold and you have to open up it up first. 
Conclusion: Nice to have this keyboard in case you don't have a working X, but 
you don't really want to use it for anything else. Exspecially you cannot use 
it for daily work.
Moreover if you just want to type in a part of an url, you don't want to move 
your finger from the touchscreen for doing that. It's uncomfortable. 

So I wanted a virtual keyboard on my desktop which I can use on the 
touchscreen. I wanted it to open up quickly on the screen (one tap for that!), 
type my stuff and close it with only one tap, too. 
And I wanted to use my touchscreen for ten-finger writing. 7-inch should be 
just enough for that. But only if the screen space is used optimally. 

So I searched for a virtual keyboard application, but I didn't find anything 
useable for me. None I found could be opened and close with only one tap. And 
most put completly unneeded buttons on the screen. Numpads, Function-keys - I 
didn't need that stuff. But it took so much space that I couldn't think of 
writing with ten fingers. The important alphanumeric buttons got too small.
By the way, most virtual keyboards were ugly. 

That's why I came to the conclusion that there is a virtual keyboard plasmoid 
is needed. And I thought that would be a good for my first real KDE 
contribution.

------------
Part 2:

So, why should you care about my story? 
I think any other person who tries out KDE on a system with touchscreen will 
feel very similar. Until now, touchscreens are quite rare. Not many people 
have one. But that is about to change.
There are more and more cheap touchscreen devices already out there. ASUS has 
a Atom-device which you can place at the wall and use by touchscreen. More and 
more MIDs like the nokia n810 are getting out. And there are the first so 
called netvertibles in the stores, more promising ones will come to the stores 
in the second quarter of the year. 
And perhaps you read that there are ARM-powered devices planned that should 
run ubuntu. As they will be very cheap and slim they have a good potential on 
the market.

So I think the number of devices with touchscreen (and maybe even without a 
keyboard) will drastically increase in the next months. There are 
UserInterfaces for such devices in development, but only for gnome and 
Ubunutu-derivats as far as I can see. 

If KDE does not work well with touchscreens, users may be forced to stick with 
gnome. Or more worth, even with Windows. 
Luckily the missing virtual keyboard is the only critical problem KDE has with 
touchscreens, as far as I can tell. Support of certain mousegestures (i.e. for 
grabbing) is another one, but that's only a nice to have, no must.


--------------
Part 3:

So, luckily I'm not only committing ideas, but even code. You can find my first 
implentation of a virtual keyboard named plasmaboard in playground. I think it 
is even useable. It produces only a german keyboard at the moment, but it is 
possible to switch to the basic latin keyboard by changing only one line of 
code.
Thanks to notmart for some help in the channel regarding the plasma stuff.
If you want to have look, check out 
trunk/playground/base/plasma/applets/plasmaboard/

Please share your ideas regarding this applet with me. This is my first real 
programm in C++, Qt and plasma of course. So I'm eager to read your 
suggestions regarding the code.

It's an PopupApplet which puts the keyboard in an extender. The keyboard 
itself consists of 63 hand positioned Plasma::PushButtons. The keyboard should 
be freely resizeable and remember its size. It can be used as an applet on the 
desktop and as an extender in front of your videos. These functionalitys are 
already given by the Extender classes.

I have some question though:

- Where to get the key layouts from?
  Of course it's not optimal to have key layouts hard coded in the code. 
Although it may even do the job, what other ways are there?
  The other problem are the keymaps. What key does produce which symbol? 
Currently I hardcoded that too. It may even be a good solution, if I can get 
XKeycodeToKeysym() to work. This Xlib function should recognize the xkb-layout 
and the pressed key modifiers for me, so I don't have to worry about it. Does 
anybody can help me using that function? It's not too well documented and I 
cannot find sourcecode.

- X-Dependency
  Currently I use the function XTestFakeKeyEvent to produce key events that 
should work with everything you  can have on your desktop. This means that 
plasmaboard needs X to work correctly. Is there any alternative for faking key 
events?
  Is X-dependency a problem at all? Windows users do indeed have alternatives 
for a virtual keyboard.

- plasma crashes when typing fast
  When typing a lot and call XTestFakeKeyEvent often, plasma crashs with a "X 
server busy" message. You can probably easy reproduce that by typing some 
sentences with plasmaboard. That's a bug that needs to be solved. Can a plasma 
dev debug it? It may be okay to loose a key when typing, but crashing plasma 
is bad.

- How to find out the display?
  I currently use KWin::display() to get the acutal display. I need it for 
XTestFakeKeyEvent . It works for me but probably it is totally wrong. Where to 
get the display from correctly?

- Plasma::PushButtons
  I currently use Plasma::PushButton for my Buttons. I tested svg and 
iconwidget, too, but they didn't work for me. I have currently two problems 
with it: Fist I need a way to put a button in a pressed state (for example 
Control should stay pressed until another key is pressed). I didn't find one. 
Can you help?
  Second I want a better formed Return-Button. My current overlapping solution 
is not beautiful. Any ideas how I can create a button that is not formed as an 
rectangle?

- Problems with Extender
  I recently implemented the keyboard as an Extender. Since that I can resize 
the keyboard only horizontally any more. Not vertically. When put on the 
desktop, the keyboard is not resizeable at all. What do I wrong?
  Moreover, when I drag the keyboard on the desktop and click on the icon in 
the panel again, I only get a small empty window, but no new keyboard. How to 
do it right?
  
  I noticed the extender to be quite buggy. Moving around my keyboard (but it 
happens with the battery extender as well) is likely to crash plasma. 
  I exspecially have a problem when I try to resize the keyboard to full 
screen width. Try it, suddenly around 30 pixels at the left are clipped away.
  

Well, this was a long mail. I appreciate suggestions, tipps, answers and of 
course help in coding. Perhaps somebody can tell me what to do for making 
plasmaboard ready to get into a plasma-package like plasma-addons?

Greetings,

Björn