Review Request 129727: kdeconnect-kde: Add remotekeyboard plugin

Holger Kaelberer holger.k at elberer.de
Wed Jan 11 19:48:14 UTC 2017 


> On Jan. 10, 2017, 9:35 p.m., Albert Vaca Cintora wrote:
> > >For the json-payload I used the syntax of the key-events as sent by mousepad-plugin with the addition of a "sendAck"-flag. If "sendAck" is set to true the remote peer should echo a key-event if it could be handled, thus allowing the local client to find out whether the key was accepted. For performance reasons, it's allowed to send multi-char strings in the "key" property (performs much better if you send a whole string via "echo '...' | kdeconnect-cli ..." e.g.)
> > 
> > I like the fact that you used the same protocol. This actually should allow us to send keys phone-to-phone and kde-to-kde, but as of now it's not working because the package type used phone-to-desktop are called "kdeconnect.mousepad" and the desktop-to-phone are called "kdeconnect.remotekeyboard". We tried changing your code to use "kdeconnect.mousepad" and it worked Android to Android :) so it might be a good idea to rename the package type (even though it only implements the keyboard part and not the cursor part, but it's not a big deal). It's true that you "extended" the protocol by adding the "echos" (maybe they can be renamed to "kdeconnect.mousepad.echos" to make it more clear?) but I don't think it will break if echos are not present (eg: desktop-to-desktop).
> > 
> > >kdeconnect-cli: For now takes a string and transforms it into single key-events for visible characters only. In a first implementation I used a kbhit() helper that used termios.h to catch and relay keypresses interactively (including some special-events), which was not optimal. A better approch might be to use linux input-api directly. Would this be an option regarding cross-platform compatibility or can I assume to develop for Linux only? Being a command-line guy, I'd really like to have a fully featured kdeconnect-cli interface ;-)
> > 
> > I like your approach because it's simple and works multi-platform.
> > 
> > >Factor out the Qt::Key-to-internal keymap to some core-helper because it corresponds to the mapping in the mousepad-plugin?
> > 
> > It sounds like a good idea, we can do it in a separate patch.
> > 
> > >The plasmoid is not perfect as it is: A single line containing a non-echoing TextField (i.e. it eats all the KeyPress events), and only ack-ed keypress-packets from the peer device are injected if they contain visible keys. Advantage: the user sees whether his key-presses are accepted by the peer device. Disadvantage: The echoed text does not correspond 1:1 to what is shown on the peer's display, user might be confused when typing without success. I played around with different variations each of which with its proper shortcomings:
> > 1. An echoing Textfield for typing: Has the advantage that the user can directly see what he is typing, which makes interaction in the typing field easier, BUT messes up interaction if the Editor on the peer is changed silently and does not notify the user if his keypresses are not handled by the peer.
> > 2. A non-echoing TextField for typing PLUS a readonly one for printing visible echoed keys. Disadvantage: same as for the previous one and uses more space on the plasmoid.
> > Comments? Ideas?
> > 
> > Agreed. I also found it a bit confusing that it is no 1:1. Maybe if we echo a few more things like the Backspaces it will get better?
> > 
> > 
> > One extra thought: right now the Plasmoid doesn't know if the keyboard is opened on the Android side or not. If we make the Android side notify the desktop when the keyboard is open or closed we can use that info on the desktop. For example: gain focus of the input field automatically, or hide it when there is no receiver on Android. Any thoughts?

> We tried changing your code to use "kdeconnect.mousepad" and it worked Android to Android :) so it might be a good idea to rename the package type (even though it only implements the keyboard part and not the cursor part, but it's not a big deal).

Ok, will change it to mousepad!

>> The plasmoid is not perfect as it is: A single line containing a non-echoing TextField (i.e. it eats all the KeyPress events), and only ack-ed keypress-packets from the peer device are injected if they contain visible keys. Advantage: the user sees whether his key-presses are accepted by the peer device. Disadvantage: The echoed text does not correspond 1:1 to what is shown on the peer's display, user might be confused when typing without success. I played around with different variations each of which with its proper shortcomings:
1. An echoing Textfield for typing: Has the advantage that the user can directly see what he is typing, which makes interaction in the typing field easier, BUT messes up interaction if the Editor on the peer is changed silently and does not notify the user if his keypresses are not handled by the peer.
2. A non-echoing TextField for typing PLUS a readonly one for printing visible echoed keys. Disadvantage: same as for the previous one and uses more space on the plasmoid.
Comments? Ideas?

> Agreed. I also found it a bit confusing that it is no 1:1. Maybe if we echo a few more things like the Backspaces it will get better?

Not sure, unless we interpret it completely as is done on android side it will feel incomplete. My first try was to inject a real QKeyEvent upon receiving an echo package into the Textfield, which would result in a (almost) perfect behavoir as the Android keyhandling tries to mimic TextField's handling of special keys on desktop. I tried hard via a native wrapper for QCoreApplication::postEvent() but there was never a event arriving at the QQuickItem. Hints from the Qt-experts?

Another option would be to simply disable echoing in the plasmoid completely and let the user look at the result on the Android side.

> One extra thought: right now the Plasmoid doesn't know if the keyboard is opened on the Android side or not. If we make the Android side notify the desktop when the keyboard is open or closed we can use that info on the desktop. For example: gain focus of the input field automatically, or hide it when there is no receiver on Android. Any thoughts?

Thought about an extra package for that case, too. For now the echoing in the plasmoid is supposed to give the user feedback about whether his keypresses are handled remotely or not. We could add a kdeconnect.mousepad.keyboard_status package and act as you proposed (gray-out vs. activate the textfield or so). With this feedback keypress echoing could also be disabled in the plasmoid completely because we give other feedback to the user.


- Holger


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://git.reviewboard.kde.org/r/129727/#review101931
-----------------------------------------------------------


On Jan. 11, 2017, 4:32 p.m., Holger Kaelberer wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://git.reviewboard.kde.org/r/129727/
> -----------------------------------------------------------
> 
> (Updated Jan. 11, 2017, 4:32 p.m.)
> 
> 
> Review request for kdeconnect.
> 
> 
> Bugs: 370919
>     http://bugs.kde.org/show_bug.cgi?id=370919
> 
> 
> Repository: kdeconnect-kde
> 
> 
> Description
> -------
> 
> Allow to inject keypress events to remote peers (most notably Android devices)
> 
> Notes / open issues / possible improvements:
> 
> - For the json-payload I used the syntax of the key-events as sent by mousepad-plugin with the addition of a "sendAck"-flag. If "sendAck" is set to true the remote peer should echo a key-event if it could be handled, thus allowing the local client to find out whether the key was accepted. For performance reasons, it's allowed to send multi-char strings in the "key" property (performs much better if you send a whole string via "echo '...' |  kdeconnect-cli ..." e.g.)
> 
> - kdeconnect-cli: For now takes a string and transforms it into single key-events for visible characters only. In a first implementation I used a kbhit() helper that used termios.h to catch and relay keypresses interactively (including some special-events), which was not optimal. A better approch might be to use linux input-api directly. Would this be an option regarding cross-platform compatibility or can I assume to develop for Linux only? Being a command-line guy, I'd really like to have a fully featured kdeconnect-cli interface ;-)
> 
> - Factor out the Qt::Key-to-internal keymap to some core-helper because it corresponds to the mapping in the mousepad-plugin?
> 
> - The plasmoid is not perfect as it is: A single line containing a non-echoing TextField (i.e. it eats all the KeyPress events), and only ack-ed keypress-packets from the peer device are injected if they contain visible keys. Advantage: the user sees whether his key-presses are accepted by the peer device. Disadvantage: The echoed text does not correspond 1:1 to what is shown on the peer's display, user might be confused when typing without success. I played around with different variations each of which with its proper shortcomings:
> 1. An echoing Textfield for typing: Has the advantage that the user can directly see what he is typing, which makes interaction in the typing field easier, BUT messes up interaction if the Editor on the peer is changed silently and does not notify the user if his keypresses are not handled by the peer.
> 2. A non-echoing TextField for typing PLUS a readonly one for printing visible echoed keys. Disadvantage: same as for the previous one and uses more space on the plasmoid.
> Comments? Ideas?
> 
> 
> Diffs
> -----
> 
>   cli/kdeconnect-cli.cpp 1acb723 
>   interfaces/CMakeLists.txt 19a9cdc 
>   interfaces/dbusinterfaces.h 9848d0c 
>   interfaces/dbusinterfaces.cpp aab3e05 
>   plasmoid/declarativeplugin/kdeconnectdeclarativeplugin.cpp 282f2a9 
>   plasmoid/package/contents/ui/DeviceDelegate.qml e90e021 
>   plasmoid/package/contents/ui/RemoteKeyboard.qml PRE-CREATION 
>   plugins/CMakeLists.txt 9d230ac 
>   plugins/mousepad/kdeconnect_mousepad.json 850e855 
>   plugins/remotekeyboard/CMakeLists.txt PRE-CREATION 
>   plugins/remotekeyboard/README PRE-CREATION 
>   plugins/remotekeyboard/kdeconnect_remotekeyboard.json PRE-CREATION 
>   plugins/remotekeyboard/remotekeyboardplugin.h PRE-CREATION 
>   plugins/remotekeyboard/remotekeyboardplugin.cpp PRE-CREATION 
> 
> Diff: https://git.reviewboard.kde.org/r/129727/diff/
> 
> 
> Testing
> -------
> 
> A lot!
> 
> 
> Thanks,
> 
> Holger Kaelberer
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kdeconnect/attachments/20170111/7c5d69d2/attachment-0001.html>

More information about the KDEConnect mailing list