Solid::Control::RemoteControl + KDE shortcut system [was: KRemoteControl moved to kdereview]

[todd rme]

On Thu, Mar 18, 2010 at 4:29 AM, Michael Zanetti
<michael_zanetti at gmx.net> wrote:
>> What about just integrating it into the normal KDE shortcut system, so
>> that remote control buttons could be used the same way keyboard
>> shortcuts are now?  A pluggable system where people could provide new
>> backends for the normal KDE shortcut system would be very handy in my
>> opinion.  It wouldn't just be useful for remote controls.  For
>> instance I imagine the simon speech recognition software being
>> discussed on the mailing list right now could probably use such a
>> system to allow voice commands to be integrated into KDE's shortcut
>> system.  It could also be used, with the appropriate backends, for
>> mice and joysticks, both of which KDE applications currently need to
>> program their own shortcut system for (as plasma does now for mice and
>> some games do for joysticks).  It would mean people would only needs a
>> single, consistent interface for assigned functions to all of their
>> input devices.
>>
>
> That would be a somewhat bigger design decision introducing new opportunities
> but also quite big limitations.
>
> + It would be nice for people with simple use cases to set up their remote
> controls. Just open your media player app, configure shortcuts, and select the
> desired buttons. No need to use apps like KRemoteControl/KHotKeys.
>
> * KRemoteControl and KHotKeys would merge. In my opinion, while they have some
> common stuff, they are still quite different and I cannot yet immagine a way how
> to combine all the use-cases of keyboards and remote controls in one app.

No, this is not really what I am proposing.  My thinking was that
KRemoteControl would remain around for things like setting up remotes,
assigning names to buttons, and so on.  The only thing that would
change is that you would not use KRemoteControl for assigning actions
to buttons.  That would be done within the normal shortcut system.
KRemoteControl would maintain all its own functionality for stuff
related specifically to remote controls, but it would send out all of
the button presses it receives to the shortcut system so they could be
used by KDE applications without needing any extra work on the part of
application developers.

So in other words, what I am suggesting is a separation in the
configuration of input devices.  Everything that is specific to a
particular type of device would be in that device's configuration.
However, things that are more general across input devices, like
assigning buttons to KDE application actions, would use a single
interface for all devices.

For example, with the voice recognition you will still need the module
to learn voice commands and assign names to the voice commands that
the shortcut system can make use of, so it would need its own module
for that sort of thing.  However, once that is done, you would use the
normal shortcut system for actually getting those voice commands to do
something useful.

Similarly, you would still need a joystick configuration module for
things like calibration.  But assigning actions to joystick buttons
would be done within the shortcut system.

You would still need a bluetooth configuration module for pairing
devices, setting trust, handling audio and data, etc, but assigning
actions to bluetooth device buttons would be done within the shortcut
system.

You would still need a mouse configuration module for things like
mouse speed, left vs. right-handedness, double click speed, etc, but
assigning actions (besides the normal left/right/middle mouse clicks
and scroll wheel) would be done within the shortcut system.

So in other words, things that are the same across all input devices,
like the need to assign actions to buttons, would use a common
interface.  The configuration of device-specific properties, however,
would use device-specific modules.

There is another potential benefit as well.  If the shortcut system is
aware of all the buttons available to each device, it might be
possible to set it up to map the buttons from one device to the
buttons of another.  Like mapping the buttons on a bluetooth phone
(that supports it) to numbers and letters on a keyboard, or mapping
remote control buttons to mouse presses.  This is not essential to my
idea, but it could potentially be a side benefit of this sort of
implementation.

> - People lose the ability to use their remote control as a multi-purpose
> remote. KRemoteControl has a mechanism named mode-switching. Using this, you
> can switch a remote to different modes. This lets you configure an unlimited
> number of actions with very small remote controls. You can find more
> information about this in the KRemoteControl handbook, sitting in kdereview
> currently :) Integrating remotes into the shortcut system would kill this, for
> remote controls quite essential feature. I hardly would use my remote control
> any more if the Play button could only launch/play/pause a single application
> or I would need to grab the keyboard/mouse to change the input focus. Well, I
> actually could set a button on the remote to do Alt+Tab, but still it would be
> way more limited than what its now.

Yes, this is the sort of thing you would need to keep KRemoteControl
around for, because it is unique to remote controls.  But that doesn't
mean KRemoteControl should duplicate the functionality of the shortcut
system when it comes to assigning buttons to ordinary actions.

In this case, actually, this something that I think the entire
shortcut system could benefit from.  For instance I think multimedia
keyboards would benefit a lot from this, as would joysticks in more
complex games.  So this may be a case where I think, rather than it
being something that should be kept unique to remote controls or
something that should be done away with, it is something that should
be incorporated into the shortcut system as a whole.

> - Another problem is the future of such an interface. With AVRCP version 1.3
> and greater, a remote control has the ability to query the controlled target
> for the the current audio/video metadata, browse the filesystem etc. (they
> might have displays nowadays :). While the current
> Solid::Control::RemoteControl doesn't support this yet, the api coud be
> extended. I guess there wouldn't be any reasonable way to introduce such a
> feature into the shortcut system.

Yes, once again this is something that would require KRemoteControl to
stick around.  I am not suggesting replacing KRemoteControl, I am
saying KRemoteControl should hook into the shortcut system just for
the purposes of making shortcuts.  It can keep doing anything else it
needs to do on its own.

> In my opinion the shortcut system in its current state does not really fit into
> the purpose of remote controls. For example you cannot use button combinations
> on a remote control while it doesn't make really sense to have a number of
> modes for a keyboard, just to get more virtual buttons. However, lets see what
> the metalworkers think about this.

Why doesn't it make sense for a keyboard?  My multi-media keyboard
already has a built-in mode switching ability (although it only works
on some keys and only has two modes).  I think it would be very useful
for a number of buttons on my device.  All in all the use cases for
multimedia keys on a keyboard I would say would be fairly similar to
the use-cases of remote controls, anything you would want to do with
one you would probably also want to be able to do with the other.



There is one detail I forgot to mention: devices would have to be able
to deny certain button presses or button press combinations.  For
instance KRemoteControl would have to be able to tell the shortcut
system that it can't use the buttons used for mode switching.  The
mouse backend would have to be able to say you can't use the left
button, right button, middle button, or scroll wheel on their own (but
it would allow, say the left button combined with one or more
non-mouse buttons, such as left mouse+keyboard meta).  This shouldn't
be too difficult because it should only be limited to blocking one or
more of the device's own buttons in combination with or not in
combination with the same device or a different device.  So the
backend could just provide a list with some or all of these values:

Buttons that cannot be used for shortcuts: a,b,c
Buttons that cannot be used for shortcuts unless combined with another
button from any device: d,e,f
Buttons that cannot be used for shortcuts unless combined with another
button from this device: g,h,i
Buttons that cannot be used for shortcuts unless combined with another
button from this device type: j,k,l
Buttons that cannot be used for shortcuts unless combined with another
button from a different device: m,n,o
Buttons that cannot be used for shortcuts unless combined with another
button from a different device type: p,q,r

Or, alternatively, the shortcut system could send any proposed
shortcuts to the backends that provide buttons used in that shortcut
(along with data like source device and source device type for each
button press in the shortcut) and the backends could all report
whether that shortcut is acceptable.  That would allow for more
complicated rules to be implemented inside the backends.

-Todd