Solid::Control::RemoteControl + KDE shortcut system [was: KRemoteControl moved to kdereview]

[Michael Zanetti]

On Thursday 18 March 2010 17:23:40 todd rme wrote:
> On Thu, Mar 18, 2010 at 4:29 AM, Michael Zanetti
> 
> <michael_zanetti at gmx.net> wrote:
> >> What about just integrating it into the normal KDE shortcut system, so
> >> that remote control buttons could be used the same way keyboard
> >> shortcuts are now?  A pluggable system where people could provide new
> >> backends for the normal KDE shortcut system would be very handy in my
> >> opinion.  It wouldn't just be useful for remote controls.  For
> >> instance I imagine the simon speech recognition software being
> >> discussed on the mailing list right now could probably use such a
> >> system to allow voice commands to be integrated into KDE's shortcut
> >> system.  It could also be used, with the appropriate backends, for
> >> mice and joysticks, both of which KDE applications currently need to
> >> program their own shortcut system for (as plasma does now for mice and
> >> some games do for joysticks).  It would mean people would only needs a
> >> single, consistent interface for assigned functions to all of their
> >> input devices.
> > 
> > That would be a somewhat bigger design decision introducing new
> > opportunities but also quite big limitations.
> > 
> > + It would be nice for people with simple use cases to set up their
> > remote controls. Just open your media player app, configure shortcuts,
> > and select the desired buttons. No need to use apps like
> > KRemoteControl/KHotKeys.
> > 
> > * KRemoteControl and KHotKeys would merge. In my opinion, while they have
> > some common stuff, they are still quite different and I cannot yet
> > immagine a way how to combine all the use-cases of keyboards and remote
> > controls in one app.
> 
> No, this is not really what I am proposing.  My thinking was that
> KRemoteControl would remain around for things like setting up remotes,
> assigning names to buttons, and so on.  The only thing that would
> change is that you would not use KRemoteControl for assigning actions
> to buttons.  That would be done within the normal shortcut system.
> KRemoteControl would maintain all its own functionality for stuff
> related specifically to remote controls, but it would send out all of
> the button presses it receives to the shortcut system so they could be
> used by KDE applications without needing any extra work on the part of
> application developers.
> 

Assigning actions to buttons is actually the only purpose of KRemoteControl. 
There is no such configuration as "assigning names to buttons". This is done by 
the backend without any configuration on the KDE side.

However, I can immagine now what you are talking about.

> So in other words, what I am suggesting is a separation in the
> configuration of input devices.  Everything that is specific to a
> particular type of device would be in that device's configuration.
> However, things that are more general across input devices, like
> assigning buttons to KDE application actions, would use a single
> interface for all devices.
> 
> For example, with the voice recognition you will still need the module
> to learn voice commands and assign names to the voice commands that
> the shortcut system can make use of, so it would need its own module
> for that sort of thing.  However, once that is done, you would use the
> normal shortcut system for actually getting those voice commands to do
> something useful.
> 
> Similarly, you would still need a joystick configuration module for
> things like calibration.  But assigning actions to joystick buttons
> would be done within the shortcut system.
> 
> You would still need a bluetooth configuration module for pairing
> devices, setting trust, handling audio and data, etc, but assigning
> actions to bluetooth device buttons would be done within the shortcut
> system.
> 

Bluetooth is somewhat special... For example AVRCP 1.4 specifies "player 
selection". That means a remote control is able to specify to which 
application it gets routed. Wouldn't work out with current KHotkeys system as 
it always gets routed to the current input-focus...

> You would still need a mouse configuration module for things like
> mouse speed, left vs. right-handedness, double click speed, etc, but
> assigning actions (besides the normal left/right/middle mouse clicks
> and scroll wheel) would be done within the shortcut system.
> 
> So in other words, things that are the same across all input devices,
> like the need to assign actions to buttons, would use a common
> interface.  The configuration of device-specific properties, however,
> would use device-specific modules.
> 
> There is another potential benefit as well.  If the shortcut system is
> aware of all the buttons available to each device, it might be
> possible to set it up to map the buttons from one device to the
> buttons of another.  Like mapping the buttons on a bluetooth phone
> (that supports it) to numbers and letters on a keyboard, or mapping
> remote control buttons to mouse presses.  This is not essential to my
> idea, but it could potentially be a side benefit of this sort of
> implementation.
> 
> > - People lose the ability to use their remote control as a multi-purpose
> > remote. KRemoteControl has a mechanism named mode-switching. Using this,
> > you can switch a remote to different modes. This lets you configure an
> > unlimited number of actions with very small remote controls. You can
> > find more information about this in the KRemoteControl handbook, sitting
> > in kdereview currently :) Integrating remotes into the shortcut system
> > would kill this, for remote controls quite essential feature. I hardly
> > would use my remote control any more if the Play button could only
> > launch/play/pause a single application or I would need to grab the
> > keyboard/mouse to change the input focus. Well, I actually could set a
> > button on the remote to do Alt+Tab, but still it would be way more
> > limited than what its now.
> 
> Yes, this is the sort of thing you would need to keep KRemoteControl
> around for, because it is unique to remote controls.  But that doesn't
> mean KRemoteControl should duplicate the functionality of the shortcut
> system when it comes to assigning buttons to ordinary actions.
> 

Anyone knows if it would be possible to domething like this:
Button "Music" is used to switch the remote to the music-mode. So when you 
press the "Play" button while the remote is in the "Music" mode, the shortcut 
system gets a key combination like "Music + Play". In other words, is it 
possible to specify additional modifier keys like Ctrl, Alt, Meta etc?

> In this case, actually, this something that I think the entire
> shortcut system could benefit from.  For instance I think multimedia
> keyboards would benefit a lot from this, as would joysticks in more
> complex games.  So this may be a case where I think, rather than it
> being something that should be kept unique to remote controls or
> something that should be done away with, it is something that should
> be incorporated into the shortcut system as a whole.
> 
> > - Another problem is the future of such an interface. With AVRCP version
> > 1.3 and greater, a remote control has the ability to query the
> > controlled target for the the current audio/video metadata, browse the
> > filesystem etc. (they might have displays nowadays :). While the current
> > Solid::Control::RemoteControl doesn't support this yet, the api coud be
> > extended. I guess there wouldn't be any reasonable way to introduce such
> > a feature into the shortcut system.
> 
> Yes, once again this is something that would require KRemoteControl to
> stick around.  I am not suggesting replacing KRemoteControl, I am
> saying KRemoteControl should hook into the shortcut system just for
> the purposes of making shortcuts.  It can keep doing anything else it
> needs to do on its own.
> 

Nope... Also KRemoteControl isn't useful for that. It would require 
Music/Video Players to register at the RemoteControl API so that a Bluetooth 
remote control can browse registered players, and select the target 
application by itself. Each registered application must be able to provide 
Metadata to the remote control system by itself.

> > In my opinion the shortcut system in its current state does not really
> > fit into the purpose of remote controls. For example you cannot use
> > button combinations on a remote control while it doesn't make really
> > sense to have a number of modes for a keyboard, just to get more virtual
> > buttons. However, lets see what the metalworkers think about this.
> 
> Why doesn't it make sense for a keyboard?  My multi-media keyboard
> already has a built-in mode switching ability (although it only works
> on some keys and only has two modes).  I think it would be very useful
> for a number of buttons on my device.  All in all the use cases for
> multimedia keys on a keyboard I would say would be fairly similar to
> the use-cases of remote controls, anything you would want to do with
> one you would probably also want to be able to do with the other.
> 

The main difference here is that your keyboard handles this mode switching in 
its hardware. The software just receives some different key presses. While for 
remote controls the mode switching stuff is done completely in software (As 
long as you haven't got a real multi-purpose-remote). This means that remote 
controls can have an unlimited number of user-defined modes, but the software 
receives always the same button name from the hardware, regardless of what 
mode it is currently in.

> 
> There is one detail I forgot to mention: devices would have to be able
> to deny certain button presses or button press combinations.  For
> instance KRemoteControl would have to be able to tell the shortcut
> system that it can't use the buttons used for mode switching.  The
> mouse backend would have to be able to say you can't use the left
> button, right button, middle button, or scroll wheel on their own (but
> it would allow, say the left button combined with one or more
> non-mouse buttons, such as left mouse+keyboard meta).  This shouldn't
> be too difficult because it should only be limited to blocking one or
> more of the device's own buttons in combination with or not in
> combination with the same device or a different device.  So the
> backend could just provide a list with some or all of these values:
> 
> Buttons that cannot be used for shortcuts: a,b,c
> Buttons that cannot be used for shortcuts unless combined with another
> button from any device: d,e,f
> Buttons that cannot be used for shortcuts unless combined with another
> button from this device: g,h,i
> Buttons that cannot be used for shortcuts unless combined with another
> button from this device type: j,k,l
> Buttons that cannot be used for shortcuts unless combined with another
> button from a different device: m,n,o
> Buttons that cannot be used for shortcuts unless combined with another
> button from a different device type: p,q,r
> 
> Or, alternatively, the shortcut system could send any proposed
> shortcuts to the backends that provide buttons used in that shortcut
> (along with data like source device and source device type for each
> button press in the shortcut) and the backends could all report
> whether that shortcut is acceptable.  That would allow for more
> complicated rules to be implemented inside the backends.
> 

I guess this would just be same as currently the modifier keys for keyboards. 
Try setting a shortcut by pressing just ctrl without any other button. The 
same could be done with Mouse buttons. Mode switches for remote controls are 
somewhat special here. They shouldn't just be passed up to KHotKeys but 
instead directly switch the remotes mode inside the backend. The next button 
press should be send up like there were two buttons pressed (ModeName + 
ButtonName).

I think we are getting somewhere but there are still many many issues and use-
cases to work out. Especially the Bluetooth AVRCP things produce some headache 
here... Also I think even with the mode-switching stuff beeing solved 
underneath the shortcut system, this approach is still way more limited to 
what it is now.
For example with KRemoteControl you can assign multiple actions to one button. 
The shortcut system doesn't allow this. KRemoteControl allows specifying what 
to do when the button is held down or you can launch the target application if 
not already running. A standard shortcut ends up in nowhere if the respective 
application is not running. With KRemoteControl you can send the action to all 
instances of an application while shortcuts always go to the one having focus 
etc...
As another example you can configure KRemoteControl like this:
We are in mode "Music" and amarok is running. If the user presses the "TV" 
button, amarok is closed, the remote is switched to the "TV" mode and a TV 
application is launched. All this just with one button press. Don't think this 
would be possible with the shortcut-system-solution.

Another approach I could think of would be some sort of special mode in 
KRemoteControl whitch doesn't send out D-Bus calls but generates key presses. 
This way, you could use KRemoteControl as usual but when switching the remote 
to the according mode all button presses are routed to the shortcut system. 
This even could be the default if the user doesn't set up any actions in 
KRemoteControl.

Cheers,
Michael

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-core-devel/attachments/20100319/185b9234/attachment.sig>