Flake event handling changes - draft proposal

[Inge Wallin]

Hi Tomas,

It is obvious that you have really thought this through.  But although your 
technical solution seems solid, I think you are missing one aspect: how the 
user interaction should work.

The problem, as you say, right now is that the user can edit shapes but then 
has no easy way to get back to the default operation. The only way is to go to 
the toolbox and select the right tool, i.e. visual tab in the toolbox for the 
operation. This is very unituitive and we have actual bug reports that say "I 
can't get back to edit cells in Sheets after inserting a chart".

In Words this default operation is text editing and in sheets cell editing. In 
Stage, it would be the same as in Words, except the text boxes are actually 
movable and resizable so they are selectable. In Karbon and Flow everything 
you do is edit shapes, so there is no default operation other than deselecting 
all shapes.  (I could be wrong here, please inform me if I am.)

So, what can be done?  The full solution is probably complex, but here is what 
I think is the simplest solution for the user interaction:

The Simple Solution
----------------------------

 1. Single click in a shape selects that shape and activates the default shape 
tool where you can move, resize, scale and rotate the shape.
 2. Double click in a shape selects that shape and activates the shape specific 
tool. Examples would be the chart tool for chart shapes and path editing for 
path shapes.
 3. Click anywhere outside any shape when the default shape is active 
unselects all shapes and returns to normal editor mode, which is specific for 
each application.
 4. <Escape> key deactivates the shape specific tool if one is active and 
activates the default shape tool. (Some special tools may have internal states 
that use <escape> ending some user interaction. If so, then the user may have 
to press <escape> twice.)

Naturally, the current method of changing tool to change state should still 
work. Also, there are small variations, like ctrl-click to toggle selection of 
individual shapes, etc.

The good thing about this solution, as far as I can see, is that it would mean 
very small changes to flake and none to any shape. So this should be possible 
to get in before 2.9. We shouldn't be doing this now, but the current user 
interaction problems are so horrible that I think we should make an exception.  
And this is especially since 3.0 will probably be quite unstable and buggy and 
should be regarded more like a tech preview than a real release.

There is one thing we would lose with this solution, though: rubber band 
selection of shapes. If any click outside a shape leads automatically to 
deselecting all shapes and reverting to the default tool for each application 
then we can't use drag operations to select shapes.

So therefore I present:

The Intermediate Solution
------------------------------------

This is just like the simple solution, except that if a shape is selected 
already, and the click outside any shape is actually the start of a drag 
operation, then it turns on rubberband selection.

As it happens, this is exactly what LibreOffice does. It treats drags differently 
when a shape is selected and when it's not. So if we implement this, we also 
gain user action compatibility with LO. (We already have rubberband selection, 
but it is in our current clunky system, so this would be a slight change. We 
would not have to implement it from scratch, though).

I don't know how difficult this would be to implement but I suspect that since 
it's already there it will be quite possible to achieve.

The full solution, would be something like Tomas proposal, but I am unsure if 
we gain much from that compared to the intermediate solution here.

-----

What I have not done is to analyze how the technical implementation of this 
should be done. But from my current understanding of Flake it seems to me that 
it should not be extremely difficult. It is far from impossible that I am 
missing something important, though.

But in any discussion that we have, I think we should always start from the 
point of view of the user interaction.

	-Inge


On Monday, February 02, 2015 22:50:37 Tomas Mecir wrote:
> As discussed on irc earlier today, here's my initial proposal to improve
> event handling in flake tools. Very much a work in progress.
> 
> Problem:
> The current method of handling flake tools feels very rigid and inflexible,
> which gives a rather poor impression from the usability standpoint. I've
> been going over the related code during my yesterday's attempts to improve
> the related functionality somewhat in Sheets, only to come to a realisation
> that doing so will require a series of improvements in flake itself.
> 
> This document contains the initial draft of my proposal to do exactly that.
> It is obviously meant to be introduced after the 3.0 release, as a change
> as significant as this cannot be merged this close to the 2.9 release.
> 
> Usability is the main motivation here, no actual new functionality will be
> added (initially, at least, as I believe the changes will allow new
> functionality to emerge). As I note, currently the shape tools are fairly
> isolated from each other, and the result feels very clumsy. For example if
> the user is editing a text document and wants to resize a picture, he needs
> to find the shape manipulation tool in the panel, resize, then reselect the
> former text manipulation tool. With the proposed changes, he will be able
> to simply move the mouse over the picture (and maybe click on it, though I
> am a fan of the hover effect here), resize it, then get back to text
> editing right away. Many more similar use cases can be devised.
> 
> From the technical standpoint, the main change is that events shall be
> processed by all applicable tools, rather than only the active tool. The
> tool manager will serve as the arbiter, deciding which tool actually
> receives the event based on information that the tools provide. The concept
> of the active tool remains, but becomes much less significant, as users
> will be able to use functionality offered by these tools even if they are
> not currently active (this mostly applies to the shape manipulation too,
> which I would ideally like to make entirely transparent, if possible).
> 
> Technical list of changes - some of these may need alterations:
> - We construct a priority list of tools belonging to each shape - some
> tools are generic, most are specific to a single shape (this mostly already
> exists, though some priorities will change)
> - Event processing gets split into two stages
> - Stage 1 - preprocessing:
>   * here we query all the tool/shape combinations that are at the affected
> (clicked, whatever) position, topmost shape first; for keypress events, we
> query the active tool (TODO - maybe more than that?)
>   * each tool determines whether it wants to process the event, and its
> priority in doing so
>   * the tool will mainly decide this based on whether it is active, most
> tools will just be idle while inactive
> - Stage 2 - actual event processing:
>   * the tool with the highest priority receives the actual event
>   * if the tool returns that it did not process the event after all, the
> next tool receives it, etc (this may not even be necessary, but it may
> simplify the tool logic)
>   * upon processing the event, the tool can return one of these return
> codes: "not processed", "processed", "processed and make this tool the
> active one"
>   * these return code options are one of the most important parts of this
> proposal - they will allow us to switch between tools much more fluidly
> than currently, and it will also allow some tools to process events without
> actually becoming active; for example, we could allow the manipulation tool
> to process shape resizing attempts, while still keeping the editing tool
> active
> - Additionally, there will be a stack of "default" tools, which will be
> processed if the user clicks in the area where no shape is located (this is
> mainly for Sheets, as its main canvas is not a shape, but other apps may
> benefit too)
> - Furthermore, some events will be partially processed by the tool manager
> itself - this will mainly affect mouse dragging; each tool currently
> processes this itself, which will no longer be feasible under the new
> rules. As such, the mouse movement event will also include the initial
> point of the drag action - this will allow us, for example, to have click
> events routed to one tool, while allowing 'click and drag' functionality to
> be processed by another (e.g. situations where clicking would be processed
> by the editing tool, while dragging would cause the shape to be moved).
> - To be determined: need to decide if we want to keep the "currently active
> shape" concept at the tool level (ie, each tool has its own), or move it to
> the shape/tool manager level; I'm thinking the latter. Overall I'd like to
> de-emphasise the "currently active shape" concept as much as possible,
> letting users e.g. fluidly work on multiple shapes at once, but this may
> not be feasible, mostly because of keyboard shortcuts.
> 
> This is just the initial draft, so please point out anything obvious that I
> may be missing.
> 
> I also volunteer to get all this done and working :)
> 
> Thoughts?
> 
> / Tomas