[Kde-accessibility] Questions regarding AT-SPI and ATK

[David Bolter]

Peter Korn wrote:

>Hi Gunnar,
>
>  
>
>>The KDE Accessibility Project is currently elaborating on what is the best 
>>way to add AT-SPI support to Qt and KDE. Clearly this will be done by 
>>using the Qt Accessibility Architecture (which currently gets extended by 
>>Trolltech). However, there are a number of questions that IMO need to get
>>solved:
>>
>>1. What are the exact events that are used in AT-SPI and ATK?
>>2. Is there a possibility to determine whether there is an assistive
>>technology?
>>3. Is there a possibility for applications to do mouse (and keyboard) 
>>grabs without stealing the AT applications mouse clicks?
>>4. As far as I have heard AT-SPI supports custom interfaces if an 
>>application wants to provide information that is not handled within the
>>AT-SPI standard.
>>How exactly does this work?
>>5. What are the exact dependencies of the required party of the AT-SPI
>>architecture?
>>
>>I will give some more details (and some background) for these questions:
>>
>>1. In order to extend their accessibility framework, the Trolltech 
>>developers need to know which events are used in AT-SPI and ATK and which 
>>details are used to describe these events. As far as I looked there is no 
>>documentation about the event types in AT-SPI. In ATK the documentation 
>>for the events is spread over all interfaces. So far I found the
>>following events:
>>
>>Key press, Key release, Children change, Focus change, Property change,
>>State change, Visible data change, Selection changed, Column deleted,
>>Column inserted, Column reordered, Model changed, Row deleted,
>>Row inserted, Row reordered, Text attributes changed, Text caret moved,
>>Text changed, Text selection changed
>>
>>Is this list complete? Which information describing those events is 
>>included in these events?
>>    
>>
>
>I will leave it to BAUM and UToronto (and the Dasher folks) to detail which
>of these events they use.  By far the most used are:
>
> focus change
> property change
> state change
> selection change
> text caret moved
> text changed
> text selection changed
>

Oops,

I just noticed this message (doing some inbox dusting). Gunnar, are you 
still interested in the events gok currently pays attention to?

cheers,

David

>
>There should also be a text inserted event.  Visible data change and text
>attribute changed are also used a fair amount, but I think to a lesser
>extent.
>
>  
>
>>2. In some cases preparing the events can be expensive. Therefore the Qt
>>Accessibility Architecture contains a method that returns true if there 
>>are AT clients, and false if there are no AT clients. Clearly the easiest
>>implementation for this method when bridging to AT-SPI is to always return
>>true if the application got registered with the broker. On the other hand 
>>it might be worth to do a more sophisticated implementation. For that we 
>>need to be able to ask the broker whether there are AT clients. In that 
>>case we also have to make sure that it is save to turn the sending of the 
>>events off when there is no AT client.
>>    
>>
>
>In GNOME there is a per-user setting for whether accessibility API support
>should be on or off, and both Gnopernicus and GOK will warn the user if the
>flag is turned off when the they are launched.  This is stored in the user's
>gconf database.  In StarOffice/OpenOffice.org (for example), if the setting
>is on then a Java VM is invoked to be (part of) the bridge.  You might use
>that idiom.
>
>  
>
>>3. One of the KDE developers wrote a prove-of-concept application that 
>>uses the current Qt Accessibility Architecture. This application is called
>>Klaviatura and serves for two purposes: It can be used to remotely control
>>the menu bars, tool bars and popup menus of KDE applications, and it 
>>contains an on-screen keyboard. Technically Klaviatura consists of a 
>>plug-in for Qt and an external application. Both parts communicate with
>>each other via DCOP.
>>
>>When writing this application the developer found that applications often 
>>make mouse grabs when showing pop up menus. If then a mouse click was 
>>intended to go to an assistive technology it will be used by the wrong
>>application.
>>
>>In Klaviatura this problem is solved as follows:
>>The plug-in intercepts the event handling in order to be able to process 
>>mouse clicks before the application processes them. If the mouse click was 
>>directed to a window of Klaviatura that mouse click is intercepted and 
>>sent to the Klaviatura application (via DCOP). Otherwise the mouse click 
>>is not intercepted (and therefore will be processed by the application).
>>
>>There are multiple possibilities for solving this issue in AT-SPI:
>>- - Avoid popup menus (and other things that require mouse grabs). This is 
>>not a good solution as it interferes with the current user interface 
>>design of many applications. Not showing a popup menu means either less 
>>functionality or (in the case that you need an assistive technology to 
>>activate an entry in the popup menu) less usability.
>>- - Do the popup menus without mouse grabs. This is not a good solution as 
>>pop up menus need mouse grabs in order to work properly.
>>- -Send all mouse clicks to the ATs first and ask for a permission to 
>>process them.
>>- -Send those mouse clicks that are not directed to an application window 
>>to the ATs and ask for a permission to process them.
>>
>>The last two possibilities sound acceptable to me. I am not sure which is 
>>the better one. Many assistive technologies might not be interested in 
>>processing all mouse clicks, but there might be some few ATs that need top 
>>process them all.
>>
>>Solving this problem will be important as some KDE developers will 
>>otherwise have objections against AT-SPI.
>>    
>>
>
>There is yet another approach, which is being pursued by Sun in
>communication with folks in X.org and the XFree86 community - a new X
>extension that provides all keystokes and mouse events just after
>XKB/AccessX and just before anything else.  An AT tool would register for
>the keystrokes it cares about, and either listen and let them contine on, or
>consume them.  The problem with this approach is that it will take time for
>the X extension to be adopted and deployed.
>
>  
>
>>4. Some KDE developers have concerns that AT-SPI is not powerfull enough. 
>>In order to convince them that we will be able to extend the protocol if
>>necessary we need support for custom interfaces.
>>    
>>
>
>AT-SPI is explicitly designed to be extensible.  That's why we have
>sub-interfaces for things like text, tables, etc.  In the Java predecessor
>to AT-SPI, we evolved the interface several times (tables & relationships
>are two notable examples of things that weren't in the first release).
>
>Are there specific questions about how to evolve AT-SPI?
>
>  
>
>>5. In order to get an acceptable solution for KDE we have to evaluate the
>>dependencies of those parts of the GNOME Accessibility Architecture that 
>>will be used within KDE. When looking at ATK and the broker I fear that 
>>both will most likely have too much dependencies. However, as we currently 
>>do not have a clear position of KDE to AT-SPI, so I cannot say which 
>>dependencies will be OK and which have to be changed.
>>
>>If what I heard is correct the ATK dependencies are due a dependency on 
>>GAIL which was introduced in order to force the user to have a working
>>architecture once ATK is installed. This dependency on GAIL pulls a
>>dependency on GNOME into ATK and should be avoided if we decide to use ATK 
>>in KDE.
>>    
>>
>
>Hmmm... Perhaps Padraig or someone else can speak to this, but I didn't
>think that ATK depends upgon GAIL.  The reverse is true.
>
>  
>
>>The broker currently requires GTK+2. From a KDE perspective this 
>>dependency is too much. When looking from outside on the broker one would 
>>think that dependencies to glib and ORBIT2 would be enough.
>>
>>Also the CORBA dependency might turn out to become a problem for many KDE
>>developers.
>>
>>In either case I hope that we will find a clear position of KDE on the KDE
>>Developers Conference at the end of August.
>>    
>>
>
>
>Regards,
>
>Peter Korn
>Sun Accessibility team
>_______________________________________________
>gnome-accessibility-list mailing list
>gnome-accessibility-list at gnome.org
>http://mail.gnome.org/mailman/listinfo/gnome-accessibility-list
>
>  
>