[Kde-accessibility] Fwd: KDE Accessibility

[Peter Korn]

Hi Volker, guys,

Daniel forwarded your e-mail to the kde-accessibility list.  I'm one of the
folks at Sun involved in the GNOME Accessibility project.  Prior to that, I
was one of the two folks at Sun who created and implemented the Java
Accessibility architecture.  Prior to that, I was with Berkeley Access who
made the first graphical screen reader and first graphical screen magnifier
for any platform (the Macintosh at that time), and did a lot of the work on
our Windows screen reader.  I also gave Microsoft a lot of feedback on MSAA
(Microsoft Active Accessibility) while at Berkeley Access.

In your e-mail to Daniel, forwarded to kde-accessibility, you wrote:

> ...
>
> One of the issues mentioned that the Qt API for accessibility support
> was not versatile enough to be used by KDE, and that the implementation
> we have would need to be extended (and implemented on UNIX). Well, here
> is some infrastructure overview (from a Windows perspective), since the
> documentation of the Qt stuff is rather thin this might give some
> insight:
> 
> On Windows, an application is queried by accessibility clients (like a
> magnifier or screen reader) for the accessibility interface (COM
> interface IAccessible) for a specific object. This is done via a window
> message WM_GETOBJECT to a window handle.
> 
> ...
> 
> - Additionally to that, Active Accessibility provides a notification
> mechanism that allows applications to tell accessibility clients that
> something important has changed or happend (e.g. a checkbox changed it's
> state). The static QAccessible::updateAccessibility() is provided for
> that, and a large set of "event types" (e.g. ValueChanged) is defined in
> an enum (the enum values are identical to the respective values in
> Windows, but can easily be mapped to something else).
> 
> Since Active Accessibility has been around for a while and is the
> standard and obviously approved infrastructure around on a lot of
> Windows desktops I would assume that the flexibility it offers is
> sufficient (so the interface we have in Qt is sufficient - it is even
> more flexible). If it is not sufficient it is trivial to add another
> interface and implement that as well in the plugins (e.g. right now we
> only have one default action, and can execute that action, but we could
> implement an interface that returns the list of public slots (without
> parameters), and that executes any of those slots etc).

Unfortunately your assumption about MSAA - "that the flexibility it offers
is sufficient" is not true.  MSAA was developed in a climate where there
were already ~13 screen readers and ~10 screen magnifiers for Windows 3.1/95
shipping.  These products used a variety of hacks, patches, and driver
replacements to obtain the contents of the screen render it to users with
little or no sight.  Specifically screen readers would either patch GDI
(what ours did) or patch/replace the video driver (what most others did) in
order to filter all text rendering and other graphics calls, and build an
Off-Screen-Model (OSM) of the text contents of the screen.  As the user
tabbed through the UI (or used a "review mode" of the screen reader), the
screen reader would walk through its OSM and send the text it found there to
the speech synthesizer or Braille display.  

While a big mess, it was a system that was working (albiet poorly).  But the
biggest problems screen access vendors were having at the time was with
working with Microsoft Office.  Office used non-standard UI elements, and
the patching/hackery was having a lot more trouble with Office.  It was in
this environment that Microsoft developed Active Accessibility.

MSAA did NOT remove the need to patch the rendering sub-system and build an
Off-Screen-Model.  It was assumed that the text content region of all
applications would be captured by existing techniques and retrieved from the
OSM.  While a number of people from the disability community, including
several assistive technology vendors (yours truly) argued forcefully for a
complete API that would do away with the need for OSMs, those voices did not
prevail.

Today MSAA is widely seen by the community as a poor and insufficient
solution to the problem, and Microsoft is rumored to be working on a
replacement.  It's also worth noting that the more sophisticated screen
readers make only limited use of MSAA (e.g. JAWS for Windows), and further
make heavy use of application-specific COM APIs for improved access to
specific applications (MS Word, Excel, and IE just to name a few).


Specifically, MSAA lacks detailed action information (as you cited), as well
as text and table support - both crucial for access on UNIX desktops (or
anywhere where you don't already have an OSM approach in place).  Further,
Sun and the GNOME community have developed a number of additional
accessibility interfaces that are proving to be very useful, such as the
selection interface, and the new relation interface which is being used
fairly extensively by OpenOffice as well as the open source Gnopernicus
screen reader.

> ...
>
> Well, please fwd. this to the respective lists/people. I would of course
> like to know more details about what parts seem to be insufficient or
> not flexible enough, so that we can add some fu in Qt. At least the
> implementations we provide for all the Qt widgets should be reusable.

You may want to take a close look at the Mozilla accessibility effort.  They
originally had a small staff working on MSAA support in their underlying
widget set (nsiAccessible).  Additional staff joined the effort and expanded
the nsiAccessible object through a series of sub-interfaces which are then
bridged to the GNOME ATK and thence AT-SPI interfaces.  We now have early
speech and Braille access to web pages via Gnopernicus and Mozilla on
GNU/Linux and Solaris thanks to this work.  Had we limited ourselves to the
original nsiAccessible work that was needed to support MSAA in Windows, this
would not have been possible.


Regards,

Peter Korn
Sun Accessibility team