testing UIs and improving a11y all at once!
Arjen Hiemstra
ahiemstra at heimr.nl
Tue Aug 16 11:30:17 BST 2022
On Wednesday, 10 August 2022 12:29:14 CEST Harald Sitter wrote:
> Servus,
>
> A while ago I prototyped a "new" approach to UI testing and I'm
> wondering if there's general interest in doing more Plasma testing
> using it. I'm able to invest time in polishing the experience for us.
>
> Very rough prototype:
> https://invent.kde.org/sitter/selenium-webdriver-at-spi
>
> The testing is run through the accessibility API we have on Linux
> https://www.freedesktop.org/wiki/Accessibility/AT-SPI2/ as well as the
> testing API selenium https://www.selenium.dev/ respectively the more
> specific appium https://appium.io/
>
> Architecturally appium is an extension to selenium and selenium is a
> client-server system where the client is the test and the server is a
> so called webdriver. Webdriver is a standardized well-defined API of
> various UI interactions https://www.w3.org/TR/webdriver/ and we'd
> implement one based on the a11y APIs (the feature sets match fairly
> well).
>
> Since selenium has wide spread use across the industry we get to use
> excellent tooling on the client without any extra work from us. And
> because it is so wide spread the stuff is generally very well
> maintained. All we need to maintain is the webdriver that interacts
> with the a11y API.
>
> The way this type of testing works is by UI interaction and state
> validation. There is a kcalc test available in the prototype repo [1]
> - the test operates the various UI elements to perform a calculation
> and then checks that the output UI element contains the expected
> value.
>
> A simple plasma test might open kickoff, and launch one of the
> favorites, then validate that indeed a new window has opened.
>
> Since all this is driven by the a11y API there is the additional
> advantage of making us notice a11y problems and deal with them,
> resulting in bettery a11y support in the long run. Two birds, one
> stone!
>
> What do you reckon?
One of my conclusions from spending time looking at behaviour/integration
testing was that you don't want some tool that uses screenshots for the tests,
as they become a maintenance nightmare. I also thought about using the
accessibility APIs instead, which provide a way to get at the actual behaviour
without relying too much on stuff like positions. It's great to see that same
idea proposed here.
It's also nice to be able to plug this into existing tools that are already
widely used for this sort of thing, it makes it a lot simpler to reuse other
things that already talk to Selenium. For example, I quite like behave[1],
which allows writing tests in more of a natural language format and which is
pretty easy to glue to Selenium.
Have you thought about how to integrate this into our projects? Ideally I
suppose we would have the tests as part of each project's repo, but that might
not work for things like Plasma, where running things depend on a lot of other
dependencies. On the other hand having some sort of "plasma-tests" repo is
much more likely to lead to unmaintained tests. But maybe we can tie things
together with Gitlab pipeline triggers?
- Arjen
>
> [1]
> https://invent.kde.org/sitter/selenium-webdriver-at-spi/-/blob/91486b50995a
> de23c6b03a54f77347c263c2f03a/calculatortest.py
>
> HS
More information about the Plasma-devel
mailing list