New project: Aktions

[Zak Jensen]

You bring up several interesting arguments that I have not encountered
yet. Thanks for doing so, because these are things that my group can
use to improve the presentation we have to give at the end of may :)

On 3/19/06, Ellen Reitmayr <ellen at kde.org> wrote:
>
> Hi Zak,
>
> let me first introduce myself to make my point of view more clear: I'm member
> of the KDE usability and HCI project, and working as a usability
> professional.

> While I agree that it's the dream of each usability expert to gain statistical
> data how users use their applications from the whole user base, I have big
> concerns using anonymous feedback mechanism like you described in the HTML
> document. This discussion is not new to me, so far it has come up in many OSS
> projects I've worked in. It is always an idea developed by developers, and
> IMHO an attempt to solve a *usability* problem technically.

Maybe collecting statistics overall is not a good idea, then. I can
see places where it could still be quite useful. For instance, one
could collect generic statistics from a wide audience, then look for
patterns which closely match limited test groups. Or, one could use it
to find patterns in the generic statistics, then attempt to figure out
why it occurs so much (possibly by giving questionaires out to limited
test groups).

The above, of course, is mostly my idealistic ramblings, and a
half-hearted defense of my project ;) Having read the next (<snip>'d)
part of your message, I see the need for limited test groups.

> From a bunch of anonymous data, you will get several high and low peaks, and
> sequences of actions. Without knowing about their tasks it's impossible to
> know if actions were not used because they were not part of the task they
> wanted to perform, or if they did not find them.

However, you could, for instance, use it to identify places where
functionality should be moved around. For instance, one of the initial
uses of this toolkit could be to identify "KActions" which are in the
wrong place... such as items on the toolbar that are seldom used.

> Taking Thomas' example of
> DnD in Konqui, you'll never know it the people don't do it because they've
> already learned it does not work, or if they don't use dnd.

You will be able to say that they don't use DnD, however. So, you can
then ask a selected group why not in a tailored survey.

> Even worse,
> information about the user is missing! How do you know if the people who sent
> in their data belong to your targetted user group? Maybe all your data comes
> from the developers themselves, how should you know??

The problem with any form of statistics is ensuring the group that you
are analyzing is both statistically significant and random. We are
well aware that, because Aktions features an opt-out, it will exclude
a portion of the audience. We have 2 refutations to this argument:
1) We assume that the people who exclude themselves from Aktions will
not skew the statistics... as they should be constituted of as random
a population as the rest of the audience.
2) If the assumption made in (1) is false, then those who opt out have
accepted the consequences of that decision--that the interface will
not necessarially improve towards the way they use their computer.

> And here we get to my second, even bigger concern: A software that is
> *potentially* able to call home, is not trustworth for many users. I very
> well remember when Microsoft XP was launched: The first thing everybody who
> got a new PC running XP was told by his peers was to switch off the 'call
> home' function. Still, people were unsure if it really was switched off. If a
> software has the power to observe you and send the data to an unkown
> recipient, a user cannot be sure if a simple button to switch off the
> function really works.

We did not approach this concern at all within our present
documentation, because it was out of the scope of those documents.
However, this is the predominant reason why we chose to make our
project open-source. Users can inspect the source code themselves if
they don't know if it is true to it's word. If they lack the
capabilities to do this themselves, then they can pay someone else to
do it instead.

We anticipate a large blowback from the open source community, and we
intend to use it to our advantage: the OSS community is very sensitive
to privacy concerns, and will audit the code for us. The fallout there
would be if we were doing any less than we reported would be
astronomical, and then our software would be worthless.

Of course, we plan to have a little bit of muscle to back the claim
up. Similarly to the Mozilla Project or the Linux Kernel, we were
planning on using a trademark to back up the authenticity of our code.
That trademark would be licensed in such a way that we could prevent
people from tampering with the code and saying it was Aktions.

> However, KTips is another thing. In applications where you know about
> weaknesses of the interface that can't be fixed by the one or the other
> reason, you might specifically support the user. If he uses some bad
> workarounds we know that are likely to occure, we can tell the user.

I was thinking it could be used to help customize the toolbars of KDE
as well. Imagine an applet that changes from a star to an exclamation
point (or something similar) maybe one or twice a day (at most). If
the user clicks on the applet at this point, an extender comes out
saying (maybe) "You seem to use Print Preview often. Would you like me
to add it to your toolbar?". Or something similar. The idea is to help
while being as unintrusive as possible. (It doesn't bounce, jiggle,
etc... and stays out of the users way... if they turn it off, it
doesn't pop back up, etc.)

> Here, the approach is the other way round: Instead of tracking the user and
> taking his data as a basis of our usability work, we do usability work first,
> identify the major pitfalls, and provide proper support in these situations.
> The difference is that these tips are *context-based*, we observe certain
> actions, assume a context of use, ask the user if it's the case and if so, we
> provide help. Fine-tuning will be a challenge, though :)

Hence why the applet has since been dropped from Aktions... we can't
implement it in the time we have left for the project.

> If you also like that idea of context-based tips, I'd be happy to support you
> to find out major pitfalls in a certain application. Kontact keeps coming to
> my mind... ;-)
>
> Cheers,
> /el

We appreaciate all the help we can get ;)

-Zak