New project: Aktions

Sun Mar 19 14:56:18 CET 2006

Hi Zak,

let me first introduce myself to make my point of view more clear: I'm member 
of the KDE usability and HCI project, and working as a usability 
professional.

While I agree that it's the dream of each usability expert to gain statistical 
data how users use their applications from the whole user base, I have big 
concerns using anonymous feedback mechanism like you described in the HTML 
document. This discussion is not new to me, so far it has come up in many OSS 
projects I've worked in. It is always an idea developed by developers, and 
IMHO an attempt to solve a *usability* problem technically. 

The reason why we do usability testing and user observation in controlled 
environments (either in a lab or sitting next to a user in his/her actual 
work environment and just watching them) is that it provides the ability to 
collect *context information*. Without knowing the context of a certain use 
situation, you are not able to evaluate if the actions a user performs are 
appropriate or not. Without knowing the goals a user has, you can not say if 
the chosen action was helpful to fulfill the goal. Even worse, you don't know 
when or if a goal was achieved at all. We do usability testing instead of 
collecting anonymous data because we need to *ask* for such goals and the 
user's background, only then we can explain a user's behaviour.

From a bunch of anonymous data, you will get several high and low peaks, and 
sequences of actions. Without knowing about their tasks it's impossible to 
know if actions were not used because they were not part of the task they 
wanted to perform, or if they did not find them. Taking Thomas' example of 
DnD in Konqui, you'll never know it the people don't do it because they've 
already learned it does not work, or if they don't use dnd. Even worse, 
information about the user is missing! How do you know if the people who sent 
in their data belong to your targetted user group? Maybe all your data comes 
from the developers themselves, how should you know?? Even if Microsoft used 
these mechanisms successfully, the method does not convince me. Of course 
you'll get some basic data, but after all, they also had to supplemented it 
by extensive user testing and observation of users in their work environment 
to gain a usable application.

And here we get to my second, even bigger concern: A software that is 
*potentially* able to call home, is not trustworth for many users. I very 
well remember when Microsoft XP was launched: The first thing everybody who 
got a new PC running XP was told by his peers was to switch off the 'call 
home' function. Still, people were unsure if it really was switched off. If a 
software has the power to observe you and send the data to an unkown 
recipient, a user cannot be sure if a simple button to switch off the 
function really works. 

You may argue that an additional software that needs to be installed manually 
might be a solution - but then I wonder how we should address all those 
non-power users whose data is of most interest for KDE?

I'm sorry to be so harsh, but I've heard that solution quite often. In the 
first run I somehow liked the idea to get a big bunch of statistical data. 
But the benefits (which are less than you might first expect due to the 
missing contextual information) do not outweigh the costs, namely loss in the 
user's trust for KDE. When it comes to sending usage data back home, there is 
a clear answer from my side: Don't!

However, KTips is another thing. In applications where you know about 
weaknesses of the interface that can't be fixed by the one or the other 
reason, you might specifically support the user. If he uses some bad 
workarounds we know that are likely to occure, we can tell the user. 

Here, the approach is the other way round: Instead of tracking the user and 
taking his data as a basis of our usability work, we do usability work first, 
identify the major pitfalls, and provide proper support in these situations. 
The difference is that these tips are *context-based*, we observe certain 
actions, assume a context of use, ask the user if it's the case and if so, we 
provide help. Fine-tuning will be a challenge, though :)

If you also like that idea of context-based tips, I'd be happy to support you 
to find out major pitfalls in a certain application. Kontact keeps coming to 
my mind... ;-) 

Cheers,
/el

On Tuesday 14 March 2006 23:15, Zak Jensen wrote:
> Note: CC'ed the kde-quality and kde-usability mailing lists.
>
> I guess I should first introduce myself, for those who do not know me.
> I am Zak Jensen, a senior at Drexel University, and a long time
> supporter of open source, the KDE project in particular. Some of you
> may know me from kdedevelopers.org (where I have responded to some of
> your blog posts), the kde-usability mailing list, or kde-artists. I
> have been lurking on the kde-devel, kde-core-devel, and kde-quality
> mailing lists for about 2 months now.
>
> About 4 months ago, I proposed a replacement for the KTips dialog: a
> mix between non-intrusive version of clippy, a plasmoid, and the
> present KTips functionality.
>
> http://lists.kde.org/?l=kde-usability&m=112609922114054&w=2
> http://lists.kde.org/?l=kde-usability&m=112611125600245&w=2
> http://lists.kde.org/?l=kde-usability&m=112611417629919&w=2
> http://lists.kde.org/?l=kde-usability&m=112670485931144&w=2
>
> And have also talked about it a bit on the kde-artists forums.
> Unfortunately, the kollaboration forums have been relocated, and I
> cannot find the original post at this time.
>
> In any case, the idea has mutated from being an improvement of KTips
> to a generic "action monitoring" service. The basic idea is to collect
> abstract usage information from users, and analyze it. That analysis
> could be used to target interfaces which should be investigated. The
> information can also be used to pinpoint seldom used interfaces or
> features.
>
> It is important to mention at this point that we are providing an
> abstraction to prevent the perception of our software as malignant
> software. One way that we are doing this is by limiting the type of
> information which the system will accept. We are limiting the input
> information (in future versions) to abstractions like "typing",
> "button click", and "toggle checkbox".
>
> At the moment, we are finding out how the mechanism will work, or if
> it will work at all. Future versions of our software will be built
> with a proper outlook on security and privacy.
>
> My senior design team wishes to integrate with an already existing
> project, and I recommended KDE to them. To those ends, I was wondering
> how interested the community is in the project. We are open to ideas
> for features, feasibility, and recommendations on how to integrate
> with KDE. We are not looking to be incorporated into the project at
> this time. Rather, we require an audience to help us tailor and focus
> our project, and we hope that there will be a few interested projects
> within KDE.
>
> Another portion of our project is defining a generic model for
> graphical user interfaces which can be applied across several
> platforms. If anyone knows of such a model which already exists
> (excepting that defined by a set of libraries) details for that would
> be greatly appreciated.
>
> Attached are 4 documents (in the tar file) providing more detail for
> our project. The best viewing of the html files occurs within
> Konqueror, but they render alright within FireFox 1.5 as well.
>
> Please forward any questions back to this thread or, if I  have posted
> to a particular mailing list in err of its guidelines, to me
> personally.
>
> Thanks,
>
> Zak

-- 

Ellen Reitmayr
KDE Usability Project
usability.kde.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.kde.org/pipermail/kde-quality/attachments/20060320/1c8d2e6b/attachment.pgp 