[dot] Pete Gordon on Portable Usability Labs

[Dot Stories]

URL: http://dot.kde.org/1097763942/

From: Frans Englich <frans.englich |a| telia | com>
Dept: portable-interview
Date: Thursday14/Oct/2004, @16:25

Pete Gordon on Portable Usability Labs
======================================

   What is a portable usability laboratory [http://www.usersfirst.com/]?
And could KDE development make use of one? Pete Gordon, an engineer
behind such a one, shares his thoughts in this interview on exactly that
topic.

     (Frans Englich): Pete, feel free to introduce yourself.

     (Pete Gordon): Thanks, Frans for having me discuss Portable
Usability it is definitely a passion of mine.

     Just a little bit about how I got into this. I like most of your
readers am a coder; I was thrown into customer interactions when a 120
employee company I was working for went down to six employees of which I
was one. The insight that I gained from working with the customers on a
regular basis was amazing up to that point I was in the Software
Engineering group and we had a couple dozen that pounded out code.

     Immediately, I saw from the customers so much that was missing in
the functional specification documents, and design documents that were
passed into the Software Engineering group and developed from. So much,
that we had spent months working on that did not meet the customers
needs and that they were not using, and so much that could be done for
little or no effort to meet those same customers needs.

     At first reading this you may think that the company was slow
moving and didn't have a good process in place. Where in fact, we were
practicing XP for the last year or so of our large existence and were
focused on the "Voice of the Customer" our representative of the
customer. I don't think the "Voice of the customer" was necessarily the
problem, I think the problem was far greater than that.

     But, the fact remains that when developers can talk to, interact
with, and see real people using their software it is enlightening. And,
that is what lead me to this. I wanted to come up with a way to
communicate the user experience throughout the organization (all the way
to developers, sales people, everyone). That's the goal of UsersFirst
and VisualMark.

     What is a portable usability lab, and in what way does it aid
usability development?

     Portable Usability Labs have been around for a while, mostly made
up of large suitcases to transport analog video/audio equipment to
remote locations to set up a temporary observation room. Here's some
links:

     http://www.normwilcox.com/lab-in-a-box.htm
http://www.userworks.com/default.asp?page=products&sub=box
http://www.usabilitysystems.com/prod_usability_portlab.html
 [http://www.usabilitysystems.com/prod_usability_portlab.html]
     This equipment was very specific, developed and sold only for the
purpose of observation setups. The idea that someone can go into the
field (into someone's home or wherever) to watch interactions is not
new. The most famous comes from anthropology. Watching and Observing
Chimps in the wild like Jane Goodall. The art of watching and learning
from peoples interactions soon followed in Product Design look at
companies like IDEO, Fitch, Frog Design which focus on Product
Innovation and Design. You really innovate and get creative thoughts and
understanding from stepping out from where you are and watching and
learning from others.

     It is not surprising with the PC revolution of the 80's that a need
to watch others interacting with PCs was needed. That said, we have now
moved into a point where watching people is secondary to usability, I
believe. The most important thing is communicating those interactions to
all involved with a product, to then come up with creative solutions to
most accurately meet the users needs. This is a little bit of a
different perspective, but similar to Joel On Software's recent comments
[http://www.joelonsoftware.com/articles/NotJustUsability.html] on Social
Interface Design. Bottom line, it aids Communication, allowing you to
communicate what you have observed to others involved.

     But is it at all necessary? Isn't it enough to test it by oneself,
and make sure any Human Interface Guidelines are followed? Why is user
testing important?

     The answer to this lies in the huge benefit and huge disappointment
to free (open source) software. The huge benefit, when developers
develop for themselves they already have a deep understanding of the
user needs, and the software is amazing in meeting those needs.

     Look at Linux, MySQL, PHP, Perl, and so much more (I personally
love Mac OS X Desktop Manager--Richard Wareham you are amazing!) these
are great examples of where developers develop for themselves, and the
community at large benefits amazingly.

     The other side is when developers develop for themselves and it
only meets a small portion of the user needs; or even worse the code
(solution or whatever you want to call the product) goes in the wrong
direction from meeting the primary users needs in favor of the few. This
is where some software dies on the vine, where other software can push
through it just fine, because people want it bad enough they will put up
with some inconvenience and difficulty or lack of it meeting the need
perfectly.

     The key I believe is striking the balance. The balance between the
users primary goals and the solution to meet those goals. The way that
this balance is reached is by understanding your users, whether it be
yourself alone, or the world.

     So, to answer your question. Is it all necessary? Depends. If your
the only user and you will always be the only user and your writing it
yourself--no it is not at all necessary to have others test it. But, if
you are to have others use it and you want others to use it, make sure
that you understand what it is those others want and don't go it alone.
That is why user testing is important to get a clear understanding of
user needs.

     One portable usability laboratory is for example Morae
[http://www.morae.com/] or Spectator
[http://www.usability.biobserve.com/], but another is the one your
company produces: Visual Mark [http://www.usersfirst.com/]. What
features does it have?

     Great. Here's where I can get into the corporate BS, thanks. The
biggest difference right now, is you can download mine and try it out.
Please do! I love to get feedback and if you think it would work on a
project drop me a line. The product is in beta and I want more and more
people to use it.

     Other differences include the focus on communicating the
experience. The other solutions spit out multiple files that you can
edit together or use their proprietary viewer to give to others. I went
with Quicktime because I can get everything video,audio, screen all into
one Quicktime Movie (still maintaining separate tracks for editing,
overlays, etc.) but simplifying the output, thus making it easier to
communicate it to others. No matter what lab you look at, the files are
huge, capturing pixel perfect screen as video is costly in terms of
bytes, so having it all neatly available as a Quicktime Movie that can
be burned simply to DVD is a big advantage.

     Another big benefit is the multi-platform nature of my solution. I
have based the VisualMark Lab on a VNC Client, so it does the screen
capture using RFB (Rectangle Frame Buffer) coming from any VNC Server.
Although, it may get replaced eventually because of performance, we'll
see what the VNC 4.0 stuff is like.

     The system is pretty simple in its usage, turn on your inputs
(screen,audio,video) log some observation notes and time stamps, close
your session, and check out your HTML/Quicktime, maybe make some edits
changes at that point, then burn it to DVD or put it on the network were
your entire team can view it. I really want it to be simple, and the
focus again is to communicate the user experience.

     What is the implementation details? For example, what is the data
format of the result, and how is the screen capture handled?

     Well, I guess I just answered this mostly.

     The output is MPEG4 for the videos, I recommend IMA4:1 for the
audio inputs (but it can be anything in the QT SoundManager), and the
Screen is captured as Animation Codec. Again, these all get thrown into
the same multi-track Quicktime Movie file, so the output is simple, big
advantage of quicktime there.

     The video inputs come from IIDC webcams (like iSight) and the
screen comes over the network as RFB (Rectangle Frame Buffer). For
Windows previewing/capturing I have built a Rendezvous wrapper
application around UltraVNC which allows you to execute but not install
UltraVNC Server off of CD, and it is dynamically located by the
VisualMark App on the Mac, so there is no need to even know the IP
address of the subjects computer.

     This is cool, I would like to have it on Linux and Mac also (right
now you have to code the IP address in the lab to do those), but there
are bigger issues related to performance and mouse movements right now
capturing those platforms. Stay tuned, or contacting me about getting
involved.

     What is the license terms for Visual Mark? It is closed source?

     Yes for the time being. : ) But, if people are interested in
helping out, and eventually open sourcing it I would love to talk to
them about it. I could use some help especially with the low level
performance stuff. It is written in Objective-C and C.

     Often mentioned but often forgotten is to develop user oriented
software, in contrast to only designing for the developers themselves.
In what way does Visual Mark put the usability engineer in focus, and
make the usage easy and simple?

     Much of what I have already said. Single file output for everyone
to see and use is definitely easy and simple. HTML and Quicktime are
ideal. I think it really empowers "usability engineers" to grab a lot of
information quickly, that's why it has multiple camera inputs in
addition to the screen capture. If you wanted to just setup multiple
video cameras around a focus group, you could get a lot of information
about users and what they thought of a conceptual product or prototype
really fast, and communicate that information really quickly to others;
even if it was just a paper prototype or something.

     The key is in developing user oriented software. When we are
developing software for others besides ourselves, we need to understand
others. And, the Usability Engineer or the developer that has a free
couple hours and can meet with users, can capture that user experience
and communicate it back to others.

     Now maybe Usability Professionals will frown on me saying
developers can do it, but I can't help to say they can do it--I am a
developer and I do it--a lot. Granted some people are more suited to do
it than others.

     You need to be able to listen more than talk, you need to watch
carefully rather than wanting to lead and teach, and you need to be
empathetic to the user. But, if you can do those things why can't you
capture the user experience and communicate it to the team. And, all the
better if you can do it in a way that they can watch it for themselves,
and begin to think creatively also about the solution to the users
problems, that's were a lab is useful.

     What is the typical development mistakes a usability lab session
reveals?

     Wow, tough one to answer. I think more than anything here we are
talking about qualitative analysis. Where there is a misunderstanding
between what the developer has developed or the concepts the developer
used and the users goals? Often times the developer develops because
he/she can, and users just want to get a goal done. These sometimes line
up and sometimes they don't.

     I have seen screens extremely cluttered with functionality, when
the users primary goals could be boiled down to just a few interaction
points. I would say the biggest mistakes in smaller developer led
projects are building too much into a product without understanding what
the user really wants to do (their primary goals). This comes back to
things like Alan Cooper ("The Inmates are Running the Asylum") and
User-Centered Design of software. Don't build it without understanding
what they want.

     Do you think open source projects can make use of such an usability
tool, considering their extensive use of online communication?

     I sure do, think they can! I would love to see my system grow into
a more robust system to help with this need even more than it does. I
want things like remote real-time streaming of the information, and
chatting among observers, etc. But that is a bigger application, and
more robust in its feature set. And, I must stay focused on the beta and
performance issues to get to general release before getting into more
features.

     Sorry, back to your question. Definitely. If anybody wants to use
my tool for open source let me know, I want to help where I can. It is
really powerful to see people/users interacting with your software it
gives you a sense of really meeting peoples needs and allows you to tap
into those needs really clearly. It allows developers to be empathetic,
really taking on the users concerns, goals, thoughts--and that makes for
better solutions.