[Kde-accessibility] [KDE Usability] Bad Studies Are Worse: About the Recent Results on Detail in Icon Design

[Björn Balazs]

Dear mutlu,

thank you for at least giving your reply a softening frame before cross-
posting it all over the world.

I am very happy to talk to you and others about the criticism you have 
(Preferably in the comments of the post, so others gain a benefit from it). I 
personally think there is some misunderstanding. So let me clarify a few 
points.

You insinuate that there is a call to action

You say:
> please don't encourage the F/OSS community to invest resources into design
> based on well-meaning but simply wrongly executed studies.

I do not do that - I would not dare to do that based on the results I present.

I simply say:
> Whenever you create icons, try to be as iconic as possible – or to phrase it
> differently: try to remove as much details from your icons as possible. It
> is likely that the quality of your icons will win.

This is something I would quote even without any data as a background. Use as 
much detail as you need, but not more. Anything fancy and additional will make 
the icon worse. Nothing special here. Nothing like: Throw all icons away and 
redo them. I am not even pointing to a single icon in this study and say 
'throw it away'. So your warning seems to be exaggerated to me.

Next you say:

> This study is so flawed  that the value of its results is nonexistent.

The misunderstanding here is that there is no 'study' concerning high- and 
low-level details in icons. The word 'study' is used first in the comments.

I say:

>It seems to be an easy truth: Too much detail in icons confuses the users. So
>we wondered whether we could find any evidence for this truth in the data of
>our large scale test of the LibreOffice Icons.

If you read this, it becomes clear that we look at the results of something 
that was never designed for testing low- vs. high-detail icons. We are simply 
looking at a prejudice about icon design - in this case: more detail is worse. 
And we are using something we accidentally have - the results of the 
LibreOffice Icon Test study.

The results are not useless. I tell you this as someone who has actually done 
a lot of statistics in his life. We look whether in this set of data we can 
find evidence that the prejudice is wrong - and we cannot. The methodology is 
well respected: We did a rating of icons with two experts and discussed icons 
we disagreed. The rest is simple statistics. 

This is exactly what I describe in the post. So where might I be wrong?

I think I made one mistake in this whole article - and I would like to 
apologize for that: The formulation of the headline and the teaser probably is 
a bit too brave and might hence be misleading.

I want to take the chance, as you are so heavily concerned about the question 
of detail in icons:

As you already outlined the design of a study that would directly address the 
question of high- vs. low-detail icons, I happily invite you - and others 
interested as well: I will support you as much as I can with conducting such a 
study. I personally would be interested in the results as well.


All the best,

Björn


Am Donnerstag, 10. Januar 2013, 10:22:10 schrieb mutlu_inek:
> Dear Icon Designers, Usability People and Survey Makers,
> 
> I am cross-posting a response I left on the blog entry "More is worse:
> About Detail in Icons" posted by Björn Balazs on User Promt and
> aggregated on Planet KDE. It can be found here:
> http://user-prompt.com/more-is-worse-about-detail-in-icons/
> 
> I do not intend to deride the work that has
> gone into this study, but I fear that the community may be misled by the
> results of a highly flawed methodology. Mind that I am not a statistician,
> but I know enough about this topic that I can spot some major flaws in
> no-time. Please find a part of the text (with less typos) below.
> 
> 
> """
> 
> This
> study is so flawed  that the value of its results is nonexistent. Taking it
> seriously may even be harmful.
> 
> 
> First of all, the distinction between low-detail
> and high-detail icons is highly inconsistent. Second, those in the first
> category point to rather well-known, well-entrenched and often-used actions
> the icons of which many will easily recognize, while the latter are less
> often used. Third, the latter category also includes icons such as the
> "auto-spellcheck" and the "spelling and grammar" icons that are very
> close to one another both in meaning and in iconic representation. Thus, an
> allegedly low-detail icons like "save" will score higher than either of the
> spell-related icons due to the inherent ambiguity. Concluding that this has
> to do with the level of detail (which in my opinion is very similar) is not
> the way to go.
> 
> Instead,
>  you would need two sets of icons that represent the same actions, one
> with more and one with less detail. Both icon sets would have to be
> equally known or unknown to your audience. A statistician might tell you
> more.
> 
> I do appreciate your work, but please
> don't encourage the F/OSS community to invest resources into design that
>  are misdirected based on well-meaning but simply wrongly executed
> studies.
> 
> Thank you.
> """
> 
> All the best,
> 
> mutlu
> 
> _______________________________________________
> kde-usability mailing list
> kde-usability at kde.org
> https://mail.kde.org/mailman/listinfo/kde-usability
-- 
Dipl.-Psych. Björn Balazs
Business Management & Research
T +49 30 6098548-21 | M +49 179 4541949

User Prompt GmbH | Psychologic IT Expertise 
Grünberger Str. 49, 10245 Berlin | www.user-prompt.com 
HRB 142277 | AG Berlin Charlottenburg | Geschäftsführer Björn Balazs