Many software programs come with built-in help features. This study attempts to understand how users of Microsoft Word, a popular word-processing application, use and perceive its various help features. A web-based survey was conducted and the findings presented here.
Over
the years Microsoft Word incorporated various help features, including the
infamous "paperclip" Office Assistant. This study looks at which help
features users prefer and which ones they actually use. It also attempts to
find trends in usage patterns and if any particular category of users prefers a
certain help feature over the others. Some interesting results were found. This
study will serve as a pilot study upon which further follow-up studies can be
conducted.
An
online web-based survey consisting of 18 questions was set up at http://www.stanford.edu/~geksiong/survey/survey1.fft. Information was gathered about the user's
familiarity, experience with the various help features, and their perceptions
of these help features. Most of the questions used a 7-point scale or are
multiple-choice. The survey was conducted anonymously.
The
respondents to the web-based survey were students from UC Berkeley (where the
author did his undergraduate work) belonging to a certain student organization.
The students belong to various departments and majors, and differ widely in
experience. Altogether there were 20 respondents to the survey over the data
collection period.
An
email was sent to the student organization's mailing list to ask for volunteers
to participate in the web-based survey. The anonymity of the survey was
stressed in the email. Data was collected over a period of 3 days.
The
online survey was set up on Stanford's Leland web servers on the author's
student account. The Formage application was used to generate the appropriate
HTML forms and to automate the data collection. Microsoft Excel was used for the
presentation and analysis of the data.
Most
of the users are frequent users of Microsoft Word. Half of them use MS Word
2000, and the rest use previous versions. For the purposes of this study, I
divided the users into 2 groups according to their expertise level, which I
shall call the "average" and "expert" users. However, it
should be noted that most of the users see themselves as being slightly above
average in expertise (mean 4.70 +/- 0.86). Only 3 users gave themselves a
"6" rating and only 2 users gave themselves a "3" rating.
Not
surprisingly, 60% of users usually look at the menus. Also, about half of them
look at the toolbar and use the help contents and index. Asking a friend (25%)
or asking the Office Assistant (35%) ranks next on the list of common help
sources. Surprisingly, right-clicking to get the context-sensitive menu is used
often only by 10% of users, despite the fact that this right-click
functionality has been built into all of Microsoft Windows and many other
applications. Reference books or manuals appear to be unpopular, as only one
user does that.
In
terms of experience, almost all users have used the help index, and more than
half have used the right-click menu and Office Assistant. Again, somewhat
surprisingly, 3 users actually said they were not aware that they can
right-click to get a context-sensitive menu, and 2 of them are expert MS Word
2000 users.
Looking
at the menu or toolbar appears to be common across all versions of MS Word.
However, there is a disparity between users who tend to access the help index
or ask the Office Assistant. Users using versions of MS Word prior to MS Word
2000 tend to use the help index, and users of MS Word 2000 prefer to use the
Office Assistant. This disparity might be explained by the fact that the
appearance of the Office Assistant was revamped in MS Word 2000 to look less intrusive.
The previous incarnations of the Office Assistant appear boxed in a floating window
and has a color scheme that sticks out, which might be why users of previous MS
Word versions do not like to use it.
The
most popular sources of help for "average" users are the menu and
toolbar and the help index, with the help index being the most popular (85.7%).
For the more "expert" users, their choices of help sources are more
spread out. Asking the Office Assistant actually ranks above using the help
index for the "expert" users.
“Average?
users appear to like the help contents and index. The rating was slightly lower
for “experts? The contents scored lower than the index, which is not
surprising since the index is a more efficient search method than manually
searching through different chapters.
Right-clicking
was rated highest by both groups, even though few users actually use it often
to look for help. One possible reason might be the fact that context-sensitive
menus work only in context. If a user does not know where to find a particular
feature, he/she is not likely to know under what context to find the menu (that
is, where to right-click).
Interestingly,
the Office Assistant was rated fairly high too by both groups in effectiveness
and in its ability to “understand?English, but it fared poorly in its attempts
to offer help automatically. The animations also scored fairly low among the “average?
users. Overall, the “average?users seem to dislike the Office Assistant more,
but there is much more variability in their liking for it.
The results suggest that
the Office Assistant is effective in providing help. Overall, it does score
higher than using the help contents and index. However, users dislike the
Office Assistant’s feature of offering help automatically, and to a lesser
degree its animations. These factors affect its overall likability. It is
recommended that either the automatic help is made more appropriate and useful,
or it is made less automatic. Still, the finding that the majority of MS Word
2000 users use the Office Assistant more than the help index shows that the
right steps were being made in its design.
A
follow-up study is suggested to assess the actual effectiveness of the help
features.
Although
this study revealed several interesting facts about the surveyed group of MS
Word users, the results cannot be applied to the general group of “MS Word
users?due to the sample used. The sample is made up of non-American college students,
and thus cultural biases might exist, and the results may be different when
applied to American office workers who are the main target users of MS Word.
Even
though 20 users responded to the survey (which are usually enough), the results
are not likely to be statistically significant due to the breakdown of the
sample into smaller subgroups. A larger sample would improve the repeatability
of the study.
Although
attempts have been made to keep the survey as short as possible, it is possible
that the entire section on the Office Assistant have been skipped over by some
respondents. More users were expected to have used the Office Assistant before.
It
is noticed that in almost all the questions using the 7-point scale, no user
used either end of the scales. It might be due to reluctance on the user’s part
to overly praise or criticize the particular MS Word feature. In the case of
rating the user’s own expertise level, the user’s modesty or shame might be at
work to prevent he/she from using the highest and lowest ratings, even though
the survey was conducted anonymously. This is a problem of surveys and such
biases are to be expected. A longer scale may or may not be better, since
longer scales require deeper thought in answering and survey respondents may answer
spuriously as they don’t want to expend the effort in deciding on the best
answer.
One
problem with conducting a survey on a popular software application such as MS
Word is that it had gone through several versions, and users have been trained
by their use of previous versions. For example, a user’s dislike of the Office
Assistant may have been due to its implementation in MS Word 95, and persisted
to his present use of MS Word 2000. The survey should have attempted to address
this issue. However, this kind of data is hard to acquire through the use of
surveys, because of the wide variety of answers possible. Interviews and other
methods with both new and old users might be better solutions.