Handout #10: - Survey Research
The interview and the questionnaire both utilise the question asking approach. In an interview, data are gathered through face-to-face or telephone interaction between the interviewer and the respondent. On the other hand, the questionnaire obtains information through the respondent’s written responses to a list of questions. Each method has advantages and disadvantages and the researcher needs to decide which of the two methods is more appropriate for the needs and characteristics of a particular study.
INTERVIEWS
¨
One of the most important advantages of the interview is its
flexibility. The subject is able to respond in the total situation in which he
or she is responding. The researcher is able to obtain clarifications or
meanings from the respondents.
¨
Low response rates usually associated with mail questionnaire reduce
the sample size and may bias the results. Interviews increase the response
rate. Personal contact increases the likelihood that the individual will
participate.
¨
The interviewer controls the order with which questions are considered.
In many cases it is important that the respondent not know the nature of the
later questions because their responses to these questions might influence
earlier responses and bias the results.
¨
Cost and time-consumption are the main drawbacks of interviewers.
¨
Open and close-ended items can be used in interviews. Open-ended items
permit free responses from respondents while close-ended items ensure that all
respondents will have the same frame of reference in responding. A limitation
of the closed question is that it does not provide much insight into
respondent’s perception of a particular issue. A combination of both open and
close-ended items may be used in an interview.
Conducting the Interview
The main task of the interviewer is to ask the question in such a way
as to obtain a valid response and to record the responses accurately and
completely. The interviewer must put the respondent at ease.
¨
Introduce yourself in a friendly way.
¨
Briefly state the purpose of the interview without giving too much
information about the study (this could create bias).
¨
Keep the respondent’s attention focused on the task.
¨
Keep the interview moving along smoothly by being familiar with the
sequence of your questions.
¨
Refrain from expressing approval, surprise or shock at any of the
respondents answers.
¨
If comparable data are to be obtained, the process must be
standardised. (Questions must be worded and presented in the same order for all
respondents)
Where the respondent digresses or gives an
irrelevant answer, the interviewer needs to probe.
¨
A complete and accurate recording of the respondent’s exact words must
be made. The recording can be facilitated by use of abbreviations. Taping (with
the interviewee’s permission) is the best way of recording information
verbatim.
QUESTIONNAIRES
¨
Interviewers are expensive and time-consuming and often the same
information can be gathered by less expensive means. Because a questionnaire is
designed for self-administration and can be mailed, hand-delivered or done via
telephone, it is possible to include a larger number of items and subjects.
¨
Questionnaires protect the respondent’s identity and guaranteed
confidentiality is more likely to elicit more truthful responses than would be
obtained with s personal interview.
¨
A major disadvantage of the questionnaire is the possibility of
misinterpretation of questions by respondents. It is often difficult to
formulate questionnaires that are exactly the same to all respondents.
¨
The response rate for questionnaires is usually lower than for
interviews (40% is the average response rate for most surveys). A low response
rate limits the generalisability of the results of a (questionnaire) study. The
researcher cannot assume that non-response is randomly distributed throughout
the sample/population.
Common Factors Influencing the Rate of Returns
1. Length
of the questionnaire
2. The
cover letter (vague, impersonal, unattractive)
3. Sponsorship
of the study
4. Attractiveness
and presentation of the questionnaire
5. Ease
or lack of ease in completing the questionnaire
6. Ease
or lack of ease in returning the completed questionnaire
7. The
interest aroused by the content
8. Follow-up
procedures used
Constructing the Questionnaire
Construction of a questionnaire is a
difficult and time-consuming task. A well-constructed questionnaire is more
likely to elicit good and complete responses than do ill-constructed ones.
¨
Construct the instrument to reflect quality. During the process of
constructing the questionnaire, numerous revisions may be necessary to
eliminate ambiguous or unnecessary items.
¨
The questionnaire should require the minimum amount of the respondent’s
time. Keep it as brief as possible as respondents are more likely to complete
and return a brief questionnaire rather than a long complex one. All the items
should seek to address a research question or hypothesis. (Do not ask questions
if the answers are not useful in the data analysis process).
¨
Ensure that respondents are knowledgeable about the “problem”. Avoid
items that are unfamiliar to the respondents.
¨
Construct each item so that every respondent can understand it. The
vocabulary should be non-technical and should be geared to the “average”
subject in the group. On the other hand, the researcher should not “talk down”
to the respondent. Where necessary, operational definitions should be provided
to ensure consistency in interpretation of items and answers provided.
¨
Try to keep items short and simple. If items are complex, separate them
into short and simple ones.
¨
Items should be phrased so that they generate unambiguous answers. When
possible, quantity responses. For example rather than asking for sometimes,
always, never…ask how many times per week/month/day, etc.
¨
The wording of a question should not influence the respondent in a
certain direction. Avoid stereotyped, prestige-carrying, or emotionally loaded
words.
¨
Avoid misleading items (because of unstated assumptions).
¨
Ensure that all alternatives to an item are exhaustive. For example, if
you need to know the marital status of a subject, do not only include married
and single, but include widowed, divorced, separated etc.
¨
Avoid items that are likely to elicit embarrassment, suspicion or
hostility in the respondent. In other words, items should not put respondents
on the defensive.
¨ Avoid double-barrelled questions. Do not ask two questions at once.
Types of Questions/Items
A researcher may use both open- and close-ended items in a
questionnaire.
1. Completion, fill-in. These are open-ended items to which respondents must supply their own answers stated in their own words.
2. Checklists. These items present a number of possible answers from which respondents are asked to check (√) all those which apply.
3. Scaled
items. Respondents are asked to rate a concept, event or situation on degree of
a certain characteristic: quantity, intensity, and frequency.
4. Ranking.
These items ask respondents to indicate the order of their preference among a
number of options. Rankings should not involve more than six (6) options as it
becomes too difficult and time-consuming for respondents to make comparisons.
5. Likert-type
items. Allows the respondent to indicate his/her response to selected
statements on a continuum from highly positive to highly negative.
Arrangement of the Questionnaire
¨
Pay careful attention to the very first item on the questionnaire. This
might determine whether or not respondents continue or “toss it in the
bin.” The first items should be clear,
interesting, simple and easy to understand.
¨
Questions that are similar in content should be grouped together.
¨
A good psychological or logical arrangement contributes to better
thought-out answers. For example, before asking respondents to recommend
changes to an office environment, first ask them how they perceive the present
arrangement.
¨
If general and specific questions are asked on the same instrument,
start with all the general ones first and then include the specific ones. Items
that are less likely to be objectionable should precede those that are more
objectionable. For example, if a researcher wanted to find out the extent to
which college students drink alcohol; he/she could first ask “How would you
describe alcohol consumption in your college?” Objective (multiple choice)
responses could be provided eg. Serious problem, moderate problem, slight
problem, no problem. Then do you think alcohol consumption among students has
increased over the last year? Do you
know students who consume alcohol? Then finally, have you ever consumed
alcohol?
Layout of Questionnaire
The researcher should ensure that the questionnaire is attractive, easy
for the respondent to read and answer and convenient for the researcher to code
and score.
1. Number
questions consecutively.
2. Differentiate
between question number and answer categories
3. Use
numbers to identify various answer categories
4. Be
consistent in assigning numbers to the various answer categories
5. Response
categories should be arranged in vertical rather than horizontal format
6. Reproduce
questionnaires by high-quality printing/copying method – this ensures a more
professional appearance.
Pre-testing the Instrument
1. Ask your colleagues to provide you with
feedback on the questionnaire. Ask them specific questions such as:
¨
Will the instrument provide the desired
data?
¨
Is it attractive?
¨
Is it easy to follow?
¨
Are the instructions clear?
¨
Are all answer categories exhaustive?
2. Administer the questionnaire to a ‘focus group’ which consists of
persons similar to the study population and/or experts in the area. The
researcher needs to pay attention to the questions respondents ask, such as;
“what does this mean?”, “although this item asks for one answer, more than one
applies to me”. The researcher needs to note the items which respondents spend
an unusually long time on. In ensuring “a quality questionnaire”, the
researcher needs to ask:
¨
Do the respondents appear to be comfortable
with the questionnaire?
¨
Do they appear motivated to complete it?
¨
Are some items confusing?
¨
Are some items likely to cause embarrassment
or hostility?
¨
Are the instructions clear?
¨
How long does it take a respondent to
complete the questionnaire?
¨
Do all respondents interpret the items in
the same way?
3. Even after pre-testing with a small group,
the researcher may still need to “pilot test” the instrument on a sample
similar to the “real” one.
Administering The
Questionnaire
In administering
the questionnaire, the researcher needs to attach a cover letter.
The cover letter introduces the researcher and the research and serves to alert
the subject to the study so that he/she is not overwhelmed. It also serves as a
“selling” tool for the researcher.
Essential Elements
of a Cover Letter
1. The purpose of the
study. This should be stated in the first
paragraph of the letter and should explain the purpose and usefulness of the
study. In establishing the purpose, the researcher should be very creative in including the possible benefits for the
participant.
2. A request for cooperation. The researcher needs to explain
why the potential respondent was included in the
sample and appeal for cooperation. It is important for the respondent to feel
that he/she can contribute to something meaningful.
3. Protection of the
respondent.
The letter must clearly state that confidentiality will be assured and HOW it
will be maintained. To facilitate a high return rate, identification numbers
can be used. If identification is missing, the problem of non-response bias is
compounded because there is no way of knowing who responded and those who did not. It might be prudent not to use
any identification especially on sensitive issues. A post card or tear-off-slip
can be included so that the respondent can mail it in at the same time the
survey is mailed (although in a separate envelope).
4. Sponsorship of
the study. The signature on the letter is very
critical. If this is a university requirement, it might be helpful if a person who is well-known to the
respondents signs or countersigns the letter. Such a signature is likely to be
more effective than that of an unknown student. A university letterhead might also be useful.
5. Promise of
results. If the researcher makes an offer to share
the findings of the study with the respondents, this should be honoured. The
researcher could have respondents indicate if they wish to have a copy of the
findings.
6. Appreciation. The researcher must include an expression of appreciation for
respondents’ assistance and cooperation.
7. Date of the
letter. The cover should bear a date close to the
time of mailing.
8. Request for
immediate return. If too much time is
given (four weeks), the respondent may procrastinate and forget about the instrument. It is advisable to give a
specific date.
Although all elements are important and should be included, the cover
letter should be a brief as possible; not more than a page long. Always include
a self-addressed, stamped envelope for mail surveys.
Summarised from
Ary, Jacobs and Razavelh (1990)