Back
to Project 5.3 Toolkit Homepage
Back
to Project 5.3 Toolkit Homepage
University of Michigan
National Center for Postsecondary Improvement
Project
5.3 on Student Learning and Assessment
A Checklist for Survey Administrators
Preparation
Conduct a thorough review of the literature. Search peer-reviewed journal articles, books/monographs, conference papers, and other relevant references.
Organize the literature around the topic. Design a conceptual map of the literature.
Develop research questions or objectives based upon theory
Develop a conceptual framework to guide the research, preferable a visual representation. If using a diagram, place the dependent variable on the right side and the independent variables toward the left. Insert arrows to represent the hypothesized relationships between variables.
Budget Considerations
Staff time to plan and implement the study.
Sample selection costs.
Labor and material costs for pre-testing the questionnaire and field procedures.
Supervisory costs for interviewer hiring, training, and monitoring (if needed).
Interviewer labor costs and travel expenses, including meals and lodging.
Expenses associated with redoing some interviews, or following-up on non-respondents.
Labor/material costs related to data entry.
Cost of reviewing the quality of the computerized paper questionnaires.
Cost of “cleaning” the final data set.
Labor or material costs for data analyses and report preparation
Telephone charges, postage, reproduction, and printing costs for all stages of the survey – from planning activities to distribution of the results
Questionnaire Design
Determine a timeline for administering the survey
Plan each area of the survey, including the cover letter and closing instructions
Determine what variables the survey will target (e.g., demographic, attitudinal, behavioral, factual)
Variables should cross-reference the research questions and survey items
Design the survey questions.
Define objectives. Specify
the kind of answers needed to meet these objectives.
Make sure all respondents will share a common understanding of each question
Be reasonably sure that respondents will know the answers to each question.
Ask questions that respondents can answer using the terminology
provided.
Ask questions that respondents are willing to answer accurately.
Decide what types of measurement scales will be included
Rating scales (e.g., strongly agree to strongly disagree)
Categorical scales (e.g., yes, no)
Rank-ordered scales (e.g., rank from highest to lowest
Field test the survey instrument.
To
establish face validity.
To
improve survey design.
Sampling
Determine who will be sampled and what is to be learned about the sample.
Data Collection
Select a method of data collection
Surveys
Use a standardized (commercial) or locally-developed instrument
Administer on location, by mail, or on-line (web survey)
Exams (machine readable)
Qualitative approaches (focus groups, interviews, video taping, etc.)
To ensure a high response rate to mailed surveys:
Initial mailing – cover letter and questionnaire
Second mailing – second letter and questionnaire
Postcard follow-up – reminder
Data Processing – Confidentiality Issues
Use only number codes to link respondents to questionnaires
Store the name-to-code information separate from the questionnaires
Do not share respondent information with anyone outside of the survey group
Destroy questionnaires and identifying information following data entry
Omit names of survey respondents from computer files used for analysis
Present statistical tabulations using broad categories so individuals cannot be identified
Data Analyses
Analyze returns
Check for response bias – the effect of non-responses on survey estimates
Collapse terms into scales using factor analysis
Check for reliability of scales, as needed
Run multivariate statistics to answer the research questions, as needed
Link to suggested readings: Articles of Survey Research
back to Survey Intro Page
back to Project 5.3 Toolkit Homepage