1. What are the different types of rating scales? The researcher is also being rated. Sample questions are, "how would you rate his writing ability" or "how would you rate the subject’s ability to play the violin compared to others his age"? But these are not Guttman scales. Types of rating scales are: (1) graphic or numeric (subjects rated on certain quality (like "low, medium, high" or "1 through n"), (2) category scales (ask raters to put subjects into 5 to 7 categories; 3 are probably too few and 9 are probably too many), and (3) comparative scales (ask person to compare subject based on other subjects of same classification (top 1%, top 5%, top 10%).

Possible errors are: (1) Halo effect (a rater’s general impression of a person can cause the rating to be subjective and cause a loss of neutrality), (2) generosity effect (the tendancy to always rate too high; workaround is to have multiple raters grading the same subject to identify the generous rater), (3) error of severity (the tendancy to always rate too low), and (4) error of central tendancy (the tendancy to call everything average with little or no variability; once again, the workaround is multiple raters).

2. What do sociometric techniques measure? This instrument lets the researcher determine how people interact or determine social differences between them. Who can get close to a particular person, and who cannot? How well does the subject interact with others? What family members talk to each other, and which ones do not? The researcher must look for behavior patterns, such as hostility or friendliness. Types of sociometric techniques are: (1) sociogram (a pictorial pattern of interpersonal relationships with everyone’s name on it and with lines drawn to show who talks to whom. A star, or stars, are the most popular. Clicks are 3 or more people who stick together, and the isolate is the one that no one talks to.) and (2) Bogardus’ social distance scale (very similar to Guttman scale (it would be acceptable to me for a communist to (a) live in the United States, (b) live in my state, (c) live in my city, (d) live in my neighborhood, (e) live next door, (f) be my friend, (g) be a close friend, and (h) marry my child.) but Bogardus can relate to other things than just social).

3. How should questionaires be developed? An example is, "Which of the following social issues are most important to you? Choose 5." Questionaires can be: (1) close-ended to force the subject to make a choice or (2) open-ended (OTHER) to let the subject respond.

To develop a questionaire, (1) keep it highly specific, (2) keep it short so people will do it, (3) do not ask for information that can be collected elsewhere, (4) avoid redudancy, (5) arrange questions from nonthreatening to threatening, (6) avoid ambiguous questions that will destroy the questionaire, (7) avoid double barrelled questions that have multiple answers ("should school spend more on basics AND less on extra curricular activities" – use two questions), (8) avoid unstated assumptions and unfamiliar terms (strive for 4th grade reading level), (9) make questions relevant to the subject, (10) avoid negative items (do not use the word "not" if at all possible), (11) avoid biased terms or phrases (do you think Russia got what it deserved when Communism fell? – wording questions for abortion questionaires can be very difficult), (12) avoid leaving out possible choices (think through all the possibilities), (13) make the questionaire neat and easy to follow, and (14) make the questionaire easy to score.

When administering the questionaire, (1) get permission to do the questionaire, if necessary, (2) provide a cover letter for questions (also state due date), (3) conduct a follow-up, and (4) give a reasonable deadline and a stamped envelope.

4. What are the types of observation? Types of observation are: (1) direct (researcher observes subjects while they are in their natural setting), (2) indirect/contrived (researcher observes in a simulated setting), and (3) mechanical (researcher uses a video camera to collect the information and later reviews it).

The steps in using observation are: (1) select a behavior to observe, (2) clearly define that behavior so other raters will know what to look for, (3) train observers so everyone will know exactly what to look for, (4) develop a method for quantifying an observed behavior, and (5) develop a coding system that will help the observers quickly encode their observations.

Two approaches to assessing observed behavior are: (1) describe everything that is observed without trying to evaluate and (2) describe everything that is observed and make quantifying remarks (very bad tantrum).

Two approaches to gathering data are: (1) use a systematic approach when using a lot of observers and give them a coding system and (2) allow informal observation, which would be less stringent and even antedotal.

5. What are some examples of research that might use each type of instrument? For measures of ability, the research can be performed by a teacher who tests the students on lectured material, or the research can be performed by a school system that administers yearly, standardized aptitude tests. For measures of personality, the research could be to give a group of depressed people the statement, "Today, I feel like I could just ….," and ask them to complete the statement. For altitude scales/opinionaires, the research could be a Likert scale for college students to rate their professor, where the responses would range from strongly disagree to strongly agree. For interest inventories, college graduate students could be asked if they would rather write a thesis or take extra classes. For rating scales, the research could be for college students to rate their professor based on a continuum from 1 to 10. Also, sports fans could be asked to assess a group of athletes based on scales like, "in the top 1% of their sport, between 1% and 10%, between 10% and 25%, or between 25% and 100%. For sociometric techniques, a researcher could draw a sociogram to measure interaction between family members in an abusive home. For questionaires, the research could be to ask subjects, in an open-ended question, to name the top athlete of the 20th century. The question, or statement, would list 4 or 5 obvious choices and then the choice "OTHER" so that the subject could provide his or her own candidate. For observation, the research could be to observe the behavior of family members in an abusive home.

				Tom of Bethany

Back To TLEE's Home Page

Index to Selected Essays And Book Reviews

Lesson 18. Experimental Validity (Part I)

Reaction to Video Lesson 17

Send email to: tlee6040@aol.com