Methods

Iowa Tests of Basic Skills (ITBS)

Tests of Achievement and Proficiency (TAP)

Background Questionnaires

Bob Jones University Press Testing and Evaluation Service

Data Generation Procedures

1. Parents contracted with Bob Jones University to be administered the Iowa Tests of Basic Skills or the Tests of Achievement Proficiency (39,607 students in probably 22,000 families).
2. Bob Jones certified test administrators, many of whom were the students' parents.
3. BJU sent questionnaires and answer forms to the test administrators.
4. Tests and questionnaires were returned to BJU. BJU bundled the tests and sent them to Riverside Publishers for machine scoring. BJU bundled the questionnaires and sent them to National Computer Systems for scanning. Unlike in previous studies, the parents did not know their scores ahead of time.
5. Electronic copy of the 23,415 test results and 23,311 questionnaire results were sent to the author of this report. These sets were merged to provide 20,900 cases with matching identification numbers. In order to weight by state public school enrollment, 140 cases with missing state data were dropped. A total of 20,760 students formed the initial dataset used in the study. After we formed the dataset with 20,760 students, we asked for the remainder of the 39,607 achievement test scores. We were informed that it would not be possible to disaggregate the remaining home school students from students in private schools also contracting testing services.

Characteristics of Home School Students and Families

          As shown in Table 2.1, respondents came from each of the fifty states. Several states, including Ohio, Georgia, and Virginia, have exceptionally high representation given their size. This is probably due to the fact that these states require testing of home school students. To reduce the effects of these and other overrepresented states, the data were weighted in all subsequent analyses by the number of public school students in each state. While we would have preferred to weight by the number of home schooled students in each state, such data are not available for all 50 states (Lines, 1998).

Student Age and Gender

          Table 2.2 shows the distribution of the respondents by gender and age. About 50.4% or 10,471 of the respondents were females; 49.6% (10,319) were males. These figures are comparable to that of the population of 3 to 34 years old enrolled in school (see U.S. Bureau of the Census, 1998, Table A-2). Some 51.4% of school enrollees nationally are male. The percentages are comparable at all age levels.

Student Grade

          Home school student grade placement was identified by their parents, presumably based on the grade level of the instructional materials. That grade was used by BJU to determine the test levels and used in this report as a grouping variable. Tables 2.3 shows the distribution of respondents and the nation by grade. There is a large difference in the proportions of high school (grades 9-12) home school students and the nation. Compared to the national data, a relatively small percentage of home school students are enrolled in high school. Possible reasons for this lower participation for high school students may be the relative newness of the home school movement, early graduation from high school, and possibly a desire on the part of some home school parents to enroll their children in a traditional high school. The distributional differences for students in grades 1 through 8 are minor.

Student Race

          Table 2.4 shows the racial distribution of home school students in 1998 and for the students enrolled in elementary and secondary public and private schools nationally in 1994. The distributions are quite different. The vast majority of home schooled children are non-Hispanic White. The largest minority groups for home school students (not shown in the table) are American Indians and Asian students who comprise some 2.4% and 1.2% of the home school students, respectively.

          The great majority of home school students are in married couple families. In contrast, only 72% of the families with at least one child enrolled in school nationwide are in married couple families (Bruno and Curry, 1997, Table 19).

Children at Home

          Table 2.6 shows the distribution of children in home school families and families with children under 18 nationwide. On average, home school students are in larger families. Nationwide, most families with school-age children (79.6%) have only 1 or 2 children with a mean of about 1.9 children per family. Most home school families (62.1%) have 3 or more children with a mean of about 3.1 children per family.

Mother's Religion

          We asked the home school families to identify the religious preference of each student's mother by selecting from a list of 27 religions. As shown in Table 2.7, the largest percentage of mothers identified themselves as Independent Fundamental, Baptist, Independent Charismatic, Roman Catholic, Assembly of God, or Presbyterian. The religious preference of the father was the same as that of the mother 93.1% of the time.

Parent Academic Attainment

          As shown in Table 2.8, home school parents have more formal education than the general population. While slightly less than half of the general population attended or graduated from college, almost 88% of home school students have parents who continued their education after high school.

          National data on family income are available for 1995. As shown in Table 2.9, home school families span all income levels. On average, home school families have a higher income level than do families with children nationwide and all families nationwide. The median family income level for home school families in 1997 is about $52,000. The median income for families with children in 1995, nationwide, was about $36,000.

Television Viewing

          The National Assessment of Educational Progress collects information on the television viewing habits of fourth-graders. Home school fourth-graders and fourth-graders nationally differ markedly in terms of television viewing. Home school students rarely watch more than 3 hours of television per day; nearly 40% of the students nationwide watch that much television.

Computer Use

          The Condition of Education provides a tabulation of the percent of students nationwide who report using a computer by frequency of use for 4th, 8th, and 11th graders in 1996. At each grade level, the distribution of computer use in 1998 by home school students is different from that of the nation in 1996. At each of these three grade levels, much larger percentages of home school students never use a computer. At the fourth-grade level, a much larger percent of home school students use a computer every day.

Money Spent on Educational Materials

          The amount of money spent in 1997 on home school education for textbooks, lesson materials, tutoring and enrichment services, and testing ranged from less than $200 to more than $2000. As shown in Table 2.12, the median amount of money spent was about $400.

Other Demographic Characteristics

Academic Achievement

          This same DSS scale is used for all tests and levels of the ITBS and TAP. The main advantages of the DSS are that it mirrors reality well, spans all grade levels, and provides a quasi equal interval scale which has a variety of attractive statistical properties. Most importantly, DSS scores can be compared to each other and can be meaningfully averaged.
          The main disadvantage of DSS scores is that they have no built-in meaning. Reference points are needed to interpret DSS scores. "Grade level" is one possible reference point. A DSS score of 170 in reading, for example, is about equal to the typical reading score for second-grade students in public and private schools in the spring of the year. A more refined reference is the percentile score that corresponds to each DSS score. The 170 in reading, for example, corresponds to the 54th percentile of second graders. That is, this score is better than the score received by 54 percent of the second graders using the 1995 spring norms.
          The reader should note that while all tests of the ITBS/TAP have the same median DSS score at each grade level, the distributions within each subject area vary. A DSS score of 310 for a tenth grader in reading, for example, corresponds to the 87th percentile. A DSS score of 170 in mathematics for a tenth grader would place the student at the 79th percentile.
          Percentiles are always defined in terms of a grade level. This can be problematic when analyzing data for home school students. In this study, 24.5% of the home school students were one or more grades above the grade usually associated with that student's age (see Table 3.2). A strong case can be made that rather than using the percentile corresponding to the enrolled grade, as we did in this study, one should use the percentile associated with the student's nominal grade, i.e., the grade usually associated with the student's age. The argument is that a 10-year-old home school student enrolled in 5th grade should be compared to his age peers in 4th grade. The counter argument is that the percentiles already consider the fact that students are not always in their nominal grade since the standardization sample had students above and below grade level. We initially analyzed the data both ways. Rather than expose our analysis to criticism, we chose to take the more conservative route by employing the enrolled grade.
          While very meaningful, percentiles do not provide a complete picture of a student's or group's academic performance. In this study, we used grade equivalent scores as an additional reference point for interpreting DSS scores. A grade equivalent score approximates a child's development in terms of grade and month within grade. A DSS reading score of 170 can be viewed as the typical DSS score earned by students in the ninth month of the second grade or a GES score of 2.9. Just as the percentile associated with a DSS scores varies by subtest, so do the properties of GES scores vary across subjects.
          Grade Equivalent Scores are particularly useful for estimating a student's developmental status in terms of grade. But, these scores must be interpreted carefully. An GES Score of 6.3 in reading for an 9 year old in the 3rd grade, for example, clearly indicates that the third grader is doing well. This does not, however, mean that the third grader belongs in the 6th grade. It only means that the third grader can read as well as a sixth grader.
          The usual interpretation of a Grade Equivalent Score of 6.3 for a third grader is that this third grade student can read third grade material as well as a sixth grader can read third grade material, not that he or she can read sixth grade material. The DSS of the ITBS/TAP, however, is unique. The DSS scales were developed by administering the same special scaling test to students in grades K-3, another common scaling test to students in grades 3 to 9, and another to students in grades 8-12. Thus, in the scaling study, the third graders did take the same test as the sixth graders in each subject area.

  Grade Placement

          Home school students are able to progress through instructional material at the student's rate. Thus, it is easy for home school students to be enrolled one or more grades above their public and private school-age peers. To evaluate the frequency of advanced placement, we compared students' enrolled and nominal grades. The enrolled grade was identified by the parents and used to determine the ITBS/TAP level. The nominal grade is the public school grade in which the student would normally be enrolled in based on the child's month and year of birth.
          As shown in Table 3.2, almost one fourth of the home school students (24.5%) are enrolled one or more grades above their nominal grade. While comparable figures nationally do not exist, one research director in a large school district estimated that less than 5% of their students are enrolled above grade level.

          Table 3.3 shows the median scaled score (DSS score) for home school students on the Composite with Computation, Reading Total, Language, Mathematics Total with Computation, Social Studies, and Science subtest scores by grade. The corresponding percentiles shown in the table are the within grade percentile scores for the nation that correspond to the given scaled scores. For example, home school students in Grade 3 have a median composite scaled score of 207 which corresponds to the 81st percentile nationwide. The median home school student in third grade out- performs 81% of the third graders nationwide. As an additional comparison, we provide the national median for each grade in the last column. By definition this is the 50th percentile of students nationwide.

Figure 1. Academic Achievement of Home School,
Catholic/Private and the Nation's Students

 
          The grade equivalent score comparisons for home school students and the nation are shown in Figure 2. In grades one through four, the median ITBS/TAP composite scaled scores for home school students are a full grade above that of their public/private school peers. The gap starts to widen in grade five. By the time home school students reach grade 8, their median scores are almost 4 grade equivalents above their public/private school peers.

Figure 2. Home School Students Compared to the National Norm Group in Grade Equivalent Units

Years of Home Schooling

          Almost half of the respondents (47%) indicated that they have been home schooled for each grade prior to their current grade, i.e., their entire academic life. Table 3.6 shows that students who are home schooled for their entire academic life do better than students who have been home schooled for only a few years (F academic life =108.2; df=1,9750; p<.01). There is also a significant interaction between grade and years home schooled (F=7.4; df=9,9750, p< .01), indicating that the effectiveness of home schooling varies with the student's grade. The differences are most meaningful starting in Grade 6.
          [All F ratios reported here are from a two-way analysis of variance with composite scaled scores as the dependent measure, grade as a blocking variable, and one independent variable. Because the students are within families, the dataset was trimmed by randomly selecting one child from each family. Had the full dataset been used, the variance of the children within a family would have been artificially smaller than the variance of among children in the population of inference. This would have increased the risk of Type I error, showing significance when significance may not be so. To assure adequate cell sizes, the analyses were also restricted to Grades 1 through 10. A statistically significant difference only means that there is evidence of a difference in population values. The difference may be small and not meaningful. "n.s." is used to indicate not significant.]
          One reviewer questioned whether this significant difference was due to life-long home schooling or was life-long home schooling serving as a proxy for parent education or income. The correlation of life-long home schooling and whether either parent has a college degree is .12, indicating there is some, but not a great deal of overlap between these variables. The correlation with income level was .02, indicating no relationship. Thus, whether a student is home schooled his or her entire life appears to be significantly related to achievement.

[The percentiles (%ile) shown in this and the following tables are the within-grade percentiles corresponding to the mean composite scale scores, differences and ranges refer to differences in and ranges of mean composite scale scores, sd refers to standard deviation, N is the number of students within each cell.]

  Enrolled in a Full-Service Curriculum

          There is no significant difference in the mean composite scaled scores of home school students enrolled in a full-service curriculum and home school students not so enrolled. As shown in Table 3.7, the means are quite close at all grade levels (F enrollment=.24; df=1,9750; n.s.).

  Student Gender

          There are no significant differences in the achievement levels of male versus female home school students (F for gender=.01; df=1,9750; n.s.). As shown in Table 3.8, the means are virtually identical at all grade levels.

  Money Spent on Educational Materials

          There is a significant difference in the achievement levels of home school students depending on the amount of money spent per child on educational materials including textbooks, lesson materials, tutoring, enrichment services, and testing (see Table 3.9). At almost every grade level, students in families spending $600 or more outperform students in families spending less than $200 (F for money spent=41.1; df=3,9585; p <.01). There is also a significant interaction between grade and money spent (F=2.7; df=27,9585; p <.01) indicating that the amount of money spent on education makes a bigger difference at the higher grade levels. The correlation between money spent on educational materials and income is significant (r=.24, p <.01), indicating that this effect may be due to family characteristics rather than expenditures.

  Family Income

          There is a significant difference in the achievement of home school students based on family income. As shown in Table 3.10, students in higher income families consistently have higher mean composite scaled scores (F for income = 79.1; df=3,9186; p < .01). There is also a significant interaction of income and grade (F =2.6; df=27,9186; p<.01). Achievement differences due to income are more pronounced for students in higher grades.

  Parent Certification as a Teacher

          To determine whether there is a difference in achievement for students in households where at least one parent holds a state issued teaching certificate, we analyzed the data for the 7,607 students with at least one parent that has a college degree. As shown in Table 3.11, the achievement levels across groups are remarkably similar. Controlling for grade and parent education level, there is no significant difference in the achievement levels of home school students whose parents are certified and those that are not (F for certification=2.9; df=1,7587; n.s.).

  Parent Education Levels

          The National Assessment of Educational Progress has consistently shown marked differences in the performance levels of students nationwide as a function of parent's educational level. Similar differences appear in the performance levels of home school students. As shown in Table 3.12, at every grade level, children of college graduates out perform children whose parents do not have a college degree (F=566.4; df=2,9744; p < .01). There is also a significant interaction between grade and parent education (F=8.7; df=18,9744; p < .01), indicating that the effect of parent education is more pronounced in some grades. It is worthy to note that, at every grade level, the mean performance of home school students whose parents do not have a college degree is much higher than the mean performance of students in public schools. Their percentiles are mostly in the 65th to 69th percentile range.

  Television Watching

          It was pointed out above that home school students spend significantly less time watching television than do the general population of school-age students. Like the nation as a whole, increased amounts of television viewing for home school students is associated with lower achievement test scores. Table 3.13 shows that at every grade level, there is a steady decline in achievement as the amount of television viewing increases (F for televison viewing =142.5; df=3,9685; p <.01). The interaction of grade and amount of television viewing is also significant (F=5.5; df=27,9685; p <.01). The effects of television on achievement are more pronounced with students in higher grades.

Summary of Major Findings

Major findings: Demographics

• Home school parents have more formal education than parents in the general population; 88% continued their education beyond high school compared to 50% for the nation as a whole.
• The median income for home school families ($52,000) is significantly higher than that of all families with children ($36,000) in the United States.
• Almost all home school students (98%) are in married couple families. Most home school mothers (77%) do not participate in the labor force; almost all home school fathers (98%) do work.
• Home school students watch much less television than students nationwide; 65% of home school students watch one hour or less per day compared to 25% nationally.
• The median amount of money spent annually on educational materials is about $400 per home school student.
• The distribution of home school students by grade in grades 1-6 is consistent with that of all school children. Proportionally fewer home school students are enrolled at the high school level.

Major findings: Achievement

• Almost 25% of home school students are enrolled one or more grades above their age-level peers in public and private schools.
• Home school student achievement test scores are exceptionally high. The median scores for every subtest at every grade (typically in the 70th to 80th percentile) are well above those of public and Catholic/Private school students.
• On average, home school students in grades 1 to 4 perform one grade level above their age-level public/private school peers on achievement tests.
• The achievement test score gap between home school students and public/private school students starts to widen in grade 5.
• Students who have been home schooled their entire academic life have higher scholastic achievement test scores than students who have also attended other educational programs.
• There are no meaningful differences in achievement by gender, whether the student is enrolled in a full-service curriculum, or whether a parent holds a state issued teaching certificate.
• There are significant achievement differences among home school students when classified by amount of money spent on education, family income, parent education, and television viewing.

Discussion

Note

ACT, Inc. (1998). The 1998 ACT High School Profile Report--National Data. Iowa City, IA. Available on-line: http://www.act.org/news/98/98data.html

Bruno, Rosaline and Andrea Curry (1997). Current Population Reports. Population Characteristics: School Enrollment--Social and Economic Characteristics of Students: October 1995 (update).Available on-line: http://www.census.gov/prod/2/pop/p20/p20-492u.pdf

Calvery, Robert; and Others (1992). The Difference in Achievement between Home Schooled and Public Schooled Students for Grades Four, Seven, and Ten in Arkansas. Paper presented at the Annual Meeting of the Mid-South Educational Research Association (21st, Knoxville, TN, November 11-13, 1992).

Day, Jennifer and Andrea Curry (1998). Current Population Survey (CPS) for the Nation. United States Census Bureau. Available on-line: http://www.census.gov/population/www/socdemo/school.html

Drahozal, Edward (1997). Validity Information for the Iowa Test of Basic Skills (ITBS) and Iowa Test of Educational Development (ITED), Forms K, L, M. Riverside Publishing Company, working draft.

Home School Court Report (Dec. 1990). A Nationwide Study of Home. Available from ERIC Document Reproduction Service ED381725.

Hoover, H.D., N. Hieronymous, D.A. Frisbie, S.B. Dunbar (1996). Catholic/Private Norms: ITBS. Itasca: IL: Riverside Publishing Company.

Lines, Patricia (1998). Home schoolers: Estimating Numbers and Growth. Technical paper.

Ray, Brian (1997). Home Education Across the United States. Purcellville, VA: Home School Legal Defense Association. Available on-line: http://www.hslda.org/media/statsandreports/ray1997/index.stm

Riverside Publishing Company (1994). Riverside 2000 Integrated Assessment Program, Technical Summary. Chicago: Riverside Publishing Company.

Scannell, D.P, O.M. Haugh, B.H. Loyd and C.F. Risinger (1996). Catholic/Private Norms: TAP. Itasca: IL: Riverside Publishing Company.

Snyder, Thomas and John Wirt (1998). The Condition of Education, US Department of Education, National Center for Educational Statistics.

US Census Bureau (1996). Educational Attainment in the United States March 1996 (Update). Available on-line: http://www.census.gov/prod/2/pop/p20/p20-493u.pdf

US Census Bureau (1997a). Statistical Abstract of the United States, 1997. Available on-line: http://www.census.gov/prod/3/97pubs/97statab/labor.pdf

US Census Bureau (1997b). Statistical Abstract of the United States, 1997. Available on-line: http://www.census.gov/prod/3/97pubs/97statab/educ.pdf

US Department of Education (1996). Youth Indicators, Washington, DC: National Center for Educational Statistics.

US Department of Education (1997). National Assessment of Educational Progress, 1996 National Mathematics Results. Washington, DC: National Center for Education Statistics.

Acknowledgments

This report relied on the creativity and expertise of several individuals. Michael Farris, Esquire, President of the Home School Legal Defense Association conceived the study and secured the cooperation of the author and three contributing organizations - Bob Jones University Testing Service, National Computer Systems, and HSLDA. Earl Hall of HSLDA worked out the numerous logistics and details of working, was the primary architect of the background questionnaire, and provided responses to my thousand questions regarding home schooling. Janet Abbott of BJU provided information about the testing program and the datasets. BJU staff also hand coded the student identification numbers to make corresponding background and achievement information possible. Tom Perry of National Computer Systems handled the logistics of scanning the background questions. Edward Drahozal, Janet Adair and Vesna Plavsic of Riverside Publishing Customer Support were wonderful, knowledgeable resources helping the author understand the ITBS/TAP norming process and data disks. Finally, the author is deeply indebted to H.D. Hoover of the Iowa Testing Program, Gene V Glass of Arizona State University, and Michael Scriven of Claremont Graduate University for their invaluable comments on drafts of this report and for helping to assure appropriate analytical methodology.

About the Author

Email: rudner@ericae.net

Dr. Rudner is with the College of Library and Information Services, University of Maryland, College Park. He has been involved in quantitative analysis for over 30 years, having served as a university professor, a branch chief in the U.S. Department of Education, and a classroom teacher. For the past 12 years, he has been the Director of the ERIC Clearinghouse on Assessment and Evaluation, an information service sponsored by the National Library of Education, U.S. Department of Education which acquires and abstracts articles and manuscripts pertaining to educational assessment, evaluation, and research; builds and maintains on-line databases; publishes articles and books; and provides a wide range of user services. Dr. Rudner holds a Ph.D. in Educational Psychology (1977), an MBA in Finance (1991), and lifetime teaching certificates from two states. His two children attend public school.