Fairtest's Alternative Assessment  Model

FairTest has worked with the Massachusetts Coalition for Authentic Reform in Education to develop an authentic assessment and accountability program. We also have collaborated with a coalition of organizations in Chicago to develop a similar plan. We enclose descriptions of each plan. Below, we outline key elements of these proposals and then suggest how the federal government could modify the proposed ESEA accountability provisions to meet this model.

Components of the model.

1) Classroom-based information. Evidence of teaching and learning is retained by each teacher showing assignments, student work, and the teacher's observations of the learning processes, strengths and weaknesses of the students. Teachers would be expected to use a variety of appropriate assessment methods and tools, including observations, student classwork, homework, projects, essays, tests, presentations in a variety of formats, and portfolios to summarize and evaluate the evidence of learning. These are the basis for teacher reports, qualitative and numerical, as to each student's progress in meeting state, district and school standards and goals as well as goals the teacher or student may have. The summary information about each student should be supported by the classroom evidence. [This approach is found in the Learning Record http://www.learningrecord.org and the Work Sampling System http://www.rebusinc.com.]

2) Limited standardized testing. Testing should be in literacy and numeracy only and provide one means of checking on school level information. For accountability purposes, sampling could be used. Marked discrepancies between test results and school's classroom-based information should be investigated. No major decisions about students or schools should be based on test scores alone.

3) School quality reviews. Independent, well-prepared teams periodically conduct reviews of every school. Each school first prepares a self-evaluation. Then the team visits, sitting in on classrooms, shadowing students, interviewing students and educators and parents, reviewing random samples of student work, considering academic and other aspects of school life. The team then prepares a report with recommendations, which is given to the school and is available to the public in summary and complete form. Schools respond to the review and use the review in planning improvement.

Having the data does not mean schools are able to use it. Schools set up systems to summarize this information. More importantly, they develop means through which teachers look at each other's work and student work as a means of learning how to improve curriculum and teaching.

Each school prepares an annual public report. The report summarizes information from the assessments and evaluations, and then describes how the school uses information to improve, changes in processes and results over the years, and steps to be taken to improve processes and outcomes that need improvement. It would use the variety of kinds of evidence, quantitative and qualitative, to "paint a picture" of the school.

The information on learning outcomes would provide disaggregated data by race, gender, income (poverty), English proficiency and disability. The data would be reported when the sample sizes are large enough to be meaningful and when individuals are not identifiable. School reports would have to discuss the progress of the groups for whom data is disaggregated.

School reports would also include data on grade retention, suspensions, graduation/drop out, and school climate (e.g., through surveys). Over time, schools would gather information about their graduates, from how well elementary students succeeded in high school to whether high school graduates attended college and successfully graduated to employment or civic participation or satisfaction data. The goal for school, after all, is not only academic success but life success.

The plans call for intervention in schools which clearly demonstrate they are not succeeding, according to the multiple measures, for all or for significant portions of their students. The first step is an investigation. Indeed, the Quality Review process should first include schools where current information suggests low levels of performance. A review must include: factors beyond a school's control, the resources a school has, what it does with its resources, and how it might use resources better.

The investigation and support should lead to carefully targeted and planned assistance. Where a school does not make progress even with assistance, stronger interventions should take place. However, interventions such as school reconstitution or state takeovers have been tried with at best limited success (Mary Levy of Lawyers Committee did a study for DC Voice; most efforts at reconstitution failed). Too little is known about how to make such strong interventions succeed. Therefore, states should develop and implement the stronger actions with caution and keep very close track of what does and does not work so as to learn from them.

[The examples are the Massachusetts Coalition for Authentic Reform in Education (CARE) Call for an Authentic Assessment System, on the web at http://www.fairtest.org/arn/masspage.html in short and longer versions; and the New ERA plan from a coalition of Chicago groups, on the web at http://www.pureparents.org./ERAplanAug00email.htm .]

Federal implications.

The conference committee on ESEA should rewrite the legislation to adopt the provisions described here. That is, states would be required to implement comprehensive assessment, accountability and assistance plans. Schools and districts would develop plans based on the outline above. Two cautions:
- This will take substantial time to implement. The key is further education of educators and the time and resources to do it. By the end of this authorization, schools, districts and states should have the basic structures in place and operating: quality reviews, school improvement plans and processes, and a range of data emanating from classrooms that can be used to understand school progress. Thus, in most cases interventions based on this approach are not likely during this authorization.
- It will not be cheap. While all the elements of this plan exist somewhere, we have not seen any summary research on likely costs. Nonetheless, based on informal assessments of costs, we believe the cost is feasible. More importantly, it would enable a means to develop accountability that will support development of high-quality schools with strong outcomes, not just gains on test scores. The largest costs will be professional development and paying teachers for the time to engage in the sharing and reflective processes required for genuine school improvement.

Some questions and answers about the plan:

Q: Shouldn't the current test-based accountability processes be used to identify and act on schools?

A: Only very cautiously. States can use current data to determine which schools should first undergo Quality Reviews, and it can use current data to identify which schools most need additional assistance (though this is more mystification than mystery: almost all schools serving low income children need additional assistance). The final sanctions in the current ESEA versions should not be employed in any event as they will cause more harm than good (as discussed above). Disaggregated data should be compiled as soon as it can, and it should be reported and used as part of making initial determinations for QR or assistance.

Q. Without numerical targets with firm sanctions, won't schools just ignore the process?

A. This gets to the heart of the improvement issue, and thus deserves some discussion.

First, most teachers currently working have substantial knowledge, work hard, and care. They do not always have sufficient knowledge. We believe teachers are mostly willing to address that issue, but need support and resources to increase their skills. They do not need attacks or attempts to de-skill them. Too often, current accountability is, or is perceived by educators as, a means of attack and blame, while teaching to the test is de-skilling.

If it is true that teachers are less skilled or motivated than we believe, we are in a quandary that current accountability attempts cannot solve. High-stakes testing does not improve skills or increase motivation to teach well (as distinguished from seeking short cuts for boosting test scores) or to learn more. There is no reason to believe that in a high-stakes environment, high-quality applicants will flood the teaching field, which already faces severe and growing shortages. The nation has little choice but to work with current teachers and to create positive inducements to attract new teachers. The approach to school improvement we propose would be far more likely to accomplish this goal than is high-stakes tests with numerical triggers. We would add that teachers are particularly likely to leave schools which are likely to fail to make adequate improvement on tests, which will only exacerbate the problems.

In short, school improvement cannot succeed without or against teachers, and it mostly needs more skilled teachers who actively participate in improvement efforts. This cannot be well attained by threats or fear.

The second critical element is the target: is it a target of ensuring high quality educational opportunities for all students, or will low-income and minority-group students be consigned to test preparation for "basic skills" in a process that never gets beyond such basics? Evidence clearly demonstrates that most all students can in fact learn to rather high levels in most areas, but only when provided rigorous high quality intellectual environment and opportunity. Test-based reform fails to provide those opportunities, which is why progress toward higher levels is often so slow in schools with low-achieving students (see recent reports from Chicago Consortium on School Research) and why parents who believe their children have good schools are increasingly angry about tests even when most of the children pass them. Finally, the goal of focusing only on tests seems to lead to schools which ignore too many other academic and non-academic dimensions, which deprives children of opportunities and supports they need and deserve.

The argument here is that real school improvement must proceed with teachers and must have goals worth honoring. Current accountability procedures often proceed by attacking or de-skilling teachers to focus on too-limited targets. Thus, it probably won't succeed on its own narrow terms and surely cannot succeed in providing a truly high-quality education for all students. Thus, schools and teachers may pay attention – but they will pay attention to the wrong things or to inadequate things.

The plans also provide for transparency: claims of success or progress must be backed up with public and verifiable information. That it will be public will be a spur to taking the process seriously even if one is not initially motivated to do so. That it is verifiable is a spur to accuracy, rather than to inflated claims. As it is based in student work, it will actually be far more understandable to the public than are test scores, about which most people understand very little. Samples of high-quality student work can be used to help parents, students and teachers learn what such work looks like and thus seek to attain similar levels of quality.

All that said, we also recognize that for a variety of reasons, even given adequate resources, some schools may fail to do a good job and will not improve despite publicizing the situation and providing support for improvement. FairTest does not believe such schools should be left to continue to fail their students, and the assessment and accountability plans call for stronger measures if needed. That would occur based on a variety of evidence, not simply test score gains that are of questionable validity and reliability.

Q: But won't this take forever?

A: There is no reason to believe that schools serving predominantly low-income children can provide high quality opportunity for all unless they have a major infusion of resources, and no reason to believe they can succeed unless this country develops as strong a will to solve poverty as it now has to develop tests. That, barring such changes, success is not likely for more than a relatively few.

The goal of all students reaching a level of "proficiency" equivalent to "proficient" on NAEP is not feasible in the 10-12 years called for in ESEA. This is indeed unfortunate, but it will not get better by pretending the impossible can be attained. That is, the goal proposed is itself not reachable and thus will "take forever."

Schools and districts taking the approach we propose will, we conclude, be more likely to make substantive, real progress on valuable, real-world outcomes, than will schools focusing on just boosting test scores every year. The reason is that it will tap into the energy, talent, and caring of educators and students – the human factor that a test-based approach ignores and undermines. That is not to say it will happen quickly, but that the goals will be superior and reached as quickly as happens with test-based accountability. It will happen faster if the nation decides it really means to leave no children behind and takes serious measures, in education and elsewhere. We have proposed one such serious measure. It should not be dismissed in favor of narrow and illusory test-scores.

August 1, 2001