Who's a Good Teacher?

Walt Gardner's EDUCATION WEEK'S Reality Check

With pressure mounting to assure that all students have a highly qualified teacher, attention is increasingly focusing on the use of the value added model. Its supporters maintain that since it measures the progress that students make on standardized tests, rather than the proficiency they reach, there is no incentive to teach in affluent suburban schools. In fact, the likelihood of receiving the designation of effective is greater in poor inner-city schools because it is easier for teachers to demonstrate gains for those at the very bottom than for those already at the very top.

Nevertheless, a contentious debate is underway about the fairness and accuracy of the VAM. The latest venue is the District of Columbia, where Mayor Vincent Gray expressed concern. He argues that evaluating teachers who teach students from chaotic backgrounds unavoidably puts them at a distinct disadvantage compared with teachers who teach more fortunate students. He takes this position even though D.C. schools under former Chancellor Michelle Rhee put in place IMPACT, a teacher evaluation system that uses the controversial VAM.

Taxpayers are entitled to know if students are being well educated. But they won't find satisfactory answers when the issue is compounded by claims made by dueling experts. Nevertheless, it's important to bear in mind that it's common for educational investigators to look at the same data and reach opposite conclusions. After Milwaukee became the site of the nation's first voucher school program, for example, the Wall Street Journal published a front-page story in Oct. 1996 about the public spat between highly regarded professors over the relative merits of the plan.

This same kind of confusing brawl was on display in Aug. 2010 when ten scholars convened by the Economic Policy Institute issued a report titled "Problems With The Use of Student Test Scores To Evaluate Teachers." They cautioned that the VAM has proved to be "unstable across ... years and classes that teachers teach." As a result, test scores alone are not sufficiently reliable and valid indicators for use in high-stakes personnel decisions.

They cited one study finding that fewer than a third of the teachers who were ranked in the top 20 percent of effectiveness in the first year in five large urban districts were in that top group the second year, and another third dropped all the way down to the bottom 40 percent. It's hard to believe that these teachers became so dramatically ineffective in one year. Clearly, something else was going on that accounted for the changes.

What was happening in all likelihood was that these same teachers happened to inherit students with fundamentally different characteristics. When this happens, outcomes can vary widely among even the best teachers. Lesson plans that were successful for years with students from similar backgrounds suddenly flopped with students from different backgrounds.

Even when steps are taken to control for student demographics, teachers who are assigned non-English speaking students, low-income students, and special education students have been found to receive lower ratings than teachers who teach affluent students from educated homes. This disparity is not supposed to happen under the VAM because progress is the basis for evaluation. But studies so far have shown otherwise.

So what can be done to determine who is a good teacher? Test scores can be used as one of many measures. For example, New York State in 2010 passed a law stipulating that by 2013, 25 percent of a teacher's evaluation will be based on a value-added system. Other states will undoubtedly follow suit.

Whether the weight given to standardized tests should be higher or lower is a matter for debate. But overreliance on such tests is bound to backfire. If teaching is ever going to be viewed as a profession, as it is in Finland, then the U.S. has to confront its testing obsession. It is slowly sucking the intellectual life out of the classroom.