Colorado Study Disputes LA Times Value-Added Project

Huck/Konopacki Labor Cartoons

Last year, the Los Angeles Times published LAUSD teacher rankings based on their Value-Added Measurements (VAM). One teacher was so distraught over the publication of his data that he committed suicide. It turns out that the LA Times’ analysis of the data on teacher effectiveness was “demonstrably inadequate to support the published rankings,” according to a new study done by the University of Colorado at Boulder.

According to the LA Times, the Colorado study validated their findings for the best and worst teachers, but suggested that their findings for average teachers may have been imprecise and subject to bias. In reality, when the Colorado study analyzed the data using a 95% confidence interval, they found that between 43% and 52% of teachers could not “be distinguished from a teacher of ‘average’ effectiveness.” As a result of the LA Times’ abuse of statistics, a significant number of teachers were likely misclassified.

The authors of the study also pointed out that the data could have been skewed by administrative decisions on teacher assignments. For example, teachers assigned to low income and low performing schools would have students with lower test scores on average and their students would have potentially less improvement each year. They conclude that at best the study’s estimates of teacher effectiveness are a “biased proxy for teacher quality.”

It is important to point out that an excellent teacher in a bad school could easily have a lower Value-Added score than a terrible teacher in a good school. Students in good schools may improve their test scores each year due to the privileges of family wealth and support.

It is also important to point out that there isn’t just one VAM methodology for ranking teachers—the concept is far from scientific. Different schools and districts determine the value-added by teachers in different ways. When the authors of the study used the same data, but an alternative  method of accounting, only 46.6% of reading teachers retained the same effectiveness (for math outcomes, only 60.8 teachers retained the same effectiveness rating.