Value-added assessment is hard, let’s go shopping!

16Mar11

An article in the New York Times last week looked at the statistical model for evaluating teachers known as value-added assessment. Value-added assessment is a way of assessing student performance by comparing a student’s current performance to their past performance to see how much they have improved; this is usually assessed using standardized tests. The belief is that since a teacher is responsible for most of the learning a student will have done within a given year, the value-added assessment is a way of determining how effective a teacher is at instilling knowledge in their students within a given year.

However, the article makes a couple points pretty clear. First, the statistical analysis is far from trivial.

As you can see, this equation is pretty complicated, and could potentially contain hundreds of variables. This is not a bad thing in itself, but for any statistical model like this to be valid, it must contain the correct quantity and quality of data. I’m sure with all those test scores, the quantity is not an issue. It’s the quality that I worry about.

While it seems they’ve taken some factors into account (such as “Student Characteristics” and “True Total Classroom Effect”), within those single factors are hundreds of hidden variables that aren’t taken into account: whether a student is tired or stressed that day, how much studying they do on their own, whether they’re eating properly, what their early childhood was like and many other variables that are completely unrelated to their classroom teacher.

The problem with using this statistical evaluation as the sole determinant of whether a teacher gets tenure or not is that it requires an immensely complicated model that attempts to embody a teacher’s sole worth within a single number. This evaluation can never be nuanced enough to contain the all-encompassing value of a teacher, and it will also be subject to corruption pressures, as embodied by Campbell’s Law:

“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

As a quantitative indicator becomes increasingly important, people will attempt to cheat the system and the quantitative indicator will lose it’s value; it will no longer measure what you think it is measuring. One way around this is to minimize the importance of any single quantitative indicator, as measures in the social sciences can never be as precise one would like; it’s hard to describe things in social sciences with just a number.

Which brings me to the second point. According to the article, this teacher is generally agreed by her peers, supervisors, students and parents to be outstanding. However, the numbers don’t agree with her, and she may lose her job as a result. This is the problem with using quantitative measures to make decisions in teaching (I don’t want to generalize to other social sciences, although it may be true there as well), as I described above. Unfortunately, this type of teacher evaluation appears to be the trend with teacher and school evaluations in the United States and (increasingly) in Canada.

Advertisements


No Responses Yet to “Value-added assessment is hard, let’s go shopping!”

  1. Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: