Saturday, August 05, 2006

Evaluating Evaluations

Just like with any other course, at the end of a summer class students are given the chance of evaluating the performance of the instructor. The standard practice at the University of Virginia in summer classes is that students can complete their evaluations even after they have taken their final exam, or even after having received a letter grade in the course.

This is an odd practice and would not only lead to biased evaluations, but also to non-representative scores for the students.

Let's briefly analyze the incentives that this practice creates. First, because several students' opinions are going to be influenced by the grade they got in the class, this system will lead to negatively biased evaluations (say, if the instructor gave a final exam that was hard in the eyes of the average student).

Because the students are likely to respond in this way, instructors fearing bad evaluations are also likely to become more generous when writing a final exam, when grading it, or when assigning final grades for the course. (Read Freakonomics for actual data on how teachers react when their students grades are used to reflect the performance of themselves as teachers). In the end, exams or grades don't really reflect the level of knowledge of students.

Yes, stating these likely reactions makes some people mad, for they say they would never be influcenced in such a way. This is true for some people, but not for others. But why risk having such biases whenever a simple change can make things a lot better?

For classes in the economics department, the students can fill their evaluations anytime before final exams start, and instructors are not allowed to see the results of their evaluations until after final grades have been assigned. I don't think there is a perfect mechanism that would lead to no biases, but in my point of view, the one described in this paragraph strictly dominates the previous one.

It came as a surprise to me that in summer classes the office of the registrar allows evaluations to be completed even after students have received a final grade. It was far more surprising to realize that this is the standard practice used by most departments at the University during Fall or Spring semesters. The economics department is one exception, and is apparently seen as a strange being for choosing that things be done in a better way.

0 Comments:

Post a Comment

<< Home