As we head into the annual salary and tenure and promotion evaluation processes it is worthwhile to review something of what is known about the use of SET scores in the teaching evaluation process. Attached (to this page) are some resources that instruct the reader on the severe limitations of SET scores in the process of evaluation of teaching performance. Professor Phillip Stark, Professor of Statistics, UC Berkley, and his colleagues have analyzed the value of SET scores as they are widely relied upon across academic institutions for salary raises, promotion and tenure decisions and, even for the firing of non-tenured faculty. He summarizes his careful evaluation of SET scores in the evaluation process in the title of his talk on the subject: “Teaching Evaluations: Biased Beyond Measure.” The Youtube of his talk on the subject can be found here: https://www.youtube.com/watch?v=ZgJVShVYh0w&t=1239s.
For those who do not want to spend a worthwhile one hour, eighteen minutes and 52 seconds with Professor Stark’s talk, which has some very amusing moments, there are other, less time consuming ways to get his message. Attached are a couple of articles on the subject that are very useful. A brief (four minutes and 22 seconds) Youtube presentation of a study by one of Professor Stark’s graduate students, Kellie Ottononi, and a colleague identify systematic gender biases using two distinct data sets. Male professors are found to get more favorable ratings by students than female professors. Her presentation can be found here: https://www.youtube.com/watch?v=HbB_2aRHB8o.The message from these and other such rigorous analyses of the use of SET scores in assessing teaching is: don’t.
However, here at WSU, as in so many universities and colleges, we mandate their use. The Union is at fault here, as well as the Administration, in that the Collective Bargaining Agreement (CBA) mandates the use of SET scores in the salary and other evaluation processes. The CBA also mandates peer evaluations of teaching, but these are more difficult to integrate into that process. It is much more convenient for both faculty and administrators to rely on the SET scores, rather than figuring out how to apply peer evaluations. There is a joint committee at work on the SET process trying to improve the use of such scores in faculty evaluations.
The advice of the Union to the salary and tenure and promotion committees is to proceed with caution in the evaluation of colleagues’ teaching performance and, at least, to cast a cold eye on SET scores.