Are Student Attributions of Instructor Racism and Sexism on Course Evaluation Forms Valid?

Stanley Coren
Psychology
University of British Columbia

We are now entering the second year of using our new teaching evaluation forms here in the Psychology Department of the University of British Columbia. When the current student course evaluation forms were introduced, some faculty members were concerned about the addition of several items which contained political or social content. Specifically three items asked the class about whether, in the eyes of the students, the instructor had displayed any apparent cultural bias, racism or sexism. I now have some data that I think should give us pause about the usefulness of this type of student opinion survey.

Let me start by giving a brief bit of background. I have always had two basic concerns about the usefulness of student responses for evaluating the classroom performance of instructors, especially if such evaluations involved considerations dealing with social or political attitudes and behaviors. My first concern had to do with the likelihood that students would be unable to separate the message from the messenger (the fundamental attribution error). My second fear was that the student's global feelings about the course would carry over and influence his or her responses to the more specific "politically sensitive" questions. If this were true it would cause a pedagogically poor instructor to also be down-rated on these nonpedagogical items (the halo effect).

The fundamental attribution error is a well-known phenomenon in Social Psychology. Research on this problem began with the psychologist Fritz Heider (1958). He basically was able to show that we undervalue the likelihood that a person's behavior is due to outside or situational factors, and are more likely to ascribe the behavior to internal (personality or belief driven) factors. The classic experimental demonstration of this comes from a study by Jones and Harris (1967) who had students read a debater's speech that either supported or attacked Fidel Castro, Cuba's leader. The students were explicitly told that the debaters had been randomly assigned by the debating team coach to argue for or against Castro. In other words, no debater had a choice as to the position that they were to represent. In spite of this knowledge, when the students were asked to estimate the debaters' actual attitude towards Castro, they concluded that the debaters' actual beliefs were closely in accord with the position that they argued for. Thus a student who argued in favor of Castro must be doing so because he or she really does like Castro's politics and actions.

In a recent empirical study (Coren, 1993), I showed that this effect holds for instructor evaluations of racism or sexism. For example, 198 students were given the scenario:

Professor X gives a lecture about intelligence. In it he describes some evidence for biological factors, such as genes, that effect intelligence. He suggests that although culture and experience are important in determining scores on intelligence tests, genetic factors can explain some portion of the difference in IQ scores obtained when different races take intelligence tests.

Notice that in this item Professor X suggests that there is a genetic contribution to intelligence. Although he acknowledges contributions from culture and environment, he concludes that the genetic contribution to intelligence might account for "some portion" of the differences observed in IQ scores between the races. When asked what the professor's motives might have been for presenting this material, nearly one out of every four of the students (24%) specifically mentioned "racist," "racism," or notions of "racial superiority" as motivating the presentation of this material. Thus the very discussion of genetic and racial differences in intelligence, if the conclusion is that they exist, renders the lecturer a racist in the minds of nearly one quarter of these students. These opinions probably represent an instance of the fundamental attribution error. This same study showed that a lecture which presented data showing sex differences in spatial ability raised charges of sexism in about one third of the students.

My second concern had to do with halo effects. Halo effects are well known in psychology. The concept is used to refer to the fact that if an individual is viewed as having some good qualities there is a bias towards assuming that the person in question has all good qualities. This assumption might lead a teacher to assume that an attractive child who is well behaved in class is also serious, hardworking, and intelligent--in other words, an all around "good student." The flip side of the coin is that a person who is seen as being "bad" in one area is likely to be judged "bad" across a broad spectrum of behaviors. While data like this has been known since the classic work of Solomon Asch (e.g., 1946), there are many more recent empirical demonstrations of it. Thus Nisbett and Wilson (1977) found that a person who acted in a socially "cold" manner to experimental subjects was not only rated as having a "cold" and "negative" personality, but was also rated as less physically attractive than a person who acted in a "warm" and "outgoing" manner.

The question of halo effects in the context of student evaluations of instructors can then be viewed as a question of whether student responses to questions of racism or sexism in the classroom are truly independent of whether the students generally like or dislike the course. To test this hypothesis I recently collected 248 course evaluation forms. These forms were actually submitted to the department as part of our annual evaluation process and were volunteered by one instructor. These forms came from Introductory Psychology classes. Because of the possibility that different instructors might produce different patterns all 248 of the responses were taken from the same instructor, meaning that the "stimulus" that every student is evaluating (namely the instructor's actual lectures) would be constant. The results provide a pattern of support suggesting that halo effects do appear, and students' evaluations of classroom matters do seem to generalize to their evaluations of social and political items.

I have tabled the results below. To summarize the results briefly, all of the correlations noted are significant at p<0.001 or better. The pattern is quite clear. If students do not like the course then they are more likely to rate the instructor as culturally biased, sexist and racist. Not being available at office hours, not encouraging student participation, not explaining course requirements clearly, not showing interest in the subject matter, or even using a low-rated textbook are all correlated with the instructor being labeled biased, sexist and racist. Remember that all of the responses analyzed were from the same instructor, which means that some alternate interpretation, such as "racist instructors are also bad teachers" is not possible in this context.

At the time that these items were added to the course evaluation forms, concerns that students could not (or would not) make distinctions between academic content and political content were raised. We were assured by people who favoured inclusion of such "classroom climate questions," that students had the ability to separate matters of attitude from those of content. However, it turns out, as any social psychologist might have been able to predict from already published experimental data, that students cannot do this. This current analysis of actual classroom responses suggests that if the students do not like an instructor's teaching style, class organization or even the course textbook, when given the opportunity to do so, they are likely to label that instructor as racist, sexist and culturally biased. I suppose it is time to ask, "Do we really need such items on our evaluation form?" What is gained by having them? Is it not possible for a person to be a bad instructor or to have a bad fit with a particular class, but still be unbiased concerning racial or gender issues?

There are some distressing implications that may be drawn here. If, as an instructor, you present data on individual differences, and if you accede to the possibility that some politically unpopular conclusions may be scientifically valid (e.g., males and females differ in behavioral or cognitive realms, or individuals of different races may have different capacities due to genetic causes) you are opening yourself to charges of racism and/or sexism due to the fundamental attribution error. Furthermore, if you are not a very popular instructor, if you are disorganized, if you give difficult exams, chose an unlikeable textbook, or if you are not a charismatic lecturer, you again open yourself to charges of racism and sexism, simply because such items appear on the student evaluation forms. If students are making negative comments on your course organization and your classroom presentation, it is likely that the operation of the halo effect will cause students to generalize their poor ratings to any questions with social and political content. Obviously, if enough students rate you poorly on questions of racism or sexism, this can lead to formal institutional charges of discrimination. Because of this, the inclusion of such items on course evaluation forms should give members of the academy cause for concern.

Taken together, my formerly published data and the analysis of this new set of data suggest that student judgments of the political and social attitudes of their instructors may be confounded with their attitudes towards the instructor's teaching style, and with the nature of research results reported in lectures. Consequently I think it is reasonable to suggest that items dealing with the apparent social and political actions of the instructor not be included as part of formal teacher competency evaluations. The issue of racism and sexism in the classroom is too important to be left to assessments by something as demonstrably flawed and confounded as these student questionnaire items appear to be.

References

Asch, S.E. (1946). Forming impressions of personality. Journal of Abnormal and Social Psychology, 41, 258-290.

Coren, S. (1993). When teaching is evaluated on political grounds. Academic Questions, 6, 73-79.

Heider, F. (1958). The psychology of interpersonal relations. New York: Wiley.

Jones, E.E. & Harris, V.A. (1967). The attribution of attitudes. Journal of Personality and Social Psychology, 3, 124.

Nisbett, R.W. & Wilson, T.D. (1977). The halo effect: Evidence of unconscious alteration of judgments. Journal of Personality and Social Psychology, 35, 250-256.