T h e Canadian Journal of Higher Education, Vol. IX-1, 1979 Bibliographie choisie sur 1'evaluation de l'enseignement au niveau supérieur Select Bibliography of Evaluation of Instruction in Higher Education E A R L B . DAVEY* The considerable escalation in recent years of b o t h substantiated argument and uninformed opinion concerning the evaluation of instruction in higher education suggests that this issue has finally come of age. Reaction to evaluation is varied and undeniably complex. Nonetheless, in order to construct a concise analysis of the responses to the evaluation of instruction I have adopted a tri-partite categorization developed by Patricia Cross in her assessment of a related issue. The three basic positions include: (i) the proponents of evaluation — "Reformers" — who lobby for improved teaching and propose various systems of evaluation and means of facilitating the evaluation of instruction; (ii) the "Analysts", who assert that instruction in higher education is clearly in need of improvement but are, for the most part, pessimistic regarding the likelihood that effective innovation will be introduced; (iii) the "Educational Conservatives", comprised for the most part of faculty members, who embrace the traditions of instruction in the university and resist pressures for change. The responses of each * Brandon University Au fil des dernières années, nous avons témoigné une croissance du débat sur l'enseignement supérieur mené par des personnes bien renseignées ainsi que par d'autres qui ignorent le fond de la situation. Ce débat nous signale que nous sommes arrivés à l'heure de l'enseignement supérieur. Les réactions à l'évaluation sont variées et indéniablement complexes. Néanmoins, pour faire une analyse concise des réponses à l'évaluation de l'instruction, j'ai adopté une catégorisation tri-partite développée par Patricia Cross dans son évaluation d'une question connexe. Il s'agit: i) des défenseurs de l'évaluation — "Réformateurs" — ceux qui oeuvrent pour un enseignement amélioré et qui proposent divers systèmes d'évaluation ainsi que des moyens pour faciliter l'évaluation de l'enseignement; ii) des "Analystes", qui affirment que l'instruction au niveau supérieur devrait s'améliorer, mais, en grande partie, ils sont pessimistes quant à la possibilité d'innovation efficace; iii) "les Conservateurs en enseignement", surtout des membres des facultés, qui soutiennent l'enseignement traditionnel à l'université 66 Charles E. Pascal and Earl B. Davey of these groups constitute a literature of considerable quantity, which, as may be expected, is largely comprised of the contributions of "Reformers" and "Analysts". Despite the diversity of positions taken regarding evaluation, several themes relating to this issue recur throughout the literature and can be identified. First, most writers agree that the primary purposes of evaluation are twofold: to diagnose the instructor's weaknesses in order to assist in the improvement of his or her professional development (formative evaluation) and t o collect data which will be used in the decision-making process relative to promotion, salary increases and tenure (summative evaluation). Regardless of their perspective, virtually all writers affirm the primacy of the formative function, b u t , there are many who question whether the proper relation of the two functions is not, in practice, reversed. This suspicion is, to some extent, corroborated by recent research which indicates that the evaluation process has not effectively stimulated faculty development. A second issue frequently identified in the literature relates to the initiation and operation of the evaluation process. Many commentators suggest that both functions are almost entirely undertaken by members of the educational administration with the assistance of specialists whose primary interest is the evaluation and development of instruction. Faculty participation in the formation and operation of the process is minimal. Predictably, faculty response to evaluation is, in general, negative, ranging from cautious acquiescence to outright hostility. Of the various methods of evaluation frequently cited, the most vigorously debated is the role of student evaluations. Rumery, Rhodes and Johnson, for example, argue that students are not automatically et résistent aux vents de changement. Les réponses de chacun de ses trois groupes constituent une documentation importante — surtout des apports des "Réformateurs" et des "Analystes". Malgré la divergence d'opinion concernant l'évaluation, plusieurs thèmes reliés à cette question reviennent toujours dans la littérature sur le sujet et peuvent être identifiés. Premièrement, la plupart des rédacteurs avouent qu'il existe un double objectif de base dans l'évaluation: a) diagnostiquer les faiblesses de l'enseignant pour aider celui-ci à parfaire son développement professionnel; et, b ) recueillir des données qui serviront à la prise de décisions relatives aux promotions, aux augmentations de salaire et aux durées d'engagements. Peu importe leur perspective, on pourrait dire que tous les rédacteurs mettent l'accent sur a). Toutefois, bien d'entre eux se demandent si, en réalité, l'objectif visé n'est pas plutôt b). Ce soupçon est confirmé en partie par de récentes recherches indiquant que le processus d'évalution n'a pas efficacement stimulé la mise en valeur des talents du corps enseignant. Il existe une deuxième question qui survient fréquemment dans la documentation reliée à l'initiation et à l'exploitation du processus d'évaluation. Bien des rédacteurs suggèrent que les deux fonctions sont presque entièrement assumées par les membres de l'administration académique avec l'aide des spécialistes dont l'intérêt principal est l'évaluation et la mise en valeur de l'enseignement. La participation du corps enseignant à la mise en place ainsi qu'à la mise en valeur du processus est minimale. Ainsi, comme on peut l'imaginer, la réaction du corps enseignant à l'évaluation est, en général, négative — allant d'un acquiescement circonspect jusqu'à une hostilité sans ambages. De toutes les méthodes d'évaluation fréquemment citées, celle dont on discute le 67 The Politics of Evaluating Teaching competent evaluators or even competent observers. Others maintain that students as consumers are primary and logical agents for evaluating instruction. Further to this issue, if the findings of Cross (1976) and Hildebrand (1972) are generalizable, this debate may be of little substance; for they report that students are "notoriously generous raters", superceded only by faculty and their benevolent assessment of each other. Few disagree that " s o u n d " evaluation procedures could yield significant benefits. The task at hand is to eliminate the multiple hazards of inadequately designed evaluation systems and to formulate a process which can facilitate the development of instruction and gain the confidence of the university's most essential component — the faculty. plus vigoureusement est l'évaluation menée par des étudiants. Par exemple, Rumery, Rhodes et Johnson soutiennent que les étudiants n'ont pas nécessairement la compétence de répondre soit en tant qu'évaluateurs ou tout simplement en tant qu'observateurs. D'autres, par contre, soutiennent que les étudiants, en tant que consommateurs, sont les évaluateurs essentiels et logiques de l'enseignement. Si les résultats des recherches de Cross (1976) et de Hildebrand (1972) peuvent être généralisés, ce débat aura peu de substance. Ils concluent que les étudiants s'avèrent des "évaluateurs fort généreux" du corps enseignant. Leur générosité n'est surpassées que par les évaluations du corps enseignant à son propre égard, et par les évaluations des étudiants envers leurs confrères et consoeurs. On est presque unanime en concluant qu'un processus " s û r " d'évaluation offrirait des bénéfices significatifs. Il s'agit d'éliminer les dangers multiples des méthodes inadéquates d'évaluation et de mettre au point un processus qui faciliterait la mise en valeur de l'enseignement et qui jouirait de la confiance du composant principal de l'université — le corps enseignant. " S t a t e m e n t on teaching e v a l u a t i o n . " ^ OTBulletin 6 1 ( 2 ) 200-2 (Aug. '75) Aleamoni, L.M. " T h e usefulness of student evaluations in improving college teaching," m Proceedings of the first invitational conference on faculty effectiveness as evaluated by students. Edited by A. Sockloff. Philadelphia, Temple University, Measurement and Research Center, 1973. Astin, A.W., and Lee, C.B.T. "Current practices in the evaluation and training of college teachers." Improving College Teaching, 1967. Baker, E.L., and Alin, M.C. "Formative evaluation of instructional development." A-V Review 21, 389-418 ( 1 9 7 3 ) Communication Batista, Enrique E. " T h e place of colleague evaluation in the appraisal of college teaching: A review of the literature." Research in Higher Education 4 (3) 257.-71 ('76) Bejar, 1.1., and Doyle, K.O. Generalizability of factor structures underlying University of Minnesota, Measurement Services Center, 1974. student ratings of instruction. Bentley, G.E. " R e p o r t of the President's c o m m i t t e e for a pilot study in student evaluation at Princeton," in The recognition and evaluation of teaching. Edited by K.E. Eble, Washington, D.C., American Association of University Professors, 1970. Blaney, J. Developing and evaluating instruction in continuing education: A guide to basic concepts and procedures. Vancouver, Centre for Continuing Education, University of British Columbia, 1974. 68 Charles E. Pascal and Earl B. Davey Borich, Gary D. The appraisal of teaching: 1977. Concepts and process. Don Mills, Ontario, Addison-Wesley, Braunstein, D.N., and Benston, G.J. " S t u d e n t and department chairman views of the performance of university professors." Journal of Applied Psychology 58, 244-249 ('73) Centra, J.A. "Self-ratings of college teachers: A comparison with student ratings." Journal Educational Measurement of 10, 287-295. 1973 "The h o w and why of evaluating teaching." New Directions Cohen, A.M., and Brawer, F.B. Measuring faculty for Higher Education performance. 17, 93-106 (Spr '77) Washington, D.C. ERIC Clearinghouse for Junior Colleges, American Association of Junior Colleges, 1969. Cook, J.M., and Neville, R.F. The faculty as teachers: A perspective on evaluation, report no. 13. Washington, D.C., ERIC Clearinghouse on Higher Education, 1971. Costin, F. Intercorrelations between students' and course chairmen's ratings of instructors. Urbana, 111., University of Illinois, Division of General Studies, 1966. Costin, F., Greenough, W.T., and Menges, R.J. " S t u d e n t ratings of college teaching: Reliability, validity and usefulness." Review of Educational Research 41, 511-535. 1971. Crawford, P.L., and Bradshaw, H.L. "Perceptions of characteristics of effective university teachers: A scaling analysis." Educational and Psychological Measurement 28, 1079-1085. 1968 Cronbach, L.J. "Course improvement through evaluation." Teachers College Record 64, 672-683. 1963 Cross, K. Petricia, " N o t can, but will college teaching be i m p r o v e d ? " New Directions for Education Higher 17, 1-15, 1977. Curry, Jerome. " S t u d e n t s - generally lack the critical ability necessary for faculty evaluation." College Student Journal 10 (4) 306-311 (Win. '76) Doyle, K.O. "Faculty evaluation: Some considerations and a m o d e l , " in Proceedings of the first invitational conference on faculty effectiveness as evaluated by students. Edited by A. Sockloff. Philadelphia, Temple University, Measurement and Research Center, 1973. Dressel, P.L. Evaluation in higher education. Boston, Houghton Mifflin, 1961. " S t u d e n t evaluation of the faculty: Why? What? H o w ? " in Proceedings of the first invitational conference on faculty effectiveness as evaluated by students. Philadelphia, Temple University, Measurement and Research Center, 1973. Eble, Kenneth E. The recognition and evaluation of teaching. Washington, D.C., American Association of University of Professors, 1971. Feldhusen, J., and McDaniel, E. "Using a model of instruction as the base for course and instructor evaluation." Paper presented at the meeting of the American Psychological Association, Honolulu, August 1972. Fletcher, O.J., and Welser, J.R. "Evaluation of instructional m e t h o d s for teaching veterinary students the c o m p o n e n t s of i n f l a m a t i o n . " Journal of Veterinary Medical Education 2 (2) 2-6 (F '75) Frey, Peter W. "Validity of student instructional ratings: Does timing m a t t e r ? " Journal Education of Higher 47 (3) 327-336 (May/June '76) Gadzella, B.M. "College student views and ratings of an ideal professor." College and University 44, 89-96. 1968. Gage, N.L. "The appraisal of college teaching." Journal of Higher Education Geis, G.L., and Rogers, J. Characteristics of effective Gessner, P.K. "Evaluation of Instruction." Science, instruction. 32, 17-22. 1961 Montreal, G / P Associates, 1975. 1973. Glaser, R. "Evaluation of instruction and changing education models." Proceedings of the symposium on problems in evaluation of instruction (December 1967). Los Angeles, University of California, Center for the Study of Evaluation of Instructional Problems, 1968. Glasman, Naftaly S. "Evaluation of instructors in higher education: An administrative f u n c t i o n . " Journal of Higher Education 47 (3) 309-326 (May/June '76) G o o d , Harold M., and T r o t t e r , Bernard. "Accountability for effective and efficient university teaching." The Canadian Journal of Higher Education 4 (1). 1974 69 The Politics of Evaluating Teaching Granzin, K.L., and Painter, J.J. " A new explanation for students' course evaluation tendencies." "American Educational Research Journal 10, 115-124. 1973. Gray, C.E. "The teaching model and evaluation of teaching p e r f o r m a n c e . " Journal of Higher Education 40, 636-643. 1969 Gustad, J.W. "Evaluation of teaching p e r f o r m a n c e : Issues and possibilities." Improving Teaching, 1967. Guthrie, E.R. The evaluation College of teaching: A progress report. Seattle, University of Washington, 1954. Haich, George D. "Developing a student questionnaire for teacher evaluation." Teaching English in the Two-Year College 2 (3) 163-166 (Spr '76) Halstead, J.S. "Model for research on ratings of courses and instructors." Paper presented at t h e meeting of the American Psychological Association, August 1970. Hildebrand, M. " H o w to r e c o m m e n d p r o m o t i o n for a mediocre teacher w i t h o u t actually lying." Journal of Higher Education, 4 3 , 4 4 - 6 2 , 1972. Hildebrand, M., Wilson, R.C., and Dienst, E.R. Evaluating University Teaching. Berkeley: Centre for Research and Development in Higher Education, University of California, 1971 Hildebrand, M., and Wilson, R.C. Effective university teaching and its evaluation. Berkeley, Center for Research and Development in Higher Education, 1970. Hocking, Joan M. "College students evaluations of faculty are directly related to course interest and grade e x p e c t a t i o n . " College Student Journal 10 (4) 312-316 (Win. '76) Hogan, T., and Hartley, E.L. " S o m e additional factors in student evaluation of courses." Educational Research Journal American 9, 241-250. 1972. Holmes, D. "The teaching assessment blank (TAB): A form for the student assessment of college instructors." Journal of Experimental Education 39, 34-38. 1971 Housego, I. The assessment of teacher effectiveness. R e p o r t of the Faculty of Education C o m m i t t e e on t h e Assessment of Teacher Effectiveness. Vancouver, University of British Columbia, 1972. Hoyt, D.P. " T h e Kansas State University program for assessing and improving instructional effectiveness," in Proceedings of the first invitational conference on faculty effectiveness as evaluated by students. Philadelphia, Temple University, Measurement and Research Center, 1973. J o h n s o n , Henry C. Jr. (et al). " T h e assessment of teaching in higher education: A critical retrospect and a proposal. Part II: A proposal." Higher Education 4 (3) 273-303 (August '75) Justiz, T.B. " A m e t h o d of identifying the effective t e a c h e r . " Doctoral thesis, University of California, 1969. Kent, L. " S t u d e n t evaluation of teaching." improving College Teaching, 1967. Kerlinger, F.N. " S t u d e n t evaluation of university professors." School and Society 99, 353-356. 1971 Klein, S.P., and Alkin, M.C. "Evaluating teachers for o u t c o m e a c c o u n t a b i l i t y . " Evaluation Comment 3 (30) 5-11. 1972 Knapper, C., McFarlane, B., and Scanlon, J. " S t u d e n t evaluation: An aspect of teaching effectiveness." CAUT Bulletin 21 (2) 26-34. 1972 Knapper, C., Geis, G.L., Pascal, C.E., and Shore, B.M. (Eds.) Scaling the ivory tower: teaching in colleges and universities. T o r o n t o , Clarke-Erwin, 1975. Appraising Kohlan, R.G. "A comparison of faculty evaluations early and late in the course." Journal Education of Higher 14, 587-594. 1973 Levenson, H., and Brooks, D.W. " S t u d e n t evaluation of lectures versus graduate student laboratory instructors in i n t r o d u c t o r y college chemistry." Journal of College Science (Nov. '75) Teaching 5 (2) 85-88 Mansour, George P. " A n eclectic approach to assessing T.A. effectiveness." Bulletin of Departments of Foreign Languages of the Association 8 (4) 24-26 (May '77) McClelland, J.N. " T h e effect of student evaluations of college instruction u p o n subsequent evaluations." California Journal of Educational Research 21, 88-95. 1970 70 Charles E. Pascal and Earl B. Davey McKeachie, W.J., Lin, Y., and Mann, W. " S t u d e n t ratings of teaching effectiveness-. Validity studies." American Educational Research Journal 8, 435-445. 1971 McKee, B.K., and Michaelson, L.W. " S t u d e n t evaluation of college teachers as a judgment factor by department heads and administrators." Journal of English Teaching Techniques 9 (1 and 2) 91-101 (Spr-Sum '76) Miller, M.T. "Instructor attitudes toward, and their use of, student ratings of teachers." Journal Educational Psychology Miller, R.I. Evaluating faculty Developing of 62, 235-239. 1971 performance. programs for faculty evaluation. San Francisco, Jossey-Bass, 1972. San Francisco, Jossey-Bass, 1974. Mody, R a y m o n d . " S t u d e n t achievement and student evaluations of teaching in Spanish." Modern Language Journal 60 (8) 454-463 (Dec '76) M o o m a n , W. Edward. "Practices and problems in evaluating instruction." New Directions for Education. Higher 17, 77-91 (Spr '77). Morris, Victor D. " A positive approach to t h e utilization of student feedback in medical e d u c a t i o n . " Journal of Medical Education 51 (7) 541-549 (Jul '76) Murray, H.G. A guide to teaching evaluation. T o r o n t o , Ontario Confederation of University Faculty Associations, 1973. (a) Nadeau, G.G. " S t u d e n t evaluation of college teaching-state of research in the field." Paper presented at the Learned Societies, Kingston, May 1973. Nadeau, G.G. "Implementing student evaluation of instruction." Paper presented at the ACFAS, Quebec City, May 1974. "Attitudes, evaluation and exam practices of faculty in a small university." Paper presented at AERA-NCME Meeting, Washington, April 1975. "Validity of student ratings of instruction: Validity for what purpose and what k i n d . " Paper presented as part of a symposium on "Validity of S t u d e n t Ratings", A E R A Meeting, Washington, April 1975. Norr, J.L., and Crittenden, K.S. "Evaluating college teaching as leadership." Higher Education 317-333 (Aug '75) P a c e , C. R o b e r t . Evaluating learning and teaching. 4 (3) Sun F r a n c i s c o , J o s s e y - B a s s , 1 9 7 3 . Perry, R . R . "Evaluation of teaching behavior seeks to measure effectiveness." College and University Business 47, 18-22. 1969 Perry, R.R., and Baumann, R.R. "Criteria for evaluation of college teaching: Their reliability and validity at t h e University of T o l e d o . " Proceedings of the first invitational conference on faculty effectiveness as evaluated by students. Philadelphia, Temple University, Measurement and Research Center, 1973. Pohlmann, J.T. "A description of effective college teaching in five disciplines as measured by student ratings." Research in Higher Education 4 (4) 335-346. 1976 P o p h a m , J.W. Designing teacher evaluation systems: A series of suggestions for establishing teacher assessment procedures as required by the Stull Bill (AB 293), 1971 California Legislature. Los Angeles, T h e Instructional Objectives Exchange (Box 24095), 1971 (a). Remmers, H.H. "On students' perceptions of teachers' effectiveness," in The appraisal of teaching in large universities. Edited by W.J. McKeachie. Ann Arbor, Michigan, University of Michigan Extension Service, 1959. Roberts, K.L. Proposal for the administration in teaching. and processing of the survey of student opinion of L o n d o n , Ontario, University of Western Ontario, C o m p u t e r Center, 1973. Rodin, M„ and Rodin, B. " S t u d e n t evaluation of teachers." Science 177, 1164-1166. 1972 Roid, G.H. "Towards a system of course evaluation." Learning and Development Roid, G.H. System design and course evaluation. and Development, 1972. 2 (6) 1-5. 1971. Montreal, McGill University, Centre for Learning ability 71 The Politics of Evaluating Teaching R o t e m , A., and Glasman, N.S. "Evaluation of university instructors in the United States: The c o n t e x t . " Higher Education 6 (1) 75-92 (Feb '77) R u m e r y , R.E. (et al). " T h e role of student reports in the evaluation of teaching in higher e d u c a t i o n . " Higher Education Bulletin 3 (2) 93-99 1975. Schuh, A.J., and Crivelli, M.A. "Animadversion error in student evaluations of faculty teaching effectiveness." Journal of Applied Psychology 5 8 , 2 5 9 - 2 6 0 . 1973. Scott, R.O., and Yelon, S.L. " T h e student as co-author: The first step in formative evaluation." Educational Technology Scriven, M. The methodology 9, 76-78. 1969 of evaluation. (AERA Monograph Series No. 1) Chicago, Rand McNally, 1967. Sharon, A.T. "Eliminating bias f r o m student ratings of college instructors." Journal of Applied chology Psy- 54, 278-281. 1970 Sheffield, E.F. "Characteristics of effective teaching in Canadian universities: An analysis based on the testimony of a thousand graduates." The Canadian Journal of Higher Education 4 (I). 1974 Sherman, T.M. "Trick or trait: A look at student evaluation of instruction." Educational Technology 16 (4) 38-40 (Apr '76) Smock, R., and Crooks, T. "A plan for the comprehensive evaluation of college teaching." Paper presented at the meeting of t h e American Educational Research Association, Chicago, February 1973. Sockloff, Alan L, (Ed.). Proceedings: Faculty effectiveness as evaluated by students. Philadelphia, Temple University, Measurement and Research Center. 1973 Sorey, K.E. " A study of the distinguishing personality characteristics of college faculty w h o are superior in regard to the teaching f u n c t i o n . " Dissertation Abstracts 28, 4 9 1 6 . 1968 Spencer, R.E., and Aleamoni, L.M. " A student course evaluation questionnaire." Journal of Educational Measurement 7, 209-210. 1970 Stake, R.E. "The c o u n t e n a n c e of educational evaluation." Teachers College Record 68, 523-540. 1967 T o u q , M.S., and Feldhusen, J.F. " T h e relationship between student ratings of instructors and their participation in classroom discussion." Paper presented at t h e meeting of the National Council on Measurement in Education, New Orleans, February 1973. Toug, M.S., Feldhusen, J.F., and Halstead, J. "Criterion referenced validity of student ratings of instructors." Paper presented at the annual meeting of the American Educational Research Association, New Orleans, February 1973. T r u e x , M.H. " F a c t o r s critical to college teaching success or failure." Improving College and University Teaching 23 (4) 236-238 (Fall '75) T u c k m a n , B.W., and Oliver, W.F. "Effectiveness of feedback to teachers as a function of source." Journal of Educational Psychology 69, 297-301. 1968 Tyler, R.W. "The evaluation of teaching," in The two sides of the log. Edited by R.M. Cooper. Minneapolis, University of Minnesota Press, 1958. University of Ottawa. The evaluation of academic personnel at the University of Ottawa. (Final Report of the Senate Ad Hoc C o m m i t t e e on t h e Evaluation of Teaching.) O t t a w a , March 1974. Villano, M.W., Rosenstock, E.H., and Estes, C. "A decade with a student evaluation form at a major university." Paper presented at the meeting of the American Educational Research Association, Chicago, April 1974. Wallace, C.E. "Self-appraisal: Systems for i m p r o v e m e n t . " California School Boards 31 (14). 1972 Warrington, W.G. " S t u d e n t evaluation of instruction at Michigan STate University." Proceedings: Faculty effectiveness as evaluated by students. Philadelphia, Temple University, Measurement and Research Center, 1973. Webb, W.B., and Nolan, C.Y. " S t u d e n t , supervisor, and self-ratings of instructional p r o f i c i e n c y . " Journal of Educational Psychology 46, 42-46. 1955 72 Charles E. Pascal and Earl B. Davey Whitely, S.E., and Doyle, K.O. Student ratings and criteria for teaching effectiveness. Minnesota, Measurement Services Center, 1973. Dimensions of effective Center, 1974. teaching: Factors or artifacts. University of University of Minnesota, Measurement Services Widlac, F.W., McDaniel, E.D., and Feldhusen, J.F. " F a c t o r analysis of an instructor rating scale." Paper presented at the meeting of t h e American Educational Research Association, New Orleans, February 1973. Wilson, R.C., Dienot, E.R., and Watson, N.L. "Characteristics of effective college teachers as perceived by their colleagues." Journal of Educational Measurement 10, 31-38. 1973 Wittrock, M.C., and Wiley, D.E. (Eds.) The evaluation Holt, Rinehart and Winston, 1970. of instruction: Issues and problems. New York,