Teaching Excellence and How It Is Awarded: A Canadian Case Study Janice Miller-Young University of ALBERTA Melina Sinclair University of ALBERTA Sarah Forgie University of ALBERTA Abstract Quality teaching and how to assess and award it, continue to be an area of scholarship and debate in higher education. While the literature demonstrates that assessment should be multifaceted, operationalizing this is no easy task. To gain insight into how teaching excellence is defined in Canadian higher education, this empirical study collected and analysed the criteria, evidence, and standards for institutional teaching awards from 89 institutions and 204 award programs across Canada. The majority of awards included criteria such as specific characteristics of teaching performance and student-centredness; while activities that had impact outside an individual’s teaching practice were also prevalent, including campus leadership, scholarship of teaching and learning, and contributions to curriculum. Lists of potential sources of evidence were heavily weighted towards student perceptions and artefacts from instructors’ teaching. Recommendations for individuals and institutions wanting to foster excellence in teaching are offered along with suggestions for future research. Keywords: teaching, awards, excellence, assessment, criteria, evidence, standards Résumé L’enseignement de qualité, et les manières de l’évaluer et de le récompenser, continuent d’être un sujet d’étude et de débat dans le domaine de l’enseignement supérieur. Bien que la littérature démontre que l’évaluation devrait être multidimensionnelle, la mise en pratique de la théorie n’est pas une tâche facile. Afin de mieux comprendre comment on définit l’excellence en enseignement dans l’éducation supérieure canadienne, cette étude empirique a rassemblé et analysé les critères, les justificatifs et les normes des prix d’enseignement dans 89 établissements d’éducation supérieure au Canada et pour 204 programmes de prix. La majorité des prix avaient pour critères les caractéristiques de l’enseignement et l’accent mis sur les étudiants. Les activités ayant un impact en dehors de la pratique d’enseignement étaient également répandues, notamment le leadership sur le campus, l’avancement des connaissances en enseignement et en apprentissage, et les contributions au programme. Les listes de justificatifs potentiels donnaient un poids particulier aux perceptions des étudiants et au matériel d’enseignement. Des recommandations pour les individus et établissements souhaitant promouvoir l’excellence dans l’enseignement sont proposées, ainsi que des suggestions pour poursuivre la recherche. Mots-clés : prix d’enseignement, excellence, évaluation, critères, justificatifs, normes Introduction The assessment of teaching is receiving increased attention in higher education, with a growing body of scholarship to inform and inspire practice, and national awards to recognize excellent teaching (Stockley, Smith, Ahmad, & Hastings Truelove, 2019; Broughan, Steventon, & Clouder, 2018). However, several barriers to rigorous assessment of teaching exist. First, while Australia and the UK have recently developed national frame- works for defining and evaluating teaching (Chalmers et al., 2014; Gunn, 2018), Canada lacks such a framework and thus the responsibility for defining and promoting teaching excellence resides at the institutional level or lower. Second, given the complexity of teaching, a rigorous evaluation process would require training and an investment of time on the part of both instructors and evaluators. This is because best practices in evaluating teaching suggest that it requires multiple sources, methods and points in time, with both students and peers giv- Canadian Journal of Higher Education | Revue canadienne d’enseignement supérieur 50:1 (2020) Teaching Excellence and How It Is Awarded J. MIller-Young, M. Sinclair, & S. Forgie ing feedback on instruction, yet it is still difficult to capture its nuanced, adaptive nature in any metric (Chism, 1999; Gourlay & Stevenson, 2017). Third, any definition or set of evaluation criteria must take into account factors such as institutional context and the discipline-specific nature of knowledge and learning goals (Abbas A., Abbas, J., Brayman, Brennan, & Gantogtokh, 2016; Tucker & Chalmers, 2018). Given these complexities as well as the lack of a national framework in Canada, we wondered, “How do Canadian institutions conceptualize and award excellent teaching?” Our study investigates this question by examining teaching awards of Canadian post-secondary institutions, including their criteria, required evidence, and stated standards. Background Defining and Assessing Excellent Teaching Many studies have developed lists of characteristics, practices, and modalities that define excellent teaching (e.g., Bain, 2004; Gunn & Fisk, 2013; Macfarlane, 2007), which include many similarities and some differences. They tend to focus on process dimensions such as interactions and close contact between instructor and students (Chickering & Gamson, 1987; Pascarella, 1980) as well as student engagement and students’ perceptions of the quality of teaching (Gibbs, 2010), but also show that more experienced instructors place a higher emphasis on a broader range of teaching qualities (e.g., Keeley, Ismail, & Buskist, 2016). One difficulty in evaluating teaching using these dimensions is that different stakeholders have different perspectives. From the student perspective, excellent teachers are assertive, responsive, clear, relevant, competent, trustworthy, caring, immediate, humorous, and disclose relevant personal information (Goldman, Cranmer, Sollitto, Labelle, & Lancaster, 2017; Lubicz-Nawrocka & Bunting, 2019). For example, a recent study in New Zealand ran focus groups of students with B+ or higher grade averages and looked at their definitions and understandings of good teachers (Anderson et al., 2017). Students mentioned attributes and actions that they perceived in good teachers. The attributes they discussed included passion, approachability, knowledge, 41 and communication. Preferred actions included talking in an animated tone, making the information relevant through stories and different examples, facilitating discussion and collaboration between the teachers and the students, and considering new concepts from many different perspectives. Generally, students prefer teachers who are enthusiastic, student-centred, and who communicate course material effectively. Instructors’ perspectives tend to focus more on the facilitation of student learning. In Carnell’s (2007) study in the United Kingdom (UK), university teachers were interviewed about their perceptions of what constitutes an effective teacher. They agreed that support for student learning was important, with three main identified themes: the learning goals should be transparent, there must be communication and dialogue around learning with students, and that teaching should foster an environment where the students themselves help each other learn (Carnell, 2007). More recently, one study found that UK instructors loosely described teaching effectiveness as the effectiveness of academics in enabling students to learn but articulated this in many different ways and also expressed doubt that teaching excellence could be measured (Wood & Su, 2017). Bartram, Hathaway, and Rao (2018) found that UK and Australian instructors, when asked about excellent teaching, mentioned not only subject expertise but also facilitative, interactive pedagogies and the importance of relating to students and seeking feedback as important aspects. Overall, rather than focusing on personality traits, higher education instructors’ perspectives tend to focus on the students’ learning and their experiences in the classroom. The scholarship of teaching and learning (SoTL) movement, originating with Boyer (1990) has also influenced the higher education community’s thinking about teaching. SoTL’s focus on evidence of student learning as well as its emphasis on pedagogical content knowledge and sharing of findings (Shulman, 1986; Potter & Kustra, 2011) aligns with a growing sense that teaching in higher education should not be an individual activity, but one that involves continuous learning, reflection, and the creation of knowledge that can be shared and built upon (Gunn & Fisk, 2013). Some have gone as far as to suggest that such a scholarly approach is required in order to be considered an excellent teacher (e.g., Olsson & Roxå, 2013). Indeed, the 3M National Teaching Fellowship (3MNTF), Canada’s national teaching award, requires evidence not only of teaching excellence, but Canadian Journal of Higher Education | Revue canadienne d’enseignement supérieur 50:1 (2020) Teaching Excellence and How It Is Awarded J. MIller-Young, M. Sinclair, & S. Forgie also of leadership in teaching (Stockley et al. 2019), also signalling that excellent teachers are expected to have an impact on teaching and learning beyond their individual classroom. As a result of this complexity and definitional debate, a number of reviews of the literature have recently been compiled. Bartram et al.’s (2018) literature review found a broadly shared definition of teaching excellence at the classroom level which was based on elements of personal qualities, practical skills and professional commitment. In a comprehensive review of research published from 2012–2016, Strang, Bélanger, Manville, and Meads (2016) pointed to a lack of a body of evidence for indicators of quality teaching, and summarized how it is typically operationalized through a number of indicators of student experience and teacher performance. In contrast, Australia has developed a national framework for dimensions of effective teaching, to which each institution may map their context-specific criteria and standards (Tucker & Chalmers, 2018). The framework is based on practices which research which has improve student learning and engagement, and includes seven dimensions: designing and planning learning activities, teaching and supporting student learning, assessing and giving feedback to students on their learning, developing effective learning environments, integrating scholarship, research and professional activities, evaluation of practice and continuing professional development, and professional and personal effectiveness (Chalmers et al., 2014). While dimensions and indicators of quality teaching will likely continue to be debated, it is clear that any reasonable assessment of teaching must be multifaceted. Teaching Awards Despite the difficulty defining and assessing it, institutional awards programs are important in order to recognize teaching excellence and to encourage other faculty to strive to be excellent teachers (Jenrette & Hays, 1996; Chism, 2006; Olsson & Roxå, 2013). Awards programs “unofficially…seek to counter the privileged position of research by improving the importance and status of teaching in universities” (Halse, Deane, Hobson, & Jones, 2007, p. 731). However, they have also been criticized for being cosmetic, simply being token awards without making an impact on the institution, and tools of a neoliberal agenda (Carusetta, 2001; Olsson & Roxå, 42 2013; Saunders & Ramírez, 2017; Warren & Plumb, 1999). Further, despite the value of both student and peer perspectives for assessing teaching, past studies have found an overreliance on student perspectives. A case study on the teaching awards at a Canadian university found that student nominations were valued more than nominations from faculty (Carusetta, 2001). Teaching award committees were more hesitant to reward nominations by faculty members and colleagues, as they believed that faculty could be using the awards as a way to help their colleagues with promotion and tenure rather than a way of rewarding excellence in teaching. Another study in Australia found that teachers who have won teaching awards tend to rely on student behaviours and reactions to their teaching more than they use feedback from their peers or self-assessment (Dunkin & Precians, 1992). One way to understand how institutions are defining excellence in teaching is to look at their awards programs. In 2006, Chism surveyed the teaching award programs in 85 post-secondary institutions across the United States. The study evaluated the criteria, evidence and standards used by the award programs. Chism found that very few institutions listed specific criteria, and many used a nonspecific statement about teaching excellence as the sole criterion. As well, none of the award programs in the sample used standards to help define excellence in any of the criteria. A more recent study of teaching awards in Pharmacy faculties across the United States found that only a small number of universities used a rubric or standards for their awards (Kiersma et al., 2016). A comprehensive survey of post-secondary teaching awards in Canada has, as yet, not been conducted. To address this gap, this study used Chism’s methods to conduct a survey of teaching awards in Canada. The goals were not to make direct comparisons with the 2006 study since the current study surveyed a different population, but rather to describe the current state of teaching awards in Canada, provide a baseline for tracking future trends, and provide enough detail about the variation across institutions so that others may reflect on whether their institution’s teaching awards are appropriate for their context. Therefore, this empirical study sought to examine how Canadian institutions conceptualize excellence in teaching, as expressed through the criteria, evidence, and standards for their teaching awards. The specific Canadian Journal of Higher Education | Revue canadienne d’enseignement supérieur 50:1 (2020) Teaching Excellence and How It Is Awarded J. MIller-Young, M. Sinclair, & S. Forgie questions we set out to answer were: 1. What are the criteria, evidence, and standards currently used to award teaching in higher education institutions across Canada? 2. What are the differences between institution type? 3. How does leadership and scholarship in teaching and learning feature in these awards? Methods Data Collection Information about the criteria, evidence, standards, and purposes of teaching awards was collected from 89 post-secondary institutions across all provinces in Canada, using a list of public post-secondary institutions from Statistics Canada’s website (Government of Canada, 2019). Initially, searches of the publicly available websites of the Teaching and Learning Centres for information about institutional teaching awards were carried out and, if insufficient information was found, a search of the entire institution’s website was conducted. If insufficient or no information was found online, institutions were contacted by email. Directors of the Teaching and Learning Centres were contacted first, and if there was no one available, an email was sent to the ‘general information queries’ contact. Awards for team teaching, teaching units, graduate supervision and curriculum development were excluded from the award sample; their criteria varied significantly from most of the teaching awards, making them difficult to code and to make comparisons. All other awards were included in the data set if the minimum data set of award criteria and evidence was available. Data collection occurred in May and June 2017. In total, 204 awards programs had sufficient award information to be included in the study. Data Analysis Using the qualitative data analysis software NVivo (QSR International Pty Ltd.), the data was coded for type of institution, award purpose, types of evidence, criteria, and standards, and definitions of teaching excellence. Initial coding used Chism’s (2006) 11 criteria codes (Table 1), 43 27 forms of evidence (Table 2), and yes/no for standards. Of the three authors, two (JM and SF) coded the data for one province each, while the other (MS) coded all provinces. Comparisons and discussions about the coding of the two common provinces (Alberta and British Columbia) led to agreement on a consistent coding scheme; one author (MS) coded the full data set after consensus was reached. After the first round of comparisons and discussions, three new codes were added (Tables 1 and 2). In the Chism study, the “innovation” code included innovation in both pedagogy and curriculum, however in our data these criteria were described sufficiently differently to justify two separate codes. Similarly, wording about introducing discipline specific research into the classroom was sufficiently different from that about content knowledge criterion and thus did not fit under any of the Chism codes, requiring a new code called “research integration” (Table 1). A new code was also required for types of evidence. We saw regular mention of types of evidence which came directly from students, such as artefacts or analysis of student work, that a new code “evidence of student learning or success” was also added. All codes for criteria and evidence are summarized in Tables 1 and 2, respectively. Awards were coded as having standards if there were any evaluation materials available, such as a rubric. Included rubrics had descriptions which illustrated poor, good, and excellent examples related to the criteria. Rubrics that scored the criteria out of a certain number of points, without specifying how those scores were obtained were coded as not having standards. To answer the second research question around institutional types, institutions were divided into two samples. Sample 1 consisted of community colleges, institutes, polytechnics and undergraduate universities (Undergraduate Group), and sample 2 included graduate degree granting universities, comprehensive universities, and research-intensive universities (Graduate Group). Results for each group were then analysed and compared. To assess the third research question about how scholarship and leadership are integrated into teaching awards, the criteria were classified into two groups — those that only included criteria and evidence related to an individual’s excellent pedagogical practice (within the classroom), and those which also included criteria and evidence related to dissemination or other activities Canadian Journal of Higher Education | Revue canadienne d’enseignement supérieur 50:1 (2020) Teaching Excellence and How It Is Awarded J. MIller-Young, M. Sinclair, & S. Forgie 44 Table 1. Criteria codes used in this study; codes from Chism (2006) are listed in order of decreasing frequency found in that study Criteria codes Description Chism (2006) codes G Global Global statements about teaching excellence as the only criterion, or no criteria specified TE Teaching Specific characteristics of teaching performance listed IM Impact Impact on student learning SC Student Centredness Student-centred approach, shows concern for growth and development CK Content Knowledge Content knowledge, mastery of subject CL Campus Leadership Leadership in promoting teaching on campus RG Range Range of teaching activities undertaken IN Innovation Innovations in classroom practice (in 2006 also included curriculum development efforts) O Other ST Scholarship of Teaching Scholarship related to teaching activities PD Professional Development Professional development efforts New codes CP Curriculum and Programs Innovation in curriculum and programs RE Research integration Integrating research into the classroom Table 2. Evidence codes used in this study; codes from Chism (2006) are listed in order of decreasing frequency found in that study Evidence codes Chism (2006) codes Letters Student ratings of instruction Curriculum vitae Philosophy of teaching statement List of teaching responsibilities Other Syllabi or other course materials Peer review or classroom observation summary Canadian Journal of Higher Education | Revue canadienne d’enseignement supérieur 50:1 (2020) Teaching Excellence and How It Is Awarded J. MIller-Young, M. Sinclair, & S. Forgie 45 Evidence codes Chism (2006) codes Teaching portfolio List of professional contributions in teaching List of contributions to promotion of teaching on campus List of previous rewards or recognitions for teaching Documentation of involvement with students outside classroom No evidence specified at all Description of growth in teaching and self-learning over time List of professional development activities in teaching Descriptions of innovations in teaching Unspecified “additional documentation” New code Evidence of student learning or success which would have an influence on teaching beyond the individual’s practice (Table 3). There were four categories of teaching awards in the sample: general awards, awards based on length of teaching career, student-focused awards, and awards for methods of teaching. General awards accounted for most of the awards in our sample, and their purpose was to recognize an excellent teacher. Often, these would be the only award an institution would have, particularly for smaller institutions which had fewer overall (Table 4). Awards based on length of teaching career were awards which categorized teachers based on whether they were in their early career, later career with leadership or lifetime achievement awards, or sessional/ part-time teaching awards. These awards would differ their criteria slightly to adjust to the differing expectations of these positions. For example, an early career achievement award may base 25% of the award criteria on campus leadership, while a later career achievement award may have leadership account 50% of the criteria. Student-focused awards include excellent teaching to Aboriginal students, first year students or international students. Finally, awards for methods of teaching include such categories as excellence in experiential learning, innovative teaching, teaching for deep learning, and research-inspired teaching. All four categories of awards were focused on teaching, and thus all were included in the analysis. Findings and Discussion The results were analysed to see differences and patterns in the sample of award programs. Criteria, Evidence, and Standards The criteria, evidence, and standards of the awards were coded, comparing their frequencies and differences overall and by type of institution. Criteria Of the 204 teaching excellence awards across Canada, all listed some form of criteria. There was variation in the type and number of criteria used by the awards; some award programs provided extensive and detailed lists of characteristics and expectations of excellent teachers, while others offered only a short description of their criteria. The most common criteria used by 61% of awards involved specific characteristics of teaching performance (Figure 1). These characteristics varied from communication skills and preparation for classes to assessment Canadian Journal of Higher Education | Revue canadienne d’enseignement supérieur 50:1 (2020) Teaching Excellence and How It Is Awarded J. MIller-Young, M. Sinclair, & S. Forgie 46 Table 3. Criteria codes grouped into two themes: (A) individual excellence and (B) scholarship and leadership A Criteria related to individual excellence B Criteria related to scholarship and leadership in teaching G Global CL Campus Leadership TE Teaching ST Scholarship of Teaching IM Impact CP Curriculum and Programs SC Student Centredness CK Content Knowledge RG Range IN Innovation O Other PD Professional Development RE Research integration Table 4. Institution types and numbers of awards Type # of Institutions # of Awards Avg #/Institution Undergraduate University 13 14 1.1 Community College 20 24 1.2 Polytechnic 4 6 1.5 Master’s University 12 31 2.6 Comprehensive University 27 70 2.6 Research Intensive 13 59 4.5 Total 89 204 methods, and frequently awards listed more than one characteristic. This contrasts with Chism’s study (2006), which found that 52% of awards only gave a global statement about requiring excellent teaching, and 43% listed specific characteristics of teaching. Similar to the Chism study, other relatively high-ranking criteria included student centredness (55%) and innovation (49%). Student centredness describes the teacher’s care and concern for their students, and interest in their learning. This includes encouraging students to learn the material and being approachable and flexible to students when they need help outside of the classroom. The innovation criterion was sometimes used broadly, like “innovation and creativity,” while other award programs more explicitly defined innovation as the use of new technology or teaching methods in the classroom. Compared to the 2006 study, there was a high frequency of criteria related to activities that have influence beyond the individual’s classroom. For example, the campus leadership criterion appeared in 43% of the awards compared to 32% found by Chism, curriculum development was seen in 18% of awards (whereas before it was 23% but combined with innovation in teaching), and scholarship of teaching and learning was listed in 24% of the awards, compared to only 8% in Chism (2006). A third of the awards program had some criteria Canadian Journal of Higher Education | Revue canadienne d’enseignement supérieur 50:1 (2020) Teaching Excellence and How It Is Awarded J. MIller-Young, M. Sinclair, & S. Forgie 47 Figure 1. Frequency of criteria codes by percentage of the sample (percentages reflect the appearance of each criterion, with many of the awards using more than one criterion, and therefore do not add up to 100%) which were classified as “other” (Figure 1). This covered a wide range of criteria including faith-based criteria, work done within the teacher’s own field or discipline unrelated to teaching, and criteria related to meeting the institution’s mission statement. Evidence Seventy-seven percent of awards required letters of support as evidence of excellent teaching (Table 5). Of these, the most common source of the required letters was from students and alumni (58%), closely followed by letters from faculty or peers (53%). One quarter of the awards requested letters without specifying who the letter must be written by, or requested letters by individuals outside of the university such as members of the community. As well, a few of the awards required letters from administrators separate from nomination letters (many of the awards required nomination letters by chairs or deans). Some of the award programs specifically outlined what the nominators needed to discuss, while other programs did not outline specifics for the nomination letter, stating only that a nomination letter was required. Ten programs had forms for the nominators, offering prompts or questions (especially common for student nominators). A surprisingly small proportion of the awards specifically required any evidence of self-reflection on teaching or peer review (Table 5). For example, 11.4% mentioned classroom observation and only 1% required a teaching portfolio. 43% of the awards required evidence categorized as “other,” which included interviews with the award candidate by the selection committee, or evidence of responding to feedback from assessments. A few of the awards included open-ended statements, such as “include any other evidence of meeting crite- ria,” and “additional materials.” Overall, the forms of evidence were heavily weighted to student perceptions (student evaluations, 51.5%) and instructor-generated forms of evidence. Standards Only 4 of the 89 institutions described standards for their teaching awards. The standards used by the four institutions were in the form of rubrics, and evaluated the extent to which the applicant was meeting the criteria. For example, one university listed each of the criteria, offering a definition of what beginning, developing and exemplary quality of the criteria looked like. Differences by Institution Type In order to answer our second research question about the differences between institution types, we compared both the criteria and evidence in two separate groups. While undergraduate and graduate institutions used similar criteria in their award programs, there were some differences between the two groups (Figure 2). The undergraduate group used a higher percentage of global criteria such as “excellent overall quality” or “outstanding teaching.” Both groups had a higher percentage of criteria for a student-centered approach compared to Chism’s 2006 study, which found 7% for the undergraduate group and 31% for the graduate group. Other differences between the groups may be due to differences in roles and responsibilities of instructors at different institution types, for example instructors in colleges may teach in transfer programs where they would have less control over curriculum and programs, and by virtue of their contract would have less expectations for scholar- Canadian Journal of Higher Education | Revue canadienne d’enseignement supérieur 50:1 (2020) Teaching Excellence and How It Is Awarded J. MIller-Young, M. Sinclair, & S. Forgie 48 Figure 2. Award criteria in community colleges, polytechnic institutions and undergraduate universities (undergraduate group), compared to U15s, master’s and comprehensive universities (graduate group) ship. This would explain why curriculum and program development was used as a criterion for only 7% of the awards in the college group, and scholarship of teaching was 18% lower in the undergraduate group compared to the graduate group. The fact that the graduate group had a lower percentage of awards citing campus leadership as a criterion can be explained by the fact that they usually had multiple awards, with only the senior awards listing leadership as a criterion. Overall, undergraduate institutions tend to use less specific criteria compared to research (in Canada, termed the U15) and comprehensive universities. Another major difference between the two groups involved the required amount of evidence. The undergraduate group required less evidence, and fewer nomination and support letters, while awards in the graduate group generally required multiple pieces of evidence (Figures 3a & 3b). Further, any of the awards for the undergraduate group which required letters tend to rely solely on nomination letters, while U15s and comprehensive universities relied on both nomination letters and letters of support. However, it was surprising that forms of evidence such as teaching philosophy statements and descriptions of growth as a teacher did not feature more highly within the undergraduate group (Figure 3b). Growth in Criteria for SoTL and Leadership We classified campus leadership, innovations in curriculum and programs, and SoTL as criteria which indicated an expectation that excellent teachers should have impact beyond their own classroom. Because innovations in curriculum did not have its own code in Chism’s paper, its frequency may be conflated with campus leadership when compared to our study, and thus we cannot comment on differences between our study and the Chism (2006) study. However, it is interesting to note that innovations in curriculum and programs appeared frequently enough to warrant its own category in our 2017 survey, in addition to the fact that the frequency of SoTL as a criterion was twice as high in our study, for both undergraduate and graduate institutions, compared to the 2006 study conducted in the US (Figure 4). Thus, we speculate that institutional expectations of excellent teachers have changed over time to include influence on teaching and learning beyond one’s own classroom. Summary and Implications This study contributes to the literature about perspectives of teaching excellence, the nature of teaching awards in Canada, and provides a baseline for future tracking of national trends as well as for comparison to other countries. The study is based on publicly available information (or what individuals were permitted to share) from 89 institutions and 204 award programs. The criteria found in this study are aligned with current scholarship concerning the dimensions of quality teaching (e.g., Chalmers et al., 2014), however forms of evidence rely very little on self and peer assessments. We found that specific characteristics of teaching effectiveness are listed more often than vague, global statements about “teaching excellence.” Several other Canadian Journal of Higher Education | Revue canadienne d’enseignement supérieur 50:1 (2020) Teaching Excellence and How It Is Awarded J. MIller-Young, M. Sinclair, & S. Forgie 49 Figure 3a. Forms of evidence required by community colleges, polytechnic institutions and undergraduate universities (undergraduate group), compared to U15s, master’s and comprehensive universities (graduate group) Figure 3b. Forms of evidence by institution type, cont’d positive differences compared to a 2006 American study included a higher frequency of criteria related to impact outside of the classroom, such as campus leadership (43%), curriculum contributions (18%), and scholarship of teaching and learning (24%). Letters of support were the most commonly listed form of evidence, followed by teaching philosophy and student evaluations. However, beyond the teaching philosophy (57%), few forms of evidence were of the sort which would demonstrate any self-reflection or growth in teaching and only four institutions described standards for their awards. Classroom observations were listed as forms of evidence in only 11% of the awards programs. Major differences between undergraduate and graduate institutions were that undergraduate institutions listed student centredness and scholarship of teaching and learning as criteria less often and had global statements of teaching excellence more often. Canadian Journal of Higher Education | Revue canadienne d’enseignement supérieur 50:1 (2020) Teaching Excellence and How It Is Awarded J. MIller-Young, M. Sinclair, & S. Forgie 50 Figure 4. Differences in frequency of SoTL as a criterion for institutional teaching awards If institutions wish to reward and enhance the quality of teaching and the student learning experience, we suggest it would be worth the institutional investment of time and resources to discuss and devise clear criteria and evidence guidelines to guide instructors in their reflection and portfolio preparation. To reduce reliance on students’ perceptions of teaching, systems for peer review of teaching also need to be developed and supported, which would require opportunities and training for classroom observations and other forms of peer review including both formative as well as summative evaluations; these would serve the dual purposes of not only assessing teaching but also facilitating conversations about teaching, thus strengthening the teaching culture (Smith, 2014). We recognize this is no small task and depending on institutional context, could require significant effort in terms of financial support, policy development, incentives, and professional development for faculty (Hunt, 2006). However authentic institutional engagement in the process of defining and assessing meaningful criteria is necessary for teaching awards to truly inspire and enable excellent teaching and not be perceived as cosmetic or token awards. Opportunities for future work in this area are plentiful, as this study raises as many questions as it answers. For example, while stated criteria, evidence and standards of teaching awards programs were analysed for this study, it is unknown how they are actually applied in the institutional evaluation and ranking process, nor how well reviewers are trained in the assessment of teaching. It is also unknown how many institutions have robust processes for peer review of teaching. Future studies might focus on the prevalence of peer review and training of peer reviewers, and analyse the applications and portfolios of teaching award winners for their alignment with stated criteria and standards. Acknowledgements This project was supported by funds from the Office of the Provost, University of Alberta. References Abbas, A., Abbas, J., Brayman, K., Brennan, J., & Gantogtokh, O. (2016). Teaching excellence in the disciplines. York, UK: Higher Education Academy. Anderson, V., Wass, R., Rangi, A., Eteuati, E., Golding, C., & Rabello, R. (2017, June). Reframing ‘curriculum transformation’ through attention to university students’ conceptions of good teaching. Paper presented at Higher Education Research and Development of Society of Australasia, Sydney, Australia. Canadian Journal of Higher Education | Revue canadienne d’enseignement supérieur 50:1 (2020) Teaching Excellence and How It Is Awarded J. MIller-Young, M. Sinclair, & S. Forgie Bain, K. (2004). What the best college teachers do. Cambridge, MA: Harvard University Press. Bartram, B., Hathaway, T., & Rao, N. (2018). ‘Teaching excellence’ in higher education: a comparative study of English and Australian academics’ perspectives. Journal of Further and Higher Education, 43(9), 1284-1298. doi: 10.1080/0309877X.2018.1479518 Boyer, E. L. (1990). Scholarship reconsidered: Priorities of the professoriate. Lawrenceville, NJ: Princeton University Press. Broughan, C., Steventon, G., & Clouder, L. (Eds.). (2018). Global perspectives on teaching excellence: A new era for higher education. London, UK: Routledge. Chalmers, D., Cummings , R., Elliott, S., Stoney, S., Tucker, B., Wicking, R., & Jorre de St Jorre, T. (2014). Australian university teaching criteria and standards project: Final report. Australian Government Office for Teaching and Learning, Canberra, A.C.T. http://dro.deakin.edu.au/view/DU:30105033 Carnell, E. (2007). Conceptions of effective teaching in higher education: Extending the boundaries. Teaching in Higher Education, 12(1), 25–40. doi:10.1080/13562510601102081 Carusetta, E. (2001). Evaluating teaching through teaching awards. New Directions for Teaching and Learning, 2001(88), 31–40. doi:10.1002/tl.35 Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. Racine, WI: The Johnson Foundation. Chism, N. V. N. (1999). Peer review of teaching. A sourcebook. Bolton, MA: Anker Publishing Company. Chism, N. V. N. (2006). Teaching awards: What do they award? The Journal of Higher Education, 77(4), 589–617. doi:10.1353/jhe.2006.0031 Dunkin, M. J., & Precians, R. P. (1992). Award-winning university teachers' concepts of teaching. Higher Education, 24(4), 483–502. doi:10.1007/ BF00137244 Gibbs, G. (2010). Dimensions of quality. York, UK: Higher Education Academy. 51 Goldman, Z. W., Cranmer, G. A., Sollitto, M., Labelle, S., & Lancaster, A. L. (2017). What do college students want? A prioritization of instructional behaviors and characteristics. Communication Education, 66(3), 280–298. doi:10.1080/03634523.2016.1265135 Gourlay, G., & Jacqueline Stevenson, J. (2017) Teaching excellence in higher education: Critical perspectives. Teaching in Higher Education, 22(4), 391– 395. doi: 10.1080/13562517.2017.1304632 Government of Canada. (2018). List of post-secondary institutions. Retrieved from http://publications.gc.ca/ Collection/Statcan/81-582-X/institution.pdf Gunn, V. (2018). Metrics and methodologies for measuring teaching quality in higher education: developing the Teaching Excellence Framework (TEF). Educational Review, 70(2), 129-148, doi:10.1080/0013191 1.2017.1410106 Gunn, V., & Fisk, A. (2013). Considering teaching excellence in higher education: 2007-2013: A literature review since the CHERI Report 2007. York, UK: Higher Education Academy. Halse, C., Deane, E., Hobson, J., & Jones, G. (2007). The research-teaching nexus: What do national teaching awards tell us? Studies in Higher Education, 32(6), 727–746. doi:10.1080/03075070701685155 Hunt, L. (2006). A community development model of change: The role of teaching and learning centres. In L. Hunt, A. Bromage, & B. Tomkinson (Eds.), The realities of change in higher education (pp. 82–95). London, UK: Routledge. Jenrette, M., & Hays, K. (1996). Honoring exemplary teaching: The two-year college setting. New Directions for Teaching and Learning, 1996(65), 77–83. doi:10.1002/tl.37219966514 Keeley, J. W., Ismail, E., & Buskist, W. (2016). Excellent teachers’ perspectives on excellent teaching. Teaching of Psychology, 43(3), 175–179. doi:10.1177/0098628316649307 Kiersma, M. E., Chen, A. M. H., Kleppinger, E. L., Blake, E. W., Fusco, N. M., Mody, V., Gillespie, M., Knell, M., & Zavod, R. M. (2016). Evaluation of criteria utilized in the recognition of teaching excellence Canadian Journal of Higher Education | Revue canadienne d’enseignement supérieur 50:1 (2020) Teaching Excellence and How It Is Awarded J. MIller-Young, M. Sinclair, & S. Forgie awards. Currents in Pharmacy Teaching and Learning, 8(4), 477–484. doi:10.1016/j.cptl.2016.03.003 Lubicz-Nawrocka, T., & Bunting, K. (2019). Student perceptions of teaching excellence: an analysis of student-led teaching award nomination data. Teaching in Higher Education, 24(1), 63–80. doi:10.1080/ 13562517.2018.1461620 Macfarlane, B. (2007). Beyond performance in teaching excellence. In A. Skelton (Ed.), International perspectives on teaching excellence in higher education (pp. 62–73). London, UK: Routledge. Napier, J., Riazi, M., & Jacenyik-Trawoger, C. (2014). Leadership: A cultural perspective on review as quality assurance versus quality enhancement. In J. Sachs, & M. Parsell, (Eds.), Peer review of learning and teaching in higher education: International perspectives (pp. 53–66), Dordrecht, Netherlands: Springer Olsson, T., & Roxå, T. (2013). Assessing and rewarding excellent academic teachers for the benefit of an organization. European Journal of Higher Education, 3(1), 40–61. doi:10.1080/21568235.2013.778041 Pascarella, E. T. (1980). Student-faculty informal contact and college outcomes. Review of Educational Research, 50(4), 545–595. doi:10.3102/00346543050004545 Potter, M. K., & Kustra, E. D. (2011). The relationship between scholarly teaching and SoTL: Models, distinctions, and clarifications. International Journal for the Scholarship of Teaching and Learning, 5(1). doi:10.20429/ijsotl.2011.050123 52 Strang, L., Bélanger, J., Manville, C., & Meads, C. (2016). Review of the research literature on defining and demonstrating quality teaching and impact in higher education. York, UK: Higher Education Academy. Stockley, D., Smith, R., Ahmad, A., & Hastings Truelove, A. (2019). Making a difference: Three decades of Canada’s only national teaching award. The Canadian Journal for the Scholarship of Teaching and Learning, 10(1), Article 4. doi:10.5206/cjsotl-rcacea.2019.1.7998 Tucker, B., & Chalmers, D. (2018). A national strategy for teaching excellence – one university at a time. In C. Broughan, G. Steventon, & L. Clouder (Eds.), Global perspectives on teaching excellence (pp. 115–127). London, UK: Routledge. Warren, R., & Plumb, E. (1999). Survey of distinguished teacher award schemes in higher education. Journal of Further and Higher Education, 23(2), 245–255. doi:10.1080/0309877990230208 Wood, M., & Su, F. (2017). What makes an excellent lecturer? Academics’ perspectives on the discourse of ‘teaching excellence’ in higher education. Teaching in Higher Education, 22(4), 451–466. doi:10.1080/1 3562517.2017.1301911 Contact Information Janice Miller-Young, University of Alberta [email protected] Saunders, D. B., & Blanco Ramírez, G. (2017). Against ‘teaching excellence’: Ideology, commodification, and enabling the neoliberalization of postsecondary education. Teaching in Higher Education, 22(4), 396–407. doi:10.1080/13562517.2017.1301913 Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14. doi: 10.3102/0013189X015002004 Smith, S. L. (2014). Peer collaboration: Improving teaching through comprehensive peer review. To Improve the Academy, 33(1), 94–112. doi:10.1002/tia2.20007 Canadian Journal of Higher Education | Revue canadienne d’enseignement supérieur 50:1 (2020)