Learning (About) Outcomes / R. S. Ascough 44 CSSHE SCÉES Canadian Journal of Higher Education Revue canadienne d’enseignement supérieur Volume 41, No. 2, 2011, pages 44 - 61 Learning (About) Outcomes: How the Focus on Assessment Can Help Overall Course Design Richard S. Ascough Queen’s University Abstract The demand for quantitative assessment by external agencies and internal administrators can leave post-secondary instructors confused about the nature and purpose of learning outcomes and fearful that the demand is simply part of the increasing corporatization of the university system. This need not be the case. Developing learning outcomes has a number of benefits for course design that go beyond program assessment. This article clarifies some key aspects of the push toward using learning outcomes and introduces a tripartite nomenclature for distinguishing among course outcomes, outputs, and objectives. It then outlines a process for instructors to use these three categories to develop and design courses that meet institutional assessment demands while also improving overall teaching effectiveness. Résumé L’évaluation quantitative que demandent les agences externes et les administrateurs internes peut confondre les instructeurs de niveau postsecondaires quant à la nature et à l’objectif des « résultats d’apprentissage », et leur faire craindre que cette demande ne fasse simplement partie de la privatisation croissante du système universitaire. Ce n’est pas forcément le cas. La création de résultats d’apprentissage présente de nombreux avantages sur le plan de la conception de cours, avantages qui vont au-delà de l’évaluation de programme. L’article clarifie quelques aspects principaux de la poussée vers l’utilisation de « résultats d’apprentissage » et présente une nomenclature tripartite pour faire la distinction entre les résultats de cours, le rendement et les objectifs. Il décrit ensuite un processus pour CJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough 45 que les instructeurs emploient ces trois catégories afin de concevoir des cours qui répondent aux exigences en évaluation de l’institution, tout en améliorant l’efficacité de l’enseignement dans son ensemble. Clearly articulated learning outcomes are required by most accrediting agencies and often by those who hold the purse strings of publicly funded higher education. Shanahan summarizes the situation in Canada, no doubt replicated in other countries: “We have seen provincial governments increasingly adopting market mechanisms in funding and resource allocation. Business and private sector criteria are employed to make education decisions.... And we have seen our accountability frameworks become infused with market discourse, market principles and market mechanisms” (Shanahan, 2009, pp. 4–5; see also Côté & Allahar, 2007; Sims, 2002). Although the potential harm that can be attached to using corporate language in higher education is great (see Woodhouse, 2009), the situation is unlikely to change quickly. The demand for quantifiable measurements for assessing student learning is increasing (Diamond, 2008). For example, in summarizing the Ontario Council of Academic Vice-Presidents’ Guidelines for University Undergraduate Degree Level Expectations Shanahan states, “University Faculties or Departments must articulate standardized outcomes and degree level expectations for every degree offered” (2009, p. 6). In turn, each course must articulate learning outcomes that are in alignment with these expectations and with degree outcomes. The pressure to articulate outcomes is not simply a top-down process; it also arises from our students themselves. There is a big difference between students now and students 30, 20, or even 10 years ago. A full decade into the 21st century we are now facing students who have been labelled, for better or worse, the millennial generation or Net Geners (those born after 1982; see Tapscott, 2009). These are students for whom digital technologies such as computers, iPods, and cellphones are everyday tools that they have owned and used for much of their lives, and for whom multi-tasking – working on multiple projects simultaneously or even having multiple (online) conversations at the same time – is common. As part of their culture, such students want and often demand a clear idea of the return on investment of a given activity. On the downside, this contributes to the corporatization of the university, where students demand service for their (tuition) fees and often have a sense of entitlement to top grades, in part fed by the grade compression of their high school years (see Côté & Allahar, 2007). On the upside, however, such students respond favourably to clear directives and are task oriented. Thus, clearly articulated learning outcomes give them a focus for their learning, which they can incorporate into their busy, often hectic schedules. The overall pressure not only to articulate learning outcomes but also to design clear modes of assessing the realization of these outcomes understandably has many instructors stressed, confused, angry, or some combination thereof. The current economic climate is not helpful, since faculty members are required to do more teaching and research with fewer resources. The move toward learning outCJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough 46 comes can feel like an unwelcome burden adding to an already heavy load, and faculty members can become exhausted and cynical about the process (Shanahan, 2009). Nor does it help that legislation aimed at promoting quality seems to dilute educational principles by promoting simplistic rubrics to measure learning (Bresciani, 2006). This is unfortunate and need not be the case. Understanding and planning learning outcomes are essential aspects of effective course design. Embracing the positive aspects of the outcomes initiatives can create opportunities to assess teaching effectiveness and determine how to improve upon it. It goes to the heart of who we are as teachers in institutions of higher learning. Despite popular stereotypes to the contrary, many of us teaching in post-secondary institutions are deeply committed to our students’ learning. As a faculty member myself I have wrestled with the critical issues in the learning outcomes movement for some time and offer the following guidelines for designing three critically linked aspects of student learning: course learning outcomes, student outputs, and instructional objectives. Defining the Terms Before discussing how to design learning outcomes, it is necessary to examine the terminology that is being employed and introduce some clarifications. Many terms have been used to designate the end result of teaching and learning: aims, purposes, goals, guiding outcomes, content standards, curriculum standards, accomplishments. Anderson and Krathwohl prefer the term objectives, by which they mean “intended student learning outcomes” (2001, p. 3). Even this, however, merges what for others are two distinct but related aspects of teaching: what the instructor intends for the learning in a particular course (objectives) and how the student demonstrates learning in the course (outcomes). My own experience as a faculty member wading through the literature left me confused and frustrated. The same mixing of terms is present in works discussing program or institutional goals. For example, Maki (2004) discusses objectives as the broad learning of students at the program level and learning outcomes as the measurable work by which students demonstrate their learning. When discussing the institution as a whole, however, she uses the term learning outcomes for the broader learning rather than the measurable work (Maki, 2004, p. 61; cf. Pratt & Associates, 1998). Overall, such muddiness in the use of the term learning outcomes is not helpful to the discussion as it can cause instructors and administrators to talk at crosspurposes with one another and among themselves. In my experience, much more clear and helpful is the maintenance of a tripartite distinction that separates learning outcomes, student outputs, and teaching objectives. The separation of outcomes and outputs is drawn from the business world (see Performance Measurement Net, 2004), not a particularly popular place for academic metaphors among many faculty members. Nevertheless, I do think it is a helpful distinction to use as accrediting agencies and administrative bodies press institutions more and more toward demonstrating “deliverables.” CJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough 47 Learning outcomes identify the deeper learning that a course intends to produce – the overall impact of the course. The outcomes define the attitudes and abilities that should result from the learning but are not necessarily measurable in the short term. For example, an undergraduate course in world religions might identify an outcome as “respect for religious diversity.” The value “respect,” however, is not easily measured by typical course assessment techniques. It is something an instructor hopes to instill in students but cannot grade through a typical essay or exam format (she cannot know whether a student has truly embraced respect for diversity as a virtue even if the student demonstrates knowledge about diversity). Student outputs delineate what the student is responsible for in the course, including the nature of the work that will be required for assessment. Assignments provide the occasion for students to demonstrate how they interact with course material while presenting them with opportunities to show the connections and extensions they have made. In order to assist students to understand output expectations it is helpful to distribute a grading rubric for that assignment before it is due. Students need clear directions in order to prepare for and participate in regular learning experiences, and they deserve unambiguous descriptions of course assignments and assessment criteria. Course objectives explain what the instructor is responsible for in the course. Such input should be linked to one’s teaching philosophy and be manifest in one’s teaching style. Objectives can be linked to constructivist learning principles that inform course design and delivery, principles such as focusing on subject-centred learning, encouraging deep learning, facilitating interactive learning, teaching to a variety of learning styles, and modelling high expectations. Morrison, Ross, and Kemp (2004) rightly note that most course objectives aim at short-term goals attainable during a course of study, with only a few contributing to long-term goals such as analytical skill development or decision-making ability that fall outside the direct control of the instructor and must be measured later. Somewhat confusingly, however, they claim that one benefit of naming course objectives is giving direction to learners as to what is to be mastered in the course, yet they admit that high-level objectives are not fully attainable, or at least not fully measurable, within the parameters of the course. Such mixing of terms and descriptions necessitates that an important distinction be made between what is measurable (outputs) and what is desirable (outcomes), while allowing the term objectives to indicate the intention of the instructor in presenting the course subject matter. Designing the Outcomes, Outputs, and Objectives The seminal work of Wiggins and McTighe (1998) on “backward design” has proven influential among excellent teachers (Bain, 2004; Richlin, 2006). Through this process the course learning outcomes can be identified, then the outputs can be determined, and finally the learning experiences and instruction modules can be designed (the objectives). This planning process assists instructors to define the CJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough 48 student learning for a course and select the strategies they will use to facilitate and measure that learning (see Mager, 1972). Identify Student Learning Outcomes In developing a course it is best to begin at the end with the learning outcomes. Instructors can identify three or four key desired results of their course in one or more of the three primary learning domains: cognitive, psychomotor/skill, and identity (see Figure 1). This is consistent with Bain (2004), who found that effective college teachers were very clear about the promises and opportunities the course would offer students, giving them a sense of the questions it would address or the abilities it would help them develop. Being as clear as possible in framing outcomes statements is essential as these will form the basis of student assessment and the overall design of the course, and thus will serve the key goal of fostering deep student learning (Diamond, 2008). Psychomotor/ Skills Learning Domain Cognitive Learning outcomes explain the deeper learning that the course intends to produce—the overall impact of the course. The outcomes define the attitudes and abilities that should result from the learning but are not necessarily measurable, especially in the short term. What do I want my students to understand, know, learn, and/or unlearn? How do I want my students to think? How do I want my students to apply their understanding? What should students be able to do? Identity What ethos/attitudes will students embody? How do I want my students to “be” in the world? Figure 1. Identify Student Learning Outcomes In the cognitive domain, which focuses on mental operations, questions might be posed such as these: What do I want my students to understand, know, learn, and/or unlearn? How do I want my students to think? How do I want my students to apply their understanding? Responses should articulate wide-ranging CJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough 49 ideas and concepts that need to be explained or perhaps even deconstructed. They should be broadly provocative while having enough specificity to provide guidance for teaching and allow for clear assessment of learning (Wiggins & McTighe, 1998). In the psychomotor domain, which focuses on physical skills and abilities, one can concentrate on the kinesthetic movements students should be able to perform. In many courses, however, especially in the humanities, one might think here of the specific applications of cognitive operations – the skills the course will engender. In the identity domain, which encompasses things like beliefs, attitudes, and values, the instructor can ask: What ethos and/or attitudes will students embody? How do I want my students to “be” in the world? A few brief exercises can be employed in order to stimulate thinking about the course learning outcomes. Weimer (2002) suggests that an instructor imagine being in a shopping mall and bumping into a former student who recalls the course from five years earlier. As an instructor, what thoughts would you want to have running through the student’s mind in that meeting, what memories brought to the fore? Another way to identify learning outcomes is through a game-show format based on Jeopardy! (Wiggins & McTighe, 1998). Begin with the answer to be learned and then articulate the question(s) that would lead to this answer. The key is to identify essential issues that form the core of a discipline, represent the most important and/or controversial topics, recur frequently in the field, and lead to other important questions and issues (Wiggins & McTighe, 1998). These outcomes are generally quite broad and not immediately measurable during or at the end of the course. For example, one potential outcome for many university courses is that “students will learn to think like a _________,” where the blank is filled in with the name of the particular discipline such as historian, scholar of religion, biologist, or the like. Although this response is not measurable in the short term, many instructors hope it will occur as a result of the course, at least for program majors. With this in mind, instructors must also be aware that they are modelling the discipline every time they engage with students. The core questions and issues in a course may be linked to broad philosophical or ideological commitments, whether addressing the meaning of life (Kronman, 2007) or conveying a body of knowledge and analytical skills (Fish, 2008). Either way, some thought must be given to the overall educational undertaking of a university and the instructor’s place in it, beyond what is avowed in the institutional mission statement itself. Why is it one does what one does? How is one sustained in the classroom? Students learn best when they are able to connect their new learning with their own pre-existing questions and goals in a way that causes them to “reconcile, explain, modify, or integrate” their new learning with their existing knowledge (Bain, 2004; Côté & Allahar, 2007). Defining “big questions” to focus a course helps make connections with what students need or desire to learn from the course, given its broad topic, in order to meet disciplinary needs (Richlin, 2006). For the instructor to keep herself engaged and excited, she should also reflect the current CJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough 50 problems and issues within the field of scholarship. At the very least, an instructor should be able to identify what excites him about the course he is teaching. At this stage, instructors also need to recognize who it is they are teaching. As noted, the current undergraduate students have grown up interacting with one another through digital technology, and these students expect to be a contributing part of the learning environment (Tapscott, 2009). They are not much interested in a one-size-fits-all model of distributed learning (the prevailing model in higher education) but want to engage and be engaged. It will not be sufficient to identify learning outcomes for these students with words like understand or grasp, unless there is a commitment to engage their learning styles and needs in a way that fits with their norms. This is a pedagogy that differs greatly from the ways in which most professors experienced their own formative education. To ignore this reality, however, is to doom the educational system to failure and irrelevance for the next generation of leaders and scholars. Alongside these characteristics, however, these students tend to be instrumental about their studies, giving greater value to course materials and assignments that have a direct link to what they see as their future employment (Côté & Allahar, 2007). As a result, students can become obsessed about their grades rather than their learning, a disturbing trend that needs to be reversed, so that students do not continue to be disengaged from their studies (Côté & Allahar, 2007). Recognizing that this is a factor in the classroom and naming it for students can begin to address the problem. Often those of us teaching courses that are not obviously connected to a particular professional skill assume that students will bridge the gap. For example, we assume that students will notice that they are learning to be critical thinkers and will know that this will help them problem-solve in their future profession (whatever it might be), and thus they will be engaged. Unfortunately, such bridging on the part of the students is not taking place (Côté & Allahar, 2007). So while the notion that we are teaching transferable skills is not misplaced, it is incorrect to assume that students recognize it on their own. Articulating transference of classroom learning and skill acquisition to the workplace can be one way to help students engage with course material. The obvious place to articulate the value of learning transferable skills is the course learning outcomes on the syllabus. Determine Student Outputs Once the broader concerns of the course and the large issues and questions that the course addresses are identified, the specific, measurable aspects of student learning can be established (see Figure 2). At this stage of course development instructors can identify the type of evidence needed to determine whether students are making progress toward or attaining the learning outcomes, one of the most important steps in course design (Richlin, 2006). In order to do so, one must attempt to identify what students need to do to provide convincing evidence that CJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough 51 they have indeed learned from participation in the course (Diamond, 2008). Some evidence may be present within the class itself; other evidence may need to be assessed through student assignments. Student outputs delineate what the student is responsible for in the course, including the nature of the work that will be required for assessment in the course. Skills Identity Learning Domain Cognitive Type of evidence to be observed Type of evidence to be assessed Key verbs: write, list, define, describe, identify, show, label, collect, examine; explain, interpret, contrast, estimate, differentiate, discuss, illustrate, compose, formulate, rewrite; analyze, connect, classify, decide, rank, test, measure, recommend, support, argue, compare, etc. Key actions: attempt, copy, imitate, mimic, play, perform, produce, perform masterfully, adapt, alter, customize, etc. Key actions: being attentive to a particular context, showing some new behaviours as a result of experience, showing some definite involvement or commitment, integrating a new value into one’s general set of values, giving something a ranking among one’s general priorities, etc. Figure 2. Determine Student Outputs It is critical that the descriptions of student outputs are framed in a way that is measurable. Too often the types of outputs given on syllabuses are vague and unmeasurable; that is, they contain no verbs for performance that identify how the instructor could know that the learner has made progress, or, worse, they are simply so vague as to be meaningless (Morrison et al., 2004; Mager, 1975; DiaCJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough 52 mond, 2008). Poorly crafted outputs contain phrases such as “have a thorough understanding of,” “demonstrate comprehension of,” or “have an awareness of” or make reference to ephemeral, internalized traits such as self-confidence, self-respect, or self-esteem. While these latter traits are not to be eschewed in the learning process, they are not measurable in and of themselves, even when linked to a particular action (e.g., “demonstrate confidence in presenting an oral report,” which may really measure only students’ acting ability and not their level of confidence). One must use clear language that will explain what the students will do to attain the learning outcomes identified (Bain, 2004); it may be framed by wording such as “Students completing this course will be able to,” with the actions that follow listed in ascending level of difficulty (Richlin, 2006, p. 120). Selecting the action verb that best describes the learning behaviour required of a student is difficult. Nevertheless, appropriate outputs are the key to the process, and without them the objectives will fall flat and the outcomes will not be attained (Morrison et al., 2004; Vella, 2000). Verbs that are vague, unclear, or open to multiple interpretations are not helpful guides for students and will not allow for clear assessment by the instructor. A description of student outputs should state what the learner is expected to do (performance), clarify the circumstances under which the learner is to do it (conditions), and describe the levels of acceptable performance (criteria) (Mager, 1975; Richlin, 2006; Diamond, 2008). For example, an assignment might state: “Given the circumstances at the end of the Second World War [condition], write [performance] a 10-page paper [condition] analyzing [performance] the Allies’ position on the surrender of Germany, using three academic resources [condition]. It will be graded according to the rubric set out in class [criteria].” It is important to focus on the proper learning domain (cognitive; psychomotor/skill; identity) and to develop the student outputs accordingly. Moreover, within each domain the emphasis on lower-order skills or higher-order skills, or an appropriate balance between them, must be maintained. Bloom’s taxonomy is the usual paradigm for determining the levels of learning in each of these domains (1956), although Anderson and Krathwohl (2001) have produced an important and helpful update that rightly reorders and redefines some of Bloom’s seminal work. Their update of the understanding of the cognitive learning domain has developed outcome-oriented language by changing all the labels from nouns to verbs in order to emphasize the active nature of learning (see figure 3, adapted from Anderson & Krathwohl, 2001 and from McKeachie & Svinicki, 2006). The inclusion of a variety of verbs allows instructors to select outputs that will appeal to different learning styles, another key aspect of student learning (McKeachie & Svinicki, 2006). Once course outputs are identified, the challenge remains to determine where they fit best into the course in terms of student learning, keeping in mind that a course must use sequencing or scaffolding. Not only must the assignments follow learning experiences in which students are given the tools necessary to complete the assignments, but the overall flow of the course must build from a foundation CJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough 53 up (see Weimer, 2002). In order to reflect this process adequately, the identification of outputs on the course syllabus should reflect the order in which they will be discussed within the class (likewise, learning outcomes and instructor objectives should mirror the scaffolding used in the course). Lower-Order Higher-Order Remember Retrieve pertinent facts from long-term memory Understand Construct new meaning by mixing new material with existing ideas Analyze Subdivide content into meaningful parts and relate the parts Apply Use procedures to solve problems or complete tasks Evaluate Come to a conclusion about something based on standards/ criteria Generate Reorganize elements into a new pattern, structure, or purpose Recognize Recall List Name Identify Show Define State Visualize Tell Describe Label Collect Examine Quote Record Interpret Exemplify Classify Summarize Compare Explain Describe Paraphrase Differentiate Demonstrate Defend Distinguish Paraphrase Predict Recognize Summarize Differentiate Organize Attribute Analyze Deduce Contrast Compare Distinguish Discuss Plan Devise Diagram Inspect Examine Categorize Appraise Solve Illustrate Calculate Use Interpret Manipulate Apply Modify Complete Implement Show Examine Relate Experiment Discover Classify Critique Judge Choose Estimate Defend Criticize Compare Rate Value Assess Estimate Measure Select Score Revise Create Plan Produce Design Hypothesize Support Schematize Report Justify Categorize Combine Compile Compose Organize Synthesize Ideas Comprehension  Scaffolding  Extension Figure 3. Cognitive Learning Domain At this point one can draw up a skeleton of the units in the course, without any content but showing the distribution of the student outputs that will be submitted to the instructor (for examples see Walvoord & Anderson, 1998). The individual course units can be placed within the skeleton so as to provide the students with the data and tools they need in order to complete a particular assignment. This runs against the tendency to want to provide students with “coverage” and focuses on empowering students to progress through the course. Grading is also an output consideration, since it goes hand in hand with the setting of student outputs (Hussey & Smith, 2002; Leskes & Wright, 2005). In asCJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough 54 sessing student work it is important to convey to the students the criteria used to measure the degree to which their output has attained the expectations for the assignment (and the criteria must match the aim of the assignment). Lack of adequate assessment will inhibit attempts by the instructor and the student to comprehend the effectiveness of the learning process (Bain, 2004; Mager, 1972). Once the student outputs have been articulated, they can be evaluated in terms of the extent to which each one provides “a valid and reliable measure of the targeted understandings,” “sufficient information to support inferences about each student’s understanding,” and “opportunities for students to exhibit their understandings through authentic performance tasks” (Wiggins & McTighe, 1998, p. 187, their emphasis). Plan Teaching and Learning Objectives In the third stage of the design process attention turns to what is often one of the first items to appear on a course syllabus: the objectives. Among the best college teachers, the syllabus represents the opening of a dialogue between instructor and student in which they explore together how learning will proceed and how it will be adjusted throughout the semester (Bain, 2004). The teaching objectives outline how this process will take place by conveying to students the intentions of the instructor (Mager, 1975). In a nutshell, the objectives represent all aspects of the course for which the instructor has responsibility and to which students can hold her accountable (see Figure 4). Course objectives explain what the instructor is responsible for in the course. Such input should be linked to one’s teaching philosophy and be manifest in one’s teaching style. Learning experiences that will prepare students to demonstrate their learning Skills Identity Learning Domains Cognitive Type of evidence (from Figure 2) Figure 4. Planning Teaching and Learning Objectives CJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough 55 For each student output the instructor needs to determine the kinds of learning activities and teaching support needed by the students in order for them to be able to produce the required outputs. These activities are not (yet) lesson plans but do delineate the types of knowledge and skills that will orient the students; the kinds of inquiry, research, problem solving, or experimentation in which the students will engage; and the nature of the formative feedback they will receive along the way (Wiggins & McTighe, 1998). Assess Connections Across the Design A final step allows the instructor to view the overall design and construct the course description (see Figure 5). Having listed the outcomes, outputs, and objectives in the appropriate columns, one can check whether they line up and support one another (this may take some rearranging of the order in which they are listed). If so, the course description can be composed. If not, the omissions must be addressed by determining whether to remove orphaned items or add material so that there is integration. This exercise provides an opportunity to check the consistency and coherence of the design and assess the learning outcomes, outputs, and objectives. OUTPUTS Skills Course Description Identity Learning Domain Cognitive OUTCOMES Figure 5. Assess Connections Across the Design CJHE / RCES Volume 41, No. 2, 2011 OBJECTIVES Learning (About) Outcomes / R. S. Ascough 56 Application and Results Although no formal quantitative testing of this model has been conducted, preliminary applications in my own courses indicate that students value the clear articulation of course outcomes and outputs. I first used this model in an upperyear theological studies course on the contexts and contents of first-century Christian letters. On the syllabus distributed to students I clarified the language I was using by adding some descriptions: • Objectives: “What you can expect from me in this course” • Outputs: “What I will expect from you in this course (and will measure)” • Outcomes: “What I hope you will develop through this course (but can’t measure)” Students’ comments during the course and on the course evaluations expressed appreciation for the clear connection between their assignments and the specific learning that course units and readings were designed to facilitate. In the summative formal course evaluations the average student ranking was a perfect 5/5 for each of the questions concerning the excellence of the course, the effectiveness of the instructor, and the amount of learning the course facilitated. To be sure, this is gratifying, but of more consequence is my own observation that students participated better in class discussions and produced essays that showed a deeper engagement with the material than had been my experience in previous years. Although my experience in using the tripartite design model in revising a course was positive, I found the process of thinking through outcomes, outputs, and objectives particularly helpful when designing a new course ancillary to my direct field of research and teaching. When invited to teach a short (five-week) course on business ethics, I employed the rubric to help establish what could and could not be accomplished in the given time frame, which resulted in figure 6. The final course description was transferred to the first page of the course syllabus and was followed by the objectives, outputs, and outcomes (see Figure 7). Not having taught the course before, I cannot compare the results with previous years of student learning. I can attest, however, that I found the process of identifying outcomes, outputs, and objectives helpful in focusing course material toward learning and in resisting a typical tendency (for me) to attempt to cover too much material at the expense of focused and engaged work in the classroom. As I have developed and worked with increasing attention to assessment in course design, I have had the opportunity to introduce my model to my institutional colleagues and to instructors across North America through consultations and workshops. The response has been favourable as many instructors have found, as did I, that confusion around the language of learning outcomes and assessment masks their true utility for improving teaching and learning. CJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough OUTPUTS Demonstrate pre-class preparation (reading and reflection) and articulate Understand the need for key concepts and ideas in ethics in business. discussions. OBJECTIVES Introduce students to analytical tools for resolving complex ethical issues in business contexts. Demonstrate an ability to identify and analyze ethical issues, conflicts, and responsibilities in business contexts. Provide examples of analyses of corporate codes of ethics and complex business cases. Learning Domain Cognitive OUTCOMES Understand four key ethical approaches. 57 Provide ample time and opportunities for students to participate in class discussions of cases. Skills Identity Construct and communicate rational, responsible, and realistic responses to a current business case. Develop students’ analytical skills for resolving ethical issues. Develop a clear sense of one’s own ethical obligations and personal responsibilities in pursuing a career. Course Description Develop skills to analyze, assess, and address ethical conflicts or dilemmas in business cases. Demonstrate critical awareness of values and morality in case discussions in class. Facilitate students’ understanding of themselves as ethical thinkers through various in-class analytic and evaluative tools. Demonstrate critical Identify factors that will awareness of biases. challenge and change one’s moral compass. Acquire courage to make principled choices in the face of ethical challenges. This course is designed to provide students with opportunities to apply ethical reasoning to the demands of business behaviour through the examination of case studies. We will apply four key ethical principles to case studies in order to explore the various ways in which business practitioners and ethical theorists have addressed issues that arise in business contexts. Figure 6. Business Ethics Course Design CJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough 58 OBJECTIVES • to expose students to the complexity of ethical decision making in business organizations • to provide ample time and opportunities for students to participate in class • to develop students’ analytical skills for resolving ethical issues • to facilitate students’ understanding of themselves as ethical thinkers OUTPUTS By the end of the course students will be able to • demonstrate pre-class preparation (reading and reflection) and comprehension of key concepts and ideas during class discussions • identify and analyze ethical issues, conflicts, and responsibilities in business contexts • construct and communicate rational, responsible, and realistic responses to ethical issues • articulate their own values and morality Students will exhibit this through • pre-class preparation and substantive contribution to class discussions and activities (30%) • a written analysis of a code of ethics from a business setting (20%) • a take-home examination that uses two key ethical principles to analyze a recent business ethics case in the news (50%) OUTCOMES • develop skills to analyze, assess, and address ethical conflicts or dilemmas in business cases • comprehend four core ethical approaches and the need for ethics in business • develop a clear sense of one’s own moral obligations and personal responsibilities in pursuing a career and the factors that will challenge and change one’s moral compass • acquire courage to make principled choices in the face of ethical challenges Figure 7: Business Ethics Syllabus Conclusion Pocklington and Tupper (2002) note in their rather scathing, if not depressing, outlook on Canadian universities that the future of teaching and learning involves commitment to moving beyond “adult daycare” to challenge students and stimulate engagement. Embracing the positive aspects of using learning outcomes will assist instructors in the design of such educational experiences by providing a structured mechanism for developing courses that integrate student needs, instructor expertise, and disciplinary requirements. The investment of thought and CJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough 59 effort in developing learning outcomes, student outputs, and course objectives will increase student learning while meeting the demand from institutions and accrediting agencies for precise and measurable ways of demonstrating that learning. References Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York, NY: Longman. Bain, K. (2004). What the best college teachers do. Cambridge, MA: Harvard. Bloom, B. S. (Ed). (1956). Taxonomy of educational objectives: Classification of educational goals. Handbook 1: Cognitive domain. New York, NY: David McKay. Bresciani, M. J. (2006). Outcomes-based academic and co-curricular program review: A compilation of institutional good practices. Sterling, VA: Stylus Publishing. Côté, J. E., & Allahar, A. L. (2007). Ivory tower blues: A university system in crisis. Toronto, ON: University of Toronto Press. Diamond, R. M. (2008). Designing and assessing courses and curricula: A practical guide. San Francisco, CA: Jossey-Bass. Fish, S. (2008). Save the world on your own time. Oxford, UK: Oxford University Press. Hussey, T., & Smith, P. (2002). The trouble with learning outcomes. Active Learning in Higher Education 3(3), 220–233. Kronman, A. T. (2007). Education’s end: Why our colleges and universities have given up on the meaning of life. New Haven, CT: Yale University Press. Leskes, A., & Wright, B. D. (2005). The art & science of assessing general education outcomes: A practical guide. Washington, DC: Association of American Colleges and Universities. Mager, R. F. (1972). Goals analysis: How to clarify your goals so you can actually achieve them. Belmont, CA: Fearon. Mager, R. F. (1975). Preparing instructional objectives (2nd ed.). Belmont, CA: Fearon. Maki, P. L. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: American Association for Higher Education. McKeachie, W. J., & Svinicki, M. (2006). McKeachie’s teaching tips: Strategies, research, and theory for college and university teachers (12th ed.). College Teaching Series. Boston, MA: Houghton Mifflin. Morrison, G. R., Ross, S. M., & Kemp, J. E. (2004). Designing effective instruction (4th ed.). Hoboken, NJ: Wiley. Performance Measurement Net. (2004). Outcomes vs outputs: Why it’s important to understand the difference? Retrieved from http://www.performance-measurement.net/news-detail.asp?nID=221 CJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough 60 Pocklington, T., & Tupper, A. (2002). No place to learn: Why universities aren’t working. Vancouver, BC: UBC Press. Pratt, D. D., & Associates. (1998). Five perspectives on teaching in adult and higher education. Malabar, FL: Krieger. Richlin, L. (2006). Blueprint for learning: Constructing college courses to facilitate, assess, and document learning. Sterling, VA: Stylus Publishing. Shanahan, T. (2009). Accountability initiatives in higher education: An overview of the impetus to accountability, its expressions and implications. In Accounting or accountability in higher education: Proceedings from the January 8, 2009 OCUFA conference (pp. 3–15). Sims, R. R. (2002). Teaching business ethics for effective learning. Santa Barbara, CA: Quorum Books. Tapscott, D. (2009). Grown up digital: How the Net Generation is changing your world. Toronto, ON: McGraw-Hill. Vella, J. (2000). Taking learning to task: Creative strategies for teaching adults. San Francisco, CA: Jossey-Bass. Walvoord, B. E., & Anderson, V. J. (1998). Effective grading: A tool for learning and assessment. San Francisco, CA: Jossey-Bass. Weimer, M. (2002). Learner-centered teaching: Five key changes to practice. San Francisco, CA: Jossey-Bass. Wiggins, G., & McTighe, J. (1998). Understanding by design (expanded 2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development. Woodhouse, H. (2009). Selling out: Academic freedom and the corporate market. Montreal, QC: McGill-Queen’s University Press. ACKNOWLEDGEMENTS This research was funded by the Wabash Center for Teaching and Learning in Theology and Religion (Lilly Endowment Inc.) in Crawfordsville, Indiana. I am also grateful for support and feedback from Denise Stockley at the Centre for Teaching and Learning at Queen’s University. Many colleagues at Queen’s and elsewhere in North America helped me test and refine these ideas through workshops, consultations, and discussions. Contact Information Richard S. Ascough School of Religion Queen’s University Kingston, Ontario K7L 3N6 Telephone: 613-533-6000 (ext. 78066) [email protected] CJHE / RCES Volume 41, No. 2, 2011 Learning (About) Outcomes / R. S. Ascough 61 Richard S. Ascough is associate professor of New Testament at the School of Religion, Queen’s University, Kingston, Ontario. His teaching and research specialty focuses on Christian origins and Greco-Roman religious culture. He also offers numerous teaching and learning workshops and consultations across North America. He has been recognized for his innovative teaching in a number of ways, including both of the university-wide teaching awards at Queen’s: the Alumni Award for Excellence in Teaching (2002) and the Chancellor A. Charles Baillie Teaching Award (2009).  CJHE / RCES Volume 41, No. 2, 2011