The Canadian Journal of Higher Education 111 Immediate Assessment Technique La Feedback revue canadienne d’enseignement supérieur Volume XXXV, No. 4, 2005 pages 111 - 131 The Immediate Feedback Assessment Technique: A Learner-centered Multiple-choice Response Form DAVID DiBATTISTA Brock University ABSTRACT The Immediate Feedback Assessment Technique (IFAT) is a new multiple-choice response form that has advantages over more commonly used response techniques. The IFAT, which is commercially available at reasonable cost and can be used conveniently with large classes, has an answer-until-correct format that provides students with immediate, corrective, item-by-item feedback. Advantages of this learner-centered response form are that it: (a) actively promotes learning; (b) allows students’ partial knowledge to be rewarded with partial credit; (c) is strongly preferred by students over other response techniques; and (d) lets instructors more easily maintain the security of multiple choice (MC) items so that they can be reused from one semester to the next. The IFAT’s major shortcoming is that grading must be done manually because it does not yet have a compatible optical scanning device. Helpful suggestions are presented for instructors who may be considering using the IFAT for the first time. The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 112 D. DiBattista RÉSUMÉ La Technique D’Évaluation Immédiat (Immediate Feedback Assessment Technique ou IFAT) est un nouveau formulaire pour examens à choix multiple qui a plusieurs avantages. Le IFAT, disponible à un prix raisonable et convenable pour les cours suivis par de nombreux étudiants, est constitué d’un format dans lequel les édudiants selectionnent alternative-paralternative parmi les choix disponibles jusqu’à ce que la réponse correcte soit indiquée. En suite, la correction est automatique et informe la réponse correcte immédiatement. Le IFAT a plusieurs avantages: (a) il favorise l’apprentissage; (b) les étudiants peuvent obtenir des points partiels avec connaissances partiels; (c) les étudiants préferent ce formulaire à comparer à autres formats à choix multiple; et (d) les instructeurs peuvent maintenir plus facilement leurs questions et alternatives en sécurité et les réutiliser au cours des prochaines sessions. Le défaut principal du IFAT est que la notation est manuele car il n’y a pas encore de lecteur optique compatible avec ce formulaire. Des suggestions utiles sont ici données pour les instructeurs qui envisagent d’utiliser cette technique pour la première fois. INTRODUCTION Multiple-choice (MC) testing is by far the most widely used selectedresponse assessment technique in academic settings. Generally speaking, well-constructed MC tests permit broad sampling from the course material covered, and they can be used to assess achievement at various levels of cognitive ability (Bloom, 1956). Furthermore, MC tests have high reliability, and scoring can be done rapidly and efficiently. Although MC tests are not suitable for assessing certain abilities, such as writing skill and creativity, their many advantages make them a practical assessment technique, especially when used with large classes. With most MC tests, students either mark their answers directly on the test paper or they use a pencil to fill in “bubbles” on a response sheet that is later run through an optical scanner and graded automatically. Of course, these response techniques do not provide students with feedback about their The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 Immediate Feedback Assessment Technique 113 performance either on the individual MC items or on the test as a whole. Furthermore, it is usually only days, or even weeks, later that students learn their test grade, and often they never find out which items they answered correctly and which they answered incorrectly. Several years ago a new MC response form called the Immediate Feedback Assessment Technique (IFAT) entered the marketplace.1 Developed by Professor Michael Epstein of Rider University, the IFAT provides students with immediate, corrective, item-by-item feedback even as they write the test, and it also allows them to determine their grade as soon as they have finished the test. The IFAT is commercially available at a reasonable price, and unlike computerized MC testing, it can be used conveniently even with very large classes. In addition, the immediate, corrective feedback provided by the IFAT directly contributes to students’ learning, allowing the MC test to serve as both an assessment instrument and a learning instrument. Moreover, the IFAT is student-friendly, with the great majority of students reporting that they would prefer to use the IFAT rather than a traditional response form on their MC tests. Thus, in contrast to the response techniques that are commonly used with MC tests, the IFAT is a learner-centered technique that may enhance students’ learning, motivation, and success (McCombs, 2003). Having now used the IFAT with my students for more than five years, I am convinced that it has distinct advantages over the traditional MC response techniques. In this article, I will describe the IFAT and outline some of the advantages it provides for both students and instructors. These advantages are summarized in Table 1, with further details provided in the text below. In addition, based on my own experiences, I will provide a few tips that may be helpful for instructors who wish to give the IFAT a try. How the IFAT Works Figure 1 shows an IFAT form that a student has used to take a 10-item quiz. The form has a series of boxes corresponding to the four alternatives for each of 50 MC items. (A five-alternative version of the IFAT is also available.) Each box is covered by an opaque, waxy coating like that found on scratch-off lottery tickets. The one box associated with the correct alternative for a particular item has a small star in it, and the other boxes The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 114 D. DiBattista Table 1 Summary of IFAT Advantages Students learn more with the IFAT than with other response techniques. The IFAT allows the instructor to award partial credit. Students show a strong preference for the IFAT. The IFAT is easy to use and does not interfere with students’ performance. The IFAT does not disadvantage any identified groups of students. Students feel that the IFAT is fairer than traditional MC answer forms. The IFAT allows students to obtain immediate feedback on their overall test performance. The IFAT eliminates the need to return tests to students, permitting the instructor to keep MC items secure so that they can be reused on future tests. The IFAT shows students that the instructor cares about their learning, and not solely about evaluating their performance. are blank. For each MC item, the student chooses the alternative he or she believes to be correct and scratches the coating off the corresponding box. If the student uncovers a star, this provides confirmation that the chosen alternative was correct. In this case, the student receives full credit and goes on to the next item. However, if the box that the student scratches is blank, the chosen alternative was not correct. The student then must reflect on the problem once again, reconsider the remaining alternatives, and continue scratching boxes until the star is uncovered. Partial credit is normally awarded when students find the star on either the second or third attempt. Of course, the student’s final selection for each item is always the correct answer. Several aspects of the IFAT are particularly noteworthy. First, the IFAT provides immediate feedback on performance. Students do not have to wait for days to find out their test grade and to learn which of their answers were correct and which were incorrect. Second, by allowing the student to continue to respond until the star appears, the IFAT provides corrective feedback on items that are answered incorrectly on the first attempt. Thus, The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 Immediate Feedback Assessment Technique 115 Figure 1. A 50-item, four-alternative IFAT form that has been used for a 10-item quiz. For each item, the star indicates the location of the correct answer. Although this form has not yet been graded, the form provides space to record the student’s score for each item. The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 116 D. DiBattista the student is not simply told “Your answer was incorrect,” but instead is given the opportunity to continue working and thus to learn the correct answer for every test item. Finally, after completing the test, the student can easily obtain feedback on overall performance by computing his or her own MC test score before handing in the test materials. The 50-item, four-alternative IFAT form shown in Figure 1 is available in 12 different versions, with the pattern of stars in each version arranged in a unique configuration. When constructing a test, the instructor arranges the MC items so that the correct answer for each item corresponds to the location of the star on the version of the IFAT form that is to be used. Each version of the IFAT is identified by a code number in the lower right hand corner of the form. (The code number is blanked out in Figure 1). Customized versions of the IFAT can also be ordered. Promoting Learning As mentioned above, a distinct advantage of the IFAT is that it provides students with immediate, corrective feedback on each MC item. The importance of providing prompt feedback in educational settings is well recognized, with research showing that immediate feedback is more effective than delayed feedback in promoting learning in classroom situations (Kulik & Kulik, 1988; Sassenrath, Yonge, & Schrable, 1968). Indeed, Chickering and Gamson (1987) have emphasized the importance of providing prompt feedback by citing it as one of seven principles for good practice in undergraduate education. Furthermore, feedback that is corrective, like that provided by the IFAT, has been shown to promote learning more effectively than feedback that provides information only about the correctness or incorrectness of responses (Bangert-Drowns, Kulik, Kulik, & Morgan, 1991; Kluger & DeNisi, 1996). By providing students with feedback that is both immediate and corrective, the IFAT sets the stage for students to learn even while they are being tested. Research studies in both classroom (Dihoff, Brosvic, & Epstein, 2003; Epstein, Epstein & Brosvic, 2001) and laboratory settings (Epstein et al., 2002) have consistently shown that students do indeed learn substantially more when they use the IFAT rather than an optically scanned (OS) response form that provides no feedback on performance. For example, The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 Immediate Feedback Assessment Technique 117 Epstein, et al. (2001) had introductory psychology students use either the IFAT or an OS form on a series of unit tests, and then presented some of the same MC items on the final examination. All students used the OS form on the final examination. Students who had used the IFAT and OS form on the unit tests had similar levels of performance on final-examination items that had not appeared on unit tests. However, on items that had previously appeared on unit tests, the IFAT group scored 8.3 percentage points higher than the OS group. Moreover, IFAT students were more than twice as likely as OS students to respond correctly on the final examination to items that they had previously answered incorrectly on the unit tests. Thus, the IFAT not only serves as an effective assessment tool, but in contrast to other commonly used MC response techniques, it also makes a significant contribution to students’ learning. Partial Credit With essays and other types of constructed-response test items, it is common practice to reward students’ partial knowledge by awarding partial credit. However, the awarding of partial credit in the context of MC testing is quite uncommon, and although several methods have been developed for this purpose, these tend to be cumbersome and to have a variety of shortcomings (Ben-Simon, Budescu, & Nevo, 1997). In contrast, the IFAT’s answer-until-correct format makes it a simple matter for the instructor to award partial credit on MC items. When grading MC tests that make use of a traditional answer form, most instructors use a number-correct grading scheme (NCGS), awarding full credit for those items that are answered correctly, and no credit for items answered incorrectly. Thus, a NCGS awards credit on an all-ornothing basis, and the awarding of partial credit is not an option. As it happens however, students considering a MC item will often have partial knowledge. For instance, a student may not be absolutely certain which alternative is correct, but may nevertheless be able to determine that, say, two of the alternatives are far more likely than the others to be correct. With a NCGS, the student must make a choice between these two alternatives, earning either full credit if the alternative chosen is correct or no credit at all if it is incorrect. In contrast, students using the IFAT may be awarded The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 118 D. DiBattista full credit for finding the correct answer on the first attempt, and they may be awarded progressively less credit for answering correctly on later attempts. Thus, with the IFAT, students may be awarded partial credit in recognition of their partial knowledge of the material covered in the test item. An added benefit of a partial-credit grading scheme (PCGS) is that it provides incentive for students to continue seeking the answer to every MC item even when they are incorrect on the first attempt. If a NCGS is used with the IFAT, students who answer an item incorrectly on the first attempt may have little incentive to continue searching for the correct answer, and as a result, they very well may not do so. Instead, they may choose to move right along to the next MC item rather than to waste valuable time that they could use to earn marks elsewhere on the test. Of course, when this happens students will not get the benefit of the immediate, corrective feedback that is essential if learning is to occur when the IFAT is used. When I use the IFAT with my students, I use a PCGS that awards 100, 25, 10 and 0 percent of full marks for correct answers given on the first, second, third, and fourth attempts, respectively. I should add that these values are not carved in stone, by any means. Some instructors I know who use a PCGS are somewhat more generous in awarding marks for second and third attempts, while others are less generous. However, my students and I have been quite happy with this 100-25-10-0 arrangement. In a survey that I conducted several years ago, only 6% of my students felt that this arrangement was not fair and reasonable. Compared to a NCGS, the PCGS that I use naturally leads to an increase in the average test grade. In casual discussions with instructors, I have found that many of them expect that this increase will be a full letter grade or even more. However, I have found that the actual increase is typically much smaller. For instance, for a MC test of normal difficulty, I have found that my 100-25-10-0 grading scheme increases the class average by about 5 percentage points. In one typical test that I gave, the class average would have been 64.3 with a NCGS, and it was 69.4 with the PCGS. Instructors who are concerned about possible grade inflation can compensate for the slightly higher grades produced by the PCGS by making a few of the easier MC items a bit more difficult than would otherwise be the case. The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 Immediate Feedback Assessment Technique 119 Students’ Acceptance of the IFAT Yet another advantage of the IFAT is that students not only readily accept its use, but they actually welcome it and would like to see its use expanded. When my colleagues and I surveyed a large group of undergraduates using the IFAT for the first time (DiBattista, et al., 2004), 84% said that they would like to use the IFAT in all of their courses that have MC tests. A number of factors seem to underlie this highly favourable response to the IFAT. For example, 90% of students said that the IFAT was easy to use, and although they were using it during a major examination, 78% nevertheless indicated that the IFAT made the test feel something like a game. Some even commented that using the IFAT made the test fun! In addition, 64% of students found the IFAT to be fairer than the traditional answer form they were accustomed to using, and 84% felt that they learned more with the IFAT than they would have with a traditional answer form. We have also found that more than half of first-time users indicate that they would prefer to use the IFAT even if the grading scheme did not allow for partial credit, suggesting that the availability of partial credit is not a critical factor influencing students’ acceptance of the IFAT (DiBattista & Gosse, in press). Also we have found that acceptance of the IFAT seems to be essentially independent of a number of test-related variables and students’ personal characteristics (DiBattista, et al., 2004; DiBattista & Gosse, 2004). For example, preference for the IFAT has not been found to be related to variables such as overall test performance, performance on the MC portion of the test, age, number of courses previously taken, self-reported degree of preparedness for the test, perception of test difficulty, level of test anxiety, and impulsiveness. Overall, reactions to the IFAT are overwhelmingly positive across students with a broad range of characteristics. In addition, use of the IFAT sends students the message that the instructor really cares about their learning and is not only interested in evaluating them. Thus, use of the IFAT may contribute to the important but usually neglected goal of creating a more positive reaction to testing among students (McMorris, Boothroyd, & Pietrangelo, 1997). The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 120 D. DiBattista Reuse of MC Items Because creating high-quality MC items requires a considerable amount of time, effort, and skill (Haladyna, 1994), it is extremely helpful for instructors to be able to reuse their MC items from one semester to the next rather than to use them only once. Accordingly, the instructor who uses a traditional MC response technique is faced with a dilemma. If the graded MC test papers are returned to the students so that they can receive (delayed) feedback on their performance, the MC items will no longer be secure. As a result, it is likely that the items will circulate through the campus grapevine and make their way into the hands of some of the students who will take the course in a subsequent semester. Because this breach of item security makes it unwise to use these same items again, the instructor is forced to take the time to develop a new set of MC items each time a course is offered. Of course, it is possible for the instructor to maintain the security of MC items when a traditional response technique is used, but the various strategies that are available to do so are plagued by shortcomings. For example, the instructor may inform students of their grades, but not return their test papers to them. Unfortunately, when this strategy is used, students receive no feedback at all on their performance on the individual MC items and therefore have no opportunity to learn from their mistakes. Even if the instructor reviews the MC items during class time, or allows students to review the items during office hours, a major drawback is that feedback is delayed rather than immediate. In addition, covering items in class can be quite time-consuming, especially if all of the items on a test are to be reviewed, and instructors may well feel that there are more productive uses of precious class time. Furthermore, instructors too often find that when students are allowed to review their tests during office hours, very few of them actually take advantage of the opportunity to do so. In contrast, with the IFAT, it is a relatively simple matter to maintain item security. Because students obtain immediate, corrective feedback on every MC item while they write the test, there is no need for the instructor to return the graded test papers to students. Rather, the instructor can inform students of their grades, which many of them will have already computed themselves before handing in the test, and if desired, spend a few minutes The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 Immediate Feedback Assessment Technique 121 in class covering selected MC items that may be of particular interest. And of course, students with concerns about their overall performance or about specific MC items will always have the option of consulting with the instructor during office hours. Thus, because the IFAT lets instructors conveniently maintain the security of their MC items, it allows them to reuse items from one semester to the next, and it frees them from the timeconsuming task of developing fresh items each time a course is offered. Shortcomings and Concerns in Using the IFAT Having focussed thus far on the IFAT’s advantages, it is now only fair to consider its shortcomings, which are summarized in Table 2. In my opinion, the major drawback is that there is not yet an optical scanning device compatible with the IFAT. As a result, all IFAT grading must be done by hand. More importantly, however, the computerized item analysis that is easily carried out when traditional MC response sheets are graded using an optical scanner is not available with the IFAT. Since I started using the IFAT, I have often examined my own MC items and computed measures of item difficulty and discrimination, which are useful in determining whether individual items should be either deleted or modified before being used again (Haladyna, 1994). However, this is a time-consuming and tedious process because information about students’ responses to individual MC items must be entered into a spreadsheet by hand before being analyzed. Clearly, the development of an optical scanning device that is compatible with the IFAT would be a huge step forward. Table 2 Summary of IFAT Shortcomings There is no commercially available optical scanning device for scoring IFAT. Item analysis is a laborious and time-consuming process. The IFAT may increase the risk that cheating will occur. Layout of the IFAT form could be improved. Test construction takes a bit more time with the IFAT. IFAT forms cost a bit more than optically scanned response forms. The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 122 D. DiBattista A second drawback of the IFAT is that if it is not used carefully, it can increase the risk of cheating. Keep in mind that with a standard MC response technique, a student who looks over at a classmate’s response form will learn what the classmate thinks is the correct answer for that item. In contrast, with the IFAT, a student who looks over at someone else’s response form and sees a star will know the correct answer for that item. For this reason, the temptation to look at nearby response forms may be greater when the IFAT is used. Generally speaking, this problem can be overcome in most situations by using alternate seating so that students are not sitting too closely together during the test. If alternate seating is not possible, it may be necessary to develop two versions of the test, each corresponding to a different version of the IFAT form. In my opinion, the IFAT’s other shortcomings are all relatively minor. For example, students sometimes say that the boxes on the form are a little too close together, and that this makes it too easy to accidentally scratch the wrong box when answering a question. However, my discussions with students suggest that crowding seems to become less of an issue for them after they have used the form a couple of times. Nevertheless, it seems to me that with just a bit more spacing between adjacent boxes, the form would be somewhat easier to use. Also, test construction takes a little bit more time with the IFAT because the instructor must arrange the MC items so that the correct answers map onto the version of the IFAT that is being used. Finally, IFAT forms are somewhat more costly than the OS response forms that are so widely used. For example, at my home university, it costs about $100 to have 1,000 OS response forms graded, while 1,000 IFAT forms (250 copies of 4 different versions) can be purchased for $190 (all values in U.S. funds). However, in considering the relative costs, the IFAT’s many advantages over OS response forms must be taken into consideration. A concern expressed by many instructors is that students may need more time to complete a test when using the IFAT rather than a standard response technique. Of course, any significant increase in the time requirement might make it necessary to put fewer MC items on the test. However, in my personal experience, I have seen no evidence that students need more time when using the IFAT, and as a result, I make my tests just as long now The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 Immediate Feedback Assessment Technique 123 as I did before adopting the IFAT. The results of our research also suggest that using the IFAT has little or no effect on the time needed to complete a test (DiBattista, et al., 2004) . Specifically, we have found students to be just about equally divided as to whether using the IFAT takes more or less time. Almost half say that using the IFAT takes somewhat more time because if they answer an item incorrectly on the first attempt, they keep working on the item until they find the right answer. Just as many students say that using the IFAT takes less time because the immediate, corrective feedback it provides completely eliminates the second guessing that can occur with a standard response form. Thus, if using the IFAT does increase the amount of time needed to answer MC questions, the effect is likely to be minor. Consequently, I do not see a need to put fewer MC items than usual on a test when using the IFAT. Tips for Using the IFAT After having considered the strengths and shortcomings of the IFAT, some instructors may decide to give it a try. Of course, instructors inevitably face challenges whenever they decide to adopt any new technology in the classroom, and for this reason, I present below a few suggestions that may prove helpful to those who are considering using the IFAT for the first time. 1. Tell your students about the IFAT well in advance. I always give my students full information about the IFAT several weeks before the first test. After describing how the IFAT works, I tell them about the research that shows that it actively promotes learning and is preferred by students. I also find it helpful to give students small scraps of IFAT forms during class time so that they can become familiar with the technique. (My tests always have fewer than 50 MC items, so after each test, I cut off the unused portion at the bottom of the submitted IFAT forms and use them for this purpose.) Because most of my students have never encountered the IFAT before, I like to create a positive attitude about using this new technology by referring to it as the “scratch-and-win” response form. The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 124 D. DiBattista 2. Make the test preparation process as easy for yourself as you can. After selecting the particular IFAT form that you will use for your test, you must arrange the MC items on your test so that the correct answers are congruent with the boxes containing the stars on the IFAT form. I suggest that you first take the IFAT form that you will be using and count the number of A, B, C, and D responses that you will need to have on your test. Then count the number of A, B, C, and D responses that are on the draft of your test. For instance, suppose that you are going to administer a 30-item, four-alternative test, and the IFAT form calls for the following response frequencies: A=8, B=6, C=9, and D=7. If the initial draft of your test has response frequencies of A=10, B=5, C=9, and D=6, you will take two items that have A as the correct answer and rearrange the alternatives so that one has B as the correct answer and another has D as the correct answer. When you have done this, you will arrange the 30 test items so that the order in which they appear on the test matches the pattern of correct answers on the IFAT form. When I am modifying the individual items as described in the preceding paragraph, I always make sure to shop around for the items that are most easily changed. For instance, if I have to change the correct answer for one of my test items from A to D, I first look for an item that has A as the correct answer and numerical values as the alternatives. I then simply reverse the order of the four alternatives, thus making the value that was alternative D become alternative A. Also, when working with the draft MC items on my computer, I routinely use the highlighting function of the word-processing program to make the correct alternative for each item more easily identifiable. When I have finished constructing the test, I remove the highlighting before printing the finished version. 3. Be sure that the answers to the MC items match the pattern on the IFAT form. When a traditional response technique is being used, having an error in the answer key is usually not a major problem, provided it is detected before the test grading is complete. However, the IFAT is not as forgiving of errors–a lesson that I have learned the hard way! During one test that I gave, one of my best students raised her hand with a puzzled look on her The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 Immediate Feedback Assessment Technique 125 face. She was absolutely sure that the correct answer for Question 17 was B, but the star on her IFAT form was in box C. Naturally, she was baffled by the discrepancy. As it happens, I had made an error when preparing the item, putting the correct alternative in position B rather in position C, where it should have been in order to be congruent with the IFAT form. To avoid running into this type of embarrassing problem, it is important to take pains to ensure that the correct answers to the test items are perfectly mapped onto the stars on the IFAT form. 4. Use a partial-credit grading scheme. As mentioned above, the grading scheme that I use awards 100, 25, 10 and 0 percent of full marks for correct answers given on the first through fourth attempts, respectively. These amounts have worked well for me and I recommend them. A PCGS not only gives students credit for their partial knowledge, but unlike a NCGS, it also gives students a strong incentive to keep working on items until they find the star and learn the correct answer. Keep in mind that a PCGS may boost the class average by about five percentage points. To avoid grade inflation, you can compensate for the awarding of partial credit by slightly adjusting the average difficulty of the test items, perhaps by modifying some of the easier items to make them somewhat more difficult. 5. Take steps to minimize copying. The vast majority of students do not intend to cheat when they go into a test, and instructors have a responsibility not to put honest students into situations that may tempt them to cheat. Because the stars on the IFAT form indicate correct answers, students – even honest students – may be tempted to look over at a classmate’s response form. Alternate seating and alternate versions of the test can greatly reduce the likelihood of this type of behaviour. In addition, I always make it a point to remind students that it is their responsibility to ensure that classmates cannot see their IFAT form. 6. Whenever using the IFAT, give each student a copy of “Suggestions for using the IFAT form” (see Table 3). I always give students as much information as possible about the IFAT before they use it on a test. In addition, I distribute copies of the suggestions in Table 3 right along with the test papers and IFAT forms. The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 126 D. DiBattista Table 3 Suggestions distributed to students using the IFAT The corner of your ID card is perfect for scratching the boxes. Scratch the IFAT form with care so that you do not tear it. Use the edge of this piece of paper to ensure that you are scratching on the correct line of the IFAT form. The star may appear anywhere within a box. It is not always in the same location, so scratch the entire box. If you scratch any portion of a box, it will be counted as a response. Think before answering! Once a box is scratched, you cannot “unscratch” it! You can earn part marks even if your first attempt is not correct. Keep scratching until you find the star. You can immediately determine your mark on the multiple-choice portion of the test. If you choose to do this on the form, please do it neatly! Keep your IFAT form secure! It is your responsibility to ensure that other students cannot see your answer sheet. Place it face down when not in use. These suggestions are very practical and most have already been touched upon previously. One point that has not already been mentioned is that the star that signifies the correct answer is not always in exactly the same location within the box (see Figure 1). I inform students that scratching any portion of a box will count as a response, and because the location of the star is not constant, they cannot scratch off just one tiny portion of the waxy coating to see whether a box contains the star. Also, I always make sure that the handout in Table 3 is distinctively coloured and easy to spot. This allows proctors to easily identify it as a legitimate item for students to possess rather than a crib sheet. An added bonus is that students can use the edge of the handout to ensure that they are scratching on the correct line of the IFAT form. 7. Take steps to insure the security of your MC test items. If you want to use the same MC test items from one semester to the next, it is wise to keep them secure. To prevent students from taking the test script with them when they leave the test room, I always include the following wording as part of the instructions on my tests: “You must submit this test script, the test booklet(s), and the IFAT form before leaving The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 Immediate Feedback Assessment Technique 127 the room. Failure to submit all items will result in your getting a grade of zero!” This is a major deterrent, and I have never once had a student walk out with the test script. As added insurance, I also always have my students write the answers to some portion of the non-MC portion of the test directly on the test script. For instance, I might make it a point to have some short answer items on the test, instructing students to write their answers in the space provided on the test script. The instructions are in large, boldface print that students cannot miss, and because some of their answers are written right on the test script, they have a strong incentive to hand it in. Finally, when the tests have been graded, I always tear off the pages that have the MC items on them and return only the non-MC portion of the test to the students. 8. Make the grading process as easy for yourself as you can. Because IFAT forms must be graded manually, you may wish to let your students grade their own forms, and then spot check the grading for accuracy. This works quite well for shorter tests, and it can save you quite a bit of grading time. However, at the end of a longer test, such as a threehour final exam, students often won’t want to be bothered grading their own IFAT form, and I have found that although I tell them that they can grade the form if they wish, very few students actually do it. I have developed a system that works well for me when I have to grade the forms myself. Rather than putting the student’s score for the item in the margin of the IFAT form, I use the symbols , X, –––, and 0 to indicate that the student has answered correctly on the first, second, third or fourth attempt, respectively. I then count up the symbols and use the form in Figure 2 to compute the student’s grade on the MC portion of the test. I write the student’s name on the form and fill in the middle and rightmost columns. The completed grading sheet is placed in the test booklet and returned to the student. The IFAT form is not returned, nor are the MC questions. I find that for a 30-item test, it takes me about an hour to grade 40 IFAT forms. The grading sheet shown in Figure 2 has been used to grade a 30-item test that makes use of my PCGS. As you can see, this student has earned 22.7/30 marks on the test and would have earned only 21/30 if a NCGS had been used. Also, by adding up the values in the middle column, I get The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 128 D. DiBattista Figure 2. Grading form for use with the IFAT. This form has been used to grade a 30item MC test with a partial-credit grading scheme. some verification that I have counted the symbols properly; if the sum in this column had not been 30, I would have known that I had made an error somewhere. CONCLUSIONS The IFAT provides an excellent alternative to the response techniques that have been traditionally used for MC testing. Among its many advantages are that it is commercially available at a reasonable cost and can be used even with very large classes. In addition, the IFAT is learner-centered. It is enthusiastically accepted by students, and by virtue of its immediate, corrective, item-by-item feedback, it actively promotes learning. Furthermore, the IFAT allows instructors to reward students’ partial knowledge by awarding partial credit, and it makes it easier for them to reuse MC items from one semester to the next. The most serious shortcoming of the IFAT, I believe, is that grading must be done by The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 Immediate Feedback Assessment Technique 129 hand. In my opinion, the advantages of using the IFAT far outweigh the disadvantages, and for this reason, I now always use the IFAT when giving MC tests. Finally, although the focus of this paper has been on using the IFAT in testing situations, it must be mentioned that the learner-centered IFAT is a versatile tool that instructors can use in a variety of settings. For example, some instructors let students use the IFAT when doing homework and assignments. Because it provides immediate, corrective feedback, the IFAT allows students to assess their learning even as they study. The IFAT can also prove extremely valuable in the context of team learning in the classroom (Michaelsen, Knight, & Fink, 2004). Because feedback is immediate, the IFAT encourages more thorough consideration of MC items, fosters more effective communication within teams, and contributes to greater learning. For a demonstration of the use of the IFAT in a teamlearning setting, readers may visit www.teambasedlearning.org. NOTES 1 Information about the IFAT, including details about cost and availability, can be obtained at www.epsteineducation.com. The author has no financial interest whatsoever in the IFAT. Correspondence information: David DiBattista Department of Psychology Brock University St. Catharines, Ontario Canada L2S 3A1 E-mail: [email protected]. Telephone: 905-688-5550, ext. 3467 Fax: 905-641-5076 The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 130 D. DiBattista REFERENCES Bangert-Drowns, R.L., Kulik, C.C., Kulik, J.A., & Morgan, M.T. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61, 213-238. Ben-Simon, A., Budescu, D.V., & Nevo, B. (1997). A comparative study of measures of partial knowledge in multiple-choice tests. Applied Psychological Measurement, 21, 65-88. Bloom, B. S. (1956). Taxonomy of educational objectives: The classification of educational goals. New York : McKay. Chickering, A.W., & Gamson, Z.F. (1987). Seven principles for good practice in undergraduate education. American Association for Higher Education Bulletin, 39, 3-7. DiBattista, D., & Gosse, L. (2004). Test anxiety, impulsiveness, and students’ preference for the Immediate Feedback Assessment Technique for multiplechoice testing. Presented at the meeting of the Canadian Psychological Association, St. John’s, Newfoundland. DiBattista, D., & Gosse, L. (in press). Test anxiety and the Immediate Feedback Assessment Technique. Journal of Experimental Education. DiBattista, D., Mitterer, J.O., & Gosse, L. (2004). Acceptance by undergraduates of the Immediate Feedback Assessment Technique for multiple-choice testing. Teaching in Higher Education, 9, 17-28. Dihoff, R.E., Brosvic, G.M., & Epstein, M.L. (2003). The role of feedback during academic testing: the delay retention effect revisited. The Psychological Record, 53, 533-548. Epstein, M.L., Epstein, B.B., & Brosvic, G.M. (2001). Immediate feedback during academic testing. Psychological Reports, 88, 889-894. Epstein, M.L., Lazarus, A.D., Calvano, T.B., Matthews, K.A., Hendel, R.A., Epstein, B.B., & Brosvic, G.M. (2002). Immediate feedback assessment technique promotes learning and corrects inaccurate first responses. The Psychological Record, 52, 187-201. Haladyna, T.M. (1994). Developing and Validating Multiple-Choice Test Items. Hillsdale, New Jersey: Lawrence Erlbaum Associates. Kluger, A.N., & DeNisi, A. (1996). The effects of feedback intervention on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254-284. Kulik, J.A. & Kulik, C.C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58, 79-97. McCombs, B. L. (2003). Defining tools for teacher reflection: The Assessment of Learner-centered Practices (ALCP). Presented at the annual meeting of the American Educational Research Association, Chicago, IL. The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005 Immediate Feedback Assessment Technique 131 McMorris, R.F., Boothroyd, R.A., & Pietrangelo, D.J. (1997). Humor in educational testing: A review and discussion. Applied Measurement in Education, 10, 269297. Michaelsen, L. K., Knight A. B., & Fink, L. D. (2004). Team-Based Learning: A Transformative Use of Small Groups in College Teaching. Sterling, Virginia: Stylus. Sassenrath, J.M., Yonge, G.D., & Schrable, K. (1968). Immediate and delayed feedback on examinations and immediate and delayed retention. California Journal of Educational Research, 19, 226-231. The Canadian Journal of Higher Education Volume XXXV, No. 4, 2005