Graduate Student Teaching Development / J.S. Boman 100 CSSHE SCÉES Canadian Journal of Higher Education Revue canadienne d’enseignement supérieur Volume 43, No. 1, 2013, pages 100-114 Graduate Student Teaching Development: Evaluating the Effectiveness of Training in Relation to Graduate Student Characteristics Jennifer S. Boman Mount Royal University Abstract In recent years, much attention has been given to the need for more empirical research to evaluate training programs that help prepare graduate students for their current and future teaching responsibilities. The present study investigated the effectiveness of a training workshop for graduate students who had varying levels of experience and diverse cultural backgrounds. Results indicated that over the course of training participants significantly increased their self-efficacy and effective teaching behaviours and decreased their public speaking apprehension. Although participants with varying levels of experience as well as participants with Canadian and international backgrounds benefited from the program, the results highlighted the need for additional teaching development opportunities for international graduate students. Résumé Ces dernières années, on a beaucoup insisté sur la nécessité de pousser la recherche empirique en vue d’évaluer les programmes de formation qui préparent les futurs enseignants à assumer les charges d’enseignement actuelles et à venir. Cette étude a investigué l’efficacité d’un atelier de formation à l’intention de finissants ayant une expérience préalable variée et provenant CJHE / RCES Volume 43, No. 1, 2013 Graduate Student Teaching Development / J.S. Boman 101 d’horizons culturels divers. Les résultats indiquent que l’atelier a eu un effet significatif favorable sur le niveau d’auto-efficacité des participants, ainsi que sur leur efficacité pédagogique et qu’il a diminué leur crainte de parler en public. Bien que l’atelier se soit révélé profitable pour les participants, peu importe leur expérience préalable ou leur origine, les résultats indiquent qu’il faut créer de nouvelles occasions de formation pour les finissants d’origine étrangère. Historically the primary focus of graduate training has been the training of research scholars, but competency in teaching is now increasingly recognized as a necessary component of graduate education (Austin & Wulff, 2004). Not only do many graduate students require teaching skills to fulfill their roles as teaching assistants (TAs) in undergraduate courses, many new doctoral graduates obtain positions at institutions where teaching is a major component of their careers as faculty members (Gaff & Lambert, 1996; Golde & Dore, 2001). Recognition of the need for training opportunities for TAs increased in the 1990s and was accompanied by the emergence of centralized programs for TA teaching development and national conferences on TA issues (Chism, 1998). Indeed, a diverse range of training opportunities for graduate students now exists across institutions in North America (Meyers & Prieto, 2000; Mueller, Perlman, McCann, & McFadden, 1997; Piccinin, Farquharson, & Mihu, 1993). Although many institutions have responded to the call for greater TA training, there has been a delay between the implementation of these instructional interventions for TAs and rigorous research that evaluates the effectiveness of that training (Weimer & Lenze, 1997). Some recent evaluation work has been done in Canadian contexts (Taylor, Schönwetter, Ellis, & Roberts, 2008), but there is a pressing need for further research that demonstrates the extent to which programs are successful in helping prepare graduate students to teach in their disciplines. Preparing graduate students for their role as future faculty members remains an important issue in higher education (Austin & Wulff, 2004; Piccinin, 2006), and the evaluation and improvement of teaching development programs is a necessary step in achieving this goal. Research on TA Training Past evaluations of TA training programs have frequently been criticized for their limited scope and reliance on the satisfaction ratings of participants as the sole measure of their impact (Chism, 1998). To assess how to better prepare graduate students for their various teaching roles, empirical research is needed that both identifies the outcomes of specific types of training programs and investigates the effectiveness of programs for diverse groups of graduate students. Understanding the outcomes of graduate students’ early training experiences is particularly important because these experiences may help lay the foundation for their future expectations regarding the teaching role. One such type of training program for new graduate teaching assistants is an information and skills workshop in which participants are introduced to general information about teaching coupled with skills training activities such as microteaching (brief presentations through which participants can practise and get feedback on specific teaching behaviours). CJHE / RCES Volume 43, No. 1, 2013 Graduate Student Teaching Development / J.S. Boman 102 Research on graduate teaching assistants and medical residents learning to teach has identified several possible training outcomes for information and skills workshops such as improved confidence or self-efficacy (Crowe, Harris, & Ham, 2000; Salinas, Kozuh, & Seraphine, 1999), decreased anxiety (Williams, 1991), and changed teaching behaviours (D’Eon, 2004; Dunnington & DaRosa, 1998; for an early review, see Abbott, Wulff, & Szego, 1989). First, teacher self-efficacy, based on Bandura’s (1977, 1986) efficacy expectations, refers to the conviction that one can successfully execute the necessary teaching behaviour to produce a desired outcome (Prieto & Meyers, 1999). Performance accomplishments, namely successful experiences in executing behaviours, are one way to increase self-efficacy (Bandura, 1986). With respect to TA training, practising the actual skills of teaching may help increase TAs’ sense of self-efficacy about their teaching roles. A second possible outcome of training, the reduction of anxiety, may be affected by feedback and consultation about one’s teaching performance (Williams, 1991). Indeed, effective interventions for communication apprehension, which can include anxiety about communicating in a public speaking context, have included skills training, video feedback, and consultation (Pribyl, Keaten, & Sakamoto, 2001). Finally, with respect to teaching behaviours, instructional interventions have shown improvements in specific teaching skills, such as using instructional objectives (D’Eon, 2004). Aspects of training workshops that could be responsible for these changes include the provision of information, direct practice, and feedback, as well as modelling of skills by workshop facilitators. Although the initial results of such training evaluations show promise in identifying common results of training, further empirical research is needed to determine the outcomes of specific programs as well as to investigate whether or not specific program and participant characteristics affect these outcomes. Graduate Student Characteristics and Training The issue of whether specific training programs are equally successful for diverse groups of participants has been relatively unexplored in evaluations of graduate student teaching programs. Although demographic variables, such as department or educational level, were frequently used to characterize a sample of graduate students before training, little research has examined how these differences affect training outcomes. Two such characteristics of graduate student teachers that warrant particular consideration for orientation programs are teaching experience and international student status. Both of these characteristics may influence how graduate students respond to the pace and level of skill instruction inherent in this type of training. For example, graduate students with teaching experience may find basic skills information redundant, whereas international students may need additional time to process cultural differences in the Canadian teaching context. Information regarding whether graduate students with diverse backgrounds and experiences respond differently to basic skills and information workshops could help determine the effectiveness of providing general skills training for graduate students as a whole versus targeting training modules for different populations. With respect to teaching experience, research indicates that more experienced TAs may differ somewhat from their novice counterparts in their expectations toward teaching. For example, research on TA training suggested a positive correlation between amount of experience and teacher self-efficacy (Prieto & Altmaier, 1994; Tollerud, 1990). TA orien- CJHE / RCES Volume 43, No. 1, 2013 Graduate Student Teaching Development / J.S. Boman 103 tation workshops are often intended to introduce new TAs to general teaching strategies and skills that are useful for the TA role (Park, 2004), but some graduate students, while new to the TA role, have taught in other instructional contexts such as primary school teaching, tutoring, or even college or university teaching (Austin, 2002). Research has indicated that these more experienced TAs may differ somewhat from their novice counterparts in areas such as self-efficacy; however, it is less clear how these differences have affected the way TAs respond to various training experiences. For example, it would be useful to know the extent to which TAs with teaching experience benefit in meaningful ways from attending TA orientation workshops with colleagues who have no practical experience in teaching. A second issue that needs to be considered in TA orientation and skill workshops is the extent to which Canadian TAs (CTAs) and International TAs (ITAs, or graduate students from places other than North America) benefit from common or separate training activities. Although members of both groups of TAs may share the common experience of being new to TA work and university teaching, ITAs may face the additional challenges of adapting to the cultural norms of the Canadian classroom and potentially teaching in a foreign language (Bauer & Tanner, 1994; Weimer, Svinicki, & Bauer, 1989). Indeed, research on teaching behaviours has suggested that ITAs may receive lower student ratings of teaching than North American TAs on dimensions such as performance, clarity of speech, rapport, and approachability (Twale, Shannon, & Moore, 1997). It is difficult to know to what extent these ratings reflected differences in teaching effectiveness and to what extent they were influenced by undergraduate students’ reactions to cultural unfamiliarity; however, these findings highlighted the additional challenges faced by many ITAs. Although many ITAs were recognized as needing specialized instruction to address pronunciation, crosscultural communication skills, and a new teaching culture (Constantinides, 1989), both ITAs and CTAs shared a common need for training that targeted general teaching skills. Outcome studies of TA training could clarify the extent to which ITAs benefit from skills training that is targeted toward a general population of new TAs. In summary, given the growing need for providing evidence-based training opportunities in teaching for graduate students, further empirical research on training programs is needed. In particular, there is a need to identify the outcomes of training that are relevant to future teaching performance (e.g., self-efficacy, effective teaching behaviours) and to provide evidence that training programs lead to improvements in these outcomes. Furthermore, it is important to determine how TAs with different experiences (e.g., prior teaching practice and differences in cultural background) respond to training. Answers to these questions can inform future training endeavours so that programs can be improved to meet the needs of different graduate student populations. Investigation of Program Efficacy The present research used a pretest/posttest design to evaluate changes in TAs after a two-and-a-half-day information and skills program. The goals of the study were twofold: to assess whether or not TAs changed their self-perceptions (e.g., self-efficacy) and teaching behaviours over the course of training, and to investigate the extent to which pre-existing differences among TAs (i.e., experience versus no experience, and Canadian versus international student status) influenced the extent to which TAs benefited from training. CJHE / RCES Volume 43, No. 1, 2013 Graduate Student Teaching Development / J.S. Boman 104 Based on a pilot study in which program participants responded to open-ended questions about the usefulness of the training program, it was hypothesized that over the course of training TAs would show significant increases in self-efficacy, improved attitudes toward teaching, and increased use of effective teaching behaviours, and a significant decrease in anxiety about public speaking. Based on previous research (Prieto & Altmaier, 1994; Twale et al., 1997), it was further hypothesized that TAs would show initial differences in these variables before training. Specifically, it was predicted that ITAs would show initial lower observer ratings of effectiveness than Canadian TAs, and that more experienced TAs would show higher initial self-efficacy and observer ratings of effectiveness than TAs with no experience. Because the question of how background differences (i.e., experience, international status) affect how much TAs gain from an orientation workshop was exploratory in nature, no specific hypotheses were generated for whether demographic characteristics would be associated with differential gains in outcomes. Method Participants A total of 125 TAs from the University of Western Ontario participated in this research. This number reflects an 85% participation rate of all TAs who attended an offering of the program over a two-month period. Analyses are reported for the 108 participants who completed both pretest and posttest questionnaires. Video recordings were also available for 86 of these participants. Reasons for missing data included equipment problems (e.g., failure to record, sound difficulties) and absence of a participant for one or more workshop activities. Participant characteristics. The sample included 49 males and 59 females with a mean age of 26.28 (SD = 4.84) years. The majority of participants were enrolled in either the first year of a masters or doctoral program (65.7% and 22.2%, respectively). In total, 56.5% of participants reported no teaching experience, whereas 43.5% reported experience as a teaching assistant, undergraduate instructor, or school teacher. With respect to international student status, 45.4% of participants were International TAs and 54.6% were Canadian TAs. Participants reported affiliation with a wide range of academic disciplines. Description of the Training Program The program evaluated in this study, the Teaching Assistant Training Program (TATP), is a two-and-a-half-day workshop intended for graduate students who are relatively new to teaching. This program is one of several professional development opportunities offered by the centralized educational development unit on campus. Participation in the program was voluntary, but TAs received a certificate of training upon completion. Training consisted primarily of active learning sessions regarding effective teaching practices (constituting approximately eight hours of training time) and two microteaching sessions. The topics of the learning sessions included effective lecturing, giving and receiving feedback, instructional aids, marking practices, student diversity, ethical issues in teaching, leading labs, tutorials, and office hours, and a choice of leading discussions or vocal health for teachers. The sessions included information about effective teaching practices and incorporated opportunities for participants to complete practical exercises or to generate CJHE / RCES Volume 43, No. 1, 2013 Graduate Student Teaching Development / J.S. Boman 105 and discuss ideas about how these principles applied to actual teaching scenarios. For the microteaching component of training, TAs prepared two 10-minute teaching segments and presented these to a small group of four or five fellow participants and a trained facilitator. They received immediate oral and written feedback on their teaching from their group as well as a video recording of their session. A third component of the program was the opportunity for informal learning that existed throughout the training days. The facilitators created a sense of welcome and community on the first day with ice breakers, snacks, and a discussion of facilitators’ and participants’ expectations of the program. TAs then had continued opportunities to converse with fellow participants and the facilitators over lunches and breaks throughout training. As a final component of training, each participant received a handbook with written information from the learning sessions as well as lists of resources. Measures In addition to the measures below, all participants were asked to provide basic demographic information (i.e., age, gender, student status, department and program information, country of origin, and first language) and information about their past training and teaching experiences. TA Self-Efficacy Scale (TSE). The TSE is a 34-item self-report scale that assessed the degree to which TAs felt confident in executing specific teaching behaviours (e.g., “motivate student interest in a lecture”). The scale was adapted from Tollerud’s (1990) Self-Efficacy Towards Teaching Inventory (SET-I) for use in the current research and includes 21 adapted items from Tollerud’s 43-item scale, 5 items derived from Streveler’s (1993) confidence scale for TAs, and 8 new items. Participants responded to items using a 5-point Likert scale ranging from 1 (not confident) to 5 (completely confident). The internal consistency of the TSE (α = .91) was excellent and similar to that of Tollerud’s original scale. For all following analyses, one item (confidence in giving a lab demonstration) was deleted because of a poor response rate. Attitudes toward teaching. A 13-item self-report measure that assessed participants’ global attitudes and motivation toward classroom teaching was developed for use in this study. Three of these items were based on Gray and Buerkel-Rothfuss’s (1993) survey of TA attitudes toward various facets of teaching. Participants rated each of 13 items (e.g., “If I had the chance to teach a class as the sole instructor, I would look forward to the opportunity”) on a 5-point scale from 1 (strongly disagree) to 5 (strongly agree). Cronbach’s alpha for this measure was .82, indicating good internal consistency. Personal Report of Communication Apprehension (PRCA-24). Public speaking apprehension was assessed using the 6-item public speaking subscale of the PRCA-24 (McCroskey, 1982). Participants were asked to report their feelings about communicating with others using a 5-point Likert-type scale (e.g., “While giving a speech, I get so nervous I forget facts I really know”). Subscale scores range from 6 to 30, with higher scores indicating greater communication apprehension. Cronbach’s alpha for the subscale was .86, which is similar to past research on this instrument (Levine & McCroskey, 1990). Teacher Behaviors Inventory (TBI-A). An abbreviated 15-item version of the Teacher Behaviors Inventory (Murray, 1983) was used to assess the frequency and effectiveness of TAs’ teaching behaviours across five main categories: clarity, enthusiasm, CJHE / RCES Volume 43, No. 1, 2013 Graduate Student Teaching Development / J.S. Boman 106 student interaction, organization, and vocal quality. Respondents were asked to rate the frequency of 14 specific, low inference teaching behaviours (e.g., “defines new or unfamiliar terms,” “speaks in a dramatic or expressive way”) on a 5-point scale (1 = almost never, 2 = rarely, 3 = sometimes, 4 = often, 5 = almost always) and to complete a single-item scale that rated overall teaching effectiveness. The single-item scale allowed responses in half-point increments from one to five. Two versions of the instrument’s instructions were adapted for use with observer ratings of teaching and with graduate students’ selfratings of teaching. Three trained observers who were blind to whether each video was from the first or second microteaching session coded the microteaching data. Observer 1 coded the complete data set, whereas Observers 2 and 3 coded a random sample of 24 and 22 microteaching sessions, respectively. Using percent agreement across the 14 items on the TBI-A, Observers 1 and 2 agreed within 1 point in their ratings 80.47% of the time. Observers 1 and 3 agreed within 1 point in their ratings 80.68% of the time. For the single item that assessed overall instructional effectiveness, Observers 1 and 2 agreed within 1 point in their ratings 87.50% of the time. Observers 1 and 3 agreed within 1 point of their ratings 95.45% of the time. These ratings suggested good inter-rater reliability between Observer 1 and the two additional raters. Consequently, the ratings from Observer 1 were used in the following analyses. Procedure The questionnaires, including the TSE, Attitudes Towards Teaching survey and the PRCA-24 were administered twice during training: once during the opening session of the workshop (Time 1) and once during the closing session at the end of the third day of training (Time 2). Video recordings of participants’ microteaching and self-reports of teaching behaviours using the TBI-A were collected at two points during training. The first microteaching session occurred in the morning of the second day of training (Time 1), and the second microteaching session occurred on the third and final day of training (Time 2). Observers later coded the video recordings of the microteaching sessions. Descriptive Data Analyses Results Means and standard deviations for pretest and posttest scores for TA participants as a whole as well as for International and Canadian TAs, and TAs with and without experience, are presented in Table 1. With respect to experience, at the start of the program TAs with experience showed higher self-efficacy than TAs with no experience, t(105) = 2.62, p = .010, d = 0.51. With respect to student status, Canadian TAs had significantly higher self-ratings of overall effectiveness, t(93) = 4.28, p < .001, d = 0.88, and observer ratings of overall effectiveness, t(83) = 6.15, p < .001, d = 1.33, than International TAs. No other pretest differences were significant at the .05 level. CJHE / RCES Volume 43, No. 1, 2013 Graduate Student Teaching Development / J.S. Boman 107 Table 1 Means, Standard Deviations, and n Sizes for Each Dependent Measure by TA Sample as a Whole and by TA Status and Experience Measures Self Efficacy ITA CTA No Experience Prior Experience Attitudes ITA CTA No Experience Prior Experience Public Speaking Apprehension ITA CTA No Experience Prior Experience Self-Rated Frequency of Behaviours ITA CTA No Experience Prior Experience Self-Rated Effectiveness ITA CTA No Experience Prior Experience Observer-Rated Frequency ITA CTA No Experience Prior Experience Observer-Rated Effectiveness ITA CTA No Experience Prior Experience Mean 112.52 111.51 113.39 109.00 117.03 51.74 51.06 52.32 51.36 52.21 18.27 17.77 18.68 18.53 17.93 55.36 54.20 56.17 55.18 55.61 3.67 3.41 3.85 3.68 3.65 55.80 49.41 60.73 56.67 54.68 3.62 3.18 3.97 3.70 3.53 Time 1 SD 16.18 17.08 15.49 14.42 17.32 6.35 6.07 6.58 6.04 6.76 4.25 4.49 4.04 4.01 4.56 5.69 6.20 5.20 5.87 5.51 0.53 0.55 0.45 0.55 0.52 9.87 9.98 6.40 9.86 9.91 0.71 0.70 0.49 0.74 0.66 n 107 49 58 60 47 106 49 57 59 47 104 46 58 58 46 100 41 59 55 45 95 39 56 53 42 85 37 48 48 37 85 37 48 48 37 Mean 131.02 127.41 134.07 130.22 132.03 52.39 50.86 53.70 52.84 51.82 16.75 15.99 17.34 16.72 16.78 59.10 56.80 60.69 59.75 58.31 3.93 3.79 4.02 3.94 3.90 57.68 52.27 61.85 58.25 56.95 3.84 3.46 4.14 3.90 3.77 Time 2 SD 13.96 13.05 14.08 13.80 14.25 6.78 6.49 6.80 6.94 6.60 4.53 4.04 4.83 4.54 4.56 6.34 6.10 6.06 6.22 6.46 0.44 0.47 0.40 0.46 0.43 8.82 8.64 6.40 9.38 8.10 0.65 0.65 0.48 0.65 0.65 n 107 49 58 60 47 106 49 57 59 47 104 46 58 58 46 100 41 59 55 45 95 39 56 53 42 85 37 48 48 37 85 37 48 48 37 Note. Different n sizes reflect the different number of data points available as some participants did not complete all of the self-report measures. CJHE / RCES Volume 43, No. 1, 2013 Graduate Student Teaching Development / J.S. Boman 108 Program Impact To assess the overall effectiveness of the training program, a 3-factor mixed-model analysis of variance (ANOVA) was conducted on each of the dependent variables (TA selfefficacy, attitudes toward teaching, public speaking apprehension, observer-rated and self-rated frequency of teaching behaviours, and observer-rated and self-rated teaching effectiveness). Between-subject factors were experience (no experience versus prior experience) and student status (ITA versus CTA); the within-subject factor was time (Time 1 versus Time 2). Only main effects and interactions that were significant at the p = .05 level are reported. Bonferroni post hoc tests were conducted on significant interactions to further explore the nature of the interaction. TA self-efficacy. The mixed-model ANOVA conducted on TA self-efficacy produced a significant main effect for time, F(1, 103) = 180.39, p < .001, η2 = .64, with participants showing greater self-efficacy at Time 2. The main effect was subsumed by a significant interaction between time and experience, F(1, 103) = 4.47, p = .037, η2 = .04. Post hoc tests showed that the nature of this interaction was that participants with no experience showed a significant increase in self-efficacy, t = 12.28, p < .001, as did participants with experience, t = 7.73, p < .001, but the gain was greatest for those with no experience. Attitudes toward teaching. The mixed-model ANOVA conducted on attitude scores did not show a significant main effect of time, F(1, 102) = 1.55, ns. However, there was a significant interaction between time and experience, F(1, 102) = 4.44, p = .038, η2 = .04, where individuals with no experience showed a significant increase in positive attitudes from Time 1 to Time 2, t = 2.83, p < .006, and individuals with experience showed no significant change. Public speaking apprehension. The mixed-model ANOVA on the public speaking apprehension scores showed a significant effect for time, F(1, 100) = 35.66, p < .001, η2 = .26, with participants showing lower anxiety at Time 2. (For this measure, higher scores were indicative of greater anxiety.) Teaching behaviours and effectiveness. With respect to frequency of effective teaching behaviours, the mixed-model ANOVA conducted on observer-ratings showed a significant main effect for time, F(1, 81) = 4.21, p = .043, η2 = .05, with higher ratings at Time 2 than Time 1. A mixed-model ANOVA conducted on self-ratings of teaching behaviours showed a similar significant main effect for time, F(1, 96) = 46.56, p < .001, η2 = .33, with higher ratings at Time 2 than Time 1. Finally, the mixed-model ANOVA conducted on overall effectiveness scores showed a significant main effect for time for observer-ratings, F(1, 81) = 9.51, p = .003, η2 = .11, with higher ratings at Time 2 than Time 1. For self-ratings of overall effectiveness, a mixedmodel ANOVA showed a significant main effect for time, F(1, 91) = 31.66, p < .001, η2 = .26, with higher ratings at Time 2 than Time 1. The main effect was subsumed by an interaction between time and TA status, F(1, 91) = 5.02, p = .028, η2 = .05. Post hoc tests indicated that International TAs showed a significant improvement in self-rating of overall teaching effectiveness from Time 1 to Time 2, t = 5.04, p < .001. Canadian TAs also showed significant increases in ratings of overall effectiveness from Time 1 to Time 2, t = 2.61, p = .011, but the gain in self-rated teacher effectiveness from Time 1 to Time 2 was larger in absolute terms for International TAs than Canadian TAs. CJHE / RCES Volume 43, No. 1, 2013 Graduate Student Teaching Development / J.S. Boman 109 Discussion Main Program Effects The first purpose of the current research was to evaluate whether a TA training workshop significantly affected TA self-perceptions and behaviour. Results for TAs as a whole indicated significant increases in self-efficacy, self-rated and observer-rated teaching behaviours and effectiveness, and a significant decrease in communication apprehension over the course of training. These outcomes make sense in the context of Nyquist and Sprague’s (1998) framework of TA development (see also Sprague & Nyquist, 1991). According to this model, TAs experience changes in their teaching concerns over time from self-oriented concerns to a concern for student learning. In the early stages of development, TAs primarily orient toward surviving the teaching experience and acquiring teaching skills. The participants in this study largely fit with these initial concerns relating to skill improvement and building confidence in teaching. This focus makes sense given that the training program is designed for TAs who are relatively new to teaching. Although participants may have increased their awareness of the relationship between effective teaching behaviours and student learning over the course of training, it seems likely that further experience and feelings of mastery are needed for TAs to shift their thinking from their own performance to student outcomes. If a primary purpose of the TATP is viewed as skill acquisition, the results also fit well within a social cognitive framework (Bandura, 1986) whereby factors such as self-efficacy are influential in determining whether knowledge will be translated into behaviour. As such, if skill improvement is a desired outcome of early TA training, educational developers may need to ensure that they address the affective dimensions of performance, such as confidence and anxiety, in their programs. Implications for TA Characteristics Experience. The second purpose of the current research was to investigate whether TA characteristics influenced training outcomes in order to provide information regarding the usefulness of training for specific groups of TAs. First, with respect to experience level, the only significant difference noted between novice and more experienced TAs at the start of training was that more experienced TAs showed greater levels of initial selfefficacy. This finding was not surprising as past research has identified a positive relationship between experience and self-efficacy (Prieto & Altmaier, 1994; Prieto & Meyers, 1999). Interestingly, both groups of TAs increased their self-efficacy over the course of training even though gains in self-efficacy were greater for novice TAs. This finding suggests that even more experienced TAs benefited from the program in terms of enhanced confidence. The only other difference observed between experience levels was that novice TAs substantially increased their positive attitudes toward teaching, whereas experienced TAs did not. It should be noted that all attitudes were very positive to begin with, so the range of attitudes was very narrow. This finding, however, reinforces the importance of making the initial teaching experiences of novice TAs positive ones because attitudes toward teaching may be linked to how much time and effort TAs put into teaching later on. Interestingly, there were no differences in effectiveness between TAs with experience or no experience at the start of the study. One interpretation of this finding is that experi- CJHE / RCES Volume 43, No. 1, 2013 Graduate Student Teaching Development / J.S. Boman 110 ence alone was not a sufficient means for this group of more advanced TAs to improve their teaching. Alternatively, one could argue that the experience level of many TAs in this group, though higher than TAs with no experience, was still not extensive. Interestingly, level of experience did not differentially affect whether or not TAs improved their teaching skills over the course of the program. This finding suggests that TAs with varying levels of experience were able to benefit from basic skills training and feedback. Perhaps more experienced TAs benefited from training as much as their novice counterparts because they still received new information about their own teaching. Indeed, much research suggests that feedback and consultation regarding teaching is important to teaching improvement (Bray & Howard, 1980; Cohen, 1980). International TA status. With respect to International TA status, a comparison of pretest scores suggested that ITAs entered training with significant disadvantages in self-rated and observer-rated effectiveness when compared to CTAs. Despite these initial differences in pretest scores, both CTAs and ITAs showed significant improvements in effectiveness. For self-rated effectiveness, ITAs showed even greater gains than CTAs; however, given the initial differences in scores, they also had more ground to gain. Regardless, these results indicate that both CTAs and ITAs benefited from a general orientation skills program. Nonetheless, although this conclusion is encouraging, the training was insufficient to bridge the initial gap between CTAs and ITAs. In other words, even though ITAs showed improvement over the course of the program, their self-rated and observer-rated effectiveness scores were still lower than those of CTAs at the end of training. The results have several important implications for ITA training. First, given that the TATP was used successfully with ITAs, workshops that target general teaching strategies may be especially appropriate for the training of ITAs and CTAs together. In fact, ITAs may have benefited from seeing examples of CTAs teaching in the Canadian teaching context, as well as from practice in using their second language to communicate with a primarily English-language audience. In addition, the finding that both CTAs and ITAs show similar improvements for variables such as self-efficacy and public speaking apprehension suggests that both groups of TAs share several common affective concerns in learning to teach. Nonetheless, given the persistence of initial differences between ITAs and CTAs in the areas of self-rated and observer-rated effectiveness, more intensive training is needed for ITAs. It seems likely that linguistic difficulties and an unfamiliarity with the Canadian teaching culture contributed to initial differences among TAs. Consequently, ITAs may benefit from additional training programs that target these issues. Indeed, a dual approach to training in which ITAs experience general orientation training programs along with separate courses on cross-cultural communication is not new (Constantinides, 1989). Perhaps given the diverse types of programming available for ITAs, the next step is research on how best to order or structure these training opportunities to maximize ITA outcomes. Limitations and Future Research A key limitation of the present research design was the absence of a control group of TAs who did not participate in training. Without a control group, it is unclear to what extent TAs who did not participate in training showed similar improvements in outcomes. Second, outcomes of training were measured immediately at posttest time points , so it is CJHE / RCES Volume 43, No. 1, 2013 Graduate Student Teaching Development / J.S. Boman 111 unclear to what extent gains persisted over time or transferred to actual classroom teaching. Indeed, some research on interventions has suggested that continued practice and feedback are an important component of sustained behaviour change (Weimer & Lenze, 1997). Longitudinal research is needed to clarify how long training gains persist and whether there are differences in classroom teaching between TAs who have and have not received training. An additional limitation of the current research is that ITAs were referred to as a homogenous group. Although small sample sizes prohibited analyzing data according to specific cultural backgrounds, there was much individual variation among the graduate students. Future research needs to differentiate between subgroups of ITAs to examine how specific cultural teaching experiences interact with learning to teach in the Canadian context. Conclusion In light of the need for empirical investigations of TA training initiatives, the present study provides evidence for the efficacy of an information and skills program in increasing TAs’ self-efficacy and effective teaching behaviours, and decreasing communication apprehension. Although the training program investigated here was effective for TAs of varying experience levels from a variety of cultural and linguistic backgrounds, the results emphasize a need for additional training endeavours for ITAs to further enhance their skill development in a Canadian teaching context. Finally, skills training programs are only an initial step in preparing TAs for the complex roles and responsibilities inherent in teaching. Given that TAs have the potential to significantly influence student learning, future research is essential to help understand the contribution of ongoing educational programming to the development of graduate students as teachers. References Abbott, R. D., Wulff, D. H., & Szego, C. K. (1989). Review of research on TA training. In J. D. Nyquist, R. D. Abbott, & D. H. Wulff (Eds.), Teaching assistant training in the 1990s. New Directions for Teaching and Learning, No. 39, 111-124. San Francisco, CA: Jossey-Bass. Austin, A. E. (2002). Preparing the next generation of faculty: Graduate school socialization to the academic career. The Journal of Higher Education, 73, 94–122. Austin, A. E., & Wulff, D. H. (2004). The challenge to prepare the next generation of faculty. In D. H. Wulff & A. E. Austin (Eds.), Paths to the professoriate: Strategies for enriching the preparation of future faculty (pp. 3–16). San Francisco, CA: Jossey-Bass Publishers. Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioural change. Psychological Review, 84, 191–215. Bandura, A. (1986). Social foundations of thought and action. New York, NY: Prentice-Hall. Bauer, G., & Tanner, M. (1994). Current approaches to international TA preparation in higher education: A collection of program descriptions. Seattle, WA: University of Washington, Center for Instructional Development and Research. (ERIC Document Reproduction Service No. ED371654) CJHE / RCES Volume 43, No. 1, 2013 Graduate Student Teaching Development / J.S. Boman 112 Bray, J. H., & Howard, G. S. (1980). Methodological considerations in the evaluation of a teacher-training program. Journal of Educational Psychology, 72, 62–70. Chism, N. V. N. (1998). Evaluating TA programs. In M. Marincovich, J. Prostok, & F. Stout (Eds.), The professional development of graduate teaching assistants (pp. 249– 262). Bolton, MA: Anker. Cohen, P. A. (1980). Effectiveness of student-rating feedback for improving college instruction: A meta-analysis of findings. Research in Higher Education, 13, 321–341. Constantinides, J. C. (1989). ITA training programs. In J. D. Nyquist, R. D. Abbott, & D. H. Wulff (Eds.), Teaching assistant training in the 1990s. New Directions for Teaching and Learning, No. 39, 71-77. San Francisco, CA: Jossey-Bass. Crowe, P., Harris, P., & Ham, J. (2000). Teaching on the run: Teaching skills for surgical trainees. The Australian and New Zealand Journal of Surgery, 70, 227–230. D’Eon, M. F. (2004). Evaluation of a teaching workshop for residents at the University of Saskatchewan: A pilot study [Electronic version]. Academic Medicine, 79, 791–797. Denham, C., & Michael, J. (1981). Teacher sense of efficacy: A definition of the construct and a model for further research. Educational Research Quarterly, 6, 39–61. Dunnington, G. L., & DaRosa, D. (1998). A prospective randomized trial of a residentsas-teachers training program [Electronic version]. Academic Medicine, 73, 696–700. Gaff, J. G., & Lambert, L. M. (1996). Socializing future faculty to the values of undergraduate education [Electronic version]. Change, 28, 38–42. Golde, C. M., & Dore, T. M. (2001). At cross purposes: What the experiences of doctoral students reveal about doctoral education (www.phd-survey.org). Philadelphia, PA: Pew Charitable Trusts. Gray, P. L., & Buerkel-Rothfuss, N. L. (1993). Adapting to attitude differences: A factor analytic profile of graduate teaching assistants. In K. G. Lewis (Ed.), The TA experience: Preparing for multiple roles (pp. 296–312) Selected readings from the Third National Conference on the Training and Employment of Graduate Teaching Assistants at Austin, Texas. Stillwater, OK: New Forums Press. Levine, T. R., & McCroskey, J. C. (1990). Measuring trait communication apprehension: A test of rival measurement models of the PRCA-24. Communication Monographs, 57, 62–72. McCroskey, J. C. (1982). An introduction to rhetorical communication. (4th ed.). Englewood Cliffs, NJ: Prentice-Hall. Meyers, S. A., & Prieto, L. R. (2000). Training in the teaching of psychology: What is done and examining the differences [Electronic version]. Teaching of Psychology, 27, 258–261. Mueller, A., Perlman, B., McCann, L. I., & McFadden, S. H. (1997). A faculty perspective on teaching assistant training [Electronic version]. Teaching of Psychology, 24, 167–171. Murray, H. G. (1983). Low-inference classroom teaching behaviors and student ratings of college teaching effectiveness. Journal of Educational Psychology, 75, 138–149. CJHE / RCES Volume 43, No. 1, 2013 Graduate Student Teaching Development / J.S. Boman 113 Nyquist, J. D., & Sprague, J. (1998). Thinking developmentally about TAs. In M. Marincovich, J. Prostok, & F. Stout (Eds.), The professional development of graduate teaching assistants, 61-88. Bolton, MA: Anker. Park, C. (2004). The graduate teaching assistant (GTA): Lessons from North American experience [Electronic version]. Teaching in Higher Education, 9, 349–361. Piccinin, S. J. (2006, Winter). Hot topics in teaching and learning in higher education. Teaching and Learning in Higher Education, 42, 3–4. Piccinin, S. J., Farquharson, A., & Mihu, E. (1993). Teaching assistants in Canadian universities: An unknown resource. Canadian Journal of Higher Education, 23, 104–117. Pribyl, C. B., Keaten, J., & Sakamoto, M. (2001). The effectiveness of a skills-based program in reducing public speaking anxiety. Japanese Psychological Research, 43(3), 148–155. Prieto, L. R., & Altmaier, E. M. (1994). The relationship of prior training and previous teaching experience to self-efficacy among graduate teaching assistants [Electronic version]. Research in Higher Education, 35, 481­–497. Prieto, L. R., & Meyers, S. A. (1999). Effects of training and supervision on the selfefficacy of psychology graduate teaching assistants [Electronic version]. Teaching of Psychology, 26, 264–266. Salinas, M. F., Kozuh, G., & Seraphine, A. E. (1999). I think I can: Improving teaching self-confidence of international teaching assistants. Journal of Graduate Teaching Assistant Development, 6, 149–156. Sprague, J., & Nyquist, J. D. (1991). A developmental perspective on the TA role. In J. D. Nyquist, R. D. Abbott, D. H. Wulff, & J. Sprague (Eds.), Preparing the professoriate of tomorrow to teach: Selected readings in TA training, 295-312. Dubuque, IA: Kendall. Streveler, R. A. (1993). Investigating differences in TA confidence. In K. G. Lewis (Ed.), The TA experience: Preparing for multiple roles (pp. 37–40). Selected readings from the Third National Conference on the Training and Employment of Graduate Teaching Assistants at Austin, Texas. Stillwater, OK: New Forums Press. Taylor, K. L., Schönwetter, D. J., Ellis, D. E., & Roberts, M. (2008). An approach to evaluating the impact of two Certification in University Teaching programs for graduate students. The Journal of Graduate & Professional Student Development, 11, (1), 78–108. Tollerud, T. (1990). The perceived self-efficacy of teaching skills of advanced doctoral students and graduates from counsellor education programs. Dissertation Abstracts International 51, 12A. Twale, D. J., Shannon, D. M., & Moore, M. S. (1997). NGTA and IGTA training and experience: Comparisons between self-ratings and undergraduate student evaluations. Innovative Higher Education, 22, 61–77. Weimer, M., & Lenze, L. F. (1997). Instructional interventions: A review of the literature on efforts to improve instruction. In R. P. Perry & J. C. Smart (Eds.), Effective teaching in higher education: Research and practice (pp. 205–240). New York, NY: Agathon Press. CJHE / RCES Volume 43, No. 1, 2013 Graduate Student Teaching Development / J.S. Boman 114 Weimer, M., Svinicki, M. D., & Bauer, G. (1989). Designing programs to prepare TAs to teach. In J. D. Nyquist, R. D. Abbott, & D. H. Wulff (Eds.), Teaching assistant training in the 1990s. New Directions for Teaching and Learning, No. 39, 57-70. San Francisco, CA: Jossey-Bass. Williams, L. S. (1991). The effects of a comprehensive teaching assistant training program on teaching anxiety and effectiveness. Research in Higher Education, 32, 585–598. Acknowledgements The research presented in this article is based on a portion of the author’s doctoral dissertation. Thanks to the Teaching Support Centre at the University of Western Ontario for assistance with data collection and to Dr. Harry Murray for valuable feedback. The author would also like to acknowledge Dr. Andrew Szeto for helpful comments on an earlier draft of this manuscript. Contact Information Jennifer S. Boman Academic Development Centre Mount Royal University 4825 Mount Royal Gate SW Calgary, AB T3E 6K6 Canada [email protected] Jennifer Boman is an Assistant Professor and Faculty Development Consultant in the Academic Development Centre at Mount Royal University. This research study was conducted when she was a doctoral student in the Department of Psychology at the University of Western Ontario. Her research interests include faculty development, the design and evaluation of programs to improve teaching effectiveness, and the impact of teaching behaviours and teaching methods on student motivation and learning. CJHE / RCES Volume 43, No. 1, 2013