GradProSkills/ V. Venkatesh , J. Rabah, L. Lamoureux-Scholes, I. Pelczer, K. Urbaniak, & F. Martin 39 CSSHE SCÉES Canadian Journal of Higher Education Revue canadienne d’enseignement supérieur Volume 44, No. 3, 2014, pages 39 - 53 Development, Implementation, and Evaluation of a Professional Skills Development Program: The Case of Concordia University’s GradProSkills Vivek Venkatesh1, Jihan Rabah1, Laurie Lamoureux-Scholes, Ildiko Pelczer, Kathryn Urbaniak, & Frédérica Martin Concordia University Abstract The Graduate and Professional Skills (GradProSkills) program is an initiative developed and run by Concordia University’s School of Graduate Studies and the university’s Office of the Vice-President, Research and Graduate Studies. This paper presents a case study in which we describe the development, implementation, and evaluative components of the GradProSkills program. Drawing on data from a rigorously validated workshop evaluation tool and from attendance figures spanning a period of a little over two and a half years, we provide a picture of how Concordia’s graduate students are benefiting from the GradProSkills program. We present the results of both descriptive and inferential statistical analyses run on a pilot group of registrants who provided evaluation data (n = 3,292). Our case study contributes an empirically derived model of extracurricular programming, with contextualizing details of administrative structures, curriculum-development initiatives, and partnership efforts that have been used in operating the GradProSkills program. Résumé Le programme « Graduate and Professional Skills » (GradProSkills) est une initiative créée et dirigée par l’École des études supérieures et le Vice-rectorat à la recherche et aux études supérieures de l’Université Concordia. Cet article présente une étude de cas dans laquelle nous décrivons la création, la mise en œuvre et l’évaluation de GradProSkills. En nous fondant sur les données obtenues d’un outil d’évaluation d’ateliers rigoureux et validé, et sur des taux de participation s’échelonnant sur une période d’un peu plus de deux ans CJHE / RCES Volume 44, No. 3, 2014 GradProSkills/ V. Venkatesh , J. Rabah, L. Lamoureux-Scholes, I. Pelczer, K. Urbaniak, & F. Martin 40 et demi, nous traçons le portrait de la façon dont les étudiants aux cycles supérieurs de l’Université Concordia profitent du programme GradProSkills. Nous présentons des résultats d’analyses statistiques descriptives et déductives tirés d’un groupe pilote de personnes inscrites à des ateliers qui ont fourni des données d’évaluation (n = 3 292). Notre étude de cas fournit un modèle de programmation hors-programme dérivé empiriquement avec des détails contextualisés concernant les structures administratives, les initiatives d’élaboration de programmes et les partenariats dont le programme GradProSkills s’est servi. Context: Graduate Professional Development Programs at Canadian Universities With a competitive market and long-documented discrepancies between job offers and the skills of newly minted master’s and doctoral degree holders (Charbonneau, 2011; Newhouse, 1999; Tamburri, 2010), graduate students must develop professional skills, in addition to their disciplinary knowledge, to be fully equipped for transitioning to employment markets within the academy and/or the corporate environment (Chillas, 2010; Ducheny, Alletzhauser, Crandell, & Schneider, 1997; Galt, 2012; Poock, 2001). In fact, the Canadian Association of Graduate Studies (CAGS) published in 2008 a report titled Professional Skills Development for Graduate Students, which emphasized that the obligations of higher education institutions should include “providing graduate students with the best possible preparation for their future roles whether within academic or in other sectors” (p. 4). A recent comprehensive survey of professional development programs (PDPs) available across Canadian universities, by Marilyn Rose (2012), prepared for CAGS in association with the federally funded Social Sciences and Humanities Research Council of Canada (SSHRC), analyzed PDPs offered across Canadian universities and recommended best practices for their development in our nation. Results of Rose’s (2012) extensive review show that PDPs operate on a graduated scale of categories, with some universities having highly structured organization and high designated budget and personnel (category one), while on the other side of the scale are small universities that are witnessing some PDP activities but with no centralization and operating from small released operational budgets (category four). Elsewhere, Holaday, Weaver, and Nilson (2007) published an analysis of the elements that create a successful PDP; they assert that finding collaborative partners is one of the most important elements of securing a successful PDP. It has already been well established in prior research across well over two decades that successful PDPs possess institutional support, take into account participant evaluations, and offer training of relevance to their contingents (e.g., Birman, Desimore, Porter, & Garet, 2000; Guskey & Huberman, 1995; Norton, 2001; Richardson, 2000; Wood & Thompson, 1993). Simon Fraser University’s (SFU) Report on Graduate Student Professional Development at SFU: Findings and Recommendations (2013) addresses the need to develop a program evaluation that measures learners’ satisfaction with PDPs and the skills gained as a result of participating in a PDP-related event, together with other evaluations targeting organization, delivery mechanisms, and ability to meet intended learning objectives. CJHE / RCES Volume 44, No. 3, 2014 GradProSkills/ V. Venkatesh , J. Rabah, L. Lamoureux-Scholes, I. Pelczer, K. Urbaniak, & F. Martin 41 Such program evaluations can be achieved through post-attendance questionnaires, professional portfolios, informal interviews, as well as focus groups with attendees (Holaday et al., 2007; Rose, 2012). Importantly, Rose (2012) exhorts universities to undertake studies to track the relation between success in gaining employment and students’ attendance of PDPs. Concordia University’s Graduate and Professional Skills Development Program (GradProSkills) GradProSkills is a PDP that responds to the market demand of graduate students being well prepared by their respective universities before they exit to the workforce. GradProSkills emphasizes giving graduate students better experience and more varied exposure to skills rather than focused proficiency in their respective specialized programs of study. The overarching aim of GradProSkills is to arm them with a whole array of professional skills before they search for jobs in the labour market. The GradProSkills initiative consists of non-credit workshops and resources for graduate students. It is designed to complement academic skills by training students in domain skills that they may not learn in their respective classrooms. GradProSkills has adopted the nine skill domains recognized by the CAGS (2008) report Professional Skills Development for Graduate Students as the most significant skills to be developed by graduate students seeking various career pathways, be those within academia, business/industry, health and biotechnology, arts and creative industries, government, or nongovernmental organizations. Post-workshop feedback forms were adopted as the formative course evaluation tools to help improve student experiences (see http://graduatestudies.concordia. ca/gradproskills/ind_workshop_feedback.php). GradProSkills is closely linked to Concordia University’s Academic Plan (2012), for which experiential learning and community engagement are essential activities fundamental to the university’s academic mission. These two elements help students transfer academic competencies into practical knowledge in society and increase their chances of success and motivation after they attain their academic degree. These components of the plan also help Concordia reach a goal of strengthening university–industry partnerships and tightening the connections between academics and the real world. GradProSkills responds to assertions made in the Academic Plan, wherein the university commits to creating funding opportunities for learners seeking to participate in applied learning via industry internships. In an effort to ensure efficient coordination among existing departments and faculties, GradProSkills offers a centralized administrative office via the School of Graduate Studies as it relates to professional development and career links, pooling resources that internal and external stakeholder institutions and placement providers can access. In addition, the GradProSkills webportal has a Build Your Toolkit option (see http://graduatestudies.concordia.ca/gradproskills/index.php?sid=3), whereby students can customize their course selections according to the career path they are seeking, connecting the latter to the related domain skills to be developed. Build Your Toolkit delineates a direct linkage between the courses and the nine essential domain skills recognized by the CAGS 2008 report. GradProSkills also reflects the increased focus on professional skill development recommended by the Gouvernement du Québec (2013). In their report titled Québec’s Eco- CJHE / RCES Volume 44, No. 3, 2014 GradProSkills/ V. Venkatesh , J. Rabah, L. Lamoureux-Scholes, I. Pelczer, K. Urbaniak, & F. Martin 42 nomic Policy—Putting Jobs First: Investing in Jobs is Investing in Québec, the government disseminated the province’s National Research and Innovation Policy for 2014 to 2019. The Ministère de l’Enseignement supérieur, de la Recherche, de la Science et de la Technologie, in conjunction with the Fonds de recherche du Québec, will work with universities to organize 1,200 additional internships in institutions for postsecondary students working in research and/or technology support and innovation. Part One: Development In the fall of 2010, Concordia University established a steering committee of senior academic administrators to oversee a three-year pilot project within the School of Graduate Studies that would offer all graduate students and postdoctoral fellows free access to a comprehensive suite of graduate and professional skills training resources. To ensure GradProSkills program would respond to specific student needs, the working committee established in January 2011 included graduate student representatives from each of the four faculties (master’s from Fine Arts; doctoral from Engineering and Computer Science; doctoral from the John Molson School of Business; and doctoral from Arts and Science), one postdoctoral fellow from Arts and Science, and support staff from the School of Graduate Studies and the Office of Research. This student-driven approach has become a defining feature of the program, with the GradProSkills team employing up to 14 graduate students over the fall and winter semesters of an academic year as workshop assistants, language group leaders, web maintenance personnel, and program development team members. For example, presently, the GradProSkills program development team includes five graduate students. The events team member focuses on the planning and delivery of a variety of services, including career panels, alumni workshops, and network-building activities. The outreach team member promotes the program through orientation presentations and coordinates an ambassador program. Two graduate students make up the academic skills development team. The fifth and final online team member focuses on managing the social media content for the program’s blog (called GradProBlog) as well as its Twitter, Facebook, and LinkedIn profiles. The initial task of the working committee was two-fold. First, the committee catalogued existing training resources available through various student services on campus. The exercise identified over one hundred potential workshops that were either already directed at graduate students or could be adapted to meet their needs. The second, related, task reviewed existing graduate student PDPs across Canada to identify trends in the range and organization of resources. This review led the working committee to adopt the nine skill domains identified in the CAGS (2008) report: leadership; research management; strategic communication; information and digital intelligence; career building; teaching and knowledge transfer; public spirit and social consciousness; wellness and life balance; and learning to learn. With funding from the Ministère de l’Éducation, du Loisir et du Sport (MELS), the steering committee also added a tenth skill domain area—secondlanguage training in French—to facilitate students’ transition into the Québec workforce. GradProSkills requires a centralized registration system to track and monitor participation and to provide students with easy access to a record of participation. With the skill domains and registration system in place, the committee worked with web designers and CJHE / RCES Volume 44, No. 3, 2014 GradProSkills/ V. Venkatesh , J. Rabah, L. Lamoureux-Scholes, I. Pelczer, K. Urbaniak, & F. Martin 43 conducted focus groups with current graduate students, alumni, and faculty to develop a website that would allow students to browse workshop details and online resources by either skill domain, professional sector, or graduate training or through searches of the workshops calendar or individual partner pages. The final task of phase one was to negotiate partnership agreements with all service departments. GradProSkills offered each partner: (i) a dedicated partner page on the GradProSkills website (http://graduatestudies.concordia.ca/gradproskills/); (ii) the opportunity to maximize workshop participation through promotion on the website, in bi-weekly newsletters, and through social media; (iii) a workshop assistant to collect attendance and support the workshop leader; and (iv) feedback reports to review and revise content as required. Partnerships were finalized by July 2011 and workshop/partner details added to the GradProSkills website for the program launch on August 4, 2011. Part Two: Implementation In its first year, GradProSkills and its partners delivered 196 workshops with 4,154 registrations by 1,182 unique students across all faculties and streams (graduate certificate, graduate diploma, master’s, doctoral, and postdoctoral fellows), with almost 70% of the workshops filled to capacity. Using workshop attendants’ feedback and a second series of focus groups, the second year of the GradProSkills program focused on refining the existing programming, developing graduate-level reading and writing strategies, and expanding career-building opportunities. An advisory committee of graduate program directors from each of the four faculties, a coordinator with the ESL program, and a learning specialist from the University Writing Centre provided guidance to develop a graduate-level reading and writing program, including a three-day Graduate School Base Camp for new graduate students, held in the last week of August. Base Camp is a series of workshops designed for newly admitted graduate students to highlight the base-line or fundamental resources critical to a successful graduate school experience. These workshops are recommended for international students new to the Canadian university experience, graduate students returning to university after an extended absence, or current students looking for additional strategies to negotiate the graduate school academic workload. The second year also focused on the development of network-building events and outreach to external partners, including the International Association of Business Communicators – Montréal Chapter and the Young Chamber of Commerce Montréal. At networking events, the online team coordinated a video capture station and invited external mentors to share their best practices and professional journey stories. To date, over 40 videos have been captured and posted to the GradProSkills website. The workshop evaluation system was also refined in March of 2013 (see http://graduatestudies.concordia.ca/gradproskills/ind_workshop_ feedback.php). Additions to second-year programming expanded the workshops offered to 286, with registrations of 6,045 representing 1,866 unique students. The focus for the third year of GradProSkills was to expand the offerings through continued refinements of the existing structure and to develop programs that would speak more directly to strategies for academic research career development and entrepreneurial opportunities. For each of the fall and winter academic terms, four faculty members with expertise in issues that students select via focus groups host discussions with graduate students on a particular research theme. The theme for the fall 2013 series was research- CJHE / RCES Volume 44, No. 3, 2014 GradProSkills/ V. Venkatesh , J. Rabah, L. Lamoureux-Scholes, I. Pelczer, K. Urbaniak, & F. Martin 44 creation; the theme for the winter 2014 series was managing multidisciplinary research projects. GradProSkills has also partnered with SAJE accompagnateur d’entrepreneurs, a Montréal-based non-profit organization that provides advisory services, coaching, and training to new entrepreneurs in Montréal. SAJE representatives have delivered workshops, participated in networking events, and hosted a “Meet the Entrepreneur” event with four entrepreneurs who participated in their program. Concordia alumni have also been invited to propose workshops for sharing their postgraduate experiences with current graduate students. The graduate academic skills read/write/present program has also expanded to emphasize the importance of peer-review techniques to polish writing and presentation skills, as well as build important editorial skills necessary for graduate students interested in pursuing an academic career. These new program additions increased the fall 2013 offerings to 222 workshops with 3,402 registrations by 1,256 unique graduate students. At the start of 2014, GradProSkills was approved to transition from pilot to permanent program. As a result, the administrative structure and funding of the program are currently being reorganized. Part Three: Results of Evaluation Instrument development. As part of the systematic development and evaluation of the GradProSkills program, we conducted classroom-based evaluations of the various offerings between March 2013 and December 2013 using the feedback form available at http://graduatestudies.concordia.ca/gradproskills/ind_workshop_feedback.php. The items developed for the evaluation instrument focused on individual graduate students’ perceptions of the effectiveness of the GradProSkills offering upon their completion of the program; it comprised a total of 12 statements, which students rated on a five-point Likert scale for their level of agreement. The items measured GradProSkills participants’ attitudes in three broad categories: course satisfaction, course quality, and instructor quality. The evaluation instrument was created in close consultation with experts in educational psychology, instructional design, as well as formative and summative evaluation methods. Sample and methodology. We received the voluntary participation of 3,292 registrants out of a possible 5,403, yielding a 60.9% return rate. Table 1 shows the demographic characteristics of the sample, which is representative of the larger population from which it is drawn in terms of the distribution of levels of study, stage of completion of program, and faculties to which participants belong. We used a combination of descriptive and inferential statistics to analyze the data from the evaluation instrument. To ensure the statistical validity of the analyses, we used the individual registrant as the unit of analysis, as opposed to each graduate student (note that each student might have registered in more than one workshop). In moving from individual student to registrant as the unit of analysis, we employed intra-sample statistical analysis (ISSA) techniques (Shaffer & Serlin, 2004; Venkatesh & Shaikh, 2011). When one is confronted with data organized and analyzed by the learner as the unit of analysis, it is not uncommon to notice that the lack of a large sample, combined with repeated measure procedures, leaves very little room for powerful statistical results. Treating the registrant as the unit of analysis enables us to employ powerful statistical procedures with a relatively larger sample. For the purposes of our analyses, therefore, we were able to generalize only to all possible registrants between March 2013 and December 2013, which is, CJHE / RCES Volume 44, No. 3, 2014 GradProSkills/ V. Venkatesh , J. Rabah, L. Lamoureux-Scholes, I. Pelczer, K. Urbaniak, & F. Martin 45 by definition, limited to the graduate students who responded to the evaluation form. In addition, as per Shaffer and Serlin’s (2004) description of the notion of “exchangeability,” we treated each graduate student who responded to the evaluation form as a fixed effect in any statistical model so as to contextualize the results with respect to the sample of individuals from which the registrations were drawn. Missing values were not replaced, to preserve the assumption of exchangeability required to conduct ISSA. Table1 Demographic Characteristics of Respondents to Evaluation (n = 3,292) Items Level of study Diploma Master Doctoral Post-doctoral Other Stage of completion Beginning cycle Mid-cycle Near completion Not specified Faculty John Molson Business School Arts and Science Engineering Fine Arts Not specified Frequency 253 2,083 652 38 266 995 598 469 1,230 245 520 1,443 167 917 Composite creation. To create composites related to perceptions of course satisfaction, course quality, and instructor quality, which are employed in inferential statistical analyses, we report, in Table 2, means and standard deviations for all items included in the composites. We also report the inter-correlation ranges as well as the Cronbach’s alpha values, which are high enough to merit the creation of the composites. We were obliged to remove one of the evaluation items in the course quality composite, namely, “this workshop met its objectives” due to its high correlation (> .90) with each of the other three items in the composite, which inflates the inter-correlation ranges as well as Cronbach’s alpha, thereby diminishing the validity of the composite. Overall, graduate students partaking of the GradProSkills program were satisfied with the offerings they completed, as evidenced by the relatively large means values reported in Table 2. Apart from charting predictive relationships amongst the perceptual factors that impact course satisfaction, our analyses will also slice the data by demographic variables such as faculty, stage of program completion, level of study, and the skill domain covered in the GradProSkills offering that the graduate student evaluated. CJHE / RCES Volume 44, No. 3, 2014 GradProSkills/ V. Venkatesh , J. Rabah, L. Lamoureux-Scholes, I. Pelczer, K. Urbaniak, & F. Martin 46 Table 2 Descriptive Statistics, Intercorrelation Range, and Internal Consistency (Cronbach’s α) for Items in Composite Variables Composite items1 M SD 4.12 1.009 4.19 1.003 4.08 1.041 4.14 1.017 Course satisfaction This workshop identified new skills important for my professional development. This workshop increased my knowledge about the topic. This workshop addressed my expectations of the training course. This workshop provided knowledge and skills I will be able to apply to my studies/work. Course Quality InterCronbach’s α correlation Range & p value .728 to .794, .925 p<.001 This workshop had clear objectives. This workshop offered practical techniques/ strategies for applying new skills. This workshop provided appropriate resources to use and take away. Instructor Quality 4.31 4.14 0.959 1.011 4.11 4.28 0.952 4.28 .925 0.913 4.29 0.740 to .808, p<.001 0.993 4.42 .846 1.019 The instructor of this workshop presented content appropriately. The instructor of this workshop presented material in a clear and organized manner. The workshop had a presenter who responded to questions. The instructor of this workshop used methods of presentation that were appropriate to the topic. .625 to .672, p<.001 0.950 1 Response measured on a five-point scale from strongly disagree (1) to strongly agree (5) Perceptual factors predicting course satisfaction. We separately regressed the course satisfaction composite variable on the individual items from the other composites, as well as on the two composites of course quality and instructor quality. Overall, we were able to predict 81.4 % of the variance in course satisfaction with six items in the evaluation instrument (see Table 3). The only item that did not significantly regress into the equation was the one that addressed whether the instructor “presented material in a clear and organized manner.” Further, the composites of course quality (β = .722) and instructor quality (β = .201) predicted 80.8% variance in course satisfaction (see Table 4). CJHE / RCES Volume 44, No. 3, 2014 GradProSkills/ V. Venkatesh , J. Rabah, L. Lamoureux-Scholes, I. Pelczer, K. Urbaniak, & F. Martin 47 Table 3 Factors Predicting Composite Variable of Perceptions of Course Satisfaction Predicted Variable: Course Satisfaction (Composite) Items1 The instructor of this workshop presented content appropriately. The instructor of this workshop presented material in a clear and organized manner. The workshop had a presenter who responded to questions. The instructor of this workshop used methods of presentation that were appropriate to the topic. This workshop had clear objectives. This workshop provided appropriate resources to use and take away. This workshop offered practical techniques/strategies for applying new skills. 2 R F Adjusted R2 B .048 SE B .016 β .050** .025 .015 .028 .055 .015 .055*** .110 .154 .271 .014 .013 .012 .115*** .162*** .303*** .300 .012 .333*** .815 1665.367*** .814 *p<.05, **p<.01, ***p<.001 1 Response measured on a five-point scale from strongly disagree (1) to strongly agree (5) Table 4 Factors Predicting Composite Variable of Students’ Course Satisfaction Predicted Variable: Course Satisfaction (Composite) Factors (composite items) Course quality Instructor quality R2 F Adjusted R2 B SE B .752 .017 .213 .017 .809 5601.633*** .808 β .722*** .201*** *p<.05, **p< .01, ***p<.001 Differences in course satisfaction by level of study, faculty, stage of completion, and skill domain. We investigated the differences in perceptions about course satisfaction across multiple independent variables using both one-way and factorial ANOVAs; note that n dips to 2,779 for all these results, due to missing values, and that all the results of post-hoc analyses are reported at a p value of 0.05 or less, along with CJHE / RCES Volume 44, No. 3, 2014 GradProSkills/ V. Venkatesh , J. Rabah, L. Lamoureux-Scholes, I. Pelczer, K. Urbaniak, & F. Martin 48 details of the statistically significant omnibus F tests. In using level of study as an independent variable, we found that doctoral students (M = 3.97, SD = 0.99, n = 554) were significantly less satisfied with their courses than postdoctoral (M = 4.19, SD = 0.74, n = 28), master’s (M = 4.17, SD = 0.89, n = 1,751), and diploma students (M = 4.19, SD = 0.99, n = 554), omnibus F(4, 2774) = 4.78, p < .001, Effect Size (ES) = .01. In using faculty as an independent variable, we found that students from the John Molson School of Business (M = 4.33, SD = 0.85, n = 225) were significantly more satisfied with their courses than those from the faculties of arts and science (M = 4.04, SD = 0.97, n = 436), engineering and computer science (M = 4.11, SD = 0.91, n = 1,228), and fine arts (M = 4.07, SD = 0.99, n = 144), omnibus F(4, 2774) = 4.17, p = .002, ES = .01. The independent variable of stage of completion did not yield a significant omnibus F when used in a one-way ANOVA with the course satisfaction composite as a dependent variable. To investigate whether and how the course satisfaction composite variable differed across the 10 different skill domains used to classify GradProSkills offerings and how it interacted with the other independent variables, we conducted a factorial ANOVA using the course satisfaction composite as a dependent variable, as well as level of study, skill domain, and faculty as independent variables. The only significant main effect found was for skill domain, F(9, 2730) = 2.89, p = .002, ES = .01. Neither the main effects for faculty or level of study, nor the various interactions between faculty, skill domain, and level were statistically significant. Follow-up univariate analyses revealed that course satisfaction scores for GradProSkills offerings for the teaching and knowledge transfer skill domain were significantly lower (p < .05) than for all the other skill domains except the wellness and life balance skill domain (see Table 5 for descriptive statistics of course satisfaction by skill domain). Discussion Our results demonstrate that the development, implementation, and testing of a theoretically grounded evaluation instrument is a necessary step in the systematic evaluation and upgrading of professional development offerings via a centralized administrative framework such as the one we offer at Concordia University. The rigorous application of needs analysis with stakeholders in the creation of the evaluation instrument was validated by strong internal consistency between items that formed the composite variables of course satisfaction, course quality, and instructor quality. Our analyses revealed that course satisfaction was predicted most by variables that fell under the aegis of course quality rather than instructor quality; this might reflect the emphasis that registrants placed on the individual elements of the course, such as its objectives, the kinds of techniques it offered, and the resources that it allowed registrants to access, over the individual qualities of the instructor. These results could also be an artifact of the amount of time registrants were in contact with the instructor; many of the GradProSkills offerings last between 90 minutes and three hours, but only a few of the offerings involved repeated contact between workshop leaders and graduate students, which might have made it more difficult to create a perceptual relationship between the quality of the instructor and the students’ satisfaction. Our data analyses reveal that course satisfaction was significantly higher for students in the John Molson School of Business than for those in other faculties and significantly lower for doctoral students than for those in other levels of study. CJHE / RCES Volume 44, No. 3, 2014 49 GradProSkills/ V. Venkatesh , J. Rabah, L. Lamoureux-Scholes, I. Pelczer, K. Urbaniak, & F. Martin However, these differences were rendered opaque when considered in a factorial analysis with the skill domains under which the course offerings were categorized. The analyses show that the teaching and knowledge transfer skill domain and the wellness and life balance skill domain elicited lower levels of course satisfaction than the other skill domains, regardless of the student’s faculty and study level. Offerings that fall under the teaching and knowledge transfer skill domain are designed to help graduate students develop pedagogical and administrative skills in preparation for being teaching assistants or teaching university courses; hence, it can be difficult to achieve the goals of such workshops, and the participants’ motivational outlook differs qualitatively from that of participants in other workshops. To explore these differences, future qualitative evaluations should take into account the different instructional approaches employed by workshop leaders, the content being covered in workshops, as well as the duration of the workshops. Table 5 Course Satisfaction by Workshops’ Skill Domains (N = 2,779) Workshops’ skill domain Course satisfaction M1 Career building 4.21 0.84 653 Digital intelligence 4.40 0.78 186 Leadership 4.31 0.82 154 Learning to learn 4.09 0.97 424 Language training 4.10 0.90 271 Public spirit 4.44 0.48 23 Research management 4.10 0.93 310 Strategic communication 4.09 0.94 357 Teaching and knowledge transfer 3.85 0.99 328 Awareness and life balance 1 SD 4.04 1.03 73 N Response measured on a five-point scale from strongly disagree (1) to strongly agree (5) Conclusion Following from the initial evaluation of perceptions of effectiveness of the GradProSkills offering, we believe that we need to draw upon several other triangulated sources of data, including feedback from workshop instructors and seminar leaders, focus groups with participants, and longitudinal achievement and attitudinal data from participants in any future evaluations of the program. In addition, it would also be desirable to prepare and apply to the GradProSkills program a quality-assurance model that reflects the contextual reality of Concordia University. One of the first steps that we plan to take is to integrate the administrative components of the GradProSkills program into the broader unit of Academic Programs and Development at the School of Graduate Studies. The first project that would result from this integration would be to ensure the efficient and effective alignment of curricular and extracurricular outcomes from the variety of graduate programs offered CJHE / RCES Volume 44, No. 3, 2014 GradProSkills/ V. Venkatesh , J. Rabah, L. Lamoureux-Scholes, I. Pelczer, K. Urbaniak, & F. Martin 50 at Concordia University with those of the GradProSkills offerings. One of the major outcomes of the integration of GradProSkills into the Academic Programs and Development unit at the School of Graduate Studies will be knowledge exchange, which means that we will work with graduate students, graduate program directors, professors, researchers, administrators, and industry practitioners to develop ways to mobilize and update the insights generated by our GradProSkills team. Key foci include: (i) creating frameworks for evaluating graduate professional development across disciplines; (ii) providing training in evidence-based techno-pedagogical and instructional practices for graduate students seeking academic jobs; and (iii) developing policies on how best to track the employment history of Concordia alumni so as to create professional development opportunities that respond to the workforce-related needs of local and international industries. We are well aware of the two solitudes of the scientist and the practitioner or policy maker. There is too little engagement between researchers and practitioners in the field of graduate education, and this must be changed if the quality of professional development training is to improve. The question is how to do so, and the persistent and vexing nature of this challenge highlights the importance of creating a sustainable version of GradProSkills anchored in the scholarship of learning efficacy—one that will help illuminate the obstacles and affordances surrounding knowledge exchange in the domain of graduate-level professional development. Our commitment is to work proactively with the community within and without Concordia and to select partners to integrate evidence-based tools and techniques into GradProSkills offerings. Further, we will focus on implementation fidelity, along with widescale, long-term use of our empirically grounded curricular framework, thereby having a meaningful impact on curricular and co-curricular development, higher-order learning, critical thinking, and professional skill development in graduate programs. Also, by sharing expertise with educational administrators and policy makers, we hope to increase understanding of the uses of evidence-based practices and the optimal conditions for implementing professional development models in graduate education. References Birman, B. F., Desimore, C. L., Porter, A. C., & Garet, M. S. (2000). Designing professional development that works. Educational Leadership, 57(8), 28–33. Canadian Association for Graduate Studies (CAGS). (2008, November 5). Professional skills development for graduate students. Retrieved from http://www.cags.ca/ documents/publications/working/Prof%20Skills%20Dev%20for%20Grad%20Stud%20 %20Final%2008%2011%2005.pdf Charbonneau, L. (2011, November 30). Is Canada producing too many PhDs? Yes, no and maybe. University Affairs. Retrieved from http://www.universityaffairs.ca/iscanada-producing-too-many-phds.aspx Chillas, S. (2010). Degrees of fit? Matching in the graduate labour market. Employee Relations, 32(2), 156–170. Concordia University. (2012). Concordia University academic plan 2012–2016. Retrieved from http://www.concordia.ca/content/dam/concordia/docs/academic-plan.pdf CJHE / RCES Volume 44, No. 3, 2014 GradProSkills/ V. Venkatesh , J. Rabah, L. Lamoureux-Scholes, I. Pelczer, K. Urbaniak, & F. Martin 51 Ducheny, K., Alletzhauser, H., Crandell, D., & Schneider, T. (1997). Graduate student professional development. Professional Psychology: Research and Practice, 28(1), 87–91. Galt, V. (2012, December 20). Hot jobs: The growing divide in the labour market. The Globe and Mail. Retrieved from http://www.theglobeandmail.com/report-on-business/ careers/careers-leadership/hot-jobs-the-growing-divide-in-the-labour-market/ article6605254/ Gouvernement du Québec. (2013). Québec’s economic policy—Putting jobs first: Investing in jobs is investing in Québec. Retrieved from http://www.economie.gouv.qc.ca/ fileadmin/contenu/publications/administratives/politiques/politique_economique_ en.pdf Guskey, T. R., & Huberman, M. (Eds.). (1995). Professional development in education: New paradigms and practices. New York, NY: Teachers College Press. Holaday, B., Weaver, K. A., & Nilson, L. B. (2007). Revisioning graduate professional development programs. College Teaching, 55(3), 99–103. Newhouse, M. (1999, July 30). Using graduate school strategically. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/Using-GraduateSchool/45663/ Norton, J. (2001). Grounded in research. Journal of Staff Development, 22(3), 30–32. Poock, M. C. (2001). A model for integrating professional development in graduate education. College Student Journal, 35(3), 345–352. Richardson, J. (2000). Learning benefits everyone. Journal of Staff Development, 21(1), 54–59. Rose, M. (2012). Graduate student professional development: A survey with recommendations. Ottawa, ON: Social Sciences and Humanities Research Council. Retrieved from http://www.sshrc-crsh.gc.ca/about-au_sujet/publications/SSHRC_ report_Graduate_Students_Professional_Skills_March_2012_eng.pdf Shaffer, D. W., & Serlin, R. C. (2004). What good are statistics that don’t generalize? Educational Researcher, 33, 14–25.  Simon Fraser University (SFU). (2013, May 31). Report on graduate student professional development at SFU: Findings and recommendations. Office of Graduate Studies and Post-doctoral Fellows, SFU. Retrieved from http://www.sfu.ca/content/ dam/sfu/dean-gradstudies/ProfessionalDevelopment/SFU%20Report%20on%20 Professional%20Development.pdf Tamburri, R. (2010, January 11). Give us the dirt on jobs. University Affairs. Retrieved from http://www.universityaffairs.ca/give-us-the-dirt-on-jobs.aspx Venkatesh, V., & Shaikh, K. (2011). Uncovering relationships between task understanding and monitoring proficiencies in post-secondary learners: Comparing work task and learner as statistical units of analyses. Education Research International, 2011. doi:10.1155/2011/735643 CJHE / RCES Volume 44, No. 3, 2014 GradProSkills/ V. Venkatesh , J. Rabah, L. Lamoureux-Scholes, I. Pelczer, K. Urbaniak, & F. Martin 52 Wood, F. W., & Thompson, S. R. (1993). Assumptions about staff development based on research and best practice. Journal of Staff Development, 14(4), 52–57. Note 1. Venkatesh and Rabah are joint first authors. Venkatesh was responsible for the overall development and execution of the research, including elaboration of the methodology and discussion sections. Rabah oversaw the literature review and the analyses of results, and she revised and redrafted the manuscript after peer review. Contact Information Vivek Venkatesh School of Graduate Studies Concordia University [email protected] Vivek Venkatesh is at Concordia University, where he is associate dean, Academic Programs and Development, in the School of Graduate Studies, associate professor in the Educational Technology program, and acting director of the Centre for the Study of Learning and Performance. His interdisciplinary research program has yielded highly cited publications in the broad fields of learning sciences, the integration of information and communication technologies in university settings, and extreme metal music studies. Since 2008, Vivek Venkatesh has helped secure, as either principal investigator, co-principal investigator, or co-investigator on multi-institutional research teams, more than $4.5 million in funding from provincial, federal, and international agencies. Jihan Rabah is a Concordia University doctoral candidate specializing in educational studies. She currently holds a two-year Social Sciences and Humanities Research Council of Canada doctoral fellowship. Her academic degrees consist of a Master of Arts in educational psychology and a Bachelor of Arts in public administration, In addition, she holds a certificate in project management and has specialized in educating students with needs, having completed a two-year postgraduate diploma in special education. Her research interests include the theoretical and practical processes that affect the development of pedagogical curricula, specifically gendered citizenships and the integration of computer technologies therein. Laurie Lamoureux Scholes completed doctoral studies in Concordia University’s Department of Religion. Her research applies social-historical and fieldwork methodologies to examine how religious identity is nurtured and maintained within contemporary cosmopolitan environments. Lamoureux Scholes has delivered papers at annual meetings of the American Academy of Religion, the Canadian Society for the Study of Religion, and Metropolis. She has co-authored a chapter on research ethics in religious studies (2012) and was the lead contributing author for a study of interfaith activity in Canada, conducted for Heritage Canada (2010). Since 2011, she has been director of the Concordia Graduate and Professional Skills Program. CJHE / RCES Volume 44, No. 3, 2014 GradProSkills/ V. Venkatesh , J. Rabah, L. Lamoureux-Scholes, I. Pelczer, K. Urbaniak, & F. Martin 53 Ildiko Pelczer is currently a PhD student at Concordia University in mathematics education. Her professional interests lie at the crossroads of education—especially mathematical education—and computer science (artificial intelligence). Ildiko has a special interest in bringing technology to the aid of teaching by offering personalized support and guidance to students and creating virtual spaces for discussion and collaborative learning. Kathryn Urbaniak is a research associate/coordinator at Concordia University. She has an MA in educational technology from Concordia and graduated with a BA (Hons.) in business studies from Robert Gordon University and a PGDip in information technology from Stirling University. Frédérica Martin is the Academic Programs Analyst at the School of Graduate Studies, Concordia University. She conducts comparative analyses of graduate programs and advises programs and faculties on new graduate program development and major curriculum revisions. In the context of her work, she has studied the use of degree-level expectations in program design, mechanisms for facilitating the creation of interdisciplinary programs, and models of curricular development and revision processes. She holds two master’s degrees, one in psychology and one in educational studies. CJHE / RCES Volume 44, No. 3, 2014