Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 1 CSSHE SCÉES Canadian Journal of Higher Education Revue canadienne d’enseignement supérieur Volume 48, No. 1, 2018, pages 1 - 21 Knowledge Mobilization Practices of Educational Researchers Across Canada Amanda Cooper Queen’s University Joelle Rodway Ontario Institute for Studies in Education, University of Toronto Robyn Read Ontario Institute for Studies in Education, University of Toronto Abstract Researchers are under increasing pressure to disseminate research more widely with non-academic audiences (efforts we call knowledge mobilization, KMb) and to articulate the value of their research beyond academia to broader society. This study surveyed SSHRC-funded education researchers to explore how universities are supporting researchers with these new demands. Overall, the study found that there are few supports available to researchers to assist them in KMb efforts. Even where supports do exist, they are not heavily accessed by researchers. Researchers spend less than 10% of their time on non-academic outreach. Researchers who do the highest levels of academic publishing also report the highest levels of non-academic dissemination. These findings suggest many opportunities to make improvements at individual and institutional levels. We recommend (a) leveraging intermediaries to improve KMb, (b) creating institutionally embedded KMb capacity, and (c) having funders take a leadership role in training and capacity-building. Résumé Les chercheurs sont de plus en plus pressés de diffuser la recherche plus largement auprès de publics non universitaires (des efforts que nous appelons mobilisation de connaissance, KMb) et d’exprimer la valeur de leur recherche au-delà du milieu universitaire vers une société élargie. Cette étude a enquêté sur les chercheurs financés par le CRSH afin d’explorer comment les universités soutiennent les chercheurs avec ces nouvelles demandes. Dans CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 2 l’ensemble, l’étude a révélé qu’il y a peu de soutien offert aux chercheurs pour les aider avec les efforts KMb. Même lorsque le soutien existe, il n’est pas grandement utilisé par les chercheurs. Les chercheurs dépensent moins de 10  % de leur temps à des activités non universitaires. Les chercheurs qui accomplissent le plus haut degré de publication universitaire rapportent aussi les plus hauts niveaux de diffusions non universitaires. Ces résultats suggèrent de nombreuses occasions d’amélioration à des niveaux tant individuels et institutionnels. Nous recommandons (a) de mobiliser les intermédiaires pour améliorer le KMb, (b) de créer une capacité KMb institutionnellement intégrée, et (c) de faire les bailleurs de fonds jouer un rôle de leadership dans la formation et le renforcement des capacités. Introduction Research impact agendas are being championed by governments and funders across the globe (Bastow & Tankler, 2014; Hicks, 2012; King’s College London and Digital Science, 2015; Tetroe et al., 2008; Wilson, Petticrew, Calnan, & Nazareth, 2010). The rationale behind impact movements is that publicly funded research should have tangible benefits for citizens and governments need to demonstrate a return on investment in relation to research. As a result, researchers and universities are under increasing pressure to share research more widely with non-academic audiences in more accessible formats to strengthen the impact of publicly funded research on policy and practice across sectors (Nutley, Walter, & Davies, 2007; Sá, Li, & Faubert, 2011). The Social Sciences and Humanities Research Council (SSHRC) of Canada refers to these efforts as knowledge mobilization (KMb) and articulates the underlying purpose of this global movement: Knowledge mobilization is about ensuring that all citizens benefit from publicly funded research. It can take many forms, but the essential objective is to allow research knowledge to flow both within the academic world and between academic researchers and the wider community. By moving research knowledge into society, knowledge mobilization increases its intellectual, economic, social and cultural impact. (SSHRC, 2014) SSHRC now requires Canadian researchers to submit a KMb plan and a section articulating potential impacts expected from research grants. These global trends—research impact and KMb—are changing the landscape of higher education (Hicks, 2012; Wilsdon et al., 2015). Historically, academia has been marked by the pursuit of publishing high-quality, peer-reviewed publications (Smith, 2010; Wilson et al., 2010); however, these trends are problematizing academic production, highlighting that academic journal articles usually fail to have an impact on those beyond the ivory towers (Sá et al., 2011). We surveyed researchers from Faculties of Education across Canada to explore their KMb practices and asked if institutional supports were available in their universities to support researchers with the growing demands of mobilizing research for non-academic audiences. We surveyed researchers who had completed their SSHRC projects in 2011, because the literature on impact highlights that research impact takes time to develop and often manifests years after the end of a particular project (Grant, Brutscher, Kirk, Butler, & Wooding, 2005). CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 3 At the time of this study, while there were studies exploring academic dissemination, there was no empirical work exploring KMb practices of researchers or comparing researchers’ academic and non-academic dissemination efforts. It is the focus on non-academic outreach, KMb practices, and institutional supports for KMb with non-academic audiences that makes this study distinct. Why Does This Research Topic and Study Matter? This area of research is important because of a number of global trends that are affecting Canadian researchers: • Research funding agencies, including SSHRC, are increasing expectations for researchers in relation to KMb efforts with non-academic audiences and tracing the impact of their research (Tetroe et al., 2008; Wixted & Beaudry, 2012). • There has been a rise of performance-based research funding systems in 14 jurisdictions that link research impact assessments to direct funding of universities (Hicks, 2012). • SSHRC requires KMb plans and proposed impacts from Canadian researchers (SSHRC, 2014). • There has been a lack of empirical work on KMb globally in relation to researchers and universities (Mitton, Adair, Mckenzie, Patten, & Perry, 2007; Nutley et al., 2007) but also in relation to tracing impact and KMb efforts in the social sciences (Bastow & Tankler, 2014; Davies, Nutley, & Walter, 2005; Wilsdon et al., 2015) Consequently, this study is significant because it provides baseline data on (a) researchers’ KMb practices and (b) existing institutional supports for KMb activities in universities1 across Canada—an area where very little data exists. We argue that if researchers are expected to mobilize their research in new ways and with potentially new audiences, institutional training and supports should be provided to assist researchers with these efforts. However, before we advance this argument, we need to know what types and proportions of institutional supports exist, how heavily they are accessed, and how researchers are disseminating their work to non-academic audiences. While this study focuses on education researchers, we argue that suggestions to improve KMb efforts in the discussion are applicable to researchers across disciplines, including (a) researchers leveraging intermediaries to improve KMb efforts and impact, (b) the need for universities to create institutional KMb support structures for researchers across disciplines, and (c) the need for funders to take a leadership role in capacity-building efforts. Literature Review: A Brief Synopsis in Relation to Our Conceptual Framework Researchers in the field of organizational theory have long explored what factors explain the way people work within organizations in order to determine which structures are most important (Bolman & Terrence, 2008; Scott, 2004). Educational research has similarly explored how people shape—and are shaped by—the educational organizations in which they are involved on a daily basis (Owens & Valesky, 2015). Because humans are social creatures, social norms can have a powerful influence in shaping their behaviour; therefore, particular organizational cultures can be quite pervasive in determining how CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 4 people think and act in professional environments (Hemsley-Brown, 2004). This also means that organizational norms and culture can be a powerful influence on how professionals view competing priorities in relation to their core work. Consequently, to understand educational researchers’ KMb activities, considering the institutional structures that facilitate or inhibit these processes might be important to improve our understanding of why researchers are, or are not, engaging in KMb activities. Similarly, we have little empirical evidence on the kinds of supports that exist, and data resulting from our study may also guide faculty improvement planning in response to increasing funding requirements to engage in KMb. For KMb to happen consistently, policies and strategies must be developed and embedded at the organizational level (Nutley et al., 2007); similarly, researchers should not be expected to increase KMb activities without funders and/or universities providing the requisite capacity-building and support required for researchers to be successful in non-academic outreach. The empirical evidence suggests that KMb is not well aligned with the priorities of academia, which focuses on generating publications within academic communities rather than making research accessible and useful to practitioners and policy makers (Hargreaves, 1999; Sá et al., 2011; Smith, 2010; Willinsky, 2000). The pressure on faculty members to produce peer-reviewed journal articles and continually chase grant funding, what Smith (2010) refers to as academic treadmills, is well documented (McGrail, Rickard, & Jones, 2006; Smith, 2010). Incentive structures throughout academia, including tenure and promotion, heavily favour academic production over non-academic outreach (Sá et al., 2011) despite the fact that funders are now changing what is expected of researchers (Tetroe et al., 2008). Exploring the institutional supports for research mobilization at universities is important for a few reasons. First, their availability (or lack thereof) speaks to institutional priorities. In addition, it speaks to universities’ capacity to engage in this work. For instance, researchers may need support in articulating their work’s implications for practitioners (Cordingley, 2008), as researchers from specialist areas are not necessarily adept at articulating the implications of their findings to practitioners or policy makers (Oliver, Innvaer, Lorenc, Woodman, & Thomas, 2014). Types of KMb Activities A typology of three major KMb strategies has emerged from the literature: researchbased products, events, and networks (Cooper, Levin, & Campbell, 2009). Various forms of each of these strategies have been described (Barrington Research Group Inc., 2004; Cooper, 2014; Kishchuk, 2003; Meagher, Lyall, & Nutley, 2008), such as academic research publications (books, book chapters, refereed publications), non-academic publications (plain language summaries, recommendations, alternative formats), academic and non-academic events (those to which researchers are invited and those initiated by researchers), and various kinds of networks to support research use and KMb efforts. We asked educational researchers about their academic and non-academic outreach in relation to KMb products, events, and networks to gauge their dissemination efforts. Although there is literature discussing the pressure and focus on academic production, there is little to no empirical evidence that we knew of at the time of the study comparing the academic and non-academic outreach activities across products, events, and networks. The literature does highlight active strategies as being more influential than CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 5 passive strategies (Mitton et al., 2007), so we also built in queries for events and networks to see which were invited versus initiated. Our thinking was that initiated events are a proxy for how active researchers are going out and making these opportunities happen with different stakeholders rather than being asked to be involved (which would be more passive on the researcher’s part). This is not to say that invited events and networks have any less impact than those initiated by the researcher, but we still thought this factor could be used to interpret levels of KMb activities. Methodology This study, approved by our institutional Ethics Review Board, investigated the KMb efforts of Canadian educational researchers and the institutional supports available at their universities to assist them with their research dissemination efforts. Research Questions and Conceptual Framework 1. What supports are available in Canadian universities to support educational researchers’ KMb efforts? Where supports are available, how often are they accessed by educational researchers? 2. How do levels of educational researchers’ academic outreach compare with their non-academic outreach in relation to products, events, and networks? 3. What proportion of time do educational researchers report spending on research versus other kinds of KMb activities? 4. What types of dissemination mechanism (print, online, media, intermediaries) do educational researchers use? How prepared do educational researchers feel to deal with various KMb activities (e.g., plain language writing or being involved in stakeholder collaboration)? Our conceptual framework for this study (Figure 1) includes four dimensions corresponding to the research questions. Survey The survey included a demographic section as well as four sections corresponding to our conceptual framework on institutional supports, academic versus non-academic outreach, allocation of researchers’ time, and dissemination (see Appendix A). Validity is defined as the trustworthiness of the data and asks whether we are measuring what we think we are measuring (Eisenhart & Howe, 1992). Face validity (which assesses whether a test looks valid to participants who take it) was assessed by piloting the survey with researchers outside the KMb field. Content validity (which requires the use of recognized experts in a particular area) was assessed by piloting the survey with key advisors from the field who have expertise in KMb. The survey questions were adapted and refined in relation to the feedback provided by both groups (researchers and KMb experts). Unfortunately, criterion validity (how well items in this survey compare with other measures of surveys on levels of researchers’ KMb) was not possible because there were no other studies that we were aware of at the time of our survey that attempted to measure KMb activity. Construct validity was also not evaluated for this survey because each survey question asked about frequencies of different kinds of activities; hence, there was not a CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 6 number of items that needed to be aggregated to check a particular construct, such as trust or depression, as in other studies. Reliability measures the consistency of a survey’s measurement (Eisenhart & Howe, 1992). We did not conduct reliability testing or use the instrument multiple times on the same group to test measurement reliability. So, while the survey has face and content validity as a measure of the KMb activity of Canadian educational researchers, we are unsure of its reliability since the survey would need to be implemented more than once in order to ascertain reliability. Figure 1. Conceptual framework: KMb efforts of Canadian researchers. Sample The sample of Canadian educational researchers was compiled from the online SSHRC database for researchers who held projects that were completed in 2011. The rationale behind this purposeful sample was that SSHRC grant holders were likely to be among the most active educational researchers in the country, given the competitive nature of the national funding mechanism, and could therefore teach us the most about research dissemination practices in Canada. After removing duplicate names of researchers who had received multiple grants over the years, 278 researchers from across Canada remained. Knowing that dissemination and uptake occur over time, we also hoped that asking researchers about SSHRC projects they had already completed would mean that some longitudinal uptake or dissemination might be captured. CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 7 A total of 278 researchers were contacted by email, informed of the study’s purpose, and invited to complete an online survey via an embedded URL. We received 111 responses in total, of which 14 were excluded because of non-completion. The response rate was 35% (N = 97/278), which is average for an online survey (Kittleson, 1995). Most participants (66%) were female. Almost all participants had completed their PhD more than 10 years ago (91%), with the largest proportion of respondents completing their PhD more than 20 years earlier (39%). Most of the sample (91%) consisted of tenured faculty members. Researchers reported the value of their SSHRC awards between $50,000 and $100,000 (32%), $101,000 and $150,000 (44%), and $151,000 and $200,000 (24%). All provinces except Newfoundland and Labrador and Prince Edward Island were represented in the sample, and our sample compares closely with the national levels of SSHRC funding by province (Table 1). Table 1. Comparison of SSHRC Distribution of Research Grants (2004–5) and Our Sample by Region Canadian Region Maritimes Prairies Ontario Quebec British Columbia SSHRC Distribution (%) 6 12 39 28 14 Our Sample (%) 8 11 41 28 12 Data Analysis Survey data were analyzed in SPSS, focusing predominantly on descriptive statistics (means, medians, standard deviations, and range) to map the terrain in relation to KMb efforts and institutional supports. To deepen the investigation of academic versus non-academic outreach activities, Pearson’s chi-squared analyses were conducted to determine whether there was a relationship between these two types of outreach activities. Effect sizes were also measured using Cramer’s V statistic. The survey broke down the concept of academic/non-academic publications (products) into different product types such as refereed publications and plain language summaries. To investigate whether there was a relationship between academic and non-academic products, the reported frequencies for each publication type were aggregated to produce variables reporting the total number of academic/non-academic publications to meet the minimum expected cell frequencies to facilitate the chi-squared analyses. Each composite variable (the total number of both academic and non-academic publications) was then split into two groups to produce a 2x2 contingency table: (a) people who reported a below-average number of academic/non-academic publications, and (b) people who reported an average or above-average number of academic/non-academic publications. Chi-squared analyses were conducted using these new variables. CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 8 Findings Canadian SSHRC-funded researchers who held education grants reported lower levels of KMb occurring with non-academic audiences than with academic audiences. Academic versus non-academic outreach was occurring roughly at a ratio of two to one. These proportions align with the incentive structures at universities in relation to how academic activities are valued for renewal, tenure, and promotion over non-academic outreach. There are few institutional supports available to researchers for their mobilization efforts, especially with non-academic audiences, despite increasing expectations from funders and universities that researchers engage in these efforts; and even at universities where these supports do exist, they are not heavily accessed by researchers. Researchers feel confident writing plain language summaries of their work and collaborating with stakeholders but are unprepared to use technology to disseminate their work and to deal with media. Findings will be reported in more detail in relation to each dimension of the conceptual framework; we asked researchers to keep just one SSHRC-funded study in mind when responding to the survey. Institutional Supports Available to Assist Researchers with KMb Efforts Respondents reported the availability of at least some type of administrative supports (61%) and/or communication supports (69%) to help them in their KMb efforts. Communication supports were distinguished from administrative supports because they were targeted specifically at disseminating research outside of the university, whereas administrative supports were used within the university or for a particular project. The three administrative supports reported to be the most frequently available across institutions (KMb efforts incorporated into tenure and promotion decisions, special personnel to manage or support research-sharing activities, and funding specifically for sharing research) were also the most frequently accessed supports (Figure 2). The proportion of researchers who report actually using the available resources ranges widely from as low as 20% of researchers accessing supports in relation to awards or a recognition for sharing research to 55% accessing funding specifically for sharing research. Funding remains an important incentive for researchers, and it was the most heavily accessed by researchers at universities where it was available for KMb. Communication supports showed similar trends to administrative supports, with researchers reporting low availability of these structures with very few researchers accessing these resources even at universities where they did exist (see Figure 3). Approximately half of the respondents reported communications departments (47%) and training available to deal with media (40%), and about a third of researchers had training available at their university focused on using technologies for research dissemination (31%). Less than 20% of researchers reported communication supports available to assist in producing plain language summaries or executive summaries for non-academic target audiences. Similar to the administrative supports, even where communication supports were available at universities, researchers did not report accessing them. In fact, communication supports were accessed at even lower rates than internal administrative supports, ranging from as low as 17% of researchers accessing media training to 32% accessing training to use technologies for research dissemination. Few researchers in either catego- CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 9 ry reported other supports, which hopefully means that we captured many of the supports that are available, but the few additional items listed by researchers included lending of equipment for KMb activities (i.e., to make multimedia dissemination products) and staff who wrote press releases about grants and research projects. Figure 2. Types of administrative supports available at universities and proportion of researchers who report accessing these resources. Figure 3. Types of communication supports available at universities and proportion of researchers who report accessing these resources. CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 10 These data suggest two potential barriers to knowledge mobilization within Canadian faculties of education: (a) a general lack of availability of institutional resources geared towards building researchers’ capacity to share their research findings with diverse audiences, and (b) apparent low levels of use (access) on the part of researchers in circumstances where these resources are available. Low levels of institutional supports overall, combined with even lower levels of resource use (particularly for communication supports), likely constrain researchers’ capacity to find opportunities for and carry out KMb activities. Producing high-quality research is time intensive, and expecting researchers to divert efforts from research production to engaging with non-academic target audiences is potentially an unrealistic goal. We will discuss this issue further in the discussion section where we argue for the active use of research brokering organizations and networks to facilitate KMb efforts between academic researchers and the communities that could potentially benefit from their work. Academic Versus Non-Academic Outreach We explored academic versus non-academic outreach in relation to products, events, and networks, which are categories that have arisen from previous studies (Cooper & Levin, 2010). Researchers were asked to identify the number of publications produced from their studies classified into different types based on whether the intended audience was primarily academic or non-academic (Table 2). When considered in aggregate, researchers reported a ratio of academic to non-academic publications at a rate of roughly two to one. Productivity rates varied greatly from one type of publication to another, and distributions for each type indicate that, with the exception of books, academic publications are strongly positively skewed, whereas non-academic publications are negatively skewed. The expected length of a book-length manuscript may be a contributing factor to the lower number of book publications in comparison to other types of academic publications. Alternative non-academic publications identified by researchers included brochures, school newsletters, handbooks, PowerPoint presentations for stakeholders, pedagogical guides, videos, professional journal articles, photography exhibits, performances (such as plays), CDs, databases, websites, and workbooks for workshops with teachers, students, and community members. Perhaps not surprisingly, respondents reported many more academic publications overall (M = 7.78, SD = 3.49) than non-academic publications (M = 3.51, SD = 2.82) when the data for each type of publication were aggregated.2 Academic publication was just over double non-academic outputs with aggregates being 909 academic products versus 395 non-academic products reported by researchers. We were also interested in the relationship between the number of academic and nonacademic publications a researcher produced. We wondered whether high academic producers would be low non-academic producers, and conversely, if researchers producing a high number of non-academic publications would produce a lower number of academic publications. We were interested in this trend because, in our anecdotal discussion with researchers during training and workshops where we offer to build KMb skills, researchers often say that they cannot focus on non-academic publications due to the amount of time it takes to produce academic publications. We wanted to test the hypothesis that there is no relationship between the frequency of academic publications and the frequency of non-academic publications. Results from Pearson’s chi-squared analyses reject the CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 11 null hypothesis, indicating that there is a statistically significant association between the frequency of academic publications and the frequency of non-academic publications, χ2 (1, N = 74) = 17.839, p = .000. Cramer’s V, the effect size statistic used in this analysis, was moderate at 0.49. Individuals who are producing a higher number of academic publications also appear to be those who produce a higher number of non-academic publications; conversely, individuals who have the fewest number of academic publications also appear to be those with the least non-academic publications. We found this finding surprising due to our original hypothesis that production across researchers would disaggregate into two groups—high academic producers versus high non-academic producers. Table 2. Number of Academic Versus Non-Academic Publications Reported by Researchers Number of Publications by Type (%) 0 1 2 3 4+ Mean Academic Publications Refereed articles Book chapters Books Other (e.g., reports) Non-academic Publications Alternative forms Plain language summaries Recommendations SD 3.4 2.6 0.8 2.5 0.9 1.4 1.0 1.3 0 11 42 7 5 12 48 19 17 22 4 24 16 22 0 19 62 34 6 31 1.4 1.4 1.1 1.5 1.3 1.2 42 28 45 12 30 22 21 22 23 11 12 6 14 9 5 Researchers’ participation in events related to their studies was measured on two dimensions: whether participation was invited or initiated by the researcher, and whether the event was primarily academic, such as an academic conference, or focused on other non-academic audiences, such as a professional development session or a public talk (Table 3). Like products, participation in academic events was greater than in non-academic events. Data distributions indicate that participation in academic events (regardless if by invitation or initiation) is again strongly positively skewed; however, unlike the frequency distributions of non-academic publications, participation was much more evenly distributed across the two groups of non-academic events (invited/initiated). Researchers more often initiated participation in academic events than in non-academic events. Similarly, researchers were also more often invited to participate in academic events than in nonacademic events. Again, these findings suggest a prioritization of academic work over connecting with non-academic audiences. However, many researchers reported participating in four or more of each type of event. Pearson’s chi-squared tests were conducted to examine whether participation in one type of academic event was associated with participation in a particular non-academic event. For this analysis, contingency tables were created by grouping the frequencies of participation in invited or initiated academic and non-academic events into two groups for each type: (a) participation in three or fewer events, and (b) participation in four or CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 12 more events. Pearson’s chi-squared test results reveal that there is a relationship between participation in invited academic and non-academic events, χ2 (1, N = 66) = 12.069, p = .001. The effect size for this finding, Cramer’s V, was moderate at 0.43. No other statistically significant relationships were identified among the other types of events. Table 3. Frequency of Participation in Academic and Non-Academic Events Number of Events by Type (%) Mean SD 0 1 2 3 4+ Initiated events 3.4 1.2 7 3 8 9 74 Invited events 2.9 1.3 6 10 24 10 52 Initiated events 1.8 1.5 31 13 20 20 16 Invited events 2.1 1.6 24 13 21 11 30 Academic Events Non-Academic Events Networks were defined as sustained, collaborative interaction in an effort to facilitate learning. Researchers reported interaction with three types of networks identified by the survey: existing networks in which they were previously involved (35%), educational organizations with established networks (21%), and new networks created as part of their dissemination efforts (12%). Researchers listed many examples of existing networks and educational organizations that they utilized to disseminate research, including school districts, professional associations such as the Canadian Teachers’ Federation, provincespecific groups such as the Early Years Education Ontario Network, academic associations such as the Canadian Society for the Study of Education, as well as a variety of nongovernmental organizations. Networks were underutilized; only a third of the researchers were even using networks they were already a part of. Allocation of Time Table 4. Proportion of Time Spent on Specified KMb Activities (%) KMb Activity Conducting research Mean 47.0 SD 16.6 Min 15 Academic publications 26.5 14.4 5 100 20–30 Non-academic publications 8.2 7.6 0 50 5–10 Academic events 11.8 12.2 2 100 5–15 Non-academic events 7.1 7.4 0 50 4.25–10 Academic networks 3.9 3.5 0 10 0–5 Non-academic networks 3.8 5.0 0 30 0–5 CJHE / RCES Volume 48, No. 1, 2018 Max 90 IQR 35–60 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 13 Respondents were asked to indicate the proportion of time they spent on various KMb activities for this research project on a scale from 0 to 100% (Table 4). Overall, researchers reported spending the majority of their time on conducting research (interquartile range between 35% and 60% of their time, but on average 47% of their time) with very little time spent focusing on non-academic activities (ranging on average from 4% to 8% of their time), a result that is consistent with the academic and non-academic outreach findings. It is interesting that both academic and non-academic networking (collaborating and interacting with others) received very little of researchers’ time (less than 4%). Another 26% of researchers’ time, on average, is devoted to producing academic publications, and 12% of their time is spent attending academic events. These data again suggest that academic mobilization work was prioritized over non-academic mobilization work. Dissemination Mechanisms We asked researchers about the dissemination mechanisms they used in three categories: online strategies, communication through the media, and work with intermediary organizations. Researchers were asked to identify the types of online strategies they used to share their research findings. Researchers reported low use of online strategies including websites (43%), listservs (15%), social networking tools (2%), and blogs (3%). These low levels of social media use could reflect the fact that we asked researchers about studies beginning between 2000 and 2005, when social media was just starting to take prominence (Facebook was founded in 2004 and Twitter in 2006). Websites, however, were also underutilized. Just under half of the researchers (47%) reported coverage of their SSHRC project in the media. These researchers were then asked a branching question about how contact with the media took place with the direction to select all items that applied from the list. In most cases, researchers indicated that the media picked up the story on their own (62%), or that faculty or university communication departments were active in contacting the media to initiate coverage (53%), while only 15% of researchers reported actively seeking out the media by contacting the media themselves. Researchers also reported sharing their findings in local and national newspapers, radio and television broadcasts, institutional media (e.g., faculty newsletters), or popular magazines (e.g., Today’s Parent, Psychotherapy Networker). The survey also asked about researchers’ engagement with intermediary organizations for the purpose of disseminating research. One in four researchers (25%) reported interacting with intermediaries as a way to increase the impact and dissemination of their work with non-academic audiences. Researchers reported that intermediaries disseminated research through their networks; organized events and created products based on their research; provided professional development based on their research; facilitated interaction with a user group; partnered in research; and disseminated their research to the media. Researchers were asked to report on their comfort levels carrying out a variety of KMb activities that focus on non-academic audiences (Figure 4). Respondents felt the most prepared to create plain language summaries and to collaborate with stakeholders, and they felt the least prepared to use technology to disseminate research, find and work with intermediaries, and interact with media and reporters. CJHE / RCES Volume 48, No. 1, 2018 14 Percentage of Respondents (%) Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read Figure 4. Researchers’ comfort levels with non-academic outreach activities. Discussion Taken as a whole, this study suggests that levels of KMb in Canadian education research with non-academic audiences are modest. These conclusions are consistent with research from the health sector as well (Hemsley-Brown, 2004). Although researchers recognize the importance of their work to external audiences, they invest relatively little time and effort in building connections with those audiences. This is arguably due to the time-intensive nature of producing high quality empirical research. These results seem problematic from several standpoints. First, researchers themselves reported that their work has implications that presumably are not being exploited. Second, governments and research funders are increasingly concerned about the impact of research, so more efforts to mobilize research knowledge would seem to be important to maintain public support for research. And most importantly, the education of young people in Canada could presumably be improved if there were strong and more consistent connections of high-quality research to policy and practice (Cooper, Levin, & Campbell, 2009). For all these reasons, it is important that we continue to build KMb capacity between researchers and potential end users and communities that could benefit from this work. We organize our discussion in relation to three categories of recommendations to improve KMb: (a) researchers leveraging intermediaries to improve KMb efforts and impact, (b) the need for universities to create institutional KMb support structures for researchers across disciplines, and (c) the need for funders to take a leadership role in capacity-building efforts. Our conclusion includes lists of concrete suggestions for researchers, universities, and funders. Leveraging Intermediaries to Increase KMb Efforts and Impact Intermediaries are an underutilized, untapped opportunity that (a) has emerging support from empirical evidence (Fisher, 2012; Hargadon, 2002; Ward, House, & Hamer, 2009), and (b) might alleviate the tension between time-intensive research commitments alongside what needs to be time-intensive investments in KMb if these efforts are to be robust and actually have a chance at improving society or the lives of Canadian citizens. Rather than putting the onus on researchers to build new networks and learn all the skills CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 15 associated with good KMb (communication, marketing, and so on), we argue that researchers should merely be responsible for building relationships and consistent communication mechanisms with intermediaries who are better suited to actually translate the research in innovative ways and already have strong networks in communities and with policy makers and practitioners. These organizations often also have the credibility with target audiences to increase uptake more than a researcher might have—for instance, collaborating with a teachers’ union to communicate important findings or tools that could support the work of teachers: (a) they have access to all teachers in a given province, (b) they already have consistent communication channels, and (c) teachers know that their union has their best interests at heart and so credibility has already been established. The same is true for user groups in other sectors such as nursing unions in the medical field. Institutionally Embedded KMb Capacity Needed at University Level KMb now appears to be on the radar at universities across the globe, though our data suggest that universities still demonstrate modest institutional support for KMb efforts. Even where resources are known to be available, they are often not used, suggesting that providing support services in a passive way is an insufficient strategy to increase mobilization efforts, especially with non-academic communities. KMb efforts also need to be valued by universities in renewal, tenure, and promotion processes. We argue that dedicated KMb roles and units need to be created in universities across Canada. KMb faculty positions are arising in other jurisdictions (such as Australia) to meet these new demands (Sá et al., 2011); the KMb faculty member is responsible for working with other faculty members in their department to create and implement KMb efforts. Another example of Canadian universities building sustained capacity for KMb is the Research Impact Network (www.researchimpact.ca). Founded in 2006, it is a network of 11 universities that have invested in small KMb research units to support both researchers and communities. KMb units help researchers with the KMb portions of grants but, more importantly, also field requests from communities about needs in a particular area that researchers might be able to help with. It is through this two-way reciprocal engagement that deeper KMb relationships happen (Phipps & Shapson, 2009). It is only through diverse partnerships with trust and sustained interaction that the challenges facing our society will be improved through collective action and system alignment across priority areas. In our view, every university in Canada could have a KMb unit that supports researchers from across disciplines in their KMb efforts. Funders Need to Go Beyond Rhetorical Commitment to KMb and Take a Leadership Role Many explanations could illuminate the low levels of institutional supports and activities in relation to KMb. For example, researchers may feel pressure to chase new research funding and move on quickly from finished projects, rather than taking the time necessary to make an earnest effort at disseminating the research findings in a way that might be applicable in policy and practice contexts, which arguably requires building and sustaining partnerships across diverse organizations and stakeholder groups. Part of this is due to funding mechanisms that do not adequately fund and focus on the requisite time for dissemination CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 16 (Tetroe et al., 2008). KMb is in its infancy, so researchers and funding agencies overwhelmingly underfund this portion of research grants. Funders need to take a leadership role in (a) creating dedicated funding streams for KMb and network development (a growing area of SSHRC), and (b) coordinating national training for universities and researchers on KMb and research impact, as these areas are still not well understood in academic communities. Conclusion In our view, increasing the mobilization of research knowledge needs to be seen as requiring concrete actions on a number of fronts (Table 5)—providing resources, changing incentive systems, building relationships that would support networking, providing social media platforms, and so on. These structures need to be accompanied with a variety of other factors, such as providing training to develop the skills necessary to do this work, creating more time and funding for researchers to spend on KMb efforts, building ongoing partnerships with end users and third parties that would support both more research access and better mobilization of results, and integrating an acknowledgement of these efforts into institutional reward systems to cultivate an academic culture in which outreach to non-academic groups is valued and recognized. Many researchers are already making efforts to mobilize their research with both academic and non-academic audiences. It is encouraging that, in our study, researchers with the most academic output also had the most non-academic output. Consequently, we argue that engaging in non-academic KMb is not necessarily associated with less academic dissemination; in fact, those who report more of the former also report more of the latter. Our data suggest that researchers are highly committed to their work and to engaging with stakeholders, so the will is there on the part of the research community; however, that will has not yet been matched by the requisite capacity-building efforts and KMb infrastructure that will allow society to accelerate the integration of research to benefit Canadian communities. Table 5. Concrete recommendations for researchers, universities, and funders Level Researchers Recommendations • Add links for newest publications below email signature line • Keep website updated with new research findings (while this seems simple, other studies have shown academics rarely do this) • Work on parallel publishing: for every academic journal article, also target a practitioner and/or policy publication to increase spread to nonacademic audiences and communities • Create social media plan by semester and then automate it, creating tweets and links to research reports and KMb products in Excel at the beginning of the semester for the whole term; automate these tweets using software (Hootsuite, etc). CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read Level Leveraging Intermediaries Universities Funders 17 Recommendations • Create a list of intermediaries in your research area (including governmental organizations, non-governmental organizations and not-for-profits, and membership/network organizations) • Send short summaries of research reports and materials to these organizations with a request for your short summary and links to your work to be included in the e-bulletins to their members • Social media: identify individuals and/or organizations with large twitter networks and approach them to ask if they will commit to retweeting research reports, summaries, and links to your work to their network to amplify the spread of your research • Create dedicated KMb and impact roles (KMb personnel or units) • Offer training sessions for (a) KMb planning, (b) stakeholder engagement, and (c) methods to trace research impact • Coordinate face-to-face events for communication departments and researchers in order to increase linkages and relationships • Create small, short-term KMb grants where research creation is not included—only focus on creating KMb efforts and disseminating these products • Develop strategic planning tools for universities that will guide development of KMb personnel and units (job descriptions, examples of budget costs for KMb units). Funders could even create competitive funding stream for KMb units at universities for three-to-five year periods • Create awards for universities to be recognized for their KMb support of faculty members and communities (currently, impact awards exist for researchers, but not for universities’ KMb services) • Circulate toolkits and training materials created for KMb planning and research impact to research service offices that are embedded in every university Notes 1. While we sampled educational researchers, institutional supports are often centralized across the university, not embedded in one particular faculty; so, while the data on academic and non-academic production relate to educational researchers, data on institutional supports often apply more broadly across disciplines since these services are usually centralized at the university and offered across faculties (for example, KMb units in the research impact network, office of research services, communication departments, and so on). 2. To determine a minimum average number of publications, the total number of publications for each publication type was aggregated to create a new variable. However, the means may underestimate the true average number of publications as the final CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 18 category in the survey scale was “4+” and the means reported here were calculated using a value of 4 for these responses. Thus, if the researcher produced more than four publications, the reported mean would increase. Nonetheless, we can be confident that the means reported here are the minimum average number of publications given the type of data collected. Acknowledgements This study was supported by funding from the Social Sciences and Humanities Research Council of Canada (SSHRC) and completed by the former Research Supporting Practice in Education Research Program (Principal Investigator: Dr. Ben Levin) at the Ontario Institute for Studies in Education (OISE), University of Toronto. References Barrington Research Group Inc. (2004). Community-University Research Alliances (CURA) Program: Analysis of data contained within the milestone and year 1 reports (Prepared for SSHRC–Social Sciences and Humanities Research Council). Calgary, AB: Barrington Research Group. Retrieved from http://www.sshrc-crsh.gc.ca/about-au_ sujet/publications/CURA_milestone_e.pdf Bastow, S., & Tankler, J. (2014). The impact of the social sciences: How academics and their research make a difference. Los Angeles, CA: Sage Publications. Bolman, L. G., & Terrence, E. D. (2008). Reframing organizations: Artistry, choice, and leadership (4th ed.). San Francisco, CA: Jossey-Bass. Cooper, A. (2014). Knowledge mobilisation in education across Canada: A cross-case analysis of 44 research brokering organisations. Evidence & Policy: A Journal of Research, Debate and Practice, 10(1), 29–59. http://doi.org/10.1332/174426413X662806 Cooper, A., Levin, B., & Campbell, C. (2009). The growing (but still limited) importance of evidence in education policy and practice. Journal of Educational Change, 10(2–3), 159–171. Cooper, A., & Levin, B. (2010). Some Canadian contributions to understanding knowledge mobilization. Evidence and Policy, 6(3), 351–369. Cordingley, P. (2008). Research and evidence-informed practice: Focusing on practice and practitioners. Cambridge Journal of Education, 38(1), 37–52. http://doi. org/10.1080/03057640801889964 Davies, H., Nutley, S., & Walter, I. (2005). Approaches to assessing the non-academic impact of social science research. In Report of the ESRC symposium on assessing the non-academic impact of research 12th/13th May 2005 (pp. 1–33). London, England: Research Unit for Research Utilisation. Retrieved from http://www.ruru.ac.uk/pdf/ ESRC Impact Symposium Final Report 040705.pdf Eisenhart, M. A., & Howe, K. R. (1992). Validity in educational research. In M. D. LeCompte, W. L. Millroy, & J. Preissle (Eds.), The handbook of qualitative research in education (pp. 642–680). San Diego, CA: Academic Press. CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 19 Fisher, C. (2012). Knowledge brokering and intermediary concepts: Analysis of an e-discussion on the Knowledge Brokers’ Forum. Knowledge Brokers’ Forum. Brighton, England: Institute of Development Studies. Retrieved from http://www. knowledgebrokersforum.org/blogs/item/knowledge-brokering-and-intermediaryconcepts-e-discussion-analysis Grant, J., Brutscher, P. B., Kirk, S. E., Butler, L., & Wooding, S. (2005). Capturing research impacts: A review of international practice. Santa Monica, CA: RAND Corporation. Retrieved from http://eric.ed.gov/?id=ED508108 Hargadon, A. B. (2002). Brokering knowledge: Linking learning and innovation. Research in Organizational Behavior, 24, 41–85. http://doi.org/10.1016/S01913085(02)24003-4 Hargreaves, D. H. (1999). Revitalising educational research: Lessons from the past and proposals for the future. Cambridge Journal of Education, 29(2), 239–249. http:// doi.org/10.1080/0305764990290207 Hemsley-Brown, J. (2004). Facilitating research utilisation: A cross-sector review of research evidence. International Journal of Public Sector Management, 17(6), 534–552. http://doi.org/10.1108/09513550410554805 Hicks, D. (2012). Performance-based university research funding systems. Research Policy, 41(2), 251–261. http://doi.org/10.1016/j.respol.2011.09.007 King’s College London and Digital Science. (2015). The nature, scale and beneficiaries of research impact: An initial analysis of Research Excellence Framework (REF) 2014 impact case studies. Bristol, England: Higher Education Funding Council for England. Retrieved from http://www.kcl.ac.uk/sspp/policy-institute/publications/Analysis-ofREF-impact.pdf Kishchuk, N. (2003). Performance report: Phase 1 of the Community-University Research Alliances ( CURA ) Program. Kirkland, QC: Social Sciences and Humanities Research Council. Retrieved from http://www.sshrc-crsh.gc.ca/about-au_sujet/ publications/cura_e.pdf Kittleson, M. J. (1995). An assessment of the response rate via the postal service and e-mail. Health Values: The Journal of Health Behaviour, Education and Promotion, 19(2), 27–39. Retrieved from https://www.researchgate.net/publication/232488537_ An_Assessment_of_the_Response_Rate_via_the_Postal_Service_and_E-mail McGrail, M. R., Rickard, C. M., & Jones, R. (2006). Publish or perish: A systematic review of interventions to increase academic publication rates. Higher Education Research & Development, 25(1), 19–35. http://doi.org/10.1080/07294360500453053 Meagher, L., Lyall, C., & Nutley, S. (2008). Flows of knowledge, expertise and influence: A method for assessing policy and practice impacts from social science research. Research Evaluation, 17(3), 163–173. http://doi.org/10.3152/095820208X331720 Mitton, C., Adair, C. E., Mckenzie, E., Patten, S. B., & Perry, B. W. (2007). Knowledge transfer and exchange: Review and synthesis of the literature. The Milbank Quarterly, 85(4), 729–768. http://doi.org/10.1111/j.1468-0009.2007.00506.x CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 20 Nutley, S. M., Walter, I., & Davies, H. T. O. (2007). Using evidence: How research can inform public services. Bristol, England: Policy Press. Oliver, K., Innvaer, S., Lorenc, T., Woodman, J., & Thomas, J. (2014). Barriers and facilitators of the use of evidence by policymakers: An updated systematic review. BMC Health Services Research, 14, 1–12. Retrieved from http://www.manchester.ac.uk/ escholar/uk-ac-man-scw:193505 Owens, R. E., & Valesky, T. C. (2015). Organizational behavior in education: Leadership and school reform (11th ed.). Boston, MA: Pearson Allyn and Bacon. Phipps, D. J., & Shapson, S. (2009). Knowledge mobilisation builds local research collaborations for social innovation. Evidence & Policy: A Journal of Research, Debate and Practice, 5(3), 211–227. http://doi.org/10.1332/174426409X463767 Sá, C., Li, S. X., & Faubert, B. (2011). Faculties of education and institutional strategies for knowledge mobilization: An exploratory study. Higher Education, 61, 501–512. http:// doi.org/10.1007/s10734-010-9344-4 Scott, W. R. (2004). Organizations: Rational, natural and open systems (4th ed.). Upper Saddle River, NJ: Prentice Hall. Smith, K. E. (2010). Academic treadmills and the squeeze on imaginative, intellectual spaces. The British Journal of Sociology, 61(1), 176–195. http://doi.org/10.1111/j.14684446.2009.01307.x Social Sciences and Humanities Research Council. (2014). Knowledge mobilization. Retrieved from http://www.sshrc-crsh.gc.ca/society-societe/community-communite/ index-eng.aspx#2 Tetroe, J. M., Graham, I. D., Foy, R., Robinson, N., Eccles, M. P., Wensing, M., … Grimshaw, J. M. (2008). Health research funding agencies’ support and promotion of knowledge translation: An international study. The Milbank Quarterly, 86(1), 125–155. http://doi.org/10.1111/j.1468-0009.2007.00515.x Ward, V. L., House, A. O., & Hamer, S. (2009). Knowledge brokering: Exploring the process of transferring knowledge into action. BMC Health Services Research, 9, 1–6. http://doi.org/10.1186/1472-6963-9-12 Willinsky, J. (2000). If only we knew: Increasing the public value of social science research. New York, NY: Routledge. Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S., … Johnson, B. (2015). The metric tide: Report of the independent review of the role of metrics in research assessment and management. London, England: Higher Education Funding Council of England. Wilson, P. M., Petticrew, M., Calnan, M. W., & Nazareth, I. (2010). Does dissemination extend beyond publication: A survey of a cross section of public funded research in the UK. Implementation Science, 5, 1–8. http://doi.org/10.1186/1748-5908-5-61 Wixted, B., & Beaudry, C. (2012). “Capturing the impacts” of research: Discussion paper. Ottawa, ON. Retrieved from http://publications.gc.ca/site/eng/445230/ publication.html CJHE / RCES Volume 48, No. 1, 2018 Knowledge Mobilization Practices of Researchers / A. Cooper, J. Rodway, & R. Read 21 Contact Information Amanda Cooper Faculty of Education Duncan McArthur Hall [email protected] Dr. Amanda Cooper, Assistant Professor in Educational Policy and Leadership at Queen’s University, is the founder of RIPPLE (Research Informing Policy, Practice and Leadership in Education): a program of research, training and knowledge mobilization (KMb) aimed at learning more about how knowledge brokering can increase research use and its impact in education by facilitating collaboration between multi-stakeholder networks (www.ripplenetwork.ca). Dr. Joelle Rodway is an assistant professor in Educational Leadership at the Memorial University of Newfoundland. Her primary research interests focus on the role of social capital in educational change in K-12 contexts. Specifically, she examines the ways educators’ social networks mediate the ways in which they develop, understand, and implement educational policy and how these processes affect school improvement outcomes. Dr. Robyn Read is a Post-Doctoral Associate in comparative, international, and development education at the University of Western Ontario. Her work focuses on the intersections between knowledge mobilization, education finance, education policy and the international development of education. CJHE / RCES Volume 48, No. 1, 2018