The Canadian Journal of Higher Education, Vol. XVII-2, 1987 La revue canadienne d'enseignement supérieur, Vol. XVII-2, 1987 Canada Council's Killam Research Fellowships: Distribution of Awards 1968-84 JOHN K. CHADWICK-JONES, A. CARCELLER and DOUGLAS VAISEY* ABSTRACT An analysis of the distribution of Killam Research Fellowships awarded by Canada Council over a seventeen-year period reveals a pattern which is examined in relation to the distribution of applications from universities. The existing situation is discussed with reference to previous studies of peer review systems and to implications for Canadian research. Encouragement of increased participation across Canada is suggested as one possible concern for Canada Council policy. RÉSUMÉ Les résultats d'une analyse de la répartition des bourses de chercheur-boursier de la fondation Killam, allouées par le Conseil des Arts du Canada sur une période de dix-sept années, et la répartition des demandes de bourses provenant de différentes universités ont fait l'objet d'une étude comparative. L'on discute de la situation actuellement par rapport à des études préalables sur les mécanismes d'évaluation par les pairs et leurs répercussions sur la recherche au Canada. On suggère que la politique du Conseil des Arts encourage, entre autres, une participation active et grandissante dans tout le Canada. Questions about priorities in the allocation of research funds, and the advantages of large university centres as against dispersion at many locations were discussed by Weinberg (1963, 1964, 1974) and Singer (1982). We wish to raise similar questions about the allocation of research funds in Canada Council's Killam Research Fellowships programme for university faculty (see "Killam Programme", Ottawa, Canada Council, 1986). More particularly, we ask the following questions: 1. What are the differentials in the research participation and success of Canadian universities? *John K. Chadwick-Jones, Saint M a r y ' s University, Halifax, A. Carceller, Mount Saint Vincent University, Halifax and Douglas Vaisey, Saint M a r y ' s University, Halifax. 28 John K . C h a d w i c k - J o n e s , A . Carceller and D o u g l a s Vaisey 2. Are the stated objectives of the funding agency being attained most effectively? If there is a moral order of research fund allocation, then it seems clear that this should be based on a well-defined reviewing process for individual applications leading to selection of the most meritorious. This selection process should involve assessment by experts who consider each applicant's research proposal together with referees' comments. The procedure should follow systematic rules so that decisions are taken as rationally as possible. There are probably difficult choices to be made in this (Killam) competition with a large number of applicants and a relatively small number of awards. The funding agency must have some notions of fair process as guides for its judging committees as they proceed to the point of allocating awards to individuals by vote or consensus. Probably the process itself is complex, because there may be a number of dimensions on which each proposal is to be judged (e.g. applicant's capacities, realistic plan, appropriate budget, sufficient detail) and choices have to be made between disciplines and sub-disciplines, or between pure and applied orientations. At the moment we do not have information from Canada Council on how these choices are made. Any system of funding usually and probably inevitably will produce unfair negatives for some applicants - if only because no system can operate perfectly according to rational-decision rules (Bauer, 1968). However, one aim of examining how a given system of funding actually operates, is to discover whether the likelihood of inequitable losing is influenced by chance factors or by some biassing factors which might disadvantage some applicants. In the Killam fellowships programme, certain universities provide many of the successful applicants, but this may indicate only that these applicants are of better quality. On the one hand, if the limitation of awards to a few universities were so extreme as to suggest a perfect correlation of talent with university affiliation, then the existing policies might be questioned, legitimately. On the other hand, a university probably needs some concentration of resources and of faculty members (a notion akin to "critical mass") before it can generate applicants to win Killam fellowships. In this kind of situation, the distribution of awards, even if it seems skewed in favour of some universities, may be the result not so much of decisions taken by the selection committees as an unavoidable outcome of the current size and organization of Canadian universities. In the administration of any awards system, there must be accountability. This is a main issue of our analysis and accountability may require re-appraisals with, eventually, modifications of existing procedures. However, no criticism of the selection procedures that are currently in use is implied in the analysis that follows. The Killam Programme The Killam Research Fellowships programme of the Canada Council is, according to its brochure for applicants, "intended to support scholars of exceptional ability engaged in research projects of broad significance and wide-spread interest" 29 Canada Council's Killam Research Fellowships: Distribution of Awards 1 9 6 8 - 8 4 TABLE 1 KILLAM FELLOWSHIP ( F I R S T YEAR) AWARDS TOTAL AWARDS GRANTED ( 1 8 8 ) No. Universities Toronto British of 1968-1984 fellowships received 55 Columbia 21 McGill 16 Queen's 13 York 13 Montreal 11 Laval 9 McMaster 7 Ottawa 5 Manitoba Simon 4 Fraser 4 Western Ontario 4 Private Scholars 2 14 O t h e r U n i v e r s i t i e s R e c e i v i n g 1 - 3 Awards TOTAL 24 188 through replacement of salary, enabling periods of leave to be taken, usually of one or two years. The Killam awards were made possible by a bequest of Mrs. Dorothy J. Killam. Between 12-20 fellowships are given per year, each renewable for a second year: the total annual expenditure is more than two million dollars currently. In Dorothy Killam's own words: "my purpose in establishing the Killam trust is to help in the building of Canada's future by encouraging advanced studies. Thereby, I hope to increase the scientific and scholastic attainment of Canadians, to develop and to expand the work of Canadian universities..." Nothing is said in the brochure about the distribution of awards across universities and provinces of Canada. In fact, as will be shown, some universities receive a high proportion of the awards. The pattern of the Killam awards shows, in effect, a 3-tier structure of the Canadian universities: the first tier consisting of a few universities with a high proportion of successful applications (possibly those with the highest concentration of talent). The University of Toronto has received about one-third of all awards - 29 percent over the total period of 17 years (Table I) and 30 percent of awards made in the last 5 years of the analysis (Table II). The University of British Columbia and McGill University (Quebec) each received around 10 percent over the total period and between 13 and 17 percent for the last 5 years. Queen's and 30 John K. Chadwick-Jones, A. Carceller and Douglas Vaisey TABLE II KILLAM FELLOWSHIP ( F I R S T YEAR) AWARDS TOTAL AWARDS GRANDED (57) No. of Universities Toronto British 1980-1984 fellowships received 17 Columbia 9 McGill 7 Queen's 5 Ottawa 4 York 3 Manitoba 3 Western Ontario 2 Simon F r a s e r 2 Alberta 1 McMaster 1 Waterloo 1 Concordia 1 Sherbrooke _1_ TOTAL 57 York Universities in Ontario, also did relatively well, each with 7 percent of the total awards. Below these are a group of universities with modest success (14 of them with only 1 - 3 awards over the entire period (Table I). The third tier consists of universities with no awards. Altogether, the University of Toronto obtained 55 of the total 188 awards. If the University of Toronto share is added to that of the University of British Columbia, McGill and Queen's, then the summation comprises well over half of all awards (Tables I, II). Analysis of total funding confirms the general picture. The University of Toronto has received 3 5 , 3 5 , 3 2 , 3 1 and 24 percent of the total funds over the five years. During the same period the nearest runner-up was the University of British Columbia with 15 percent and 14 percent of the total funds in 1981 and 1982 (Table III). It might be uncharitable to assume that the universities whose faculty numbers have not achieved Killam fellowships have little or no research output. In a Canada Council report of 1977 the authors cite a survey of small universities where "most faculty members interviewed felt that research forms an essential part of any university department" but the survey also noted that "research" was interpreted in these institutions in a broader sense to the definition used in "the most prestigious academic circles". This broader definition included "applied research, literature 31 Canada Council's Killam Research Fellowships: Distribution of Awards 1 9 6 8 - 8 4 TABLE III FINANCIAL DISTRIBUTION OF AWARDS: LARGEST ALLOCATIONS 1980-84 PERCENTAGES OF TOTAL FUNDS ALLOCATED IN KILLAM PROGRAMME 1980 University Toronto of U n i v e r s i t y of B r i t i s h Columbia Queen's University 1981 1982 1983 1984 35 35 32 31 24 4 15 14 11 12 11 11 7 6 15 reviews, course development and small-scale experimentation". Also, there is no doubt that many universities produce few applicants for the Killam programme. Information on applications has been made available to us (back to 1977) by the Canada Council (Table IV). Altogether, over the period 1980-84, applications were received from faculty members in 36 universities. The University of Toronto provided, consistently, thirty or more applications per year, submitting 22 percent of the total (172 out of 787) during the five years, 1980-84. The relatively low applications rate of a large number of "also ran" Canadian universities, even the larger ones, in this competition contrasts with the high participation by the University of Toronto, well ahead of the University of British Columbia (63 applicants) and McGill University (44 applicants). One interesting feature of the period 1980-84 is the absence of awards to the francophone universities of Quebec although these universities did gain awards during the first decade of the analysis (Table I). Possibly, able individuals in this province may have turned to increased research funding from provincial sources for Francophone scholars in Quebec but this must remain conjectural given the limits of our current analysis. The first-time fellowships are in most cases renewed for a further year. However, some have been made for more than two years. For example, one individual at McGill University had his fellowship renewed for four consecutive years 1970-74 and received a new fellowship in 1976, renewed in 1977. Another McGill University scholar's fellowship of 1973-74 was renewed for three further years from 1974 to 1977. A Queen's University recipient received fellowships over the five years (1968-72) and a University of Toronto recipient's fellowship lasted for four years (1975-79). These multiple awards serve to emphasize the allocations of available funds to a few universities even more than has appeared from the distribution of the initial awards. The selection committee of 12-13 members is the principal instrument for awards: most are academics, with 1 - 3 ex-officio members (one is secretary of the Killam programme). Since the beginning of the competition in 1968, 27 32 John K. Chadwick-Jones, A. Carceller and Douglas Vaisey TABLE IV APPLICATIONS RECEIVED: Year KILLAM RESEARCH FELLOWSHIPS N o . of u n i v e r s i t i e s from which a p p l i c a t i o n s were r e c e i v e d Total applicants 1980 27 138 1981 27 126 1982 30 139 1983 30 167 1984 28 217 universities have been represented on this committee, a wide spread of representation. The University of Toronto is the only one that has been represented on the committee in each of the 17 years (1968-84); for 7 years it had 2 representatives and in 3 years it had 3 representatives (1976-77, 1981-82, 1982-83). In the last decade McGill University has had representatives during each year, with 2 representatives during 6 of those years. That members of the selection committee, reviewers and applicants may be personally known to each other does not constitute evidence of bias. On the contrary, such personal contacts are often seen as positive in assisting the decision process and, in any case, are observed in a variety of research funding contexts (Hall, 1986). There is still the question of whether the existing distribution of awards achieves the specific aim of the Killam trust "to develop and expand the work of Canadian universities". Or is it the case that the resources of the Killam programme, available by open competition to the wider academic community across Canada, actually support the research activities of a minority of universities rather than encouraging the general development of research skills in Canada? In the Killam fellowship programme, the Prairies and Atlantic provinces receive few awards. For example, from 1980 to 1984 four Prairie universities (the Universities of Alberta, Calgary, Manitoba, Saskatchewan) made 77 applications and received four initial awards. Eight of the Atlantic universities (Dalhousie, Memorial University of Newfoundland, Mount Allison, St. Francis Xavier, Saint Mary's, Mount Saint Vincent, Université de Moncton, University of New Brunswick) submitted a total of 37 applications; they received no awards. Queen's University in the same period submitted 44 applications and received 5 awards. Further light might be thrown on this question in the future, if Canada Council were to consider carrying out an enquiry into the quality of applications from the Atlantic provinces. We can examine the participation of these eight Atlantic Canada Council's Killam Research Fellowships: 33 Distribution of Awards 1 9 6 8 - 8 4 TABLE V KILLAM FELLOWSHIP ( F I R S T YEAR) AWARDS: UNIVERSITY COMPARISONS 1980-1984 Universities Toronto No o f Applications N o . of S u c c e s s f u l Applications P e r c e n t a g e of Successful Applications Queen's Prairies 172 44 77 11.4 (4) Atlantic (8) 5.2 37 17 9.8 universities. If we take the University of Toronto as our model (using data from Table V) then let us suppose that each university in the Atlantic provinces should provide annual applications at an equivalent rate (assuming, that is, their faculty members are research-oriented to a similar extent). Based on the University of Toronto participation (Table V), this yields us a formula for the expected participation of: 172/5 2,373 (No. of applicants over 5 years) (Total faculty members U. of Toronto)* No. of full-time faculty _ Expected no. of applications By this formula, a small university with, say, 275 faculty would need to supply 4 applications per year to have an equal rate of participation. For the Atlantic universities in the five years 1980-84, 37 applications have been submitted from eight universities with a total faculty of approximately 3,107 members. Applying the formula, the resulting figure of expected applications would be: 172/5 3,107 - X — ; — = 45 per year 2,373 1 The actual participation rate is, thus, very much lower than the expected level. In fact the total applications made in the five years are less than the suggested rate, using the above formula, for only one year's applications. •Source of figures for 1983-84: Statistics Canada. Salaries and Salary Scales of Full-Time Teaching Staff at Canadian Universities ( C S 8 1 - 2 5 8 ) Ottawa: Statistics Canada. Figures include medical and dental schools. 34 John K . C h a d w i c k - J o n e s , A . Carceller and D o u g l a s V a i s e y Discussion The success of members of major universities in the Killam competition merits comparison with similar tendencies documented in discussions of peer review systems for scientists in the United States. The literature on peer review has concentrated on the sciences; we do not have equivalent information on the humanities and social sciences, so the relevance of this literature for the Killam awards allocations is only partial. Cole and Cole (1967, 1968) considered the hypothetical situation of an individual who produces high quality work but is a member of a low prestige department; they identified university affiliation as an important influence on success. The Coles found, in their study of 120 physicists, that there was a high correlation between scientific recognition and prestige rank of department. They suggested also that published work by members of major (as contrasted with minor) universities gains most attention. However, Mitroff and Chubin (1972) discussed two studies of peer review in the National Science Foundation and in these studies the results were rather less decidedly in favour of the effect of institutional prestige. More recently, Bélanger and Lacroix (1986) have discussed the likelihood of institutional status effects on bias in the wider context of the federal grants agencies of Canada and they have discounted institutional influence, pointing to the importance of individual productivity: "the correlation between publication productivity and grant recipientship is very high and credible" (p. 27). Thus, there is no evidence of institutional bias in the incomparably larger scene of federal funding, where the individual productivity of applicants appears to be the decisive influence. On the other hand, in the United States, Crane (1965) had marshalled evidence showing that scientists at major universities were more likely to be productive and to receive recognition than those located at minor universities. The more prestigious institutions were considered to provide an encouraging ambiance for research, including advantages gained from the presence of successful senior academics, who could provide "an adequate model for the optimum way to proceed". In addition Crane noted that the effect of location "may actually be due to contacts it facilitates between junior men and eminent scientists who play important roles in allocating scientific rewards". This kind of contact was thought also to influence the selection of good research topics by applicants at major universities and to increase a researcher's "visibility" in his/her profession. According to a Canada Council report of 1977, the faculty members of smaller universities suffer isolation because of lack of departmental colleagues in the same field; because of heavy teaching loads and lack of opportunities to make academic contacts at other universities. Research productivity, according to Crane (1965), "did not make the scientist as visible to his colleagues as did a position at a major university". Scientific recognition was the subject of several conjectures by Crane, for example: 1. recognition may be relative to achievement, 2. location at a major university creates a "halo" effect so that a scientist's work looks better than it actually is, 3. contact with eminent colleagues enhances visibility and affects the Canada Council's Killam Research Fellowships: 35 Distribution of Awards 1 9 6 8 - 8 4 chances for recognition. Crane's discussion concluded with speculations about possible future improvements in standards outside the group of elite institutions and made the prediction of an expansion of research activity beyond the major institutions. The study by Zuckerman and Merton (1971) of peer review systems for selection of articles for scientific journals also has some interesting comments about successful performance. Scientists at major universities may be more successful, they suggest, "because their critical associates (university colleagues) and they themselves have demanding internal standards, then the manuscripts they decide to submit for publication are apt to be rigorously pre-selected, with consequently high rates of acceptance by referees applying similar ... criteria". It could also be, as was mentioned by Crane (1965), that scientists who have the advantage of the criticism and help of such colleagues will also tend to select more attractive research problems. To summarize, we have discussed the awards distribution of the Killam research fellowships and have shown evidence of a pattern which may, in fact, be the best outcome under the existing condition of the competition. The faculty members of some universities are continually successful in the Killam fellowships competition. It is also the case for other universities that, over long periods, relatively few applications are submitted. Could the stated objective of the Killam research fellowships programme "to develop and expand the work of Canadian universities" be attained more effectively by a wider distribution of awards? There is also a second question: could the participation of Canadian universities be increased? One situation, perhaps comparable, that comes to mind is the concentration of French scholarship and research at the University of the Sorbonne in Paris. Singer (1982) has discussed the options of funding institutions at the centre, as against spreading the research allocations across a number of locations. He ventures the opinion, in the case of French universities, that the encouragement of academic talents at provincial locations would increase research output on a greater scale "than concentration in one location". In our case, even if we were to regard this kind of opinion favourably, there is the problem of how to encourage, in the Killam competition, higher rates of participation across Canada. It might be a problem worthy of consideration in Canada Council policy. REFERENCES Bauer, R. A . "The Study of Policy Formation: An Introduction," in Bauer, R. A . and Gergen, K. J. (eds.). The Study of Policy Formation N e w York: The Free Press, 1968. Bélanger C. H . & Lacroix R. "Measuring the effectiveness of research grant getting," The Journal of Higher Education, Canadian 1986, X V I pp. 2 5 - 3 8 . Canada Council Report. "Needs of Scholars at Small Universities," Ottawa: Canada Council, 1977. Cole, S. & Cole, J. R. "Scientific Output and Recognition: A Study in the Operation of the Reward System in Science," American Sociological Review, 1967, XXXII, pp. 3 7 7 - 3 9 0 . Cole, S. & Cole, J. R. "Visibility and the Structural Bases of Awareness of Scientific Research," American Sociological Review, 1968, XXXIII, pp. 3 9 7 - 4 1 3 . 36 John K. Chadwick-Jones, A . Carceller and Douglas Vaisey Crane, D. "Scientists at M a j o r and Minor Universities: A Study of Productivity and Recognition," American Sociological Review, 1965, X X X , pp. 6 9 9 - 7 1 4 . Hall, N . " E n d of the peer s h o w ? " The Times Higher Education Supplement, 1986, 737, p. 11. Isaacs, C. F. "Avoiding Stagnation in Small University Departments," (unpublished) cited in Canada Council Report (1977) Needs of Scholars at Small Universities, Ottawa: Canada Council. Mitroff 1.1. & Chubin D . E . "Peer review of the NSF: a dialectical policy analysis", Social Studies of Science, 1972, 2, pp. 1 9 9 - 2 3 2 . Singer, B. " T h e Ascendancy of the Sorbonne: the Relations between Centre and Periphery in the Academic Order of the Third Republic," Minerva, 1982, X X , pp. 2 6 9 - 3 0 0 . Weinberg, A . M . "Criteria for Scientific C h o i c e , " Minerva, 1963,1, pp. 1 5 9 - 1 7 1 . Weinberg, A . M . "Criteria for Scientific Choice II: The two cultures," Minerva, 1964, II, pp. 3 - 1 4 . Weinberg, A . M . "Institutions and Strategies in the Planning of Research," Minerva, 1974, XII, pp. 3-17. Z u c k e r m a n , H . and Merton, R. K. "Patterns of Evaluation in Science: Institutionalization, Structure and Function of the Referee S y s t e m , " Minerva, 1971, I, pp. 6 6 - 1 0 0 .