Canadian Journal of Higher Education, Vol. IX-2, 1979 The OECD Review and Higher Education EDWARD SHEFFIELD,* HAROLD J. NOAH** and HILDEGARD HAMM-BRÜCHERt ABSTRACT Professor Sheffield tells how the OECD review of educational policies in Canada proceeded, who were involved, and what the examiners stressed in their report. He reports also on a tentative assessment of its impact, most of which is potential or not provable. Professor Noah describes the history and procedures of OECD's reviews of the national education policies of its member states - exercises in the comparative study of education, with practical benefits. Dr. Hamm-Briicher reflects in a highly personal way on her experience as a member of the examining team and her disappointment with the meagre response to the final report on the part of Canadian governments and academics. Nevertheless, she believes strongly in the kind of international co-operation represented by the OECD reviews. RÉSUMÉ Le professeur Sheffield explique a) comment l'OCED a passé en revue les politiques canadiennes de l'enseignement, b) qui a été engagé comme enquêteurs et c) où ces personnes ont mis l'accent dans leur rapport. Il donne également une évaluation provisoire d l'impact du rapport - impact potentiel pour lequel il n'existe aucune preuve. Le professeur Noah donne l'historique et les procédures des revues de l'OCED des politiques nationales d'enseignement de ses états-membres — une étude comparée de l'enseignment avec des avantages pratiques. Le docteur Hamm-Briicher donne de façon personnelle des réflexions sur ses expériences en tant que membre de l'équipe des enquêteurs. Elle indique aussi sa déception à l'égard du concours peu encourageant en vue du rapport final de la part des gouvernments au Canada et des pédagogues canadiens. Elle croit fortement, néanmoins, au genre de coopération internationale qu 'impliquent les revues de l'OCED. * Professor emeritus of higher education, University of Toronto. ** Dean, Teachers College, Columbia University, t Minister of State in the Federal Ministry for Foreign Affairs, Federal Republic of Germany. 2 Edward Sheffield, Harold J. Noah and Hildegard Hamm-Briicher CANADA EXAMINED Edward Sheffield Although Canada has been a member of the Organization for Economic Co-operation and Development (OECD) since it succeeded the Organization for European Economic Cooperation in 1960, and although member states are expected to undergo reviews of their national economic, science and education policies from time to time, Canada did not invite an OECD general review of its policies for education until 1973. (A limited examination of the training of and demand for high-level scientific and technical personnel was undertaken ten years earlier.) One reason for the delay was that the federal government could not speak for the provinces in the field of education and no way had been found for the provinces to speak for themselves. What was agreed to involved a nice choice of words, of the kind to which Canadians have been accustomed throughout the history of federalprovincial relations: a review not of national policy for education but of educational policies in Canada. The Council of Ministers of Education, Canada (CMEC) acted for the provinces and the Department of the Secretary of State (SOS) acted for the federal government in preparing a series of six background reports on education in Canada, serving as an internal review as well as providing information for the team of examiners chosen by OECD to undertake the external review. The series, published in 1975, was entitled Review of Educational Policies in Canada and included a Foreword and Introduction, an Atlantic Region Report and a Government of Canada Report.* Most of the work on these reports was done in government departments. On the whole, they represented official statements emphasizing description and the identification of current concerns rather than evaluation or criticism. The OECD examiners were: Professor Michel Crozier, director of the Institute for the Study of the Sociology of Organizations, Paris; Dr. Kjell Eide, director of planning and research in the Ministry of Education, Norway; Dr. Hildegard Hamm-Brucher, member of the legislature of Bavaria and former Secretary of State, FRG; Professor Harold J. Noah, dean of Teachers College, Columbia University, New York (rapporteur); and Professor Pierre Vanbergen, director-general of the organization of studies, Ministry of National Education and French Culture, Belgium. After studying the documentation provided by the Canadian authorities, the examiners spent the month of June 1975 visiting various parts of the country to discuss points raised by the background reports and to seek answers to questions they thought appropriate. The draft of the examiners' report was circulated to Canadian authorities in order that they might ready themselves for the "confrontation" with the OECD Education Committee. At this stage the draft report was released unofficially by the Canadian Association for Adult Education and the Students' Administrative Council of the University of Toronto, although normal procedure would have been for OECD to publish it, together with a synopsis of the original Canadian submission and a report of the discussion at the confrontation, somewhat later. The official report, Reviews of National Policies for Education: Canada, was published by OECD in 1976. The examiners' chief criticism was that Canada had no national policy, no national goals, for education. They observed that educational policy was pragmatic and curiously unpoli* All of the reports mentioned in this article were published in both French and English. 3 The OECD Review and Higher Education ticized. The greatest need for reform, they felt, was for improved mechanisms for interprovincial and federal-provincial co-operation. Granting the limits imposed by the constitution, they identified the Council of Ministers of Education as the body in the best position to provide leadership in "defining and co-ordinating provincial, federal and national interests and policy in higher education." To this end they urged that the CMEC "be developed into a national forum for the working out of educational policies, so that the Federal government may be involved in a systematic and open manner in discussions of educational policy that transcend Provincial boundaries." They noted that the federal government had covert but not overt roles in educational policy making, and that its efforts were unfocused, in spite of the co-ordinating function assigned to the Secretary of State. Referring specifically to higher education, the examiners were critical of the massive and expensive expansion of facilities during the sixties, the lack of definition of the role of universities, the unpersuasiveness of university demands for autonomy and public support, the unresponsiveness of the universities to their local communities, inadequate inter-university co-operation, the failure to improve access for underprivileged groups, and what they saw as an institutional pecking order rather than equality of institutional status. On the other hand, they were complimentary to the community colleges and noted with approval the achievements of the Maritime Provinces Higher Education Commission. At the confrontation, held at OECD's Paris headquarters in December 1975, there were present the members of the Organization's Education Committee, the five examiners, and the nineteen-man Canadian delegation: the Hon. Ben Hanuschak, Minister of Education and of Colleges and Universities Affairs, Manitoba (head of delegation); twelve other officials of provincial ministeries of education: Newfoundland — Mr. C. Roebothan;New Brunswick — the Hon. Gerald S. Merrithew and Messrs. J.P. McCluskey and G.EM. McLeod; Québec — Messrs. Maurice Mercier, Claude Marin and Pierre Fontaine; Ontario — Messrs. H.E. Gillies and W.E.P. Fleck; Saskatchewan — Mr. L.H. Bergstrom; Alberta — Dr. James S. Hrabi; British Columbia — Mr. J. Phillipson; and six officials of federal government departments: Secretary of State — Messrs. Peter Roberts, R. Lachapelle and M. Spalding; External Affairs — Mr. Graham Mitchell; Indian and Northern Affairs — Mr. R. Connelly; Manpower and Immigration — Mr. Bernard Dufresne. The record of that meeting indicates that the examiners were sharply critical and, for the most part, the Canadian spokesmen were defensive. Late in 1976 the CMEC had the excellent idea of inviting a score of the national associations concerned with education to prepare their comments on the Review and send them to the Council. Most responded and some of the points they made were referred to briefly in A Statement by Canadian Authorities for the OECD Appraisal of Country Educational Policy Reviews "prepared by the Council of Ministers of Education, Canada in collaboration with the Government of Canada, January 1978." The Canadian Association of University Teachers responded, the Association of Canadian Community Colleges did not, and the response of the Association of Canadian Universities and Colleges arrived after the Statement had been prepared. The Statement was made in response to an invitation from OECD to participate in an appraisal of country educational policy reviews based on experience of the previous 4 Edward Sheffield, Harold J. Noah and Hildegard Hamm-Briicher decade. It did offer such an appraisal, based on the Canadian exercise, 1973-75, and also attempted a tentative assessment of the impact of the review. Events were grouped in three categories: "those related to the review and external report, features of the report which evoked negative responses, and developments congruent with but not necessarily related to the report." The national associations were generally in favour of the examiners' thrust for national goals and a national forum, but the provincial authorities were not. Items mentioned in the third category (much the largest) indicated progress on many of the matters of concern which had been mentioned in the original, internal reports. On the whole, the Statement reflected primarily the satisfaction of the provincial authorities with existing structures and directions, and acknowledgement that improments within those limits were desirable. One might have expected the CMEC to do something more with the responses received from national education associations, if only by publishing them, or a synopsis of them, under one cover. Each responding organization distributed copies of its responses within its membership, but few people have seen them all or even several of them. A number of these organizations and some others arranged for discussion of the OECD Review at annual or special conferences. The Canadian Teachers' Federation held one such special conference in May 1978. Professor Harold J. Noah, who had been rapporteur for the OECD examing team, was an invited speaker, and it was on this occasion that he presented the paper "Reviewing the OECD Reviews of Educational Policy" which follows. Later in that month Dr. Hildegard Hamm-Briicher, who had been one of the examiners, shared her reflections on "Multilateral Co-operation in Education — Possibilities and Limitations" at the annual conference of the Canadian Society for the Study of Higher Education. Her paper follows Professor Noah's. (Both have been slightly condensed.) As is indicated above, the review has been given little credit for changes that are just beginning to be apparent, and it is hard to say that they would not have come about anyway. The examiners correctly pointed to the salient characteristic of the Canadian system — that it is not one system but many — and made it clear that the lack of central planning and policy making appeared to outsiders to be a grave deficiency. When negotiations leading to what became the Federal Provincial Fiscal Arrangements and Established Programs Act 1977 were in progress, the Prime Minister proposed "the establishment of a continuing federal-provincial forum at the ministerial level" (much as the OECD examiners had suggested), but with only partial success. The provinces were unwilling to act as if the federal government had any jurisdiction in the field. For some years, but more frequently since 1976, the CMEC has invited federal representatives to join in discussion of particular subjects of mutual interest, from time to time, and has opened lines of communication with national education associations. But it is cautious, constantly aware that it has no power of its own: only its members have power and then only at home. Nevertheless, the CMEC has been showing awareness of its potential, and must be encouraged to take Canada-wide initiatives. The national associations, too, must continue to play their national roles, and so must the Government of Canada — identifying Canadian interests, challenging the principal actors to serve those interests, and offerint its support. It is encouraging to note, therefore, that in an address in November 1978 the Secretary of State acknowledged the weight of the OECD examiners' criticisms and reaffirmed the Federal Government's concern for the national interest in education. 5 The OECD Review and Higher Education REVIEWING THE OECD REVIEWS OF EDUCATIONAL POLICY Harold J. Noah I propose to examine the OECD reviews of national policies for education under eight headings: Some background on OECD as an organization, and some history of the programme of educational policy reviews. The five elements of a review. Who does the reviewing? Who wants to get what from the reviews? Who and what gets examined? Preparing for the review. Results of the reviews. Prospects for the review programme. While I will do this with reference to all the reviews that have taken place so far, I'll be making particular reference to the German, Austrian and Canadian reviews, in which I had a direct role as examiner, and for two of which I acted as rapporteur. General and historical background One valued feature of comparative inquiry into educational phenomena is the opportunity to bring at least a fresh, and perhaps a more objective, perspective to bear on the educational affairs of another country. Recall that Marc-Antoine Jullien de Paris as far back as the second decade of the nineteenth century proposed that each European nation be required to supply answers to a lengthy annual questionnaire on its educational establishment. As the questions he suggested were many of them rather pointed, Jullien hoped that a proper sense of pride (and shame) would stimulate each nation to ever greater educational efforts. At least, the laggards would be exposed to international disapproval, and more progressively inclined nations could learn from the good example of others. Jullien's notion of nations undertaking systematic, regular, self-administered examinations of their educational systems was ignored for the whole of the nineteenth century, but has been partially carried forward in this century by the work of the International Bureau of Education, Geneva, and by UNESCO. Examinations of member countries by the OECD are another strand in the process of international examination of educational policies. The Organization for Economic Co-operation and Development is an international body with its headquarters in Paris. Its present membership of 25 countries includes the governments of Western Europe, Turkey and Yugoslavia, the United States and Canada, Japan, Australia and New Zealand. OECD had its origins in the consortium of countries that came together under the post-Second World War Marshall Plan. Originally named the OEEC (Organization for European Economic Co-operation), the name was changed to the non-regional appellation OECD in 1960. OECD's activities in industrial and agricultural policies, the commercial and monetary fields, education and science, and social policy are by now well developed. The OECD seeks to be a locus for intergovernmental discussion, collaboration and co-ordination. In these activities, OECD is acting in much the same way as the more familiar UN agencies, ILO, WHO, UNESCO and so on. However, the OECD is unlike these UN agencies with respect to one thing it does not do, and one thing it does do. The OECD does not seek to incorporate non-governmental institutions or representation into its work, and it does conduct regular reviews of the economic (and lately) of the educational policies of member countries. OECD economic policy reviews were established in the early 1950s. They are conducted by members of the OECD secretariat and attract considerable attention, as authoritative 6 Edward Sheffield, Harold J. Noah and Hildegard Hamm-Briicher judgments about the means used by countries to achieve their stated economic and financial goals and the degree of success in attaining them. OECD reviews of member countries' educational policy goals commenced somewhat later, but are already beginning to be seen as a series of considered evaluations of an important area of each government's social and administrative effort. While the OECD secretariat organizes and co-ordinates the educational policy reviews (in co-operation with officials of the host country), it does not undertake the review itself. This is done by a group of between three and six so-called examiners, appointed ad hoc by the OECD for a particular country review, with the approval of the government concerned. However, it would be fair to say that the educational policy reviews have not, as yet, attained the regularity, sustained achievement, and publicity achieved by the economic policy reviews, though, given time, this will perhaps develop. The review process grew out of the OECD Conference on Economic Growth and Investment, held in Washington, D.C. in 1961. One conclusion of that Conference was that there should be regular national surveys of educational policies in member countries. The emphasis of the Conference, and of the proposal for reviews, was upon the manpower implications of educational policy and at least the first review, that of Ireland, was conducted in that spirit. In fact, the series of reviews began by being piggybacked on a threeyear survey of education in Ireland, undertaken 1962-65, and published as an Irish government document, entitled Investment in Education. A team of OECD examiners visited Ireland for a week in June 1966, using this document as their background report. In October 1966 a so-called confrontation meeting was held in Paris at the OECD between the examiners and the Irish educational and political authorities, in the presence of the relevant OECD multi-government committee. The examiners' report and the proceedings of the confrontation meeting were then published by OECD. Thus, was the pattern established that was used, with minor modification only, for subsequent reviews: a background report prepared by the country; an on-site examination and report by external assessors; an agenda of questions for discussion at a confrontation meeting; and a series of publications to conclude the process of review. Sweden, Italy, Austria, France, Japan, the Netherlands, United States (educational research policy), Germany, England and Wales, Austria again (higher education), Norway, Canada, Australia, the Netherlands (again), and Austria (school policy) have since followed. Aspects of United States educational policy will be reviewed shortly (educational policies for the disadvantaged), and Yugoslavia, Sweden, and New Zealand are on the future Schedule. The five elements of a review As I have mentioned, a complete country review has at least five important documentary stages: 1. A report, prepared by the educational authorities of the country under review. This document is usually known as a "background document." I will have something more to say about it later. For the moment suffice it to say that the background document provides examiners with a description of the relevant parts of the country's educational system, a statement of policy goals as seen by the country's authorities, and some 7 The OECD Review and Higher Education reference at least to the major policy problems under discussion within the country. The background document is, in a sense, the examiners' primary source document for their review. 2. In a number of countries, and especially in Canada, a series of special research reports are prepared during the course of the writing of the background report. These are normally made available to the examiners, at their request, but are not intended for eventual publication as part of the OECD report. Other papers that are available to the examiners may come from special interest groups — particularly from teachers and school board associations, and from university groups. 3. After spending a period of time in the country, which may be as short as three to four days or as long as a month, the examiners produce their report on what they have seen and heard together with a list of questions which they wish to have discussed with the representatives of the country's educational authorities. 4. The examiners' report and the list of questions form the agenda for a session of the education Committee of the OECD in Paris. Representatives from all of the OECD member countries are present and one or two whole days of discussion are devoted to the so-called "confrontation meeting." During the course of discussion based on the examiners' questions, it is expected that representatives from the other member countries will intervene, and over the years such interventions have become more and more the rule. In that sense, the confrontation meeting in Paris becomes a type of multi-national seminar on the problems of educational policy experienced by all the member countries, but with particular reference to the issues raised by the experience of the country under examination. 5. In at most a year from the date of the confrontation meeting, there will appear from the OECD a publication which includes the background report, or a summary thereof, the full text of the examiners' report and the questions they have riased for discussion, plus a summary of the discussion held at the confrontation meeting. From the point of view of the OECD, this publication marks the end of the country review process. Of course, this is by no means the end of the matter for the country concerned. Most often, a vigorous public discussion ensues at home that often continues for a number of years. Who are the examiners? The team of examiners consists of a minimum of three and a maximum of six persons, accompanied by an official from the OECD secretariat. Many of the examiners are university professors, especially those who have had some hand in the discussion and setting of educational policy in their own countries. In addition, examiners are drawn from the ranks of ex-ministers or deputy ministers of education, members of legislative committees concerned with education, and administrators (that is, civil servants) who have dealt with education at one stage or another of their careers. Each team usually has an individual who has been on the receiving end of the examinatioi in his own country. In addition, because of the number of reviews that have now taken place, the examining teams tend to include a rather large proportion of examiners who have taken part in examinations before. As befits an international organization, the OECD attempts to achieve a good spread 8 Edward Sheffield, Harold J. Noah and Hildegard Hamm-Briicher of examiners in terms of nationality. In no circumstances, may an examiner be a national of the country under examination. In Canada, for example, the examining team consisted of a Frenchman, a Belgian, a German, a Norwegian, and an Englishman, together with the requisite official of the OECD. One of the examiners, usually the member with the most diplomatic experience, is designated chairman, to make the formal speeches of introduction, and so forth. There is also a rapporteur (usually, though not always, not the same person as the chairman), who is responsible for co-ordinating the writing of the examiners' report. In thé development of the work of the examiners, there is a great deal of collégial consultation, discussion, and group definition of outlines of the questions and conclusions. The schedule of interviews, and the itinerary within the country, usually provides for the examiners to stay together as a group for most of the time, so that there is a chance for an examiners' group consensus to emerge on important issues that must be dealt with in the examiners' report. However, a certain division of labour among the examiners also occurs, so that individual examiners tend to specialize in one or another aspect of education policy. During the course of the country visit, and for a day or two at the end of the visit, the examiners engage in intensive periods of discussion and writing, in order that by the time they disperse, the rapporteur will have in hand a clear outline of basic arguments to be made in the final draft of the examiners' report. In the intervening period between the end of the examiners' visit and the confrontation meeting in Paris, a three-way process of discussion of the draft examiners' report ensues among the examiners, the country authorities and the OECD secretariat. The goal now is to ensure that the final draft is free of errors of fact and gross misinterpretations of evidence that has been presented to the examiners. In addition, gaps in documentation and statistics are remedied as far as possible at this stage. My own experience demonstrates conclusively, I believe, that in no way is there censorship exercised on the examiners by either the country under review or the OECD secretariat. This is not to say that suggestions are not made for change in this or that aspect of the examiners' drafts, but the examiners are, in the final instance, free to render their own judgment in their own words. Who wants to get what from the reviews? It is, I believe, important to recognize that the reviews of country educational policies do not occur in a political vacuum. They are the result of a specific agreement between the authorities of the country concerned and the OECD secretariat, and are intended to serve a variety of interests. The OECD secretariat has at least five interests in the review process. There is first the desire to have knowledge of the member country's experience, potentials, and problems in education shared as widely as possible internationally. On the basis of the review work, the secretariat hopes, too, to build an ever-increasing spirit of international co-operation. Third, such work helps to cement further the particular elements of international cooperation. that are represented by the OECD itself. Fourth, any particular review is looked upon as an opportunity to improve the educational review process in general. It is hoped that practice makes perfect here as elsewhere. Last, the content of a review helps to test the validity of generally agreed OECD policies in educational affairs against experience 9 The OECD Review and Higher Education of the member countries. This leads to the considerable enrichment and modification of OECD educational policies over the years. A country accepting a review also has a number of interests in mind. There is stimulus provided to self-examination. Certainly that was a most important interest of the educational authorities of Canada, and the background documentation in Canada was the outcome of a co-ordinated effort among and between the provincial and federal authorities that represented a "first" for Canada. Second, the review does provide a good platform from which a country can hope to inform the governments of many other countries concerning its achievements in the field of education, and its ambitions for the future. There is also an important plus of the opportunity to receive an external, and it is hoped objective, critique of the conduct of educational affairs that will be of value internally in the future construction of policy. Last, and this has often been of decisive importance for the timing of particular reviews, country authorities believe that a review and the surrounding publicity will help to mobilize support for a desired programme of innovation and change. There were strong elements of this motivation for the timing of the reviews that have taken place in Germany, Austria and Australia. The examiners' interests tend to be somewhat more personal than the foregoing. Invariably, they have a vivid interest in learning more about the country they are visiting, and the process of visits and discussions makes for what amounts to a travelling seminar for the examiners. Many of them tend to be interested in the comparative study of educational phenomena and are eager to test in practice the validity of Jullien de Paris' notion that international examination can help improve national educational policies. In any case, there is an intrinsic attraction in the opportunity given the examiners to hold up a mirror, reflecting back to the country an image of its educational system as seen through the eyes of the examiners. The mirror is, no doubt, a distorting lens to some degree. Indeed, I would argue, it should not be a perfect mirror, merely giving back what was shown to it. There exists a strong, and properly strong, tension between the problems and "facts" as the representatives of a country see them and the way those same phenomena are viewed and evaluated by the examiners. It is precisely from these often unexpected, and not always welcome, reflections that the contribution of an examination will emerge, if it is to emerge at all. Who and what undergoes these examinations? I noted already that some twelve nations have had reviews of their policies for education, some more than once. The OECD does not force itself upon a country. Rather, it is a matter of a member country finding it convenient or useful to request that a review be undertaken at a particular year. Within each country, it is largely official governmental policies and the administrative and institutional apparatus for regular, formal education that is examined. In most countries relatively little engagement from the non-formal, non-public, and non-governmental sectors of education is sought, or achieved. The explanation for this is to be found in the emphasis upon official, governmental relationships which is the hallmark of the OECD. An explanation is not, of course, a justification. Country examinations are regularly criticized for giving insufficient attention to the non-governmental sectors. In Canada, the 10 Edward Sheffield, Harold J. Noah and Hildegard Hamm-Briicher so-called "interest groups" (representing teachers, school trustees, associations dealing with education of exceptional children, and so forth) were extremely critical of what they conceived to be a one-sided process of review. In Germany, too, there was little indication that the work of teachers' associations was considered worth devoting time to, so that whatever success the examiners had in meeting with representatives of teachers' associations was the result of ad hoc and informal arrangements. The review procedure in England and Wales formed a somewhat surprising set of arrangements. The examiners there met with Ministry officials only, and this caused something of a stir. In Austria, although the examiners did indeed meet with representatives of employers and trade union associations, they did so only in the presence of the officials responsible for education. However, the final judgment must be that, given the constraints on the time available, most countries do manage to have the examiners meet rather freely with an extraordinarily wide range of representative interest groups and individuals. Preparing for the review Some candidates for an examination are able to prepare for the Big Day methodically and in good time; others stay up all night, cramming together whatever they can in a helterskelter rush. So it is with country examinations. The background reports for Canada and for Austria, to give just two examples, were assembled in the most thorough and painstaking way. They provided the examiners with comprehensive surveys of the educational systems and their operation. The background reports from some other countries have not been so thoroughly prepared (for example, in Germany), and they were in consequence of less use to the examiners. The tasks facing a country preparing for an examination are basically threefold. It is necessary first to identify for examination a set of institutions, issues, policies, and plans. Some countries have taken a wide definition of their educational system for examination, including as much as possible in the survey. The reviews in France, Sweden, Germany and Canada were of this all-inclusive variety. In Austria and the United States segments of the educational system were identified, and there seems to be an increasing preference by countries and the OECD secretariat to structure reviews more narrowly within specified sectors of the educational system. This stage of preliminary definition and discussion can take upwards of a year. In Canada it took about four years to get from the initial idea of a review to the end of the process of definition of the structure. The second and main stage of preparation is the work on the background report, which must summarize the system of education, describe its objectives and modes of policy making, and estimate the success and shortcomings of the system with respect to the stated objectives. Finally, in consultation with the OECD secretariat, the country authorities must establish a schedule of visits, discussions, mini-conferences, and appointments for the examiners, to occupy between a week and a month of full-time examiners' work. In general, country authorities tended to be over-optimistic about the lead time that is necessary to get arrangements in final shape for an examination. Partly this is the result of delay that is inevitably associated with diplomatic protocol, but there is also a tendency to under-estimate the complexities of the background report and the schedule of examiners' appointments. Both are fraught with potential political aspects. As the process of pre- 11 The OECD Review and Higher Education paration for the review comes toward its end, more and more offices and non-official groups come to realize that a quite significant event is about to take place, and demand that they be granted some piece of the action, or some access to the background report before it is finally submitted. It is not therefore unusual to find that the date for the beginning of an examiners' visit is postponed, sometimes more than once. Results of reviews As a result of an educational policy review, the visibility of educational policy both nationally and internationally is quite definitely increased. The immediate result takes the form of assessing "how we did" in an international scrutiny. Any and all elements of praise for the country and its educational system are, of course, seized upon with alacrity, and repeated in the press with many references to the examiners' objectivity and sagacity. At the same time, there is a marked tendency to rebut points of criticism (real or imagined) while underlining the ignorance of the examiners, stemming either from their lack of time or opportunity to acquaint themselves more thoroughly with conditions in the country, or from their sheer innate obtuseness. Such immediate reactions, while they generate a good deal of temporary heat, are not the most important. In assessing the longer term effects of a country review, it is necessary to recognize that any given review cannot and should not be understood simply in terms of a single country report. The programme of reviews is essentially a serial process, in which there is a gradual accretion of a body of international audits, judgments, and identifications of trends of policy. Hence, the context in which a given country review takes place is extremely significant for understanding why the examiners have concentrated on this or that aspect of the educational system, or emphasized this or that approach to educational policy. Thus, policy reviews have tended to concentrate on a fairly limited range of issues: equality of educational opportunity (both as a goal and in terms of actual achievement); curriculum structures (especially the fit between schooling and work); the aptness of government and school governance structures; the role, influence, skills, and training of teachers; policies and practices for the education of certain special groups of children (linguistic, ethnic, socially disadvantaged, learning disabled and so forth); and thefiscal implications of educational policy. These particular perspectives arise from the major policy concerns of the OECD as an organization. The politicians and administrators who attend OECD meetings, and all who read OECD documents regularly, thus accumulate gradually a body of case-study and comparative information that helps them to set their own, more intimately known problems and achievements in a wider context of understanding. Within the given country, ideas and judgments that at first shock and seem unpalatable often become assimilable in time. In Germany in particular, there was great resistance to the ideas in the examiners' report, especially in relationship to the overloaded curriculum, hours of schooling, and the tight relationship between particular educational qualifications and specific occupations. But over the years there has developed increasing willingness within Germany to consider ways in which some of the areas of examiners' criticism might be dealt with. 12 Edward Sheffield, Harold J. Noah and Hildegard Hamm-Briicher Prospects for the review programme The future of the review programme must depend upon the continuing support for it provided by the OECD member countries. In a sense, they are continually gauging what the review programme costs them in time, money and energies deflected from other worthwhile enterprises versus what they judge they are deriving in terms of benefits. Regarded from this point of view, the prospects for the review programme seem to be good. One gets the impression that, although the immediate governmental reaction to a review is "never again," the disenchantment does not last long. One may suppose, moreover, that there is a certain automatic momentum built into the review programme. Having jumped across the review hurdles itself, a country may be quite enthusiastic about other countries doing the same. There are certainly arguments for tightening up and tidying up some aspects of the review process. Some suggestions that have been frequently made point toward making much clearer and more specific the particular areas, topics and aspects of education to be reviewed. Also, it is claimed that the examiners could well use better briefing of the country that they are to visit, and that the nature of the examiners' reports should be less value-laden and more objective and "scientific." And hopes are continually expressed that the entire review process might be held to tighter time schedules than has usually been possible. There may indeed be much of value in these, and other, suggestions. I would merely warn against trying to go too far in the direction of routinizing and bureaucratizing the review procedures. It is, I think, worth emphasizing that the review process is essentially a set of acts of "discovery" — often self-discovery by the country concerned, and creative discovery by the examiners and the Education Committee of the OECD in Paris. It is fashionable nowadays to ask of every investment of public funds "What will the product be?" With respect to the reviews of educational policy, it might be useful to tolerate a certain amount of untidiness, in return for many of the benefits flowing from this unusual international exercise. The process of reviews helps to build an international community in a way that is positive, practical, and non-threatening. Nations go about their educational business in their own manner, according to their best lights. Their sovereignty in education is in no way impaired, but through the reviews they demonstrate their willingness to "pay a decent respect to the opinions of mankind," as Thomas Jefferson put it on another occasion. In that spirit the reviews of educational policy have deserved and earned attention and support. MULTILATERAL CO-OPERATION IN EDUCATION: POSSIBILITIES AND LIMITATIONS (with special reference to the OECD review of educational policy in Canada) Hildegard Hamm-Briicher I have twice participated in an OECD educational review — once when my own country was appraised by an OECD team, after which I led the German delegation during the confrontation meeting. (The Federal Republic of Germany fared much worse than Canada.) On the second occasion I was one of the examiners of the Canadian educational system. The process entailed a rather full working program including three months for 13 The OECD Review and Higher Education preparatory work, a stay in Canada, and evaluation of reports, impressions and a heavy load of data now stored in the cellar of our house, and constituting a comprehensive archive of reference material on Canadian education. I have thus gone through both experiences. I know how uncomfortable one feels when the educational institutions of one's own country, which were developed over many decades, are critically examined by foreign experts. And on the other hand, I recall that as an examiner I felt just as much unease because, despite much good will and hard work, it does not seem possible to adequately perform the tremendous task. During both procedures, I asked myself more than once if the amount of money, time and intellectual and physical effort that go into such an exercise are really warranted by its results. I should like to leave this question unanswered for the time being, but return to it at the end of my paper. To begin with, I should like to comment on the review procedure itself, because it explains many weaknesses of the report and helps clarify some politically controversial statements, which can then be further explored. Following that, I should like to evaluate the comments from Canadian institutions and organizations on the OECD report — to the extent that I had access to them. Against the background of these comments, I should then like to focus on those problems that appear particularly relevant for the future development of Canadian education. Finally, I should like to consider in somewhat greater detail whether and in what way this novel form of international co-operation can be of practical use. The review procedure Even though the final composition of the team of examiners was not clear until the last moment, the six internal reports were sent to us much too late, and we did not have a final itinerary before we began it, the review process carried out in June 1975 went off extremely well. I shall always remember the readiness of the staff of the Council of Ministers of Education, the federal government and provincial and local authorities to help us in every conceivable way. The discussions with numerous representatives of universities and other post-secondary institutions, teachers, parents, school administrators, school trustees and many others were thorough and frank. Given my many years of experience and the possibilities for international comparison at my disposal, I can truly say that their commitment to, knowledge of, and interest in, the Canadian educational system, as witnessed by us, are likely unique in the world. Everyone participating in the meetings had this willingness and desire to make good use of the opportunities offered by the review. This impressed me all the more because, during the review conducted in Germany, more than once those "examined" demonstrated opposition and lack of frankness, and even occasionally provided incorrect information. As a result, the examiners were more critical, which in turn awakened nations defensiveness, and ultimately the results were disappointing. Nothing of this kind happened in Canada. Despite the strain and difficulties, our intensive four-week collaboration was harmonious, fruitful and even friendly, and the vigour with which, upon our return, we wrote our report was due very much to this friendship. We had not set out believing that we knew more than others, nor did we in 14 Edward Sheffield, Harold J. Noah and Hildegard Hamm-Briicher fact know more than others; we informed ourselves on the spot and made certain observations, comparisions and reflections. There was nothing else we could do, and nothing else could have been expected of us. I trust that our report, as well as my remarks, wil be understood and received in this spirit. I should now like to mention a number of deficiencies inherent in the external review procedure. We were, as many of our critics noted, a "European" team, and perhaps too European in the philosophy underlying our approach. The Canadian authorities, however, who expressly approved our nomination knew this from the beginning and could have objected. The time we had to prepare ourselves was too short, and during our four-week trip, the periods for quiet work after a tiring day were also too short and too infrequent. And it was only in the course of our tour that we were able to devise a suitable approach. After the visit to Canada we faced the almost insurmountable task of evaluating the fantastic amount of documentation that had been supplied to us. As the OECD report observes, "Canadian educational administration threatens to drown in a sea of unread, unstudied and unevaluated reports." The literature confirmed that Statistics Canada had played an important role in recording, quantitatively, the development of Canadian education over the past five decades. The report recognized this, and suggested greater future involvement for Statistics Canada in similar ventures. From what I know, the factual information in the study has not been challenged, which in my opinion and experience deserves recognition. Let us briefly return to the deficiencies of the review procedure. The so-called confrontation meeting in December 1975 in Paris had to be arranged hastily. There was no time for the examiners and their major Canadian partners to carry out an interim evaluation. The meeting itself went off in a manner which — I say this quite openly — was rather unsatisfactory for us examiners. Amidst the large group of all the OECD member states, the official Canadian delegation was "hedging" for the first time, and did not respond openly to our questions, so that some doubts arose in my mind that our work had been positively received by the Canadian representatives, and that it would lead to any practical consequences. Nevertheless, we, in particular Harold Noah and myself, did not want to give up, and irrespective of other commitments and work, we resolved to pursue the fate and possible consequences of our report. Response to the report This brings me to the second part of my remarks, namely a review and evaluation of the response to our report. First, there was the interest of the mass media, which was lively only as long as our report was kept secret. After it had been officially published, response was muted. I was also surprised that the report has not been subjected to critical academic scrutiny. I have been unable to discover many reviews or articles, although it has been available for almost two years. The academic community has welcomed many of its suggestions, but their response has yet to crystallize in formal publications. A few remarks now about the statements and comments of our official partners in the discussions, as well as those of non-government organizations. I have carefully studied the document titled A Statement by Canadian Authorities for the OECD Appraisal of Country Educational Policy Reviews, which was published by the 15 The OECD Review and Higher Education Council of Ministers of Education in co-operation with the Canadian government in April 1978.1 am well acquainted with the cautious, restrictive and complacent language of such official papers; therefore its content caused no serious disappointment to me. I think no authority anywhere in the world would officially admit that certain things could be done better or differently. Therefore, on this understanding, the statement contains many hopeful approaches. But these must be followed up not by the former examiners but by democratic institutions and responsible organizations such as the Canadian Society for the Study of Higher Education. For example, in an official press release in the Fall of 1976, the Council of Ministers of Education said that the report should not be considered the final word on education in Canada, and that the Council would explore various means of promoting wide public discussion of the major issues raised, including the possibility of a Canada-wide conference. Twenty months later such a conference had not even been announced, let alone taken place. It was left to non-government organizations to assess the report, on invitation of the Council of Ministers. Perhaps the CSSHE and other interested groups should insist that all the positive reactions and promises of the Council of Ministers of Education be implement The Statement mentions five points in the report about which public response is said to have been mainly negative (some of them had aroused controversy even among the examiners The statement goes on to mention no fewer than fourteen much more substantial points which have been favourably received — a comparison which deserves recognition! All in all, I feel that despite its "ifs" and "whens" this statement could be a cautious step forward, provided that its positive approaches generate effective public pressure for future action. The official reactions of the provinces appeared to be motivated by constitutional considerations and an apparent desire to maintain the status quo. Some questioned the basic thrust as well as specific observations. Others maintained that since the Review took a national perspective, many observations did not reflect the situation in their particular province. Since I am quite familiar with this line of reasoning from discussions in my own federalist country, I should like to stress that I understand many of the points, while at the same time expressing my hope that the whole process will lead to intensified co-operation among the provinces and to rapid, tangible progress jointly achieved with the federal authorities. This brings me to the reactions of sixteen non-government organizations — the so-called "interest groups" — which, taken together, made a considerable impression on me, even when they contained negative or critical remarks. All of them showed that our report had been carefully studied, and indicated a great factual knowledge and an untiring commitment to the cause of education. Canada can be proud of this proof of strength and vitality. Let me single out reactions to some of the more contentious issues raised in the report. The Canadian Education Association suggested that the report was written from an OECD perspective (similar views were expressed by government authorities). It is difficult to respond to so general a comment but, as I indicated earlier, available time and resources made it almost impossible to detect all the forces shaping Canadian education. The short time for background preparation, and lack of research and support staff, limited the examiners' opportunities to pursue some controversial issues. Moreover, it is always easy for a country to claim to be an exception. However, it is my impression that Canada differs from other OECD countries in degree rather than in kind. Although I believe that 16 Edward Sheffield, Harold J. Noah and Hildegard Hamm-Briicher the examiners might have qualified some of their statements, this would be unlikely to change the message of the report. The second most disputed issue was federal involvement, specifically the claim that no federal policy exists, but that federal participation in education occurs under other labels — for example, manpower policy, science policy, research support, student assistance, and bilingualism. This topic was raised continually during our tour of Canada, and the almost schizophrenic nature of federal-provincial relations in educational policy was considered a major stumbling block. In an article I wrote about my Canadian experience, I compared education policy relations between the federal and provincial governments to "sex as it used to be in times of prudery. You do it but you do not talk about it and even if you should allude to it, you never use the right words." The OECD report recommends greater federal involvement through the Council of Ministers of Education. This suggestions was picked up by the Prime Minister who proposed a permanent structural arrangement. The provinces vetoed this proposal, offering instead to allow federal representatives to participate, on invitation, at regular meetings of the Council if the necessity arises. Although this arrangement goes some way in the direction the report envisages, it will not resolve the anomalies of federal-provincial relations in education. The report expressed the hope that the role and functions of the Council of Ministers of Education could be clarified and expanded. The Council manifested its potential by sponsoring the OECD study, but has yet to become an effective organ serving provincial, inter-provincial and national purposes. The examiners noted that the Council's small secretariat, despite its dedicated staff, is unable to generate the research and documentation necessary for policy formulation. This observation also applies to the federal scene where the designated department, that of the Secretary of State, does not have sufficient expertise in matters of education. Canada has too few institutions concerned with education from a national perspective. The lack of what is called "Bildungsforschung" and "Bildungsplanung" in Germany (research and planning in education) is one of the weaknesses perceived by the OECD examiners and, I have been told, the situation has not changed since 1975. Nor have the goals of Canadian education become clearer, although this, too, was another desideratum identified in the report. It is generally agreed that because of a number of factors such as demography, labour market conditions and rising costs, education in Canada is facing major challenges. The momentum which the OECD .Review might have created has not yet materialized. Problems related to future development Following this rather general response to the comments on the OECD report, I should now like to mention a few problems which appear to me relevant to the future development of education in Canada. First, there is the continuous task of giving all citizens equal educational opportunities. This cannot be done as long as there is no overall perspective ranging from pre-school through the tertiary sector to institutions of adult education. The aim should be to reduce regional, demographic and ethnic differences, and provide specifically for both handi- 17 The OECD Review and Higher Education capped and gifted children. (I really do regret that the Canadian Association for Adult Education complained that the report did not pay sufficient tribute to its activities. This impression is erroneous: in the opinion of the examiners and myself, adult education in Canada is among the most highly developed, innovative and diversified.) In the decades ahead, continuously changing demands and conditions in professional, social and personal life will cause adult education to assume growing importance all over the world. Second, there is the necessity of improving the possibility of acquiring professional skills in school, and providing better education in fine and applied arts including music. Third, all educational institutions are confronted with the challenge of responding ever more effectively to their environment, to problems of community life and to changing living conditions. This cannot be achieved without a carefully structured network of planning, research and development institutions. Nor can this be done without assigning greater responsibility to the national government or without improving co-operation. The difficult issue of relations between the Council of Ministers of Education and the "interest groups" is a case in point. The examiners questioned the attitude of Canadian authorities toward interest groups and the strangely ambivalent relations with them. The report concludes: "It would help if the interest groups had a stronger and more secure role in the process of educational decision-making." The frequent absence of government co-ordination gives enormous importance to these national organizations. But although the Council of Ministers increasingly solicits their advice, their participation in decision making has remained haphazard. Much depends on whether co-operation will improve in the future. I also hope that the provincial and federal parliaments will deal with the OECD report in the near future. In my opinion, in view of their responsibility for major national decisions, the non-involvement of political institutions in education should be overcome. In fundamental matters, at least, education policy, like all other fields of domestic policy, needs parliamentary control and decision. The framework of my paper forced me to limit myself to these few remarks. I hope, however, that they convey the considerable and sincere interest which the OECD examiners, and I personally, have in the further development of the Canadian educational system. The benefits of international co-operation Now I should like to return to the possibilities and benefits of international co-operation in education policy. All Canadian statements on the OECD Report repeatedly stress how important and useful its very preparation had been for all participants. For example, the Council of Ministers' Statement speaks of the processes and activities involved in the internal and external components of the project, and in the preparation of the reports of these phases. These activities gave rise to numerous special studies and research at the provincial and federal levels, and to highly co-ordinated inter-government consultation and collaboration. This experience in itself may well serve as a model, and influence further evolution of inter-government co-operation. Nevertheless, in my opinion, one should not be content and self-complacent with this undoubtedly gratifying progress. Of course, the examiners were and are aware that our report would not be a revelation 18 Edward Sheffield, Harold J. Noah and Hildegard Hamm-Briicher to our Canadian partners, nor could it be expected to contain panaceas. Being a politician, I have always known that outsiders can do even less than insiders to change established structures, and that accepting outside criticism and advice is difficult in itself. Nevertheless, despite these and other limitations to multilateral co-operation in education, I feel that an intensified exchange of information at an international level is necessary and possible. Many national issues can be examined by friendly third parties from a fresh perspective, and this may have a favourable influence on the parties directly concerned. Such an exercise can be helpful in that one is compelled to look at one's own situation with greater objectivity and understanding, and thereby define one's position more clearly. Initiatives are encouraged and tensions are reduced to normal proportions. In this respect, I perceive many opportunities for further co-operation. These were just a few personal remarks and reflections on the OECD review procedure and its consequences. They strengthen me in my conviction that, despite the deficiencies and disappointments, and the hard work involved, international co-operation in the OECD report has proved its worth. I hope that in the manner of a seed corn it will bear some fruit — certainly in minor matters and one of these days, perhaps, even on a larger scale.