The Canadian Journal of Higher Education, Vol. XIII-3, 1983 La revue canadienne d'enseignement supérieur, Vol. XIII3, 1983 An Examination of Performance of First Year Students at an Ontario University: An Admission Perspective* LORRAINE G. ALLAN, ALEXANDER L. DARLING, ROBERT C. HUGHES and JACK M. ROSENFELD** ABSTRACT Major changes in the Ontario secondary school system over the last 15 years have increased the concerns within some universities about the use of Grade 13 marks as the principal criterion for evaluating applicants. These concerns focus on grade inflation, variability of marks among schools, and how appropriate some subjects are for university admission. This paper reports on a study of these issues conducted at McMaster University. The relationship between Grade 13 admission and Year I university averages was examined. Two issues that have received little attention elsewhere were also examined: how Year I performance varied by applicant status, and the relationship between individual Grade 13 subjects and Year I performance. The results indicate that Grade 13 marks, which are the only measure of a student's achievement available, continue to be a reasonable predictor of subsequent performance in Year I. Grade 13 subjects were identified that were more consistently associated with Year I performance than others. There was also evidence that some Grade 13 subjects may be contributing to an "inflated" admission average. The results of this study should be useful in evaluating existing admission policies and in the development of the new Ontario Academic Courses. * A working paper of the same title containing complete data may be obtained from the Office of Institutional Analysis, McMaster University. ** The order of authorship is alphabetical. We would like to express our appreciation to Dr. Fred Hall for his comments and suggestions. Lorraine G. Allan is Professor of Psychology and Chairman of Undergraduate Council; Alexander L. Darling is Registrar and Director of Institutional Analysis; Robert C. Hughes is Analyst, Office of Institutional Analysis; and Jack M. Rosenfeld is Associate Professor of Pathology and Chairman of the Admissions Committee. 38 Lorraine G. Allan, Alexander L. Darling, Robert C. Hughes and Jack M. Rosenfeld RÉSUMÉ Au cours des 15 dernières années, de sérieux changements dans le système des écoles secondaires de l'Ontario ont ajouté à l'inquiétude qui règne au sein de certaines universités face à l'emploi des notes de la 13e année comme critère principal d'évaluation des candidats universitaires. Ces craintes se centrent sur des notes "gonflées", sur une notation variable d'école en école et sur la pertinence de certains cours à l'entrée universitaire. Ce papier traite d'une étude à ce sujet, menée à l'université McMaster, qui visait à percevoir les rapports entre la moyenne d'admission de la 13e année et celle de la lere année universitaire. L'étude porta également sur l'écart entre la performance universitaire et le statut du candidat et sur la performance universitaire reliée à certaines matières de la 13e année, deux questions qui avaient suscité jusqu 'ici peu d'intérêt. Les résultats de cette étude indiquent que les notes de la 13e année, qui sont le seul système de mesure actuel du rendement scolaire, n'en demeurent pas moins un indice valide de la performance post-secondaire. Dans ce rapport, certaines matières de 13e année furent identifiées et étroitement associées au succès universitaire. Les découvertes signalèrent également un rapport probable entre certains sujets de 13e année et la hausse dans la moyenne d'admission. Toutes ces données devraient servira évaluer les procédures actuelles d'admission et servir à la mise sur pied de nouveaux 'cours pré-universitaires'. Admission to Ontario universities is based primarily upon a student's performance in the final grade of secondary school, Grade 13, which prior to 1968 was assessed through provincial examinations administered by the Ontario Department of Education. During the late 1960's two policy initiatives which reflected changing educational philosophies were developed: the elimination of the Grade 13 departmental examinations in 1968 and the decentralization of the responsibility for curriculum development. These two initiatives were noted in the 1967 annual Report of the Minister of Education (Note 1): ". . . t h e c o n c e p t o f c u r r i c u l u m e v o l v e s i n t h e d i r e c t i o n o f i n c r e a s i n g l o c a l i n i t i a t i v e a n d f l e x i b i l i t y in c o u r s e d e v e l o p m e n t . . . " ( p . 1). and ". . . W i t h a b o l i t i o n o f u n i f o r m p r o v i n c e - w i d e e x a m i n a t i o n s , p r i n c i p a l s and teachers n o w have the same responsibilities for Grade 12 and 13 p r o g r a m m e s as f o r o t h e r g r a d e s . " (p. 3) The prevailing philosophies about curriculum and evaluation were captured in the 1968 Report of the Provincial Committee on Aims and Objectives of Education in the Schools of Ontario, Living and Learning (Note 2): An Examination o f Performance of First Year Students at an 39 Ontario University : An Admission Perspective " T h e l o c k - s t e p structure of past t i m e s m u s t give w a y to a s y s t e m in w h i c h the child will progress f r o m year t o year t h r o u g h o u t the s c h o o l s y s t e m w i t h o u t t h e h a z a r d s a n d f r u s t r a t i o n s o f f a i l u r e . . . " (p. 1 4 ) These philosophies were reflected in Recommendation 75 (p. 186) that the use of formal examinations be abandoned, and in Recommendation 83 (p. 187) that responsibility for curriculum design and implementation be placed with school boards and individual schools. During 1975, educational standards were the subject of public debate, through the media of public speeches and the editorial columns of provincial newspapers and in the 1975 Ontario provincial election campaign. During this debate there was some call for the return of standardized tests, at least in English and Mathematics. In the Spring of 1975 the Ministries of Education and of Colleges and Universities agreed to the establishment of a joint study of the interface between secondary and postsecondary education, and of the adequacy of preparation of secondary school graduates for postsecondary education. The release of the report of the Secondary-Postsecondary Interface Project (Note 3) in 1977 was accompanied by an announcement by the Minister of Education of changes in the secondary school curriculum, including greater specification of the subjects required for the secondary school diploma. In recent years, some universities have continued to voice concern over issues related to admissions and scholarships. These issues are the possibility of increased variability in marks among schools for similar performance, of inflation of secondary school marks, and of some secondary school subjects being inappropriate for university admission. The concern about variability is most focussed on questions about the potential for inequitable treatment of applicants for admission to limited enrolment programmes and of candidates for scholarships in cases where Grade 13 performance is the principal criterion for selection. Inflation of marks and an inappropriate selection of subjects may permit students who are inadequately prepared to enter university with little chance of success. Admission requirements and policies of universities have traditionally served four purposes. One is to identify those applicants who have the greatest chance of succeeding in their university studies. A second is to ensure the equitable treatment of applicants for admission to programmes of limited enrolment and of candidates for scholarships. A third is to ensure that a student has an adequate foundation for a specific university programme, which is normally achieved through the specification of individual subject requirements. The last is to ensure that the student entering university has the general education expected of a secondary school graduate. Within Canadian society one might argue that this general education should include some competence in the two official languages of the country, numeracy, some understanding of science, and knowledge of Canadian history and culture. While the development of such attributes may not be primarily the responsibility of the university, it may seek to encourage their development through its specification of admission requirements. 40 Lorraine G. Allan, Alexander L. Darling, Robert C. Hughes and Jack M. Rosenfeld This paper reports on a study of Year I performance conducted at McMaster University, The study follows earlier studies of Year I students registered at McMaster in 1969-70 (Note 4) and 1974-75 (Note 5). The goals of the study relate to the first two purposes of admission requirements and policies articulated above. While the results of this kind of study are useful in evaluating current policies, they should not be the sole factor in such an evaluation. Investigations related to admission policies are not new as noted in the review of literature below, and are relevant in view of the Ontario Minister of Education's response to the 1981 Final Report of the Secondary Education Review Project (Note 6). The response, The Renewal of Secondary Education (Note 7), was released in November 1982 and includes five items which are relevant to university admission, namely: 1. 2. 3. 4. 5. review and rewriting of the curriculum for Grades 7 to the termination of secondary school study; replacement of the two graduation diplomas (the Ontario Secondary School Graduation Diploma for which 27 credits are required, and the Ontario Secondary School Honour Graduation Diploma which requires 6 Grade 13 credits) by a single diploma of 30 credits; an increase in the number of required credits for the diploma from nine to sixteen; development of prescriptive, province-wide curricula for the Ontario Academic Courses which are to replace Grade 13 courses; and further development of the Ontario Assessment Instrument Pool, a bank of items to evaluate performance, with priority being given to the development of items for the new Ontario Academic Courses. These proposals, following those announced in 1977, represent a change in direction from the trend developed in the 1960's. REVIEW OF LITERATURE Studies of student performance in other provinces of Canada and in the United States have been widely reported, though many such studies have been conducted in-house and are not available in the published literature. This review is confined to studies conducted in Ontario, which are representative of those conducted elsewhere. 1. Atkinson Study of Utilization of Student Resources The first comprehensive study in Ontario was the Atkinson Study of Utilization of Student Resources (Note 8) which was conducted during the time when students were admitted on the basis of Grade 13 provincial examination results. Of the thirteen volumes of the study, the three made available in 1959, 1962, and 1965, are particularly relevant. The best predictor of university performance was Grade 13, but the Grade 12 average made a sufficiently independent contri- An Examination of Performance of First Year Students at an 41 Ontario University : An Admission Perspective bution to warrant its inclusion in addition to the Grade 13 results, whereas the value of Scholastic Aptitude Test scores and principals' ratings was marginal. Students who needed more than one year in Grade 13 appeared to need an average of at least 5 per cent higher in that grade to have the same chance of success as those who completed the grade in a single year. The correlations between Grade 13 and Year I averages varied among programmes with the highest occurring in Applied Science (Engineering) and lowest in Arts programmes. 2. Studies by Ontario Institute for Studies in Education Following the abolition of provincial examinations, a series of studies on students completing Grade 13 in 1967, 1968, and 1969 was conducted under the auspices of the Ontario Institute for Studies in Education (Note 9). These studies, which involved students at a number of Ontario universities, reported that the correlations between first-year university achievement and measures of previous achievement (irrespective of whether they were based on provincial examinations or teacher-assigned marks) were higher than the correlations between first-year achievement and aptitude test scores. Higher correlations were reported in Applied Science/Engineering and Science than for Arts disciplines, and for females in relation to males. 3. McMaster Studies At the same time studies on the performance of students entering each of the Faculties of McMaster University in 1969-70 were conducted at the University (Note 4), and the findings paralleled those of earlier studies. The best single predictor of success was one of the measures of achievement in Grade 13, the admission average computed on the basis of either final or interim Grade 13 marks. The Pearson correlation coefficients ranged from a high of 0.78 in Engineering to a low of 0.52 for students entering Social Sciences I without Mathematics in Grade 13. Those who needed more than one year to complete Grade 13 required a higher average to have the same chance of success as those who had completed Grade 13 in a single year. Oksanen and Spencer (1975) in reporting on the performance of students in Year I Social Science courses at McMaster University noted that the secondary school average and Ontario Scholastic Aptitude Test-Verbal were significant predictors with the former accounting for more of the variance. It was also noted that in the case of most of the subjects examined females tended to perform better than males, all other things being equal. In a subsequent examination of performance of students entering McMaster University in 1974-75 (Note 5), it was found that the correlation coefficients for the 1974 entering class did not differ significantly from those reported earlier for the 1969 class. In general, those who had completed Grade 13 in an earlier year performed as well as those who completed Grade 13 in 1974 and proceeded directly to university. Those who had multiple attempts at Grade 13 did not 42 Lorraine G. Allan, Alexander L. Darling, Robert C. Hughes and Jack M. Rosenfeld appear to perform as well as those who completed Grade 13 in a single year. The performance of students with low admission averages was significantly below that of those with higher averages. 4. Secondary & Postsecondary Interface Project Within the Secondary-Postsecondary Interface Project mounted in 1975, the Nature of Students was examined (Note 10). Inter alia the project examined differences in marking standards among schools, the levels of performance in secondary schools and postsecondary institutions, the relationship between the two, and the differences that may have occurred over the preceding decade. The investigators concluded that a student's performance in first-year university was as predictable as it had been when province-wide examinations had been used to determine Grade 13 performance. However, while achievement test results suggested that there had been reasonable stability in mathematical standards, there appeared to be a "serious decline" in student achievement in Physics in secondary school (p. 306). While variability among the marks assigned by schools was noted, the authors suggested that the differences were not sufficient to warrant a student changing schools in an attempt to obtain higher marks, although the differences could affect a student's admission to a selective programme (pp. 307-309, 312-313). 5. Council of Ontario Universities Experimental Testing The Council of Ontario Universities (Note 11) reported in 1979 on its experimental testing of freshmen in English and Mathematics in 1975 and 1976. The 1975 administration involved students at four institutions and that in 1976at two institutions; only McMaster University was involved in both. Self-selection among participants occurred in 1975, whereas almost all freshmen were included in 1976. The timing and method of administration may have detracted from the validity of the results, and different standardized tests were used in the two years. Notwithstanding these difficulties, the Admission Average based on Grade 13 marks was the best predictor of Year I performance, and the correlation coefficients were comparable to those obtained when province-wide examinations were administered. For the 1975 group entering McMaster, the coefficients ranged from 0.68 in Natural Sciences I to 0.49 in Business I, and for the 1976 group, from 0.76 in Engineering I to 0.54 in Business I. The correlation between averages based on interim marks and those based on final marks was 0.94 with the former being slightly poorer predictors; the final average could be predicted from the interim value within 2.5 marks on a scale of 0 to 100 for 68 per cent of the applicants. The addition of the test scores to high school performance data would have provided only slight gains in the efficiency of student selection. The available data suggested substantial variability in marking standards of various high schools, but the additional information provided by a single test score was not found to correct for this variation to any great extent. An Examination of Performance of First Year Students at an 43 Ontario University : An Admission Perspective 6. Summary The results of all the studies are consistent. Grade 13 grades are the best predictor of Year I performance, and are a better predictor for females than for males. The correlations between Admission Averages and Year I performance for Science and Engineering programmes arc higher than those for Arts programmes. Students who need more than one year to complete Grade 13 require a higher Grade 13 Admission Average to have the same chance of success as those who completed Grade 13 in a single year. Variability in marking standards across secondary schools has been documented. FOCUS OF THE CURRENT STUDY The 1976 class is the most recent group reported in the published literature. The recent decision of the University of Toronto to require applicants to write achievement tests prompted the investigators to examine whether the strength of the relationship between Grade 13 Admission Average and Year I performance in various programmes has decreased in recent years. As in previous studies, this relationship was examined separately for females and males. Since secondary school records no longer allow one to differentiate, with confidence, between students who complete Grade 13 in one year and those who do not, this variable was not examined in this study. The sample was too small to examine meaningfully the issues of variability in marking standards in different secondary schools or even school boards. This study focussed on two issues which have received little attention in previous studies. Since McMaster had changed its admissions policy in recent years, especially for applicants from Ontario Community Colleges and applicants from other provinces, there was interest in determining whether Year I performance was related to Applicant Status. The other issue of interest was related to the decision of the University of Toronto in 1981 not to accept selected Grade 13 subjects for admission. (The subjects specified as unacceptable included data processing, secretarial science, technological studies, and other business studies. Grade 13 Courses still under review include accounting, dramatic arts, family studies, law, multidisciplinary studies, other arts studies, physical and health education, screen education and visual arts.) The relationship between marks in individual Grade 13 courses and Year I performance was examined. In addition there was a desire to determine whether there were Grade 13 courses in which the assigned grades were consistently high, since there was concern that inclusion of a number of high mark courses could result in an inflated Admission Average. METHODOLOGY A data file was created for all full-time undergraduate students registered at McMaster University during the 1980-81 Winter Session in the eight Year I pro- 44 Lorraine G. Allan, Alexander L. Darling, Robert C. Hughes and Jack M. Rosenfeld TABLE 1 ANALYSIS OF RESULTS OF SESSION BY APPLICANT STATUS Results of Session Failed/Withdrew Incomplete Programme Applicant Status Clear Business Grade 13 Current 224 172 35 N/A 17 14 11 6 4 1 0.92 1.76 205 13 147 N/A 12 13 1 - 15 17 1.95 3.69 N/A Grade 13 Prior Repeaters Engineering Grade 13 Current Grade 13 Prior Repeaters x' Humanities Grade 13 Current 144 31 14 Science Grade 13 Current 249 86 112 N/A 23 49 11 10 13 11 0.84 5.17 Grade 13 Current 268 91 63 N/A Grade 13 Prior Mature Students Repeaters 36 18 40 20 9 10 10 8 17 2.65 2.32 5.22 Grade 13 Current 83 49 27 Grade 13 Prior Repeaters 5 25 13 8 4 2 Grade 13 Prior Repeaters Social Sciences Physical Education N/A 7.98" 4.96 1 The chi-square value reported is for comparisons between individual applicant status categories and the Grade 13 Current category within a particular programme. With 2 degrees of freedom, the critical value of chl-square = 5.99 ( p < . 0 5 ) * Significant grammes: Business I, Engineering I, Humanities I, Music I, Natural Sciences I, Social Sciences I, Physical Education I, and Nursing I. Music I is a small programme and for this study Music I students were grouped with Humanities I students. Since Nursing I is a small programme which is highly selective in admissions and which could not logically be combined with any of the other Year I programmes for this analysis, students in Nursing I were not included in the study. To allow for a longitudinal comparison, partial data files were created for full-time Year I students registered in the 1974-75 and 1981-82 Winter Sessions. Grade 13 performance was measured using Final and Interim Admission Averages and final marks in individual courses. Both Admission Averages are weighted means of all required courses, as established by the University programme, plus enough courses to yield a total of 6 credits. The Interim Average is determined mid-year, the Final Average in June. Success in Year I University was measured using Year I University Average and Result of Session. The University Average is the weighted mean of the percentage marks in the courses attempted in Year I. Results of Session is a categorical variable which represents academic standing upon completion of Year I. There are four categories, as follows: Clear to Continue in Programme: The student has obtained credit in all courses for Year I and obtained an average exceeding the minimum prescribed for the programme; Failed Year: The student has failed courses which account for more than 40 per cent of the units for the Year; An Examination of Performance of First Year Students at an 45 Ontario University : An Admission Perspective TABLE 2 PEARSON CORRELATION COEFFICIENTS FOR FINAL GRADE 13 ADMISSION AVERAGE AND YEAR 1 AVERAGE FOR FIVE ENTERING CLASSES Entering Class of 1969 19741 1980 1981 n/a n/a n/a n/a .56 .75 .54 .71 .63 .62 Males Females overall n/a n/a .63 n/a n/a .75 n/a n/a .67 .73 .77 .73 .68 .70 .68 Humanities 1 (including Music 1) Males Females .58 .73 .47 .67 .56 .63 .54 .60 .59 .73 Natural Sciences 1 Males Females .75 .81 .59 .77 .75 .74 .62 .72 .66 .66 Social Sciences 1 Males Females .45 .64 .53 .56 .53 .61 .42 .53 .41 .49 Physical Education 1 Males Females n/a n/a n/a n/a .42 .63 .39 .61 .50 .48 Programme Sex Business 1 Males Females Engineering 1 1967 Sources: 1967 S. B. Khan, Pat Ransom, Martin Herbert. Prediction of First-Year Achievement in Ontario Universities. O.I.S.E., Toronto, 1970. 1969 S. B. Khan, S. A, Rickard. The Prediction of University Achievement in Ontario Universities. O.I.S.E., Toronto, 1971. 1974 Office of Institutional Analysis, McMaster University. 1980 Office of Institutional Analysis, McMaster University. 1981 Office of Institutional Analysis, McMaster University. 1 Other Pearson r values are available from 1974 calculated on a different sample of Year 1 students. See: A. L. Darling. The Academic Performance of Students Admitted to an Ontario University. Unpublished M.Ed, thesis, The University of Alberta, Edmonton, 1976. n/a: Not Available. Incomplete Year: The student is not clear to continue in the programme, nor has the Year been failed. This condition can result from a reduced load, failures in 40 per cent or less of the units, or an average below the minimum prescribed for the programme; and Student Withdrew: The student withdrew from the University prior to the last date for withdrawal in February. Since most Year I programmes have quotas, the Failed Year and Withdrew categories were combined for the analyses, because students in these categories occupied spaces which were limited in number. It should be noted that the marks necessary to calculate a Year I University Average are not always available. Therefore, the number of students contributing to analyses based on Result of Session is slightly larger than the number contributing to analyses based on Year I Average. RESULTS 1. Relationship between Applicant Status and Year I Performance The question investigated was whether Year I performance, as measured by Result of Session, is different for groups with different educational backgrounds. The applicants were partitioned into 14 Applicant Status categories. Those who 46 Lorraine G. Allan, Alexander L. Darling, Robert C. Hughes and Jack M. Rosenfeld TABLE 3 COMPARISON OF FINAL ADMISSION AVERAGE AND YEAR 1 AVERAGE FOR ACADEMIC YEARS 1974-75, 1980-81, 1981-82 Slope* Standard Error of Estimate -0.75 1.44 4.97 0.91 0.90 0.85 10.05 8.12 6.90 13.7 13.5 13.2 -19.64 -32.64 -39.02 1.11 1.20 1.26 10.20 9.24 9.73 68.5 68.0 72.0 10.3 10.8 9.1 13.07 5.45 11.03 0.76 0.84 0.80 8.16 8.77 6.53 9.4 8.7 8.7 66.8 59.4 62.3 14.2 15.5 13.9 -20.30 -29.12 -14.83 1.13 1.16 1.01 9.40 11.64 10.65 69.2 69.3 69.2 7.7 6.9 7.2 63.5 62.8 61.4 11.6 13.0 13.1 1.53 0.67 4.97 0.90 0.90 0.81 9.31 11.42 11.64 68.0 69.3 68.8 7.1 6.4 6.4 61.9 63.3 63.3 8.8 10.3 12.1 11.21 2.44 -1.71 0.75 0.88 0.94 7.12 8.63 10.52 Final A d m i s s i o n Standard Average Deviation Year 1 Average Standard Deviation Programme N r' Business 1 74-75 80-81 81-82 246 409 407 .58 .60 .61 69.5 73.2 74.5 7.9 6.6 6.3 62.6 67.7 68.7 12.4 10.1 8.7 Engineering I 165 74-75 337 80-81 81-82 326 .67 .73 .68 77.9 79.4 81.3 8.3 8.0 7.0 66.5 63.0 63.5 Humanities I 74-75 80-81 81-82 208 183 143 .61 .59 .70 73.3 74.4 75.1 8.3 7.6 8.1 Natural Sciences I 352 74-75 80-81 408 81-82 560 .75 .66 .64 76.8 76.0 75.8 Social Sciences I 418 74-75 396 80-81 467 81-82 .60 .48 .47 Physical Education I 74-75 156 144 80-81 81-82 155 .60 .55 .50 Intercept ' All values are significantly different from zero (p<.0001). completed Grade 13 in 1980 and entered the University that year, Grade 13 Current, were treated as the group against which the performance of other groups was compared. For each programme, the number of students in most of the Applicant Status categories was too small to allow for a statistical examination using the X 2 test for independence. (The criterion that for a test to be performed more than 80% of the cells required expected values > 5 was used). Results are presented in Table 1 for those groups for which tests could be conducted. The two categories where tests could be applied most commonly were (1) those who had completed Grade 13 in a year prior to 1980 and (2) those who were repeating Year I. Of the 11 instances where comparisons were possible, only one statistically significant relationship was found. (No correction was applied for multiple tests). Overall, the data suggest that there is no statistical difference in Year I performance among students in different Applicant Status categories. Because of recent changes in McMaster's admission policy, the number of students in the various categories, such as those entering from Community College, will probably increase in future years, and our, tentative conclusion should be re-evaluated. For all subsequent analyses only those students who came directly from Ontario Grade 13 were considered. Two reasons formed the basis for this decision: (i) this was the only group that was large enough to allow for statistical tests An Examination of Performance of First Year Students at an 47 Ontario University : An Admission Perspective TABLE 4 CORRELATIONS BETWEEN GRADE 13 AND YEAR I PERFORMANCE FOR INDVIDUAL GRADE 13 SUBJECTS' -^Programme Subject ^ Group A English Second Language Biology Chemistry Physics Economics Urban Geography History Mathematics Engineering Business N r N 405 114 111 150 109 195 85 120 400 .37 .46 .51 .46 .52 .53 .38 .32 .51 268 43 103 328 334 19 35 23 337 97 13 26 210 13 37 .27 r .36 .52 .61 .66 .59 .48 œ .58 .38 Ns .66 Humanities N . 183 137 54 42 13 35 40 107 91 Natural Sciences r N .56 .50 .61 .55 361 95 284 395 352 26 63 37 407 - .55 .65 .52 .40 r Social Sciences N r Physical Education N r .46 .52 .55 .56 .54 .41NS .48 .57 .59 379 112 182 124 66 145 132 226 242 .38 .37 .32 .31 .00 NS .35 .38 .47 .29 141 .44 36 .33 NS 121 .45 80 .57 31 .35 " s NS 30 -.09 54 .42 57 .53 102 .31 .62 212 22 32 76 27 20 .46 .16 Ns .35 NS 56 -.28 NS 2 67 .57 16 ,20 NS 12 11 - Group B Family Studies Music Physical Education Accounting Media C o m m u n i c a t i o n Math..of Investment - . 19 .51 - .54 NS 7 9 6 33 16 18 - .47 .49 NS .79 83 13 9 21 23 4 .54 - .20 NS .20 NS - 29 14 14 32 19 8 - .75 .65 - 2 3 NS NS .41 .55 ' If a student had more than one Grade 13 credit in a subject area, a mean mark was calculated for use in the analysis. Correlation coefficients are not reported when the number of students was less than 15. NS: not significant at p = .01. when broken down , by selected criterion variables; and (ii) being by far the largest source of first year students, data for this group would be the most relevant to evaluation of the admission process. 2. Relationship between Grade 13 Average and Year I Average Early admission at McMaster is based on the Grade 13 Interim Admission Average. Normally the offer is withdrawn only if the applicant fails to obtain six Grade 13 credits on the final marks. In order to evaluate these practices, an investigation was made to determine whether the relationship between the Interim Average and Year I University Average was as strong as that between the Final Admission Average and Year I University Average. The Pearson correlation coefficients were calculated separately for males and females for each of the programmes. The coefficients for the two Grade 13 Averages were similar, with a tendency for the Final Admission Average to show the stronger relationship. The correlation between the two Admission Averages was 0.96. The Final Average could be predicted from the Interim Average within 2.4 marks on a scale of 0 to 100 for 68% of the applicants. These findings were similar to those reported by theUouncil of Ontario Universities (Note 11). Discussions about admissions policy may include the desirability of raising the average required for admission, the performance of those with low admission averages, and the effect upon failure and withdrawal rates. In order to examine these issues the sample was divided into quartiles on the basis of Final Admission Averages, and the distributions of Results of Session for the quartile groups were analysed using the X 2 test. Not surprisingly, it was noted that the 48 Lorraine G. Allan, Alexander L. Darling, Robert C. Hughes and Jack M. Rosenfeld TABLE 5 DISTRIBUTION OF CHANGES IN GRADE 13 AVERAGE AFTER REMOVAL OF GROUP B COURSE MARKS' Change in fo/a?e Business N % Engineering N % Humanities N % Natural Sciences Social Sciences W N > +4.5 + 3.1 — +4.5 4 (1.3) 1 (1.1) 2 (1.6) 2 + 1.6 — +3.0 12 (3.8) 4 (4.6) 6 (4.8) 6 0.0 — + 1.5 52 (16.5) - - - % % 1 3 17 (5.5) - 27 (31.0) 35 (27.8) 35 (29.2) 69 (22.3) % Total N % 1 (0.1) - (1.0) (5-0) W (0.3) (1.7) Physical Education 12 (1.1) (4.0) 50 - 5 22 (17.6) 240 (46) (22.1) 123 (38.9) 28 (32.2) 49 (38.9) 50 (41.7) 108 (34.8) 33 (26.4) 391 (36.1) -3.0 83 (26.3) 21 (24.1) 20 (15.9) 21 (17.5) 66 (21.3) 37 (29.6) 248 (22.9) 4.5 30 (9.5) 4 (4.6) 9 (7.1) (5.0) 29 (9.4) 18 (14.4) 96 (8.9) < -4.5 12 (3.8) 2 (2.3) 5 (4.0) - 17 (5.5) 10 (8.0) 46 (4.2) -0.1 — - 1 . 5 -1.6 -3.1 Total 316 (100.0) 87 (100.0) 126 6 (100.0) 120 (100.0) 310 (100.0) 125 (100.0) 1084 (100.0) ' In the recalculated average, the removed Group B courses were not replaced by Group A courses that were not included originally in the Final Admission Average. percentage of students Clear to Continue increased with increasing Admission Average. Comparisons between adjoining quartiles indicated that in Business I, Engineering I, and Natural Sciences I, students with higher Grade 13 Averages tended to perform significantly better in Year I than those with lower Grade 13 Averages. In contrast, in Social Sciences I adjacent quartiles did not differ significantly. In examining those with lower admission averages it was noted that while in three programmes, Humanities I, Natural Sciences I, and Physical Education I, the b o t t o m two quartiles differed significantly, in the other three programmes, Business I, Engineering I, and Social Sciences I, they did not. This suggests that, particularly in the last three named programmes, simply raising the minimum Admission Average would administer rough justice and a more careful examination of an applicant's record would be warranted. In Table 2 Pearson's r between Final Admission Average and Year I University Average is shown for five entering classes, separately for males and females. (An overall correlation is given for Engineering I because of the low proportion of females). In general, as reported in other studies, the correlations for females are higher than the correlations for males, and the coefficients vary among programmes with the highest coefficients occurring for Engineering I and Natural Sciences I and the lowest in Social Sciences I and Physical Education I. The correlations for females decreased over the period from 1967 to 1981, whereas the pattern for males is quite variable. It should be noted that the size of Pearson's r is influenced by the range spanned by the variables being correlated. It is possible that some of the variations seen in the value of r over time is attributable to a constriction of the range of the Final Admission Average, especially in the limited enrolment programmes of Engineering I and Business I for which the minimum average for admission has increased. An Examination of Performance of First Year Students at an 49 Ontario University : An Admission Perspective The correlation coefficient measures the degree of asssociation; it does not provide information about the Final Admission Average normally associated with a criterion value of Year I performance. For each Year I programme, the Year I University Average was regressed on Final Admission Average separately for the entering classes of 1974, 1980, and 1981. The slopes, intercepts and standard errors of estimate are shown in Table 3 and the regression equations were plotted. For Engineering I, Natural Sciences I, and Social Sciences I, the regression lines suggested that, for a constant level of 60% in Year I, the Admissions Average required in recent years is higher than in previous years. The trend was reversed in Business I and there was no change in Humanities I or Physical Education I. The increase in Admission Average for a constant Year I performance seen in some programmes could be interpreted as "grade inflation" in the secondary school system. One must be cautious, however, since other factors, such as variations in University marking standards, were not controlled. Stronger evidence for "grade inflation" is the dramatic increase in the percentage of all Secondary School Honours Graduation Diploma (SSHGD) recipients in the Province who obtained first class honours on the requirements for the diploma. There is a steady increase from 6.6% in 1965-66 to 25.3% in 1979-80 (Note 12). The number of credits required for the SSHGD decreased from nine to seven in 1966 and then to six in 1971. The increase in Ontario Scholars may be partly attributable to the reduction in the number of required credits. However, the increase continued beyong the proportion of 14.6% in 1971 when the last reduction in the number of credits required for the diploma occurred. Further evidence for grade inflation is the increase in the percentage of Grade 13 papers passed (Note 12). In 1968 the proportion of papers passed was 85.2%, and the following year this rose to 92.5%. Since then the proportion has risen slowly to 94.3% in 1980. 3. Relationship between Grade 13 Subjects and Year I Performance The use of an overall Grade 13 average might mask potentially significant variations that exist among Year I students with differing Grade 13 subject backgrounds. This is particularly important in light of the expansion of secondary school courses in recent years into areas that could be considered "non-traditional". All the Grade 13 courses appearing on the 1980-81 data file were classified into two groups: Group A contained those traditional Grade 13 courses that are related to McMaster's academic curriculum, 1 and Group B contained more recently developed courses and courses that do not bear a direct relationship to the Year I programmes. 2 This classification of subjects paralleled the classification by the University of Toronto of acceptable and unacceptable Grade 13 subjects. During the classification of Grade 13 courses into groups, it became clear that the number of students enrolled in some courses, especially those in Group B, was small. It was necessary, in many instances, to aggregate a number of courses into subject areas to allow for meaningful comparisons. In creating these subject areas, an attempt was made to maximize homogeneity in terms of course content. 50 Lorraine G. Allan, Alexander L. Darling, Robert C. Hughes and Jack M. Rosenfeld Even after aggregation, the number of students enrolled in seven of the 13 subject areas in Group B was still too small (less than 60) to allow for meaningful statistical analysis in all the Year I programmes. In all Year I programmes there are many Group B courses that are rarely taken by McMaster students. Correlation coefficients were calculated to determine the degree of association between Year I University Average and individual Grade 13 subjects from both Group A and Group B and these are reported in Table 4. The correlation coefficients range, over the six Year I programmes, from a low o f - . 2 8 to a high of .79 with the majority being in the range .45 to .60. Not surprisingly, Grade 13 Mathematics correlates the highest with Year I performance in Business, Engineering and Natural Sciences. Conversely, Grade 13 English correlates the highest with Year I performance in Humanities. For other individual subjects, the interFaculty differences in the size of the correlation coefficients are not as intuitively consistent. The individual subject correlations do confirm, however, the trends in performance that have already been identified, that students who perform better in Grade 13 generally perform at a higher level in Year I University. While variations in the magnitude of the correlations are evident for individual subjects from both Group A and Group B, it is interesting to note the pattern of subjects that are not significantly associated with Year I performance. Of the 53 Group A subject correlations (over the six programmes), 7 (13%) were nonsignificant whereas, of the 22 Group B subject correlations, 10 (45%) were nonsignificant. A X 2 test indicated this difference was significant ( X 2 j = 9.16, p < .005). This suggests that performance in the Group A subject areas is, on the whole, more consistently associated with Year I performance than is performance in the Group B subject areas. An examination of the means revealed that Grade 13 marks in Group B subject areas were generally higher than marks in Group A subject areas, and for each of the six Year I programmes the mean of the Group B marks was higher than the mean of the Group A marks. A Mann-Whitney U test performed on the combined data from all the programmes confirms the trend noted above in that the distribution of Group B subject area marks differs significantly from the distribution of Group A subject area marks (U = 319, Z = 3.07, p < .01). 4. The Impact of Group B Course Grades on Final Admission Average Our analyses suggested that Group B subject area marks were, in general, higher than Group A subject area marks. In four of the programmes, Business I, Humanities I, Social Sciences I and Physical Education I, the percentage of students with at least one Group B course was high: 73.3, 66.7, 73.3, and 78.6 respectively. In contrast only 23.8 per cent and 26.8 per cent in Engineering and Science respectively included a Group B in the subjects used for admission. All Group B course marks were eliminated and an average based only on the remaining Group A courses was calculated. In the re-calculated average, based only on Group A courses (Group A Average), the removed Group B courses were not replaced by Group A courses that were originally not included in the 51 An Examination o f Performance of First Year Students at an Ontario University : An Admission Perspective Final Admission Average. In general, as shown in Table 5, the Group A Average was lower than the Final Admission Average before the removal of Group B courses. The distribution of differences between Final Admission Average and Group A Average is shown in Table 5. For each programme the number of students who showed a decrease greater than 3 percentage points was larger than the number who showed an increase greater than 3 percentage points. Over all programmes, 13.1 per cent of students showed a decrease greater than 3 percentage points, while only 1.2 per cent showed an increase greater than 3 percentage points. Further analyses were performed that disaggregated the information contained in Table 5 by the number of Group B courses included in the Admission Average. These data indicated that the difference between the Group A Average and the Final Admission Average was influenced by the number of Group B courses excluded. Of the students whose Group A Average was more than 3 percentage points lower than their Admission Average, the majority had two or more Group B courses excluded (67% in Business, 86% in Humanities I, 87% in Social Science I, and 71% in Physical Education I). In contrast, of the students who had only one Group B course removed, the majority showed changes no greater than ±1.5% (61% in Business I, 78% in Humanities I, 71% in Social Sciences I, and 71% in Physical Education I). Further evidence of the impact of increasing numbers of Group B courses was seen by an examination of the size of the change after removal of one, two, or three Group B courses; the mean decline increased with number of Group B courses removed. CONCLUDING COMMENTS The findings of this study may assist in the discussions and evaluation of the secondary school curriculum initiated as a result of the publication in November 1981 of the report of the Secondary Education Review Project (Note 6) and the response of the Ministry of Education in November 1982 (Note 7). Committees have been established to revise the curriculum from Grade 7 to graduation from secondary school and to develop Ontario Academic Courses which will be used for university admission. In addition the Ministry announced that it will continue to develop the Ontario Assessment Instrument Pool which is to be used on a sampling basis to monitor the Ministry's programmes. In addition to re-examining matters that have been the subject of earlier investigations the present study focussed on two issues which have received little attention previously. These were the relationship between performance in individual Grade 13 courses and Year I, and a comparison of those admitted on the basis of different academic qualifications. The examination of the relationship between marks in individual Grade 13 courses and Year I performance revealed that the "traditional" subject areas (Group A) are more consistently associated with Year I performance than are other subject areas (Group B). The data also indicated that marks in Grade 13 Group B subject areas are, in general, higher than in Group A subject areas. A 52 Lorraine G. Allan, Alexander L. Darling, Robert C. Hughes and Jack M. Rosenfeld Group A Average was determined for each student by excluding all marks in Group B subject areas, and it was noted that the Group A Average tended to be lower than the Final Admission Average which was based on courses in both Groups. The difference between the Group A Average and the Final Admission Average increased as the number of excluded Group B credits increased. These results suggest two possible courses of action. One would be for individual universities to change their admission requirements either by not accepting specified subjects for admission or by limiting the number of courses from a group or groups acceptable for admission. The second would be for the Ministry of Education to limit the number of Ontario Academic Courses to fewer than the current Grade 13 courses. On the second issue, the comparison of the Year I performance of students from different educational backgrounds, the samples for many of the Applicant Status categories were too small for statistical evaluation. Where comparisons were possible, the data suggested that no statistically significant differences exist in Year I performance among students from different educational backgrounds. As in previous studies, the Interim Admission Average and the Final Admission Average were highly correlated which suggests that the practice of granting early admission on the basis of interim marks does not create inequities, particularly since other students may be admitted on the basis of final marks at a later date. The correlation between Grade 13 Final Admission Average and Year I performance was examined; the correlations for the Science and Engineering programmes were higher than those for the Arts programmes, and the correlations for females were higher than those for males. Over the period from 1967 to 1981, the correlations for females have been decreasing; for males, no consistent trend is apparent. The results also indicate that Grade 13 marks continue to be a reasonable predictor of subsequent performance in Year I. Difficulties still exist in the use of this average as the principal criterion for selection because of variations in marking standards among secondary schools. Variability in such grading practices has been noted in previous studies, but this report was unable to address this issue for two reasons: (i) small sample sizes from individual secondary schools; and (ii) inability to measure adequately the quality of an entering student relative to that student's secondary school class. However, since some measures of academic achievement must continue to be used for university admissions, the development of the Ontario Assessment Instrument Pool, especially for the Ontario Academic Courses, may assist in the development of more consistent standards of grading. Another option might be the use of comprehensive examinations as has been announced in Alberta, where compulsory tests will be administered in four subjects. Regression analyses suggested that the admission average associated with a 60% average in Year I has been rising in some programmes. While this is not conclusive evidence of "grade inflation" in the secondary schools, it appears to be similar to the trends evident in the Provincial data on Ontario Scholars and pro- 53 An Examination of Performance of First Year Students at an Ontario University : An Admission Perspective portion of successful Grade 13 papers. Both percentages have steadily increased over the period from 1966-67 to 1979-80. These trends suggest that grade inflation has occurred in the secondary schools, and is similar to what has been reported elsewhere following the elimination of standardized examinations (Note 13). It is important that information such as contained within this report be utilized in the current evaluation of secondary school curriculum in Ontario. In order to ensure that these deliberations take into account current data, the replication of these investigations at other universities would be desirable. It is important that information such as contained within this report be utilized in the current evaluation of secondary school curriculum in Ontario, particularly in the case of the Ontario Academic Courses. REFERENCE NOTES 1. Annual Report of the Minister of Education, 1967. 2. Committee on Aims and Objectives of Education in the Schools of Ontario (E.M. Hall and L. Dennis, Chairmen), Living and Learning, Toronto: Department of Education, 1968. 3. The Secondary-Postsecondary Interface Project. Toronto: Ministry of Education and Ministry of Colleges and Universities, 1977. 4. Borankay, F.Y. Final Report for Business I 1969-70; Final Report for Engineering I 1969-70; Final Report for Humanities I 1969-70; Final Report for Natural Sciences I 1969-70; Final Report for Social Sciences I 1969-70. Series of unpublished reports, McMaster University, 1971. 5. Darling, A.L. The Academic Performance of Students Admitted to an Ontario sity. Unpublished M.Ed. Thesis, The University of Alberta, Edmonton, 1976. 6. Report 1981. of the Secondary of Secondary Education Education. Review Project. Univer- Toronto: Ministry of Education, 7. The Renewal 8. Fleming, W.G. Personal and Academic Factors as Predictors of First-Year Success in Ontario Universities. Toronto: The Ontario Institute for Studies in Education, 1959. The Use of Predictive Factors for the Improvement of University Admission Requirements. Toronto: The Ontario Institute for Studies in Education, 1962. Characteristics and Achievement of Students in Ontario Universities. Toronto: The Ontario Institute for Studies in Education, 1965. Toronto: Ministry of Education, 1982. 9. Khan, S.B., Ransom, P., & Herbert, M. Prediction of First-Year Achievement in Ontario Universities. Toronto: The Ontario Institute for Studies in Education, 1970. Khan, S.B., & Rickard, S.A. The Prediction of Academic Achievement in Ontario Universities. Toronto: The Ontario Institute for Studies in Education, 1971. Khan, S.B., & Rickard, S.A. The Prediction of University Achievement in Ontario Universities. Toronto; The Ontario Institute for Studies in Education, 1971. 10. Traub, R., Wolfe, R., Wolfe, C., Evans, P., & Russell, H.H. Secondary-Postsecondary Interface Project II; Nature of Students, Vol. I. Toronto: Ministry of Education and Ministry of Colleges and Universities, 1977. 11. Council of Ontario Universities. Experimental Summary Report. Toronto, 1979. Achievement Testing Programmes- 54 Lorraine G. Allan, Alexander L. Darling, Robert C. Hughes and Jack M. Rosenfeld 12. Reports of the Ministry of Education, 1966-67 to 1970-71. Education Statistics Ontario, 1972-73 to 1980-81. 13. Mehra, N. High School Grades and University Performance. Edmonton: Office of Institutional Research and Planning, the University of Alberta, 1977. REFERENCES Oksanen, E.H., & Spencer, B.G. On the Determinants of Student Performance in Introductory Courses in the Social Sciences. The American Sociologist, 1 9 7 5 , 1 0 , 103-109. FOOTNOTES 1. The courses in Group A were English, Other Languages, Biology, Chemistry, Physics, Economics, Geography, History, Core Mathematics. 2. The courses and course groupings in Group B were Family Studies, Music, Physical Education, Accounting, Media Communication, Mathematics of Investment, Secretarial Science, Computer Science, Social Sciences, Theatre Arts, Technical Training, Statistics, Other Mathematics.