The Canadian Journal of Higher Education, Vol. XXIV-141, 1994 La revue canadienne d'enseignement supérieur, Vol. XXIV-141, 1994 First Year Science in a Commuter University: Where to Intervene J. PAUL GRAYSON* Abstract A study of first year science students in a large Canadian commuter university shows that integration/involvement variables are important in explaining a number of desired outcomes. Most evident is the fact that students who were satisfied with the quality of instruction, and students who felt that topics covered in classes were important to future career success, were more likely than others to score high on a number of desired outcomes. By way of comparison, variables falling in the social integration/involvement category, by and large, were not important in explaining desired outcomes. For example, in contrast to previous findings on factors affecting student development, the number of outof-class contacts with faculty had no impact on any of the outcomes under consideration Overall, the findings, and possible avenues of improvement that they suggest, reflect the fact that in a large commuter university classroom contact represents the main link between the institution and the individual. As a result, it is likely that improvements in curriculum and teaching would have large payoffs for both students and the institution. The university under study is a commuter university in a large city with large first year science classes. Where similar conditions can be found in other Canadian universities, it is likely that various outcomes can be explained by reference to similar factors. By extension, it is equally probable that changes likely to enhance outcomes would also lead to improvements elsewhere. * Institute for Social Research, York University First Year Science in a Commuter University: Where to Intervene 150 Résumé D'après une enquête menée dans une grande université canadienne auprès d'étudiant(e)s de première année en sciences, on constate que les variables intégration/participation son déterminantes afin d'obtenir certains résultats souhaités. De toute évidence, les étudiant(e)s qui sont satisfait(e)s de la qualité de l'enseignement et ceux et celles qui croient que les sujets abordés en classe amélioreront leurs chances de succès pour leurs futures carrières sont plus susceptibles que d'autres d'atteindre de hautes performances quant à certains résultats souhaités. En comparaison, les variables qui tombent dans la catégorie sociale participation/intégration ne sont pas vraiment importantes pour expliquer les résultats escomptés. Par exemple, en contraste avec les découvertes antérieures sur les facteurs qui modifient le développement des étudiant(e)s, on a constaté que le nombre de contacts hors-classes avec les professeur(e)s n'avait aucun impact sur les résultats à l'étude. En général, les résultats de cette étude et les possibilités d'amélioration qui en découlent, démontrent que dans une grande université, les contacts établis en classe représentent le lien principal entre l'institution et l'individu. En conséquence, il est probable qu 'une amélioration des programmes scolaires et de l'enseignement bénéficierait grandement à la population étudiante ainsi qu'à l'institution. L'université qui a servi de modèle pour cette enquête est située dans une grande ville et elle comporte de larges classes de première année en sciences. Si l'on retrouvait des conditions semblables dans d'autres universités canadiennes, il est probable que les résultats attendus des étudiant(e)s s'expliqueraient à partir de facteurs similaires. On peut donc déduire qu'il est également probable que des changements qui donneraient des résultats positifs ici puissent donner lieu à une amélioration ailleurs. Introduction In Canada as elsewhere, a number of recent reports have focused on the quality of undergraduate education in general (Smith, 1991) and science education in particular (Human Resource Development Committee, 1991). While there is no doubt that in many areas Canadian universities are doing a splendid job, many would argue that there is much room for improvement. Areas frequently targeted for improvement include attrition rates, teaching, the encouragement of female students in areas such as science, and the graduation of students with demonstrable generic skills. 18 J. Paul Grayson If it is evident to some that there are problems with undergraduate education in Canada, it is apparent to others that universities have not turned their research efforts inward (Dennison, 1992; Smith, 1991). For example, we know little with regard to the types of university experiences in Canada that are most conducive to the attainment of various educational goals. As a result, many changes intended to improve the teaching/learning environment in Canadian universities are based on intuition or imitation of innovations elsewhere. Frequently, the latter themselves are based on feeling rather than research. Moreover, where changes are introduced, it is seldom that their effects are systematically measured to determine if the objective of the change has been met. When research on the impact of universities on students is undertaken, it is useful to distinguish among three major sets of factors: pre-entry characteristics of students; environmental factors; and outcomes (Astin, 1991). Included among pre-entry characteristics are family income, parental education level, high school attainment, gender, and so on. Each of these may be conducive to university success. For example, certain institutions may graduate students who later have successful careers not because of the university's impact, but because the university attracts students from high socio-economic backgrounds who participate in networks useful to career success. Environmental factors include curriculum, peer climate, and the degree to which the institution facilitates the social and academic involvement of students. Holding pre-entry characteristics constant, environmental factors may have implications for certain university outcomes. For example, two universities may have different effects on students similar in terms of their prior academic achievement and social background. It is these effects that are important in assessing the relevance of the university experience. Outcomes can be defined as, "those aspects of the student's development that the institution either does influence or attempts to influence through its educational programs and practices" (Astin, 1991, p. 38). For current purposes, outcomes can be divided into two kinds: a. final outcomes that may be the ultimate objectives of an institution with regard to student development; b. intermediate outcomes that may be contributory to final outcomes, but are themselves interim products of the student-environment interaction. Matters such as learning and intellectual development are obvious desired university final outcomes. Less obvious, but, it can be argued, equally desirable, are outcomes such as satisfaction with academic programs, value changes, the intent to return to the institution to complete a degree, and so on. Interim outcomes would include products such as term marks.1 First Year Science in a Commuter University: Where to Intervene 150 Consistent with the foregoing approach to the study of university outcomes, in order to understand the impact of the university on its students better, and to facilitate change where needed, in 1991, York University initiated a program of student studies. The university, located in Metropolitan Toronto, is a large, multi-racial and multi-ethnic university, with 1992-93 full-time and part-time undergraduate enrollments of 25,235 and 14,402 respectively. Among its fulltime undergraduate population, approximately 10% of students live on campus while 77% live at home with their parents. Although the University offers programs in many diverse areas, roughly 72% of all full-time undergraduate students are enrolled in the Faculty of Arts. Those in science represent only 8% of the full-time undergraduate population. As is the case in many other universities, first year science students are required to take a core curriculum taught in very large classes and laboratories. It is important to note that York is located on the suburban fringe of Metro Toronto. As a result, because few students live on campus, large numbers spend a considerable part of their day travelling. For example, 38% of students spend 1.5 hours or more per day travelling to and from school. In addition, because there is no residential settlement to either the north or east of the campus and no 'student spots' such as pubs and restaurants close to the University, York is less integrated into the local community than many Canadian universities. These points are emphasized for the reason that residence on, or close to, a university campus makes possible greater involvement in various aspects of university life than living a considerable distance from campus. Moreover, readily accessible off-campus 'student spots' frequently provide welcome alternatives and/or supplements to services offered on campus. By way of comparison, when commuting times are long, and the availability of readily accessible offcampus services, etc., is limited, it is likely that the university will be required to take extra steps to achieve outcomes that may be expected of universities in general. In the first year of this research, the investigation at York involved three surveys of full-time first year science students and the holding of focus group meetings to coincide with the surveys. The first survey was conducted on the second day of classes before impressions of the university crystallized; the second, in November after an anticipated initial period of adjustment; and the third, in late February and early March. In the following pages attention will focus on the outcomes of first year science at York; the processes that contributed to the outcomes; and an identification of areas for intervention that likely would enhance desired outcomes. In conclusion, it will be argued that Canadian 20 J. Paul Grayson science programs similar to those at York, in comparable commuter universities, might profit from the same interventions. The Commuter University The characteristic of York that is of major concern in this study is its status as a commuter university. As noted by Pascarella and Terenzini (1991, p. 632), even in the United States, where far more research has been conducted on university students than in this country, commuter students have received short shrift: by and large, generalizations and theories relating to the effect of post secondary education have been based on examinations of traditional students (young, fulltime) at residential institutions. An appreciation of the problems encountered by students in a commuter university must begin with the recognition that student learning and development is a function of at least six factors: • Maturation. • The formal curriculum (what is learned in class/labs and readings). • Interactions with faculty outside of class. • Participation in non-mandatory academic activities such as special seminars/lectures. • Interactions with students in organized activities such as clubs, sports, and so on. • Interactions with students in informal activities such as talking before and after class, visits to pubs, etc. Participation in the formal curriculum, interactions with faculty, academic achievement, and participation in academic activities can be viewed as academic integration/involvement. Organized or informal associations with students can be identified as social integration/involvement. 2 Although many US studies of university attrition have conceptual and methodological limitations, and the results of some cannot be applied to Canada without qualification (Corman et al, 1992), in the United States a considerable body of research shows that various aspects of academic and/or social involvement are important to attrition and other university outcomes. For example: • Several studies have indicated that students displaying the greatest degree of cognitive development: see faculty as concerned with teaching and students' development; tend to have developed an intellectual relationship with at least one faculty member; and state that peer interaction importantly affected their development (Endo & Harpel, 1983; Pascarella, etai, 1983a; Volkwein, etal., 1986). First Year Science in a Commuter University: Where to Intervene 150 • Even after they controlled for pre-entry characteristics, such as level of high school achievement, Pascarella and Terenzini (1976, 1977, 1979a, 1979b) found that first year persistence was related to faculty student contact outside of class, particularly to contact dealing with intellectual concerns. • Based on a study of 1,064 students from 27 small liberal arts colleges Centra and Rock (1971) concluded that frequent student-faculty interactions, and the perception that faculty were interested in teaching and treating students as individuals, contributed to the development of an effective learning climate. • In a study involving 27,065 science students enrolled in 388 colleges and universities in the United States, Astin and Astin (1992) found that out-of-class interaction with faculty had important implications for degree aspirations and satisfaction with faculty and curriculum. Despite these findings it should not be assumed that the effects of academic and social involvement are uniform. For example, after examining several studies, Pascarella and Terenzini (1991, p. 411) conclude that academic integration, measured by grades, intellectual development, and faculty interaction, is most influential for persistence and degree attainment for students with low social integration. With increased social integration, however, the importance of academic integration decreases. Similarly, Pascarella and Terenzini (1979a) found that informal contact with faculty to deal with intellectual matters, and the reported quality of the contacts, were most influential in explaining the persistence of students with initial low levels of commitment to graduation or who had parents with low education. In essence, faculty contact may assist two potentially at risk groups: those who are not socially involved and those who have a low commitment to graduation and/or parents with low education. Although the formal and informal components of academic and social integration/involvement may be contributory to certain university outcomes, commuter students, because of the relatively limited time spent on campus, may have less opportunity than students at residential universities to engage in activities other than those associated with the formal curriculum. As a result, it might be reasonable to assume some less desirable outcomes from commuter as compared to residential universities. This expectation was borne out by a classic study by Chickering (1974). In an examination of students from 270 institutions of higher learning in the United States, he noted that commuter students ranked themselves relatively lower than others on public speaking and on academic, writing, artistic, and 22 J. Paul Grayson leadership skills. Moreover, after controlling for pre-entry characteristics, it was found that living at home was inversely related to the social confidence of students. Similarly, Astin and Astin (1992, pp. 7-9), in a large study of U.S. science students, discovered that, "students who live at home or in private off-campus housing are also likely to decrease their [degree] aspirations." The general thrust of Chickering's and the Astins' research is supported by other studies (see Astin, 1977; and Welty, 1976). By way of explanation for observations such as these, Pascarella and Terenzini (1991, p. 402) argue that, "[t]he commuter institution's social system may simply not be potent enough to play more than a relatively trivial role in the persistence or educational attainment process." Conversely, they add, "one would expect academic factors, such as academic achievement, to be relatively important." In this expectation they are backed up by a number of studies. To be more explicit, in the United States, many investigations show little relationship between social integration and phenomena such as persistence at commuter institutions, even after controls have been introduced for pre-entry characteristics of students (Braxton & Brier, 1989; Fox, 1986; Garcia, 1988; Williamson & Creamer, 1988). By way of comparison, academic achievement has been found to account for, among other things, persistence at the same type of institution (Fox, 1986; Garcia, 1988; Pascarella, et al., 1983b). In Canada, the results of a study of 3,817 commuter students in a community college point in the same direction. Dietsche (1990) found that academic integration and educational commitment were more important in accounting for persistence than social integration and institutional commitment. To summarize, in residential institutions, academic and social involvement, in different ways for different students, contributes to desired outcomes such as persistence and academic attainment. However, in commuter institutions, like York, the possibility of social involvement is reduced; moreover, the relationship between social involvement and matters such as persistence is tenuous. By way of contrast, in commuter institutions, academic achievement has important implications for certain outcomes. Survey Characteristics In 1992, the student studies initiated at York were designed in such a way that the relative impact of pre-entry characteristics, intermediate outcomes, and environmental variables on 498 entering science students could be assessed. Data relevant to the investigation were collected from surveys, focus group First Year Science in a Commuter University: Where to Intervene 150 meetings, and administrative records. Among the total sample, 57% of students were male; the mean high school average in the final year was 77%; and 31% considered themselves to be members of a visible minority group. The percentages of those responding to surveys conducted on the second day of classes, in mid-November and in late March and early April were 89%, 84%, and 68% respectively. Among the respondents for each survey, approximately 70% to 80% were willing to provide student ID numbers so that their responses to various surveys could be compared; unfortunately, it was not always the same students who provided information from one survey to the next. As a result, while the overall response rate to each survey was acceptable to high, only 141 students provided identification across all three surveys. As a result, for purposes of this report, it was decided to rely primarily on the results of survey three. Data collected in surveys one and two will be referred to when they can shed light on the first year outcomes under consideration. First Year Outcomes In examinations of university outcomes, attention is frequently restricted to academic attainment; however, among others, Gilbert (1989) has argued that assessments of the university experience should be based on a number of wideranging outcomes. Moreover, Gilbert and Evers (1991) have shown that university students in part attribute development in a number of areas to their university experience. In the area of skills, for example, students most frequently credit the university experience for the development of thinking and reasoning skills; problem solving skills; planning and organizing skills; time management skills; the ability to conceptualize; learning skills; and quantitative mathematical and technical skills (Gilbert & Evers, 1991, p. 74). Such abilities may or may not be reflected in marks. At York, a number of changes that occurred from one survey to the next, as well as a specification of certain outcomes of the first year science experience, have been analyzed elsewhere (Grayson, 1993a). Although important to the current analysis, the results of this study will not be repeated here. Suffice it to say that for some final outcomes it was possible to make comparisons between measures in the final survey and earlier surveys after adjustments had been made for gender and high school marks; for others, it was appropriate only to examine outcomes as measured in the final survey without controlling for gender or high school marks. Some of the outcomes included in the analyses were chosen for study because of their embodiment in the University's Academic Plan. Others were 24 J. Paul Grayson selected for examination because of their frequent mention in academic literature or in policy documents. Still others were examined as a result of particular challenges faced by York with respect to gender and the visible minority status of a large number of students. The outcomes dealt with, and the surveys used in their measurement, are as follows: • Satisfaction with marks (November, March). • Satisfaction with the science program (November, March). • Beliefs that females, minority students, and gays should be treated in the same way as other students (September, March). • Conformity (September, March). • The extent to which students prefer being taught over learning on their own (September, March). • Assessments of the importance of a degree in general and a York degree in particular (September, March). • Assessments of the extent to which students feel that their knowledge of science increased over the academic year (March). • Assessment of an increase in intellectual development (March). • The likelihood that the student will return to York the following year (March). • The likelihood that the student will return to science at York (March). Outcomes for which information on more than one survey was useful are documented in Graph 1 (see Grayson, 1993a, for more detailed analysis). For these, scores were standardized across surveys, and then comparisons were made between surveys after adjustments had been made for gender and high school marks. Had there been no change from one survey to the next, means for each survey would have been zero. The horizontal axis of the graph indicates the difference in percentile points between means of the appropriate surveys.3 The vertical axis indicates, in descending order of change, phenomena that can be treated as outcomes of the first year experience. The graph shows that a tolerance for females and preference for learning by one!s self, have the greatest increase between September and March. An increase in tolerance for minorities and gays between September and March is next in magnitude. Although there were slight changes in satisfaction with marks between November and March, differences are not statistically significant. Change is evident in the assessed importance of a York degree, and a university degree, between September and March. This time, however, the change First Year Science in a Commuter University: Where to Intervene 150 Graph 1: First Year Outcomes (September, November, February Surveys) Outcomes T o l e r a n c e Females S e l f Taught Tolerance Minorities T o l e r a n c e Gays Satisfaction Mark Conformity Satisfaction Science Imp. D e g r e e Imp. York D e g r e e -15 - 10 -5 0 5 10 P e r c e n t i l e Points Between M e a n s Graph 2: First Year Outcomes (February Survey Only) More Knowledge Sci. Intellect. Developed Return to York Return to Science 15 26 J. Paul Grayson is negative: in September more students believed that a York degree, and a degree, were important than in March. The slight negative difference between November and March with regard to satisfaction with science is not statistically significant. The only measure for which there was absolutely no change from one survey to the next was conformity. In March as in September, on a five point scale in which students were asked to disagree or agree (with a score of 1 for strongly disagree and 5 for strongly agree) with a statement focusing on the extent to which they usually tried to do and say what others expected, the mean score was 2.79. As a result, the difference between the means was zero. In the graph, a minimal value was assigned so that the variable is visible. Outcome measures for which only the March observation is relevant are outlined in Graph 2. From the graph, it is clear that the majority report both an increase in their knowledge of science and an increase in intellectual development over their first year. In addition, the vast majority express the intent to return to York and to science in the following year. It is clear that the first year experience in science at York had both positive (increase in tolerance, increased knowledge of science, etc.) and negative (decreased importance of a university degree and a York degree) outcomes. What is not evident from the data are the pre-entry characteristics and environmental factors responsible for various outcomes. By extension, it is not known what interventions might result in increases in desired outcomes. These matters are the focus of the current article. Variables for Analysis The variables initially chosen for potential explanation of the outcomes detailed above were selected in terms of one or more of: • Their pre-entry and environmental status as discussed by Astin (1991). • Their theoretical relevance as discussed earlier. • Their being raised in focus group interviews with students or in written comments on questionnaires. • The logic of their inclusion. The variables used in analysis, as well as response ranges, mean scores, and standard deviations, are outlined in Tables la to lc. 4 The single pre-entry characteristic that will be examined here is average grade in the final year of high school - Table la. Items listed under intermediate outcomes - Table lb - were First Year Science in a Commuter University: Where to Intervene 150 included because of the logical possibility that they could affect outcomes such as satisfaction with marks. Those listed under environmental factors - Table lc - include matters of social and academic integration/involvement as discussed earlier. While it is not necessary to discuss the contents of Tables la to lc in detail, a few general observations are in order. First, as seen from Table lc, variables focusing on the helpfulness of professors, the interest of student advisors, the relevance of course topics, and satisfaction with instruction and class size, score no higher than 2.94 on a 5 point scale. Although it is somewhat of a simplification, these figures do not suggest high evaluations on the part of students. Second, the numbers of monthly out-of-class contacts with faculty and teaching assistants (TAs) or lab demonstrators, with means of 1.55 and 1.04 respectively, are quite low. The mean scores for the number of campus organizations and sports participated in, the number of sports watched, and the number of pub visits, indicate a low level of student engagement in these activities. By way of comparison, students have made on average 14 new friends since September. Overall, however, there appears to be a low level of informal interaction with faculty and TAs, and a low degree of involvement in formal social activities. While similarities do not exist on all dimensions, these findings are nonetheless generally consistent with those of a study of first year experiences at the University of Alberta, another large Canadian commuter university. For example, in the latter, Holdaway and Kelloway (1987) emphasize that there is little involvement in extracurricular physical education and academic and cultural events; nor is there much involvement in student clubs. Stepwise regression analysis was used to determine the impact of each of the variables in Tables la to lc on individual outcomes described in Graphs 1 and 2.5 For variables described in Tables la and lb values used in the regressions ranged from 1 for the lowest category to the appropriate value for the highest category (e.g., 8 for grades in final year of high school). After pairwise deletion, the minimum number of students available for analysis was 233; the maximum, 336.6 Utilizing stepwise regression facilitates a determination of the impact of the environmental variables controlling for pre-entry characteristics and intermediate outcomes. For example, when examining satisfaction with the science program, the procedure enables a consideration of the influence of environmental variables after considering the effect of high school marks, Fall marks, and hours spent on studying, each of which might be confused with the effect of environmental variables. In addition, the procedure enables a determination of which among the environmental variables statistically contributes to the 28 J. Paul Grayson Table la Variables in Analyses - Pre-entry characteristics Count Average grade in final year high school 50-54 55-59 60-65 66-69 70-74 75-79 80-85 86+ Total Col % 1 1 2 15 59 92 79 84 .3% .3% .6% 4.5% 17.7% 27.6% 23.7% 25.2% 333 100.0% Table lb Variables in Analyses - Intermediate Outcomes Count Col % 38 59 56 60 50 29 20 22 11.4% 17.7% 16.8% 18.0% 15.0% 8.7% 6.0% 6.6% 334 100.0% Average fall term grade this year F E D D+ C C+ B B+ A A+ 2 3 20 31 69 52 61 46 34 7 .6% .9% 6.2% 9.5% 21.2% 16.0% 18.8% 14.2% 10.5% 2.2% Total 325 100.0% Hours per week on studies outside class < 10 hrs. 10-14 hrs 15-19 hrs. 20-24 hrs. 25-29 hrs. 30-34 hrs. 35-40 hrs. >40 hrs. Total First Year Science in a Commuter University: Where to Intervene 150 Table lc Variables in Analyses - Environmental Factors Range Monthly contacts profs outside class Monthly contacts TAs outside class Profs went out of way to be helpful Advisors took interest in students Number of non-required academic activities over two months Topics covered important future career success Satisfaction instruction quality Satisfaction class size No. of campus organizations member of No. of on-campus sports participate in No. of sports events watched since Sept. No. of weekly pub. visits No. of new friends since Sept. Note: Mean 0-6+ 0-6+ 1-4 1-4 1.55 1.04 2.59 2.47 0-4+ .75 1.21 2.86 2.89 2.94 .69 .43 .49 .40 14.11 .75 .99 1.14 .87 .70 .96 .68 9.11 1-3 1-4 1-4 0-3+ 0-2+ 0-3+ 0-2+ 0-36+ Std Deviation 1.97 1.56 1.16 1.13 A '+' following a maximum value indicates that values above those recorded were truncated to facilitate reasonable analysis. outcome under consideration. This latter benefit facilitates decisions regarding the relative importance of academic and social involvement at a commuter university as discussed previously. These benefits will become more apparent once analysis is under way. It might be emphasized that the primary concern of the analysis is not to understand with a great degree of precision the impact of pre-entry characteristics, intermediate outcomes, and environmental factors, on each of the relevant outcomes. Instead, the interest is in identifying any general explanatory patterns that emerge in examining a number of different outcomes. Should it be found, for example, that certain factors, such as the helpfulness of professors, are important in explanations of a variety of outcomes, it can be assumed that such factors represent areas in which improvements might be made with beneficial results for student development. A decision as to whether or not such changes can be made, and their ultimate impact, is beyond the scope of the study. 30 J. Paul Grayson Expected Marks It has been shown elsewhere that although students entered science in September with high expectations regarding the marks they could achieve, by the November survey their expectations had undergone drastic revision (Grayson, 1993a). The results of a regression analysis carried out in the way described earlier, as summarized in Table 2, shed light on factors that help explain the expected marks of students as measured in the March survey. For Table 2 and subsequent tables, the information in parentheses beneath the table title represents the minimum and maximum values of the outcome variable under consideration, the mean, and the standard deviation. In this case, expected marks could have a minimum value of 1 and a maximum value of 10; the mean score is 6.53 (corresponding to a letter grade of C); and the standard deviation is 1.60. Variables that have been included in the regression by the stepwise procedure, and their associated betas, are outlined in the body of the table. (Although they will not be discussed, regression coefficients have also been included in the tables.) Additional information on variables was documented in Tables la to lc. 7 As can be seen from Table 2, the beta for high school marks is (.05). (Parentheses indicate that the variable made a statistically significant contribution when first entered, but not in the final equation. For all intents and purposes, the implications of this fact can be ignored in the current analysis.) This means that for each increase of one standard deviation in high school marks, an increase of .05 of a standard deviation can be expected in anticipated first year marks. In essence, high school marks have a small impact on expected final first year marks.8 By way of comparison, the beta for Fall marks is .73, indicating a greater effect than high school marks on expected final marks. The betas for academic activities engaged in and satisfaction with instruction are .14 and .10 respectively. These values suggest less of an effect on expected final marks than Fall marks had. In total, as seen from the note at the bottom of the Table, the variables in the regression explain 66% of the total variance in expected final marks. In terms of the objectives of the current analysis, the most important information is provided by the betas for the two environmental variables (what goes on in the university), academic activities engaged in and satisfaction with instruction. As noted above, because in regression information on any one variable assumes constancy of all others, we can see that even after the effects of high school marks (a pre-entry characteristic) and Fall marks (an intermediate First Year Science in a Commuter University: Where to Intervene 150 Table 2 Expected Marks (range 1 to 10; Mean = 6.53; SD = 1.60) b High School Marks Fall Marks Academic Activities Satisfaction Instruction Beta .06 .63 .19 .16 (.05) .73 .14 .10 Explained Variance = 66% Table 3 Satisfaction Marks (range 1 to 5; Mean = 2.45; SD = 1.17) b High School Marks Fall Marks Academic Activities Satisfaction Instruction Beta -.08 .38 .23 .13 (-.08) .61 .19 .12 Explained Variance = 46% outcome) are considered, the two environmental variables still have an impact on expected final marks. As a result, if changes could be introduced that would increase academic activities and/or satisfaction with instruction, there could be a marginal payoff in terms of expected final marks. It must be noted, however, that there is no guarantee, for two reasons, that such changes would have the desired result. First, suggestions for changes are based on an examination of the existing situation. It is quite possible that changes that seem logical from what is currently known would have unanticipated consequences because of their impact on important emergent factors. Second, it is not known whether or not changes are logistically possible. In essence, suggestions advanced represent best guesses for improvement based on an examination of the existing situation. Satisfaction With Marks Although satisfaction with marks, like satisfaction with other measures that will be dealt with later, can be seen as a desirable outcome of the university experience, data presented earlier in Graph 1 indicated little change in satisfaction 32 J. Paul Grayson with marks between November and March. Information pertaining to satisfaction with marks in the latter survey is presented in Table 3. It is evident from the Table that there is a slight inverse relationship between average mark in the final year of high school and satisfaction with marks in university (beta = -.08): the higher the high school average, the lower the satisfaction with marks in university. This relationship may be explained by the fact that in university marks are lower than in high school; as a result, those who have high marks in high school may be particularly dissatisfied with lower levels of achievement in university. Consistent with this explanation is the fact that high Fall marks translate into high satisfaction with university marks. Indeed, the beta for Fall marks, .61, is higher than others. More importantly, the figures also show that even when high school and Fall marks are held constant, two academic involvement variables, satisfaction with instruction and the belief that advisors are interested in students, with respective betas of .19 and .12, help explain satisfaction with marks. The total variance explained by all variables is 46%. With regard to measures that might be taken to enhance the outcome under discussion, it can be argued that increases in satisfaction with instruction, that might be attained through enhanced teaching methods, could be of consequence. It is equally clear that increasing the interest of faculty advisors in students might have a similar effect. Satisfaction With Science Earlier it was shown that between November and March there was a very slight, not statistically significant, decrease in satisfaction with the science program at York. Information relating to overall satisfaction with the science program as measured in the March survey can be found in Table 4. Initially most noticeable is that the pre-entry characteristic of average grade in the final year of high school did not enter the equation. Fall marks, an intermediate outcome, are included, with a comparatively modest beta of .15. Perhaps most evident is that the .37 beta for an assessment that classroom topics are important to future career success is the highest in the equation. As with the previous two outcomes studied, satisfaction with the quality of instruction (beta = .31) also is of importance in explaining satisfaction with the science program. The beta for the helpfulness of professors is a lower .15. In total, variables in the equation explain 45% of the variance. To mention only variables that have not previously entered the regressions, an increase in courses of topics relevant to career success, as well as increased First Year Science in a Commuter University: Where to Intervene Table 4 Satisfaction Science 150 (range 1 to 5; Mean = 3.36; SD = 1.04) b Fall Marks Satisfaction Instruction Topics Important Professors Helpful Beta .08 .32 .51 .14 .15 .31 .37 .15 Explained Variance = 45% helpfulness of professors, has the potential to affect satisfaction with science positively. Increases in Tolerance It was shown in Graph 1 that tolerance increased substantially from September to March. Nonetheless, none of the variables under consideration helps explain tolerance toward females, members of visible minority groups, or gays. Such findings are consistent with the point made in an earlier report that increases in tolerance may not be related to participation in the science program; instead, they may be a function of general liberalism encountered in a university environment (Grayson, 1993a). In terms of the thrust of this report, such conclusions suggest that there may be little that can be done in the science faculty to increase tolerance. To put this comment in perspective, it must be noted that tolerance was relatively high to begin with. Preference for Being Taught In a university context, it is desirable that students become independent learners. In fact, in a study of CEGEP's students in Quebec, Bateman and Donald (1988) show that students who accept responsibility for their own learning are more successful academically than others. Accordingly, a preference for learning on one's own rather than being taught can be regarded as a positive outcome. Moreover, as was shown in Graph 1, at York, there was an increase in preference for learning on one's own between September and March. A factor that helps explain being taught, the opposite of learning on one's own, is identified in Table 5. 34 J. Paul Grayson Table 5 Preference For Being Taught (range 1 to 5; Mean = 3.18; SD = 1.21) b Satisfaction Instruction Beta -.21 -.17 Explained Variance = 3% Table 6 York Degree Important (range 1 to 4; Mean = 1.53; SD = .78) b Satisfaction Instruction Beta .13 .16 Explained Variance = 3% Evidentially, only satisfaction with instruction has implications for preference for being taught; however, the beta of -.17 suggests that students who are satisfied with their instruction are inclined to prefer the opposite to being taught; that is, they prefer learning on their own. This single variable, however, explains only 3% of the variance. In addition, it is difficult to specify the direction of causality. Are those who prefer being taught less inclined to be satisfied with instruction or does being dissatisfied with instruction promote a preference for being taught? Importance of Degrees Earlier it was noted that there was a decline from September to March in terms of students' assessments of the importance of a university degree in general, and a York degree in particular. This fact notwithstanding, none of the variables chosen for analysis helps explain a decline in the assessed importance of a degree. At best it can be suggested that the decreased assessments of the importance of obtaining a degree may be a function of a poor economic climate and/or unrealized expectations regarding university life. As shown in Table 6, only one variable contributes to an explanation of the importance of a York degree: satisfaction with instruction (beta = .16). The greater the satisfaction, the more highly rated the importance of a York degree; however, satisfaction with instruction explains only 3% of the variance. First Year Science in a Commuter University: Where to Intervene Table 7 Increase Science Knowledge (range 1 to 5; Mean = 3.76; SD = 1.09) b Fall Marks Topics Important Satisfaction Instruction Academic Activities 150 Beta .08 .43 .13 .30 .23 .12 .26 .11 Explained Variance = 24% Increase in Science Knowledge In an earlier section, it was seen that a majority of students reported that independent of marks they had learned a considerable amount of science in the past year. The data in Table 7 indicate that a number of factors are important to an understanding of this outcome. To begin, with a beta of .13, Fall marks contribute to assessments of increases in knowledge of science. More important is that when Fall marks and other measures in the equation are controlled, a perception that topics covered in class are relevant to career success has a relatively large beta of .30. Although the betas are smaller, other academic involvement measures, satisfaction with instruction and involvement in informal academic activities, with respective betas of .23 and .12, each affect increases in knowledge. Overall, the variables included in the equation explain 24% of the variance in increase in knowledge of science. Intellectual Development As noted in an earlier section, data collected in the March survey indicate that a majority of students believed that in the first year of the science program they had developed intellectually. Some of the factors that contributed to such development are outlined in Table 8. Not surprisingly, Fall marks, with a beta of .19, were directly related to perceptions of intellectual development. Independent of Fall marks, it is also evident that satisfaction with the quality of instruction, with a beta of .28, helps explain intellectual development. The total variance explained by these two variables is 37%. 36 J. Paul Grayson Table 8 Developed Intellectually (range 1 to 5; Mean = 3.60; SD = .99) b .10 .28 Fall Marks Satisfaction Instruction Beta .19 .28 Explained Variance = 37% Table 9 Likely Return to York (range 1 to 6; Mean = 5.32; SD = 1.00) b .29 .20 Topics Important Satisfaction Instruction Beta .22 .20 Explained Variance = 1 1 % Table 10 Likely Return to Science (range 1 to 6; Mean = 5.29; SD = 1.11) b Fall Marks Topics Important Professors Helpful Beta .07 .37 .16 .13 .25 .17 Explained Variance = 1 3 % Return to York, Return to Science Information discussed earlier indicated that the vast majority of students included in the March survey intended to return to York and science in the following year. Information presented in Tables 9 and 10 suggests that once again measures might be introduced that could have some impact on retaining students in science at York. The clear observation from Table 9 is that the only variables that help explain the likelihood of returning to York can be placed in the academic involvement category. The variable with the highest beta, .22, is a perception that topics taught in class are relevant to future career success. Satisfaction with instruction has a comparable beta of .20, and the overall variance explained by the two variables is 11%. First Year Science in a Commuter University: Where to Intervene 150 Information in Table 10 indicates that some existing success in the science program, as measured by Fall marks (beta = .13), contributes to the likelihood of returning to science at York the next semester. In addition, the importance of course topics, and beliefs that professors have been helpful, have respective betas of .25 and .17. In total, however, the three variables explain only 13% of the variance. A Summary Measure It was stated earlier that the prime objective of this report was not to examine in detail the factors contributing to particular outcomes. Instead, the objective was to identify factors that might affect a number of outcomes. Once such factors are identified, it would be possible to introduce changes that might contribute to a realization of desired outcomes. Information has been presented in Table 11 in a way that should facilitate an overall understanding of general relationships among pre-entry characteristics, intermediate outcomes, environmental variables, and outcomes. The Table provides an overview of the preceding regression analyses. Outcomes are listed across the top; potential independent variables to the left. A '+' indicates a positive relationship between the independent variable and the outcome; ' ( ) ' indicates a negative relationship. To begin with the pre-entry characteristic, it is evident that average marks in the final year of high school helps explain only two outcomes. The intermediate outcome, Fall marks, is important to an understanding of five outcomes; hours of study is relevant to none. These findings suggest that initial academic success at York is more important to a number of outcomes than academic achievement in the final year of high school. Variables that contribute to an explanation of the most outcomes fall in the academic involvement category. More specifically, satisfaction with instruction helps explain 8 outcomes; an assessment that class topics are important to career success, 4; beliefs that professors are helpful, 2; involvement in nonrequired academic activities, 2; and a perception that advisors are interested in students, 1. Among other variables that fall into the academic involvement category, out-of-class contact with professors and TAs and satisfaction with class size, help explain none of the outcomes. The first of these deserves special consideration for the simple reason that in other studies out-of-class encounters with faculty have been found to be extremely important in accounting for desirable outcomes. By way of explanation for the non-importance of this variable in 38 J. Paul Grayson a U H D at Z a wh d n sz O U J > L I I J O I S O 0 S U 4 - Z H U J J W U H Z Qwooiww |S + + + + + -So- Q U O R U M « N. 0 ! UJ U. W AI + + H < D O K H O- O H O A U O I S — hOJW« 3 <> »K u. w g < j w H o S O + + + >-ooiNi I » U - ><Oa:ui + M U - B Z U U CO < J I- w < (uxa. U B: Z O O : C O U - M Z U U §<«¡¿1/3 + + 3 + + + + + + + + + First Year Science in a Commuter University: Where to Intervene 150 the analyses described here, it can be hypothesized that at York out-of-class contact occurs primarily to deal with academic problems. Important informal contacts of a more positive nature that have been found to contribute to desirable outcomes may not occur. Were they to develop, it is likely that they would positively contribute to desired outcomes. Consistent with the literature on commuter universities, social involvement variables are important to an explanation of no outcomes. Once again, however, it may be that at York such involvement is at too low a level to be of any significance. Were it to increase, it might have a corresponding effect on various outcomes. It should be obvious, however, that increasing social involvement likely would be far more difficult than taking measures to increase satisfaction with instruction or to make course topics more relevant to career success. Conclusion Research has demonstrated that holding pre-entry characteristics constant there are a number of aspects of university life that can contribute to desirable outcomes. Perhaps most important in this regard are the academic and social integration/involvement of students. Unfortunately, in commuter universities, particularly social integration/involvement is often found lacking. It can be argued from the foregoing that the experiences of first year science students at York are consistent with previous research on commuter universities. Overall, social involvement does not explain as much as academic involvement with respect to a number of desirable outcomes. Moreover, within the general category of academic involvement, experiences in the classroom rather than out-of-class contacts with faculty are particularly important in explanations of various desirable outcomes. More specifically, particularly satisfaction with the quality of instruction and a perception that courses include topics relevant to future career success have an impact on a fairly wide range of outcomes. No doubt the importance of the classroom can be related to the fact that in a large commuter university, classroom contact represents the main link between the institution and the individual. As a result, it is likely that improvements in curriculum and teaching would have large payoffs for both students and the institution. What goes on in the classroom appears to be at the centre of student success and satisfaction. As noted earlier, York is a large commuter university with large first year science classes, on the suburban fringe of a large city. Where all of these conditions can be found in other Canadian universities, it is likely that various outcomes can be explained by reference to similar factors. By extension, it is 40 J. Paul Grayson equally probable that changes likely to enhance outcomes at York would also lead to improvements elsewhere. Notes ' Gilbert and Auger (1988) noted that in the university they studied, the outcomes students expect from the university experience were different from the stated learning objectives of the university. The same can be said of the students who will be discussed in this article. In both cases, one of the primary objectives of students was to gain knowledge related to potential future careers. 2 Some scholars distinguish between integration and involvement; however, as pointed out by Corman et al. (1992), at the operational level, imprecision has characterized both. As a result, in this study, integration and involvement will be treated interchangeably. 3 See Pascarella and Terenzini, 1991, for justification of a similar usage. 4 Gender and minority group status were originally included under pre-entry characteristics; however, they were removed because some preliminary analyses of survey three that need not be discussed suggested that neither was relevant to the outcomes under discussion. (For a fuller treatment see Grayson, 1993b.) The finding with regard to gender parallels the results of a study by Nevitte et al., (1988). In that case, it was found that female science students performed as well as male science students. See Astin (1991) and Astin and Astin (1992) for a justification of this procedure in circumstances such as these. 6 All regressions were run with listwise and pairwise deletion and mean substitution. The results of the regression with pairwise deletion were comparable to those with listwise deletion and the pairwise option resulted in the fewest Type 1 errors. 7 Considerable discussion took place regarding whether beta or b(mean x) was the most appropriate measure to use for current purposes. In spite of arguments such as those by Achen (1982:71-77), beta was finally chosen. Beta provides the number of standard deviations that a dependent or outcome variable will change when the independent variable changes by one standard deviation with all other variables in the equation held constant. To over-simplify for readers unfamiliar with regression, the higher the beta the greater the impact of the independent variable on the outcome. Moreover, because beta is a standardized coefficient, betas for different variables can be directly compared to one another. 8 In addition, although not shown, the correlation of .68 between final year of high school marks and final first year marks is similar to that found for the sciences in earlier studies. See Allan et al. (1983). Although analyses also were carried out using final marks, analyses using expected marks are used for two reasons. First, because many students surveyed were reluctant to identify themselves, it was not possible to link survey data and administrative records for a substantial number of students. Second, when regression analysis was carried out using First Year Science in a Commuter University: Where to Intervene 150 final marks for students for whom survey data were available, the results were similar to those obtained when expected marks were used. As a result, it was decided to use the procedure that was based on the greatest number of students. References Allan, L.G., Darling, A., Hughes, R.C., & Rosenfeld, J.M. (1983). An examination of performance of first year students at an Ontario university: An admission perspective. The Canadian Journal of Higher Education, (13) 3. Astin, A.W. (1977). Four critical years. San Francisco: Jossey Bass. Astin, A.W. (1991). Assessment for excellence. Toronto: Collier Macmillan. Astin, A.W., & Astin, H.S. (1992). Undergraduate science education. Los Angeles: HERI, University of California. Bateman, D., & Donald, J.E. (1987). Measuring the intellectual development of college students: Testing a theoretical framework. The Canadian Journal of Higher Education, (17)\. Braxton, J., & Brier, E. (1989). Melding organizational and interactional theories of student attrition. Review of Higher Education, (13), 47-61. Centra, J., & Rock, D. (1971). College environments and student academic achievement. American Educational Research Journal, (8), 623-634. Chickering, A. (1974). Community versus residential students. San Francisco: Jossey Bass. Corman, J., Barr, L., & Caputo, T. (1992). Unpacking attrition: A change in emphasis. The Canadian Journal of Higher Education, (22)2. Dennison, J.D. (1992). Higher education as a field of inquiry in Canada. In A.D. Gregor & G. Jasmin (eds.), Higher education in Canada. Ottawa: Ministry of Supply and Services. Endo, J., & Harpel, R. (1983). Student-faculty interaction and its effect on freshman year outcomes at a major state university. Paper presented to the Meetings of the Association of Institutional Research, Toronto. Evers, F., & Gilbert, S. (1991). Outcomes assessment: How much value does university education add? The Canadian Journal of Higher Education, (21)2. Fox, R. (1986). Application of a conceptual model of college withdrawal to disadvantaged students. American Educational Research Journal, (23), 415-24. Garcia, M. (1988). Community college student persistence. Unpublished manuscript, Montclair State College, Upper Montclair, N.J. Grayson, J. P. (1993a). Outcomes and experiences of first year science in two universities. Toronto: Institute for Social Research, York University. Grayson, J. P. (1993b). The experience of female and minority students in first year science. Toronto: Institute for Social Research, York University. Holdaway, E., & Kelloway, K.R. (1987). First year at university: perceptions and experiences of students. The Canadian Journal of Higher Education., (21) 2. Human Resource Development Committee. (1991). Report of the National Advisory Board on Science and Technology. Ottawa. 42 J. Paul Grayson Nevitte, N„ Gibbins, R., & Codding, P.W. (1988). The career goals of female science students in Canada. The Canadian Journal of Higher Education, (18) 1. Pascarella, E., & Terenzini, P. (1976). Informal interaction with faculty and freshman ratings of academic and non-academic experiences of college. Journal of Educational Research, (70), 35-41. Pascarella, E., & Terenzini, P. (1977). Patterns of student-faculty informal interaction beyond the classroom and voluntary freshman attrition. Journal of Higher Education, (48), 540-552. Pascarella, E., & Terenzini, P. (1979a). Interaction effects in Spady's and Tinto's conceptual models of college dropout. Sociology of Education, (52), 197-210. Pascarella, E., & Terenzini, P. (1979b). Student-faculty informal contact and college persistence: A further investigation. Journal of Educational Research, (72), 214218. Pascarella, E., & Terenzini, P. (1991). How college affects students. San Francisco: Jossey Bass. Pascarella, E., Duby, P., & Iverson, B. (1983a). A test and reconceptualization of a theoretical model of college withdrawal in a commuter institution setting. Sociology of Education, (56), 88-100. Pascarella, E., Terenzini, P., & Iverson, B. (1983b). Student, faculty relationships and freshman year intellectual and personal growth in a non-residential setting. Journal of College Student Personnel, (24), 395-402. Smith, S. (1991). Report of the Commission of Inquiry on Canadian University Education. Ottawa: AUCC. Volkwein, J.P, King, M., & Terenzini, P. (1986). A study of the academic performance and intellectual growth of transfer students. Paper presented to the meeting of the Association for the Study of Higher Education, Chicago. Welty, J. (1976). Resident and commuter students. Journal of College Student Personnel, (17), 465-68. Williamson, D., & Creamer, D. (1988). Student attrition in 2- and 4- year colleges. Journal of College Student Development, (29), 210-17.