The Canadian Journal of H i g h e r E d u c a t i o n , Vol. XVI1-1, 1987 L a revue c a n a d i e n n e d ' e n s e i g n e m e n t supérieur, Vol. X V I I - 1 , 1987 Measuring the Intellectual Development of College Students: Testing a Theoretical Framework 1 DIANNE BATEMAN AND JANET G. DONALD* ABSTRACT The purpose of this research was to test the ability of Perry's scheme of intellectual development to measure the intellectual development of college students. Perry's scheme postulates four broad levels of intellectual development in the college years - dualism, multiplicity, relativity and commitment. A questionnaire was developedfrom items used in the research literature to describe the four levels and was administered to second and fourth semester college students. Ten college instructors also participated. To test the construct validity of Perry's scheme, different items representing a stage were examined for convergence, while the sets of items representing different stages were examined for divergence. The empirical validity of Perry's scheme was tested by examining student responses in relation to student time in college, cumulative average and gender. The results suggest that rather than stages of development, there are two possible levels or general positions that students take toward knowledge. The first is that knowledge consists of facts and data, and that professors should supply them. The second is that knowledge is a quest in which students have responsibility for their own learning, and are expected to be able to judge the validity of arguments and to identify and defend their own point of view. More successful students had a greater tendency to adopt the second position, but the majority of the students were in agreement with statements describing the second position. RÉSUMÉ L'objet du présent projet était de vérifier la capacité du modèle de développement intellectuel de Perry à mesurer le développement intellectuel des étudiants de niveau collégial. Le modèle de Perry pose comme principe qu'il existe quatre ' T h i s p a p e r is based on research f u n d e d by L a Direction G é n é r a l e de l ' E n s e i g n e m e n t Collégial and the Quebec Funds, FCAR. "Centre for University T e a c h i n g and L e a r n i n g . McGill University. 28 Dianne Bateman and Janet G . Donald grands niveaux de développement intellectuel durant les années de collège: le dualisme, la multiplicité, la relativité et /'engagement. Nous avons donc élaboré un questionnaire à partir des rubriques couramment employées dans la littérature pour décrire les quatre niveaux, et l'avons administré à des étudiants de collège au deuxième et au quatrième semestres. Dix professeurs de collège y ont également participé. Pour vérifier la validité structurelle du modèle de Perry, nous avons analysé différentes rubriques représentant un stade pour déterminer leur convergence, alors que nous avons cherché à analyser la divergence d'ensembles de rubriques représentant différents stades. Nous avons vérifié la validité empirique du modèle de Perry en examinant les réponses des étudiants par rapport au temps passé par l'étudiant au collège, à sa moyenne cumulative et à son sexe. Il ressort de nos résultats qu'au lieu de stades de développement, il y a plutôt deux niveaux ou positions générales possibles que les étudiants adoptent face au savoir. La première est que le savoir se compose de faits et de données, et que les professeurs sont là pour les inculquer aux étudiants. La seconde est que le savoir est une quête dont la responsabilité incombe à chaque étudiant, ceux-ci étant normalement capables de juger de la validité des arguments et de dégager et de défendre leur propre point de vue. On a pu constater que les étudiants les plus brillants optaient généralement pour la deuxième position, alors que la majorité des étudiants étaient d'accord avec les énoncés décrivant cette deuxième position. A widely held assumption among educators is that most students change cognitively and affectively as a result of attending college. Several researchers of college outcomes, including Feldman & Newcomb (1969) in their book The impact of college on college students, Perry (1970) and Winter, McClelland & Stewart (1981) in their studies of the effects of a liberal arts education, have suggested that intellectual development during the college years is demonstrable. What these cognitive and affective changes are, however, is not clear. Another widely held assumption is that students come to college intellectually prepared to organize and synthesize knowledge. Teachers expect their students to be capable of independent thinking, to interact in situations demanding clear and rational thought, and to combine that thought with communicative techniques (McKinnon, 1978). Students are expected to think logically and to be able to reason with abstract propositions that they will meet in their courses (Donald, 1985). However, studies show that fifty percent of the entering student population are not at the level of formal operations where they are able to think logically, abstractly and independently (Higgins-Trenke & Gaite, 1971; McKinnon, 1978; Ross, 1973; Torkia Lagace, 1981). The assumptions that students arrive intellectually equipped to deal with college and that they continue to grow cognitively and affectively while attending college create frustration for both teachers and students. Teachers complain that students Measuring the Intellectual D e v e l o p m e n t of College Students: 29 Testing a Theoretical F r a m e w o r k cannot think; students become overwhelmed with what appear to be unrealistic teacher expectations. Although teachers are required to define the content objectives of their courses, that is, the material to be learned, teachers are seldom, if ever, asked to identify the cognitive and affective abilities their curriculum and methodology are intended to develop in their students. Content objectives are relatively easy to measure at the end of a course or program since they consist of defined bodies of knowledge. However, the cognitive and affective abilities acquired by students have not to date been recognized or measured although postsecondary literature, as mentioned above, suggests that this should be so. The purpose of this research, therefore, was to test the ability of Perry's scheme of intellectual development, which proposes four broad stages of development in the college years, to measure that development and, thus, to test the construct validity of Perry's stages of intellectual development. Construct validity of the Perry scheme would be shown if the four stages of development could be identified in college students. According to Cook and Campbell (1979), construct validity is measured by testing for convergence across different measures of a construct, in this case a stage, and testing for divergence between measures of different constructs or stages. The study was also designed to test the empirical validity of the stages through an analysis of student responses in relation to students' time in college, cumulative average in college and gender. A second, important part of the research was to compare student development with teacher expectations in college. We therefore studied the fit and utility of these stages as described in statements with teachers. Teachers were asked to identify which statements best described their students and then, at what stage as expressed by the statements, they (a) expected and (b) preferred their students to be. The questions posed in this study were, therefore, how coherent is Perry's scheme, that is, do different statements of the same stage show convergent responses? Second, how explicit is the scheme, that is, do statements about different stages diverge? Third, how consistent is the scheme with other measures of student development such as time in college and cumulative average? Fourth, how useful is the scheme in measuring college student development according to their instructors? PERRY'S S C H E M E Perry (1970) developed a framework for analyzing intellectual development in college students. He proposed four principal stages of development: dualism, multiplicity, relativism and commitment. The dualistic student views the world and knowledge in absolute terms. Things are either right or wrong. In education, authority is represented by the teacher who knows the correct information. What the teacher says is accepted as truth. The student moving toward the second stage, multiplicity, begins to recognize the authorities' contrasting viewpoints but sees contrasts as the authorities' way of making the student locate the right answer. 30 Dianne Bateman and Janet G . Donald Students in the third stage, relativism, are becoming aware that there are no right or wrong answers. They recognize the need to perceive, analyze and evaluate. Students at the final stage, commitment, have developed their own approach to living and learning. Decisions regarding career, marriage, education or politics are made based on a personal philosophy. Perry's original research used a sample of Harvard male students during the 1950s and 1960s. Student volunteers completing their first year at university were asked to come and talk freely about what stood out for them in that year. This interview was repeated yearly until graduation. Although this study was performed on a restricted sample, subsequent studies by Perry (1974), and others (Kurfiss, 1975; Mentkowski, Moeser and Strait, 1983), provide supporting evidence of the relevance of the sequence in other college and university settings. In contrast to other stage theorists who view stages as upward and sequential, Perry has suggested that individuals can move across his stages more than once. In fact, the four stages are represented by ten positions. For Perry, individuals may be at different positions in different areas of their intellectual life. For example, at the beginning of the learning process in a given discipline, students must master a certain amount of technical vocabulary and data. They are not yet ready to think critically about the subject. Moving to a new discipline may put students back to an earlier stage, where they may once again want right answers (Elrick, 1985). Although other measures of cognitive development at the postsecondary level exist, for example, those used at Alverno College (Mentkowski and Strait, 1983) and that used by Winter and McClelland (1978), no published test exists using Perry's scheme to estimate the levels of students' intellectual development. Knefelkamp began work to develop instruments to measure the stages with Widick and Parker in 1975 and Sleptiza in 1976. She attempted to establish the reliability of two instruments, one using sentence completion and another, short essays. Although Knefelkamp's short essays correlate reasonably well (.78) with extensive full-length interviews, the measure focuses on career development and may meet the needs of counsellors more readily than the classroom teacher. Other researchers (Kurfiss, 1975; Erwin, 1983; Mentkowski, Moeser and Strait, 1983) have attempted to measure development according to Perry's scheme but have encountered difficulties with their methods. Kurfiss uses paraphrasing in her test. This method requires that students paraphrase short paragraphs, a skill which many college level students appear to lack. The method of paraphrasing may work where the complexity of the paragraph is not more than a step beyond the students' ability. However, comprehension difficulties may arise with the meaning of paragraphs where the structure of thought is more advanced. Erwin (1983) reports that his Scale of Intellectual Development can be used only in research settings and needs further validation. The difficulties with the Mentkowski, Moeser and Strait measures are due to their method of data collection. They collect three essay samples from their subjects each time they are evaluated, which provides a massive data base to analyze. What is missing is a test of student development which is consistent with Perry's scheme but which does not require Measuring the Intellectual D e v e l o p m e n t of College Students: 31 Testing a Theoretical F r a m e w o r k paraphrasing or analysis of essays. Therefore, a test was created which uses statements that represent Perry's four principal stages: dualism, multiplicity, relativism and commitment. If a student consistently preferred statements describing a particular stage, it could then be said that that stage represents the student's level of cognitive development. RESEARCH DESIGN This study examined the construct and empirical validity of Perry's scheme. Level of intellectual development, according to Perry, was measured by items (statements) describing the four stages. To test for construct validity, different items within a stage were examined for convergence, while the sets of items from different stages were examined for divergence. To test for empirical validity three factors which could affect level of intellectual development, time in college, cumulative average in college, and gender, were examined in relation to the level of intellectual development. Time in College Several studies on cognitive growth have focused on the four year liberal arts college. These studies (Mentkowski & Strait, 1983; Winter, 1981; Winter & McClelland, 1978) indicate that the ability to form complex concepts changes drastically from the freshman to the senior year. Mentkowski and Strait, who used a variety of measures including the Watson Glaser Test of Critical Thinking Skills, the Defining Issues Test and Loevinger's Sentence Completion Test, found that more changes occur during the first two years of college but that the effects of the learning process were more evident during the last two years. For this reason, time in college was considered to be an important independent variable. Students at the end of their second semester and students at the end of their fourth semester in a Quebec college served as subjects for the study. Cumulative Average The most widely used indicator of intellectual development is a student's academic success as measured by grade point average. To test academic success, Mentkowski and Strait used high school grade point averages and Winter and McClelland compared a high-powered university with a state college which was known to accept students who were less academically inclined. In this study, to investigate the effect of academic ability, each student's cumulative average was used as a measure of academic success. Sex of the Students A recent report published by the Social Sciences and Humanities Research Council of Canada (Eichler, 1985), stresses the importance of treating men and women equitably and objectively in all stages of the research process. Kohlberg (1964) and Loevinger (1966) did extensive research on moral and ethical development, 32 Dianne Bateman and Janet G . Donald Kohlberg with male students and Loevinger with female. The original work of both Kohlberg and Loevinger can be criticized for the exclusion of one sex. Kohlberg's theory of six stages of moral development was derived from a longitudinal study of eighty-four Harvard men. Loevinger's test for measuring ego development was drawn from studies of women. Gilligan (1979) challenged Kohlberg's theory and suggested that systematic attention to women's lives, in both theory and research, would create a more balanced conception of human development. She argued that women fail to develop according to Kohlberg's system because they bring a different point of view regarding self and morality to the life cycle. Kohlberg missed this important finding because of his restricted sample. To date, studies done on the Perry scheme have not used sex of the students as a variable. Therefore, to achieve a balanced perspective, students' sex was included as a variable. Teacher Expectations One reason for this research is the suggestion that a large proportion of college students are unable to think logically and critically. As stated earlier in this paper, studies indicate that fifty percent of the entering student population do not have the abstract thinking skills required for success in college (Higgins-Trenke & Gaite, 1971; McKinnon, 1978; Ross, 1973;Torkia Lagace, 1981). Literature on teacher expectation effects suggests that the expectancies a teacher has regarding students' ability affects the performance of those students (Brophy & Good, 1974; Braun, 1976; Brophy, 1983 and Cooper, 1983). The findings, however, are not conclusive. In a 1978 survey of the literature, 112 studies were found which investigated teacher expectations; 40 percent of those studies found significant support for the existence of teacher expectation effects (Cooper, 1983). Therefore, the development or lack of development of student thinking abilities may be related to teacher expectations. For this reason it was important to examine teacher expectations using the Perry model. The current literature suggests that teachers describe their students as being at lower levels of development but prefer that they be at higher levels of development. Hypotheses INTELLECTUAL DEVELOPMENT Hypothesis 1: Perry's four stages of intellectual development: dualism, multiplicity, relativism and commitment, have construct validity as described by statements. Hypothesis 2: Second semester students will tend to be in greater agreement with statements representing lower levels of development than fourth semester students. Second semester students will be in less agreement with statements representing higher levels of development than fourth semester students. Hypothesis 3: More successful students, that is, those with higher cumulative averages, will be in greater agreement with the statements at the more advanced levels and will be in greater disagreement with statements at the lower levels. Measuring the Intellectual D e v e l o p m e n t of College Students: 33 Testing a Theoretical F r a m e w o r k Hypothesis 4: There will be no difference due to sex of students. TEACHER EXPECTATIONS Hypothesis 5: Teachers will describe students as being at lower levels of development, but prefer that they be at higher levels of development. METHOD Subjects The subjects for the study were full time students at a Quebec college (Cegep). In order to ensure a cross-section of students from all college programs, ten Humanities classes, a subject which all students in the college are required to take, were drawn at random. The sample contained 101 females and 76 males and was relatively homogeneous with respect to age (M = 18, SD = .79). The sample closely represented the proportion of the college population in each program (Social Science, 41%; Commerce, 25%; Science, 12%; Creative Arts, 11%; Other, 11%). The Humanities classes were comprised of both second and fourth semester students. Second semester students consisted of one hundred and two who had started college in September 1984 and therefore had completed one term. Fourth semester students consisted of seventy-five who had begun college in September 1983 and therefore had completed three terms. A sample of ten college teachers involved in teaching core courses, and who would, therefore, teach a broad spectrum of students, participated in the study. Testing Procedure The test was a questionnaire based on the four principal stages of Perry's scheme: dualism, multiplicity, relativism and commitment. It consisted of twenty statements (items). Eight were from transcripts of Perry's interviews and had been cited by him as examples of the stages (1970), four were taken from the work of Kurfiss (1975) on Perry, and four expressed concepts which Perry and other researchers have used as indicators of intellectual development (Table 1). Each stage on the Perry scale was represented by four items. The questionnaire was examined by a team of five instructors to establish the face validity of the items, that is, that they were consistent with the research literature. The questionnaire was administered in the ten Humanities classes over a one week period in March 1985. Students were asked to respond on a 5-point Likert Scale to what extent they agreed or disagreed that the items described the way they felt about knowledge and learning. Teachers were asked to identify which items described their students, which represented a more advanced level of development, and at which level they preferred their students to be by checking off the items on the questionnaire. Method of Analysis To test hypothesis 1, that is, to establish the construct validity of Perry's scheme, the following steps were taken. To test for convergence across different measures 34 Dianne Bateman and Janet G . Donald Table 1 Q u e s t i o n n a i r e Items R e p r e s e n t i n g Stages of I n t e l l e c t u a l Development D1 When i t comes to knowledge, f a c t s are f a c t s : t h a t ' s b a s i c . The s t u d e n t ' s b u s i n e s s i s to master the f a c t s as the p r o f e s s o r g i v e s them. D2 Knowledge i s being able to f i q u r e out the r i g h t M3 Teachers present d i f f e r e n t p o i n t s of view because they want us to t h i n k independently - to l e a r n to f i n d the answer f o r ourselves. C4 The p r o f e s s o r i s not a g i v e r o f knowledge. The p r o f e s s o r i s a guide and a model f o r our own independent l e a r n i n g . The r e s p o n s i b i l i t y f o r l e a r n i n g or mastering a s u b j e c t i s the student's. R5 You c a n ' t a n a l y z e , c o n s i d e r and balance t h i n g s f o r e v e r ; sooner or l a t e r you have to decide and a c t . C6 Knowledge i s being able to defend a p o s i t i o n with argumentation, even though o t h e r s might d i s a g r e e . C7 L e a r n i n g i s c h a l l e n g i n g when we must look at a l l from these decide where we s t a n d . D8 Knowledge i s being able to r e c a l l R9 O p i n i o n s are o n l y as good as the evidence s u p p o r t i n g them. RIO As long as s t u d e n t s develop and support t h e i r answers they should not be p e n a l i z e d , even i f t h e i r view d i f f e r s from t h a t of the professor. M12 In areas where e x p e r t s d i s a g r e e , everyone has a r i g h t to h i s or her own o p i n i o n . R14 Knowledgeable persons use what they know to judge values. C16 Knowledgeable pesons have i d e n t i f i e d t h e i r own p o i n t of view, r e c o g n i z e that i t i s t h e i r own and act a c c o r d i n g to i t . D17 I f t e a c h e r s stuck more to the f a c t s and d i d l e s s s t u d e n t s would get more out o f t h e i r c l a s s e s . M19 The s u c c e s s f u l M20 Everyone has a r i g h t to h i s or her own o p i n i o n . t h i n g as r i g h t or wrong. answer. solid the ideas and f a c t s and data. i d e a s , data and theorizing, student has f i g u r e d out what the teacher wants. There i s no such Measuring the Intellectual D e v e l o p m e n t of College Students: 35 Testing a Theoretical F r a m e w o r k of the same stage, a matrix of Pearson-product moment correlations was computed between all items. It was expected that the four items representing each intellectual stage would show a higher percentage of significant correlations than correlations of the items from one level with items from the other levels. Convergence was then tested by factor analysis. If the test of intellectual development was valid, items measuring dualism would cluster together, and this cluster would be distinct from a cluster of items on relativity, as would those from each stage. If this occurred, it would indicate that there are different kinds or levels of intellectual development. Conversely, if unanticipated clusters were found in the test, revision of the test or the construct would be warranted (Shavelson and Stanton, 1975). Thus, to exhibit construct validity, the items describing each stage should (a) correlate and (b) cluster in a factor together. The items describing different stages should not correlate as highly as those in a stage and should not load on the same factor. To test hypothesis 2, the effect of time in college on intellectual development, an analysis of variance between second and fourth semester students was done between their responses to the statements describing the least advanced level of intellectual development (dualism) with the most advanced level (commitment). To test hypothesis 3, the effect of cumulative average on level of intellectual development, students' scores on the lowest (dualism) and highest (commitment) levels of intellectual development were regressed on'their cumulative averages. To test hypothesis 4, the effect of sex on level of intellectual development, each student's scores on dualism and commitment items were regressed on cumulative average by sex. This procedure allowed for a comparison of the effects of sex with the effects of cumulative average on level of intellectual development. To test hypothesis 5, the percentage of teachers' agreement was compiled on items which they considered to describe students, on items which they thought represented a more advanced level of development, and on items which for them described the level at which they preferred their students to be. RESULTS Description of Student Responses The percentage of student agreement with each item was calculated to determine the overall tendency of students as to preferred levels of intellectual development (Table 2). The majority of students did not tend to agree with items representing dualism, but did tend to agree with items representing the more advanced levels. Overall agreement with dualism items ranged from 23 to 44 percent, while agreement with multiplicity, relativity and commitment items ranged from 49 to 90 percent. Thus, students overall were in greater agreement with items from the higher levels of intellectual development. Hypothesis 1: Construct Validity of the Perry Scheme SIGNIFICANT CORRELATIONS B E T W E E N ITEMS The correlation matrix for the 16 items describing the four stages (Table 3) shows 36 Dianne Bateman and Janet G . D o n a l d Table 2 Percentage of Student Agreement with Items Item Percentage of Agreement Second Semester Fourth Semester Total D1 36 40 38 D2 24 23 23 D8 46 41 44 D17 28 27 28 M3 81 73 78 M12 81 81 81 M19 48 51 49 M20 57 51 55 R5 75 76 75 R9 57 60 58 RIO 89 91 90 R14 77 80 79 C4 71 76 73 C6 69 69 69 C7 70 72 71 C16 68 60 65 D = Dualism R = Relativity M = Multiplicity C = Commitment the degree of correlation within stages compared to between stages. First, we must note that the correlations are not high, although one-third of them are significant. To exhibit convergence within a stage and divergence between stages, we would expect significant correlations within each triangle and non-significant correlations outside. Table 3 shows that within a stage the highest percentage of significant correlations between items (67%) occurred between items representing commitment. Fifty percent of the correlations among items from the other three levels were significant. Outside the triangles, 28 percent of the correlations were Measuring the Intellectual D e v e l o p m e n t of College Students: 37 Testing a Theoretical F r a m e w o r k Table 3 Correlation Matrix f o r Items Describing Stages of Intellectual the Four Development significant. Thus the items within groups showed greater convergence than outside the groups. Furthermore, the matrix shows that few significant correlations occurred between dualism items and others (10%), and that the percent of significant correlations among multiplicity, relativity and commitment (46%) very closely approximates the percent of significant correlations among items within three of the groups. The matrix shows, therefore, that dualistic items do not tend to be related to the items at other levels, but that the three more advanced levels of intellectual development are related. If the items represent stages, it would be expected that more significant correlations would occur between adjacent stages than non-adjacent stages. Between the adjacent stages: dualism and multiplicity; multiplicity and relativity; 38 Dianne Bateman and Janet G . Donald and relativity and commitment, the percent of significant correlations was 19, 38, and 50 respectively, for an average of 35 percent. Between the non-adjacent stages: dualism and relativity; dualism and commitment; and multiplicity and commitment, the percent of significant correlations was 0, 13, and 50 percent, for an average of 21 percent. Although the non-adjacent items show fewer significant correlations, the relatively high percentage (50) between multiplicity and commitment items weakens the argument for the existence of stages. SEMANTIC EXAMINATION OF ITEMS These results suggested the need for a more intensive study of the individual items to determine the reasons for their relationship. Correlations between items were significant for three of the four statements representing dualism: (1, 2, 8). Each of these statements refers to knowledge as the accumulation of facts. The additional focus of item 17 on teacher methodology results in a weak relationship between it and the other dualism items. Among the multiplicity items, item 12 relates significantly to each other item so could be said to be a core item. It summarizes the multiplistic perspective that where experts disagree, everyone has a right to his or her own opinion. The three relativity items which relate significantly (items 5, 10, and 14) have as their theme the development of an individual point of view. Item 9 does not correlate significantly with these items, and adds a new dimension of supporting viewpoints with evidence. The four items representing commitment show significant correlations. These items stress the importance of being able to defend one's viewpoint and of being responsible for one's own learning. Thus factors other than the stages of development appear to be affecting the relationship among items. FACTOR A N A L Y S I S O F R E S P O N S E S T O I T E M S Because the correlation matrix suggested that factors other than those defined by the four Perry stages were affecting student response, a factor analysis was done of student responses to the sixteen items. A principal-factor orthogonal varimax procedure was used to reduce the 16X16 correlation matrix and yielded five factors. In factor analysis there is no single criterion for determining the number of factors to retain. One guideline introduced by Kaiser (1960) is to retain factors with eigenvalues greater than one. Seven factors had eigenvalues greater than one. For the present correlation matrix a five-factor solution was more interpretable than a seven-factor solution since only one item loaded on each of factors six and seven. The loadings of the items on each factor are shown in Table 4. The first factor had its highest loadings from items representing the three more advanced positions of intellectual development, multiplicity, relativity and commitment (items 4, 3, 7 & 5). Each of these items refers to the students' role in the learning process, claiming that students must think independently and are ultimately responsible for their own learning. The second factor also had its highest loadings from items representing multiplicity, relativity and commitment (items 20, 6, 10 & 14). All stress the Measuring the Intellectual D e v e l o p m e n t of College Students: 39 Testing a Theoretical F r a m e w o r k Table 4 Loading of Items on Factors Produced by the P r i n c i p a l Factor Orthogonal Varimax Procedure Factor Item D1 .05 .38 .20 .29 .50 D2 .02 .14 .02 /78" -.05 D8 .01 .16 -.04 .63 .14 D17 .20 .11 -.16 .01 .77 M3 .64 .07 .15 .28 .18 M12 .14 .28 ?44~ .04 .09 M19 .04 .05 .01 .12 .05 M20 .04 .79 .03 .10 .18 R5 .53 .21 .22 .15 .34 R9 .07 .04 .01 .12 -.08 RIO .14 .46 .35 -.21 -.04 R14 .29 .36 .33 .15 -.24 C4 .75 .10 .00 .03 .01 C6 .11 .53 .06 .29 .22 C7 .61 .21 .21 .10 .30 C16 .02 .07 TitT .01 .08 importance of being able to judge the validity of the argument of others, as well as being able to defend one's own position with evidence. The third factor is clearly dominated by ai commitment item (16), which defines knowledgeable people as those who have identified their own point of view, recognize that it is their own, and act according to it, although items 12, lOand 14 from multiplicity and relativity also contribute. Factors four and five encompass the items representing dualism. Items 2 and 8 load heavily on the fourth factor. These two items define knowledge as the rote 40 Dianne Bateman and Janet G . Donald recall of facts in one's quest to find the right answers. Factor five is dominated by items 1 and 17 which define the teacher's role in this quest. The teacher is viewed as a giver of knowledge, the one who knows the correct answers. This analysis shows that dualism items represent two factors in the analysis recall of facts and the teacher's role in this. Different multiplicity items load on the first three factors, which suggests that multiplicity is not a construct. The loading of relativity and commitment items followed a similar pattern over the three factors. Thus, rather than representing three stages or positions, the multiplicity, relativity and commitment items are explained by three other factors: students' responsibility for their own learning, using evidence, and identifying their own point of view. A second factor analysis was performed to test divergence among the four positions. To do this, the scores on the four items in each level were combined by student. The correlations between each position and the factor loadings for the one factor resulting from this analysis are shown in Table 5. The solution could not be rotated because only one factor was extracted. This factor excludes dualism and is dominated by loadings from multiplicity, relativity and commitment. This single factor accounts for 43% of the variance. As could be expected from the correlations between positions and the previous analysis, no evidence of divergence exists among these three more advanced positions of intellectual development. We must therefore conclude that rather than four stages of development, Perry has found statements which describe two positions. These positions consist of two factors which describe knowledge as facts and data (dualism) and three factors which describe knowledge as a quest in which students think independently, judge the validity of arguments, and have identified their own point of view. Hypothesis 2: Time in College An assumption of the Perry scheme is that a college student's development through the scheme is sequential; that the student may naively expect answers at the beginning of the college experience, and gradually come to terms with the relative nature of knowledge by the end of the college experience. Following this assumption, it might be expected that an individual would score high on one position, and that this position would represent the individual's level of cognitive development and have some relationship with time spent in college. For example, an entering student who sees the world and knowledge in absolute terms might agree with the statements representing dualism. In contrast, a second year student who is beginning to realize that facts must be seen in terms of their context might agree with the statements representing the more advanced position. It could then be said that that position describes the person's intellectual development within the Perry scheme. Since the factor analysis differentiated only between dualism and a more advanced position, and since the items describing commitment showed the highest overall loadings on that more advanced position, in succeeding analyses comparisons were made between the responses to the 4 dualism items and the 4 Measuring the Intellectual D e v e l o p m e n t of College Students: 41 Testing a Theoretical F r a m e w o r k Table 5 C o r r e l a t i o n and F a c t o r M a t r i x Second F a c t o r Analysis FACTOR 1 CORRELATIONS DUAL MUL REL DUAL for COM DUAL MUL MUL REL -.019 .320 COM -.064 .322 .417 .70 REL -.016 .77 COM .78 commitment items. It was expected that second semester students would agree more with the statements of dualism and less with statements of commitment than would fourth semester students. An analysis of variance between second and fourth semester students on their dualism and commitment scores was performed. There was no significant difference between these groups. Second semester students did not agree nor disagree more strongly with the items when compared with fourth semester students. Therefore, a relationship between time spent in college and attitude toward knowledge was not evident. There are several possible explanations for this result. The first is that one year does not make a substantial difference. Recent research on reasoning skills in the American College Program suggests that there are pronounced differences between first and fourth year college students but that age does not make a difference in reasoning skills (Steele, 1986). Another explanation for the lack of a difference could be the date of data collection. Because the data was collected in March, it is possible that it did not capture the expected gain in intellectual development experienced by first year students coming from high school. In addition, individuals may be at different positions in different areas of their intellectual life, making their attitude dependent upon the situation. Students referred to this dilemma while completing the questionnaire. They asked questions such as should they think about a particular subject? Should they refer their answers to the beginning or end of term? Other researchers (e.g., Kurfiss, 1977) have found that the Perry position varies with the content area addressed by the measurement instrument. This supports the argument that positions are less developmental in nature and perhaps more strategic and situation dependent. We therefore must determine if other factors are consistent with the hypothesis of two levels of development or if the Perry descriptions are situation dependent. 42 Dianne Bateman and Janet G . Donald Hypothesis 3: Cumulative Average A scheme of intellectual development would suggest that more successful students, that is, those with higher cumulative averages, would be in greater agreement with the items which describe the more advanced level. Student cumulative averages were, therefore, regressed on the dualism items and on the commitment items. Table 6 shows that all the correlations between student cumulative averages and scores on dualism and commitment were in the expected direction, and although no high correlations were found, correlations for the total group were significant. Higher averages were associated with lower scores on dualism; more successful students tend to be less dualistic. Supporting the trend, the correlations between commitment and student cumulative averages exhibited a positive relationship. These results confirm the hypothesis that more successful students, that is, those with higher cumulative averages, agree more with statements representing commitment and less with statements representing dualism. Fourth semester students showed the highest significant correlation between cumulative average and commitment scores. This also supports the idea of intellectual development in college. In summary, the negative correlation that exists between student cumulative averages and scores on dualism with the corresponding positive correlation between cumulative averages and scores on commitment support the premise that two levels of intellectual development can be discerned. Hypothesis 4: Sex of the Students It was expected that there would be no difference between males and females in items preferred. As stated above, students with higher cumulative averages preferred commitment items. This relationship became more complex when the total group was divided by sex. As female cumulative averages increased, their agreement with dualism items decreased (r = —.22, p. 001.) to a greater extent than those of their male peers (r = —. 12). However, the male commitment correlation was higher than that for females, although not significant. Therefore, results do not confirm the hypothesis that there would be no differences between males and females, but the differences are complex. The results suggest that more successful males are less in disagreement with dualism than females, but more in agreement with the commitment items. Is it possible that successful males have stronger attitudes toward the positive aspects of intellectual development? Our findings suggest that sex differences should continue to be examined in research on intellectual development. Hypothesis 5: Teacher Expectations It was expected that teachers would describe students as dualistic and multiplistic, but prefer that they be at the higher levels of development. There was unanimous Measuring the Intellectual D e v e l o p m e n t of College Students: 43 Testing a Theoretical F r a m e w o r k Table 6 C o r r e l a t i o n C o e f f i c i e n t s of Student C u m u l a t i v e Average w i t h D u a l i s m and Commitment Item S c o r e s Dualism Total Group Commitment -.19** .17* Second Semester (n=102) -.23* .15 Fourth Semester (n= 75) -.13 .27* Ferna 1e (n=101) -.22* .10 Male (n= 76) -.12 .21 * p . < .01 (n=177) **p. <.001 agreement among the 10 teachers that all dualism statements (1, 2 , 8 , 17), and two multiplicity statements, (12, 19), best described student attitudes toward knowledge and learning. This expectation was in marked contrast with student responses. Table 2 shows that with the exception of the multiplicity item twelve, which had an 81% level of agreement overall, the majority of students did not agree with the dualism and multiplicity statements which teachers had used to describe them. There was unanimous agreement among teachers that a relativity statement, (9), and two commitment statements, (4 and 7), represent an advanced level of thinking, a level at which teachers wished their students to be. The majority of students agreed with these statements, item 9 (58%), item 4 (73%), item 7 (71%), suggesting that they tend to possess these attitudes. There were mixed teacher reactions to two multiplicity statements, (3 and 20), three relativity statements, (10, 14, 5), and two commitment statements, (6 and 16). These statements focus on the role of the teacher and the role of the student, areas in which teachers have shown differences in opinion. These results support the hypothesis that teachers would describe their students as dualistic and multiplistic but prefer that they be at the higher levels of development. One possible explanation for this finding is that students may agree with the higher level statements but do not transfer their beliefs into behaviors that their teachers can see. Another is that students behave toward teachers as if teachers are authority figures, although their personal attitudes may differ. The classroom situation may impose a less developed stance on the students. 44 Dianne Bateman and Janet G . Donald CONCLUSION The purpose of this research was to test the ability of Perry's scheme of intellectual development to measure intellectual development. The results suggest that rather than stages of development, there are two possible levels or positions that students take toward knowledge. The first is that knowledge consists of facts and data, and that professors should supply them. The second is that knowledge is a quest in which students have responsibility for their own learning, and are expected to be able to judge the validity of arguments and to identify and defend their own point of view. More successful students have a greater tendency to adopt the second position, but the majority of students are in agreement with statements describing the second position. When describing students, instructors tended to choose low-level statements (dualism and multiplicity). When identifying statements which represented a more advanced level of thinking, instructors tended to choose high-level statements (relativism and commitment). The instructor responses do not fit student preferences for the statements, which may reflect a role/attitude discrepancy. The questionnaire items could be tested on university students to see if they are more sensitive when the age difference is greater than one year. Since the preferences of the college sample in this study were so advanced, however, the secondary school might be a preferable location for further testing of the scheme. The evidence questions the construct validity of the scheme, and prompts the recommendation that the items be considered measures of attitudes toward knowledge and learning, rather than measures of intellectual development. Attitudes toward knowledge and their components such as responsibility for one's own learning or the ability to judge the validity of arguments merit further attention as they likely provide an encompassing framework through which students and teachers view the learning process. REFERENCES Braun, C. (1976). T e a c h e r expectation: Sociopsychological d y n a m i c s . Review Research, 46, of Educational 185-213. Brophy, J. (1983). Research on the self-fulfilling prophecy and teacher expectations. Journal Educational Psychology, of 75, 6 3 1 - 6 6 1 . Brophy, J . , & G o o d , T . (1974). Teacher-student relationships: Causes and consequences. N e w York: Holt, Rinehart & W i n s t o n . C h i c k e r i n g , A . W . , & Associates (1981). The modern American C o o k , T . D . , & C a m p b e l l , D. T . (1979). Quasi-experimentation: college. San Francisco: Jossey-Bass. Design & analysis for field settings. Chicago: Rand McNally. C o o p e r , H . M . (1983). A historical overview of teacher expectation effects. Paper presented at the A n n u a l C o n v e n t i o n of the A m e r i c a n Psychological Association, California. D o n a l d , J. G . (1985). Intellectual skills in higher education. Canadian Journal of Higher Education, 15, 5 3 - 6 8 . Eichler, M . , & L a p o i n t e , J. (1985). On the treatment H u m a n i t i e s Research Council of C a n a d a . of the sexes in research. Social Sciences and Measuring the Intellectual D e v e l o p m e n t of College Students: 45 Testing a Theoretical F r a m e w o r k Elrick, M . (1985, J a n u a r y ) . Intellectual d e v e l o p m e n t of students. Teaching Forum, p. 4 . Erwin, T . D . (1983). T h e scale of intellectual development: Measuring Perry's scheme. Journal College Student Personnel, of 6-12. F e l d m a n , K . A . , & N e w c o m b , T . M . (1969). The impact of college on college students. San Francisco: Jossey-Bass. Flavell, J. H . (1977). Cognitive development. N e w Jersey: Prentice Hall. Gilligan, C . (1979). W o m a n ' s place in m a n ' s life cycle. Harvard Educational Review, 49(4). H i g g i n s - T r e n k e , A . , & G a i t e , A . J. H. (1971). Elusiveness of formal operations thought in adolescence. Proceedings Association. of the 79th Annual Meeting of the American Psychological 6, 2 0 1 - 2 0 2 . Kaiser, H . F. (1960). T h e application of electronic computers to factor analysis. Educational Psychological Measurement, 20, and 141-151. K n e f e l k a m p , L . , & Slepitza, R. (1976). A cognitive developmental model of career d e v e l o p m e n t - A n adaptation of the Perry s c h e m e . The Counselling Psychologist, 6(3), 53-58. K o h l b e r g , L . (1964). D e v e l o p m e n t of moral character and moral ideology. Review Development Research, of Child 1, 3 8 2 - 4 3 1 . K u r f i s s , J. (1975). Late adolescent development: A structural epistemological perspective. Unpub- lished doctoral dissertation, University of W a s h i n g t o n . L o e v i n g e r , J. (1966). T h e m e a n i n g and m e a s u r e m e n t of e g o d e v e l o p m e n t . American Psychologist, 21, 195-206. M c K i n n o n , J. W . (1978). T h e college student and formal operations. In R e n n e r e t a l . (Eds.). thinking and learning with the Piaget model. Research, N o r m a n : University of O k l a h o m a , 110-129. M e n t k o w s k i , M . , M o e s e r , M . , & Strait, M . (1983). Using the Perry s c h e m e of intellectual and ethical d e v e l o p m e n t as a college o u t c o m e s measure: A process and criteria for judging student p e r f o r m a n c e , Vols. I and II. M i l w a u k e e , Wisconsin, Alverno Productions. M e n t k o w s k i , M . , & Strait, M . J. (1983). A longitudinal development and generic abilities study of student in an outcome-centered liberal change in cognitive arts curriculum. report to the National Institute of Education, Research Report No. 6.) (Final Milwaukee, W i s c o n s i n , A l v e r n o College, O f f i c e of Research & Evaluation. Perry, W . J. (1970). Forms of intellectual and ethical development in the college years: A scheme. N e w York: Holt, Rinehart and W i n s t o n . Perry, W . J. (1974). C o u n s e l i n g . In A n n u a l Report of the Bureau of Study Counsel, Harvard University, 1973-74. C a m b r i d g e , Masachusetts: Harvard University. Ross, R . J. (1973). S o m e empirical p a r a m e t e r s of formal thinking. Journal 2(2), of Youth and Adolescence, 167-177. Shavelson, R . J . , & Stanton, G . C . (1975). Construct validation: three m e a s u r e s of cognitive structure. Journal Steele, J . M . (1986, April). Assessing reasoning M e t h o d o l o g y and application to of Educational and communication Measurement, skills in college. 12(2), 67-85. Paper presented at the annual m e e t i n g of the A m e r i c a n Education Research Association, San Francisco. Torkia L a g a c é , J. (1981). La pensée formelle chez les étudiants de collège: objectif ou réalité? (Final report to the G o u v e r n e m e n t du Q u é b e c , Ministère de l ' E d u c a t i o n , Direction générale de l ' e n s e i g n e m e n t collégial.) W i d i c k , C . , K n e f e l k a m p , L . , & Parker, C . A . (1975). T h e counselor as a developmental instructor. Counselor Education and Supervision, Winter, D . G . (1981). A new case for development. 14, 286-296. the liberal arts. Assessing institutional goals and student San Francisco: Jossey-Bass. W i n t e r , D . G . , & M c C l e l l a n d , D . C . (1978). T h e m a t i c analysis: An empirically derived measure of the e f f e c t s of liberal arts education. Journal of Educational Psychology, 70(1). 8-16. Winter, D. G . , M c C l e l l a n d , D . , & Stewart, A . (1981). A new case for the liberal institutional goals and student development. San Francisco: J o s s e y - B a s s . arts. Assessing