University of Iowa

Iowa Research Online

Theses and Dissertations

2011

The effect of study abroad on intercultural competence among undergraduate college students
Mark Hungerford Salisbury
University of Iowa

Copyright 2011 Mark Hungerford Salisbury This dissertation is available at Iowa Research Online: http://ir.uiowa.edu/etd/1073 Recommended Citation
Salisbury, Mark Hungerford. "The effect of study abroad on intercultural competence among undergraduate college students." doctoral PhD diss., University of Iowa, 2011. http://ir.uiowa.edu/etd/1073.

Follow this and additional works at: http://ir.uiowa.edu/etd Part of the Educational Administration and Supervision Commons

THE EFFECT OF STUDY ABROAD ON INTERCULTURAL COMPETENCE AMONG UNDERGRADUATE COLLEGE STUDENTS

by Mark Hungerford Salisbury

An Abstract Of a thesis submitted in partial fulfillment of the requirements for the Doctor of Philosophy degree in Educational Policy and Leadership Studies in the Graduate College of The University of Iowa May 2011 Thesis Supervisor: Professor Ernest T. Pascarella

ABSTRACT During the last decade higher education organizations and educational policy makers have substantially increased efforts to incentivize study abroad participation. These efforts are grounded in the longstanding belief that study abroad participation improves intercultural competence an educational outcome critical in a globalized 21st century economy. Yet decades of evidence that appear to support this claim are repeatedly limited by a series of methodological weaknesses including small homogenous samples, an absence of longitudinal study design, no accounting for potential selection bias, and the lack of controls for potentially confounding demographic and college experience variables. Thus, a major competing explanation for differences found between students who do and do not study abroad continues to be the possibility that these differences existed prior to participation. The current study sought to determine the effect of study abroad on intercultural competence among 1,593 participants of the 2006 cohort of the Wabash National Study on Liberal Arts Education. The Wabash National Study is a longitudinal study of undergraduates that gathered pre- and post-test measures on numerous educational outcomes, an array of institutional and self-reported pre-college characteristics, and a host of college experiences. The current study employed both propensity score matching and covariate adjustment methods to account for pre-college characteristics, college experiences, the selection effect, and the clustered nature of the data to both crossvalidate findings and provide guidance for future research. Under such rigorous analytic conditions, this study found that study abroad generated a statistically significant positive effect on intercultural competence; an effect

2 that appears to be general rather than conditional. Moreover, both covariate adjustment and propensity score matching methods generated similar results. In examining the effect of study abroad across the three constituent subscales of the overall measure of intercultural competence, this study found that study abroad influences students diversity of contact but has no statistically significant effect on relativistic appreciation of cultural differences or comfort with diversity. Finally, the results of this study suggest that the relationship between study abroad and intercultural competence is one of selection and accentuation, holding important implications for postsecondary policy makers, higher education institutions, and college impact scholars.

Abstract Approved: ____________________________________ Ernest T. Pascarella ____________________________________ Professor, Educational Policy and Leadership Studies ____________________________________ Date

THE EFFECT OF STUDY ABROAD ON INTERCULTURAL COMPETENCE AMONG UNDERGRADUATE COLLEGE STUDENTS

by Mark Hungerford Salisbury

A thesis submitted in partial fulfillment of the requirements for the Doctor of Philosophy degree in Educational Leadership and Policy Studies in the Graduate College of The University of Iowa May 2011 Thesis Supervisor: Professor Ernest T. Pascarella

Copyright by Mark Hungerford Salisbury 2011 All Rights Reserved

Graduate College The University of Iowa Iowa City, Iowa

CERTIFICATE OF APPROVAL _______________________ DOCTORAL THESIS _______________ This is to certify that the Doctoral thesis of Mark Hungerford Salisbury has been approved by the Examining Committee for the thesis requirement for the Doctor of Philosophy degree in Educational Leadership and Policy Studies at the May 2011 graduation. Thesis Committee: ___________________________________ Ernest T. Pascarella, Thesis Supervisor ___________________________________ Michael B. Paulsen ___________________________________ Brian P. An ___________________________________ Scott F. McNabb ___________________________________ Downing A. Thomas

To my parents, Anne and Lee Salisbury, who exemplify a lifelong pursuit of learning.

ii

What we obtain too cheap, we esteem too lightly; it is dearness only that gives everything its value. Thomas Paine The Crisis, December 23, 1776

iii

ACKNOWLEDGMENTS For any substantial recognition bestowed upon an individual, there are in reality many who deserve credit for the accomplishment. To my long-time friends and family, your constant encouragement and support carried me when my confidence waned. To the faculty in the Higher Education program at the University of Iowa, your wealth of expertise and perspective provided a foundation of knowledge for which I will always be grateful. To Ernie Pascarella, Mike Paulsen, and Paul Umbach, I cannot thank you enough for the opportunity you gave me to test the deep waters of educational research. I look forward to many more years of fruitful collaboration and friendship. It is hard to put into words my appreciation for the community of graduate students with whom I had the extreme fortune of sharing this experience. You created an atmosphere of support and camaraderie that made all of the hours spent together absolutely worthwhile. In particular, Kathy Goodman and Georgianna Martin, thank you for willingly engaging in the deep, albeit sometimes meandering, dialogue that challenged me to think more deeply and consider more broadly. Finally, I share this accomplishment fully with my wife, Lynn, and my two boys, Keaghan and Reid. Every day you keep me centered, motivated, and focused. I love you more than you know.

iv

ABSTRACT During the last decade higher education organizations and educational policy makers have substantially increased efforts to incentivize study abroad participation. These efforts are grounded in the longstanding belief that study abroad participation improves intercultural competence an educational outcome critical in a globalized 21st century economy. Yet decades of evidence that appear to support this claim are repeatedly limited by a series of methodological weaknesses including small homogenous samples, an absence of longitudinal study design, no accounting for potential selection bias, and the lack of controls for potentially confounding demographic and college experience variables. Thus, a major competing explanation for differences found between students who do and do not study abroad continues to be the possibility that these differences existed prior to participation. The current study sought to determine the effect of study abroad on intercultural competence among 1,593 participants of the 2006 cohort of the Wabash National Study on Liberal Arts Education. The Wabash National Study is a longitudinal study of undergraduates that gathered pre- and post-test measures on numerous educational outcomes, an array of institutional and self-reported pre-college characteristics, and a host of college experiences. The current study employed both propensity score matching and covariate adjustment methods to account for pre-college characteristics, college experiences, the selection effect, and the clustered nature of the data to both crossvalidate findings and provide guidance for future research. Under such rigorous analytic conditions, this study found that study abroad generated a statistically significant positive effect on intercultural competence; an effect

that appears to be general rather than conditional. Moreover, both covariate adjustment and propensity score matching methods generated similar results. In examining the effect of study abroad across the three constituent subscales of the overall measure of intercultural competence, this study found that study abroad influences students diversity of contact but has no statistically significant effect on relativistic appreciation of cultural differences or comfort with diversity. Finally, the results of this study suggest that the relationship between study abroad and intercultural competence is one of selection and accentuation, holding important implications for postsecondary policy makers, higher education institutions, and college impact scholars.

vi

TABLE OF CONTENTS LIST OF TABLES............................................................................................................. iv LIST OF FIGURES .............................................................................................................v CHAPTER I. INTRODUCTION ............................................................................................1 Statement of the Problem..................................................................................9 Purpose of the Study.......................................................................................10 Research Questions.........................................................................................11 Definitions ......................................................................................................12 Significance of the Study................................................................................13 II. LITERATURE REVIEW ...............................................................................17 Theoretical Background of College Impact Research ....................................19 Intercultural Competence................................................................................23 Study Abroad and Intercultural Competence .................................................33 Predicting Participation in Study Abroad .......................................................41 Summary.........................................................................................................44 III. CONCEPTUAL FRAMEWORK AND METHODOLOGY .........................46 Conceptual Framework...................................................................................46 Methodology...................................................................................................48 Sample.....................................................................................................48 Variables .................................................................................................50 Analysis...................................................................................................54 Limitations ..............................................................................................59 IV. RESULTS .......................................................................................................77 Descriptive Statistics ......................................................................................77 Analytic Results..............................................................................................78 Covariate Adjustment .............................................................................78 Propensity Score Matching .....................................................................80 Including a Propensity Score in a Covariate Adjustment Model............82 V. DISCUSSION.................................................................................................91 The Effect of Study Abroad on Intercultural Competence.............................92 Estimating Treatment Effects on College Impact Research ...........................96 Conclusion ......................................................................................................99 VI. REFERENCES .............................................................................................101

vii

LIST OF TABLES Table 1. 2. 3. 4. 5. Names, Descriptions, and Response Options for Variables Used in this Study.......68
Descriptive Statistics for Variables Included in the Analysis of General Effects ...............74

Correlation Matrix for All Variables Included in this Study ....................................75 The Effect of Study Abroad on Intercultural Competence Using Covariate Adjustment while Accounting for the Clustering Effect. .........................................86 A Comparison of Results derived from Covariate Adjustment and Propensity Score Matching Analyzing the Effect of Study Abroad on Intercultural Competence ..............................................................................................................87 A Comparison of Propensity Score Matching Results using Various Combinations of Independent Variable Blocks to Generate the Propensity Score (listed in order from least to most biased) ......................................................88 The Effect of Study Abroad on Intercultural Competence Comparing Two Covariate Adjustment Regression Equations to Explore the Impact of a Propensity Score vs. a Direct Measure of Study Abroad Intent on the Overall Model ........................................................................................................................89 The Effect of Study Abroad on each MGUDS Subscale..........................................90

6.

7.

8.

viii

LIST OF FIGURES Figure 1. 2. 3. Conceptual Model to Examine the Effect of Study Abroad Participation on Intercultural Competence..........................................................................................63 Miville-Guzman Universality-Diversity Scale Short Form (MGUDS-S) .............64 MGUDS-Short Form Scoring Key ...........................................................................66

ix

1 CHAPTER I INTRODUCTION In the wake of the 9/11 attacks on the World Trade Center and the Pentagon, the American Council on Education (ACE) published Beyond September 11: A Comprehensive National Policy on International Education (2002), urgently calling on the federal government to reverse decades of declining emphasis on international education and foreign language study and instill new national policy that would prioritize international education. Like the challenge of Sputnik in 1957, the attacks of September 11 have brought Americas international preparedness to a crossroads. The global transformations of the last decade have created an unparalleled need in the United States for expanded international knowledge and skills. But the nation is unready. And our future success or failure in international endeavors will rely almost entirely on the global competence of our people. (p. 7) ACE advocated three policy objectives for the United States during the next century: 1) produce more international experts that align with strategic needs, 2) improve the nations ability to address and resolve global challenges, and 3) develop a globally competent citizenry and workforce (p. 9-10). ACE argued that the United States simply could not achieve any of these three goals without substantially increasing the number of Americans students who annually engage in international study abroad. Beyond September 11 was not the first clarion call for increased investment and participation in postsecondary study abroad. Over twenty years earlier, the Presidents Commission on Foreign Languages and International Study (1979) warned in its final report Strength Through Wisdom: A Critique of U.S. Capability that the shrinking number and deteriorating quality of international study opportunities in postsecondary education

2 would undermine the United States future ability to succeed in effective global diplomacy, international crisis resolution, and preventative national security. In the intervening decades, advocacy reports from the Council on International Educational Exchange (National Task Force on Undergraduate Education Abroad, 1990) and the Association of American Colleges & Universities (AAC&U) (Cornwell & Stoddard, 1999; Johnston, Jr. & Edelstein, 1993) urged postsecondary institutions to increase study abroad participation by more clearly and overtly providing support for and promotion of international study. In addition, study abroad program providers colleges, universities, and private providers were encouraged to carefully reconsider whether their programs legitimately improved their participants intercultural competency skills. During the same period, the Office of Research in the Department of Education (Adelman, 1994) and the Rand Corporation (Bikson & Law, 1994) initiated studies examining the skills most desired by multi-national employers of their future work force. These studies hypothesized that the emerging trends toward economic globalization and corporate multi-nationalism might affect the way companies evaluated potential future employees. In both cases, the researchers found a growing belief among senior executives at large corporations that college graduates with internationally compatible skills such as second language fluency and knowledge of international cultures and customs would be vital to the continued profitability of the firm and, by extension, the nation. The authors suggested that unless the public and private sector intentionally collaborated to increase the number of college graduates entering the job market with these skills, the United States would struggle to maintain a trajectory of economic growth and might well lose its position as a dominant economic power.

3 This growing emphasis on study abroad seems to have influenced an increase in participation. The compound annual rate of growth between 1993 and 1997 (10.55%) was more than three times the size of the compound annual growth rate between 1987 and 1993 (3.43%)(NCES, 2008a, 2008b). However, though the number of study abroad participants increased throughout the 1990s, the proportion of all college students who embarked on an international study experience changed little, remaining below one percent (NCES, 2008a, 2008b). Prior to 2001, while higher education institutions seemed willing to exalt the importance of international educational experiences, they seemed less willing or able to institute substantial curricular changes or reshuffle budget priorities to boldly put those words into action (ACE, 2002; Altbach & Peterson, 1998; Siaya & Hayward, 2003). Since 2001, increasing the number of college graduates with the ability to effectively interact internationally has emerged as a national policy priority (APLU, 2004; Lincoln Commission, 2005; S. Res. 2005). The final report of the National Commission on Terrorist Attacks Upon the United States, commonly referred to as The 9/11 Commission (2003) appointed by President George W. Bush to identify the steps and missteps of the U.S. intelligence community leading up to the 9/11 terrorist attacks, noted that one of the weaknesses in effective intelligence gathering and diplomacy in the Arab world was the infinitesimal number of postsecondary graduates with degrees in Arabic studies. Upon the recommendations of NAFSAs Strategic Task Force on Education Abroad (2003) and the efforts of its honorary co-chair Senator Paul Simon, Congress appropriated funds in 2003 to create The Commission on the Abraham Lincoln Study Abroad Fellowship Program (Consolidated Appropriations Act, 2004) and explore

4 ways in which the federal government might facilitate a substantial increase in annual study abroad participation. President George W. Bush greeted the 2005 publication of the Lincoln Commissions final report Global Competence and National Needs: One Million Americans Studying Abroad by signing Senate Resolution 308 (2005) to declare 2006 The Year of Study Abroad. Soon thereafter the Higher Education Reconciliation Act of 2005 created the National Sciences and Mathematics Access to Retain Talent (SMART) Grants that included scholarship funds for students majoring in foreign languages deemed critical to national security interests (Deficit Reduction Act of 2005). The efforts of the Lincoln Commission represented bold aspirations to make study abroad a more central piece of the undergraduate experience. The Commissions report (2005) established a goal of one million American students studying abroad annually by 2016-17. The commission proposed that Congress appropriate annual funding of $50 million initially with incremental increases to reach $125 million annually by 2011-12 to fund non-renewable scholarships of up to $5,000 to pay the costs of studying abroad. The Lincoln Commission proposed funding protocols that would especially encourage participation among traditionally underrepresented populations and criteria that would ensure quality control in the study abroad programs supported by this effort. Finally, the Commission proposed that higher education institutions partner with the federal government by increasing their own investment in study abroad scholarships in response to this federal initiative so that federal dollars could be most efficiently used to encourage study abroad growth across all types of postsecondary institutions. The resulting legislation, titled The Paul Simon Study Abroad Foundation Act (H.R. 1469/S. 991) in honor of the late Senator Simon who died just before the Lincoln

5 Commission was established, passed the House of Representatives unanimously in June 2007 but stalled in the Senate despite broad bi-partisan support. Although it has been reintroduced in subsequent sessions of Congress and continues to garner near unanimous support (H. R. 2410, 2009; S. 473, 2009), it has struggled to survive the lawmaking process in the face of recent grim economic realities. Nonetheless, the broad public support for study abroad generated by the Lincoln Commission helped to further galvanize higher education organizations on behalf of international education generally and study abroad in particular (Blum, 2006; Chmela, 2005; Farrell, 2007). Throughout the last decade, ACE has sponsored an extensive series of programs and publications to promote internationalization among higher education institutions through its Center for International Initiatives, emphasizing the investment institutions should make in supporting study abroad participation (ACE 2008; Field, 2009; Green, 2005; Green, Luu, & Burris, 2008; Hill & Green, 2008; Olson, Green, & Hill, 2005, 2006; Siaya & Hayward, 2003). At the same time, AAC&Us Liberal Learning and Americas Promise (LEAP) (2007) initiative further situated study abroad as an ideal means of developing one of the central educational outcomes of a liberal arts education intercultural knowledge and competence. This collaborative effort, funded generously by a host of influential educational philanthropies, sought to articulate the specific types of knowledge and skills that all undergraduates must obtain from a twenty-first century college education in order to be successful in a new global century. The authors asserted, Student success in college cannot be documented as it usually is only in terms of enrollment, persistence, and degree attainment (p. 1). AAC&U detailed four broad educational outcomes that

6 would provide a framework to guide students cumulative progress from school through college (p. 2): 1) knowledge of human cultures and the physical and natural world, 2) intellectual and practical skills, 3) personal and social responsibility, and 4) integrative learning. Study abroad participation was highlighted as an example of an educational experience that would positively influence gains in each category. In 2008, the AAC&U further emphasized the value of study abroad participation in High Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter (Kuh). This publication used National Survey of Student Engagement (NSSE) data to demonstrate that certain types of educational activities are particularly influential in producing gains across a variety of the educational outcomes outlined in the LEAP initiative. Study abroad was prominently featured as one such activity. Institutions were further encouraged to support and promote study abroad participation, especially for students with few prior cross-cultural experiences. Again, although it might be difficult to demonstrate a direct causal relationship, the growing chorus of public and private support for the importance of study abroad seems to have influenced growth in institutional support and student participation. Reporting data from an ACE survey of internationalization efforts at higher education institutions, Siaya and Hayward (2003) noted comparative increases over the previous decade in the proportion of institutions that administered study abroad programs and the proportion of institutions that provided financial support for students participating in those programs. ACE followed up these findings five years later and found that the proportion of institutions administering study abroad programs had increased again 85% in 2003 to 91% in 2008 (Green, Luu, & Burris). Moreover, despite the economic

7 recession that followed 9/11 and the accompanying American fears of travel or studying abroad (Borcover, 2002; McKeown, 2003; Siaya & Hayward), the compound annual growth rate of study abroad participation between 2002 and 2007 was 8.48% only minimally less than the 8.91% compound annual growth rate between 1997 and 2002 (NCES, 2008a, 2008b) with the absolute number of study abroad participants growing from 174,629 in 2002-03 to 262,416 in 2007-08. The growing investment and emphasis at both the federal and institutional level on increasing study abroad participation turns on the popular and long accepted belief that a study abroad experience uniquely improves the intercultural skills of all participants (Lincoln Commission, 2005; NAFSA, 2003; National Task Force on Undergraduate Education Abroad, 1990). Yet some study abroad scholars have recently questioned whether the successful push to drastically increase participation rates has drowned out efforts to ensure the educational effectiveness of the experience (Vande Berg, 2003, 2007; Woolf, 2007). Describing the originating principles behind The Fulbright Program, established in 1946 as the first and arguably most prominent federally supported study abroad scholarship program (Jeffrey, 1987), Senator J. William Fulbright (1989) declared, That ethos, in sum, is the belief that international relations can be improved, and the danger of war significantly reduced, by producing generations of leaders, especially in the big countries, who through the experience of educational exchange, will have acquired some feeling and understanding of other peoples' cultures why they operate as they do, why they think as they do, why they react as they do and of the differences among these cultures. It is possible not very probable, but possible that people can find in themselves, through intercultural education, the ways and means of living together in peace. (p. 193-194)

8 Although decades of studies seem to suggest that study abroad participation can improve a range of intercultural attitudes and skills (Pascarella & Terenzini, 2005; Sell, 1983), Pascarella and Terenzini (2005) summarized the primary methodological weaknesses that undercut the validity of these claims. All of these studies are smallscale and leave uncontrolled numerous factors on which students who study abroad are known to differ from those who do not (p. 316). This leaves open the distinct possibility that the alleged positive effects of participating in study abroad are not due to the experience itself, but rather because of specific characteristics of the participants who select to embark on an international educational experience. Spurred by the growing emphasis on assessment of student learning in higher education and calls among international educators to document student learning in study abroad (Ingraham, 2003; Vande Berg, 2007), several recent large scale studies have sought to empirically demonstrate the unique educational benefit of study abroad participation on intercultural growth (Braskamp, Braskamp, & Merrill, 2009; Sutton & Rubin, 2004, 2010; Vande Berg, Connor-Linton, & Paige, 2009). However, in each case critical weaknesses in research design or analytical methodology undermine the validity of their findings. The GLOSSARI study (Sutton & Rubin, 2004, 2010) compared students from the University of Georgia System who did and did not study abroad on seven dimensions of intercultural knowledge and skills. However, in addition to a lack of demographic similarities between treatment and control groups, the researchers used a cross-sectional design and were therefore unable to rule out the possibility that the differences revealed in their study existed prior to the study abroad students departure.

9 Both Braskamp, Braskamp, and Merrills (2009) Global Perspectives Inventory (GPI) study and Vande Berg, Connor-Linton, and Paiges (2009) Georgetown Consortium Project employed a pretest-posttest design and could therefore explore the changes in intercultural competence over time. However, neither study accounted for predeparture demographic or attitudinal differences that might 1) influence the decision to select to study abroad or 2) predispose an individual to benefit from the experience. Furthermore, in addition to substantial problems in creating control groups demographically comparable to their treatment groups, neither study employed an analytical methodology designed to account for the selection effect when examining the effect of a given treatment in quasi-experimental studies (Rosenbaum & Rubin, 1983; Rubin, 2008; Schneider, Carnoy, Kilpatrick, Schmidt, & Shavelson, 2007). Although these studies demonstrate a welcomed increased effort toward the assessment of learning through study abroad, the lack of a rigorous research design or appropriate analytical method does little to provide empirical evidence of study abroads effect on intercultural competence among undergraduate college students. Statement of the Problem Over the course of several decades, postsecondary study abroad has evolved from a selective niche educational endeavor to a national educational priority. Substantial increases in institutional support for study abroad programs and potential federal funding for study abroad scholarships have helped to boost participation. International educational opportunities, once awarded to an elite few after an exhaustive process of application and interview prior to selection (Gardner, 1968; McEvoy, 1967; Wallace, 1965), are now marketed to all students with the hope of attracting as many participants

10 as possible (Lincoln Commission, 2005; NAFSA, 2003). Some institutions have instituted, or are seriously considering, substantial curricular changes to require study abroad as a prerequisite for graduation (Chmela, 2005; Fischer, 2007). Although a wide range of governmental and institutional stakeholders agree that increasing the number of students who study abroad will increase the intercultural competency of the participants and prepare more college graduates to succeed in a 21st century globally interconnected knowledge economy, evidence to support the belief that study abroad participation guarantees the acquisition of intercultural competence is far from compelling. In fact, no conclusive evidence exists to demonstrate that the benefits of study abroad participation suggested by prior research are attributable to the study abroad experience itself and not to differences between the students who choose to participate and the students who do not. In an era of shrinking fiscal resources, higher education institutions and governing bodies need to know whether the substantial investment to increase study abroad participation can be expected to produce the anticipated educational gains for all participants. College students and their families need to know whether the personal investment of time and finances necessary to participate in a study abroad experience will produce the educational advantages the study abroad advocates claim. Purpose of the Study The purpose of this study is to estimate the unique effect of study abroad participation on intercultural competence. To ensure the methodological rigor of this study, however, this effort requires multiple stages of analysis. First, this study will examine the differences between students who participate in study abroad and students who do not and develop a model to predict the likelihood of study abroad participation.

11 Second, this study will estimate the effects of participating in study abroad on the development of intercultural competence in the presence of the potentially confounding pre-college characteristics, attitudes, and interests, institutional differences, and college experience variables identified by the predictive model for participation. In addition, this study will investigate whether those effects are conditional or general, i.e. whether differences in the effects of participating in study abroad exist based upon differences in pre-college characteristics, academic major, institutional type, and first-year experiences. Third, this study will implement and then compare two analytical procedures propensity score matching methods and covariate adjustment for estimating effects of study abroad participation using observational data in order to strengthen the robustness of the study as well as provide a deeper understanding of the value of propensity score matching methods for college impact research. Research Questions 1. What are the factors that previous research might indicate predicts participation in study abroad? 2. What are the net total and direct effects of study abroad participation on intercultural competence? 3. Are the net effects of study abroad participation on intercultural competence conditional or general? 4. How do estimates generated by propensity score matching methods compare with those generated by covariate adjustment methods?

12 Definitions 1. Study Abroad: Study abroad has evolved significantly over the last century (Hoffa, 2007; Hoffa & DePaul, 2010). Traditionally, study abroad participants traveled to a university in a western European country at the beginning of their junior year, attended courses for credit at the host institutions for a full academic year, and fully immersed themselves in the local culture by either living with a host family or living in a typical university residence with other native students. Today, study abroad participants can travel to virtually any corner of the globe, enroll in programs that last as little as three weeks or as long as a year, and experience widely divergent degrees of immersion from living with a host family to spending the entire experience with a group of like-minded American students under the close tutelage of a faculty member. For the purposes of this study, a study abroad experience will be defined to include the full range of credit-earning experiences that require international travel. 2. Intercultural Competence: An extensive body of empirical research and theoretical writing has sought to describe, define, and measure intercultural competence under a wide variety of definitions and constructs. Although no full agreement exists on a definition of intercultural competence, a recent study applied both survey and Delphi methods to bring a range of intercultural relations scholars and international education administrators together and encapsulate the many perspectives on intercultural competence into a single consensus definition that could be used as a starting point for future attempts to measure the construct (Deardorff, 2004, 2006). The resulting definition used to guide the present study describes intercultural competence as a process orientation that is organized into two levels or stages an

13 individual level and an interactional level, each containing two separate steps. Within the individual level, the first step requires that one possess attitudes of respect or value for other cultures, openness and the ability to withhold judgment, and curiosity to discover while tolerating ambiguity. The second step requires that one develop specific knowledge and comprehension that would include cultural self-awareness, deep cultural knowledge, and sociolinguistic awareness. Consequently, to continually acquire and comprehend this kind of knowledge, one must possess the skills to listen, observe, evaluate, analyze, interpret, and relate. At the interactional level, this definition of intercultural competence differentiates between two types of expected outcomes: internal and external. The internal outcomes demonstrating intercultural competence are an informed frame of reference shift that would come through increased adaptability, flexibility, ethnorelativism, and empathy. The external outcome expected of this process orientation is that all of these developmental gains are integrated holistically so that the individual demonstrates effective and appropriate communication and behavior in an intercultural situation (Deardorff, 2006, p. 256). Significance of the Study This study holds potential importance for a variety of practical, policy, theoretical, and methodological considerations. Although institutional and public policy makers, intercultural relations scholars, and international education administrators seem to firmly believe in the positive effects of study abroad for most if not all students, the lack of rigorously conducted research supporting this claim suggests the possibility that this belief may qualify as one of higher educations closely held rational myths (Pascarella,

14 2006, p. 513). This study will be the first large scale, multi-institutional study to rigorously examine the effect of study abroad participation on intercultural competence in the presence of the potential confounding effects of pre-college characteristics, academic interests and career goals, institutional type differences, and college experiences. Moreover, this study will be the first to employ the analytical procedures necessary to fully account for the selection effect, i.e. the nonrandom selection of participants into studying abroad. Although this study does not account for the wide variability in study abroad location, program length, or depth of cultural immersion, the findings of this study will provide a foundation for future research that might more precisely investigate the potential existence of differential effects among these programmatic variations. The findings of this study will be of considerable importance for public higher education policy. If the findings support the longstanding belief in the positive educational value of study abroad, they will provide the first methodologically sound evidence to date regarding study abroads unique educational importance and embolden advocates of international education to argue for continued increases in postsecondary investment in study abroad infrastructure, programs, and scholarships as well as final passage of the Senator Paul Simon Study Abroad Foundation Act. On the other hand, if the findings suggest that study abroad participation, after accounting for potentially confounding variables, does not uniquely improve intercultural competence or if the findings suggest that study abroad benefits only certain types of students because of preexisting traits, this study, at the very least, will emphasize the dangers of making policy decisions in the absence of methodologically sound evidence.

15 Efforts to identify conditional effects could hold important implications for practice as well as policy. If study abroad is particularly beneficial for certain types of students, institutions might craft targeted programs to ensure that those students participate in study abroad and maximize the benefits of the experience. Moreover, by examining the effects of differences in college experiences on the development of intercultural competence of study abroad participants, institutions may be able to construct new initiatives, or improve upon existing ones, to ensure that study abroad students are fully primed to successfully benefit from an international educational opportunity. The findings from this study will be useful for theory development as well. Although not often cited in recent research on the positive effects of study abroad, earlier studies grounded their research in the application of the contact hypothesis (Allport, 1954; Hofmann & Zak, 1969; Salter & Teger, 1975), which argued that interaction between two groups would reduce the prejudicial feelings among the individuals in both groups. Despite Allports and other scholars subsequent work demonstrating that important conditions must be met before intergroup contact can be expected to reduce prejudice and increase intercultural awareness, sensitivity, or communication (Amir, 1969; Hewstone & Brown, 1986; Pettigrew & Tropp, 2006), much of the popular argument for study abroad participation seems to stems from the supposition that sustained contact by itself will produce positive growth (Fulbright, 1989; Lincoln Commission, 2005; NAFSA, 2003). The findings from this study may provide evidence through which scholars can reconsider the application of the contact hypothesis as a rationale for promoting study abroad participation and examining its effect. More

16 broadly, these findings could provide additional evidence to show that merely participating in a high impact activity (Kuh, 2008) such as study abroad does not necessarily ensure improved educational quality (Salisbury & Goodman, 2009). Rather, as Kuh notes, high impact activities are only effective when they are purposefully conducted to elicit learning. Finally, because of the thorough design of the Wabash National Study on Liberal Arts Education (WNS) (the source of the data for this study), this study will be able to compare two analytical methods for analyzing observational data when the variable of interest is participation in a non-randomly selected treatment. Statisticians have empirically demonstrated that research exploring the effect of a non-randomly selected treatment using observational data rather than data gathered through a true experimental design can be vulnerable to type I error (i.e., finding a significant effect when, in fact, there is none) because the lack of random selection leaves open the possibility that individuals who choose to participate in the treatment are systematically different from those who do not (Dehajia & Wahba, 1999; Rosenbaum & Rubin, 1983). The findings from this section of the study will provide additional insight into determining the most appropriate research design and analytical method for accurately estimating the effect of a non-randomly assigned treatment.

17 CHAPTER II LITERATURE REVIEW American educational policy makers, employers, and higher education leaders have recognized that American college graduates must acquire the ability to communicate and collaborate across racial, ethnic, and cultural differences if they are to successfully engage, compete, and contribute in the new global century (AAC&U, 2007; APLU, 2004; Bikson, Treverton, Moini, & Lindstrom, 2003; Lincoln Commission, 2005). To this end, postsecondary institutions and organizations have invested substantial resources to increase study abroad participation with the expectation that students who live and learn in the midst of another culture for an extended period of time will develop intercultural awareness, sensitivity, and communication skills that that they could not acquire through other educational mechanisms available on campus (ACE, 2002; Green, Luu, & Burris, 2008; NAFSA, 2003). While these efforts seem to have produced growth in study abroad participation rates, the goals set forth in the Lincoln Commissions final report one million students studying abroad annually by 2016-17 or the even more ambitious goals outlined by NAFSA in Securing Americas Future: Global Education for a Global Age of 20% of all undergraduates studying abroad by 2010 are still far from being achieved. Moreover, the beneficial effects of study abroad participation on intercultural competence may be more a popular narrative than an empirically grounded claim due to methodological weaknesses in prior research (Pascarella & Terenzini, 2005), the subtle conversion of fervently advocated beliefs into presumed fact (Wolf, 2007), and policy priorities among

18 international education administrators and study abroad programs that have emphasized boosting participation rates over ensuring educational quality (Vande Berg, 2003, 2007). As higher education institutions continue to invest more heavily in internationalizing undergraduates through study abroad, and the federal government continues to contemplate a significant increase in funding for study abroad scholarships (H. R. 1496/S. 991, 2007; S. 473, 2009; S. Res. 208, 2005), it is critical to determine whether these investments can be expected to produce their intended results. The purpose of this study is to rigorously examine whether participation in study abroad uniquely improves intercultural competence even after accounting for the pre-existing characteristics or prior educational experiences of the participants. This chapter reviews the literature related to study abroad and its impact on college students intercultural competence. It is divided into four sections: 1) the theoretical underpinnings of college impact research, 2) the construct of intercultural competence, 3) prior research on the effects of study abroad, and 4) the factors that might influence a selection effect, i.e., the decision to participate in study abroad. First, this chapter reviews the theory and research that grounds the field of college impact study and guides scholars understanding of how the college experience might influence students intellectual, affective, and interpersonal growth. Second, this chapter examines the theory and literature that defines and details the construct of intercultural competence. The third section of this chapter explores the research on the effects of study abroad, particularly in the area of intercultural competence. Finally, this chapter examines the theory and research examining the factors that influence study abroad participation.

19 Theoretical Background of College Impact Research A series of seminal works now trace a long and increasingly diverse evolution of research into the impact of college on students (Bloom & Webster, 1960; Boyer & Michael, 1965; Feldman & Newcomb, 1969; Freedman, 1960; Jacob, 1957; Pascarella & Terenzini, 1991, 2005; Strang, 1937). Summing the expansive research reviewed by Feldman and Newcomb in 1969 with the breadth of studies examined by Pascarella and Terenzini for their 1991 publication How College Affects Students and its equally comprehensive 2005 sequel, these authors examined approximately 6,500 college impact studies published over a seventy-year period. Moreover, the number of studies reviewed in each publication indicates an increasing interest in the field, with Feldman and Newcombs work examining 1,500 studies published over a forty-year period, Pascarella and Terenzinis 1991 book covering about 2,600 studies published within a twenty-year period, and How College Affects Students, Vol. 2: A Third Decade of Research (Pascarella & Terenzini, 2005) reviewing almost 2,500 studies produced over a period of ten years. Collectively, this body of research has repeatedly demonstrated that the college experience or particular aspects of the college experience can significantly influence the development of college students across a range of developmental outcomes beyond the inevitable effects of maturation. This ever-broadening field of study can be roughly organized into two categories studies that endeavor to identify a particular aspect (or aspects) of the college experience that influences change in students and studies that focus on the nature of the change or development attributed to the impact of college (Pascarella & Terenzini, 2005, p. 19). Both families of theories and corresponding

20 research on college student development play an important role in framing the present study. Astins Input-Environment-Output (I-E-O) Model (1970a, 1970b) and Pascarellas General Model for Assessing the Effects of Differential Environments on Student Learning and Cognitive Development (1985) lay out two theoretical models for understanding college impact that have been repeatedly and successfully utilized to identify aspects of the college experience that influence student growth (Astin, 1977, 1993; Pascarella & Terenzini, 1991, 2005). The I-E-O Model posits that the effects of college can be isolated and measured by creating an analytical model composed of three blocks: 1) inputs the pre-college characteristics of the student including demographic traits, attitudes, values, academic preparation, personal or social attributes, and family background characteristics, 2) the environment all the features that shape the college experience including faculty, staff, peers, courses, curricula, co-curricular programs, and experiences with which the student must contend during college, and 3) outputs the knowledge, skills, attitudes, values, beliefs, interests, and behavioral traits of the students upon graduation, or departure, from college. As Pascarella and Terenzini (2005) note, this model has been especially important for research examining the effects of discrete programs that are organized or coordinated by institutional faculty or staff (p. 53). In the context of research on study abroad, scholars have applied this model to examine study abroad as both an outcome (Rust, Dhanatya, Furuto, & Kheiltash, 2007) and as an element of the collegiate environment influencing subsequent educationally important activity (Gonyea, 2008).

21 Pascarellas General Model for Assessing the Effects of Differential Environments on Student Learning and Cognitive Outcomes (1985) expands upon the I-E-O Model to isolate and differentiate several critical sets of potentially confounding variables previously absent from Astins college impact model. In addition to accounting separately for inputs and outputs, Pascarellas model divided Astins conception of the environment into four separate blocks that more specifically account for the different influences that might shape the effect of a given college experience. Pascarella conceptualized Astins environment as a function of four potentially influential factors: 1) the structural/organizational characteristics of the institution, 2) the institutional environment, 3) the interactions with agents of socialization, and 4) the quality of the student effort (Pascarella & Terenzini, 2005, p. 57). This model allows researchers to account more precisely for the separate elements of the symbiotic relationship between the traits, attitudes, and interests that can shape the individual students involvement or engagement in their own educational endeavor and the institutions array of curricular offerings, co-curricular programs, and informal experiences that can both affect the individual students likelihood of engagement as well as shape the broader institutional context. Equally important, this model is particularly critical for appropriately analyzing multi-institutional data for individual level effects (p. 56). Applied to better understand college impact, the second category of theories focuses on the nature of the change or development observed in college students. While these theories attempt to explain a range of developmental constructs such as psychosocial development (Chickering, 1969; Erikson, 1950), cognitive development (Belenky, Clinchy, Goldberger, & Tarule, 1986; King & Kitchener, 1994; Perry, 1970),

22 moral development (Kohlberg, 1971; Gilligan, 1982), identity development (Cass, 1979; DAugelli, 1994; Helms, 1990; Josselson, 1987; Phinney, 1990), or more holistic conceptions of development (Baxter Magolda, 2001; Kegan, 1982), they emanate from the pioneering developmental psychology work of Eriksons theory of psychosocial development (1950, 1959, 1968) and Piagets theory of cognitive development in children (1952, 1964) and share several commonalities. They often describe a series of stages through which an individual moves over time. Each stage requires a more complex construction of understanding than the previous stage, and membership within each subsequent stage depends upon the adoption of a new construction of knowing accumulated during a previous stage. In most cases, stages function in a sequence with transitional phases between them that are often difficult to negotiate smoothly, requiring some sort of developmental effort, and in some cases the experiences within these transitional stages can operate as impediments to development (Miller and Winston, 1990). Pascarella and Terenzini (2005) also note that, in order to move into higher developmental stages, these theories require a cognitive readiness to progress, a recognition of increased complexity insufficiently explained by the present stage, an appropriately constituted challenge to the epistemological equilibrium of the current stage, and to move into the latter stages of development the individual needs to gain the ability to experience empathy and to detach from self (pp. 49-50). In each case, these developmental theories either focus specifically on changes that occur over the course of four years in college (e.g., Perry) or include a stage that is particularly germane to explaining the development of individuals as they move through the ages of a traditional college student (e.g., Erikson, Kohlberg, Baxter Magolda).

23 Examining the effect of study abroad on the development of intercultural competence necessitates grounding in both of these theoretical families. Study abroad is a distinct program provided by higher education institutions for the educational benefit of its students and thereby fits into Astins environment block or, as a representation of the students choice to engage in an educationally productive experience, fits into Pascarellas quality of student effort block. Furthermore, participating in study abroad has long been associated with development across a range of cognitive, psychosocial, and interpersonal domains. Bearing numerous similarities to the latter stages of Kegans orders of consciousness (1982, 1994), the educational goals of study abroad participation are often described as a combination of intercultural awareness, sensitivity, knowledge, and communication skills that, when taken together, clearly rely on an interdependence between multiple developmental domains (Deardorff, 2004). King and Baxter Magolda (1996, 2005) have also argued that these various domains of development psychosocial, cognitive, identity, moral are best understood as an integrated whole when considering the development of college students. The interwoven nature of these domains is poignantly exemplified by intercultural competence a primary intended educational outcome of study abroad participation. Intercultural Competence Higher education institutions have increasingly emphasized internationalization as a core element of their strategic goals for the next century (Green, Luu, & Burris, 2008; Siaya & Hayward, 2003). When institutional strategic plans and mission statements articulate the intended educational outcomes of internationalization, they often claim that the institution will instill in its students the intercultural competency skills necessary to

24 successfully engage, compete, and thrive in an increasingly diverse and globally interconnected world (Deardorff, 2004; Knight, 2004). Unfortunately, clearly defining and measuring intercultural competence has proven more difficult. But as the construct has become an essential capacity for personal and professional success both domestically and internationally, a consensus definition has begun to emerge that provides an opportunity to assess elements of intercultural competence as an educational outcome of college (Deardorff, 2009). First, this section briefly examines the concept of competence as an educational goal. Second, it describes an emerging comprehensive definition of intercultural competence that resulted from a Delphi study of the construct (Deardorff, 2004). Third, this section examines a developmental model of intercultural maturity (King & Baxter Magolda, 2005) that applied a holistic developmental model grounded in Kegans orders of consciousness (1994) to the notion of educating college students toward intercultural competence. Finally, this section examines the similarities between the multidimensional attributes necessary to exhibit intercultural competence and the construct of universal-diverse orientation (UDO) and its accompanying measurement instrument, the Miville-Guzman Universality-Diversity Scale (Miville et al, 1999). The concept of competence has itself been the subject of some debate among educational scholars. Emerging from a movement amongst educational policy makers in the 1960s and 70s toward establishing observable and measurable minimum standards of personal and professional skills and behaviors, elementary, secondary, and postsecondary educational organizations and scholars described this approach as competency-based education (Bowden and Marton, 1998; Grant & Associates, 1979; Spady, 1977). Efforts to clearly define the intended outcome of this movement soon lagged behind its

25 increasing popularity a challenge snidely articulated in the title of William Spadys article (1977) Competency Based Education: A Bandwagon in Search of a Definition. Bowden and Marton (1998) describe four increasingly complex stages through which definitions of competency have evolved (p. 106). Early definitions focused on behavior and suggested that competence could be identified primarily through observing and evaluating workplace performance. Scholars then added the importance of previously acquired knowledge and suggested that competence was an additive construct of both knowledge and performance that should be measured separately. In its third stage, Bowden and Marton suggest that scholars integrated the measurement of knowledge and performance to accommodate the situational nature of demonstrated competence. Lastly, Bowden and Marton argue that competence is best conceptualized as a holistic construct that perpetually integrates 1) the way one sees ones role in a given situation, 2) the capacity one has to undertake that role, and 3) the integration of previously acquired knowledge and current performance within the given context to effectively accomplish a given task. Stoof, Marten, van Marrienboer, and Bastiaens (2002) argue that the difficulty in defining competency originates from efforts to define it within the framework of an objectivist paradigm that presumes competence can exist outside of a unique situational context. Stoof et al. suggest that any demonstration of competence is fundamentally influenced by three variables people, goals, and context. Furthermore, they argue that competence is most effectively defined when conceived within a constructivist paradigm that recognizes its iterative formation process and contextualized manifestation. This approach to defining competence closely resembles the epistemological underpinnings of

26 the holistic development models proposed by Baxter Magolda (2001) and Kegan (1982, 1994) that integrate multiple aspects of the cognitive, intrapersonal, and interpersonal domains and recognize that, while development along all three trajectories is necessary to move to higher orders of consciousness (Kegan) or mature levels of development (King & Baxter Magolda, 2005), the interdependence of those domains can affect the nature and process of development for each individual. Bowden and Marton (1998) proposed that educators focus on developing students toward competency construct curricula that include a myriad of ill-structured problems and require simultaneous use of all three domains as students engage an assigned educational experience. In applying a holistic perspective to propose the stages through which one would progress to achieve intercultural maturity, King and Baxter Magoldas (2005) student interviews show that study abroad participation can present students with the intercultural interactions that require them to engage the kinds of ill-structured problems that foster holistic development. Intercultural competence applies the notion of competence to describe the successful engagement or collaboration toward a single or shared set of goals between individuals or groups who do not share the same cultural origins or background. These cultural differences can arise from any combination of factors including racial, ethnic, socio-economic, religious, and national differences. Individuals or groups demonstrating intercultural competence are able in a given situation to find common purpose through mutually coordinated communication across cultures and languages (Spitzberg & Changnon, 2009, p. 2). But while intercultural competence may seem like a manageable

27 if not identifiable construct, attempts to define it have generated increasingly divergent approaches that have made it difficult to reach a consensus among scholars. In a recent meta-analysis that reviewed both definitions and developmental models of intercultural competence, Spitzberg and Changnon (2009) identified over 300 conceptual frameworks of the construct. Moreover, for almost every conceptualizations of intercultural competence identified, Spitzberg and Changnon found that a unique measurement instrument was devised to assess the validity of the particular version of the construct. While most of these instruments seem to have been employed only once, several have been more broadly utilized to measure intercultural competence (Dunn, Smith, & Montoya, 2006; Sinicrope, Norris, & Watanabe, 2003) either to compare scores across diverse populations or to test the effect of a particular treatment on intercultural competence growth. Some of the more common instruments include the Behavioral Assessment Scale for Intercultural Competence (BASIC) (Koeser & Olebe 1988; Ruben & Kealey, 1979), the Intercultural Sensitivity Inventory (ICSI) (Bhawuk & Brislin, 1992), the Intercultural Development Inventory (IDI) (Hammer, Bennett, & Wiseman, 2003), the Cross-Cultural Adaptability Inventory (CCAI) (Kelley & Meyers, 1995), the Intercultural Sensitivity Index (ISI) (Olson & Kroeger, 2001), the Assessment of Intercultural Competence (AIC) (Fantini, 2000, 2006), the Miville-Guzman UniversalityDiversity Scale (M-GUDS) (Miville, Gelso, Pannu, Lui, Touradji, Holloway, & Fuertes, 1999), and the Global Perspective Inventory (GPI) (Braskamp, Braskamp, & Merrill, 2008). Several other measures have been developed for assessment and training in the counseling profession such as the Multicultural Counseling Inventory (MCI) (Sodowsky, Taffe, Gutkin, & Wise, 1994), the Cross-Cultural Counseling Inventory-Revised (CCCI-

28 R) (LaFromboise, Coleman, & Hernancez, 1991), the Multicultural Counseling Awareness Scale: Form B (MCAS:B) (Ponterotto & Alexander, 1996; Ponterotto, Gretchen, Utsey, Rieger, & Austin, 2002), and the Multicultural Awareness Knowledge Skills Scale (MAKSS) (DAndrea, Daniels, & Heck, 1991). An instrument was also developed specifically for multicultural competence development in a higher education context called the Multicultural Competence in Student Affairs-Preliminary 2 (MCSAP2) (Pope & Mueller, 2000). After reviewing the diversity of approaches to modeling the development of, or manifestation of, intercultural competence, Spitzberg and Changnon (2009) suggest that although this range of conceptualizations provides a rich conceptual and theoretical landscape (p. 44), many of these definitions or frameworks identify similar ingredients that can be categorized into five groups: motivation (affective, emotion), knowledge (cognitive), skills (behavior, actional), context (situation, environment, culture, relationship, function), and outcomes (perceived appropriateness or effectiveness, satisfaction, understanding, attraction, intimacy, assimilation, task achievement) (p. 7). Spitzberg and Changnon found that in most cases these conceptualizations differed more in terminology than in substance. They conclude that many conceptual wheels are being reinvented at the expense of legitimate progress (p. 45) toward a common definition that would better facilitate development of instruments or methodologies to measure the construct and test its validity. In an effort to move toward a broadly accepted definition of intercultural competence and thereby allow institutions to better assess educational efforts toward intercultural competence, Deardorff (2004) conducted a two-pronged study that included

29 a survey of international education administrators regarding institutional internationalization efforts and a Delphi approach to developing a consensus definition among an international panel of 23 intercultural relations experts. Deardorff found substantial overlap among scholars and international education administrators in conceptualizing intercultural competence and organized those findings into a four-part process model (2006, p. 256). These four parts are divided into two stages individual and interaction. The individual stage encompasses two sets of attitudes or attributes. The first includes a set of attitudes including respect for or valuing other cultures, openness without judgment to intercultural learning and to people from other cultures, and a curiosity for discovery that can tolerate ambiguity during the process of exploration. The second includes an interrelated set of knowledge comprehension and interactive skills. The critical aspects of knowledge and comprehension include cultural self-awareness, a deep understanding and knowledge of culture that includes contexts, the role and impact of culture on differing worldviews, culture-specific information, and sociolinguistic awareness. The interactive skills include the ability to listen, observe, and interpret as well as analyze, evaluate, and relate knowledge gained in one setting to circumstances in a new setting. The interaction stage is also composed of two stages internal outcomes and external outcomes. The internal outcomes describe the development of several psychological traits including adaptability to different communication styles and behaviors or new cultural environments, both cognitive and emotional flexibility in selecting and using appropriate communication styles and behaviors, the development of a increasingly nuanced ethnorelative view, and an increased sense of empathy across

30 cultural differences. Finally, the desired external outcome was defined as behaving and communicating effectively and appropriately based upon ones intercultural knowledge, skills, and attitudes to achieve ones goals to some degree (Deardorff, 2006, p. 254). While the external outcome described in this model is undoubtedly influenced by the social context within which the interchange occurs (and therefore outside the control of the individual), each of the three prior stages taken together (as situated in the Deardorffs process model) represent the attitudes, values, and attributes necessary to demonstrate intercultural competence. These capacities closely match the culminating traits of each of the three multidimensional domains proposed in King and Baxter Magoldas (2005) developmental model of intercultural maturity. Constructed within a college student development paradigm, this model articulates the developmental trajectories necessary to demonstrate intercultural competence. King and Baxter Magolda build their model upon the theoretical framework for holistic development proposed by Kegan (1982, 1994), which proposes three domains of development cognitive, intrapersonal, and interpersonal (King & Baxter Magolda, p. 576). The cognitive domain addresses knowledge acquisition, comprehension, and understanding and describes cognitive development as a move from conceptualizing knowledge as universal and dichotomous (i.e. answers to dilemmas are frames simply as right or wrong) to understanding the existence and effects of multiple dimensions of nuance across all manner of individual, situational, and cultural differences. The intrapersonal domain addresses psychosocial and personal identity development within which individuals move from lacking awareness of ones own values and identity apart from external associations to embracing a dynamic sense of an internal self that willingly considers differing beliefs

31 and values through an ethnorelative lens that understands and respects difference. The interpersonal domain describes the capacity for successful interaction with diverse others. Individuals who exhibit a mature level of development in the interpersonal domain demonstrate a capacity to engage in meaningful, interdependent relationships with diverse others that are grounded in an understanding and appreciation for human differences (p. 576). The traits described within the mature stages of each domain in King and Baxter Magoldas (2005) model closely mirror the culmination of the requisite attitudes, knowledge, and skills that manifest themselves in the internal outcomes of Deardorffs (2004) consensus definition of intercultural competence. Where King and Baxter Magolda suggest that individuals in the mature stage of cognitive development have the ability to consciously shift perspectives and behaviors into an alternative cultural worldview and to use multiple cultural frames (p. 576), Deardorffs (2004) consensus definition identifies the ability to adapt to different cultural environments and the cognitive flexibility to utilize culturally-specific and appropriate communication techniques among the internal outcomes that set the stage for demonstrating intercultural competence. Where the Developmental Model of Intercultural Maturity describes the mature stage of development in the intrapersonal domain as the capacity to create an internal self that . . . considers social identities (race, class, gender, etc.) in a global and national context (p. 576), Deardorffs (2004) consensus definition describes interculturally competent individuals as possessing an ethnorelative view, a cultural and sociolinguistic self-awareness, and an openness to intercultural learning while withholding judgment. Finally, where King and Baxter Magolda describe the mature

32 stage of interpersonal development as the capacity to engage in meaningful, interdependent relationships with diverse others that are grounded in an understanding and appreciation for human differences (p. 576), Deardorffs consensus definition describes intercultural competence as a multi-pronged capacity requiring respect and a valuing of cultural diversity, the skills to listen and observe among other important elements of successful social interaction, and empathy for anothers cultural context. Although both scholars use differing terminology, clearly these models share substantial overlap in describing the cognitive, intrapersonal, and interpersonal capacities required to demonstrate intercultural competence. In summary, the developmental process toward intercultural competence might be understood along three vectors: 1) cognitive development that allows for a relativistic appreciation of similarities and differences among diverse individuals, 2) psychosocial (or intrapersonal) development that facilitates increasing comfort when engaged in interactions with diverse others, and 3) interpersonal development that empowers one to seek out diverse interactions through experiences that highlight, celebrate, or examine differences among diverse individuals or groups. As student populations have diversified and both policy makers and employers have recognized the importance of intercultural competent among college graduates, college impact scholars have focused on identifying the educational experiences that enhance student attitudes toward diversity and develop intercultural competence skills (Pascarella & Terenzini, 2005). Although typically described in the context of domestic interracial interactions, this body of research has employed dependent measures that closely mirror the cognitive, intrapersonal, and interpersonal capacities required of intercultural competence, including openness to diversity, racial tolerance, respect for

33 racial and ethnic differences, cultural knowledge, multicultural understanding, increased interracial comfort, international understanding, and pluralistic orientation (Astin, 1993; Engberg, 2007; Gurin, Peng, Lopez, & Nagda, 1999; Hurtado, Engberg, & Ponjuan, 2003; Milem, 1994; Myers-Lipton, 1996; Pascarella, Edison, Nora, Hagedorn, & Terenzini, 1996). Researchers have found that a variety of college experiences and activities can positively influence intercultural competence, including living on campus (Pike, 2002), interracial friendship groups (Antonio, 2001), diversity-related course taking (Chang, 2002), racial awareness workshops (Whitt, Edison, Pascarella, Terenzini, & Nora, 2001), leadership training (Cress, Astin, Zimmerman-Oster, & Burkhardt, 2001), volunteer or service learning experiences (Gray, Ondaatjie, Fricker, & Geschwind, 2000; Sax & Astin, 1997), and participation in a study abroad program (Carlson, Burn, Useem, & Yachimovicz, 1990; Williams, 2005). Study Abroad and Intercultural Competence As study abroad participation rates have increased and diversified across academic disciplines, researchers from a wide range of academic fields have increasingly endeavored to demonstrate the educational benefits of studying abroad (Lewin, 2009; McKeown, 2009; Savicki, 2008). International education scholars have linked study abroad participation with several indicators of general academic success including graduation rates (Posey, Jr., 2003; Sutton & Rubin, 2010), time to degree (Ingraham & Peterson, 2004; Flash, 1999), retention (Kasravi, 2009; Metzger, 2006; Young, 2003), and grade point average (Posey, Jr.; Sutton & Rubin; Thomas & McMahon, 1998). More specifically, foreign language scholars have repeatedly found study abroad to positively affect second language acquisition (Brecht, Davidson, & Ginsberg, 1993; DuFon &

34 Churchill, 2006; Freed, 1995; Magnan & Back, 2006), one of the original motivations for the creation of formal international educational experiences (Hoffa, 2007). Moreover, researchers have also found study abroad to be correlated with increased engagement in other educationally beneficial activities during college (Gonyea, 2008) as well as specific career choices after college (Armstrong, 1984; Mistretta, 2008; Norris & Gillespie, 2009; Wallace, 1999). In addition, study abroad advocates have long argued that the experience of living and learning abroad has a unique effect on many of the broad educational outcomes historically associated with a liberal arts education. Study abroad participation has been linked with aspects of psychosocial often called personal development (Gmelch, 1997; Gullahorn & Gullahorn, 1966; Kauffmann, 1983; Milstein, 2005), identity development (Dolby, 2004; Talburt & Stewart, 1999), moral or values development (Jurgens & McAuliffe, 2004; Lindsey, 2005; Ryan & Twibell, 2000), intellectual development (Barrutia, 1971; McKeown, 2009; Zhai & Scheer, 2002), and holistic development conceptualized as self-authorship (Braskamp, Braskamp, & Merrill, 2009; Du, 2007). Most prominent among the educational benefits claimed of an international educational experience, study abroad has long been considered a powerful mechanism for developing intercultural competence (Fulbright, 1989; Hoffa, 2007; Hoffa & DePaul, 2010; Lincoln Commission, 2005; NAFSA, 2003; National Task Force on Undergraduate Education Abroad, 1990). The assertion that studying abroad improves intercultural competence is grounded in the contact hypothesis. Allport (1954) proposed that prejudice held by one group toward another group could be reduced if individuals from both groups participated in

35 sustained interpersonal contact. Subsequent research testing this hypothesis under a variety of conditions identified several caveats under which intergroup relations were most likely to reduce prejudice (Hewstone & Brown, 1986; Nesdale & Todd, 2000). These conditions include equality of status during contact, a social context that supports equality between groups, collaborative engagement toward a shared goal, opportunity to develop the level of intimacy necessary to contradict previously held stereotypes, and the support of applicable authority figures. In a meta-analysis of 713 independent samples from 515 studies distributed across almost 50 years of research, Pettigrew and Tropp (2006) found strong support for the contact hypothesis. In addition, their findings suggest that although the conditions identified by prior research might facilitate a greater reduction of prejudice, the existence of those conditions were not a necessary prerequisite for prejudice to be reduced. While the contact hypothesis originated within the context of improving relations between domestic racial groups, Amir (1969) reviewed and synthesized efforts to apply the contact hypothesis to improve inter-ethnic and international relations. He argued that it was crucial to specifically examine the validity of the contact hypothesis in the context of improving ethnic relations because the assumptions underlying this theory are exemplified in the explicit or tacit objectives of various international exchange programs: student exchanges or those of professional people, organized tours and visits to foreign countries, the sending of foreign students to visit or live with native families, etc. . . . International seminars, international conferences and exhibitions, the Olympic games all these are often thought to be effective because of the opportunities for contact that they afford. The basic premise is typically that personal contact can overcome difficulties where tons of paper work and memoranda have not succeeded (p. 320).

36 Amirs review of the literature found that, although the majority of findings support the assertion that contact between groups of differing ethnic origins or nationalities was likely to produce change in the attitudes of both groups toward the other, the conditions under which this contact occurs are significantly influential in determining the direction, or intensity, of the attitudinal change. In addition to the conditions outlined by Allport (1954) that might encourage positive attitudinal change, Amir noted several conditions specific to inter-ethnic or international interactions that might inhibit positive change or even increase prejudice. These unfavorable conditions include when the contact is unpleasant, involuntary or tension laden, when one group is in a state of frustration (i.e., inadequate personality structure, recent defeat or failure, economic depression, etc.) potentially leading to ethnic scapegoating, and when the two groups find each others moral or ethical values objectionable (p. 339). In the presence of one or more unfavorable conditions, sustained contact could increase or intensify prejudice rather than reduce it. Clearly, the cognitive, intrapersonal, and interpersonal capacities necessary to exhibit intercultural competence sensitivity to cultural differences, awareness of sociohistorical cultural contexts, adaptability and flexibility to view cultural differences and contextual circumstances through an informed ethnorelative lens, and the empathy to seek deeper understanding while withholding judgment are all attributes that could mitigate the presence of the unfavorable conditions listed above and thereby increase the likelihood of a positive cross-cultural outcome. Not surprisingly, study abroad advocates and international education scholars have repeatedly sought to demonstrate the positive effect of studying abroad on intercultural competence under the presumption that the

37 conditions for intergroup contact during a study abroad experience are ideal for reducing prejudice, developing intercultural competency skills, and improving relations across cultural, ethnic, and/or national differences (Sell, 1983; Pascarella & Terenzini, 2005). A few exceptions notwithstanding (Kalunian, 1997, as cited in Pascarella & Terenzini, 2005; Patterson, 2006; Wilkinson, 1998), researchers have repeatedly found that students who study abroad demonstrate positive change on several aspects of intercultural competence upon return to their home campuses. Although the language describing the outcome of interest varies across this body of work, numerous singleinstitution, small sample studies have investigated the effect of studying abroad on three broad aspects of intercultural competence: 1) the respondents view of the host culture or country, 2) the respondents global perspective or world mindedness, and 3) the respondents intercultural awareness or sensitivity. Findings from these studies seem to suggest that students who study abroad develop a more positive view of the host culture (Bicknese, 1974; Carlson & Widaman, 1988; Cushner & Karim, 2004; Nash, 1976), expand their global perspective or world mindedness (Chieffo & Griffiths, 2004; Cushner & Mahon, 2002; Douglas & Jones-Rikker, 2001; Golay, 2006), and increase their intercultural awareness and sensitivity (Anderson, Lawton, Rexeisen, & Hubbard, 2006; Black & Duhon, 2006; Pedersen, 2009; Shaheen, 2004; Williams, 2005). But while all of these studies assert evidence to support the claim that studying abroad uniquely influences the development of intercultural competence, several methodological weaknesses undercut the validity and generalizability of their findings. First, each of these studies examines a small group of students at a particular institution, comparing them to an equally small and homogenous non-randomly assigned control

38 group. In most cases, these studies are also limited to students who participated in a particular study abroad program. Furthermore, some of these studies are even restricted to students in a specific major. Since each of the samples in these studies is far from nationally representative of all undergraduates, it is exceedingly difficult to generalize any of these findings to all college students who study abroad. Second, as Pascarella and Terenzini noted in their review of college impact research (2005), none of these studies adequately accounts for the potentially confounding demographic, attitudinal, or aspirational characteristics that might systematically differentiate between students who do and do not study abroad. Even the most extensive efforts to demonstrate study abroads effect on its participants intercultural attitudes and skills suffer from notable weaknesses in design or analysis. Although the 1984-85 Study Abroad Evaluation Project (SAEP) (Carlson, Burns, Useem, and Yachimovicz, 1990) collected data from students at multiple American institutions (N=4), applied a longitudinal design, and controlled for a host of potentially confounding variables, the analysis did not account for the possibility that effects observed may have resulted from differences between the students who selected to participate in study abroad and the students who did not. In addition, the sample of student respondents came primarily from large research institutions (the control group coming almost entirely from a single large public university), thus making it difficult to generalize the findings to students at other types of institutions, especially small, private liberal arts colleges where study abroad originated and remains more deeply institutionalized. Although the study found that students who studied abroad grew in ways that the students who stayed home did not, it was impossible to know whether that

39 growth was uniquely a function of studying abroad or rather a by-product of other observed or unobserved characteristics. The Georgetown Consortium Project (Vande Berg, Connor-Litton, & Paige, 2009) examined gains on several outcomes regarding second language acquisition and intercultural awareness and competency development across a range of study abroad program differences using a pre-test/post-test design. However this study only analyzed mean differences between the pre- and post-test scores for the treatment and control groups without controlling for any other potentially confounding characteristics. Furthermore, the control group differed substantially from the treatment group in size and distribution across institutions, making it difficult to make confident assertions about the validity of the findings. Finally, The GLOSSARI (Georgia Learning Outcomes of Students Studying Abroad Research Initiative) Project (Sutton & Rubin, 2004) endeavored to assess the effects of study abroad participation across a variety of intercultural and academic progress measures using data gathered from students at institutions throughout the University System of Georgia. However, this study failed to employ a pre-test/post-test design and only controlled for GPA and academic major in its analysis, omitting any other pre-college characteristics or attitudinal differences between student groups. Furthermore, the studys means of convenience sampling left open the significant possibility of sample selection bias. Although this study is currently implementing a second phase that attempts to correct for previous methodological and design weaknesses, the sampling frame continues to be students within the University System of

40 Georgia, again making it difficult to generalize the findings to students at private institutions. Two additional and potentially more fundamental problems undermine the validity of the existing body of research on study abroad. After reviewing the state of study abroad research almost twenty-five years ago, Church (1982) and Sell (1983) noted that the vast majority of the study abroad research to date had been constructed absent grounding in a plausible theoretical framework. Black and Mendenhall (1991) found the same to be true of the research on cross-cultural adjustment among study abroad participants. Although Pascarella and Terenzini (2005) did not assert such a stark assessment of the study abroad research involving college students, they clearly noted that the existing body of findings had failed to account for differences between students who choose to study abroad and students who choose not to study abroad. In the context of Pascarellas model for examining the impact of specific college experiences on college student educational outcomes (1985), Pascarella and Terenzinis critique of research on undergraduate study abroad seems to mirror the conclusions of these earlier reviews. Undergraduate study abroad does not occur, nor does it function, in isolation from the larger postsecondary educational enterprise. As such, any effort to examine its impact without accounting for the potential confounding effects of this context would seem incomplete. Moreover, despite an extensive body of research demonstrating that college student development across a range of cognitive, affective, or holistic domains cannot be expected to occur at a uniform pace, in a linear fashion, or within a quantifiable timeframe (Baxter Magolda, 2001; Kegan, 1982, 1994; King & Kitchener, 1994; Pascarella & Terenzini, 1991, 2005; Perry, 1970), research on the effects of study abroad

41 has consistently operated under the assumption that any change resulting from the study abroad experience should be evident immediately upon its conclusion. The second methodological weakness found both in the single institution studies as well as the larger scale studies is the lack of adequate accounting for factors that might systematically differentiate students who choose to study abroad from students who choose not to study abroad. Yet both descriptive data on study abroad participants as well as the few studies examining the factors that predict study abroad intent or participation have demonstrated the degree to which students who study abroad differ from those who remain on campus (Booker, 2001; Carlson et al, 1990; Goldstein & Kim, 2006; IIE, 2009; Koester, 1985; Lincoln Commission, 2005; NAFSA, 2003; Salisbury, Paulsen, & Pascarella, 2010, in press; Salisbury, Umbach, Paulsen, & Pascarella, 2009; Spiering & Erickson, 2006). Because any effort to isolate an effect of study abroad participation on intercultural competence needs to account for the full array of potentially confounding variables, the last section of this chapter reviews the research regarding systemic differences between students who do and do not study abroad and the factors that might predict participation. Predicting Participation in Study Abroad Study abroad advocates have long lamented the disproportionate nature of study abroad participation rates among undergraduates (Council on International Educational Exchange [CIEE], 1991; Dessoff, 2006; Koester, 1985; Lambert, 1989; Redden, 2008). Students who study abroad have historically been overwhelmingly white, more female than male by a ratio of almost 2:1, and primarily majoring in the humanities, social sciences, and fine arts (IIE, 2009; Lincoln Commission, 2005; NAFSA, 2003). Salisbury,

42 Paulsen, & Pascarella (2010, in press) found that the disparities in participation by gender and race were reflected in rates of study abroad intent. Likewise, Booker (2001) and Stroud (2010) both found that choice of major influences the decision to participate in study abroad. The findings of the Lincoln Commission (2005) echoed this disparity and pointed specifically at the more prescriptive curricular requirements of the natural sciences, pre-med, and engineering as barriers to larger participation rates among those majors. In a single institution study, Goldstein and Kim (2006) found that heightened concerns about completing a major on time were a significant negative predictor of study abroad participation. Although not specifically tied to curricular complexities, Spiering and Erickson (2006) employed a diffusion of innovation theory to examine why students who attended an informational session on study abroad chose to participate or not participate and found that students who did not participate tended to rank the complexity of the issues involved in making it possible to study abroad as the number one reason they chose not to participate. Historically, small liberal arts colleges tend to send larger percentages of their students on study abroad programs than large public institutions (Hoffa, 2007; IIE; NAFSA). Salisbury, Umbach, Paulsen, and Pascarella (2009) found that students attending research universities, regional institutions, and community colleges were substantially less likely to intend to study abroad. Potentially reflecting differences across institutional types, study abroad advocates have long argued that the cost of participation has disproportionately favored students of higher socioeconomic status (National Task Force on Undergraduate Education Abroad, 1990; Lincoln Commission). Salisbury,

43 Umbach, Paulsen, and Pascarella (2009) also found that students who intend to study abroad tend to originate from a higher socioeconomic status. In addition to longstanding demographic differences between American undergraduates who do and do not study abroad, a small subset of findings suggests that study abroad participants may differ from those who do not participate in their attitudes toward intercultural interaction, their interest in intercultural experiences, and the degree to which they value intercultural interaction as a part of their college education. As a part of the SAEP study, Carlson et al. (1990) asked both the study abroad students and the control group (i.e. students who remained on campus) a series of questions about international knowledge and interest and intercultural aptitudes at the beginning of their study, prior to departure. The researchers identified clear differences in the degree to which the two groups of students viewed American foreign policy (the study abroad students were more critical of American foreign policy), the quality of postsecondary education in Western European countries (the study abroad students held more positive opinions), and degree of interest in experiencing other cultures (the study abroad students were much more interested in experiencing other cultures than the students who were not about to study abroad). Goldstein and Kim (2006) conducted a single-institution longitudinal study to identify variables that might predict study abroad participation and found that the students who studied abroad had on average differed significantly on measures of ethnocentrism, prejudice, and intercultural attitudes. Goldstein and Kim (2006) replicated earlier findings of King and Young (1994), findings that students who studied abroad tended to be more interested in learning a foreign language. Salisbury et al. (2009) and Stroud (2010) both found that the degree to

44 which students value developing cross-cultural understanding during college significantly increased the likelihood of intending to study abroad. Salisbury and his colleagues also found that diverse experiences during the first year of college could improve the likelihood of intent to study abroad, suggesting that systemic attitudinal differences between students who intend and students do not intend to study abroad may not be entirely solidified prior to matriculation. Summary Despite a growing emphasis on postsecondary study abroad participation in response to the increasing realities of living and working in a new global century ((AAC&U, 2007; APLU, 2004; Green, Luu, & Burris, 2008; NAFSA, 2003), international education advocates have yet to fully demonstrate that participating in a study abroad program will improve intercultural competence (Vande Berg, 2007). As the above review of literature suggests, this challenge is highlighted by the lack of theoretically grounded, methodologically rigorous research on the effects of study abroad participation that fully accounts for the array of potentially confounding factors. This includes a pre-test/post-test data collection, a large sample gathered from a range of institutional types, data accounting for a wide array of pre-college characteristics, educational interests and attitudes, college experiences, and a conceptual model grounded in an empirically-vetted theoretical framework addressing the impact of college and college experiences on student development. In the absence of such a study, the present body of research examining the effect of study abroad on intercultural competence has not adequately addressed 1) whether the findings can be attributed to the study abroad experience itself or are in fact the result of

45 pre-treatment differences between those who participate and those who do not, and 2) whether findings identified immediately after the conclusion of the study abroad experience are accentuated, mitigated, or otherwise affected by the overall college experience within which participation in a study abroad program exists. The present study endeavors to address all of these design factors by 1) estimating the effects of study abroad participation on intercultural competence while accounting for the range of demographic and affective characteristics on which prior research has shown that students who do and students do not study abroad differ and 2) examining whether the effects of studying abroad on intercultural competence are conditional and differ based upon the existence of specific pre-college characteristics.

46 CHAPTER III CONCEPTUAL FRAMEWORK AND METHODOLOGY Conceptual Framework The conceptual frameworks grounding this study are Pascarellas (1985) General Model for Assessing the Effects of Differential Environments on Student Learning and Cognitive Outcomes and Astins (1970a, 1970b) input - environment - output (I-E-O) model. Astins I-E-O model asserts that the impact on a given outcome of the college experience generally or a specific activity within the college experience is a function of 1) the pre-college characteristics of the student and 2) the college environment within which the student engages in a given program, activity, or curriculum. Pascarellas General Model for Assessing the Effects of Differential Environments of Student Learning and Cognitive Outcomes expands and extends the I-E-O model to more precisely represent the symbiotic relationship between the postsecondary institution and the students educational experience within it. Pascarella portrayed his model as a function of five categories of variables that can directly or indirectly affect student learning toward educational outcomes. These categories include: 1) structural/organizational characteristics of institutions such as enrollment size, faculty/student ratio, selectivity, and the percentage of students living on campus, 2) student background/pre-college traits such as aptitude, achievement, personality, aspiration, and ethnicity, 3) interaction with agents of socialization such as interaction with faculty or peers, 4) institutional environment, and 5) quality of student effort. In this model Pascarella asserted that the structural/organizational characteristics of the institution and the student background/pre-college traits influenced each other. In

47 addition, they both shaped the institutional environment. Furthermore, the characteristics of the institution and the background of the students as well as the institutional environment influenced interactions with agents of socialization. The quality of the student effort was influenced by the student background traits, the institutional environment, and interactions with agents of socialization. Finally, student learning and cognitive development could be understood as a function of student background/precollege traits, interactions with agents of socialization, and the quality of the student effort. The goal of the present study is to examine the effect of study abroad participation on intercultural competence after accounting for the factors that predict participation in study abroad or might otherwise account for change in the outcome of interest. Furthermore, this study endeavors to examine whether the effects of studying abroad on intercultural competence are conditional or general, i.e., whether the effect of studying abroad differs based upon differences in the pre-college characteristics of the student, differences in structural or environmental characteristics of the institution, or differences in the college experiences of the student. Consequently, the conceptual model for this study is derived from Pascarellas (1985) model and is populated by variables identified through prior research as predictors of study abroad participation among college students and the variables that previous research has identified as predictors of change in intercultural competence among students who study abroad. Figure 1 portrays the hypothesized conceptual model for this study. As in Pascarellas (1985) model, institutional characteristics and pre-college student characteristics influence each other. Institutional characteristics and pre-college student

48 characteristics both influence agents of socialization. Institutional characteristics precollege student characteristics, and agents of socialization influence the quality of student effort. The treatment of interest, i.e., participation in study abroad, is influenced by precollege student characteristics and quality of student effort. Finally, the outcome of interest, in this case intercultural competence, is influenced by the pre-college student characteristics, the agents of socializations, and the treatment of interest study abroad participation. Methodology This study analyzes data from the 2006 cohort of the Wabash National Study of Liberal Arts Education (WNS). Funded by the Center of Inquiry in the Liberal Arts at Wabash College, the WNS is a multi-institutional longitudinal study focused on identifying the curricular and co-curricular experiences that impact student gains on the educational outcomes traditionally associated with a liberal arts education. The educational outcomes examined by the WNS are effective reasoning and problem solving, well-being, inclination to inquire and lifelong learning, intercultural effectiveness (i.e., intercultural competence), leadership, moral character, orientation toward life and career, political orientation, and academic motivation (Pascarella & Colleagues, 2007). Sample Responding to a national call for institutions to participate in the WNS, over 60 colleges and universities applied to be selected for the 2006 cohort. 19 four- and two-year colleges and universities were purposefully selected in an effort to create a diverse institutional sample. As a result, the institutional members of the 2006 cohort represent

49 the full range of institutional types and vary widely in selectivity, size, location, religious affiliation, and public/private control. Three of the institutions are research universities, three are regional, non-doctoral granting universities, two are community colleges, and 11 are small liberal arts colleges. Because the goal of the WNS was to examine the effects of a liberal arts education, liberal arts colleges were intentionally over-sampled. Because postsecondary study abroad participation predominantly occurs during the third or fourth year of college, the two community colleges were eliminated from the current study, resulting in a final institutional sample of 17 schools. Since the WNS was designed as a longitudinal study and focused on identifying factors that influenced change over time, data was collected from students at three points during their college careers: at the beginning of the freshman year, at the end of the freshman year, and in the second semester of the senior year. Each institution invited first-time, full-time undergraduates to participate in one of two ways. For the larger institutions, students from the incoming freshmen class were randomly invited to participate. For the smaller institutions, the entire incoming freshmen class was invited to participate. Students were promised confidentiality and a $50 stipend for participating in each data collection. 4,193 students completed pencil and paper surveys during the first data collection in the fall of 2006. These students were asked to provide extensive information regarding demographic characteristics, family background, high school experiences, aspirations for college and career, and personal attitudes, values, and beliefs. Students were also asked to complete standardized instruments for each of the educational outcomes associated with the WNS. In addition, the students provided

50 permission for the institutions to share institutional admissions data involving ACT, SAT, or COMPASS scores and awarded financial aid. 2,953 students returned in the spring of 2007 to provide data at the end of the freshman year. These students completed the same standardized instruments for each of the educational outcome measures as well as a host of data regarding curricular and cocurricular experiences during the first year. In the spring of 2010 (the fourth year of the study), 2,212 students returned to provide data during the third data collection. Once again students were asked to complete each of the standardized instruments used to measure each of the educational outcomes of interest to the WNS. The students also provided data regarding their curricular and co-curricular experiences. In addition, the institutions provided institutional data regarding enrollment and academic performance. In total, 1,865 students provided data at all three data collection points. Variables The variables utilized in this study are conceptually portrayed in Figure 1. The variables within each block were chosen for this study based upon prior research on study abroad participation and its effect on intercultural competence. Variable selection was constrained by the need to account for the clustered or nested nature of the data (Groves, Fowler, Couper, Lepkowski, Singer, & Tourangeau, 2004). Properly accounting for this issue requires that the total number of variables in the model not exceed the number of institutions, or clusters. Thus, the selection of variables was limited to a total of 17 variables 1 dependent variable and 16 independent variables. Each of the variables utilized in this analysis are further described in Tables 1 with the means,

51 standard deviations, and range of responses provided in Table 2. A correlation matrix of all the variables included in this study is portrayed in Table 3. A single variable addresses systemic differences in institutional characteristics. A dichotomous variable was employed to indicate whether or not the student attended a small liberal arts college or a large regional or research university based upon Carnegie classification (liberal arts college=1, other=0). Potential differences in institutional characteristics are also addressed by the statistical adjustment for the nested nature of the data. Variables chosen to represent pre-college student characteristics include sex (male=1, female=0), race (white=1, non-white=0), ACT score (SAT and COMPASS scores were converted to an ACT metric), a pretest score for intercultural competence (MGUDS), an attitude toward literacy scale score (PATL), and the average of the parents highest educational attainment. All of these variables were derived from data collected at the beginning of the freshman year. Variables in the block representing agents of socialization address a range of curricular and co-curricular influences. Since study abroad participants have traditionally come primarily from among the humanities, social science, and fine arts majors, a dichotomous variable was created from data gathered at the end of the senior year to identify student in these majors. Data addressing the three final potentially confounding factors was gathered twice once at the end of the freshman year and once at the end of the senior year. Two categorical variables account for the number of hours per week the student was involved in co-curricular activities during the freshman year and during the senior year, respectively. Four additional variables address the two educational

52 experiences identified through the review of literature as potentially influential in intercultural competency development. Like co-curricular involvement, these items were measured once at the end of the freshman year and once at the end of the senior year. The diverse experiences scale is a continuous variable derived from nine items that ask about the frequency of various interactions across diverse differences. Finally, the integrative learning scale is a continuous variable derived from seven items that ask about frequency or degree that the student has integrated knowledge gained in one setting into an activity or project set in a different context. A single item asking whether the student intended to study abroad represents the quality of the student effort. This item was converted to a dichotomous variable to account for those students who intended to study abroad compared with those who did not intend to study abroad or didnt know (1, 0). This item was asked during the second data collection at the end of the freshmen year. The treatment of interest for this study study abroad participation was determined by a single item asked at the end of the senior year. Finally, the outcome of interest is measured by the MGUDS one of the standardized instruments chosen by the WNS to assess intercultural competence. The Miville-Guzman Universality-Diversity Scale [MGUDS] (1999) was developed and tested to measure an individuals degree of universal-diverse orientation (UDO), i.e., the degree to which an individual possesses an attitude of awareness and acceptance of both similarities and differences that exist among people (Miville, Gelso, Pannu, Liu, Touradji, Hollowa, & Fuertes, 1999). The MGUDS total scale is composed of three subscales that contribute equally to the overarching construct. The Diversity of Contact

53 subscale measures the degree to which the respondent is interested in and values engaging in diverse social and cultural activities. The Relativistic Appreciation subscale measures the degree to which the respondent appreciates the importance of recognizing both similarities and differences among diverse groups and the connection between that recognition and their own growth. The Comfort with Difference subscale measures ones degree of comfort with the idea of diverse interactions. The authors found that this instrument correlated in theoretically expected ways with a variety of other instruments measuring empathy, racial identity, homophobia, and dogmatism (Miville et al). The external validity of the MGUDS has been supported by studies that used the instrument to successfully predict attitudes among first year college students toward diversity in a college setting (Fuertes, Sedlacek, Rogers, & Mohr, 2000) and majority students perceptions of minority mental health counselors (Fuertes & Gelso, 2000; Fuertes & Brobst, 2002). While the original MGUDS included 45 items with 15 items in each subscale, Fuertes, Miville, Mohr, Sedlacek, and Gretchen (2000) conducted a multi-stage factor analysis study and derived an equally predictive short form of the MGUDS instrument that reduced the length of the instrument to 15 questions with each subscale composed of five questions. Lau and Finney (2006) replicated this factor analysis and found the MGUDS short form to be comparably predictive of the underlying Universality-Diversity construct and its component subscales. Figures 2 and 3 provide copies of the MGUDS short form and its scoring key. Because of the exhaustive nature of the WNS and the length of many of the instruments utilized by the study, the architects of the WNS chose to use the MGUDS short form instead of the long form.

54 Alpha reliabilities for the subscales among first year students in the WNS ranged from .77 to .78, while the alpha reliability for the total MGUDS score among first year students was .85 (Pascarella & Colleagues, 2007). The predictive validity of the MGUDS is exemplified by the numerous significant and theoretically expected correlations between the MGUDS pretest scores and a variety of other WNS measures of attitudes and activities that represent attitudinal elements or behavioral manifestations of intercultural competence. Among WNS participants, the MGUDS pretest score correlates significantly (p<.05) with the respondents openness to diversity as an important element of a college education (.616), belief in the importance of cultural understanding (.511), and belief in the importance of racial understanding (.459). Furthermore, the MGUDS pretest score correlates significantly with how often respondents made friends with international students during the first year (.324), made friends with students of another race during the first year (.330), had serious conversations with students of a different race or ethnicity (.335), and tried to better understand someone elses views by imagining how an issue looks from his or her perspective (.304). Analysis This study employs two methods of analysis to examine the effect of study abroad on intercultural competence in order to ensure the robustness of the overall findings as well as provide an opportunity to compare the results of two different approaches to identifying the effects of participation in a non-randomly assigned treatment. First, this study employs a stepwise, covariate adjustment method to examine the effect of study abroad participation on intercultural competence while accounting for a variety of potentially confounding variables (Gall, Gall, & Borg, 2003; McClendon, 1994). All

55 continuous variables were standardized prior to analysis in order to produce comparable effect sizes. A series of four equations regresses the end of senior year MGUDS score on the variables from the first five blocks of the model portrayed in Figure 1. The first equation regresses the dependent variable on the pretest and study abroad participation. The immediate inclusion of the pretest allows each resultant effect of all independent variables to represent its respective effect on a change in intercultural competence. The second equation adds all of remaining pre-college characteristics variables (gender, race, pre-college tested academic preparation, parents educational attainment, socioeconomic status, and positive attitude toward literacy scale) to the first equation. The third equation adds the college experience variables (institutional type, major choice, co-curricular involvement, diverse experiences and interactions scale, and integrative learning scale) to the variables included in equation two. The fourth and final equation adds the potentially confounding variable of intent to study abroad. The specific order of these equations allows for the identification of a potential total effect followed by a potential direct effect, which in turn allows for the identification of any moderating effects as additional variables were added to the model. Importantly, because of the nested nature of the data, analytic procedures are included to account for the clustering effect (Groves et al., 2004). In addition to testing for a general effect, this study explores for the possibility of conditional effects of study abroad, i.e. the possibility that the effect of studying abroad on intercultural competence might differ based upon other characteristics. Normally, potential interaction variables are created, added to the model, and tested to determine whether their addition to the model significantly increases the models explanatory power as measured by the R-square. However, because of the need to account for the clustering

56 effect, the maximum number of variables was limited to 16, the number of institutions from which data in this study was gathered. To maintain the theoretical integrity of the study, an intermediate step is employed to determine the plausibility of any conditional effects prior to creating an interaction variable and introducing it into the full model. This study explores for the possibility of conditional effects along seven different variables - institutional type, sex, race, academic ability, pretest score, degree of diverse experiences, and degree of integrative learning experiences. Thus, the procedure described below was repeated seven times. First, the full sample is divided into two groups. For the three dichotomous variables (race, gender, and institutional type), this division point is self-evident. For the four continuous variables (academic ability, pretest score, degree of diverse experiences, degree of integrative learning experiences), the sample is divided at the mean. Second, the full regression equation is applied to both subsets of the full sample. Third, a Wald Chi-Square test is employed to determine if the difference between the coefficients generated by the study abroad participation variable is significant and can therefore be attributed to the conditional trait by which the sample was divided. Identifying the existence of a conditional effect based upon gender presents a unique challenge because some of the institutional samples do not include both men and women who had studied abroad. Therefore, the clustering adjustment procedure cannot be applied since the analysis of the male sample no longer included data from 16 institutions (i.e., aggregate units). Instead, an interaction variable that generated the cross product of studying abroad and gender (male=1) is created and introduced directly to the

57 model. A global F-test is also employed to determine whether or not the introduction of the new cross product variable significantly adds to the explanatory power of the model. The second analytic method utilized to estimate the effect of studying abroad on intercultural competence is propensity score matching. In recent decades social scientists have expressed increasing concern about the validity and reliability of research examining the effect of a particular program, activity, or experience often termed treatment in observational data where participation in the treatment is not randomly assigned (Dehejia & Wahba, 1999; LaLonde, 1986; Reynolds & DesJardins, 2009; Rosenbaum & Rubin, 1983; Rubin, 2001, 2008; Schneider, Carnoy, Kilpatrick, Schmidt, & Shavelson, 2007). This concern arises primarily from the inability of traditional covariate adjustment methods to account for unobserved characteristics upon which those who select to participate in the treatment of interest might systematically differ from those who chose not to participate. Rosenbaum and Rubin (1983) devised an analytic method by which a logistic regression equation is employed to create a propensity score representing the likelihood of participation in the treatment for each member of the sample. This new variable is then used to examine the effect of the treatment across cases that share a similar propensity for selecting into the treatment, thus minimizing the potential bias introduced by the observed selection effect. Although some postsecondary education researchers have attempted to show the importance of this method for examining a range of important research questions using observational data (Brand & Halaby, 2006; Reynolds & DesJardins, 2009; Titus, 2007), other scholars have questioned the perceived uniform effectiveness of propensity score matching to produce more accurate estimates and have shown that this approach may not always improve the

58 accuracy of the findings and is vulnerable to a range of research design and analysis decisions (Padgett, Salisbury, An, & Pascarella, in press; Peikes, Moreno, & Orzol, 2008; Shadish, Clark, & Steiner, 2008; Steiner, Cook, Shadish, and Clark, in press). Utilizing a propensity score matching method in the current study requires two stages of analysis. First, a logistic regression model is constructed to predict participation in study abroad. This model is grounded in the research reviewed in chapter 2 and mirrors the predictive model for intent to study abroad created by Salisbury, Umbach, Paulson, and Pascarella (2009). This model included all of the variables from Figure 1 with the only difference being the operationalization of the academic major. Instead of two dichotomous variables to isolate the effects of being a STEM and applied science major or an applied pre-professional major, a single dichotomous variable is employed denoting humanities, social science, and fine arts majors (1, 0). This logistic regression equation produced a coefficient for each case in the sample that was converted to a probability of participation in study abroad between zero and one and converted into a new variable titled propensity score. The second stage of this analytic method requires that cases are matched with others bearing a similar propensity for study abroad participation. This matching procedure allows the creation of several strata within which cases shared similar, but not necessarily exact, propensity scores. Cases for which there were no comparable matches were withdrawn from the analysis. Then, within each stratum, the senior year MGUDS score means are compared between those who had and those had not studied abroad. A weighted average of these results is then derived based on proportion of observations within each stratum (Epstein, Allen, & Satten, 2007; Lunceford & Davidian, 2004).

59 The current study presents a unique opportunity to directly compare the findings from covariate adjustment and propensity score matching. In particular, the existence of a variable asking whether the student intends to study abroad tacitly functions as a plausible representation of non-randomized selection. Furthermore, one analysis of propensity score methods suggests that the inclusion of a pretest may eliminate the need to employ propensity score matching (Steiner, Cook, Shadish, and Clark, in press). Thus, the final stage of analysis for the current study re-ran the analytic techniques described above with and without the MGUDS pretest measure and with and without the study abroad intent variable to identify comparative reductions in bias. Limitations This study is not without limitations. First, the construct of intercultural competence as defined by Deardorff (2006) clearly includes a behavioral component that is difficult if not impossible to fully capture in a survey instrument. In the absence of a behavioral component, some might argue that the MGUDS - the dependent measure utilized by the Wabash National Study and the present investigation only addresses the capacity for intercultural competence rather than the situationally-specific demonstration of intercultural competence. Second, there are clear limitations resulting from the sample. Although extensive efforts were made by the Wabash National Study to generate a diverse sample of institutions and students, it is not nationally representative and therefore the result of this study may be limited in their generalizability. The WNS purposefully oversampled liberal arts colleges in an effort to remain true to its primary mission as an investigation of a liberal arts education. In addition, the larger institutions included in the sample are not

60 intended to be representative of all Research I or Regional institutions as defined by Carnegie classifications. Because of the variables necessary to represent the theoretical framework of the study, the sample only includes students who participated in each of the three WNS data collections. This may have unfairly limited the sample to those who did not have competing obligations at the times that data was collected. The students in this study were paid for their participation, which may have influenced their responses. Furthermore, the students in this sample do not represent the overall racial and ethnic diversity enrolled in American postsecondary education. While weighting is sometimes used in an attempt to more accurately represent the population from which the sample was derived, it is unclear how to employ weighting procedures within the propensity score matching method chosen for this study. Since one of the goals of this study was to compare the results of covariate adjustment and propensity score matching methods, the decision was made to forego any attempt to weight findings to the samples originating population. Finally, it is not possible to adjust for non-response bias. Third, the analytic methodology chosen for this study required decisions based on competing issues. Accounting for the clustered nature of the data limited the number of variables allowed in the model. Thus, this study may have omitted important confounding factors. Moreover, although the WNS collected an extensive range of data from its respondents, it is possible that key variables for this study were not collected. The WNS did not collect data regarding the various types of study abroad programs on which student embarked. As such, this study could not account for differences in study abroad program duration, destination, or degree of cultural immersion.

61 Fourth, study abroad programs vary widely by location, duration, and degree of cultural immersion. Although these variations may well influence the effect of studying abroad on intercultural competence, those details of the respondents study abroad participation are not available in the WNS dataset. However, since further examination of the kinds of programs available to students at all of the institutions in the WNS indicates a comparable range of programmatic differences, any statistically significant findings arising from this study can be considered foundational as an average effect across programmatic variations and additional grounds for future research. To complicate measures further, students who participate in study abroad do not do so at the same point in their college careers. Although the WNS includes institutional measures that indicate if students studied off-campus during a particular semester, it became clear through a series of validity checks that the number of students indicating that they had studied abroad substantially exceeded the institutional statistics of students who had studied off campus. Since study abroad experiences do not always occur during the academic year and do not always produce academic credit, this study chose to use student self-reports of studying abroad as the independent variable of interest and, in light of this decision, was unable to control for the potentially confounding factor of when the student chose to study abroad. Finally, the measure of intent to study abroad is a dichotomous variable collected at the end of the first year and may not fully represent or account for the degree to which respondents have initiated actions that would move them from intent to participation. Some students indicating that they intend to study abroad may have already met with a study abroad advisor and be in the final stages of preparing to study abroad during the sophomore year while others may have indicated intent to study abroad but only in the

62 most abstract sense. The degree to which the intent to study abroad variable fails to capture these differences may also affect the results of this analysis.

(1) Institutional Characteristics Attends a Liberal Arts College

(3) Agents of Socialization Academic Major Co-Curricular Involvement Diverse Experiences Integrative Learning Experiences (6) Outcome of Interest (4) Quality of Student Effort Intent to Study Abroad Posttest Intercultural Competence

(2) Pre-College Characteristics Gender Race ACT Score Parents Education Socioeconomic Status Attitude toward Literacy Pretest - Intercultural Competence

(5) Treatment of Interest Study Abroad Participation

Figure 1. Conceptual Model to Examine the Effect of Study Abroad Participation on Intercultural Competence

63

64 Figure 2. Miville-Guzman Universality-Diversity Scale Short Form (MGUDS-S) The following items are made up of statements using several terms which are defined below for you. Please refer to them throughout the rest of the questionnaire. Culture refers to the beliefs, values, traditions, ways of behaving, language of any social group. A social group may be racial, ethnic, religious, etc. Race or racial background refers to a sub-group of people possessing common physical or genetic characteristics. Examples include White, Black, American Indian. Ethnicity or ethnic group refers to specific social group sharing a unique cultural heritage (i.e., customs, beliefs, language, etc.). Two people can be of the same race (e.g., White), but be from different ethnic groups (e.g., Irish-American, Italian American). Country refers to groups that have been politically defined; people from these groups belong to the same government (e.g., France, Ethiopia, United States). People of different races (White, Black, Asian) or ethnicities (Italian, Japanese) can be from the same country (United States). Instructions: Please indicate how descriptive each statement is of you by filling in the number corresponding to your response. This is not a test, so there are no right or wrong, good or bad, answers. All responses are anonymous and confidential.

1 Strongly Disagree

2 Disagree

3 Disagree a little bit

4 Agree a little bit

5 Agree

6 Strongly Agree

1._____

I would like to join an organization that emphasizes getting to know people from different countries.

2._____ 3._____ 4._____

Persons with disabilities can teach me things I could not learn elsewhere. Getting to know someone of another race is generally an uncomfortable experience for me. I would like to go to dances that feature music from other countries.

65

Figure 2 continued.

5._____

I can best understand someone after I get to know how he/she is both similar and different from me.

6._____ 7._____ 8._____ 9._____ 10.____ 11.____

I am only at ease with people of my race. I often listen to music of other cultures. Knowing how a person differs from me greatly enhances our friendship. Its really hard for me to feel close to a person from another race. I am interested in learning about the many cultures that have existed in this world. In getting to know someone, I like knowing both how he/she differs from me and is similar to me.

12.____ 13.____ 14.____

It is very important that a friend agrees with me on most issues. I attend events where I might get to know people from different racial backgrounds. Knowing about the different experiences of other people helps me understand my own problems better.

15.____

I often feel irritated by persons of a different race.

1992 Marie L. Miville Permission is granted for research and clinical use of the scale. Further permission must be obtained before any modification or revision of the scale can be made.

66 Figure 3. MGUDS-Short Form Scoring Key

Please write the Likert responses for each of the following items. Find the sum of the responses to determine the subscale and total scale scores. Subscale 1: Diversity of Contact Item 1) 4) 7) 10) 13) Likert Score _____ _____ _____ _____ _____ Subtotal _______ Subscale 2: Relativistic Appreciation Item 2) 5) 8) 11) 14) Likert Score _____ _____ _____ _____ _____ Subtotal ______ Subscale 3: Comfort with Differences Item 3) 6) 9) 12) 15) Likert Score _____ _____ _____ _____ _____ Subtotal ______

67 Figure 3 continued.
Items may or may not be reversed scored on Subscale 3. If you want to use total MGUDS-S scale score (see below), then reverse score these items for scoring consistency. TOTAL MGUDS-S SCORE Sum Subscales 1, 2, and 3 subtotals Subscale 1__________ Subscale 2__________ + Subscale 3__________ +

TOTAL

__________

1992 Marie L. Miville Permission is granted for research and clinical use of the scale and scoring key. Further permission must be obtained before any modification or revision of the scale can be made.

Table 1. Names, Descriptions, and Response Options for Variables Used in this Study Variable Name - Intercultural Competence Pretest - Intercultural Competence Posttest - Study Abroad Description The Miville-Guzman Universality Diversity Scale (MGUDS) short form completed during the first data gathering at the beginning of the freshmen year The Miville-Guzman Universality Diversity Scale (MGUDS) short form completed during the third data gathering at the end of the senior year Respondent's self-report of participation in an undergraduate study abroad experience gathered at the end of the senior year Response Options See Figures 2 and 3 See Figures 2 and 3 Have not decided, Do not plan to do, Plan to do, Done (Recoded into Done=1, Other=0) Male=1, Female=0 White=1, Other=0 scores range from 16 to 36

- Gender - Race - Pre-College Tested Academic Preparation

Gender of respondent provided by institution during the first data collection Race of respondent provided by institution during the first data collection ACT, SAT, or COMPASS score provided by the institution during the first data collection then converted to an ACT metric

68

Table 1 continued. - Parents' Educational Attainment Average of the respondent's reported parents' educational attainment provided during the first data collection at the beginning of the freshmen year (If only one parent's score was provided, this score was used in place of an average) Institutional data indicating if the respondent met the threshold for federal financial aid and received a Pell Grant A six item scale measuring the degree to which the respondent enjoys and benefits from reading and writing (WNS Alpha=.69). Constituent items include: - I enjoy reading poetry and literature. - I enjoy reading about science. - I enjoy reading about history. - I enjoy expressing ideas in writing. - After writing about something, I see that subject differently. - If I have something good to read, I am never bored. Carnegie Classification of each institution participating in the WNS based on Carnegie Foundation data Respondent majored in a humanities, fine arts, or social science field strongly disagree, disagree, neutral, agree, strongly agree scores range from 11 (did not finish high school) to 20 (both parents have earned a doctorate) Yes=1, No=0

- Socioeconomic Status - Positive Attitude Toward Literacy Scale

- Institutional Type - Humanities, Fine Arts, or Social Science Major

Liberal Arts College=1, Other=0 Yes=1, No=0

69

Table 1 continued. - Co-Curricular Involvement * Respondent's self-report of participation in co-curricular activities such as student organizations, campus publications, student government, greek organizations, athletics, etc. 0 hours, 1-5 hours, 6-10 hours, 11-15 hours, 16-20 hours, 21-25 hours, 26-30 hours, more than 30 hours

- Diverse Experiences Scale *

A nine item scale measuring the extent to which respondents' engaged in conversations, interactions, and experiences with diverse individuals involving issues of difference (WNS Alpha=.80). Constituent items include: - How often did you attend a debate or lecture on a current political/social issue during this academic year? - How often did you have serious discussions with staff whose political, social, or religious opinions were different than your own? - How often did you participate in a racial or cultural awareness workshop during this academic year? - Indicate the extent to which your institution emphasizes encouraging contact among students from different economic, social, and racial or ethnic backgrounds. - During the current school year, how often have you had serious conversations with students of a different race or ethnicity than your own? Very often, Often, Sometimes, Rarely, Never

Very little, Some, Quite a bit, Very much Never, Sometimes, Often, Very often

70

Table 1 continued. - During the current school year, how often have you had serious conversations with students who are very different from you in terms of their religious beliefs, political opinions, or personal values? - How often have you had discussions regarding intergroup relations with diverse students while attending this college? - How often have you had meaningful and honest discussions about issues related to social justice with diverse students while attending this college? - How often have you shared personal feelings and problems with diverse students while attending this college? - Integrative Learning Experiences Scale * A nine item scale measuring the extent to which the respondent has integrated knowledge, information, or skills gained in one setting into learning experiences engaged in a different environment (WNS Alpha=.76). Constituent items include: - My courses have helped me understand the historical, political, and strongly disagree, social connections of past events. disagree, neutral, agree, strongly - My courses have helped me see the connections between my agree intended career and how it affects society. - Out of class experiences have helped me connect what was learned in the classroom with life events. - Out of class experiences have helped me translate knowledge and understanding from the classroom into action. Very often, Often, Sometimes, Rarely, Never 71

Very often, Often, Sometimes, Rarely, Never

Table 1 continued. - During the current school year, how often have you worked on a paper or project that required integrating ideas or information from various sources? - During the current school year, how often have you put together ideas or concepts from different courses when completing assignments or during class discussions? - During the current school year, how often have you discussed ideas from readings or classes with other outside of class (students, family members, co-workers, etc.)? - During the current school year, how much time have your spent synthesizing and organizing ideas, information, or experiences into new, more complex interpretations and relationships? - During the current school year, how much time have you spent making judgments about the value of information, arguments, or methods, such as examining how others gathered and interpreted data, and assessing the soundness of their conclusions? Very little, Some, Quite a bit, Very much Never, Sometimes, Often, Very often

72

Table 1 continued. - Intent to Study Abroad Respondent's self-report of intent to participate in an undergraduate study abroad experience gathered during the second data collection at the end of the freshmen year Have not decided, Do not plan to do, Plan to do, Done (Recoded into Plan to do=1, Other=0; 38 respondents who indicated "Done" at this data gathering point were eliminated from the analysis)

* Two versions of these variables are used in this analysis, one from data collected at the end of the first year and one from data
collected at the end of the fourth year.

73

74

Table 2.
Descriptive Statistics for Variables Included in the Analysis of General Effects a (N=1,593) Variable MGUDS Total Score - Pretest - Diverse Contact Subscale Score - Pretest - Comfort with Diversity Subscale Score - Pretest - Relativistic Appreciation Subscale Score - Pretest MGUDS Total Score - Posttest - Diverse Contact Subscale Score - Posttest - Comfort with Diversity Subscale Score - Posttest - Relativistic Appreciation Subscale Score - Posttest Study Abroad Gender (Male=1) Race (White=1) Pre-College Tested Academic Preparation Parents' Educational Attainment Socioeconomic Status (Pell Grant Recipient=1) Positive Attitude Toward Literacy Scale Institutional Type (Liberal Arts College=1) Humanities, Fine Arts, or Social Science Major Co-Curricular Involvement - First Year Diverse Experiences - First Year Integrative Learning Experiences - First Year Co-Curricular Involvement - Fourth Year Diverse Experiences - Fourth Year Integrative Learning Experiences - Fourth Year Intent to Study Abroad
a

Mean 0 0 0 0 0 0 0 0 .438 .347 .814 0 0 .129 0 .537 .433 0 0 0 0 0 0 .660

St. Dev. 1 1 1 1 1 1 1 1 .496 .477 .390 1 1 .335 1 .499 .496 1 1 1 1 1 1 .474

Min. -5.347 -3.407 -4.743 -5.583 -4.195 -3.441 -5.377 -5.253 0 0 0 -2.738 -2.175 0 -3.258 0 0 -1.137 -2.291 -3.428 -1.119 -2.475 -3.759 0

Max 2.237 1.879 1.558 1.791 2.107 1.879 1.440 1.690 1 1 1 2.074 2.063 1 2.338 1 1 3.486 2.867 2.554 3.316 3.052 1.955 1

all continuous variables are standardized

Table 3.
Correlation Matrix for All Variables Included in this Study 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
MGUDS Total Score - Pretest - Diverse Contact Score - Pretest - Comfort with Diversity Score - Pretest - Relativistic Appreciation Score - Pretest MGUDS Total Score - Posttest - Diverse Contact Score - Posttest - Comfort with Diversity Score - Posttest - Relativistic Appreciation Score - Posttest Study Abroad Gender (Male=1) Race (White=1) Pre-College Tested Academic Preparation Parents' Educational Attainment SES (Pell Grant Recipient=1) Positive Attitude Toward Literacy Scale Institutional Type (Liberal Arts College=1) Humanities, Fine Arts, or Social Science Major Co-Curricular Involvement - First Year Diverse Experiences - First Year Integrative Learning Experiences - First Year Co-Curricular Involvement - Fourth Year Diverse Experiences - Fourth Year Integrative Learning Experiences - Fourth Year Intent to Study Abroad 1.000 0.866 0.727 0.726 0.624 0.567 0.441 0.409 0.144 -0.184 -0.163 0.036 0.059 0.031 0.415 -0.002 0.196 -0.023 0.392 0.266 -0.008 0.345 0.215 0.207 1.000 0.422 0.508 0.573 0.628 0.310 0.327 0.185 -0.174 -0.154 0.031 0.085 0.033 0.427 0.019 0.192 -0.029 0.371 0.206 -0.023 0.329 0.165 0.259 1.000 0.276 0.421 0.307 0.501 0.166 0.062 -0.103 -0.101 0.066 0.051 -0.008 0.218 -0.038 0.139 -0.023 0.270 0.167 0.006 0.235 0.126 0.096 1.000 0.443 0.335 0.212 0.492 0.067 -0.147 -0.120 -0.021 -0.014 0.047 0.300 0.013 0.113 0.005 0.256 0.263 0.006 0.225 0.222 0.098 1.000 0.855 0.713 0.724 0.204 -0.134 -0.146 0.018 0.049 0.061 0.346 0.035 0.181 -0.021 0.383 0.293 0.023 0.515 0.379 0.242 1.000 0.407 0.458 0.275 -0.137 -0.151 0.018 0.083 0.065 0.037 0.074 0.223 -0.021 0.360 0.222 0.032 0.490 0.310 0.295 1.000 0.280 0.088 -0.048 -0.084 0.034 0.029 0.027 0.180 -0.010 0.093 -0.041 0.269 0.198 0.008 0.360 0.230 0.109 1.000 0.070 -0.115 -0.090 -0.014 -0.016 0.044 0.221 0.000 0.075 0.016 0.237 0.263 0.009 0.310 0.337 0.121 1.000 -0.067 0.094 0.208 0.153 -0.076 0.124 0.146 0.141 0.058 0.092 0.099 0.069 0.102 0.144 0.415 1.000 0.038 0.072 0.052 -0.057 -0.113 0.042 -0.008 0.140 0.043 0.009 0.105 0.045 -0.075 -0.118 1.000 0.249 0.211 -0.202 0.012 0.036 -0.072 0.130 -0.218 -0.034 0.045 -0.206 -0.025 0.008 1.000 0.356 -0.260 0.225 -0.215 0.009 0.010 0.048 0.012 -0.006 -0.028 -0.037 0.079

10

11

12

75

Table 3 continued.
13 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
MGUDS Total Score - Pretest - Diverse Contact Score - Pretest - Comfort with Diversity Score - Pretest - Relativistic Appreciation Score - Pretest MGUDS Total Score - Posttest - Diverse Contact Score - Posttest - Comfort with Diversity Score - Posttest - Relativistic Appreciation Score - Posttest Study Abroad Gender (Male=1) Race (White=1) Pre-College Tested Academic Preparation Parents' Educational Attainment SES (Pell Grant Recipient=1) Positive Attitude Toward Literacy Scale Institutional Type (Liberal Arts College=1) Humanities, Fine Arts, or Social Science Major Co-Curricular Involvement - First Year Diverse Experiences - First Year Integrative Learning Experiences - First Year Co-Curricular Involvement - Fourth Year Diverse Experiences - Fourth Year Integrative Learning Experiences - Fourth Year Intent to Study Abroad 1.000 -0.278 0.112 -0.076 0.060 0.044 0.004 0.017 0.059 -0.009 -0.017 0.095 1.000 -0.008 0.150 0.054 -0.012 0.082 0.035 -0.005 0.110 0.042 -0.069 1.000 0.050 0.184 -0.031 0.267 0.263 -0.041 0.263 0.183 0.176 1.000 0.195 0.090 0.014 0.106 0.133 0.095 0.134 0.103 1.000 0.000 0.182 0.130 0.023 0.222 0.139 0.190 1.000 0.042 0.095 0.497 0.083 0.077 0.051 1.000 0.476 0.098 0.581 0.310 0.164 1.000 0.110 0.371 0.531 0.067 1.000 0.177 0.141 0.031 1.000 0.470 0.165 1.000 0.111 1.000

14

15

16

17

18

19

20

21

22

23

24

76

77 CHAPTER IV RESULTS The primary goals of this study were 1) to examine the effect of study abroad on intercultural competence in the presence of pre-college characteristics, educational aspirations, and college experiences, and 2) to explore whether the effect of studying abroad is general or conditional (i.e., whether the intercultural competence of diverse student types who study abroad is systematically effected differently). To address these questions, this study utilized two analytical methods covariate adjustment and propensity score matching. This allowed the study to compare and contrast specific findings derived from each method and strengthened the overall validity of the study. Descriptive Statistics Of the 1,865 students who provided data at all three WNS data collection points, 234 cases were removed due to missing data. In addition, 38 cases were removed because these respondents indicated at the end of the first year that they had already participated in study abroad (thus eliminating the opportunity to account for study abroad intent since this particular variable was gathered at the end of the freshmen year). The final analytic sample for this study included 1,593 students. Table 2 portrays the descriptive statistics for all of the variables included in this study. Among the students in the final sample, 693 students (43.5%) indicated that they had studied abroad. While this far exceeds the national proportion of students who study abroad, it allowed for a robust comparison of students who did and did not study abroad. Of those who studied abroad, 477 were female and 216 were male. This distribution slightly exceeds the national female/male ratio of study abroad participation of 2:1 (IIE,

78 2009). The proportion of study abroad participants in our sample identifying as white (85.57%) is slightly higher the national percentage of study abroad students who identify as white (IIE, 2009). Analytic Results Covariate Adjustment The results of the stepwise covariate adjustment analyses are depicted in Table 4. Since all continuous variables were standardized prior to analysis, the coefficients generated by each independent variable can also be understood to represent the size of their explanatory effects. The first stage of analysis examined the total effect of studying abroad on the MGUDS measure of intercultural competence after accounting for the parallel MGUDS pretest score. This equation accounts for a substantial portion of the variance (R2 = .403; p < .001). Both the pretest and study abroad participation generated significant effects (p < .001) of .607 and .236, respectively. The second stage of the analysis adds the pre-college characteristic variables of gender, race, pre-college tested academic preparation, parents educational attainment, socioeconomic status, and the positive attitude toward literacy scale score to the variables included in first equation. The addition of these variables marginally increases the explanatory power of the model (R2 = .418; p < .001), with the race variable (White=1) generating a significant negative effect (-.145, p < .01) and the positive attitude toward literacy scale generating a significant positive effect (.109, p < .001). The effect of study abroad participation remained significant (p < .001), its effect size increasing slightly to .250. The pretest continued to generate a large effect of .550 (p < .001), albeit somewhat smaller than its effect size in the first equation.

79 The third stage of the analysis adds a series of college experience variables (institutional type, major choice, and senior year co-curricular involvement, diverse experiences, and integrative learning experiences) to the list of variables included in the second stage. The addition of these variables substantially increased the explanatory power of the model (R2 = .521; p < .001). Moreover, two of the five variables added to the model in this stage generated significant effects. A one standard deviation increase in the diverse experiences scale predicts .280 of a standard deviation increase in the dependent variable (p < .001). In addition, a one standard deviation increase in the integrative learning scale predicts a .133 increase in the dependent variable (p < .001). The addition of these variables reduced the effect of studying abroad to .187, and the effect remains significant at p < .001. As might be expected, the pretest continues to generate a large and significant effect of .458 (p < .001). The fourth and final stage of the covariate adjustment analysis adds a single variable to those utilized in the third stage. This variable represents quality of student effort in the conceptual framework (see block 4 in Figure 1) and is operationalized for this study by intent to study abroad. While the addition of this variable produces a marginal increase in the models variance explained (R2 = .523; p < .001), an adjusted Wald test reveals that the addition of the intent to study abroad variable significantly improves the explanatory power of the model [F (1, 15) = 6.87; Prob > F = .0193]. Study abroad participation continues to generate a significant positive effect of .147 in the final model (p < .001). Again, the effect of the pretest score is comparatively large (.454) and significantly predictive of the post-test score of intercultural competence (p < .001).

80 Propensity Score Matching In recent years, scholars of educational research have strongly encouraged higher education researchers to apply increasingly rigorous and more precisely appropriate analytical methods in postsecondary education research, particularly in examining the effect of programs or treatments where selection occurs nonrandomly, i.e., students choose whether or not to participate (Reynolds & Desjardins, 2009; Schneider et al, 2007). Therefore, in addition to covariate adjustment methods, the current study also examined the effect of study abroad on intercultural competence using a propensity score matching method available through the psmatch2 command in the STATA statistical software package (Leuven & Sianesi, 2003). The results of this analysis are portrayed in Table 5. This method reduced the analytic sample to 1,576, eliminating cases that did not fall within boundaries of common support, i.e., groupings of study abroad participants and non-participants who had a similar propensity to participate in study abroad. In addition, because propensity score matching is designed to specifically account for the selection effect occurring pre-treatment, the only independent variables included in the model had to have occurred pre-treatment. Therefore, the end of freshman year measures of co-curricular involvement, diverse experiences, and integrative learning experiences were used in place of the equivalent senior year measures. As a result, this model did not account for any potentially confounding factors occurring simultaneous to the treatment of study abroad. Given these limitations, it may not be surprising that the result of this methodological approach overestimates the effect of study abroad when compared to the findings of the covariate adjustment model (Table 5). While study abroad participation

81 generated a significant positive effect on intercultural competence (p < .01) using the psmatch2 command, the effect size generated by the propensity score matching method (.176) is 19.7% larger than the effect generated by covariate adjustment (.147). Not surprisingly, the standard error derived from the propensity score matching model (.061) was substantially higher than that resulting from covariate adjustment (.040) and, as a consequence, the t-statistic derived from the propensity score matching method (2.19) was smaller than the t-statistic that emerged from the covariate adjustment model (3.55). A benefit of the stepwise covariate adjustment model in this study (Table 4) is the opportunity it provides to see how the inclusion of each additional group of potentially confounding independent variables influences the effect size of study abroad on intercultural competence. In an attempt to both generate similar comparative data using a propensity score matching method and to better understand how different combinations of variables might affect the results of propensity score matching methods, the propensity score matching method was repeated fifteen times using each potential combination of independent variable blocks (including each block individually) as they were utilized in the stepwise covariate adjustment model. Simply described, these blocks include the pretest, pre-college characteristics, college experiences, and study abroad intent. The results of these analyses are portrayed in Table 6. Interestingly, the least biased effect results from a model that includes only the pretest, the college experiences, and the intent to study abroad variable (.175; t = 3.08). By contrast, the full model which adds the pre-college characteristics in addition to the variable blocks listed above generates a marginally larger coefficient, albeit a small t - statistic (.176; t = 2.19). The study abroad intent variable appears to be of primary importance in generating an

82 accurate propensity score matching equation with the pretest also playing an important role. By comparison, the model that utilized only the pre-college characteristics produced the most biased effect (.371; t = 7.27). Moreover, in several instances it appears that the pre-college characteristics block did nothing to reduce estimation bias, and in some cases it appears that this block of variables actually added bias to the estimation of an effect. Including a Propensity Score in a Covariate Adjustment Model Comparing these two analytic approaches reveals a set of competing methodological benefits and limitations. Although the covariate adjustment model includes a measure of study abroad intent that might function as a measure of selection, the nonrandom nature of the study design remains, leaving open the possibility of existing selection effect bias. Furthermore, the need to account for the clustered nature of the data limited the number of variables in the model to the number of aggregates (i.e. postsecondary institutions) included in the study, ultimately requiring that this model omit the three freshman year college experience variables (co-curricular involvement, diverse experiences, and integrative learning experiences) that prior research has found to be influential in predicting study abroad intent (Salisbury, Umbach, Paulsen, & Pascarella, 2008). Conversely, since the propensity score matching method is designed to account for selection but not for potentially confounding variables that occur simultaneous to the treatment, that model did not include the three senior year college experience variables accounting for co-curricular involvement, diverse experiences, and integrative learning which the covariate adjustment model suggests are important in accurately estimating the effect of study abroad on intercultural competence.

83 Another way to use a propensity score in estimating the effect of a given treatment is to include it in a traditional covariate adjustment equation (Shadish, Clark, & Steiner, 2008). This allows the model to account for both potentially influential factors that occur simultaneous to the treatment while at the same time statistically accounting for potential selection bias. To this end, a regression equation was constructed using all of the variables included in this study (pretest, pre-college characteristics, major, institutional type, and both freshman year and senior year college experience variables) and the propensity score created by the psmatch2 command during the previous stage of this analysis. As a comparison, the regression equation was repeated using the intent to study abroad variable in place of the propensity score. Finally, because the number of variables in these models exceeds the number of institutions from which the data was gathered, the command to account for the clustered nature of the data could no longer be used, leaving open the possibility that the equation would generate underestimated standard errors and increasing the possibility of Type I error. In response, the threshold for reporting significance was reduced to p<.01 from the traditionally accepted p<.05. Table 7 portrays the results of the full covariate adjustment model including the propensity score (Model 1) as well as the comparative findings of the full covariate adjustment model including a direct measure of intent to study abroad (Model 2). In both cases, study abroad participation produced the identical significant positive effect (p < .01) of .147; the same effect size generated by the clustered covariate adjustment model employed in the first stage of this study. While slight differences in effect sizes appear throughout the two models and in comparison with the prior clustered covariate adjustment model, these differences appear to be minimal. As might be expected, when

84 the propensity score is included in the model, the intent to study abroad variable is no longer significant, likely due to substantial collinearity with the propensity score. In addition to examining the effect of study abroad on the MGUDS total score, this study sought to better understand the nature of this effect by examining the effect of study abroad on each of the subscales of the MGUDS. The three subscales measure the respondents inclination to engage in diverse contact, the respondents comfort with diverse interactions, and the respondents relativistic appreciation of differences between cultures in values, attitudes, and perspectives. The descriptive statistics of the pretest and posttest measures of each subscale are portrayed in Table 2. Based on the findings of the full covariate adjustment model portrayed in Table 7, analysis of study abroad participations effects on each of the MGUDS subscales utilized the full covariate adjustment model above including the direct measure of study abroad intent. The results of these analyses are portrayed in Table 8. Study abroad participation generated a significant (p < .001) and comparatively large positive effect on the diverse contact subscale. However, the effect of study abroad on both the comfort with diversity and relativistic appreciation subscales was nonsignificant. Finally, this study endeavored to explore whether the effect of study abroad was general or conditional, i.e. whether the effect of study abroad on intercultural competence differed based upon variations of other independent variables in the model. Examination of potential differences based upon institutional type, race, pre-college tested preparation, pretest score, degree of diverse experiences, and degree of integrative learning experiences produced no evidence of conditional effects. Although the exploration of conditional effects by gender produced some indication that study abroad might affect

85 men and women differently, the results of this analysis did not meet the significance threshold of p<.01 set to address the possibility of underestimated standard errors and subsequent Type 1 error.

86

Table 4. The Effect of Study Abroad on Intercultural Competence Using Covariate Adjustment while Accounting for the Clustering Effect (N=1,593)
Variable Intercultural Competence Pretest Study Abroad Gender (Male=1) Race (White=1) Pre-College Tested Academic Preparation Parents' Educational Attainment Socioeconomic Status (Pell Grant=1) Positive Attitude Toward Literacy Institutional Type (Liberal Arts College=1) Humanities, Fine Arts, or Social Science Major Co-Curricular Involvement - Fourth Year Diverse Experiences - Fourth Year Integrative Learning Experiences - Fourth Year Intent to Study Abroad R-squared * p<.05, ** p<.01, *** p<.001 .403 *** .418 *** .521 *** Model 1 .607 *** (.024) .236 *** (.041) Model 2 .550 *** (.019) .250 *** (.034) -.012 (.051) -.145 ** (.045) -.035 (.025) .021 (.022) .118 (.070) .109 *** (.020) Model 3 .458 *** (.018) .187 *** (.034) -.071 (.037) -.033 (.042) -.018 (.031) .025 (.024) .057 (.064) .045 (.018) -.041 (.057) -.019 (.046) -.040 (.019) .280 (.024) .133 (.027) Model 4 .454 *** (.017) .147 ** (.040) -.062 (.038) -.033 (.043) -.017 (.032) .024 (.024) .068 (.067) .042 (.019) -.045 (.059) -.029 (.043) -.040 (.019) .275 (.023) .135 (.028) .114 (.043) .523

***

***

***

*** * **

87

Table 5. A Comparison of Results derived from Covariate Adjustment and Propensity Score Matching Analyzing the Effect of Study Abroad on Intercultural Competence Method of Analysis Covariate Adjustment a Propensity Score Matching b
a

N 1,593 1,576

Effect .147** .176**

St. Error .040 .061

t - Statistic 3.69 2.89

This model accounts for the intercultural competence pretest score, gender, race, pre-college tested academic preparation, parents' educational attainment, socioeconomic status, positive attitude toward literacy scale, institutional type, major, senior year co-curricular involvement, senior year diverse experiences, senior year integrative learning experiences, intent to study abroad, and the clustering effect. b This model accounts for all equivalent pre-treatment variables: intercultural competence pretest score, gender, race, pre-college tested academic preparation, parents' educational attainment, socioeconomic status, positive attitude toward literacy scale, institutional type, major, first year co-curricular involvement, first year diverse experiences, first year integrative learning experiences, and intent to study abroad.

* p<.05, ** p<01, *** p<.001

Table 6.
A Comparison of Propensity Score Matching Results using Various Combinations of Independent Variable Blocks to Generate the Propensity Score (listed in order from least to most biased) a Combination of Included Variables Pretest, college experiences, and study abroad intent Pretest, pre-college characteristics, college experiences, and study abroad intent Pretest, pre-college characteristics, and study abroad intent College experiences and study abroad intent Pretest and study abroad intent Pre-college characteristics, college experiences, and study abroad intent Pretest, pre-college characteristics, and college experiences Study abroad intent only Pretest and college experiences Pre-college characteristics and study abroad intent Pretest and pre-college characteristics Pretest only Pre-college characteristics and college experiences College experiences only Pre-college characteristics only
a

N 1,577 1,576 1,557 1,587 1,577 1,579 1,565 1,593 1,586 1,567 1,576 1,592 1,560 1,584 1,590

Coef. .175** .176** .187*** .200*** .211*** .212*** .221*** .231*** .235*** .238*** .261*** .289*** .300*** .326*** .371***

St. Error .057 .058 .058 .057 .055 .061 .054 .056 .050 .058 .051 .049 .054 .050 .051

t - Statistic 3.08 2.19 3.21 3.52 3.85 3.49 4.08 4.14 4.69 4.05 5.08 5.87 5.55 6.49 7.27

These equations includes blocks of variables organized by the pretest (MGUDS measure of intercultural competence gathered at the beginning of the freshmen year), pre-college characteristics (gender, race, pre-college tested academic preparation, parents' educational attainment, socioeconomic status, positive attitude toward literacy scale), college experiences (institutional type, major, first year co-curricular involvement, first year diverse experiences, first year integrative learning experiences) and intent to study abroad.

* p<.05, ** p<01, *** p<.001

88

89 Table 7.
The Effect of Study Abroad on Intercultural Competence Comparing Two Covariate Adjustment Regression Equations to Explore the Impact of a Propensity Score vs. a Direct Measure of Study Abroad Intent on the Overall Model (N=1,593). Variable Intercultural Competence Pretest Study Abroad Gender (Male=1) Race (White=1) Pre-College Tested Academic Preparation Parents' Educational Attainment Socioeconomic Status (Received Pell Grant=1) Positive Attitude Toward Literacy Institutional Type (Liberal Arts College=1) Humanities, Fine Arts, or Social Science Major Co-Curricular Involvement - First Year Diverse Experiences - First Year Integrative Learning Experiences - First Year Co-Curricular Involvement - Fourth Year Diverse Experiences - Fourth Year Integrative Learning Experiences - Fourth Year Propensity Score Intent to Study Abroad R-squared ** p<.01, *** p<.001 Model 1 .462 (.025) .147 (.041) -.051 (.044) -.030 (.054) -.014 (.046) .024 (.022) .071 (.056) .045 (.022) -.043 (.073) -.028 (.040) -.030 (.021) -.027 (.024) -.013 (.025) -.025 (.021) .289 (.024) .143 (.022) -.022 (.438) 0.126 (.169) .524 Model 2 .462 *** (.021) .147 *** (.040) -.050 (.039) -.031 (.050) -.016 (.021) .023 (.019) .072 (.056) .045 (.020) -.046 (.038) -.028 (.038) -.030 (.020) -.027 (.024) -.014 (.023) -.025 (.021) .289 *** (.024) .143 *** (.022)

*** **

*** ***

***

.118 (.042) .524

** **

90 Table 8. The Effect of Study Abroad on each MGUDS Subscale (1,593) a MGUDS Subscale Diverse Contact Comfort with Diversity Relativistic Appreciation
a

Coefficient .251*** .058 -.017

St. Error .040 .048 .048

t - Statistic 6.22 1.20 -.34

Each equation accounts for the equivalent subscale pretest score, gender, race, pre-college tested academic preparation, parents' educational attainment, socioeconomic status, positive attitude toward literacy scale, institutional type, major, first year co-curricular involvement, first year diverse experiences, first year integrative learning experiences, senior year cocurricular involvement, senior year diverse experiences, senior year integrative learning experiences, and intent to study abroad.

*** p<.001

91 CHAPTER V DISCUSSION Despite widespread agreement among policy makers and study abroad advocates regarding the educational role of study abroad in developing intercultural competence, no prior methodologically rigorous, theoretically grounded research could be found to empirically support this claim. This study endeavored to determine the effect of study abroad participation on intercultural competence after accounting for the influence of an equivalent pretest measure, pre-college characteristics, college experiences, and study abroad intent. In addition, it sought to determine whether any identified effect of studying abroad was general or differed systematically based on the condition of a third independent variable. The current study appears to be the first to attempt such a fully specified analysis utilizing appropriately rigorous methodological analyses. An additional purpose of this study was to compare and contrast results derived from covariate adjustment and propensity score matching methods in the hope of providing additional guidance for future scholars interested in estimating the effect of a given treatment in a quasi-experimental study where assignment to the treatment is not random. Although research methodologists have implored educational scholars to utilize increasingly rigorous analytic methods, little comparative scholarship exists to compare analytic methods and guide selection of the most appropriate rather than merely the most complicated or arcane means of analysis. The current study utilized data from the 2006 cohort of the Wabash National Study of Liberal Arts Education. This large-scale multi-institution research project of the factors that influence development on the historically articulated educational outcomes of

92 a liberal arts education gathered data at the beginning of the freshman year, the end of the freshman year, and the end of the senior year from students at 19 institutions. These colleges and universities differed across a variety of domains including size, selectivity, geographic location, public/private control, and Carnegie classification. The Effect of Study Abroad on Intercultural Competence The results of the current study suggest that, even in the presence of a pretest score, a variety of pre-college characteristics and college experiences, and either a direct measure of intent to study abroad or a statistical adjustment for the selection effect, on average studying abroad significantly affects the positive development of intercultural competence. Furthermore, this effect appears to be general rather than conditional. This analysis found no evidence to indicate that the effect of studying abroad varies systematically by gender, race, SES, institutional type, pre-college tested academic preparation, pretest score, or college experiences. In recent years, a growing number of study abroad advocates and scholars have recognized that in an age of increased calls for accountability in higher education the field of international education can no longer rely solely on a belief in study abroads unique educational value (Vande Berg, 2007). This study provides clear evidence that studying abroad provides an educational benefit regardless of the students pre-college background, educational aspirations, or college experiences. This finding holds potentially powerful implications for educational policy makers, higher education institutions, and study abroad providers. Although international education advocates have long asserted the educational benefits of study abroad, the lack of rigorously obtained evidence supporting this claim has hindered the ability of

93 postsecondary institutions and governmental bodies to enact large-scale financial legislation to increase study abroad participation. Based on these findings, the federal government can enact the Senator Paul Simon Study Abroad Act (Lincoln Commission, 2005) backed by empirically derived evidence that the passage of this legislation might produce its intended objectives. Moreover, in an era of shrinking public resources, these findings provide some evidence for postsecondary education policy makers at both the state and federal level who are faced with the unenviable responsibility of choosing which programs or scholarships to fund from an array of educational funding needs. Likewise, these findings should provide higher education institutions with even more incentive to invest in study abroad and to find ways to increase student participation. Despite extensive efforts at many institutions, diversifying the student population is often a challenge fraught with complex obstacles that often exceed institutional resources. The results of this study suggest that if institutions make the concerted effort to increase study abroad participation, not only can these opportunities provide the cross-cultural experiences less available to students on their home campuses; these experiences can be expected to develop intercultural competence, an educational outcome widely recognized as vital to the success of college graduates in the 21st century. Yet the findings of this study regarding the nature of study abroads effect suggest that study abroad participation does not influence the multiple domains that encapsulate intercultural competence equally. In fact, this study suggests that while study abroad participation increases ones inclination toward diverse contact, it has no statistically significant effect on ones comfort with diverse interactions or ones relativistic appreciation of cultural differences. While study abroad professionals and advocates

94 might be encouraged by the overarching effect identified in this study, the underlying findings regarding the nature of this effect should present reasons for concern. According to Deardorffs work toward a consensus definition of intercultural competence (2004, 2006), individuals demonstrating intercultural competence appreciate the ethnorelativity of cultural differences and demonstrate comfort when engaged in diverse interactions. If studying abroad only increases the inclination toward diverse contact but does not contribute to growth along other domains as the present research suggests, then study abroad may not be contributing to the kind of holistic transformative effect that it claims. Even though study abroad appears to significantly contribute to growth on an overall measure of intercultural competence, the underlying findings leave study abroad susceptible to being little more than educational tourism. Study abroad providers focused on influencing student growth along each of these domains might consider introducing more intentionally developed educational experiences prior to departure, throughout the experience abroad, and upon return to ensure that students engage opportunities for growth across all three domains. The findings of this study present a number of inviting opportunities for future research. Since this study examines study abroad generally and does not parse study abroad experiences by length of program, type of immersion, or location of study, these findings merely scratch the surface in seeking to understand the relationship between study abroad and intercultural competence development. Given the vast differences in study abroad experiences based on various potential combinations of location, length of stay, and depth of immersion, it is entirely plausible to suspect that variation along each of those vectors might uniquely influence the degree and nature of its effect.

95 Furthermore, such potential differences might be additionally influenced by individual student characteristics or even differences in institutional type or culture. Recognizing the degree to which this series of factors might impact the effect of the experience further emphasizes the need for extensive additional research to tease out the significant features that might make specific program designs ideally suited, or specifically unsuitable, for certain student types. Instead of focusing primarily on overall participation rates, international education and postsecondary education generally might better meet increasing standards of accountability for learning by research that reveals whether certain types of study abroad programs are better suited for certain types of students. This kind of research would allow institutions to align specific students with specific programs and better meet diverse student needs and goals. In addition, the findings of the covariate adjustment models in this study demonstrate that other college experiences contribute to intercultural competence development. The experience scales included in this study diverse experiences and integrative learning experiences largely assess experiences that are not specific to discrete programs. Instead, they describe opportunities for interaction, either between individuals or between learning contexts. By its very nature, study abroad can often provide many such experiences. College impact scholars might explore ways in which campus experiences included in this study might accentuate, or minimize, the educational benefits of study abroad. The findings of this study appear to confirm the core suppositions of Allports contact hypothesis (1954). Participation in study abroad an experience that by its very nature sets the stage for a period of increased contact across a given cultural difference

96 seems to produce a statistically significant increase in gains on a measure of intercultural competence not experienced by those who do not study abroad. However, after considering the disproportionate effects across the three domains of intercultural competence unveiled by this analysis, these findings may suggest no more than a partial confirmation of the contact hypothesis. On the other hand, it may be important not to overemphasize the import of these findings in the context of an educational outcome as multi-faceted as intercultural competence. As Baxter Magoldas research on the development of self-authorship has demonstrated (2001), an individuals cognitive, affective, and interpersonal development does not stop upon graduation from college. The study abroad participants in this study may indeed grow in their comfort with diversity and relativistic appreciation as a result of their study abroad experiences just not within the confines of the college experience. Estimating Treatment Effects in College Impact Research This study also holds some important implications for educational researchers focused on the impact of the college experience on learning. Although experimental design remains the gold standard in social science research for obtaining unbiased estimates, the ethical obligations and practicalities of engaging in educational research often require a quasi-experimental study design. Given this reality, educational scholars are regularly faced with the necessity of accounting for the non-random assignment of participants to the treatment of interest. This study used an examination of study abroad, a college experience in which students choose to participate, as an opportunity to compare two analytic methods of estimating the effect of a given treatment covariate adjustment and propensity score matching.

97 Although recommendations to use propensity score matching methods instead of traditional covariate adjustment methods in educational research have increased in recent years, the results of this study suggest that this dichotomy may at the very least be misleading. Propensity score matching methods are designed to statistically adjust for the possible existence of an observed selection effect. However, most matching procedures including the method used in this study are limited to accounting for pre-treatment differences that might, when taken together, approximate a selection effect. Any effort to isolate the unique effect of a given treatment in a quasi-experimentally designed study of college experiences will most likely also need to account for potentially confounding factors that occur simultaneous to the treatment. In such cases, most propensity score matching methods would be conceptually inappropriate. This is not to say that a propensity score cannot function as a useful or potentially important variable. In the present study, when the effect of studying abroad was measured with the full covariate adjustment model (Table 7), the propensity score and the direct measure of study abroad intent functioned similarly in impacting the estimated effect of study abroad participation. Additional comparative analysis using different combinations of variables to generate the propensity score produced similar results. But in each case, results were derived through covariate adjustment with the inclusion of a propensity score as an independent variable. The more critical issue identified by this study appears to be the study design and the array of variables gathered by researchers and available for inclusion in the analysis. The longitudinal design of the Wabash National Study of Liberal Arts Education required that pretest scores of each outcome measure were gathered at the beginning of the study.

98 The analysis portrayed in Table 6 demonstrates the important role of the pretest in reducing bias when using propensity score matching methods. Without a pretest, this study would have likely reported a substantial overestimation of study abroads effect. Likewise, Tables 4 and 7 portray the impact of the pretest in predicting intercultural competence at the end of the senior year using a covariate adjustment approach. It appears that, no matter the method of analysis, the absence of a pretest would substantially undermine the accuracy of the finding. Likewise, the direct measure of study abroad intent significantly reduced bias in estimating study abroads effect on intercultural competence in both analytic approaches. As portrayed in Tables 4, 6, and 7, the direct measure of study abroad intent played an important role in reducing estimation bias using either analytic method. Again, the longitudinal design of the Wabash National Study allowed researchers to gather this measure of intent. In the context of recent research examining the complex combination of factors that predict intent to study abroad (Salisbury, Umbach, Paulsen, & Pascarella, 2008), a cross-sectional study endeavoring to answer the research questions of this study would have provided an insufficient range of variables to rigorously obtain results. Ultimately, based on the comparative findings of this study, covariate adjustment appears to be an entirely sufficient method for estimating the effect of a treatment where selection into the treatment is non-random. However, to generate a least biased estimate, the choice of study design and the variables gathered during the course of the data collection seem more important than the method of analysis.

99 Conclusion The increasingly compelling argument for internationalization in higher education and for study abroad in particular is rooted in the undeniable realities of globalization. College students graduate into an economic, technological, and socio-cultural environment that is globally intertwined and interdependent; where a financial crisis in one country can unhinge a another nations economy and where the online support for ordering flowers occurs on the edge of a desert on the other side of the world (Friedman, 2005). At the same time, in an era of alarming increases in the costs of postsecondary education, the public has demanded greater accountability from higher education institutions in documenting and demonstrating educational value. Moreover, education policy makers have emphasized the use of rigorous analytic methods before making the causal claims on which governmental bodies might base funding decisions. In this context, international education advocates have recognized that impassioned arguments and anecdotal vignettes are no longer enough to obtain the public funding required to increase and diversify participation in study abroad. This research is the first to document the effect of study abroad on a measure of intercultural competence using conceptually and statistically appropriate analytic methods to make causal claims about a non-randomly selected treatment in large-scale data gathered through a quasi-experimental study. Furthermore, this study provides important methodological guidance for future educational researchers seeking to make causal statements about the effects of participation in a particular curricular or cocurricular experience on postsecondary educational outcomes. As postsecondary education continues to diversify and grow in the face of an increasingly complex and

100 interculturally-dependent post-graduate reality, it is critical for educational researchers to expand on these substantive and methodological findings to broaden understanding of the potentially important educational experiences that will prepare all students to succeed in a twenty-first century globally interconnected world.

101 REFERENCES Adelman, C. (1994). What employers expect of college graduates: International knowledge and second language skills. (Department of Education publication #OR-94-3215). Washington DC: U.S. Government Printing Office. Allport, G. W. (1954). The nature of prejudice. Cambridge, MA: Addison-Wesley. American Council on Education. (2002). Beyond September 11: A comprehensive national policy on international education. Washington DC: American Council on Education. American Council on Education. (2008). College-bound students interests in study abroad and other international learning activities. Washington DC: American Council on Education. Amir, Y. (1969). Contact hypothesis in ethnic relations. Psychological Bulletin, 71, 319342. Anderson, P. H., Lawton, L., Rexeisen, R. J., & Hubbard, A. C. (2006). Short-term study abroad and intercultural sensitivity: A pilot study. International Journal of Intercultural Relations, 30, 457-469. Antonio, A. (2001). Diversity and the influence of friendship groups in college. Review of Higher Education, 25, 63-89. Armstrong, G. K. (1984). Life after study abroad: A survey of undergraduate academic and career choices. The Modern Language Journal, 68, 1-6. Association of Public and Land-Grant Universities. (2004). A call to leadership: The presidential role in internationalizing the university. Final report from the Task Force on International Education to the National Association of State Universities and Land Grant Colleges. Retrieved from http://www.aplu.org/NetCommunity/Document.Doc?id=340 Association of American Colleges and Universities. (2007). College learning for the new global century. A report from the National Leadership Council for Liberal Education & Americas Promise. Retrieved from http://www.aacu.org/leap/documents/GlobalCentury_final.pdf Astin, A. (1970a). The methodology of research on college impact (I). Sociology of Education, 43, 223-254. Astin, A. (1970b). The methodology of research on college impact (II). Sociology of Education, 43, 437-450.

102 Astin, A. (1977). Four critical years: Effects of college on beliefs, attitudes, and knowledge. San Francisco: Jossey-Bass. Astin, A. (1993). What matters in college? Four critical years revisited. San Francisco: Jossey-Bass. Altbach, P. G., & Peterson, P. M. (1998). Internationalize American higher education? Not exactly. Change, 30(4), 36-39. Barrutia, R. (1971). Study abroad. The Modern Language Journal, 55, 232-234. Baxter Magolda, M. (2001). Making their own way: Narratives for transforming higher education to promote self-development. Sterling, VA: Stylus. Belenky, M., Clinchy, B., Goldberger, N., & Tarule, J. (1986). Womens ways of knowing: The development of self, voice, and mind. New York: Basic Books. Bhawuk, D. P. S., & Brislin, R. (1992). The measurement of intercultural sensitivity using the concepts of individualism and collectivism. Intercultural Journal of Intercultural Relations, 16, 413-436. Bicknese, G. (1974). Study abroad Part I: A comparative test of attitudes and opinions. Foreign Language Annals, 7, 325-336. Bikson, T. K., & Law, S. A. (1994). Global preparedness and human resources: College and corporate perspectives. Santa Monica, CA: RAND Corporation Institute on Education and Training. Bikson, T. K., Treverton, G. T., Moini, J., & Lindstrom, G. (2003). New challenges for international leadership: Lessons from organizations with global missions. Santa Monica, CA: RAND Corporation National Security Research Division. Back, H. T., & Duhon, D. L. (2006). Assessing the impact of business study abroad programs on cultural awareness and personal development. Journal of Education for Business, 81(3), 140-144. Black, J. S., & Mendenhall, M. (1991). The u-curve adjustment hypothesis revisited: A review and theoretical framework. Journal of International Business Studies, 22, 225-247. Bloom, B. S., & Webster, H. (1960). The outcomes of college. Review of Educational Research, 20, 321-333. Blum, D. E. (2006). Seeking to prepare global citizens, colleges push more students to study abroad. Chronicle of Higher Education (October 27). Retrieved from http://chronicle.com/article/Seeking-to-Prepare-Global-C/23025/

103 Booker, R. W. (2001). Differences between applicants and non-applicants relevant to the decision to apply to study abroad. (Ph.D., University of Missouri-Columbia). Bowden, J., & Marton, F. (1998). The university of learning: Beyond quality and competence. London: Kogan Page. Borcover, A. (2002, September 8). Travel industry still feeling 9/11. Chicago Tribune (Chicagoland Final ed.), 2. Retrieved from http://infoweb.newsbank.com Boyer, E. L., & Michael, W. B. (1965). Outcomes of college. Review of Educational Research, 35, 277-291. Brand, J. E., & Halaby, C. N. (2006). Regression and matching estimates of the effects of elite college attendance on educational and career achievement. Social Science Research, 35, 749-770. Braskamp, L. A., Braskamp, D. C., & Merrill, K. C. (2008). Global perspective inventory (GPI): Its purpose, construction, potential uses, and psychometric characteristics. Chicago: Global Perspective Institute Inc. Retrieved from https://gpi.central.edu/supportDocs/manual.pdf Braskamp, L. A., Braskamp, D. C., & Merrill, K. C. (2009). Assessing progress in global learning and development of students with education abroad experiences. Frontiers: The Interdisciplinary Journal of Study Abroad, 18, 101-118. Brecht, R., Davidson, D., & Ginsberg, R. (1993). Predictors of foreign language gain during study abroad. Washington DC: National Foreign Language Center. Carlson, J., Burn, B., Useem, J., & Yachimovicz, D. (1990). Study abroad: The experience of American undergraduates. Westport, CT: Greenwood Press. Carlson, J. S., & Widaman, K. F. (1988). The effect of study abroad during college on attitudes toward other cultures. International Journal of Intercultural Relations, 12, 1-17. Cass, V. (1979). Homosexual identity formation: A theoretical model. Journal of Homosexuality, 9, 219-235. Chieffo, L., & Griffiths, L. (2004). Large-scale assessment of student attitudes after a short-term study abroad program. Frontiers: The Interdisciplinary Journal of Study Abroad, 10, 165-177. Chang, M. J. (2002). The impact of an undergraduate diversity course requirement on students racial views and attitudes. Journal of General Education, 51, 21-42.

104 Chickering, A. W. & Reisser, L. (1969). Education and identity (Second Edition). San Francisco: Jossey-Bass. Chmela, H. (2005). Foreign detour en route to a college degree. New York Times, (October 19). Retrieved from http://travel.nytimes.com/2005/10/19/education/19goucher.html?_r=1&scp=1&sq =%22lincoln+commission%22+and+%22study+abroad%22&st=nyt Church, A. T. (1982). Sojourner adjustment. Psychological Bulletin, 91, 540-572. Consolidated Appropriations Act, 2004, Pub. L. No. 108-199, 118 Stat. 3 (2004). Cook, S. W., & Selltiz, C. (1955). Some factors which influence the attitudinal outcomes of personal contact. International Social Science Bulletin, 7, 51-58. Cornwell, G., & Stoddard, E. (1999). Globalizing knowledge: Connecting international and intercultural studies. Washington DC: Association of American Colleges and Universities. Council on International Educational Exchange [CIEE]. (1991). Black students and overseas programs: Broadening the base of participation. Papers and speeches presented at the 43rd CIEE International Conference on Educational Exchange/ New York: Council on International Educational Exchange. Cress, C., Astin, H., Zimmerman-Oster, K., & Burkhardt, J. (2001). Developmental outcomes of college students involvement in leadership activities. Journal of College Student Development, 42, 15-27. Cushner, K., & Karim, A. U. (2004). Study abroad at the university level. In D. Landis, J. M. Bennett, & M. J. Bennett (Eds.), Handbook of intercultural training, 3rd Edition (pp. 289-308). Thousand Oaks, CA: Sage Publications. Cushner, K., & Mahon, J. (2002). Overseas student teaching: Affecting personal, professional, and global competencies in an age of globalization. Journal of Studies in International Education, 6, 44-58. DAndrea, M., Daniels, J., & Heck, R. (1991). Evaluating the impact of multicultural counseling training. Journal of Counseling & Development, 70, 143-150. DAugelli, A. R. (1994). Identity development and sexual orientation: Toward a model of lesbian, gay, and bisexual development. In E. Trickett, R. Watts, & D. Birman (Eds.), Human diversity: Perspectives on people in context (pp. 312-333). San Francisco: Jossey-Bass.

105 Deardorff, D. K. (2004). The identification and assessment of intercultural competence as a student outcome of internationalization at institutions of higher education in the United States. (Ph.D., North Carolina State University). Deardorff, D. K. (2006). Identification and assessment of intercultural competence as a student outcome of internationalization. Journal of Studies in International Education, 10(3), 241-266. Deardorff, D. K. (2009). Understanding the challenges of assessing global citizenship. In R. Lewin (Ed.), The handbook of practice and research in study abroad: Higher educations quest for global citizenship (pp. 346-364). New York: Routledge. Deficit Reduction Act of 2005, Pub. L. No. 109-171, 120 Stat. 4 (2005). Dehejia, R. H., & Wahba, S. (1999). Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs. Journal of the American Statistical Association, 94(448), 1053-1062. Designating 2006 as the Year of Study Abroad. S. Res. 208, 109th Cong., 1st. Sess. (2005). (enacted). Dessoff, A. (2006). Whos not going abroad? International Educator, 15(2), 20-27. Dolby, N. (2004). Encountering an American self: Study abroad and national identity. Comparative Education Review, 48(2), 150-173. Douglas, C., & Jones-Rikkers, C. G. (2001). Study abroad programs and American student worldmindedness: An empirical analysis. Journal of Teaching in International Business, 13, 55-66. Du, F. (2007). Self-authorship as a learning outcome of study abroad: Towards a new approach for examining learning and learning conditions. (Ph.D., University of Minnesota). DuFon, M. A., & Churchill, E. (2006). Evolving threads in study abroad research. In M. A. DuFon & E. Churchill (Eds.), Language learners in study abroad contexts (pp. 1-29). Clevedon, UK: Multilingual Matters, Ltd. Dunn, T. W., Smith, T. B., & Montoya, J. A. (2006). Multicultural competency instrumentation: A review and analysis of reliability generalization. Journal of Counseling and Development, 84, 471-482. Engberg, M. E. (2007). Educating the workforce for the 21st century: A cross-disciplinary analysis of the impact of the undergraduate experience on the students development of a pluralistic orientation. Research in Higher Education, 48, 283317.

106 Epstein, M. P., Allen, A. S., & Satten, G. A. (2007). A simple and improved correction for population stratification in case-control studies. American Journal of Human Genetics, 8, 921-930. Erikson, E. H. (1950). Childhood and society. New York: Norton. Erikson, E. H. (1959). Identity and the life cycle. New York: International Universities Press. Erikson, E. H. (1968). Identity, youth, and crisis. New York: Norton. Fantini, A. E. (2000). A central concern: Developing intercultural competence. SIT Occasional Papers Series, I. 25-42. Retrieved from http://www.sit.edu/SITOccasionalPapers/sitops01.pdf Fantini. A. E. (2006). Exploring and assessing intercultural competence. Final Report of a Research Project conducted by the Federation of The Experiment in International Living. Retrieved from http://www.sit.edu/SITOccasionalPapers/feil_research_report.pdf Farrell, E. F. (2007). Study abroad blossoms into big business. Chronicle of Higher Education, (September 7). Retrieved from http://chronicle.com/article/StudyAbroad-Blossoms-Into-/19935/ Field, R. (Producer & Director). (2009). Engaging the world: U.S. global competence in the 21st century. (DVD). (Available from The American Council on Education, One Dupont Circle NW, Washington DC, 20036). Retrieved from http://www.usglobalcompetence.org/videos/download.html Fischer, K. (2007, June 20). All abroad! Overseas study required. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/All-AbroadOverseas-Study-/13923/ Flash, S. J. (1999). Study abroad program participation effects on academic progress. (Ph.D., State University of New York at Buffalo). Foreign Relations Authorization Act, Fiscal Years 2010 and 2011, H. R. 2410, 111th Cong. (2009). Freed, B. F. (1995). Second language acquisition in a study abroad context. Philadelphia, PA: Benjamins Publishing Company. Freedman, M. B. (1960). Impact of college. New Dimensions in Higher Education, No. 4. U.S Department of Health, Education, and Welfare. Washington DC: U.S. Government Printing Office.

107 Friedman, T. L. (2005). The world is flat: A brief history of the twenty-first century. New York: Farra, Strauss, & Giroux. Fuertes, J. N., & Brobst, K. (2002). Clients ratings of counselor multicultural competency. Cultural Diversity and Ethnic Minority Psychology, 8, 214-223. Fuertes, J. N., & Gelso, C. J. (2000). Hispanic counselors race and accent and EuroAmericans universal-diverse orientation: A study of initial perceptions. Cultural Diversity and Ethnic Minority Psychology, 6, 211-219. Fuertes, J. N., Miville, M. L., Mohr, J. J., Sedlacek, W. E., & Gretchen, D. (2000). Factor structure and short form of the Miville-Guzman Universality-Diversity Scale. Measurement and Evaluation in Counseling and Development, 33, 157-169. Fuertes, J. N., Sedlacek, W. E., Roger, P. R., & Mohr, J. J. (2000). Correlates of universal-diverse orientation among first-year university students. Journal of First-Year Experience and Students in Transition, 12, 45-59. Fulbright, J. W. (1989). The price of empire. New York: Pantheon. Gall, M. D., Gall, J. P., & Borg, W. R. (2003). Educational research: An introduction. 7th Edition. Boston: Allyn and Bacon. Gardner, E. G. (1968). Selection of American undergraduate students for participation in study programs overseas. . Occasional Paper on International Educational Exchange No. 13. Council on International Educational Exchange. Retrieved from http://www.ciee.org/images/uploaded/pdf/occasional13.pdf Gilligan, C. (1982). In a different voice: Psychological theory and womens development. Cambridge, MA: Harvard University Press. Gmelch, G. (1997). Crossing cultures: Student travel and personal development. International Journal of Intercultural Relations, 21, 475-490. Golay, P. (2006). The effects of study abroad on the development of global-mindedness among students enrolled in international programs at Florida State University. (Ed.D., The Florida State University). Goldstein, S. B., & Kim, R. I. (2006). Predictors of US college students participation in study abroad programs: A longitudinal study. International Journal of Intercultural Relations, 30, 507-521. Gonyea, R. M. (2008, November). The impact of study abroad on senior year engagement. Paper presented at the Annual Meeting of the Association for the Study of Higher Education, Jacksonville, FL.

108 Grant, G., & Associates. (1979). On competence: A critical analysis of competence-based reforms in higher education. San Francisco: Jossey-Bass. Gray, M., Ondaatjie, E., Fricker, R., & Geschwind, S. (2000). Assessing service learning: Results from a survey of Learn and Serve American, Higher Education. Change, 32, 30-39. Green, M. (2005). Internationalization in U.S. higher education: The student perspective. Washington DC: American Council on Education. Green, M., Luu, D. T., & Burris, B. (2008). Mapping internationalization on U.S. campuses: 2008 edition. Washington DC: American Council on Education. Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2004). Survey methodology. Hoboken, NJ: Wiley-Interscience. Gullahorn, J. E., & Gullahorn, J. T. (1966). American students abroad: Professional versus personal development. Annals of the American Academy of Political and Social Science, 368, 43-59. Gurin, P., Peng, T., Lopez, G., & Nagda, B. (1999). Context, identity, and intergroup relations. In D. Prentice & D Miller (Eds.), Cultural divides: Understanding and overcoming group conflict (pp. 133-170). New York: Russell Sage Foundation. Hammer, M. R., Bennett, M. J., & Wiseman, R. (2003). Measuring intercultural sensitivity: The intercultural development inventory. International Journal of Intercultural Relations, 27, 421-443. Helms, J. (1990). Toward a model of White racial identity development. In J. Helms (Ed.), Black and White racial identity Theory, research, and practice (pp. 49-66). Westport, CT: Greenwood Press. Hewstone, M. & Brown, R. (1986). Contact is not enough: An intergroup perspective on the contact hypothesis. In M. Hewstone & R. Brown (Eds.), Contact and conflict in intergroup encounters (pp.1-44). Cambridge, MA: Addison-Wesley. Hill, B., & Green, M. (2008). A guide to internationalization for chief academic officers. Washington DC: American Council on Education. Hoffa, W. W. (2007). A history of U.S. study abroad: Beginnings to 1965. Carlisle, PA: The Forum on Education Abroad. Hoffa, W. W., & DePaul, S. C. (2010). A history of U.S. study abroad: 1965-present. Carlisle, PA: The Forum on Education Abroad.

109 Hofman, J. E., & Zak, I. (1969). Interpersonal contact and attitude change in a crosscultural situation. The Journal of Social Psychology, 78, 165-171. Hurtado, S., Engberg, M. E., & Ponjuan, L. (2003). The impact of diversity experience on students learning for a diverse democracy. Paper presented at Annual Meeting of the Association of the Study of Higher Education, Portland, OR. Ingraham, E. C. (2003). Documentation and assessment of the impact of study abroad. In G. T. M. Hult & E. C. Lashbrooke, Jr. (Eds.), Study abroad: Perspectives and experiences from business schools (pp. 1-22). Oxford, UK: Elsevier Science Ltd. Ingraham, E. C., & Peterson, D. L. (2004). Assessing the impact of study abroad on student learning at Michigan State University. Frontiers: The Interdisciplinary Journal of Study Abroad, 10, 83-100. Institute of International Education [IIE]. (2009). Open doors report 2009. Retrieved from http://opendoors.iienetwork.org/ Jacob, P. E. (1957). Changing values in college: An exploratory study of the impact of college teaching. New York: Harper. Jeffrey, H. P. (1987). Legislative origins of the Fulbright program. Annals of the American Academy of Political and Social Science, 491(May), 36-47. Johnston, Jr., J. S., & Edelstein, R. J. (1993). Beyond borders: Profiles in international education. Washington DC: Association of American Colleges and Universities. Josselson, R. (1987). Finding herself: Pathways to identity development in women. San Francisco: Jossey-Bass. Jurgens, J. C., McAuliffe, G. (2004). Short-term study-abroad experience in Ireland: An exercise in cross-cultural counseling. International Journal for the Advancement of Counseling, 26, 147-161. Kalunian, J. (1997, Fall). Correlations between global-mindedness and study abroad. International Education Forum, 17, 131-143. Kasravi, J. (2009). Factors influencing the decision to study abroad for students of color: Moving beyond the barriers. (Ph.D., University of Minnesota). Kauffmann, N. L. (1983). The impact of study abroad on personality change. (Ph.D., Indiana University). Kegan, R. (1982). The evolving self: Problems and process in human development. Cambridge, MA: Harvard University Press.

110 Kegan, R. (1994). In over our heads: The mental demands of modern life. Cambridge, MA: Harvard University Press. Kelley, C., & Meyers, J. (1995). Cross-Cultural Adaptability Inventory. Minneapolis, MN: National Computer Systems. King, P. M., & Baxter Magolda, M. (1996). A developmental perspective on learning. Journal of College Student Development, 37, 163-173. King, P.M., & Baxter Magolda, M. (2005). A developmental model of intercultural maturity. Journal of College Student Development, 46, 571-592. King, P. M., & Kitchener, K. (1994). Developing reflective judgment: Understanding and promoting intellectual growth and critical thinking in adolescents and adults. San Francisco: Jossey-Bass. King, L. J., & Young, J. A. (1994). Study abroad: Education for the 21st century. Die Unterrichtspaxis / Teaching German, 27, 77-87. Kitsantas, A., & Meyers, J. (2004). The role of college students goals on the development of cross-cultural skills and global understanding. College Student Journal, 38, 441-452. Knight, J. (2004). Internationalization remodeled: Definitions, approaches, and rationales. Journal of Studies in International Education, 8, 5-31. Koeser, J., & Olebe, M. (1988). The behavioral assessment scale for intercultural communication effectiveness. International Journal of Intercultural Relations, 12, 233-246. Koester, J. (1985). A profile of the U.S. student abroad. New York: Council on International Educational Exchange. Kohlberg, L. (1971). Stages of moral development as a basis of moral education. In C. Beck, B. Crittenden, E. Sullivan (Eds.), Moral education: Interdisciplinary approaches. Toronto: University of Toronto Press. Kuh, G. D. (2008). High impact educational practices: What they are, who has access to them, and why they matter. Washington DC: Association of American Colleges and Universities. LaFromboise, T. D., Coleman, H. L., & Hernandez, A. (1991). Development and factor structure of the Cross-Cultural Counseling Inventory-Revised. Professional Psychology: Research and Practice, 22, 380-388.

111 LaLonde, R. (1986). Evaluating the econometric evaluations of training programs with experimental data. American Economic Review, 76, 604-620. Lambert, R. D. (1989). International studies and the undergraduate. Washington, DC: American Council on Education. Lau A., & Finney, S. (2006). Measuring diversity orientation: An examination of the Miville-Guzman universality-diversity scale short form. Paper presented at the annual meeting of the Association of Psychological Sciences, New York. Leuven, E., & Sianesi, B. (2003). PSMATCH2: Stata module to perform full Mahalanobis and propensity score matching, common support graphing and covariate imbalance testing. Retrieved from http://ideas.repec.org/c/boc/bocode/s432001.html. Version 1.2.3. Lewin, R. (ed.) (2009). The handbook of practice and research in study abroad: Higher education and the quest for global citizenship. New York: Routledge. Lincoln Commission. (2005). Global competence and national needs: One million Americans studying abroad. Final Report from the Commission on the Abraham Lincoln Fellowship Program, Washington, DC. Lindsey, E. W. (2005). Study abroad and values development in social work students. Journal of Social Work Education, 41, 229-249. Lunceford, J. K., & Davidian, M. (2004). Stratification and weighting via the propensity score in estimation of causal treatment effects: A comparative study. Statistics in Medicine, 23, 2937-2960. Magnan, S. S., & Back, M. (2007). Social interaction and linguistic gain during study abroad. Foreign Language Annals, 40, 43-61. McClendon, M. J. (1994). Multiple regression and causal analysis. Long Grove, IL: Waveland Press, Inc. McEvoy, T. L. (1967). Psychological growth, cosmopolitanism, and selection of students for cross-cultural educational programs. Occasional Paper on International Educational Exchange No. 7. Council on International Educational Exchange. Retrieved from http://www.ciee.org/images/uploaded/pdf/occasional07.pdf McKeown, J. S. (2003). The impact of September 11 on study abroad student interest and concern: An exploratory study. International Education, 32(2), 85-95. McKeown, J. S. (2009). The first time effect: The impact of study abroad on college student intellectual development. Albany, NY: State University of New York Press.

112

Metzger, C. (2006). Study abroad programming: A 21st century retention strategy? College Student Affairs Journal, 25, 164-175. Milem, J. (1994). College, students, and racial understanding. Thought and Action, 9, 5194. Milstein, T. (2005). Transformation abroad: Sojourning and the perceived enhancement of self-efficacy. International Journal of Intercultural Relations, 29, 217-239. Mistretta, W. (2008). Life-enhancing: An exploration of the long-term effects of study abroad. (Ph.D., State University of New York at Buffalo). Miville, M. L., Gelso, C. J., Pannu, R., Liu, W., Touradji, P., Hollowa, P., & Fuertes, J. N. (1999). Appreciating similarities and valuing differences: The Miville-Guzman Universality Diversity Scale. Journal of Counseling Psychology, 46, 291-307. Myers-Lipton, S. (1996). Effect of service-learning on college students attitudes toward international understanding. Journal of College Student Development, 37, 659668. NAFSA: Association of International Educators. (2003). Securing Americas future: Global education for a global age. Report of the Strategic Task Force on Education Abroad. Retrieved from http://www.nafsa.org/public_policy.sec/public_policy_document/study_abroad_1/ securing_america_s_future/ NAFSA: Association of International Educators. (2008). Strengthening study abroad: Recommendations for effective institutional management for presidents, senior administrators, and study abroad professionals. Report of NAFSAs Task Force of Institutional Management of Study Abroad. Retrieved from http://www.nafsa.org/_/File/_/final_imsa_taskforce.pdf Nash, D. (1976). The personal consequences of a year of study abroad. The Journal of Higher Education, 47, 191-203. National Center for Education Statistics. (2008a). Table A-40-1. Number and percentage distribution of U.S. study abroad students, by host region: Selected academic years, 1987-88 through 2007-08. Retrieved from http://nces.ed.gov/programs/coe/2010/section5/table-ssa-1.asp National Center for Education Statistics. (2008b). Table 189. Total fall enrollment in degree-granting institutions, by attendance status, sex of student, and control of institution: Selected years, 1947 through 2008. Retrieved from http://nces.ed.gov/programs/digest/d09/tables/dt09_189.asp

113 National Commission on Terrorist Attacks upon the United States. (2003). The 9/11 commission report. Washington DC: U.S. Government Printing Office. Retrieved from http://govinfo.library.unt.edu/911/report/911Report.pdf National Task Force on Undergraduate Education Abroad. (1990). A national mandate for education abroad: Getting on with the task. New York: Council on International Educational Exchange. Norris, M. E., & Gillespie, J. (2009). How study abroad shapes global careers: Evidences from the United States. Journal of Studies in International Education, 13, 382397. Olson, C., Green, M., & Hill, B. (2005). Building a strategic framework for comprehensive internationalization. Washington DC: American Council on Education. Olson, C., Green, M., & Hill, B. (2006). A handbook for advancing comprehensive internationalization: What institutions can do and what students should learn. Washington DC: American Council on Education. Olson, C. L., & Kroeger, K. R. (2001). Global competency and intercultural sensitivity. Journal of Studies in International Education, 5(2), 116-137. Padgett, R. D., Salisbury, M. H., An, B. P., & Pascarella, E. T. (in press). Required, practical, or unnecessary? An examination and demonstration of propensity score matching using longitudinal secondary data. Assessment Issue of New Directions in Institutional Research. San Francisco: Jossey-Bass. Pascarella, E. T. (1985). College environmental influences on learning and cognitive development: A critical review and synthesis. In J. Smart (Ed.). Higher education: Handbook of theory and research (Vol.1, pp. 1-64). New York: Agathon. Pascarella, E. T. (2006). How college affects students: Ten directions for future research. Journal of College Student Development, 47(5), 508-520. Pascarella, E. T., Edison, M., Nora, A., Hagedorn, L., & Terenzini, P. T. (1996). Influences on students openness to diversity and challenge in the first year of college. Journal of Higher Education, 67, 174-195. Pascarella, E. T., & Terenzini, P. T. (1991). How college affects students. San Francisco: Jossey-Bass. Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students (Vol. 2): A third decade of research. San Francisco: Jossey-Bass.

114 Patterson, P. K. (2006). Effect of study abroad on intercultural sensitivity. (Ph.D., University of Missouri-Columbia). Pedersen, P. J. (2009). Teaching towards an ethnorelative worldview through psychology study abroad. Intercultural Education, 20, 73-86. Peikes, D. N., Moreno, L., & Orzol, S. M. (2008). Propensity score matching: A note of caution for evaluators of social programs. The American Statistician, 62, 222-231. Perry, W. (1970). Forms of intellectual and ethical development in the college years: A scheme. San Francisco: Jossey-Bass. Pettigrew, T. F., & Tropp, L. R. (2006). A meta-analytic test of intergroup contact theory. Journal of Personality and Social Psychology, 90, 751-783. Phinney, J. (1990). Ethnic identity in adolescents and adults: Review of research. Psychological Bulletin, 108, 499-514. Piaget, J. (1928). Judgment and reasoning in the child. New York: Harcourt, Brace, and Company. Piaget, J. (1952). The origins of intelligence in children. New York: International Universities Press. Pike, G. (2002). The differential effect of on- and off- campus living arrangements on students openness to diversity. NASPA Journal, 39, 283-299. Ponterotto, J. G., & Alexander, C. M. (1996). Assessing the multicultural competence of counselors and clinicians. In L. A., Suzuki, P. J. Meller, & J. G. Ponterotto (Eds.), Handbook of multicultural assessment (pp. 651-672). Thousand Oaks, CA: Sage. Ponterotto, J. G., Gretchen, D., Utsey, S. O., Rieger, B. P., & Austin, R. (2002). A revision of the Multicultural Counseling Awareness Scale. Journal of Multicultural Counseling and Development, 30, 153-180. Pope, R. L., & Mueller, J. A. (2000). Development and initial validation of Multicultural Competence in Student Affairs-Preliminary 2 Scale. Journal of College Student Development, 41, 599-608. Posey, Jr., J. T. (2003). Study abroad: Educational and employment outcomes of participants and non participants. (Ph.D., Florida State University). Presidents Commission on Foreign Languages and International Study. (1980). Strength through wisdom: A critique of U.S. capability. Modern Language Journal, 64(1), 9-57.

115 Redden, E. (2008). Women abroad and men at home. Inside Higher Ed. Retrieved from www.insidehighered.com/news/2008/12/4/genderabroad Reynolds, C. L., & DesJardins, S. L. (2009). The use of matching methods in higher education research: Answering whether attendance at a 2-year institution results in differences in educational attainment. In J. C. Smart (Ed.), Higher education: Handbook of theory and research, (pp. 47-104). New York: Springer Science + Business Media. Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55. Ruben, B. D., & Kealey, D. (1979). Behavioral assessment of communication competency and the prediction of cross-cultural adaptation. International Journal of Intercultural Relations, 3, 15-48. Rubin, D. B. (2008). For objective causal inference, design trumps analysis. Annals of Applied Statistics, 2, 808-840. Rubin, D. B. (2001). Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services and Outcomes Research Methodology, 2, 169-188. Rubin, D. L., & Sutton, R. C. (2001). Study abroad learning outcomes assessment. Paper presented at the American Association for Higher Education Assessment Forum, Denver, Colorado, June, 25. Rust, V., Dhanatya, C., Furuto, L. H. L., & Kheiltash, O. (2007). Student involvement as predictive of college freshmen plans to study abroad. Frontiers: The Interdisciplinary Journal of Study Abroad, 15(fall), 1-16. Ryan, M., & Twibell, R. (2000). Concerns, values, stress, coping, health, and educational outcomes of college students who studied abroad. International Journal of Intercultural Relations, 24, 409-435. Salisbury, M. H., & Goodman, K. (2009). Educational practices that foster intercultural competence. Diversity and Democracy, 12(2), 12-13. Salisbury, M. H., Paulsen, M. B., & Pascarella, E. T. (2010). To see the world or stay at home: Applying an integrated student choice model to explore the gender gap in the intent to study abroad. Research in Higher Education, 51. Retrieved from http://www.springerlink.com/content/713786l56010450n/

116

Salisbury, M. H., Paulsen, M. B., & Pascarella, E. T. (in press). Why do all the study abroad students look alike? Using an integrated student choice model to explore differences in the development of white and minority students intent to study abroad. Research in Higher Education. Salisbury, M. H., Umbach, P. D., Paulsen, M. B., & Pascarella, E. T. (2009). Going global: Understanding the choice process of the intent to study abroad. Research in Higher Education, 50, 119-143. Savicki, V. (ed.) (2008). Developing intercultural competence and transformation: Theory, research, and application in international education. Sterling, VA: Stylus. Sax, L. & Astin, A. (1997). The benefits of service: Evidence from undergraduates. Educational Record, 78, 25-32. Schneider, B., Carnoy, M., Kilpatrick, J., Schmidt, W. H., & Shavelson, R. J. (2007). Estimating causal effects using experimental and observational designs. Washington, DC: American Educational Research Association. Sell, D. K. (1983). Research on attitude change in U.S. students who participate in foreign study experiences: Past findings and suggestions for future research. International Journal of Intercultural Relations, 7, 131-147. Senator Paul Simon Study Abroad Foundation Act of 2007, H. R. 1469/S. 991, 110th Cong. (2007). Senator Paul Simon Study Abroad Foundation Act of 2009, S. 473, 110th Cong. (2009). Shadish, W. R., Clark, M. H., & Steiner, P. M. (2008). Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random and nonrandom assignments. Journal of the American Statistical Association, 103, 1334-1356. Shaheen, S. (2004). The effect of pre-departure preparation on student intercultural development during study abroad programs. (PhD., The Ohio State University). Siaya, L., & Hayward, F. M. (2003). Mapping internationalization on U.S. campuses. Washington DC: American Council on Education. Sinicrope, C., Norris, J., & Watanabe, Y. (2003). Understanding and assessing intercultural competence: A summary of theory, research, and practice (Technical report for the Foreign Language Program Evaluation Project). Second Language Studies, 26, 1-58. Retrieved from http://www.hawaii.edu/sls/uhwpesl/26(1)/Norris.pdf

117 Sodowsky, G. R., Taffe, R. C., Gutkin, T. B., & Wise, S. L. (1994). Development of the Multicultural Counseling Inventory: A self-report measure of multicultural competencies. Journal of Counseling Psychology, 41, 137-148. Spady, W. G. (1977). Competency based education: A bandwagon in search of a definition. Educational Researcher, 6, 9-14. Spiering, K., & Erickson, S. (2006). Study abroad as innovation: Applying the diffusion model to international education. International Education Journal, 7, 314-322. Spitzberg, B. H., & Changnon, G. (2009). Conceptualizing intercultural competence. In D. K. Deardorff (Ed.), The SAGE handbook of intercultural competence (p. 2-52). Thousand Oaks, CA: SAGE Publications, Inc. Steiner, P. M., Cook, T. D., Shadish, W. R., & Clark, M. H. (in press). The importance of covariate selection in controlling for selection bias in observational studies. Psychological Methods. Strang, R. (1937). Behavior and background of student in college and secondary school. New York: Harper. Stroud, A. H. (2010). Who plans (not) to study abroad? An examination of U.S. student intent. Journal of Studies in International Education, 20(10), 1-18. Sutton, R. C., & Rubin, D. L. (2004). The GLOSSARI Project: Initial findings from a system-wide research initiative on study abroad learning outcomes. Frontiers: The Interdisciplinary Journal of Study Abroad, 10, 65-82 Sutton, R. C., & Rubin, D. L. (2010). Documenting the academic impact of study abroad. Final report of the GLOSSARI project. Paper presented at the NAFSA Annual Conference. Kansas City, MO. Talburt, S., & Stewart, M. A. (1999). Whats the subject of study abroad?: Race, gender, and living culture. The Modern Language Journal, 83, 163-175. Thomas, S. L., & McMahon, M. E. (1998). Americans abroad: Student characteristics, pre-departure qualifications and performance. International Journal of Educational Management, 12(2), 57-64. Titus, M. A. (2007). Detecting selection bias, using propensity score matching, and estimating treatment effects: An application to the private returns to a masters degree. Research in Higher Education, 48, 487-521. Vande Berg, M. (2003). The case for assessing educational outcomes in study abroad. In G. T. M. Hult & E. C. Lashbrooke, Jr. (Eds.), Study abroad: Perspectives and experiences from business schools (pp. 23-36). Oxford, UK: Elsevier Science Ltd.

118 Vande Berg, M. (2007). Intervening in the learning of U.S. students abroad. Journal of Studies in International Education, 11(3/4), 392-399. Vande Berg, M., Connor-Linton, J., & Paige, M. R. (2009). The Georgetown Consortium Project: Interventions for student learning abroad. Frontiers: The Interdisciplinary Journal of Study Abroad, 18, 1-75. Wallace, D. H. (1999). Academic study abroad: The long-term impact on alumni careers, volunteer activities, world, and personal perspectives. (Ph.D., Claremont Graduate University). Wallace, J. A. (1965). Selection of participants. Occasional Paper on International Educational Exchange No. 3. Council on International Educational Exchange. Retrieved from http://www.ciee.org/images/uploaded/pdf/occasional03.pdf Whitt, E., Edison, M., Pascarella, E. T., Terenzini, P. T., & Nora, A. (2001). Influences on students openness to diversity and challenge in the second and third years of college. Journal of Higher Education, 72, 172-204. Wilkinson, S. (1998). Study abroad from the participants perspective: A challenge to common beliefs. Foreign Language Annals, 31, 23-49. Wilkinson, S. (2000). Emerging questions about study abroad. ADFL Bulletin, 32, 36-41. Williams, T. R. (2005). Exploring the impact of study abroad on students intercultural communications skills: Adaptability and sensitivity. Journal of Studies in International Education, 9, 356-371. Woolf, M. (2007). Impossible things before breakfast, Myths in education abroad. Journal of Studies in International Education, 11(3/4), 496-509. Young, D. Y. (2003). Participation in a study-abroad program and persistence at a liberal arts university. (Ph.D., University of North Texas). Zhai, L., & Scheer, S. D. (2002). Influence of international study abroad programs on agricultural students. Journal of International Agricultural and Extension Education, 9(3), 23-29.