Running head: QUINLAN RAMBLE EVALUATION PLAN 1

Quinlan Ramble Evaluation Plan

Fernando Perez and Jona Salamanca

Loyola University Chicago

QUINLAN RAMBLE EVALUATION PLAN 2

Table of Content

PROGRAM OVERVIEW…......………………………………………………………………...3
PROGRAM FOCUS……………………………………………………………………………..4
PROGRAM OUTCOMES………………………………………………………………………6
LOGIC MODEL DESCRIPTION………………………………...............................................9
ASSESSMENT PURPOSE…………………………………………………………………….11
ASSESSMENT QUESTIONS………………………………………………………………….13
QUANTITATIVE ASSESSMENT APPROACH………………………………….…………13
SURVEY INSTUMENT………………………………………………………….……………15
DATA ANALYSIS………………………………………………………………….…………16
QUALITATIVE ASSESSMENT APPROACH………………………………………………18
PROTOCOL INSTRUMENT…………………………………………………………………..19
DATA ANALYSIS……………………………………………………………………………..20
LIMITATIONS OF ASSESSMENT APPROACH……...…………………………………...22
TIMELINE……………………………………………………………………………………...23
BUDGET........…………………………………………………………………………………...23
NEXT STEPS…...………………………………………………………………………………24
REFERENCES…...…………………………………………………………………………….25
APPENDICES…...………………………………………………………………………….26-50
A. LOGIC MODEL……………………………………………………………………………26
B. QUANTATATIVE SURVEY INSTRUMENT..……………………………………….27-36
C. QUANTATATIVE SUREY MATRIX…………………………………………………….37
D. EMAIL TEMPLATES...……………………………………………………………………38
E. QUALATATIVE INSTRUMENT………………………………………………………39-41
F. DEMOGRAPHIC VARIABLES FORM…………………………………………………...41
G. INFORMED CONSENT ……………………………………………………………….41-42
H. POWERPOINT PRESENTATION……………………………………………………..42-50
QUINLAN RAMBLE EVALUATION PLAN 3

Quinlan Ramble Evaluation Plan

Program Overview

The Quinlan Ramble is a seven day long immersive program directed towards

undergraduate students at Loyola University Chicago, specifically those that are part of the

Quinlan School of Business. This program takes place in the spring semester, with recruitment

for it happening in the last couple of months of the fall semester. The Quinlan Ramble takes a

group of undergraduate students to a city within the United States, visiting multiple businesses

headquartered in that specific city. This upcoming year, the program will be taking students to

Seattle for the second time.

The Quinlan Ramble program is fairly new, with the first one happening in the spring of

2014. It started as a project that was put on by only one person, when and individual assembled a

group of students and took them to his hometown of Seattle. The program initially relied heavily

on the creator’s connections in the city to get access for the group to visit different sites. What

started as a one-person program has since developed to also include a graduate assistant and a

staff member of Business Career Services from the Quinlan School of Business. The program

also collaborates with the Office of Advancement on this project, although no one from that

office actually staffs the program itself. Each trip permits for sixteen students to participate.

When speaking with the Quinlan School of Business’ Student Engagement Coordinator,

Allison Davis, we discovered there is not a written mission, values, or goals for the program, so

it was a challenging finding a way to guide us through the development of a plan. After speaking

with Allison, we did gather vital information about her view of what the program should be, that

we used to develop key components that were missing from the program. The mission of this
QUINLAN RAMBLE EVALUATION PLAN 4

program is to expand knowledge of the global economy through exposure to real-life examples

of values-based leadership and business operating through an ethical and social justice based

approach. Additionally the program seeks to build a professional network for attendees. This

mission is situated perfectly among the mission of the Quinlan School of Business, which

promotes a “commitment to contribute to society through ethical and socially responsible

conduct, sound decision-making and problem-solving skills, and the knowledge to act effectively

in complex organizational settings and in a diverse global economy,” (Loyola University Chicago,

2017). The mission of the Quinlan Ramble connects with the institutional mission of Loyola

University Chicago, specifically by addressing the commitment to a global awareness and

promoting justice and values-based leadership. The Quinlan Ramble program specifically

address its commitment to awareness of a global economy through exposure to real-life

examples of values-based leadership and business operating through an ethical and social justice

based approach by being intentional in the cities that it chooses to take students. The cities are

selected on a basis of how much the city is connected globally and how many businesses we can

take students to that promote social justice and values based leadership.

Program Focus

The Quinlan Ramble is starting to shift its efforts to better align with the institutional

strategic plan, Plan 2020, which states that one priority is to develop programming for student

success (Loyola University Chicago, 2017). With the development of this program, the main

purpose is to help situate the students in a place in which they will have a better skill set for

networking and more intentionality in selecting the companies in which they will work.

The Quinlan Ramble focuses on giving the students the resources to develop a career

after graduation. By giving the students the opportunity to have a first-hand experience with
QUINLAN RAMBLE EVALUATION PLAN 5

multiple companies, it is the goal for our students to use that knowledge to build a career in

which they are able to use values-based leadership and use a social justice approach to their

practice. The Quinlan Ramble is intentional with picking companies that show a diverse range of

individuals and backgrounds. The professionals for each company not only share information

about the company, but since most of them are Loyola alumni, they also share their story. This is

done in order to help our students start to map their professional life. All the resources and

programs that they engage in really push the students to start thinking on how to be able to direct

their own life in a way that promotes the mission of the Quinlan Ramble and Loyola.

The Quinlan Ramble program encourages students to reach self-authorship. Self-

authorship is the achievement of student’s internal capacity to define one's beliefs, identity &

social relations (Magolda, 2008). The ability to do this involves student’s cognitive, identity, and

interpersonal development. The Quinlan Ramble facilitates this type of development through the

invitation to apply classroom knowledge to their experiences in the program. By showing them

examples of great leadership in great companies, we are allowing them the chance to question

and define “great leadership” on their own. The Quinlan Ramble will have built-in time to reflect

and discuss. This is done intentionally to challenge yet support student as they may potentially

experience dissonance in what they thought of as just practice versus what is reality. The site

visits and different conversations that happen throughout the program will hopefully lead to the

participant’s exploration in what all this means to them and how they can transfer this to their

personal and professional life. As Baxter Magolda (2008) explains, “awareness often prompted

exploration...these were times of confusion, ambiguity, fear, and even despair as individuals

struggled to analyze and reconstruct some aspect of their beliefs, identity, or relationships in
QUINLAN RAMBLE EVALUATION PLAN 6

various context” (p. 280). These questions will help the students shape their career path and truly

be in control of how they engage in the global economy.

Given the emphases on career exploration and building networks during this program, it

is also important to look at what different theories say about programs that help students explore

career options. Vocational Choice Theory by John Holland (1997) asserts that career interests are

an expression of an individual's personality. Programs like Quinlan Ramble allow students to

explore occupational interests through potential job shadowing and informational interviews

before committing to any one job. Holland (1997) states that people can be described as a

combination of two or more of six interest types: realistic, investigative, artistic, social,

enterprising, and/or conventional. Likewise, work environments can be described in the same

way. According to Holland, a person is more satisfied if their personality matches their work

environment. Holland asserts that within any one given career field there are realistic,

investigative, artistic, social, enterprising, and/or conventional type-jobs; including the business

field. Quinlan Ramble allows students to explore their interests through exposure and

engagement. For some participants this may be the first time they are doing job site visits. This

initial contact could potentially spur favorable reactions to the type of work that affirm their

career path. Conversely, it may persuade them to seek other opportunities. Because the Quinlan

Ramble will ask student to reflect and discuss their own values and belief systems, the program’s

exposure could lead student to some realization about the consistence of their values and interest

with the career or companies philosophies.

Program Outcomes

The Quinlan Ramble was initially created with the purpose of connecting our current

students to a global economy by giving them the opportunity to visit a globally connected city
QUINLAN RAMBLE EVALUATION PLAN 7

within the United States. The Quinlan Ramble sets the stage for the students to learn from real

life experiences, most often told by Loyola alumni. After considering this, previously mention

items, and our communicating with Allison we were able to identify four program area

outcomes: understanding global economy, understanding values-based leadership, explore career

interests, build professional networks.

The first program outcome that we developed was that as a result considering global

economies. We determined that it was important for student to be able to explain power

relationships in the economy, identify visited company’s current practices in the market place

and their impact on the global economy. Additionally we determined it was necessary for student

to be able to describe how a visited company’s mission statement positively impacts the global

economy.

The second program outcome we developed for the program was to have students

understands values-based leadership. After participating in this program students will be should

be able tot describe what social justice means to them within the business field. Students should

be able to describe steps a visited company can take to have a more just approach to their

practice. Finally students will be able to identify five qualities that are part of a company that has

a values based approach to leadership. This learning outcome will be met through the various site

visits and presentation that we will visit during the program. The goal for students is to take this

learning outcome and keep using it in their daily practice, allowing themselves to criticize

personal daily procedures in order to analyze if they are following a social justice approach in

their personal and professional lives. With students having the ability to analyze their daily

practices, we hope that they can develop this initial learning outcome and adapt social justice

approaches to their everyday practices.

QUINLAN RAMBLE EVALUATION PLAN 8

The third program outcome for this program that we developed revolves around students

being able to explore career interests. After participating students should be able to articulate

their interests in the business field, identify important skills that would be valued in the

workplace. Additionally, students will be able to identify societal needs or inequities that

influence their career interest or plan. for students to be able to examine the mission statements

of the companies we visit for commitment to the global economy and just approaches to

business, and identify the one mission statement they relate to the most. Students will be able to

do this after each site visit through their reflection activities and by participating in blog posts

through the program in order to discuss their thoughts about each company. They will have time

to develop their insights about each company through reflection and also share with peers

utilizing the blog posts.

The last of the program outcomes that we developed is for students who participate in

this program to be able to build professional networks. After participating in this program

students should be able to identify a professional in the field that exemplified great leadership

skills. Additionally, students will be able to develop and nurture at least one connection with a

professional in the field. We hope that students use this opportunity to grow their network and

this learning outcome is to ensure that the students take the right steps in doing that. Our intent

for this learning outcome is for students to be able to develop and nurture at least one connection

with a professional in the field. We will help students develop this, but having them write thank

you notes to all the professionals that we visit throughout the program and also to follow up with

any further question that they have. By forming these relationships, students should commit to

forming intentional relationships to develop a more connected business environment.

QUINLAN RAMBLE EVALUATION PLAN 9

With these updated outcomes, we are able to steer the program in new and intentional.

The program goals help tie the program to the mission of the institution and of the division and

with the new goals being developed, they will bring the program closer to aligning with the

mission.

Logic Model Description

Taking the program goals into consideration, we developed a logic model. A logic model

is a visual representation of a program’s components, the operations of a program, and the

potential effects and outcomes of the program (Kellogg Foundation, 2004). These components

include the available resources and program activities (i.e. the intentional efforts by program

coordinators). This also includes outputs, outcomes, and impacts, which reflect anticipated

results of the program (Kellogg Foundation, 2004). As a visual representation, the logic model

can provide a photographic comprehending of all program components and how effective they

are at achieving their goal. The logic model we developed (Appendix A) outlines the four

learning outcomes from which more precise student learning outcome derive. These are

commitment to understanding global economy, understanding values-based leadership, explore

career interests, build professional networks. These four larger umbrella outcomes are then

separated into three categories: short-, medium- and long-term. Each of the categorized student

learning outcomes become measureable at certain times or durations from which the program or

learning experience took place (Kellogg Foundation, 2004). Short-term outcomes, as its name

implies, are measureable a short time after the learning experience has taken place (typically

immediately upon initial experience). Medium-term outcome become measureable 2-3 years

after the experience, and long-term outcomes become measurable at least five years from the

experience.
QUINLAN RAMBLE EVALUATION PLAN 10

When looking at the learning outcomes for this program, the logic model suggests to look

at a lot of background information as well. One of the key things that need to be discussed is who

will participate in the program. For the Quinlan Ramble, only undergraduate students that are

part of the Quinlan School of Business can participate. We are looking for participants of all

majors and class standings, but it will be limited to only sixteen participants. This program is

being assessed because the department is looking to have the mission of the program connect

back to the mission of the institution. It is looking to develop new goals and procedures, as this

affects students, the department of students engagement, department of advancement, business

career services at Loyola, as well as employers and alumni, who are the ones usually doing the

company presentations.

A lot of factors also go into the development of the program, which in the Logic model

they are labeled under inputs. The most crucial inputs for this program are the staff that help

develop it. The coordinator of student engagement and the graduate assistant for this department

are the key players in developing the program, along with the help from the Department of

Advancement and Business Career Services. Another major input is the funding for this

program. Each of the students pays $400, but the department has to use its budget to make up for

the rest of the cost for each of the students. The cost takes into consideration flight, hotel for the

whole program, transportation, and most meals. Luckily, Allison stated that the program is well

funded, as the division believes it helps the students.

The final things to consider when looking at the logic model is the context of the

program. The environment in which the program is situated is crucial in the creation of the

program. For the Quinlan Ramble, it is important to note that Allison has only been in this role

for about two months, and the graduate assistant has never actually participated in the program.
QUINLAN RAMBLE EVALUATION PLAN 11

We also have to take into consideration that we are relying on companies to give the

presentations, so a lot of coordination needs to happen in order to develop a full working

schedule for the program. One of the last things that have to be considered when planning the

program is the dietary restriction of the participants. Since we are responsible for most of the

meals, planning the schedule has to revolve in including everyone and providing food for

everyone. We also have to acknowledge all the different assumptions that come with this

program. We are assuming that students want to participate, and also that students are willing to

pay for the cost of the program. We are also assuming that we will be able to find companies that

promote social justice and global awareness to align with our mission and that the professionals

for each company will be able to clearly demonstrate and relay this information to our students.

The last thing that we are assuming, and probably one of the biggest, is that career development

through global awareness and values-based leadership education is important to a student’s

success in their career.

Assessment Purpose

For this assessment proposal, the Quinlan School of Business is looking to make some

changes to the way that the Quinlan Ramble operates. The main focus of the assessment is two-

fold: (1) to establish program goals and student learning outcomes and (2) to determine whether

student participants leave the Quinlan Ramble having reached or attained the intended learning

outcomes. Since there is not much laid for this program, Allison is really interested in starting to

set the foundation for what the program can be and start aligning the practices and processes of

this program back to a central mission. Quinlan would like to incorporate more reflection and

social justice elements into the business trip. Allison mentioned that the group typically goes to

about six to eight business sites throughout the trip, for undergraduates to meet up with
QUINLAN RAMBLE EVALUATION PLAN 12

executives and employees, but they currently do not take intentional time to pause and reflect or

have a dialogue about their experiences. Establishing program outcomes can gear Allison to

develop activities focused intentionally to pause and reflect. Allison would like to incorporate

more conversations and reflections about what impact these business already have or could have

on consumers or employees in a global economy and how they are approaching leadership.

Quinlan mentioned that this would be a great addition to the Quinlan Ramble because it would

align the purpose of the trip more closely to the mission and values of the institution as a whole.

The program does not have clearly defined learning outcomes. in addition to creating

these outcomes, Allison would like to explore ways in which they could address social justice

issues and concerns within the business field and specifically within their Quinlan Ramble trip.

Additionally, Quinlan would like to know the if students who partake in this trip leave the trip or

program feeling like have a “leg up” on other students who did not take this trip. Allison is really

interested if the Quinlan Ramble situates students in a more promising position for after

graduation. Quinlan wants to know if this program provided them with the sense of being able to

advance beyond their peers in their desired business field because of this trip. The proposed

assessment will focus on both the outcome and the processes. Due to the program’s lack of

established outcomes, we will be working to establish and solidify these learning and program

outcomes. We are interested in a longitudinal approach because we are interested in measuring

the achievement of these program and learning outcome after a period of time.

In particular, Allison, the coordinator of the program, is the most interested in the results

of the assessment. Due to her being new to this position as of two months ago, she was given the

challenge to realign the purpose of this program back to the mission of the institution. She is

taking intentional steps to make sure that are being done, so this first assessment will be crucial
QUINLAN RAMBLE EVALUATION PLAN 13

in assessing the new direction of the program. This assessment will be used to formulate a clear

mission and learning outcomes for the program that are aligned to the mission of the division and

the institution. Due to the lack of documentation from the previous organizer of the event, this

first program will be used as a benchmark for the upcoming years.

Assessment Questions

We are proposing to approach this assessment as a formative assessment because we will

explore ways of improving the program as opposed to a summative assessment which seeks to

prove that the program is worth keeping. We propose to assess the program goals. Based on the

assessment purpose our assessment question is two-fold: (1) do students who participant in the

Quinlan Ramble achieve the short-term learning outcome immediately after participant in the

experience (2) do participants achieve medium term learning outcome after two to three years

after the program.

The program currently does not collect usage metrics. Out of 80 student applications they

receive on an annual average they only accept about 16 students. There is currently no protocol

or set structure for determine how to advocate or support students who meets all the

requirements to attend the program, but who can not meet the financial need to pay for the

program. Alison did mention that the cost might be significant of certain student populations.

The program does not have a record or data about who uses this program or who engages in it.

Getting a hold of graduation and employment comparison between those that participated in the

program and those that did not might prove helpful in collecting data on the immediate and

longer term impact of this program.

Quantitative Assessment Approach

QUINLAN RAMBLE EVALUATION PLAN 14

We propose a descriptive design for our quantitative assessment of the Quinlan School of

Business’ Quinlan Ramble program. The goal of descriptive designs is to understand what is

occurring in a given scenario or program without necessarily looking at what factors contribute

to the phenomenon (Henning & Roberts, 2016). Our assessment design will be descriptive

because its goal will serve to explain whether student participants are meeting learning and

developmental outcomes. The resulting data will be a statistical analysis measuring the student's

achievement of the Quinlan Rambler’s program learning and developmental outcomes. Our

assessment design is not intended to explain the relationship between the outcomes and any other

matter of causality. Our design will also consider a panel study since the cohort of student

participants is relatively small.

Our design will also be longitudinal because the goal of our assessment will be to

demonstrate change in learning and development over time as a result of the program (Henning

& Roberts, 2016). In longitudinal designs data are collected two or more times. Our assessment

plan will look at the extent to which students have achieved the (short- and medium-term)

outcomes. In order to do this we propose a pre- and post-program survey to gauge baseline level

of learning/development. Pre- and post- survey scores will be compared to determine any change

in learning/development over time. The pre-test will be administered immediately after the

program while the post-test will be administered 2-3 years after the program. Since descriptive

longitudinal designs are useful for providing detailed information regarding a specific

phenomenon over a period of time (Henning & Roberts, 2016) the design seems to fit well

within our assessment's purpose and questions. The purpose of our assessment purpose is two-

fold. The following are our two assessment questions: (1) do students who participate in this

program achieve the short-term learning outcomes immediately after program participation and
QUINLAN RAMBLE EVALUATION PLAN 15

(2) are participants achieving the medium-term outcomes expected as a result of participating in

this program after 2-3 years of program participation. Descriptive designs are also relevant for

explaining student development and the impact of interventions on student growth (Henning &

Roberts, 2016). Our assessment project looks to do this.

Survey Instrument

Our survey instrument covers four broad areas of the assessment plan and program

outcomes: understanding the global economy, understanding values-based leadership, exploring

career interest, and building professional networks. There are approximately 40 survey items that

should take approximately 15 to 20 minutes to complete. We will not be conducting a large-scale

or intensive piloting study of our survey, as it would not be feasible to find a group of students

that mimics the ones that are participating in the program except for past participants. The

likelihood that we have access to these past participants is not high. Additionally, it would be

time-intensive. However, it is suggested to have colleagues review the survey in its finished

format to ensure its clarity, bias, and proper function online (if the survey is being administered

online which ours would be; Henning & Roberts, 2016). If by chance there is a past participant

of the program who is still connected with the office, we might ask them to volunteer in an

individual small piloting of the survey. We would use this volunteer to help us gauge the time it

took to complete the survey so we give an more accurate indicator to actual survey participants.

As mentioned previously this would likely be unfeasible.

As mentioned previously, our survey would be administered online. Although our survey

is void of any branching questions, an online issue of the survey would greatly facilitate data

collection and reduce data entry errors (Henning & Roberts, 2016). Since we are doing a

longitudinal design, we will be administering the survey immediately after the program and then
QUINLAN RAMBLE EVALUATION PLAN 16

we will be sending the same survey 2-3 years after the program. The survey will consider short-

and medium-term outcomes. The timeline of our survey release and the time we leave in between

both releases of the surveys are appropriate considering the immediate survey release should

satisfy short-term learning and development while the 2-3 years leaves times to development and

satisfy the medium-term outcomes. Logically we can assume a growth through the comparison

of these two exact same surveys since we can expect that students will not reach medium-term

outcomes after immediate completion of the program. Any indifference or regression in the

comparison analysis might reveal noteworthy results; however, this might also reveal one of the

limitations of our longitudinal design which is called a response-shift bias. A response shift bias

is when pre-test scores are inflated due to the student’s over confidence when the student realizes

on the post-test that they did not know as much as they initially thought.

The student engagement manager and/or the student engagement graduate assistant will

administer the survey. We will not use an incentive to increase response rates for the initial

survey that will be conducted immediately after the program because it is a small number of

students that participate per year and it will be built seamlessly into the program curriculum.

Since the initial survey will be built into the program curriculum, we will not incorporate survey

reminders, however, the post-survey will incorporate two survey reminders a week after the

initial release and then a second one a week later.

Quantitative Analytic Plan

We would primarily use an inferential statistical analysis to answer our assessment

question. Descriptive statistical analyses will be utilized simply to seek our participant sample in

terms of demographic variables (i.e. academic standing, major, gender, etc.). Because our survey

sample includes all general student participants in the Quinlan Ramble program, we are not
QUINLAN RAMBLE EVALUATION PLAN 17

relying too much on a descriptive statistical analyses; however we will want to know which

student population we are basing our generalized result on (Henning & Roberts, 2016).

Our assessment plan would benefit from collecting sample information regarding

univariate categorical nominal data of class standing. Our assessment plan would benefit from

this because a limitation of its longitudinal design is distinguishing whether achievement of

outcomes results from program participation of nature student maturation.

An inferential statistical analysis would play a central role in our assessment plan.

Inferential statistical analysis would allow us to move beyond simply describing our sample to

generalizing the findings from the survey sample to the entire Quinlan Ramble student

population (Henning & Roberts, 2016). The null hypothesis we would seek to reject is there is no

difference between the learning outcome item means (averages) between the pre-survey and

post-survey time, suggesting no change in students’ learning or development. Our research

hypothesis would be that the learning outcome item means (averages) are not the same between

pre-survey time and post-survey time, suggesting change in students’ learning and development.

In our inferential statistics we would seek to analyze the group’s mean pre-program (i.e.

immediately after program) results versus post-program (i.e. 2-3 years after the program).

Comparing the two groups’ means would allow us to generalize about the group and their

achievement of short- and medium-term outcomes over time.

We would perform a paired-samples t test. This would allow us to compare two mean

scores across time to determine whether there is a difference. Our findings would be presented

using tables to visually represent our item mean comparisons. These finding items will be

presented in our table: group size (n), standard deviation (SD), t statistic from t test (t), and

significance level (p value).

QUINLAN RAMBLE EVALUATION PLAN 18

Qualitative Assessment Design

We will be following a narrative qualitative design. According to Henning & Roberts

(2016) narrative designs rely on participants’ stories, which reflect their experiences and the

meaning they make of those experiences. Because this assessment seeks to measure whether

students have achieved the intended learning outcomes, we will combine the narrative findings

with the quantitative survey results. As a narrative design revolves around the students’ stories,

we will be able to get a better insight on how our individual students accomplished the short and

medium term outcomes of the program and if some of the outcomes were not achieved, it will

also allow us to understand how we can make it possible for students to achieve that goal. An

interview approach would be more ideal for our specific program. We want to get in-depth

answers from our participants and a semi-structured interview method will allow us to ask

follow-up questions when needed to get a better insight on the students’ answers or when we

need clarification (Adams, 2010). By doing interviews, as opposed to a focus group, we will be

able to analyze each individual response and gauge how much growth and learning actually

occurred in the learning outcomes we are trying to measure and the aspects of the program that

relate to such learning. Another reason we will do interviews is because we have a small sample

of students that will be participating in the program, and all are from varying academic, age, and

socially groups. Completing interviews will allow us to personalize, to an extent, the type of

conversations we have in order to maximize the amount of information we get from each

participant (Robert & Henning, 2016).

Ideally we will implement the qualitative approach after the survey. It would be helpful

to complete the interviews after the survey, because we will be using the interviews as a way to

better understand the participants’ responses to the survey items. All of the students that are
QUINLAN RAMBLE EVALUATION PLAN 19

participating in this year’s program will be asked to participate in the interviews. Since there are

only a small number of participants, we will be reaching out to all of them to participate in the

interview. This will give us more information to work with and will be more telling of how our

students are able to achieve the outcomes or some hindrances that do not allow them to achieve

others. Due to the fact that students come from various backgrounds, both personally and

academically, collecting data from most of them will reveal more inclusive results than if we

were to target any one specific population at this point.

Protocol

The interview protocol will be split into an introductory section, four main questioning

sections, and a concluding section. There are distinct sections for each of the four outcomes

areas: understanding of global economies, understanding values-based leadership, career

interest/exploration, and building professional network (Appendix A). These derive from our

logic model and were identified as a relevant structure to our questionnaire to measure achieved

outcomes from the perspective of the participant. As a result, evaluators will be able to gain

further insight from quantitative survey data and understand whether students achieved the

learning outcomes and possibly for what reasons. Similarly, to the quantitative survey, the

interview questions will not be pilot tested. The small sample population of participants and our

lack of access to previous participants contact information contribute to this decision.

The 2018 Quinlan Ramble concludes on March 10th, therefore interviews will occur from

April 9th to April 30th, after participants have completed the survey. This scheduling works best

given that it will allow time for participants to first complete the survey and then to reflect on

their experiences within the program. Allison, Fernando, and Jona will conduct interviews.

Allison and Fernando have the most involvement with Quinlan Ramble, but Jona will assist with
QUINLAN RAMBLE EVALUATION PLAN 20

this process since conducting 10 to 14 interviews may be overwhelming for two people to

complete in three weeks. Since the interview is not necessarily asking them to reflect and explore

intimate social identities, having an unfamiliar evaluator, Jona, should not have a significant

impact on the participants’ comfort to answer and process the interview questionnaire with Jona.

Allison, Fernando, and Jona will individually conduct interviews on their own and take notes to

record participant responses. Interviews will be audio recorded with participant permission. By

doing this, evaluators will have the most accurate records of participant’s responses and thereby

qualitative data to accompany our quantitative results. Participants’ names will not be shared

publicly. Participants will have the ability to decline or end the interview at any point in time.

Interviews will take place in Allison’s office or reserved study rooms in the Water Tower

Campus library in order to provide some level of privacy. Interviews will occur based on the

evaluator’s schedules, but primarily from 10:00 am to 5:00 pm between Monday and Friday to

provide flexibility in scheduling around participant class and work schedules. Adams (2010)

suggests interviews be scheduled for 60 minutes in length. Therefore, ours will be scheduled for

an hour. Given the relatively small participant sample and the community building that would

have ideally taken place during the program, no incentives will be provided to participants.

Qualitative Analytic Plan

The first step that we will take to analyze our interview data is to do attribute coding. We

will be using a demographic variable form (Appendix B) that will be asking our participants

demographic variables such as class standing, age, gender, and racial/ethnic heritage. We will be

asking for this information to learn more about the specific populations that are taking advantage

of the opportunities that the Quinlan School of Business offers. We will also like to learn how

different students perceive the information that is being presented during the Quinlan Ramble.
QUINLAN RAMBLE EVALUATION PLAN 21

The next step would be to look at all the transcripts from the interview and conduct descriptive

coding, which means to look through the transcripts to identify common themes or topics. We

will be using a parallel approach when going over the answers of the interview items. We will

look at each specific question and then analyze the answer for that question from all of the

participants. This will help us when finding emerging themes for each of the questions and will

allow us to follow a more comparative analysis of the responses. We will not be creating

predetermined codes and instead will be using the emerging themes as we analyze each of the

questions at a time to find the codes that we will be using. Because the interview questions that

we will be asking are directly drawn from the learning goals and our assessment question, we

will be using pattern coding as the third step in analyzing the responses. We will be taking the

emerging codes from the responses to each of the interview questions and finding the differences

and similarities across the particular demographic groups captured on the demographic variable

form. This part of the analysis will be the most revealing, as we will be able to combine all of our

findings and see if there is a pattern among certain student populations or if some of the learning

goals that students were expected to achieve were not achieved by most students. This will help

us suggest specific improvements or changes to the overall program.

When analyzing the data, we will be following a pawing technique, which will allow us

to find emerging themes quicker by the highlighting and notes that we make when looking at

individual transcripts. Member checking is “a way of soliciting feedback on preliminary finding

from participants” (Robert & Henning, 2010 p. 164). We will be employing this by sending the

initial codes that emerge from the data to each of the participants. We want to make sure that the

participants approve of how we have interpreted their information and that we used the correct

code to categorize their answers. We will not be using inter-rater reliability as the two people
QUINLAN RAMBLE EVALUATION PLAN 22

will look at the answers at the same time and they will both decide how to interpret the answers

together. Unfortunately, we do not have the staff power to make triangulation happen although it

would be helpful to confirm our findings through multiple means (Robert & Henning, 2016). To

present the findings from our interviews, we will be using a construct map to highlight the main

emergent themes from our findings. We will use quotations that help encompass the overall

themes that we find in the answers to the questions asked during the interviews. These will be

pretty basic, only containing the main themes, and these will be distributed to administrators that

are not directly connected with the program. For those that helped develop the program or played

a key role in facilitating any aspect of the program, we will develop tables that include the

emergent themes, along with specific quotation that help showcase that theme.

Limitations

One of the biggest limitations that we have to think about is the fact that these learning

outcomes are new to the program, and there is not much previous work to help indicate if we are

moving in the right direction. The curriculum of the program is very early in its developmental

process, so some of the questions and survey items we are proposing might be out phased by the

time that the actual program happens. Our participants are also at various developmental and

maturity levels which might impose how students process the information that they are presented

with during the Quinlan Ramble. Since some of them are first- and second-year students,

developmentally, they might not be able to process the information that they receive at the same

degree as juniors and seniors might, which in turn might lead them to not meet the learning goals

that we have set up for this program. The program itself is developed, managed, facilitated, and

assessed by only two people, so there is a high possibility that our constant interactions with the

students will lead to deeper relationships. The fact that we will get to know the students while on
QUINLAN RAMBLE EVALUATION PLAN 23

the trip will possibly lead us to believe we know what they mean when they say a specific

answer, which in turn can lead us to interpret data differently or take it out of context. This

program is only taking sixteen participants, and it is safe to think that not all of them will

participate in the assessment of the program, thus giving us a limited number of responses. This

will in turn lead to limited understanding of if and how students are achieving the learning

outcomes that we have placed in the program.

Timeline

Due to the fact that we will be assessing both the short and medium term learning

outcomes, we have a pretty extensive timeline. The first survey will be sent out a week after the

students have returned from the trip, giving them a total of three weeks to complete. We will be

sending a reminder email a week before the final deadline to participate in the survey. The

interview will be administered through the next month following the deadline of the survey.

Since we are also trying to measure the medium term learning outcomes, the second and final

survey will be sent out two years after the student participated in the program. Our goal is that by

this point, the student either has had real world experience in the field, or is a either a junior or

senior and will be able to have more insight in the world of business. Following the same

procedure as the first survey, the final interview will begin the following month after the second

survey is done.

Budget

There is not much of a budget for this assessment project. Most of the work is under the

job responsibilities of both the program coordinate and the graduate assistant working on this

program. The only budget that this assessment will need is for the incentive to encourage

students to participate in the survey and the interview. For those students that choose to
QUINLAN RAMBLE EVALUATION PLAN 24

participate, we will be giving them a gift bag that includes Quinlan branded items. Each year that

we do this, the gift will be different, but the cost of it will stay around $30. For this year, we have

a budget of $480.

Next Steps

This assessment proposal will actually be used as part of the programs’ procedure. Once

the first round of surveys and interviews are completed, and we are able to gather some date and

information, we will use that to make any changes to the learning outcomes of the program. We

are truly looking for this assessment to provide some guidance on what the Quinlan Ramble will

become in the future.

QUINLAN RAMBLE EVALUATION PLAN 25

References

Adams, W.C. (2010). Conducting semi-structured interviews. In J.S. Wholey, H.P. Hatry, &

K.E. Newcomer (Eds.), Handbook of practical programs evaluation (3rd ed.) (pp. 365-

3770. San Francisco, CA: Jossey-Bass

Henning, G.W., & Roberts, D. (2016). Student affairs assessment: theory to practice. Sterling,

VA: Stylus.

Holland, J. L. (1997). Making vocational choices: A theory of vocational personalities and work

environments (3rded.). Odessa, FL: Psychological Assessment Resources.

Loyola University Chicago. (2017). Retrieved December 10, 2017, from

https://www.luc.edu/strategicplanning/plan2020/

Loyola University Chicago. (2017). Retrieved December 10, 2017, from

https://www.luc.edu/quinlan/aboutquinlan/missionandvision/

Magolda, M. B. (2008). Three Elements of Self-Authorship. Journal of College Student

Development, 49(4), 269-284. doi:10.1353/csd.0.0016

Rogers, P.J. & Goddrick, D. (2010). Qualitative data analysis. In J.S. Wholey, H.P. Hatry, &

K.E. Newcomer (Eds.), Handbook of practical program evaluation (3rd ed.) (pp. 429-

453). San Francisco, CA: Jossey-Bass

W. K. Kellogg Foundation. (2004). Using logic models to bring together planning, evaluation,

and action: Logic model development gu ide. Battle Creek, MI: Author.
QUINLAN RAMBLE EVALUATION PLAN 26

Appendix A

Logic Model
QUINLAN RAMBLE EVALUATION PLAN 27

Appendix B

Quinlan Ramble 2018 – Seattle

Thank you for participating in the Quinlan Ramble 2018 trip to Seattle. The week was full of company
visits, meeting business professionals and alumni, and fun experiences. We appreciate you taking the
time to fill out this survey so that our program can better understand students’ experiences and
improve our program.

Please complete this survey, which should take approximately 15-20 minutes. Your response will be
recorded electronically after you click submit. Should you have any trouble completing the survey or
have any questions, please reach out to Allison Davis at adavis10@luc.edu.

For each of the following questions, please select only one answer

General Information

1. At the time of my participation I was at what class standing.

o First Year
o Sophomore
o Junior
o Senior
Understand the Global Economy – We’d like to know about your experience understanding local and
global markets in the global economy.

2. Before the Quinlan Ramble I could explain what a global economy was.
o Strongly Disagree
o Disagree
o Neutral
o Agree
QUINLAN RAMBLE EVALUATION PLAN 28

o Strongly Agree
3. By participating in the Quinlan Ramble I have a better understanding of the power
power relationship between markets in the USA and global markets.
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree

4. By participating in the Quinlan Ramble, I can identify specific company practices that
have a socially just impact on the global economy.
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree

5. Before the Quinlan Ramble I was able to define (1) Sustainability (2) Well-being (3)
Economic Growth is required for a balanced global economy.
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
6. By participating in the Quinlan Ramble I am now able to define (1) Sustainability (2)
Well-being (3) Economic Growth is required for a balanced global economy.
o Strongly Disagree
o Disagree
o Neutral
QUINLAN RAMBLE EVALUATION PLAN 29

o Agree
o Strongly Agree
7. By participating in the Quinlan Ramble I am able to ask prospective employers about
their participation in the global economy.
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
8. Please rate yourself on your ability to do the following after participating in the Quinlan
Ramble.
I am able to describe the extent to which a prospective employer's mission positively
impacts the global economy.

o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree

Understand Values-based Leadership – Now we’d like to learn about your experiences related to
values-based leadership.

9. Before the Quinlan Ramble, I had learned how social justice can be applied in the
business field.
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
QUINLAN RAMBLE EVALUATION PLAN 30

10. By participating in the Quinlan Ramble I am able to provide specific examples of social
justice in the business field.
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
11. A company can both have a social justice-based approach and be successful.
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
12. By participating in the Quinlan Ramble I am able to identify steps that a company can
take to adopt more socially just practices.
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
13. I can explain how my personal values relate to possible career paths for me.
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
14. After participating in the Quinlan Ramble, I can identify the personal values that are
important to me
o Strongly Disagree
o Disagree
QUINLAN RAMBLE EVALUATION PLAN 31

o Neutral
o Agree
o Strongly Agree

15. After participating in the Quinlan Ramble, I will actively seek employment by companies
that have similar values as my own.
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree

16. My current internship/professional work follows a values-based leadership approach.

o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
Explore Career Options – Now let’s hear about your experience related to career exploration

17. Before participating in the Quinlan Ramble, I knew the career path that I wanted to take
after graduation.
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
18. By participating in the Quinlan Ramble, I learned about different business sectors that I
did not know about prior to this experience.
o Strongly Disagree
QUINLAN RAMBLE EVALUATION PLAN 32

o Disagree
o Neutral
o Agree
o Strongly Agree
19. By participating in the Quinlan Ramble, I have a better understanding of the skills that
are valued in the workplace.
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
20. I can identify the strengths that I currently hold that are valued in the workplace.
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
21. By participating in the Quinlan Ramble, I was able to learn about societal needs
(healthcare, benefits, life insurance) in the workplace
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
22. By participating in the Quinlan Ramble, I was able to identify inequalities (gender pay
gap, discrimination based social identities) in the workplace
o Strongly Disagree
o Disagree
o Neutral
QUINLAN RAMBLE EVALUATION PLAN 33

o Agree
o Strongly Agree
23. After participating in the Quinlan Ramble I have a better understanding of the career
path I want to pursue after graduation.
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
24. I have been able to match my strengths with job opportunities in the business field
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
25. I have been able to match my experiences with job opportunities in the business field
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
26. I need more help narrowing my possible career paths
o Yes
o No
27. If you answered “yes” to #26, do you need more help narrowing possible career paths to
that align to your personal values
o Yes
o No
QUINLAN RAMBLE EVALUATION PLAN 34

28. I need more help narrowing my possible career paths to a select few that fit society
inequities
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree
Build Professional Networks – Lastly, we’d like to hear about your capacity to build professional
networks.

29. By participating in the Quinlan Ramble I was able to meet professionals in the business
field.
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree

30. How many of the business professionals that you met during the Quinlan Ramble have
you contacted to follow up with questions or gratitudes?
o0
o1
o 2-3
o 4-5
o 6+
31. How many of the business professionals that you met during the Quinlan Ramble do you
plan on staying in contact with for questions or advice in your future career?
o0
o1
o 2-3
o 4-5
QUINLAN RAMBLE EVALUATION PLAN 35

o 6+
32. There are clear benefits to fostering and keeping professional relationships with other
professionals
o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree

33. Why did you decide to participate in the Quinlan Ramble?

o It was friend/professor/mentor’s recommendation
o You wanted to travel to Seattle
o You had heard of other’s experiences during Quinlan Ramble
o One or more of the visited companies is of interest to your career
o Other reason

34. I would participate in the Quinlan Ramble as a returner

o Strongly Disagree
o Disagree
o Neutral
o Agree
o Strongly Agree

35. Please provide us with your LUC ID number: __________________________

QUINLAN RAMBLE EVALUATION PLAN 36

Thank you for participating in the Quinlan Ramble 2018 trip to Seattle and
for completing this survey.
QUINLAN RAMBLE EVALUATION PLAN 37

Appendix C
QUINLAN RAMBLE EVALUATION PLAN 38

Appendix D
QUINLAN RAMBLE EVALUATION PLAN 39

Appendix E

Quinlan Ramble Spring 2018 – Interview Protocol

Assessment Questions:
1). Do students who participate in this program achieve the short-term learning outcomes
immediately after program participation
(to be read to participants)
Hello, as you may know my name is (insert name) and I am a (Program Coordinator and/or
Graduate Assistant) at Loyola’s Quinlan School of Business. As you may remember, through the
electronic survey you received, I am working on an assessment project that is seeking to learn
more about students’ experiences in the Quinlan Ramble so that we can better understand and
improve the program.
I would like to record this interview. Recording this will ensure that I have an accurate account
of your responses. Is that ok with you?
The interview should be around an hour. This interview will delve deeper into some on the
survey questions. Also your name or any identifying information will not be used in any reports
or publications that result from this study. The audio recording of this conversation will be
deleted after it’s serviced its purpose. If at any time during our interview you want to end the
conversation, please feel free to do so. Please read this consent form and sign it when done. If
you have any questions about the consent form or about how the interview information will be
used, please let me know.

Do you have any questions for me before we begin?

Ok. The following questions will prompt you and guide me in understanding your Quinlan
Ramble experience. There are no right or wrong answers and my main objective is to listen to
your story and experience. By being here you taking the time so that our program can better
understand students’ experiences and improve our program.

QUINLAN RAMBLE QUESTIONNAIRE

SECTION ONE: Understanding the Global Economy
I want to start by getting an idea of your overall understanding of the global economy and how
big or small markets impact the larger economy. So to start…
 Question 1: I’m wondering; how would you define a global economy?
o Probe 1: Who are participants in the global economy?
 Question 2: Did the Quinlan Ramble program help you understand power relationships
within the global economy?
QUINLAN RAMBLE EVALUATION PLAN 40

o Probe 1: What elements of the program contributed toward understanding power

relationships within the global economy?
 Question 3: Tell me about one of the companies/businesses you visited, how are their
current business practices impacting the global economy?
o Probe 1: What influence does a companies’ mission statement have on their
business and the global economy?

SECTION TWO: Understanding Values-based Leadership

These next few questions are concerned with your understanding of values-based leadership:
 Question 1: What does social justice in field of business mean?
 Question 2: What are some examples that you saw of social justice playing out in the
companies that we visited in the Quinlan Ramble?
 Question 3: Can you describe what values-based leadership means to you?
 Question 4: As a result of this program has your definition of social justice changed or
evolved? Please explain.
o Probe 1: Prior to the Quinlan Ramble, did you consider businesses’ practices and
it’s relation to social justice concerns? If yes, can you provide an example of a
company and their practices?
o Probe 2: Since participating in the Quinlan Ramble, have you begun to consider
how businesses’ practices and its’ relation to social justice concerns? If yes, can
you provide an example of a company and their practices?

Now let’s shift from describing your understanding of these topics to exploring topics of career
interest and professional networks.

SECTION THREE: Exploring Career Interests

These next few questions are concerned with career interests
 Question 1: Can you share with me your career interests at this time?
o Probe 1: Has your interest in the field changed as a result of participating in the
Quinlan Ramble?
 Question 2: Before participating in the Quinlan Ramble, what skills would you have said
are important to anyone going into your desired field?
o Probe 1: Would you say those skills remained the same since the Quinlan
Ramble?
o Probe 2: What are some of your strengths and skill related to those you consider
important in the field?
o Probe 3: What are some experiences that helped you gain those skills?
o Probe 4: What are experience and skills you still hope to gain in the future?

SECTION FOUR: Building Professional Networks

QUINLAN RAMBLE EVALUATION PLAN 41

These next few questions are the final topic questions before we concluded with some closing
questions. These next few question are concerned professional network in the business field
 Question 1: What are qualities of an exemplary professional in the field?
o Probe 1: Can you provide example from the Quinlan Ramble?
 Question 2: Is there someone you connected with during the Quinlan Ramble (i.e. another
participant or employer)?
o Probe 1: Do you intend to stay connected with them? What could be some
challenges or advantages to staying connected?
 Question 3: Can you speak to some behaviors or action steps you can practice to develop
your professional network?
o Probe 1: How would you describe your current professional network?

SECTION 5: Closing Questions

 What was the main reason you decided to participate in the Quinlan Ramble?
 In relation to the business field and practice, are there any topics that you were surprised
to learn about during the Quinlan Ramble?
 On the opposite end, were there any topics you had previous you previously explored to
the same amount of depth in prior experience or the classroom.
 What are your major “takeaways” from the Quinlan Ramble?
 Do you have any closing thoughts or reflections you’d like to share?

Appendix F
Quinlan Ramble Interview Participant Demographics
QUINLAN RAMBLE EVALUATION PLAN 42

Demographic information is requested for data analyses purposes only.

Date: Time: Place:

Year in School: Age:

__ First year __ Under 18 years old
__ Sophomore __ 18-24 years old
__ Junior __ 25-34 years old
__ Senior __ 35-44 years old
__ 45 years or older
Gender: Racial/Ethnic Heritage:
__ Woman __ American Indian or Alaska Native
__ Man __ Asian
__ Transwoman __ Black or African American
__ Transman __ Hispanic, Latinx, or Spanish origin
__ Genderqueer __ Middle Eastern or North African
__ Not listed: ___________________ __ Native Hawaiian or Pacific Islander
__ Prefer not to say __ White
__ Not listed: __________________________
__ Prefer not to say

Major(s): _______________________________________________________________

Appendix G
CONSENT TO PARTICIPATE IN THE QUINLAN RAMBLE INTERVIEW
QUINLAN RAMBLE EVALUATION PLAN 43

Project Title: Quinlan Ramble

Interviewer: ________________________________

Introduction:
You are being asked to take part in an interview to help inform the Quinlan School of Business
staff about how the Quinlan Ramble program exposes students to career exploration and global
business trends. You are being asked to participate based on your participation in the 2018
Quinlan Ramble.

Please read this form carefully and ask any questions you may have before deciding whether to
participate in this focus group.

Purpose:
The purpose of the interview is to gather information about students’ understanding of global
economies, understanding of values-based leadership. Additionally, we want to collect
information in regards to their career exploration and their professional networks to study the
impact of the Quinlan Ramble.

Procedures:
If you agree to participate in the interview, you will be asked to respond to a series of questions
that will last for approximately 60 minutes. You are encouraged to respond openly and honestly
to the questions asked of you, although you should only respond to questions when you feel
comfortable doing so.

Risks/Benefits:
There are no known risks involved in participating in this study. Benefits may include
improvements to the Quinlan Ramble program.

Confidentiality:
Your name will not be associated with your responses in the interview reports. We will compile
a report of basic themes and share it with you prior to finalizing it. The information gathered in
today’s interview will only be shared with staff of the Quinlan School of Business.

Voluntary Participation:
Participation in this interview is voluntary. Even if you decide to participate, you are free to
withdraw from participation at any time without penalty.

Contacts and Questions:

If you have questions about the focus group you can contact Allison Davis at adavis10@luc.edu.
QUINLAN RAMBLE EVALUATION PLAN 44

Statement of Consent:
Your signature below indicates that you have read and understood the information provided
above, have had an opportunity to ask questions, and agree to participate in this interview. You
will be given a copy of this form to keep for your records.

____________________________________________
Participant’s Signature

Date ____________

____________________________________________
Interviewer’s Signature

Date ______________

Appendix H
QUINLAN RAMBLE EVALUATION PLAN 45

Qu in lan Ram b le
t o Seat t le

Fernando Perez and Jona Salamanca

QUINLAN RAMBLE EVALUATION PLAN 46
QUINLAN RAMBLE EVALUATION PLAN 47
QUINLAN RAMBLE EVALUATION PLAN 48
QUINLAN RAMBLE EVALUATION PLAN 49
QUINLAN RAMBLE EVALUATION PLAN 50