You are on page 1of 15

Formative Evaluation Data Collection Report

Academic Tutoring: A Program Evaluation

Jeremy Bell
Holly Hicks
Caroline Kiczuk
Maria Popel
Keith Webster

The University of Tampa


EME 640
March 31, 2014

1 of 12

Evaluation Plan
The need for the Academic Tutoring Program arose when students required
academic assistance when professors were not available. Students reacted positively to
the idea of having a free tutoring program at the Academic Center for Excellence (ACE),
and it has become an increasingly popular service at The University of Tampa. By going
to a reliable tutor, students are able to make the necessary adjustments in their classes
to earn better grades and pass their courses with confidence. The tutoring team
consists of both undergraduate and graduate students; they set their own time block to
conduct tutoring sessions in their area of expertise with struggling students. Tutoring is
available to any undergraduate or graduate student at The University of Tampa, and is
available on a walk-in basis. The unique involvement of student-to-student tutoring will
be a factor for this evaluation. We want to know how involved each tutor is in assisting
students, and whether their brilliant minds are able to coach their peers to success with
quizzes, tests, and term papers.

The goal of this evaluation was to assess the effectiveness of the tutoring
program, and to make recommendations as to how it could run more effectively. We
wanted to know if the program is a viable asset to an undergraduate students
education, and whether the tutors are reliable sources for assistance. We hoped to
answer these questions, and any others that arose, by conducting interviews and
sending out surveys to both tutors and tutees. We aimed to collect qualitative data from
our written surveys, and quantitative data from our electronic survey. We hope this

2 of 12

evaluation will result in us being able to offer suggestions that will make the program
more beneficial to students in the future.

Materials Evaluated
The Academic Tutoring Program at The University of Tampa is run by the
Academic Center for Excellence (ACE). It utilizes both individual and small group
instruction settings to assist students in a variety of subjects. The program aims to help
students improve their knowledge base, and skill sets in the desired subject area. The
subjects that are currently offered are: Biology, Chemistry, Math, Phycology,
Accounting, Economics, ITM, Anatomy, Public Speaking, French, Italian, German,
Chinese, Japanese and Spanish.

Participants
The participants in the evaluation consist of tutors and students. The tutors are
successful, experienced students who passed a special training course and did well in
the subject areas they are tutoring for. Their experience in tutoring varies from three
semesters to up to four years. The range of students turning up for the tutoring varies
weekly, and consists mostly of freshmen, sophomores, and juniors. Most of them are
asking for tutoring right before their tests or exams. Tutoring is available on a walk in
basis; no appointment necessary. Six tutors and twelve students participated in this
evaluation.

Data Collection Tools


The data were collected through written questionnaires, face-to-face interviews,
3 of 12

and on-site observations. An online survey was created using SurveyMonkey.com; it


consisted of nine Likert-scale questions, and one open-ended question (Appendix A).
The survey was open for eight days.

Twenty five copies of a paper survey (Appendix B) were left at the different
tutoring sites around campus for students currently being tutored to fill out. Tutors at the
different sites were asked to have their tutees fill one out after their tutoring session.

The tutor interview questionnaires (Appendix C) included twelve open-ended


questions and were filled out by willing tutors.

Data analysis consisted of examining the student survey questions, the online
survey responses, and the tutor interview questions. The researchers looked at each
question separately and looked for trends in the responses. The results from the
SurveyMonkey survey were imputed into an excel worksheet and graphs were made to
pictorially illustrate the results.

Tool Administration Process


We decided to use both paper and electronic surveys in our evaluation. As a
group we brainstormed what questions to ask and then ran them by both our professor
and the head of the tutoring department (Janice Law) before distributing them. We
dropped off our paper surveys (Appendix B) in the IT Annex Building with the tutors and
requested that they ask their tutees to fill one out after their tutoring session. We came
back a week later to find zero surveys filled out. We again asked the tutors if they would
administer the surveys and leave them for us to collect in one week; one week later we
4 of 12

again arrived to zero responses. We decided that we needed to go to some of the other
areas where tutoring takes place. We left surveys with the language tutors in the
Riverside Center, and got their email addresses so that we would be able to track the
surveys down. We also left surveys with the Anatomy tutors for their students to fill out.
This all resulted in us collecting one student survey (Appendix B) and six Tutor surveys
(Appendix C). Members of the evaluation team meet weekly, and observed tutoring
sessions in progress.

We created a survey with ten questions (Appendix A) to send out to students


who have received tutoring from the Academic Center for Excellence in the past. The
survey was created using the online service SurveyMonkey. The email accompanying
the survey informed participants that they would be entered to win a $10 Walgreens gift
card if they participated in the survey. Due to only having the free version of
SurveyMonkey, we were only able to send the questions to 100 students on the list. The
data was collected and imputed into an Excel spreadsheet for analysis.

Results
The results of the electronic survey are summarized in Table 1. Only eleven
tutees responded to the survey, so the data may not reflect the true feelings of all tutees
who receive tutoring from the center. Generally, students felt satisfied with the
instruction they received from the tutors and that they were helped with the assignment
they were seeking help with. Most students felt that the tutors were accessible, and
think they would return for more tutoring. Students only saw a moderate change in their
overall grades after receiving instruction from the tutors. Students are somewhat likely
5 of 12

to recommend the tutoring program to their friends. The highest percentage of students
surveyed go to the center once a week for tutoring. Most students would not pay for this
tutoring service if it were not free. The tutees in the survey were not very satisfied with
the group setting for receiving tutoring.

Overall the responses lead us to believe the tutoring program is functioning


efficiently and satisfying most of its clients. The lack students who would be willing to
pay for the service could mean a few things. The students may feel that the quality of
the instruction they receive at the tutoring center is not worth their money, or maybe
they simply would not spend their money on tutoring regardless of its effectiveness.

Most tutees surveyed did not particularly like the group setting for tutoring and
felt that one-on-one instruction would be more beneficial to their specific needs. We
received mostly positive feedback for question ten (Appendix D). The most notable
response was, Tutor did not seem to know answers and helped me with an online quiz
that I ended up getting a 60% on. This is only one persons experience, however it
does speak to the qualifications of the tutors. Another tutee suggested that it would
benefit the program to have tutors available not just for introduction level classes but for
higher level classes as well.
The tutors see a large influx in tutees coming in for help right before midterms
and finals. Tutors stressed in their interviews that coming in right before a huge
assignment or exam does not leave them much time to help the student to the degree
they would like to. They also feel that they could be more effective tutors if students
came prepared with specific questions to ask them. It was recommended that tutors
6 of 12

double check answers and facts that they provide to students, so as to not confuse or
mislead tutees. As was found with the tutees, the tutors are not huge fans of the group
tutoring stetting. They feel that they could be more focused and helpful if they were only
focused on one student/assignment at a time.

Discussion
Our group was disappointed in the low response rate to both our paper surveys
and our electronic survey. We attribute the low response rate of our student survey
(Appendix B) to two main factors. Firstly, most students are stressed out when they are
seeking tutoring and dont want to take the extra time to fill out a survey, so they simply
did not. Secondly, we feel that the way we administered the surveys was not the most
effective. Tutors were asking their tutees to fill out the surveys and leave the completed
ones with them. One can see how a student would be reluctant to state their feelings
about the tutoring they had just received (especially if negative), and hand it back to the
person they were talking about. If we had come up with a more discrete way to
administer and acquire the surveys, we think we could have achieved a higher response
rate. We did not receive much qualitative feedback from tutees as, we only obtained
one response to our student survey (Appendix B).

We found some limitations in using the free version of SurveyMonkey. We could


only formulate ten questions in the survey, which really limited us. We could only send
the survey to 100 people, so we were unable to sample the full population of tutees,
which was our initial plan. The last limitation we found with the free version, was that the
data could not be imported into Excel automatically; data had to be transposed by hand.
7 of 12

We do not think that this evaluation had enough participants to be of much help to
Janice Law. While we did pick up on a few trends, the small sample size makes the
results fairly unreliable. We maintained our integrity as program evaluators; we
remained unbiased as we conducted interviews and formulated our questions, and we
did not make up data to compensate for our low response rate.

Recommendations for Improvement


Future success of Academic Tutoring at The University of Tampa stands to
benefit from some of the recommendations received while conducting this evaluation.
Our recommendations are summarized in Table 2.

8 of 12

Table 1. Electronic Survey Results


Question

1.

Average Response
Likert Scale 1-5

2.3. How satisfied were you with the instruction you


received?

3.9

1.
How much did the tutoring help with the
assignment you were being tutored for?

2.
4.5. How accessible did you find the tutors?

3.64
4

3.
6.
4.
7.

Have you seen a positive change in your grades


since receiving the tutoring?
How often do you receive tutoring from the
center?

5.

8.9. How likely are you to return to the tutoring


center?

6.
10. How likely are you to recommend this service to a
friend?

7.
11. If this service were not free, how likely would you
be willing to pay for it?

8.
12. How helpful did you find the group setting for
tutoring?

9.
13. Is there anything that could have improved your
10.

tutoring experience? ALL comments are


appreciated!

= Positive Response
= Neutral Response
= Negative Response
9 of 12

3.45
36% receive tutoring
1/week
3.64
3.45
1.36
2.73
See Appendix D

Table 2. Recommendations

Setting
1. More one-on-one tutoring availability
If a student is consistently coming to tutoring prepared with questions
and not just showing up the day before assignments are due or exams
are scheduled, they should be able to set up time with a tutor for oneon-one instruction.
The group instruction setting can continue for walk ins.
2. More resources available at tutoring locations
Computers
Whiteboards

Tutoring
1. Offer help with higher level courses
2. Make sure tutors are actually qualified to be instructing in the area they are.
If a tutor is not sure about an answer they should not pretend to be.
If a tutor is consistently misleading tutees, the tutor should face
consequences
3. Better advertising of the services offered
Inform professors what tutoring is available so they can inform their
students.
Have Gateways instructors give out the tutoring schedule to all
freshmen and transfer students and the beginning of the semester.

4. Pay tutors for their time


This would make tutoring more appealing; more students would likely
want to be tutors and therefore more one-on-one instruction could occur
and instruction for higher level courses could be offered.
If tutors are being paid, the quality of the tutoring could improve, as
tutors would not want to lose their jobs for inadequate help.

10 of 12

Appendix A
Survey Monkey Survey Questions
14.
15. How
satisfied
were you
1.
with the
instruction
you
received?

1
Not
Satisfied

How much
did the
tutoring help
2.
1
with the
Not
assignment
Helpful
you were
being
tutored for?
16.
17. How
1
accessible
Not
did you find Accessible
3.
the tutors?
18. Have you
seen a
positive
4.
change in
your grades
since
receiving
the
tutoring?
19. How often
do you
receive
5.
tutoring
from the
center?

20.
21. How likely
are you to
return to the
6.
tutoring

1
No
Change

3
Moderately
Satisfied

5
Extremely
Satisfied

3
Moderately
Helpful

5
Extremely
Helpful

3
Moderately
Accessible

5
Extremely
Accessible

3
Moderate
Change

5
Positive
Change

1/Week

2/Week

3+/Week

1/Month

2/Month

1
Not Likely

3
Moderately

5
Extremely

11 of 12

Less
than
1/Month

center?

22. How likely


are you to
recommend
7.
this service
to a friend?

Likely

1
Not Likely

23. If this
service
were not
8.
1
2
free, how
Not Likely
likely would
you be
willing to
pay for it?
24. How helpful
did you find
the group
9.
1
2
setting for
Not
tutoring?
Helpful
25. Is there
anything
10. that could
Comment Box
have
improved
your
tutoring
experience?
ALL
comments
are
appreciated!

Likely

3
Moderately
Likely

5
Extremely
Likely

3
Moderately
Likely

5
Extremely
Likely

3
Moderately
Helpful

5
Extremely
Helpful

12 of 12

Appendix B
Student Survey Questions

1. Are you a Freshman, Sophomore, Junior, or Senior?


2. Is this your first visit to the Tutoring Center?

Yes

(Please circle one)


No

(Please circle one)

3. What is your major?


4. What subject were you tutored in?
5. How many hours did you spend in the tutoring center?
6. Were you comfortable with the group tutoring or would you have preferred oneon-one instruction? Please explain.
7. What did you like/dislike about the tutoring you received?
8. Is there anything you would change about the program? If so, what?
9. How were you introduced to the program?
10. Would you be willing to pay for this service? Yes
11. Would you refer this program to a friend?

Yes

No
No

12. Any additional thoughts or comments about the program?

13 of 12

(Please circle one)


(Please circle one)

Appendix C
Tutor Interview Questions

1. How long have you been tutoring?


2. What subject(s) are your specialty?
3. How many hours a week do you tutor students?
4. Would you recommend this program for other tutors?
5. Are you being paid to tutor students?
6. Do you mostly tutor during the day or night?
7. Based on your experience, how many of your students pass their classes?
8. What subjects do your students receive the most help?
9. How many students do you meet for tutoring?
Freshman?___Sophomore?____Junior?____Senior?____
10. Do you consider this program for other struggling students?
11. What recommendations would you have to improve the tutoring sessions with the
students?
12. What are some recommendations you have as a tutor?

14 of 12

Appendix D
Question #10 Responses - Is there anything that could have improved your tutoring
experience?

Respondent #

Response

Better tutors!

Moving all the tutoring locations to a building/area that provides


access to computers for the students needing help with online
homework.
No

Have one on one sessions

Tutor did not seem to know answers and helped me with an online
quiz that I ended up getting a 60% on.

Nothing! Examples were helpful

Get more well prepared students to do tutoring

Healthy snacks! Accessible white boards! More upper level class


tutoring available

More tutors should be available.

10

Keep up the great work

11

More attention from tutors to students.

15 of 12

You might also like