Professional Documents
Culture Documents
Conceptualization of Evaluation
The University of North Carolina at Greensboro Department of Recreation and Wellness has
requested that their mental health and suicide awareness initiative, Pinwheels for Prevention, be
evaluated. The broad question we hope to answer through our evaluation is: Does the program Pinwheels
for Prevention contribute to mental health awareness? The primary purpose of the evaluation is to
determine the overall impact of this initiative based on the project objectives and whether or not this
As the evaluation is created and implemented, it is important to keep the key stakeholders
involved. The primary stakeholders are the students at UNCG. The evaluation will help to determine if
the Pinwheel display is impacting students in a positive way. Pinwheels for Prevention is located centrally
on campus, therefore, UNCG faculty and staff are also stakeholders who may be impacted by the display.
The UNCG Department of Recreation and Wellness is an additional stakeholder because they are
financially responsible for ensuring the display is installed every September. Another group of
stakeholders to consider are the members of the SPARTA Peer Health Education Program. These students
invest their time to manage the display year to year, and rely on it to connect with their peers. Finally, the
Counseling Center, housed in Student Health Services, is a stakeholder. This vested body of counselors
and staff serve as a critical resource for the mental health and well-being of UNCG students. If the
evaluation shows a positive and lasting impact, the Counseling Center may notice an increase in students
There are a series of contextual factors associated with completing this evaluation. First, we must
consider the pre-existing stigma surrounding mental health on the UNCG campus at large. Another
contextual factor to consider are college student’s perceived seriousness of mental health issues. If
students do not view mental health concerns as important, they may be unwilling or hesitant to seek
mental health services. Also, the peer health educators who are conducting the surveys will need to be
1
Through this evaluation process, certain assumptions are being made. First, we are making the
assumption that the students, faculty, and staff who are given the survey are a representative sample of the
campus as a whole. Second, by administering the survey where the pinwheel display is installed, we
assume that this will provide a high response rate and therefore, a large sample size. Additionally, we are
assuming that five days is enough time to gather data. Another assumption being made by our evaluation
team is that the students who take the survey can speak and write in English (unless we provide a
translated version). Our evaluation team is also assuming that participants will answer the survey
questions truthfully. Our final assumption is that students will care enough about the display to stop and
complete a survey.
Program Description
There are two primary purposes of the Pinwheels for Prevention initiative: To reduce the stigma
surrounding mental illness and treatment and to promote help-seeking behavior. This is accomplished
through a display that represents the 1,100 college students lost to suicide each year. It is meant to be
implemented on the grounds of the UNCG campus during Mental Health Awareness Month, which is in
September. According to Jamie Stephens, the Coordinator for Peer Education and Outreach, the target
audience is the entire Greensboro community and specifically the students, faculty, and staff at UNCG.
However, Jamie hopes that first year students, or students new to campus, are particularly impacted by the
Pinwheels for Prevention. The intention behind this focus on first year and new students is to normalize
discussing mental health from the very start of their college experience.
The Pinwheels for Prevention initiative operates under a primary conceptual model (Figure 1)
that is imbedded in PRECEDE-PROCEED. Ultimately, the target is to improve mental health, but there
are several behavioral and environmental factors that can impede this goal. The Pinwheels for Prevention
initiative was created to target the predisposing and reinforcing factors that affect mental health by
increasing awareness and hope, which theoretically leads to a reduction of the stigma associated with
mental illness. With less stigma surrounding this topic, the hope is an increase in help seeking behavior.
When students are able to seek help, this creates improvements in their overall mental health.
2
In order to reach the objectives set forth in the logic model (Figure 2), the Pinwheels for
Prevention initiative requires several critical components. First, because the display requires the
purchasing of pinwheels and the labor from peer health educators, funding must be available from the
Department of Recreation and Wellness. The pinwheels must be purchased prior to the event so that all
supplies are ready for installation. Also, the space outside of the Elliott University Center (EUC) must be
reserved. The Peer Health Education group is likely the most critical component of the initiative. These
students help lay down the grid lines for the display the day before set up and count and box up all of the
pinwheels and transport them to the EUC lawn. These students need a thorough understanding of what the
event is about, an elevator pitch that articulates what it stands for, and knowledge of appropriate
The Pinwheels for Prevention program has undergone previous evaluation. In 2013, a total of 81
responses were gathered from the prompt, ‘the pinwheels on the lawn represented...’. This was collected
in qualitative short answer form. Responses ranged from no understanding at all, to an accurate
descriptive statement. The initiative was also evaluated in 2015 and 2016. These evaluations did not help
us in terms of collecting data on help seeking behavior. However, the 2013 evaluation that asked about
representation did give us an idea of how many students understood the purpose of the display.
Evaluation Plan
In order to evaluate the Pinwheels for Prevention program efficiently, our evaluation team will
utilize a cross-sectional descriptive design which will obtain information from two moments in time. The
primary survey will be conducted one week after the display is set up on the EUC lawn. The secondary
survey will be conducted via email six months after the primary survey was conducted. One benefit of
using a cross-sectional design is that it is a relatively quick way to collect information. The following two
summative evaluation questions asked in the two surveys will help us better understand whether or not
3
● To what extent did the Pinwheels for Prevention reduce stigma of mental illness and treatment
among UNCG students, faculty, and staff?
● To what extent did the Pinwheels for Prevention increase help seeking behavior among UNCG
students, faculty, and staff?
The evaluations will gather information from the UNCG community using a modified version of
the General Help-Seeking Questionnaire (GHSQ) and a modified version of the Healthy Minds stigma
scale. Both of these utilize the Likert scale for collection of quantitative data. Only the first survey will
ask participants if they are willing to answer a brief follow-up survey. Their email address will be
recorded if they agree. Our evaluation team will propose the survey questions to the following
stakeholders: the Department of Recreation and Wellness, the Counseling Center, and SPARTA Peer
Health Educators.
The rationale for conducting the first survey one week post pinwheel set up is so that we increase
the chances that more people have seen the display. The first survey is to gather data on short-term
outcomes such as knowledge, attitudes and beliefs. The secondary survey, sent out six months after the
display has been taken down, is to gauge the lasting impact of Pinwheels for Prevention. By conducting
the second survey six months later we are making the assumption that knowledge, attitudes and beliefs
can be affected/changed over a period of time. We assume that participants will remember what the
initiative was about. We also assume that participants will care enough to complete a follow up survey.
We may need to consider an incentive to gather more responses. Our evaluation team believes that this
plan will be sufficient in collecting the necessary data to determine if stigma and help seeking behavior
4
Figure 2: Logic Model
Reduce stigma of Peer health Create budget Display Increased Improved mental
mental illness and educators exposure of awareness health
treatment Marketing students,
Volunteers faculty & staff Sought
Promote help- Buy materials professional care
seeking behavior Professional
staff Organize display Discussion of
mental health
Monetary Put up pinwheels with others
resources
Materials
Allocated time
To what extent did the Pinwheels for Prevention reduce stigma of mental illness and treatment among
UNCG students, faculty and staff?
Survey Question 1:
(Likert scale: strongly agree, agree, somewhat agree, somewhat disagree, disagree, strongly disagree)
After viewing the Pinwheels for Prevention Initiative…
1. I would think less of myself if I received mental health treatment.
2. I would think less of a person if they received mental health treatment.
3. Most people think less of a person who has received mental health treatment.
(Healthy minds network, 2017)
To what extent did the Pinwheels for Prevention increase help seeking behavior among UNCG students,
staff and faculty?
Survey Question 2:
(General Help-Seeking Questionnaire: Not likely to extremely likely)
After viewing the Pinwheels for Prevention Initiative…
5
1. If you were having a personal or emotional problem, how likely is it that you would seek help
from a friend or family member, or professional.
2. If you were experiencing suicidal thoughts, how likely is it that you would seek help from a
friend or family member, or professional.
(Wilson et al., 2005)
Reporting Plan
The purpose of reporting our evaluation findings is to indicate the effectiveness of the Pinwheels
for Prevention program. In our evaluation report we will show the degree in which the program reduced
stigma of mental illness and treatment among UNCG students, faculty, and staff. The report will also
demonstrate how likely individuals were to engage in help-seeking behaviors as a result of the program.
It is critical for the evaluation findings to be accurate and fair. Therefore, the evaluation results
will be reviewed with stakeholders. The Pinwheels for Prevention program evaluation team will report
evaluation findings to the following three stakeholders: the Department of Recreation and Wellness, the
Counseling Center, and SPARTA peer health educators. Determining the stakeholder’s values, interests
and potential questions will help inform how and what our evaluation team will report from the
evaluation. Our evaluation team will ask each stakeholder what they would like to know in advance. This
understanding can inform our reporting information for the stakeholders. When preparing evaluation
reports we will make sure to consider the rights, privacy and feelings of those involved in the evaluation.
Ensuring that we respect all parties involved can prevent unwanted ethical breaches in reporting.
In our interview with Jamie Stephens, the program coordinator, she expressed her hope that other
college campuses would adopt the Pinwheels for Prevention initiative. Therefore, our most detailed
evaluation report will be written in a format suitable for submission to an academic journal. This fifteen
page detailed report will be a useful reference for other universities interested in implementing a similar
mental health program. The first stakeholder we will report our findings to is the Department of
Recreation and Wellness as they coordinate the Pinwheels for Prevention program. We will provide them
with a comprehensive one-page executive summary. It will be easier to understand than the detailed
report as it will have less technical jargon. It also will include a brief description of the program and
6
methodology of data collection. The executive summary will include only the most important findings,
limitations and recommendations. Along with providing the executive summary, we plan to give a brief
Powerpoint presentation to the Department of Recreation and Wellness. It will include charts, graphs and
other engaging visuals to keep the audience’s attention. If the Department of Recreation and Wellness
would like to have the evaluation’s purpose, data collection methods, and interpretation for future use, it
Counseling services and the SPARTA Peer Health Educators will be given basic evaluation
results regarding whether or not the program was successful in stigma reduction and help-seeking
behavior. For these two stakeholders, we plan to deliver the main points of our findings through an
infographic. An infographic is a creative way to present findings and visuals they may want to use in the
future. It is also an effective method for presenting to SPARTA students and counseling staff whose time
is valuable and limited. If either of these stakeholders indicate the desire for more extensive information,
we will provide them with the executive summary as well. After reporting and receiving feedback from
stakeholders, our evaluation team would like to submit the evaluation findings to scholarly journals.
Publishing our findings will add to the literature on university mental health programs. Furthermore, it
may be an informative resource for other college campuses that would like to implement a similar mental
health initiative.
Budget Justification
A. Personnel
Evaluator/Primary Investigator - The primary investigator is responsible for ensuring that the surveys are
revised for the distribution year. This person is responsible for reviewing, saving and inputting data. They
are also responsible for sending out the six month follow up to patrons who supplied their email. The
evaluator will compile and compare data, create, report and distribute the final report.
responsible for ensuring that the surveys are functioning prior to evaluation. They will contact the primary
investigator if any problems need resolution. They are responsible for managing and scheduling the peer
7
health educators on the day of data collection. They are also responsible for obtaining the iPads from the
library. To quality control the peer health educators, this person will conduct two informational sessions
B. Additional Personnel
Peer Health Educators (5) - These individuals will be responsible for distributing the surveys during the
week of data collection. They serve as the primary data collection distributors. They will report directly to
the wellness graduate assistant with any concerns or changes in their schedule. They are also responsible
C. Equipment
Five iPads from the UNCG library are needed for the day of data collection. This is the primary tool for
evaluation and collection of data to be later analyzed. There is no cost for the use of this equipment (<
$5,000).
D. Travel
The primary investigator’s destination is the UNCG campus located in Greensboro North Carolina. There
will be one person traveling in August. They will be given three days allowance for travel. For the sake of
this evaluation project, we hypothetically calculated the evaluator having to travel to and from Raleigh.
We took into account that the evaluator will need time to present findings to three different sets of
stakeholders.
Primary Evaluator ● Compile and compare data (3 months allowance, paid 45 work days) 22,500
(Approx $500 per day) ● SPSS 75
● Create Report approx (2 months allowance, paid 20 work days) 10,000
● Present & Distribute Report (3 days) 1,500
● Accommodations (3 nights at $175) 525
8
Travel Cost [mileage] ● 1 Round trip (Greensboro ⇆ Raleigh) ≅ 76 miles, 2.51 per gallon. 30
Projected Schedule
The projected time frame for this evaluation will be from September of 2018 to July of 2019. The
evaluation will start during the same month that the Pinwheels for Prevention display is installed. The
data collected during the evaluation week will be compiled and input into SPSS during the month of
October. Six months after the initial survey, in March of 2019, the second survey will be provided to
individuals who agreed to participate. There will then be two months allotted for adding this data to the
previously compiled data for comparison. The evaluation team should be prepared to begin writing a
formal report during this second month of data analysis, but will have through the month of June to
complete the report. Once the report is complete, the evaluation team will present the report to the key
stakeholders mentioned above. After reporting and receiving feedback from the evaluations, the report
will be submitted for publication in peer reviewed journals. Completion of the evaluation project in July
of 2019 will provide ample opportunity for the Pinwheels for Prevention to be changed or altered if
needed before implementation in September of 2019. A table of this schedule can be viewed below
(Figure 3).
Figure 3: Schedule
9
References
Wilson, C. J., Deane, F. P., Ciarroche, J. V., & Rickwood, D. (2005). General Help-seeking
Healthy Minds Network. (2017). Perceived public stigma versus personal stigma. Retrieved from
http://healthymindsnetwork.org/research/data-point-of-the-month.
10