HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

HIV Prevention

Program Evaluation Plan

Prepared by:

Robert Avina
Kaitlin Brehaut
Abigail Lopez
Alexa Reyes

California State University, San Bernardino

June 11, 2017

1. INTRODUCTION AND STAKEHOLDER ENGAGEMENT

Evaluation Purpose

Page 1
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

The purpose of this evaluation is to assess the programs progress. Therefore,

findings of the evaluation will be used to identify any changes needed to improve the
overall program. As stated on the Student Health Center website, their main goal is to
provide health services to the students at California State University, San Bernardino
with the ultimate goal of promoting good health so that students can focus on academic
success. This evaluation aligns with the Student Health Centers goal by ensuring that
the HIV Prevention program is increasing knowledge and awareness about HIV, while
promoting the services provided at the Health Center to improve student health.

Stakeholders

Aside from the MPH grad program, the primary stakeholders of the HIV
Prevention program are the CSUSB Student Health Center and the CSUSB Department
of Health Science. All stakeholders want to ensure that the program is effective in
increasing awareness of HIV with the sole purpose of promoting health positivity among
the CSUSB community.The CSUSB Student Health Center will act as additional staff to
aide in the planning process. As the external reviewer, the CSUSB Department of
Health Science will oversee the entire HIV Prevention program.

Table 1. Stakeholder Assessment and Engagement Plan

Stakeholder Name Interest or Perspective Role in the Evaluation

CSUSB Student Health Center Staff Planning Team
CSUSB Department of HSCI Staff External Reviewer

Cultural Competence

In effort to engage our stakeholders, each stakeholder will have their own
individual role with the overall goal to reduce the transmission of HIV among CSUSB
students. To ensure the success of our program, the team will pilot test materials
provided, such as the ZINE, PSA, and pre/post tests, by handing out samples to each of
the stakeholders/intended audiences. This will allow us to indicate any small changes
that need to be made prior to the implementation of the program. Knowing the
demographics of the population including (age and race/ethnicity) can give us a better
understanding about the possible cultural barriers that can affect implementation.
Therefore, efforts can be tailored to address those barriers.

2. DESCRIPTION OF HIV PREVENTION PROGRAM ON INCREASING KNOWLEDGE

AND AWARENESS ABOUT HIV

Need
There is a need to reduce HIV transmission among CSUSB students to
encourage a healthier lifestyle rid of STIs.

Page 2
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

Context

Stigma and discrimination associated with HIV can affect the effectiveness and
implementation of the program. With that being said, providing accurate information on
HIV can lead to correcting common misconceptions, which would lead to reducing of
associated stigma and discrimination. In addition, as a result of the program, we want
students to gain confidence in being able to seek out services and further information.

Population Addressed
The intended audience for the HIV prevention program are students enrolled in
HSCI 404, HSCI 370, HSCI 271 and any other students attending the Second Annual
Run Like A Mother 5k in October 2017.

Stage of Development
The HIV prevention program has not been in place for very long. In fact, the
finalization of the program components (ZINE and PSA) were completed as of April
2017. Therefore, the program is still in the planning stage because it has yet to be
implemented to the target population.

Resources/Inputs
The available resources/inputs needed to support the program are staff/
volunteers, a location, deliverables (at least 200 copies of the ZINE, USB drive with
PSA, and any other informational handouts), and funds/donations which can include
incentives which will be given out at the event.

Activities
The first activity needed to be conducted to achieve program outcomes is to
distribute the ZINE and display the PSA to students who attend the Second Annual Run
Like a Mother 5k Run/Walk. Second is to promote the CSUSB Student Health Center on
their availability of condoms, lubrication, and STI testing.

Outputs
The outputs of the activities will result in students having the PSA and ZINE
readily available for them through social media outlets and at the Student Health
Center. This will allow students to go back and access the materials as deemed
necessary.

Outcomes
The intended outcomes of the HIV Prevention program include an increase in
knowledge about HIV and the resources available on campus or within the county. As
far as long-term outcomes, the HIV Prevention Program aims to reduce HIV mortality
and morbidity rates among CSUSB students. This can be achieved through increase in
condom usage, routine HIV testing, and safer sex methods, which can improve the
Page 3
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

overall quality of life. The time frame between the activities and the point to which we
being to see the ultimate outcomes, the HIV Prevention team will produce other
materials to post on social media outlets to continue raising awareness of HIV.

Logic Model
Logic model will be provided as an attachment to this document.

3. EVALUATION DESIGN
Stakeholder Needs
The findings of this evaluation will be used by the CSUSB Health Center in order
to create appropriate HIV intervention methods for CSUSB students. The evaluation will
inform the Health Center what the current CSUSB students know about HIV prevention
through an online pre and post test. The online pre test will be given to HSCI 404, HSCI
370, and HSCI 271 students prior to the 5k event. The post test will be given to
determine the effectiveness of the deliverables (PSA and ZINE). Test findings will be
utilized for statistical analysis to identify any gaps in our program, which will be used by
the Health Science Department. When referencing to what is credible information to our
stakeholders needs, would consist of having a pre test to gather evidence to create a
baseline and compare it to the post tests to see if HIV knowledge increased.
Furthermore, the HIV Prevention program team will encourage active involvement from
our stakeholders and will provide updates during the evaluation process.

Evaluation Questions
The evaluation will answer the following four questions:
1. How effective was our group in informing students about the HIV
resources on campus?
2. To what extent did the PSA and ZINE contribute to HIV reduction rates on
campus?
3. How useful was the PSA and ZINE in raising awareness of HIV?

Evaluation Design
The evaluation design uses an online pre and post test. A pre test will be
administered by providing a link to HSCI 404, HSCI 370, and HSCI 271 students a week
before the 5k event. The rationale behind this type of design is to determine what the
students know about HIV at the beginning of the event compared to how well the PSA
and ZINE serve to educate the students for the post test. The post test will evaluate
what remains unclear about the modes of transmission, the high, low, or no risks
associated with HIV, and their knowledge on condoms, lubricants, and PrEP.

4. GATHER CREDIBLE EVIDENCE

Data Collection Methods

Page 4
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

New data will be collected through the evaluation questions using an online pre
and post test. Both tests will be created using Survey Monkey and the link will be
provided to Dr.Verissimo to administer to her Health Science students. The readability
and reliability will be tested through the feedback given from the pilot test participants.
The validity will be tested when our results can be compared to the increase or
decrease in HIV rates at CSUSB for the year following the 5K. Cultural appropriateness
can be measured by including a section that ask for the students sex at birth, gender
identity, race, and sexual orientation. All data will be collected first hand by using Survey
Monkey and will be used for our program evaluation. Therefore, we do not plan to use
existing data, however, we may want to compare National HIV/AID Strategy Goals for
2020.

Table 3: Evaluation Questions and Associated Data Collection Methods

Evaluation Question Data Collection Method Source of Data
1. How effective was our group in Pre and post test CSUSB students
informing students about the HIV
resources on campus?
2. To what extent did the PSA and Measuring the number of CSUSB Health Center
ZINE contribute to HIV reduction rates students who got STD
on campus? tested after the event
through the Health Center
3. How useful was the PSA and ZINE Pre and post test CSUSB students
in raising awareness of HIV?

5. DATA ANALYSIS AND INTERPRETATION

Indicators and Standards

In order to measure our programs performance, an online pre-post test would be
used to evaluate CSUSB students HIV literacy. Once students take the pretest, it will
allow access to view the PSA; However, the ZINE will be available at the 5k event for
students to read. The pre/post tests will help us measure how knowledgeable students
were about HIV prior to the run and if they learned something new after seeing the PSA
and reading the ZINE. Success for our group would be that 80% of the students who
viewed the PSA and read the ZINE would become more HIV literate.

Table 4. Indicators and Success

Evaluation Question
1. How effective was our group
Page 5
Standards
Criteria or Indicator (What Constitutes
Success?)
Referred (How did you Based on the number of
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

in informing students about the hear about our attendees, we would

HIV resources on campus?
2. To what extent did the PSA
and ZINE contribute to HIV
reduction rates on campus?
services) want 25% of the students
we reached out to go to
the Student Health
Center to seek out
services.
Students seek HIV 20% of students we
screenings on campus reached out to will get
or through the county HIV tested.
Identifying what type of Practicing safer sex by
condoms are more using latex, polyisoprene,
effective in preventing and polyurethane
an HIV infection. condoms.

3. Howuseful was the PSA Identifying the 5 ways Successfully identifying

and ZINE in raising HIV HIV can be transmitted. all modes of
awareness ? transmission.

Analysis
For our Pre-post study we will be running descriptive statistics, and ANOVA at a
significance established p value lower than .05. The software of our choice to run the
data would be SPSS version 24. In doing so, it will allow us to identify the gaps of our
program and any further needs of our population.

Interpretation
Individuals that would be considered so far in drawing, interpreting and justifying
conclusions will be Alexa Reyes, Robert Avina, Abigail Lopez, and Kaitlin Brehaut in
collaboration with Dr. Verissimo, the primary organizer of the Run Like A Mother 5K
Run/Walk and Dr. Becerra, the MPH Coordinator. Dr. Verissimo will help our team in
administering the pre and post tests to her Health Science students, while Dr. Becerra
will oversee the overall project.

6. USE AND COMMUNICATION OF EVALUATION FINDINGS Use

Evaluation findings will be used by Robert Avina, Abigail Lopez, Alexa Reyes,
and Kaitlin Brehaut for data analysis purposes. The findings will help identify our
programs strengths, weaknesses, and opportunities for improvement. The data will also
be used to tailor our program to the communities needs to improve our programs
overall quality and outreach. As intended, the pilot testing will occur during the summer
2017 HSCI 120 labs to gain feedback before the actual program implementation.
Abigail, Alexa, Kaitlin, and Robert have an equal understanding of the implementation
process and will provide input during that process to have an accurate/quality program.
The follow-up would consist of bi-weekly meetings to ensure that the program is
meeting its deadlines. The team will document all changes made throughout the
program which will be kept in a USB drive.

Page 6
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

Communication
We will communicate with the Health Science Department, the external viewers
of the program, the CSUSB Student Health Center, who will be a part of our planning
team. We would hold a monthly meeting to evaluate the status of the program, along
with, resolving any issues or findings that we might come across. Stakeholders will be
contacted through email informing them about the program mission and vision. After
conducting our study it is important to keep stakeholders in the loop and we will do so
by setting up in-person meetings/ presentations and provide them with written reports.
This will give them an idea on how resources are being used as well as provide
information on the impact the program is having in the community. These methods are
appropriate because they engage our stakeholders throughout the evaluation process
since we want to ensure we build a strong relationship with the stakeholders.
7. EVALUATION MANAGEMENT
Evaluation Team

The managing and the implementation of this evaluation with be done in a team
effort. In order for this evaluation to be successful, the proper goals and objectives will
need to be refined to ensure the accuracy and quality of the program. This would
include details on how the inputs were successful and what outcomes and impacts were
reached during this program. The best approach for this program would be the pre and
post test to assess the knowledge of the student body. Our external reviewer would be
the Health Science Department, which is likely to be Dr.Becerra.

Table 5. Roles and Responsibilities of the Evaluation Team Members

Individual Title or Role Responsibilities
Robert Avina Program Evaluation The role of this position is to ensure
Coordinator that the evaluation program runs
smoothly, which consist of working
with student engagement, the
evaluation designer, and the data
collector in order for a successful
program.
Abigail Lopez Student Engagement The role of this position is to
Coordinator develop student relationships, and
provide accurate information to the
student body. This position will help
increase the number of student
participation on campus.
Kaitlin Brehaut Evaluation Designer The role of this position entails the
proper evaluation of the program,
along with other details to provide
the continuation of the program.
This position will work with the
Program Evaluation Coordinator to
execute a successful program.
Page 7
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

Alexa Reyes Data The role of this position is to collect

Collection/Analysis data and analysis data in order to
ensure that the grants/funding
continue.

Data Collection Management

Referring back to table 3, the evaluation questions we will be assessing are:
o 1. The effectiveness of informing students about HIV resources
on campus by using Pre/Post Tests.
o 2. The number of students who received an HIV test at the
CSUSB Health Center after the event.
o 3. The online Pre/Post test based on HIV transmission after the
PSA and ZINE have been presented.
The activities that are needed to carry out the data collection successfully are an
online pre/post test before and after the PSA and ZINE have been presented.The pre
test will be administered to the HSCI 404, HSCI 370, and HSCI 270 students with the
help of Dr.Verissimo to hold them accountable. The post test link will be provided at the
end of the event. The primary data collection personnel will be Alexa, but the team will
assist if needed. Robert Avina, the Program Evaluation Coordinator will work with the
team to ensure that the program meets its outcome. At anytime, the team can step in
and voice their perspective about the program to ensure the accuracy and the quality of
the program.

Table 6. Data Collection Plan

Evaluation Data Collection Activities Person(s)
Due Date
Question Method Needed Responsible
1.How effective Pre and post test Deliverables Robert Avina Oct 21st,
was our group in from the and Alexa 2017
informing students Student Reyes
about the HIV Health Center
resources on
campus?
2.To what extent Measuring the Keep a tally of Kaitlin Oct 21st,
did the PSA and number of how many Brehaut and 2017
ZINE contribute to students who got people get Robert Avina
HIV reduction rates an HIV test after tested (No
on campus? the event through names
the Health Center involved)
3.How useful was Pre and post test A Abigail Lopez Oct 21st,
the PSA and ZINE questionnaire 2017
in raising before and
awareness of HIV? after the event

Data Analysis Management and Interpretation

Page 8
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

Data security will be ensured by only allowing the group members to count and
review the information. NO personal information will be collected (ie; names, home
addresses, or emails). The data that will be analyzed from the pre and post test (please
refer back to step 5 for details). Alexa Reyes is responsible for conducting the analysis
and gathering external or internal help to collect the data for the program. Data will
presented to the stakeholders to inform them about our current analysis process. This
also includes information based on the surveys to create a baseline to see if there is a
need for HIV awareness. (ie. education, services, and prevention.) We will also allow
our stakeholders to provide assistance and feedback on analysis interpretation and
results.

Table 7. Data Analysis Plan

Data to Be Person(s)
Analysis to Be Performed Due Date
Analyzed Responsible
Demographics (Be part of the Age, Race, and Alexa, Abigail, End of Fall 2017
Pre test) Sexual orientation Kaitlin, Robert
Pre/Post Test Comparison of the Alexa, Abigail, End of Fall 2017
Pre/Post answers Kaitlin, Robert

Communicating and Reporting Management

Our intended audience is HSCI 404, HSCI 370, HSCI 271, and any others
attending the 5k event. The purpose of the communication efforts with our intended
audience is to receive their feedback based on the program to implement any changes
if needed. The most appropriate type of communication method to use our social media
outlets to allow students to provide constructive feedback to improve questions,
program, and overall quality/content. Additionally, the post test can include a section for
feedback and comments. Communication will take place during the pilot testing session,
during the planning/implementation process, and after the implementation of the
program.

Timeline
Planning and administrative tasks will occur from Spring 2017- Fall 2017. The
training for data will occur three weeks before October 21st, 2017. Pilot testing will take
place during summer 2017 during the HSCI 120 Labs. The data collection based on the
pre test will be administered one week before October 21st, 2017, the post test and
analysis will be conducted one week after October 21st, 2017, and the interpretation will
be completed by the end of November 2017. Information dissemination tasks will occur
by the end of Fall 2017 before Winter break.There should not be any issues with the
information unless the data collection is not conducted as planned (ie. not having
enough pre/post tests that match up, or not everyone participating in the tests)

Wrapping Up

At the end of the evaluation, everyone who contributed to the implementation of

the program will be acknowledged by receiving a certificate by the Health Science
Department. Evaluation lessons will be documented by comparing our theoretical goals,
Page 9
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

objectives, and intended outcomes provided in the logical model to the overall outcomes
that were presented after the program. This would include the documentation of
feedback that was provided by the community that attended the event or viewed our
material that was posted on our social media outlets and the changes made throughout
the entire process.The team will utilize the Health Science Departments computers
however, all documents, instruments, and data will be kept in USB drives for each
person on the team and one will be provided to the HSCI Department.

Page 10