Clinical Simulation in Nursing (2016) 12, 2-7

www.elsevier.com/locate/ecsn

Featured Article

After the National Council of State Boards of

Nursing Simulation StudydRecommendations and
Next Steps
Tonya Rutherford-Hemming, EdD, RN, ANP-BC, CHSEa,*,
Lori Lioce, DNP, FNP-BC, CHSE, FAANPb,
Suzan Suzie Kardong-Edgren, PhD, RN, ANEF, CHSE, FAANc,
Pamela R. Jeffries, PhD, RN, FAAN, ANEFd,
Barbara Sittner, PhD, RN, APRN-CNS, ANEFe
a

Senior Nurse Researcher, Cleveland Clinic, Cleveland, OH 44195, USA

Clinical Associate Professor/Executive Director, Learning and Technology Resource Center, The University of Alabama in
Huntsville College of Nursing, Huntsville, AL 35805, USA
c
Professor and RISE Center Director, School of Nursing and Health Sciences, Robert Morris University, Moon Township, PA
15108-1189, USA
d
Dean and Professor of Nursing, George Washington University, School of Nursing, Washington, DC 20036, USA
e
Professor, College of Nursing, Bryan College of Health Sciences, Lincoln, NE 68506-1398, USA
b

KEYWORDS
simulation;
faculty development;
NCSBN Simulation
Study

Abstract
Background: The National Council State Boards of Nursing (NCSBN) Simulation Study has generated
increased conversation about the use of simulation in nursing education.
Method: At the 14th Annual International Nursing Association for Clinical Simulation and Learning
(INACSL) conference in Atlanta Georgia, a panel discussed the results and significance of the National
Council of State Boards of Nursing (NCSBN) Simulation Study.
Results: Panel members discussed movements in nursing education in the eight months since the
studys release, implementation of the recommendations from the study in practice and academic
settings, and methods to achieve the necessary faculty development needed in simulation.
Conclusion: The use of simulation in nursing education is expanding.
Cite this article:
Rutherford-Hemming, T., Lioce, L., Kardong-Edgren, S. S., Jeffries, P. R., & Sittner, B. (2016, January).
After the National Council of State Boards of Nursing Simulation StudydRecommendations and Next
Steps. Clinical Simulation in Nursing, 12(1), 2-7. http://dx.doi.org/10.1016/j.ecns.2015.10.010.
2016 International Nursing Association for Clinical Simulation and Learning. Published by Elsevier
Inc. All rights reserved.

* Corresponding author: aUNCheel@gmail.com (T. Rutherford-Hemming).

In fall 2014, findings from the National Council State

Boards of Nursing (NCSBN) Simulation Study were released
(Hayden, Alexander, Smiley, Kardong-Edgren, & Jeffries,

1876-1399/$ - see front matter 2016 International Nursing Association for Clinical Simulation and Learning. Published by Elsevier Inc. All rights reserved.

http://dx.doi.org/10.1016/j.ecns.2015.10.010

After the National Council State Boards of Nursing Simulation Study

2014). The NCSBN study sought to provide evidence to US
boards of nursing regarding the use of simulation as a replacement for traditional clinical experiences in prelicensure
nursing education. The study aimed to determine (a) whether
simulation could be substituted for traditional clinical hours,
(b) the educational outcomes
of undergraduate nursing
Key Points
students in the core clinical

The NCSBN Simulacourses when simulation
tion Study provided
was integrated throughout
evidence on the use
the core nursing curriculum,
of simulation.
and (c) whether varying

Faculty development
levels of simulation in the
is an area of need in
undergraduate curriculum
nursing education.
impacted the practice of

A resource is the Stannew graduate nurses in their
dards of Best Practice
first
clinical
positions
in Simulation.
(Hayden et al., 2014).
At the 14th Annual International Nursing Association for Clinical Simulation and Learning (INACSL)
conference in Atlanta Georgia, a panel was formed to
discuss the results and significance of the NCSBN study
(Hayden et al., 2014), movements in nursing education in
the 8 months since the studys release, implementation of
the recommendations from the study in practice and
academic settings, and methods to achieve the necessary
faculty development needed in simulation. The session
concluded with open dialogue between panel members
and conference attendees discussing relevant simulation
topics such as simulation ratios and time, faculty development, debriefing practices, and future endeavors.

Plenary Session During the International

Nursing Association for Clinical Simulation
and Learning Conference, 2015
The following are excerpts from the panel of experts who
presented at the conference.

Suzie Kardong-Edgren, PhD, RN, ANEF, CHSE,

Director, RISE Center and Professor, School of
Nursing, Robert Morris University
The landmark NCSBN Simulation Study provided evidence that up to 50% of traditional hours in the major
clinical courses in prelicensure nursing programs could be
safely substituted with simulation (Hayden et al., 2014).
This opened the door for further discussion about the
use of simulation in prelicensure programs; however, the
devil is in the details. Many nursing programs may be
tempted to gloss over the elements that produced those results. All study faculty shared the same mental model of
how simulation would be run for the study. This began

with the initial orientation, debriefing training, and

ongoing evaluation of the dedicated simulation faculty,
over the life of the study. The INACSL Standards of
Best Practice: SimulationSM (2013) guidelines for orientation, facilitation, debriefing, and evaluation were used to
guide and standardize simulation practice across all sites.
A high-level Socratic debriefing method was used to
develop self-reflection skills. These key elements were
all standardized and controlled at all sites and used
best-known practices for the use of simulation. Very few
United States programs have the trained faculty and
standardization to provide the same level of simulation
used in the study. These realities must be addressed
when schools wish to adopt high levels of simulation
within a nursing program.

Pam Jeffries, PhD, RN, FAAN, ANEF, Dean and

Professor, George Washington University School of
Nursing
Implications from the NCSBN study (Hayden et al., 2014)
call for faculty to be trained in using simulation pedagogy.
Faculty development in designing, implementing, and evaluating clinical simulations still remains a major concern in
nursing education. It may be that a shift from training all
faculty to do simulations to a well-prepared simulation
team is needed. A key element in implementing the
simulations in the landmark NCSBN study (Hayden
et al., 2014) included faculty development and preparation.
Resources to facilitate faculty development and to ensure
quality simulations are being developed and delivered
include the use of the Standards of Best Practice in
Simulation (Decker et al., 2015; International Nursing
Association of Clinical Simulation and Learning, 2013;
Lioce et al., 2015) and becoming a Certified Healthcare
Simulation Educator (CHSE) (Society for Simulation in
Healthcare [SSH], 2014). Both are available benchmarks
for faculty preparation and credentials.
The NCSBN study tested an integrated, sustainable
simulation model across seven clinical courses (Hayden
et al., 2014). Hayden (2010) reported that 87% of Schools
of Nursing in the United States included simulations in
their nursing programs. Curriculum integration should
build on/or be used to fill-in the gaps within nursing
programs. Simulation directors and simulation teams need
to work with clinical coordinators to integrate both if
resources and support are available.
The NCSBN study (Hayden et al., 2014) provided evidence that simulations work. Research is needed in other
areas to advance the science in simulation in the United
States and internationally. Research in simulation remains
embryonic; therefore, many research questions and topics
remain available to explore and embrace (Agency for
Healthcare Research and Quality, 2015; McGaghie,
Issenberg, Petrusa, & Scalese, 2010). Funding sources,
pp 2-7
Clinical Simulation in Nursing
Volume 12
Issue 1

After the National Council State Boards of Nursing Simulation Study

such as the Agency for Healthcare Research and Quality,
are available for studies investigating outcome measures,
safety, and quality health care environments in simulation
(Agency for Healthcare Research and Quality, 2013).
The National League for Nursing (NLN) Jeffries
Simulation Framework (Jeffries & Rogers, 2012) is moving
to a mid-range theory for simulation. The new theory, based
on the NLN Jeffries framework (Jeffries & Rogers, 2012),
will be unveiled at the NLN Teaching Summit in
September 2015 in addition to being featured in the special
simulation issue of the NLN journal, Nursing Education
Perspectives in the September/October issue. The
monograph and new mid-range theory will hopefully help
to facilitate more theoretical-based research in the area of
clinical simulations and provide a direction for simulation
research.
The NCSBN Simulation Study is a landmark study that
provides needed evidence for this type of pedagogy used in
nursing education (Hayden et al., 2014). The results are
generalizable across the United States based on the study
of 10 different nursing schools that included schools n
the urban/rural settings with the study expanding across
both associate and baccalaureate degree prelicensure
nursing programs. More data/metrics on learning outcomes
using simulations are needed, so data-driven decisions can
be made to improve learner preparation, practice, and
delivery of patient care using a simulation-based
curriculum. A next step is to promote the value of
simulation by translating the importance of using clinical
simulation in education to provide better, quality care,
and patient outcomes.

Tonya Rutherford-Hemming, EdD, RN, ANP-BC,

CHSE, Senior Nurse Researcher, Cleveland Clinic
Nursing education does not cease at the end of graduate
studies. Education continues as the clinical nurse practices
within the health care setting. Similar to the increased
propensity to use simulation in academia, simulation in
practice environments has increased in the last decade. A
unique aspect of the NCSBN study (Hayden et al., 2014) is
that it followed the prelicensure nursing students
postgraduationdsomething, few, if any, studies have done.
The Cleveland Clinic, like many institutions, uses
simulation for many professionals including nurses,
physicians, dieticians, pharmacists, and so forth. Simulation is used as part of onboarding and competency testing.
It is also used to enhance procedure performance, improve
teamwork, and develop communication skills. Cleveland
Clinic has dedicated simulation faculty in the multidisciplinary simulation center who are trained in simulation
pedagogy. Staff and faculty who want to teach using
simulation attend mandatory training courses. The training,
based on the Standards of Best Practice: SimulationSM
(International Nursing Association of Clinical Simulation

Table 1

4
Items to Measure Outcomes

Measurements
1. Identify and/or hire innovators/early adopters.
2. Consider workload credit for those involved in program
development. Complete a facilitator needs assessment.
3. Select a theoretical framework for your simulation education
program and review it every 3 years.
4. Collaborate and select policy and procedures to provide
transparency and standardize processes.
5. Select a reservation request forms, design template, and
evaluation method; and use a simulation objective map. This
will decrease the amount of e-mails to coordinate and
prepare simulations.
6. Collect usage data to provide evidence for estimating cost
per simulation, course, and for the program as well as
support consistent staffing.
7. Require faculty/staff training in facilitation/debriefing
before participation in simulation and offer additional
training at the beginning or end of each semester.
8. Require dry-runs before all new scenario implementation
with facilitators. This time allows facilitators to identify
content and objective cues and set performance measures
and set mutual expectations.
9. Record an orientation to the center, specifics on the
simulation process, and specific equipment that will be used
for the day to ensure standardization. Play the recording
during prebriefing.
10. Offer annual open houses for simulation equipment
demonstration and coordinate team planning time.

and Learning, 2013), includes sessions on simulation

terminology, facilitation, and assessment/evaluation. A
half-day interactive session is focused on debriefing.

Lori Lioce, DNP, FNP-BC, NP-C, CHSE, FAANP,

Executive Director, Learning and Technology
Resource Center, University of Alabama in
Huntsville School of Nursing
Implications from the NCSBN study support and require
changes in daily operations and structure of health care
simulation education programs. Table 1 offers ten items for
consideration by administrators, directors, coordinators,
and facilitators to improve, standardize, and assist in
measuring outcomes.
It is often said, we tend to teach as were taught. This
saying offers an explanation for why formal educational
systems around the world find change difficult to
implement and even harder to measure. The traditions
associated with the education of health care professionals
are extremely difficult to change. This is especially true
when we have new technologies and approaches for
teaching that are improving educational effectiveness but
also for protecting patients and improving the care we
provide.
pp 2-7
Clinical Simulation in Nursing
Volume 12
Issue 1

After the National Council State Boards of Nursing Simulation Study

Table 2

Simulation Programs

Simulation Programs

Web sites

Boise State University

Bryan College of Health Sciences

http://hs.boisestate.edu/nursing/sgcp/
http://www.bryanhealthcollege.edu/bcohs/academic-programs/certificate-programs/simulat
ion-certificate/
http://www.drexel.edu/cnhp/academics/continuing-education/Nursing-CE-Programs/Certific
ate-in-Simulation/
http://admissions.rmu.edu/online/nursing-and-health-care/simulation-instruction
https://www.usfca.edu/nursing/msim/
https://www.usi.edu/health/certificate-programs/clinical-simulation-certificate-program

Drexel University College of Nursing

and Health Professions
Robert Morris University
University of San Francisco
University of Southern Indiana

Barbara Sittner, PhD, RN, APRN-CNS, ANEF,

Professor, Bryan College of Health Sciences
As the science of simulation continues to expand into
academic, clinical, and research settings, so does the need
for continuing education on this teaching strategy. There
are several simulation programs in the United States to
address this need and prepare individuals for Society for
Simulation in Healthcare certifications (Table 2). It is
important that these programs incorporate the standards
into their curriculum and remain up to date on revisions
and the addition of new Standards. Since the inception of
the INACSL Standards in 2011, the Standards were revised
in 2013 and presently being reviewed by the INACSL
Standards Committee. The 2015 conference was a great
example of our Standards as living documents as INACSL
unveils two new standardsdStandards of Best Practice:
Simulation Standard VIII: Simulation-Enhanced Interprofessional Education (Decker et al., 2015) and Standards
of Best Practice: Simulation Standard IX: Simulation
Design (Lioce et al., 2015).

Audience Participation
When the panel speakers concluded, discussion opened to
the audience. In general, four themes emerged from
audience questions: (a) simulation ratios and time, (b)
faculty development, (c) debriefing practices, and (d) future
of simulation.
An initial question surfaced related to the ratio of
simulation time to traditional clinical time being used.
Currently, there is no recommended standardized ratio.
The NCSBN study (Hayden et al., 2014) used a 1:1 ratio
of time (i.e., one hour of simulation time equaled one hour
of traditional clinical time) because no evidence existed to
suggest using anything different. A subcommittee of the
INACSL Research Committee has completed a national
descriptive survey that investigated what ratios are
currently being used if the ratios are standardized per
course, who makes the decision(s) about the ratio being
used, and how the ratio is decided (Breymier et al.,
2015). It was suggested that low doses of simulation,

such as a five hours of time in the simulation laboratory

per semester, would most likely not yield significant differences in student learning outcomes, no matter what ratio was used.
Audience participants discussed costs associated with
simulation and strategies to solicit support for simulation
from colleagues and leadership within their school of
nursing. It was agreed that there is a need to show return
on investment with simulation. Simulation can be used to
increase patient safety and avoid sentinel events and near
misses (Barsteiner & Disch, 2012).
Another discussion topic focused on the substitution of
traditional clinical experiences with simulation. States are
at different stages in the use of simulation as a substitution
in this manner. Some states may determine that simulation
cannot be substituted for any clinical experiences. Nurse
educators may need to secure support for the use of
simulation through documentation of competencies that
students are unable to acquire in the clinical settings and
competencies that students are not able to acquire because
clinical sites are so few.
California has a state regulation that allows schools of
nursing to substitute up to 25% of traditional clinical time
per course with simulation. Some schools and the
California Simulation Alliance would like to increase the
substitution time from 25% to 50%, but the California State
Board of Nursing is resisting this request. Participants and
the panel discussed strategies to assist state boards of
nursing to accept simulation in place of traditional clinical.
Bringing a representative from the board to the nursing
school to observe simulation is one strategy.
Florida increased the percentage of time to 50% that
simulation could be substituted for traditional clinical
experience as soon as the NCSBN study (Hayden et al.,
2014) results were announced. A member of the Florida
Simulation Alliance expressed concern that Florida schools
of nursing were/are not prepared to institute simulation at
this high percentage yet. The participant voiced a fear
that the option to use simulation today will become a
mandate for simulation in the future.
This spawned a discussion related to faculty development for simulation. The question surfaced, Are faculty
prepared to educate students using 50% simulation time to
pp 2-7
Clinical Simulation in Nursing
Volume 12
Issue 1

After the National Council State Boards of Nursing Simulation Study

replace clinical experiences? Overall, the panel and
audience members indicated that they do not think faculty
are prepared. There were examples of students receiving
poorly run simulations and faculty not using best practices
in simulation because faculty are not trained in simulation
education or were unfamiliar with the case or setup. There
is a need to produce evidence that nursing schools and
nurse educators are trained and ready to facilitate
simulation before it is actually incorporated in the nursing
curriculum. Faculty development can be achieved through
formal education such as presentations, workshops,
consortiums, and certificate programs. The Standards of
Best Practice (International Nursing Association of Clinical
Simulation and Learning, 2013) should be an integral
component of any education session. The CHSE (Society
for Simulation in Healthcare [SSH], 2014) blueprint also
provides key elements of simulation which faculty should
be familiar.
Debriefing is the most important part of the simulation
experience. One audience member asked what
evidence-based practice debriefing method is best for
interprofessional learning. Members of the expert panel
recommended Promoting Excellence and Reflective
Learning in Simulation (Eppich & Cheng, 2015). This
debriefing method incorporates reaction, plus/delta, direct
feedback, and advocacy/inquiry methods. It also gives the
educator stems of phrases to use to elicit reflective learning
in participants.
Conversation shifted when a member asked for evidence
to support traditional clinical experiences. Members of the
panel acknowledged there is little if any evidence to
support traditional clinical. However, the Institute of
Medicine (2010) report lends support that current practices
are not working. There is a need to investigate learning
outcomes associated with both traditional clinical and
simulation.
Additional comments related to the lack of clinical space
and what students actually get to do during traditional
clinical experiences were made. A panelist voiced concern
that students often only complete vital signs and bed baths
when they are in the hospital setting. This concern echoes
findings that teachers and students focus on task
completion which often overshadows the more complex
aspects of learning nursing practice (Ironside, McNelis, &
Ebright, 2014, p. 185).
Another discussion centered on the future of simulation
in health care education. One member asked if the future of
simulation lies in accredited independent independently run
simulation centers. Members of the audience and panel
agreed that this could be a next step and would be a
fundamental change to health care education as it currently
exists. Students would get didactic and high-dose simulation before entering the clinical arena so that when they
were at the patient bedside, they could take full advantage
of patient experiences.

The final discussion point concerned the use of

simulation in graduate nursing education. A member
referenced the position of the National Task Force on
Quality Nurse Practitioner Education (2012) against replacing any portion of the 500 mandatory direct patient
care clinical hours in NP education with simulation, yet
clinical site placements are lacking for graduate practicums in the same manner as undergraduate clinicals.
The panel hoped that changes would be made in the future
in graduate nursing education. The American Association
of Colleges of Nurses has formed an Advanced Practice
Registered Nurse task force to investigate the use of simulation in graduate education (American Association of
Colleges of Nursing, 2014). There is a need rigorous scientific studies in the future to provide quantitative support
simulation to be counted as clinical hours in NP programs.

Addressing Issues and Barriers That Still Exist

The NCSBN Simulation Study addressed important gaps in
the literature regarding the use of simulation in prelicensure
nursing education. Still, many barriers and questions
remain. An expert panel and audience brought many of
those barriers and questions to the forefront of discussion
during the INACSL conference in June. Simulation use
continues to grow. Discussion related to the use of
simulation in healthcare education will continue.

References
Agency for Healthcare Research and Quality. (2015). Healthcare simulation to advance safety: Responding to Ebola and other threats.
Retrieved from http://www.ahrq.gov/research/findings/factsheets/errorssafety/simulproj15/index.html.
Agency for Healthcare Research and Quality. (2013). Advances in patient
safety through simulation research (R18). Retrieved from http://grants.
nih.gov/grants/guide/pa-files/PA-14-004.html.
American Association of Colleges of Nursing. (2014). Current state of
APRN clinical education. Retrieved from http://www.aacn.nche.edu/
APRN-White-Paper.pdf.
Barsteiner, J., & Disch, J. (2012). A just culture for nurses and nursing
students. Nursing Clinics of North America, 47(3), 407-416, http://dx.
doi.org/10.1016/j.cnur.2012.05.005.
Breymier, T. L., Rutherford-Hemming, T., Horsley, T. L., Atz, T.,
Smith, L. G., Badowski, D., & Connor, K. (2015). Substitution of clinical experience with simulation in prelicensure nursing programs: A national survey in the united states. Clinical Simulation in Nursing, 11(11),
472-478, http://dx.doi.org/10.1016/j.ecns.2015.09.004.
Decker, S. I., Anderson, M., Boese, T., Epps, C., McCarthy, J., Motola, I.,
., & Scolaro, K. (2015). Standards of best practice: Simulation standard VIII: Simulation-enhanced interprofessional education (sim-IPE).
Clinical Simulation in Nursing, 11(6), 293-297. http://dx.doi.org/10.
1016/j.ecns.2015.03.010.
Eppich, W., & Cheng, A. (2015). Promoting Excellence and Reflective
Learning in Simulation (PEARLS): Development and rationale for a blended
approach to health care simulation debriefing. Simulation in Healthcare,
10(2), 106-115, http://dx.doi.org/10.1097/SIH.0000000000000072.

pp 2-7
Clinical Simulation in Nursing
Volume 12
Issue 1

After the National Council State Boards of Nursing Simulation Study

Hayden, J. (2010). Use of simulation in nursing education: National survey
results. Journal of Nursing Regulation, 1(3), 52-57.
Hayden, J. K., Smiley, R. A., Alexander, M., Kardong-Edgren, S., &
Jeffries, P. R. (2014). The NCSBN national simulation study: A longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education. Journal of Nursing Regulation,
5(2), 1-66.
Institute of Medicine. (2010). The future of nursing: Leading change,
advancing health. Washington, DC: National Academies Press.
Retrieved from www.nap.edu/catalog.php?record_id12956.
International Nursing Association of Clinical Simulation and Learning.
(2013). Standards of best practice: Simulation. Clinical Simulation in
Nursing, 9(6S), Si-S32.
Ironside, P. M., McNelis, A. M., & Ebright, P. (2014). Clinical education in
nursing: Rethinking learning in practice settings. Nursing Outlook,
62(3), 185-191, Retrieved from http://dx.doi.org/10.1016/j.outlook.
2013.12.004.

Jeffries, P. R., & Rogers, K. J. (2012). Theoretical framework for simulation design. In P. R. Jeffries (Ed.), Simulation in nursing education:
From conceptualization to evaluation (2nd ed.). New York, NY:
National League for Nursing. (pp. 25-41).
Lioce, L., Meakim, C. H., Fey, M. K., Chmil, J. V., Mariani, B., &
Alinier, G. (2015). Standards of best practice: Simulation
standard IX: Simulation design. Clinical Simulation in Nursing,
11(6), 309-315, Retrieved from http://dx.doi.org/10.1016/j.ecns.
2015.03.005.
McGaghie, W. C., Issenberg, S. B., Petrusa, E. R., & Scalese, R. J. (2010).
A critical review of simulation-based medical education research: 20032009. Medical Education, 44, 50-63.
National Task Force on Quality Nurse Practitioner Education. (2012).
Criteria for evaluation of nurse practitioner programs. Washington,
DC: National Organization of Nurse Practitioner Faculties.
Society for Simulation in Healthcare. (2014). Certified healthcare simulation educator. Retrieved from http://www.ssih.org/Certification/CHSE.

pp 2-7
Clinical Simulation in Nursing
Volume 12
Issue 1