Alyssa Olson

Professional Service Learning Journal

Clinical Practicum II
July 20, 2016

The American Registry of Radiologic Technologists (ARRT) is a nationally known

credentialing organization for many healthcare modalities including radiation therapy. The
ARRT creates several different certification board examinations for examinees with the help of
various committees. For the past four years, I have been extensively involved with the ARRTs
examination committee. The responsibilities of this particular group are to evaluate and review
newly written board questions for future examinations in addition to evaluating the accuracy of
content in the current board forms. Board question evaluation occurs throughout the duration of
the year, while board exam review is held on an annual basis.
This year, the exam committee met at the ARRT headquarters near St. Paul, Minnesota
over a three day period from July 18 to July 20, 2016. The ARRT aims to create a well-rounded
radiation therapy certification exam through appointing a variety of radiation oncology-based
professionals. This meetings attending committee members included a physicist, an educational
program director, three radiation therapists, a dosimetrist, a psychometrician, and an exam
development coordinator; this ensures question accuracy, and the testing of a variety of topics.
During this meeting, our committee individually reviewed two of the available current
board exam forms. Individual questions were flagged for review if the content appeared to be
out of date, irrelevant, or caused unnecessary confusion among examinees within their personal
comment section. In addition to the 200 scored items on each test, we also evaluated 40
unscored, or piloted questions. Pilot questions serve to merely collect statistical data for future
scored item consideration; items with an excellent p-value and r-value can be used on future
exam forms as a scored item. Following the exam, the group analyzed each flagged item and

chose item replacements from the compiled question bank. Each exam was carefully scrutinized
by our psychometrician as he determined the level of difficulty for each replacement question. It
is the goal of the ARRT to create all exams with the same level of difficulty to be fair to the
examinees. This entire process took approximately two and a half days to complete and much
care is taken to design the best exam form possible. The final afternoon of the meeting was
spent analyzing submitted board exam questions from item writers. As a group, we determined
which questions could be added to the compiled bank of questions for future piloting and made
necessary revisions to those that required a modification.
This committee has been a huge time commitment over the past four years but it has been
one of my most rewarding and interesting professional activities to date. I have been able to
meet and develop great relationships with my fellow committee members from all over the
United States and look forward to catching up with them each year. In addition, it is such an
incredible experience to be able to help design a certification exam where we have specific input
on choosing important test concepts for future therapists. Through serving on this committee, I
have learned so much about how to correctly format test questions, how to analyze questions for
answer clueing, and how to evaluate a question for accuracy and purpose. Next year will be my
last participating year. I am sad to leave the group but excited to serve my profession through
other service opportunities and perhaps continue to use these newly learned skills through future
teaching opportunities.

Examination Committee 2016: (From left) Kaylee Voorhies, Alyssa Olson,

Samantha Stantz, Sherry Bicklein, Laura Minnick