AMIA 2015 Self Assessment MOC II Booklet FINAL

AMIA 2015 Self Assessment and
Maintenance of Certification (MOC) Part II Q&A booklet

Purpose
A number of presenters at AMIA 2015 were invited by AMIAs Education Department to submit multiple
choice questions (MCQ) associated with their presentations. Attendees of AMIA 2015 may use these
MCQs to test their knowledge of what they learned in these specific sessions.
All board-certified clinical informaticians must engage in answering the MCQs associated with the
sessions for which they will claim MOC-II credit.
American Board of Pathology-boarded clinical informaticians: acceptable performance in this self
assessment activity is defined as answering correctly 75% of the MCQs for the sessions you attended.
Copy the text on the next page to send an attestation of your self assessment to pesha@amia.org.
The ACGME competencies addressed by this activity are: Patient Care, Medical Knowledge, Practicebased Learning and Improvement, Interpersonal & Communications Skills, Professionalism, and
Systems-based Practice. For a full explanation of the ACGME competencies in the context of clinical
informatics, see Safran C et al. Program requirements for fellowship education in the subspecialty of
clinical informatics. J Am Med Inform Assoc. 2009 Mar-Apr;16(2):158-66.
MOC Committee Reviewers and Disclosures: The reviewers engaged in selecting sessions for MOC and
in reviewing the submitted questions are: Laura Fochtmann, Paul Fu, Anupam Goel, Gretchen Purcell
Jackson, Karl Poterack, and Keith Woeltje. Dr. Purcell Jackson discloses that she receives
financial/material support from West Health. The other reviewers disclose that neither they nor their
life partners have relevant financial relationships with commercial interests related to the content of
this activity.
Instructions
The first half of this booklet of >300 MCQs provides the question and answer options organized by day
and by session number. The second half provides the correct answers per question. The answers are
followed with explanations and suggested readings for more information. The booklet is bookmarked.
Clinical Informatics Diplomates: How to claim MOC-II credit
Log into your account at www.amia.org and click My Profile
Click tab Invoices & Transactions
Scroll down to Events
Find AMIA 2015 Annual Symposium and click Credits
For Select Credit Type click Physician
For Select Physician Credit Type click MOC-II and select your MOC-II sessions attended
o You must claim your MOC-II credit first for AMIA to keep an accurate record of your
MOC-II credits
Click Save and Add Additional Credit Type at bottom of screen
Now for Select Physician Credit Type click Physician on drop-down menu, and claim your
regular CME sessions
Click Submit
Click My AMIA Activities to download your certificate
Claim credit by December 21, 2015. AMIA reports MOC-II credits to the ABPM and to the American
Board of Pathology. We pull data to send reports only within the same year as the activity.
Page | 1
Email attestation text for American Board of Pathology-boarded

clinical informaticians:
To: Pesha@amia.org
From: [Use the email you have on file with the American Board of Pathology]
Subject: MOC-II credits AMIA 2015
I attest that I attended the live activity AMIA 2015 and used the self assessment booklet as my Self
Assessment Module (SAM) for those sessions designated as offering MOC-II credit in the activity.
I attest that I achieved at least a 75% score on the questions I answered that were associated with the
MOC-II designated sessions that I attended.
Name
Page | 2
Sessions Designated as Available for MOC-II Credit
Saturday, November 14, 2015

T02: A Problem Well Stated is Half Solved: A Case-Based Tutorial About Approaching
Evaluation and Technical Assistance Projects Through Informatics Problem-Solving
8:30 AM 12:00 PM; Continental 7 (Hilton San Francisco Union Square)
T04: AMIA 2015 CMIO Workshop
T05: Practical Modeling Issues: Representing Coded and Structured Patient Data in EHR
Systems
1:00 PM 4:30 PM; Imperial B (Hilton San Francisco Union Square)
T06: The Art and Science of Writing Items for High Stakes Exams
1:00 PM 4:30 PM; Continental 7 (Hilton San Francisco Union Square)
WG09: Primary Care and EMRs in the 21st Century Why Havent We Got it Right Yet and
How Can We Make it Better? (Sponsored by the Primary Care Informatics Working Group)
1:00 PM 4:30 PM; Yosemite B (Hilton San Francisco Union Square)
Sunday, November 15, 2015

T07: Innovations in Standards & Standards Development: Advances in Standards
Methodologies & Implementation
8:30 AM 12:00 PM; Continental 1/2 (Hilton San Francisco Union Square)
T13: Introduction to Biomedical Informatics
8:30 AM 12:00 PM; Franciscan B (Hilton San Francisco Union Square)
S01: Interactive Panel - Open Architecture for Pathways and Care Coordination
S03: Didactic Panel - Wearable Health Data and the Quantified Self: What Role in the Clinical
Context?
S09: Podium Presentations: Information Retrieval and Data Capture
3:30 PM 5:00 PM; Yosemite A (Hilton San Francisco Union Square)
Page | 3
Monday, November 16, 2015

S13: Didactic Panel - Harmonization of ICD-11 and SNOMED CT - Not just Mapping! Practical
and Theoretical Lessons and Benefits to Users and Implementers
8:30 AM 10:00 AM; Continental 5 (Hilton San Francisco Union Square)
S20: Podium Presentations - Are We Safer Yet?
8:30 AM 10:00 AM; Imperial B (Hilton San Francisco Union Square)
S21: Systems Demonstrations - Mapping Public Health
8:30 AM 10:00 AM; Plaza A (Hilton San Francisco Union Square)
S25: Didactic Panel - The Clinical Quality Framework Initiative to Harmonize Decision Support
and Quality Measurement Standards: Defined Standards, Pilot Results, and Moving Beyond
Quality Improvement
S37: Papers - Patients Want to Know
1:45 PM 3:15 PM; Continental 1/2/3 (Hilton San Francisco Union Square)
S41: Podium Presentations - Government Initiatives in Health IT
1:45 PM 3:15 PM; Plaza B (Hilton San Francisco Union Square)
S42: Podium Presentations - Care Team Communication
1:45 PM 3:15 PM; Continental 7/8/9 (Hilton San Francisco Union Square)
S48: Didactic Panel - ClinicalTrials.gov: Adding Value through Informatics
3:30 PM 5:00 PM; Imperial B (Hilton San Francisco Union Square)
S54: Systems Demonstrations - From Patients to Research
3:30 PM 5:00 PM; Plaza A (Hilton San Francisco Union Square)
Tuesday, November 17, 2015

S57: Didactic Panel - The Best of Imaging Informatics Research 2015
8:30 AM 10:00 AM; Room: Continental 4 (Hilton San Francisco Union Square)
S58: Didactic Panel - Looking Back and Moving Forward: A Review of Public and Global Health
Informatics Literature and Events
8:30 AM 10:00 AM; Room: Continental 6 (Hilton San Francisco Union Square)
S59: Interactive Panel - What Could Go Wrong?: Migrating from One EHR to Another
Page | 4
S60: Interactive Panel - Patient Privacy and "De-identified Health Records in the Genomic Era
S61: Didactic Panel - User-centered Methods to Optimize Clinical Decision Support: Examples
from Pediatrics with Applicability to All Care Settings
10:30 AM - 12:00 PM; Continental 6 (Hilton San Francisco Union Square)
S62: Didactic Panel - Developing Natural Language Processing Systems for Healthcare
10:30 AM - 12:00 PM; Imperial A (Hilton San Francisco Union Square)
S70: Didactic Panel - Needs of the Digital Native: Adolescents and Access to PHRs
1:45 PM - 3:15 PM; Continental 4 (Hilton San Francisco Union Square)
S71: Didactic Panel - Rapid Development and Implementation of Critical Information Systems
for Ebola Treatment Centres in West Africa: Lessons for Future Events
S76: Papers - Making EHRs Useful
1:45 PM - 3:15 PM; Continental 7/8/9 (Hilton San Francisco Union Square)
S80: Didactic Panel - Health Information Technology and Large-scale Adverse Events
S81: Didactic Panel - Collaboration and Health Information Technologies: Towards Defining
and Operationalizing the Collaboration Space
3:30 PM - 5:00 PM; Imperial A (Hilton San Francisco Union Square)
S83: Didactic Panel - State of the Art of Clinical Narrative Report De-Identification and Its
Future
S84: Papers - Clinical Decision Support II
3:30 PM - 5:00 PM; Imperial B (Hilton San Francisco Union Square)
S85: Papers mHealth
3:30 PM - 5:00 PM; Continental 1/2/3 (Hilton San Francisco Union Square)
S88: Podium Presentations - Human Factors are Key
3:30 PM - 5:00 PM; Yosemite C (Hilton San Francisco Union Square)
S90: Podium Presentations - EHR Usability and Quality
3:30 PM - 5:00 PM; Yosemite A/B (Hilton San Francisco Union Square)
Page | 5
Wednesday, November 18, 2015

S91: Interactive Panel - Patient Portals: Best Practices and New Directions for Development
and Investigation
8:30 AM - 10:00 AM; Continental 4 (Hilton San Francisco Union Square)
S95: Papers - All about Handoffs
8:30 AM - 10:00 AM; Continental 1/2/3 (Hilton San Francisco Union Square)
S96: Papers - Human-computer Interaction
8:30 AM - 10:00 AM; Continental 7/8/9 (Hilton San Francisco Union Square)
S98 Papers - Taken as Directed
8:30 AM - 10:00 AM; Yosemite A (Hilton San Francisco Union Square)
S101: Systems Demonstrations - EHRs of the Future
8:30 AM - 10:00 AM; Plaza A (Hilton San Francisco Union Square)
S102: Didactic Panel - Perioperative Clinical Decision Support: Improving Care of the Surgical
Patient through Informatics
10:30 AM - 12:00 PM; Continental 4 (Hilton San Francisco Union Square)
S104: Didactic Panel - The Implementation of Online Patient Portals in Safety Net Settings:
The Realities of Meaningful Use Certification with Vulnerable Patient Populations
10:30 AM - 12:00 PM; Imperial A (Hilton San Francisco Union Square)
S105: Papers - Patients in Control: Self-management Tools
10:30 AM - 12:00 PM; Continental 7/8/9 (Hilton San Francisco Union Square)
S109: Papers - The User Perspective on Informatics Tools
10:30 AM - 12:00 PM; Yosemite C (Hilton San Francisco Union Square)
Page | 6
Saturday, Nov. 14, 2015 Questions & Answer Options

Herman Tolentino; Laura H. Franzke; Sridhar R. Papagari Sangareddy; Catherine Pepper
T02-1:
An applied definition is a statement of means and ends. Which of the following best describes
the ends in an applied definition of public health informatics?
a) Information value cycle
b) Problem solving
c) Systems thinking
d) Health system feedback loop
e) Improving population health
T02-2:
In the health system feedback loop, health consequences are affected by which of the following
components?
a) Information systems
b) Health determinants and outcomes
c) The health system
d) Collective action
e) Data measurement challenges
T02-3:
Generating value from data resources requires an iterative process. In the Information Value
Cycle (IVC) which step do we first consider the resources necessary to implement an
information system?
a) Planning
b) Evaluation
c) Capture
d) Manage
e) Analyze
Page | 7
T02-4:
In which part of the DISC Model will you find technology?
a) Data, information and knowledge
b) Information systems
c) Context
d) Capture
e) Problem solving
T02-5:
In informatics problem-solving, which of the following is essential when identifying a problem?
a) Plot of trajectory of change from past to future state
b) Knowledge of gaps between current and future state
c) List of prior trouble-shooting attempts
d) Identification of prior failure points
e) Description of past and current personnel involved in the project
T02-6:
Which components of the Problem Solving Framework focus on the problem solver?
a) Problem solving inputs and problem management
b) Problem management and performance improvement
c) Problem solving inputs and context
d) Problem solving inputs and performance improvement
e) Context and performance improvement
T02-7:
You get a request from the local health department to get help in solving a problem of
integrating community data into public health surveillance systems. Which step is the most
important to do first?
a) Start develop solutions based on what you think is going on
b) Define the problem and its context
c) Make informed recommendations based on your experience in other settings
d) Contact a local IT company to do the job
e) Design a dashboard to show where the data is coming from and how the data are
integrated in the database backend.
Page | 8
T02-8:
In informatics problem solving, which of the following best describes the timing of a context
determination?
a) At the beginning of the project before establishing project scope
b) Mid-way through the project, once initial design specifications are outlined
c) As the project is coming to a close, so that context can be integrated into training efforts
d) At multiple points, as the project unfolds
T02-9:
Systems thinking in problem solving involves:
a) As a first step, identifying solutions that will solve the problem
b) Viewing problems as part of a wider, dynamic system
c) Solving problems from one stakeholder perspective
d) Acting on surface features of a problem and assume certain root causes due to time
constraints
T02-10:
You are asked to solve a problem about integrating data from multiple information systems in a
state health department. You start analyzing the problem and learn the following:
Statement 1: Recently the State Senate has enacted legislation that mandates integration of all
injury surveillance systems.
Statement 2: The state health department asks what you can do to help link patient interview
and medical records abstraction datasets that are currently captured in separate databases in
their Medical Monitoring Program.
Select the right answer.
a) Statement 1 is the context for the problem you have to solve.
b) Statement 1 is the problem statement.
c) Statement 2 is the context for the problem you have to solve.
Page | 9

Paul Fu; Richard Schreiber; Julie Hollberg; Joseph Kannry
Practical Approaches to EHR Governance - Dick Schreiber
T04-1:
The principles of EHR governance apply to:
a) Large university hospitals
b) Teaching hospitals
c) Small community hospitals
d) Ambulatory practices
e) All of the above
T04-2:
The most critical success factor in EHR governance is:
a) assuring purchase of the newest technology
b) managing and leading change transformation processes
c) establishing best infrastructure prior to and after EHR installation
d) rewarding staff for meeting or exceeding goals
T04-3:
Effective governance requires effective communication for EHR optimization and healthcare
transformation, as well as implementation. All of the following have been shown to be
successful forms of communication, but the least effective is:
a) Rich channels such as personal interaction and audiovisual aids are better than email
alone
b) Encouraging leaders to disseminate information to users
c) Just-in-time trainingnot too soon, not too late; as well as at-the-elbow support
d) Involvement of clinicians as well as technical experts in creating the communication
plans.
Page | 10
People, Process & Technology: Putting them all together to Optimize the Physician
Experience - Julie Hollberg
T04-4:
Physicians frustrations with EHRs are largely ignored for all of the following reasons EXCEPT:
a) Physicians who voice complaints are seen as technophobic, resistant, and uncooperative
b) Healthcare IT is too fragile to withstand constructive criticism
c) Our government holds general belief that Health IT will improve care and reduce cost
d) Hospitals have a universal mandate to adopt a new technology before its effects are
fully understood
T04-5:
In order for healthcare IT to improve medical care, it must
a) Help us synthesize and analyze medical information
b) Tell our patients stories
c) Acknowledge its mistakes and learn from them
d) All of the above
T04-6:
All of the following are examples of leadership failures which can damage your implementation,
except:
a) Lack of available time
b) Lack of political skills
c) Lack of communication skills
d) Lack of vision
e) All of the above can contribute to system failures
T04-7:
Key leadership success factors for any project that involves people, processes, and technology
include all EXCEPT:
a) Executive leadership dedicated to removing barriers
b) Clearly identified and engaged local physician and clinical operation leaders to serve as
workflow change agents
c) Decentralized command centers to address local concerns during go-live
d) Strong project managers with personal relationships with providers.
T04-8:
The following are all best practice myths EXCEPT:
a) Training is not necessary for computer-savvy providers
b) Ambulatory productivity should be reduced permanently to accommodate changed
provider documentation workflow
c) Managers assigned to support implementation/conversion activities should not have
other responsibilities at go-live.
d) Additional staff is required to execute the new workflows.
CMIO Surveys: Who are we? What do we do? - Dick Schreiber

T04-9:
Pressing challenges for the CMIO in 2016 include:
a) Data analytics and business intelligence
b) Population health
c) Meaningful Use
d) EHR optimization
e) All of the above
CMIO Year-In-Review - Dick Schreiber/Paul Fu, Jr.
T04-10:
Which are the following statements are true about EHR adoption in the United States?
a) Adoption of a combination of Basic and Advanced EHR systems has increased
substantially in the past four years, reflecting a combination of the availability of
financial incentives and the impending penalties, at least from the Medicare part of the
meaningful-use program.
b) A majority of hospitals might not be ready to meet the stage 2 core objectives when
required to do so, especially Critical Access Hospitals and smaller hospitals which lag
behind in both EHR adoption and the ability to meet state 2 MU criteria.
c) More than 60% of US acute care hospitals with less than basic EHR are unable to
support online physician documentation.
d) Up-front capital costs, ongoing support costs, physician cooperation, and complexity of
meaningful-use criteria within the specified time-frame are key challenges for hospitals
able to meet and not able to meet MU Stage 2 core objectives.
e) All of the above
T04-11:
A mismatch between clinician workflow and EHR functionality is a major and growing concern,
especially in the areas of usability and patient safety. Which of the following are
recommendations to improve EHR systems?
a) Optimize EHRs to facilitate longitudinal care including the ability to support teams of
clinicians and patients
b) Avoid checkboxes or similar repetitive data entry designs to minimize inaccurate date
entry.
c) Improve the designs of interfaces so that they support and build upon how people think.
d) Use public, standards-based APIs and data standards that enable EHRs to become more
open to innovators, researchers, and patients
e) All of the above
T04-12:
Which of the following statements about Health Information Exchanges is true?
a) Studies of HIEs that used study designs having strong internal validity were significantly
more likely than studies not using such designs to find a benefit from HIEs for all
outcomes.
b) HIEs have spread across the US, are financially viable, and are very mature.
c) Emerging vendor-driven interoperability initiatives may change the way we view and
value health information exchange
d) There are many rigorous studies that link HIE adoption and health outcomes.
T04-13:
The effective visual display of data and clinical decision support alerts is critical in supporting
safe, effective health care delivery. Which of the following statements is NOT true?
a) There are no regulations governing EHR usability.
b) Evidence-based design principles from existing EHR safety design guides (e.g. SAFER) are
not widely used by existing EHR systems
c) Clinical decision support rules frequently are integrated into the EHR without end-user
usability testing, resulting in poor adoption rates
d) Many current commercial EHRs have significant limitations in graphing capabilities of
laboratory test results and often display results in non-standardized fashion
e) All of the above are true.
Page | 10c
T04-14:
The drive for value- over volume-based reimbursement is leading to the greater adoption of
new models of care delivery. The patient-centered medical home (PCMH) model of primary
care is being implemented widely. Which of the following statements is FALSE regarding the
role of the EHR in PCMH?
a) The PCMH typically involves EHRs, organizational practice change, and payment reform,
although its effects on quality are unclear.
b) In a study of PCMH adoption vs non-PCMH adoption, with and without EHR support, the
PCMH group improved significantly more over time than either the paper group or the
EHR group in 4 out of 10 process measures
c) The aspects of the PCMH that drove improvement are distinct from, but may be enabled
by, the EHR.
d) None of the above
Patient Portals and Patient Empowerment - Paul Fu, Jr.
T04-15:
What is the role of the patient portal in patient empowerment?
a) Required by the EHR Meaningful Use
b) Change outcomes and behavior
c) None, there is no literature except diabetes
d) All of the above
T04-16:
What is the most effective model for a patient portal?
a) Tethered to a sponsoring EHR vendor
b) Tethered to a sponsoring data source
c) Tethered to a sponsoring provider organization
d) Untethered
T04-17:
What are the challenges and barriers involved in deploying a patient portal?
a) Policies
b) End-user support
c) Use case diversity
d) All of the above
T04-18:
What factors predispose towards patient portal adoption?
a) Caregivers of elderly patients
b) Parents of young children
c) Patients with chronic conditions
d) Provider acceptance and promotion
e) All of the above
Page | 10e
Systems
Stanley M. Huff
T05-1:
Which one of the following statements is true?
a) The complexity of modern medicine exceeds the limits of the unaided human mind.
b) Proper and rigorous medical education can lead to error-free medical management.
c) Physicians can reliably manage up to 10 parameters when making medical decisions.
d) Physicians actions match the facts that they have been taught.
T05-2:
Which one of the following things is NOT needed to enable true data interoperability?
a) Detailed clinical information models.
b) A knowledge repository for storing medical logic modules.
c) The detailed clinical information models need to be coupled (bound) to standard
terminologies.
d) Specification of Application Programmer Interfaces.
e) Open sharing of models, terminology, and APIs.
T05-3:
Which of the following statements is true about SNOMED CT?
a) There is only one way to represent a given concept using SNOMED CT codes.
b) SNOMED CT has many concepts that are not fully defined.
c) All needed medical concepts exist in SNOMED CT.
d) There is a clean division of labor between the content of SNOMED CT and LOINC codes.
Page | 12
T05-4:
Which one of the following statements is NOT true?
a) Many countries and organizations worldwide are creating detailed clinical information
models.
b) Clinical information models need to be created for all aspects and domains of clinical
medicine.
c) Information models can be abstract and general, or very detailed and specific.
d) We need to agree on a universal information model to enable interoperability.
T05-5:
Which one of the following is a goal of the Clinical Information Modeling Initiative (CIMI)?
a) Create a shared repository of detailed clinical models.
b) Use any standard formal modeling language to represent the models.
c) Define standard application programmer interfaces (APIs) for using the models for
information sharing.
d) Use data types from standard programming languages as the elemental building blocks
of the models.
T05-6:
Which one of the following statements about medical terminologies is true?
a) Terminology tables or terminology services are needed for clinical information
modeling.
b) Terminologies are the same as ontologies.
c) Terminologies are organized according to self-evident hierarchies.
d) Terminologies assume an open world view.
T05-7:
An evaluation style model would be more appropriate to use than an assertion style model
when:
a) A particular attribute of the patient is being assessed
b) A condition or disease state that pertains to the whole individual is being assessed
c) A detailed statistical analysis will be performed
d) Very few measurements will be conducted
Page | 13
T05-8:
Which one of the following statements is true about representing subject of information in
detailed clinical information models?
a) Subject of information is always an independent attribute in models.
b) Some modeling styles for subject of information could lead to combinatorial
explosion.
c) Representation of subject of information is unrelated to the representation of family
history information.
d) Subject of information is managed in one consistent style within the LOINC terminology.
T05-9:
Which one of the following statements about modeling strategies is true?
a) There is typically a best way to model a given type of clinical data.
b) Two models can be isosemantic, that is, they have exactly the same information
content, but they have different representations.
c) Post coordination is better than precoordination.
d) The most popular medical modeling language is Abstract Syntax Notation 1 (ASN.1)
T05-10:
Which one of the following statements is NOT a requirement for good models?
a) Models are unambiguous.
b) Models are independent of the terminology used for semantic binding.
c) Models are consistent across medical domains.
d) Models evolve gracefully with the addition of new knowledge.
Benson Munger; John Finnell
T06-1:
In a multiple choice test item which one of the following is an example of Testwiseness?
a) None of the above is used as an option
b) Terms in the options are not stated consistently
c) The item contains superfluous information
d) The item contains tricky language in the stem
Page | 14
T06 2:
a) An item is partially true
b) Language in the foils is not parallel
c) The stem and a foil contain the same word
d) The item contains the word always
T06 3:
In a multiple choice test item which one of the following is an example of Irrelevant Difficulty?
a) A foil does not logically follow from the stem
b) A subset of foils is collectively exhaustive
c) The answer is connected to a previous item
d) The stem is negatively phrased
T06 4:
The length of an item stem should be which one of the following in relation to the responses?
a) Approximately the same
b) Longer than the responses
c) Shorter than the responses
d) There is no standard
T06 5:
A poorly constructed test exhibits which one of the following characteristics:
a) Content validity
b) Consistency
c) Reliability
d) Standard length
Page | 15
T06 6:
Other things being equal, the highest level of test item discrimination would be achieved with
which one of the following levels of difficulty?
a) 90%
b) 75%
c) 50%
d) 25%
e) 10%
T06 7:
Which one of the following steps should be part of developing a high quality test item?
a) Do a thorough literature search
b) Construct 5-6 plausible foils
c) Construct the foils first
d) Write out the testing point
T06 8:
The City of Chicago
1.
Became a great trade center
2.
Became an early rail center
3.
Elected a second Mayor Daley
4.
Had a great fire in 1871
The test item above is poorly constructed because it,
a) Fails to ask a definite question
b) Has foils that are too long
c) Has two foils that start with the same word
d) Lacks a definition in the stem
Page | 16
T06 9:
Where did American Slavery first start?
1.
Jamestown
2.
New York
3.
Pennsylvania
4.
Plymouth
The test item above is poorly constructed because the foils,
a) Are in alphabetical order
b) Are too specific
c) Belong to different classes
d) Lack city/state names
T06 10:
How long do historians think the first Thanksgiving lasted?
1.
2.
3.
4.
One day
Three to four days
Two days
Two to three days
The test item above is poorly constructed because the foils are:
a) Independent
b) Overlap times
c) Too short
d) Too specific
Page | 17
Stephen J. Morgan; David M. Newman; Walton Sumner; Michael E. Kordek
WG09-1:
EHRs that use process control charts for population health measures usually do not provide
similar tools for time series of data in individual patient's records because :
a) Irregular intervals between observations compromise validity
b) Physicians typically have managed patients without such charts
c) The statistical methods require aggregate data at each time point
d) Visual cues from an individuals data time series are often misleading
WG09-2:
The Systems Engineering Initiative for Patient Safety (SEIPS) 2.0 model added which of these
concepts to the original Structure-Process-Outcome model:
a) Accountability (documentation of data sources and decision makers)
b) Engagement (stakeholders have separate but collaborative activities)
c) Fail-Safe (likely lapses and mistakes are automatically flagged for review)
d) Security (physical and keyword requirement for entering and retrieving data)
WG09-3:
Usability is:
a) unprofitable and/or unproductive
b) qualitative, never quantitative
c) a personal feeling
d) an academic discipline
WG09-4:
The origin of style sheets can be credited to:
a) Steve Jobs and Apple
b) Allen & Gates: Microsoft
c) IBM
d) The New York Times
e) "Hart's Rules" for the Oxford University Press
Page | 18
WG09-5:
Which of these organizations have not written about good Electronic Health Record design?
a) NIST
b) FHIR
c) AHRQ
d) Apple
WG09-6:
In the care of children, what is the most important reason for an electronic record system to
document the time of birth?
a) Parents like to share the time of birth on social media
b) Knowing the time of birth is required to generate an entry into the patient database
registry
c) Several laboratory "normal reference ranges" widely vary based on the exact age of the
patient
d) Determining the time of birth is important to identify products of a multiple gestation
WG09-7:
The typical workflow for primary care physicians using an EHR is
a) An approximately linear progression from chief complaint to history to physical to
assessments and plans
b) Comparable to the subjective-objective-assessment-plan workflow without EHRs, then
after-visit documentation
c) Review of patient complaints and automated reminders, then focused history, physical,
assessment, and plan
d) Multiple cycles of problem identification, varied data gathering, documentation,
assessment, and planning
Page | 19
WG09-8:
The ideal EHR documentation process would be?
a) Anticipatory
b) Comprehensive
c) Almost invisible
d) Rapid
WG09-9:
Out of all the child health specific needs for electronic health record systems which is the most
challenging to implement?
a) Age specific growth charts
b) Weight based medication dosing
c) Patient portals
d) Blood pressure percentile calculations
Page | 20
Sunday, Nov. 15, 2015 - Questions & Answer Options

Charles Jaffe; Douglas B. Fridsma; Stan Huff; Christopher Chute; John D. Halamka; William E.
Hammond
T07-1:
Which is not a goal of the Argonaut project?
a) Create an implementation guide for the query/response of the Meaningful Use
Common Data Set
b) Create an implementation guide for the secure RESTful transport of data
c) Replace the CCDA
d) Create an implementation guide for the creation of a trust fabric using OAUTH
T07-2:
What is not yet in scope for the Argonaut project?
a) Enable applications to exchange data within an organization
b) Enable applications to exchange data between organizations
c) Provide a mechanism for querying a provider directory
d) Provide a mechanism for uniquely identifying a patient
T07-3:
Which of the following is not a component of FHIR?
a) Profile
b) Bundle
c) Resource
d) Extension
e) Paradigm
Page | 21
T07-4:
Which of the following statements about FHIR is incorrect?
a) FHIR permits transporting data at the lowest levels of granularity.
b) FHIR access and use require membership in HL7 International .
c) FHIR is service driven.
d) FHIR permits sending only the data that is required.
e) FHIR is based on a stateless protocol.
T07-5:
Which of the following standards is most useful in creating Restful interoperable services?
a) HL7 Version 2.X
b) HL7 Version 3
c) HL7 Clinical Document Architecture (CDA)
d) HL7 FHIR
T07-6:
Which terminology is most useful for coding the names of clinical observations?
a) SNOMED CT
b) RxNorm
c) LOINC
d) RadLex
T07-7:
The most recent draft standard of FHIR (DSTU 2.0) provides all of these enhancements EXCEPT?
a) Extends search and versioning significantly
b) Increases the power and reach of the conformance resources and tools
c) Defines a terminology service
d) Enhances human readability
e) Broadens functionality to cover new clinical, administrative and financial areas
Page | 22
T07-8:
The many attributes of FHIR include all of these benefits EXCEPT
a) Shortens development time and reduces development costs
b) Replaces V3
c) Relies upon the rule of 80/20
d) Evolved from HL7 standards that are the keystone of meaningful use
e) Written in a manner understood by both clinicians and technology developers
T07-9:
Which of the following FHIR resources are at the right level of granularity for a resource?
a) Resting HR after exercise
b) Intervention
c) Allergy
d) Hospital
T07-10:
Which of the following principles are NOT part of the guiding manifesto for FHIR?
a) Focus on implementers
b) Develop healthcare specific web technologies
c) Require human readability as base level of interoperability
d) Make content freely available
T07-11:
What are the key innovations in the ICD 11 project from an informatics perspective?
a) ICD 11 includes many more codes than ICD 10
b) ICD 11 includes traditional medicine codes
c) ICD 11 has an underpinning semantic network from which it can derive an arbitrary
number of coding linearizations
d) ICD 11 will be available in multiple languages
Page | 23
T07-12:
What is the common framework between the ICD and SNOMED?
a) ICD and SNOMED are both classifications
b) ICD and SNOMED share a common ontology
c) ICD and SNOMED include shared information
d) ICD and SNOMED are multilingual

Joseph W. Hales; Christopher Cimino
Definition
T13-1:
Biomedical informatics refers to the field of study of
a) the use of biomedical information in the assessment and delivery of medical care.
b) the use of computers and related technologies to manipulate biomedical information.
c) the optimal use of biomedical information for problem solving and decision making.
d) all aspects of electronic health record systems.
Terminology
T13-2:
Controlled vocabularies have advantages and disadvantages as compared to uncontrolled text.
Which options best captures the most positive and most negative aspects of controlled
vocabularies?
a) Pro: vocabulary controlled by experts; Con: licensing fees drive up healthcare costs.
b) Pro: expansion of the caregivers vocabulary; Con: require tedious menus for selections.
c) Pro: allow automatic analysis of health information; Con: Limit the caregivers
expressiveness.
d) Pro: single terminology shared for many purposes; Con: Competition between
vocabulary vendors makes it hard to choose the most long lasting solution.
Page | 24
T13-3:
The ideal controlled vocabulary
a) provides coverage of the topic area with minimal overlap between terms.
b) provides an other or not otherwise specified category or modifier in all hierarchies.
c) places each term exactly once within the terminology hierarchy.
d) minimizes the depth of the terminology hierarchy.
T13-4:
The National Library of Medicines unified medical language system (UMLS) semantic network is
used
a) to compute the definition of medical terms for the purpose of decision support in the
information sources map.
b) to drive a natural language processing algorithm in the lexical matcher.
c) for research purposes but connects only minimally to the other UMLS components.
d) to assist in matching new terms to existing concepts within the metathesaurus.
Medical Records
T13-5:
The primary driving force behind the development of electronic medical records has historically
been to
a) provide medico-legal documentation to address malpractice suits.
b) allow better communication between multiple care givers.
c) provide faster submission of coded bills to insurance companies.
d) track and improve the quality of medical care.
Page | 25
T13-6:
Health Level 7 (HL7) provides a standard that is supposed to increase the ease with which
different health information systems (ie pharmacy, lab, radiology) integrate into a single
medical record. Its primary approach to doing this is by defining standardized
a) table structures.
b) messages.
c) vocabularies.
d) field types.
Decision Support
T13-7:
Over time, well structured rule-based reminder systems will
a) meet initial resistance to change physician behavior but eventually they will adopt the
new practices.
b) quickly produce a change in physician behavior that is sustained even after the
reminders are discontinued.
c) quickly produce a change in physician behavior but their behavior returns to baseline
shortly after the reminders are discontinued.
d) meet initial resistance to change physician behavior and ultimately fail to create any
lasting change unless hospital policies are put in place and enforced to require the
change.
T13-8:
The Health Information Technology for Economic and Clinical Health Act (HITECH) has become a
major driver of clinical decision support implementation because it
a) provides incentives for providers who can demonstrate they have used electronic
records in a meaningful way that measures quantity and quality of health care
delivered.
b) requires all certified electronic health record systems to implement quality
improvement applications that can demonstrate a meaningful improvement in health
care quality over time.
c) provides funding for clinical decision support research.
d) requires Medicare and Medicare patients in programs receiving government funding to
allow their de-identified patient information to be used for research purposes.
Page | 26
T13-9:
The central dogma of biology describes the
a) basis for evolutionary development of life
b) fundamental relationship between DNA and its expression in proteins (and ultimately
living organisms)
c) life cycle of living organisms, from birth or multiplication to death
T13-10:
The Gene Ontology is an initiative to
a) unify the representation of genetic information across species.
b) create a controlled vocabulary of the human genome
c) name all of the genetic products of all possible combinations of gene sequences.
Steve Demuth, Matthew Burton, Robert A. Greenes, Davide Sottara, Keith Toussaint
S01-1:
Considering an interoperable app deployment infrastructure, what are the essential features
that would benefit mostly from the curation, deployment and execution of a robust set of
knowledge resources externally to EHRs?
a) The resources can be edited in a common place.
b) The resources can be incorporated in services.
c) Apps that deliver continuity of care, workflow support, and decision support must
bridge between EHRs.
d) There needs to be transparency of knowledge used in systems.
S01-2:
Considering next-generation interoperable apps as the driving use cases, there are a number of
potential benefits to having a testbed/sandbox for communal development. However, what
would be the primary goal it could provide?
a) A place to assemble the needed resources, since they are hard to obtain.
b) A place to obtain needed technical support, since this is a new framework for app
deployment.
c) A place to both evolve the infrastructure itself and support development of apps,
because the compelling use cases involve need for both.
d) A way to provide a bridge to deployment and commercialization, because the
ecosystem is not yet mature.
Page | 27
S01-3:
Considering the characteristics of the prevalent service models in cloud computing, which
service models or combinations thereof must be present to enable open architectures
supporting the execution of clinical pathways and care coordination?
a) Software-as-a-Service only
b) Infrastructure-as-a-Service only
c) Platform-as-a-Service and Infrastructure-as-a-Service
d) Software-as-a-Service and Infrastructure-as-a-Service
S01-4:
What are the central architectural principles employed in Mayo Clinics emerging knowledge
framework, as initially demonstrated by the EASE solution:
a) Deployment on public cloud computing infrastructure is a key aspect for scalability
b) Uses exclusively open-source implementations of architectural components
c) Adequate separation of concerns at the data, knowledge, service and application layers
d) Proves that the adoption of open interface standards alone is sufficient to enable
interoperability
S01-5:
What are central tenets the Healthcare Services Platform Consortium (HSPC) seeks to enable
a) Enable only non-profit business models and transactions
b) Facilitate clinical application interoperability and data sharing
c) Initiate full migration from existing EHRs and clinical systems
d) Establish a single reference architecture upon which all EHR systems are expected to
converge
S01-6:
A current limitation for the evolution of EHR solutions that this panel seeks to overcome is:
a) Silos of data and proprietary architectures
b) For-profit EHR and other clinical and life systems information systems
c) Use of existing messaging and data format standards
d) Adherence to Meaningful Use-related EHR certification
Page | 28
Context?
Emil Chiauzzi; Kevin Patrick; Cinnamon Bloss; Ernesto Ramirez
S03-1:
The Quantified Self community is a growing network of over 100 MeetUps around the world,
conference events, and focused meetings. At these different gatherings there are many ways
individuals expose their interest in self-tracking tools, the data they gather, and their
experiences. Of the following, what is the core focus of the community?
a) The gather and disseminate information about device accuracy and validity.
b) To develop methods to interface personal data with the healthcare system.
c) To share real-world, personal examples of learning from the process of self-tracking and
self-experimentation.
d) To generate generalizable findings for research publication.
S03-2:
There is increasing demand to incorporate patient-generated health data into the medical
record. Much of that data may have clinical significance. However, access to the data that apps
and devices create is unregulated, and implementation differs across companies.
Which of the following describes a best-case scenario for personal data access to data from
consumer health and wellness tools/apps?
a) Companies only integrate with personal medical records and individuals are allowed to
access it through their healthcare system.
b) Companies provide an option for the individual to export the data from their system.
c) Companies only share data with qualified researchers and research institutions.
d) Companies provide access to the individual in a structured, machine-readable format
and allow data sharing through a well-documented API.
S03-3:
There has been much discussion of the use of wearable technologies as a means of assessing
the status of chronic disease patients in their natural environments. What principle should be
considered most in the application of wearable technologies to chronic disease management?
a) It is best to focus on younger, more active chronic disease populations
b) Wearable devices should only be used if they can account for all physical activity
c) Patients with chronic disease are unable to derive much benefit from devices that stress
social comparison and athletic competition
d) The measurement need defined by patients should drive the decision to utilize devices
in chronic disease management.
Page | 29
S03-4:
Many patients utilize wearable devices as a means of improving physical activity and improving
their self-management of chronic diseases. Although these devices often include behavioral
strategies such as virtual awards and ongoing feedback, they often lack the critical elements
required to enhance behavior change. Which behavior change element is frequently lacking in
the wearable activity experience?
a) Self-monitoring
b) Behavioral instruction
c) Goal-setting
d) Feedback
S03-5:
The popularity of health self-tracking through the use of consumer health devices has primarily
been confined to fitness and wellness contexts, but there has been increasing interest in
leveraging this health data in clinical decision-making and care delivery. Why?
a) Personal tracking devices easily interface with electronic medical records
b) Personal tracking devices provide a highly granular picture of health for an individual
c) Personal tracking devices are generally covered by insurance
d) Personal tracking devices are well understood by physicians
S03-6:
Privacy is an issue that has been discussed at length in a range of fields, including law,
philosophy, and medicine. What is the agreed upon definition of privacy?
a) The well accepted definition of privacy is a legal one
b) Scholars agree that each individuals definition of privacy should be taken as accurate
c) The well accepted definition of privacy is in regards to informational privacy
d) Scholars and experts do not have a universally agreed upon definition of privacy
Page | 30

Capturing Preventive Care Services: Comparing Data Obtained from Manual Chart Review,
Automated EHR Extraction, and Insurance Claims
S.R. Bailey; M. Hoopes; H. Angier; R. Gold; J. Heintzman; M. Marino; J.P. O'Malley; J.E. DeVoe
S09-1-1:
A network of ambulatory primary care clinics with a shared electronic health record is
entertaining several possible quality improvement initiatives. One would involve selecting some
routine preventive services in order to investigate the quality of adult preventive care that is
delivered in their clinics. They are considering various data sources from which to draw their
data. Assuming their EMR is comparable to the one presented, if they chose to use claims data,
which preventive service would be a good choice to assess?
a) Mammography
b) Smoking status assessment
c) Blood pressure screening
d) BMI assessment
S09-1-2:
data Previously, the clinic used manual chart review for projects such as these. They are
wondering if an automated extraction will be as accurate for a given measure.
Assuming their EMR is comparable to the one presented, which measure might be the best first
choice because of the close agreement between manual and automated chart extraction?
a) Chlamydia screening
b) Cervical cancer screening
c) BMI assessment
d) Cholesterol screening
Page | 31

Comparing Weight Redistribution and Distance Imputation Methods for Missing Data in
Clear-text and Encrypted Record Linkage
Toan C. Ong; Lisa Schilling; Michael G. Kahn
S09-2-1:
Researchers are investigating the association between household income and diabetes
diagnoses among patients between ages 18 and 65. Since diagnoses are stored in the hospitals
electronic health record (EHR) system and income information resides in an external sociodemographic database (e.g. census data), the final dataset used in this study must be created
by linking these two datasets. After reviewing the data elements, four variables which appear in
both data sets are used to link these data sets. These variables are: first name, last name, date
of birth and gender. The personally identifiable information (PII) variables (name and DOB) in
both datasets have to be encrypted prior to data sharing and hence impact the linkage process.
The linkage result was very poor. Only 30% of the patients in the clinical data set have
matching records in the socio-demographic dataset. Which of following statements about the
cause of this result is INCORRECT?
a) Many records in both datasets have typographical errors and the linkage method is
deterministic.
b) Many records in both datasets are missing gender and the linkage method is
probabilistic.
c) There were a lot of missing data in both datasets and the FRIL-0 linkage method was
used.
d) Most of patients in the socio-demographic database belong have age 17 or less
S09-2-2:
Which of the following statements is correct regarding the approaches to improve the linkage
result in the scenario above?
a) The problem with typographical errors can be resolved entirely by using probabilistic
record linkage methods
b) Imputation methods can improve the linkage performance for missing clear-text data
only and will not work with encrypted data
c) Deterministic record linkage method will be improved by improving the accuracy of
gender data.
Page | 32

Rethinking Document Retrieval for Scientific Literature: A Learning to Rank Approach
Jesse M. Lingeman; Hong Yu
S09-3-1:
Previous work in learning to rank scientific documents uses traditional Information Retrieval
datasets to obtain relevance judgments. These traditional methods include ranking by a panel
of experts (as in TREC datasets) and ranking using pseudo-relevance feedback (i.e., by
document similarity). For the learning to rank sub-task of related document search, these
same methods are often used. Instead of relying on these traditional methods, we asked the
authors to rank documents related to their work for us. What did we find?
a) The document ranking by the authors differed significantly from the document ranking
by traditional IR systems.
b) The document ranking by the authors was the statistically the same as the document
ranking by traditional IR systems.
c) Documents ranked by traditional IR systems appeared to be more relevant than the
authors ranking.
d) The authors ranking is not a good indicator of relevance.
S09-3-2:
Learning to rank methods work by decomposing the relationship between a query and a
document into a list of features. These features can include text similarity, document length,
document age, etc. We found two features that were negatively correlated with the rankings
given by the authors. What were they?
a) Citation age and number of citations in the literature
b) Text similarity between abstracts and text similarity between titles
c) The number of times a cited paper is referenced in the query and the text similarity of
the words around each reference with the abstract.
d) The text similarity of the discussion/conclusion with both the title and abstract.
Page | 33
Monday, Nov. 16, 2015- Questions & Answer Options

Alan L. Rector; James R. Campbell; Bedirhan T. Ustun; Christopher G. Chute; Harold R. Solbrig
S13-1:
What is the difference in primary purpose between SNOMED and ICD?
a) Aim for comprehensiveness vs aim for international standardization
b) Use for clinical care vs use for statistical returns
c) Use for clinical care vs use for reimbursement
d) Use for decision support vs use for patient coding
S13-2:
How are residual categories, such as "other" or "not elsewhere classified" dealt with in the new
ICD 11 architecture?
a) They are used throughout the system as has always been the case.
b) They have been eliminated in favor of using the parent category, i.e. the most specific
category that can be fully justified.
c) They exist in the Linearizations in the usual form and are linked if required into the
Foundation Layer using queries.
d) Special legacy codes are provided for backwards compatibility, but they are otherwise
deprecated.
S13-3:
Why do the linearization codes have to be linked to the Foundation Model & Common
Ontology by queries rather than by description logic expressions? For example, why would class
for "Hypertension but NOT Pregnancy" expressed in a description logic for the Common
Ontology not capture the intended meaning of "Hypertension excluding Diseases of Pregnancy"
in ICD?
a) Because it could not be queried with the usual ICD tools.
b) Because the meaning in the description logic would be inconsistent (i.e. unsatisfiable)
because it is open world.
c) Because the intended meaning in SNOMED is "not classified under in SNOMED" which is a
closed world query.
d) Because the intended meaning in ICD is "not classified under in this llinearization" which
is a =meaningless in the Common Ontology & SNOMED.
Page | 34
S13-4:
What is meant by for the SNOMED and Common Ontology hierarchies to be reconciled (i.e.,
coherent)?
a) Every corresponding concept has the corresponding parents and children
b) One is a sub graph of the other
c) For every concept in one for which there is a corresponding concept in the other, no
ancestor of the original concept corresponds only to non-ancestors of the corresponding
concept
d) For every concept in one for which there is a corresponding concept in the other and for
which the ancestors correspond, the children also correspond
S13-5:
What is the difference in the information contained in the "Content Model" and Common
Ontology of ICD-11?
a) The Common Ontology contains information that is generally true but to which there may
be exceptions The content model contains only information that is necessarily and always
true.
b) The Common Ontology contains only information that is necessary and always true; the
content model contains information that is generally true but to which there may be
exceptions.
c) The Content Model is structured around signs and symptoms and the Common Ontology
around anatomy.
d) The Content Model is crowd-sourced while the Common Ontology is built by experts.
S13-6:
What is the primary reason that ICD, up to version 10, used a mono-hierarchy and SNOMED
uses a polyhierarchy?
a) Because early developers preferred mono-hierarchies. SNOMED was developed later.
b) Because statistical requirements for ICD required that at each level the cases sum to
100% whereas in SNOMED the hierarchies were determined by logical inference that could
infer one concept to be child of several other concepts.
c) Because ICD was written on paper and SNOMED developed for computers.
d) Because SNOMED was concerned that it be easy to find, for example, all forms of heart
disease, whereas ICD was concerned to maintain traditional distinctions, e.g. between
congenital and degenerative heart diseases.
Page | 35

A Road Map for a National Health Information Technology Safety Center
Douglas Johnston; Andrew Gettinger; Kathy Kenyon; Stephanie Rizk; Colene Byrne; Linda
Dimitropoulos
S20-1-1:
Public and private sector organizations are involved in a range of efforts to ensure and improve
the safety of activities across industries such as transportation, agriculture, energy,
manufacturing, drug development, health care delivery and others. Historically, these efforts
have used a variety of organizational models: from governmental regulatory agencies to private
sector oversight bodies.
What does the Roadmap propose as the organizational model for a national Health IT Safety
Collaborative/Collaboratory?
a) A federal regulatory agency with rule-making and enforcement powers
b) A private, non-profit patient safety or health care accreditation organization
c) A public-private partnership comprised of health IT safety stakeholders
d) An international standards development organization
S20-1-2:
Since 2011, the Office of the National Coordinator for Health Information Technology (ONC) has
supported or led a range of initiatives to understand and respond to increasing evidence of
health IT safety-related events. The most recent of these efforts the Health IT Safety
Collaborative Roadmap defines objectives for a proposed Collaborative/Collaboratory. What
are these objectives? (Select all that apply.)
a) To regulate health IT vendors, and to investigate health IT-related safety events
b) To improve the safety of health IT, and to use health IT to make care safer
c) To create safety-related criteria for the Meaningful Use incentive program
d) To include health IT safety as part of a learning health system, and to improve safety
cultures
Page | 36A
S20-1-3:
In a study of data from the Pennsylvania Patient Safety Authority (See reference 1 below),
hospitals adopting advanced electronic health records (EHRs):
a) Experienced a 27% overall reduction in reported patient safety events.
b) Found a 20% decline in medication errors
c) Found a 15% decline in procedure-related errors.
d) Did not show any improvements relative to basic EHRs.
Page | 36B

Analysis and Classification of Patient Safety Reports in Computerized Prescriber Order Entry
(CPOE) Systems and Refinement of a New Taxonomy for Classification of CPOE-Related
Medication Errors
Mary G. Amato; Alejandra Salazar; Thu-Trang Hickman; Arbor J. Quist; Lynn A. Volk; Adam
Wright; Dustin S. McEvoy; Sarah P. Slight; David W. Bates; Gordon Schiff
S20-3-1:
Which of the following is an example in which the Computerized Provider Order Entry (CPOE)
system facilitated (actively contributed to) an error?
a) A nurse practitioner enters an order for a medication which dangerously interacts with
another medication and does not receive a warning or alert about the interaction.
b) A physician in clinic enters a prescription and enters a new pharmacy requested by the
patient, but when the order is completed, the old pharmacy information auto-populates
the pharmacy field and the prescription is electronically sent to the wrong pharmacy.
c) When medication orders are entered electronically for a patient upon hospital
discharge, one of the pre-admission home medications the patient was taking is omitted
from the orders because the ordering resident overlooked it on the admission list.
d) A resident orders an extended release medication meant to be dosed once per day for
the patient to receive three times a day because they confused the medication with the
immediate release formulation.
S20-3-2:
Which of the following statements is true about what was found in this review of error reports
from several CPOE systems?
a) Report narratives generally followed a standardized format that was consistent across
sites.
b) A taxonomy category for what happened to the patient and what happened when
entering the order in the CPOE was able to be easily determined for each case.
c) Review of the error reports enabled the iterative development of a taxonomy to classify
CPOE-related medication errors.
d) Wrong patient errors were the most common type of CPOE related error in this study.
Page | 37

Leveraging Health Information Exchange to Create Neighborhood Health Records for Public
Health Agencies
Brian E. Dixon; P. Joseph Gibson; Karen F. Comer; Jian Zou; Jennifer L. Williams; Marc Rosenman
S21-1-1:
Many health systems are either contemplating or investigating how to incorporate social
determinants of health into their electronic health record (EHR) systems. What are social
determinants of health?
a) Societal constructs designed to classify sub-populations such as race, ethnicity, and
sexual orientation.
b) Conditions in the environments in which people are born, live, work, play, and age that
affect a wide range of health, functioning, and quality-of-life outcomes and risks.
c) Risk factors related to socially-oriented behaviors such as smoking, unprotected sex, or
binge drinking.
d) Environmental factors that influence health outcomes such as air quality, water quality,
and nearness to a food desert.
S21-1-2:
Which national academy recommends incorporating social determinants of health into
electronic health record (EHR) systems?
a) Institute for Healthcare Improvement
b) Integrating the Healthcare Enterprise
c) Institute of Medicine
d) Institute for Environmental Medicine
Page | 38

The SDIDS System for Integrating Global Health Surveillance Data: An Example Application to
Malaria Surveillance in Uganda
Kate Zinszer; Anya Okhmatovskaia; Arash Shaban-Nejad; Lauren N. Carroll; Neil F. Abernethy;
David Buckeridge
S21-2-1:
The manager of Monitoring and Evaluation at a national malaria control program (NMCP) is
interested in comparing the timing and location of interventions deployed by partners in a
district that is currently experiencing an outbreak of malaria cases. Interventions include the
distribution of long-lasting insecticide treated nets (LLINs) to at-risk groups for malaria (e.g.,
children < 5, pregnant women) and indoor residual spraying (IRS), which is insecticide spraying
for mosquitoes in houses. In particular, the manager would like to understand the timing of the
interventions and to investigate if there has been a positive effect of the interventions on the
outbreak, to date. What is the most likely barrier for this analysis?
a) The national surveillance system for malaria does not permit analysis by place and time.
b) The partners of the NMCP are not willing to share the intervention data.
c) The intervention data and malaria morbidity data are housed by separate systems and it
would be very difficult to share the data in a timely manner and also conduct the
analysis in a timely manner.
d) The effectiveness of LLINs and IRS is highly questionable.
S21-2-2:
A central characteristic of the Scalable Data Integration for Disease Surveillance (SDIDS)
software platform is its ability to scale-up and integrate data from other geographical regions
and for other priority diseases. This means that a wide range of data sources can be mapped
once to SDIDS and then accessed and analyzed repeatedly by a wide range of global health
users. To enable such scalability, SDIDS relies on the use of a software ontology, which provides
a uniform data representation framework. What is the best definition of an ontology, in this
context?
a) An ontology is a convention for naming concepts/types, properties, and
interrelationships for a particular domain of knowledge.
b) The branch of metaphysics dealing with the nature of being.
c) An ontology can be viewed as a graph of information for a particular domain, with
relationships as nodes of the graph and concepts (terms) as links that connect the
relationships.
d) An ontology is a formal taxonomy of terms augmented with the relationships between
concepts expressed in formal logic.
Page | 39
Quality Improvement
Kensaku Kawamoto; Marc J. Hadley; Thomas A. Oniki; Julia Skapik
S25-1:
You have been appointed Chief Medical Information Officer for your health system and have
been tasked with formulating your institutions strategy for clinical decision support. How
should you incorporate electronic clinical quality measurement into this overall strategy?
a) Do not include clinical quality measurement clinical decision support and clinical
quality measurement are completely unrelated.
b) Establish two independent, unrelated strategies: one for clinical decision support and
one for clinical quality measurement.
c) Implement a single coordinated strategy across both clinical decision support and
clinical quality measurement, with harmonized standards and approaches used to the
extent possible.
d) Ensure that the clinical decision support team does not waste time implementing
related quality measures, since clinical decision support interventions always work and
their impact never need to be measured.
S25-2:
Your colleague just moved from the clinical quality measurement team at your institution to the
clinical decision support team. She noticed how the approaches taken and technologies used
are quite different between the two teams. She heard you went to a discussion on the topic of
decision support and quality measurement standards and asks you why the two approaches are
so different. What should you tell her which would be correct?
a) Clinical decision support and clinical quality measurement have fundamentally different
requirements related to the data model.
b) Clinical decision support and clinical quality measurement have fundamentally different
requirements related to the metadata.
c) Clinical decision support and clinical quality measurement have fundamentally different
requirements related to the expression language.
d) Clinical decision support and clinical quality measurement standards have historically
been developed independent of one another.
Page | 40
S25-3:
An IT developer in your organization is really impressed with the HL7 FHIR standard and how
easy it is to use. When you mention that you went to a discussion on how people involved in
clinical decision support and clinical quality measurement are developing FHIR profiles, he asks
you why FHIR resources arent just being used without any profiles. What should you tell him
which would be correct?
a) Having a lot of models gives us choice and is therefore good for interoperability.
b) FHIR resource instances cannot communicate data attributes that are not explicitly
included in the resource definition.
c) FHIR profiles allow for the definition of additional constraints compared to base FHIR
resources (e.g., the use of a LOINC code to identify a laboratory result), thereby
facilitating semantic interoperability.
d) Only FHIR profiles can leverage a REST application programming interface.
S25-4:
A colleague asks you what the core motivations are behind the Clinical Quality Framework
(CQF) initiative. What should you tell her that would be correct?
a) The CQF initiative is seeking to reduce duplicate effort for clinical decision support and
clinical quality measurement implementation.
b) The CQF initiative is seeking to define a minimum data set for exchanging continuity of
care documents.
c) The CQF initiative is seeking to make it easier to integrate biometric device data into the
electronic health record.
d) The CQF initiative is seeking to define a large corpus of clinical decision support rules for
clinical use.
S25-5:
A developer in your organization is implementing a clinical decision support execution
framework and asks you if you have head of any clinical expression language standards that
could be leveraged for such a framework. What could you tell her is a clinical expression
language standard developed through the Clinical Quality Framework initiative?
a) The GELLO standard
b) The Arden Syntax standard
c) The FHIR standard
d) The Clinical Quality Language standard
Page | 41
S25-6:
Your CMIO is tracking developments in the health IT standards space. She asks you how the
Clinical Quality Framework (CQF) initiative relates to the Health eDecisions (HeD) initiative.
What should you tell her that would be correct?
a) The CQF initiative is taking the clinical decision support standards developed by the HeD
initiative and harmonizing those standards with electronic clinical quality measurement
standards.
b) The CQF initiative is taking the electronic clinical quality measurement standards
developed by the HeD initiative and harmonizing those standards with clinical decision
support standards.
c) The HeD initiative is taking the clinical decision support standards developed by the CQF
standards.
d) The HeD initiative is taking the electronic clinical quality measurement standards
developed by the CQF initiative and harmonizing those standards with clinical decision
support standards.

Health Literacy, Education Levels, and Patient Portal Usage During Hospitalizations
Sharon E. Davis; Chandra Y. Osborn; Sunil Kripalani; Kathryn Goggins; Gretchen P. Jackson
S37-1-1:
Patient portals are online tools supporting patients and their families as they interact with their
health care information and providers. In a retrospective study of portal use among
hospitalized patients at Vanderbilt University Medical Center, which factor was associated with
portal use during hospitalization?
a) Educational attainment
b) Health literacy
c) Socioeconomic status
d) Years of employment
Page | 42
S37-1-2:
hospitalized patients at Vanderbilt University Medical Center, use of which portal function was
associated with educational attainment?
a) Secure messaging with health care providers
b) Managing clinic appointments
c) Viewing of electronic health information
d) Accessing targeted health education materials
Strategies for Managing Mobile Devices for Use by Hospitalized Inpatients
P.C. Dykes; D.L. Stade; A.K. Dalal; S. Collins; M.Clements; F.Y. Chang; A. Fladger; G. Getty; J.R.
Hanna; R. Kandala; L. Lehmann; K. Leone; A.F. Massaro; E. Mlaver; K. McNally; S.Ravindran; K.
Schnock; D.W. Bates
S37-2-1:
Which of the following strategies is not consistent with a socio-technical approach?
a) End-users with an understanding of tasks and environments are closely involved.
b) The approach is linear with a stage gate between each phase where requirements are
reviewed and approved before continuing to the next phase.
c) The approach sheds new light on the potential roles of IT applications in health care
practices.
d) The design team includes multidisciplinary skills and perspectives.
S37-2-2:
Which of the following issues can be addressed after deploying mobile devices in hospital
(inpatient) settings?
a) Mobile device storage protocol
b) Mobile device charging strategy
c) Mobile device infection control protocol
d) Providing individual patients with secure access
Page | 43

Content and Usability Evaluation of Patient Oriented Drug-Drug Interaction Websites
Terrence J. Adam; Joe Vang
S37-4-1:
The expert consensus on the level of evidence for the clinical risk for most drug-drug
interactions is?
a) High quality with randomized control trial data to estimate clinical risk for most drug-drug
interactions
b) Has a high degree of consistency no matter which reference source or drug database is
utilized
c) Is often based on clinical case report data and drug class effects with limited information
on estimated actual risk of adverse events
d) Readily available in a comprehensive form in several public source databases
S37-4-2:
Consumer oriented drug-drug interactions websites are characterized by?
a) Consistent reporting of clinical risk data on potential drug-drug interactions
b) Appropriate grade level content on the textual descriptions of drug-drug interaction risk
c) Comprehensive estimates of multiple drug interactions
d) Variable use of patient friendly interface features and quantitative drug-drug interaction
risk communication
Page | 44

Understanding Challenges and Opportunities in Precision Medicine and Interoperability Using
Informatics Approaches
Jay Geronimo Ronquillo; Chunhua Weng; William T. Lester
S41-1-1:
An informaticist is designing a Precision Medicine reporting module that integrates patient
genomic results into their EHR system. They want to make sure that it is consistent with past
and/or existing certification criteria for EHRs. Which of the following
standards/implementation guides relating to genetic information exchange would be a good
starting point to making the reporting module more likely to be interoperable for Precision
Medicine?
a) HL7 Genetic Test Reporting
b) HL7 Family Health History
c) Global Alliance for Genomics and Health
d) HL7 Genetic variation
e) Fast Health Interoperability Resources (FHIR)
S41-1-2:
The Precision Medicine Initiative was announced at the beginning of 2015 to transform
healthcare by combining diverse patient data (clinical, genomic, etc) to provide innovative
and personalized insight into disease diagnose, treatment, and even prevention. Which of the
following clinical condition(s) represents the most important immediate priority for the
Precision Medicine Initiative?
a) Diabetes
b) Cancer
c) Mendelian diseases
d) Diseases affecting underrepresented populations
e) Cardiovascular disease
Page | 45

Early Experiences with Meaningful Use and Online Portal Implementation among
Providers/Staff and Patients/Caregivers in a Safety Net Healthcare System
Courtney Lyles; Lina Tieu; Dean Schillinger; Neda Ratanawongsa; Urmimala Sarkar
S41-2-1:
What is the Meaningful Use Stage 2 metric for patient engagement with online portals?
a) 50% registered and 5% viewing, transmitting, or downloading information
b) 25% registered and 5% viewing, transmitting, or downloading information
c) 50% registered and 15% viewing, transmitting, or downloading information
d) 100% registered and 15% viewing, transmitting, or downloading information
S41-2-2:
What is the Meaningful Use standard/recommendation for use of portals in languages other
than English?
a) Portals must be available to patients in the top three languages used within that
healthcare system
b) Portals must be available in another language if >10% of the patient population prefers
a language other than English
c) There is no specific guidance about portals related to language
d) Portals should only be provided in English
Physician Participation in Meaningful Use and Rehospitalization of Medicare Fee-for-Service
Enrollees
Mark A. Unruh; Hye-Young Jung; Joshua R. Vest; Lawrence Casalino; Rainu Kaushal
S41-3-1: If Meaningful Use is associated with lower overall rates of hospitalization among
patients of participating physicians, how would it affect our estimates (of the impact of
physician participation in Meaningful Use on the likelihood of readmission)?
a) It would likely not have any effect on the estimates.
b) It would suggest that specialists are more likely to participate in Meaningful Use
compared to primary care physicians.
c) It would bias estimates towards the null.
d) It would bias estimates away from the null.
Page | 46
S41-3-2: Why was it important to also examine mortality in a study of physician participation in
Meaningful Use and readmissions?
a) Differing mortality rates between patients of Meaningful Use participants and nonparticipants may bias estimates.
b) There is no relationship between mortality and readmission, but it is an interesting
outcome to examine.
c) Hospitals with high mortality rates are not allowed to participate in the Meaningful Use
incentive programs.
d) Meaningful Use specifically targeted physicians and hospitals with high mortality rates.
Are Meaningful Use Requirements Really Meaningful for Medication Use? Experiences from
the Field and Future Opportunities
Sarah P. Slight; Eta S. Berner; William Galanter; Stanley M. Huff; Bruce L. Lambert; Carole
Lannon; Christoph U. Lehmann; Brian J. McCourt; Michael McNamara; Nir Menachemi; Thomas
Payne; Stephen A. Spooner; Gordon Schiff; Tracy Y. Wang; Ayse Akincigil; Stephen Crystal;
Stephen P. Fortmann; Meredith L. Vandermeer; David W. Bates
S41-4-1: Which of the following EHR functionalities do Stage 3 Meaningful Use objectives
specifically mention or recommend?
a) EHR functionalities to assist with the prescribing of medications for children
b) EHR functionalities to help maintain an up-to-date accurate medication list
c) EHR functionalities to obtain patient input into the reconciliation of problems on their
problem list
d) EHR functionalities to code contraindications
S41-4-2: Which one of the following statements about Stage 3 Meaningful Use
Recommendations is correct?
a) The implementation of Meaningful Use capabilities can sometimes delay other EHR
development projects planned in organizations
b) All organizations should face the same levels of difficulty when implementing
Meaningful Use recommendations
c) The use of patient portals by patients living in rural areas is unlikely to be a challenge in
the future
d) No funding has been provided to prepare future professionals to meet emerging
workforce HIT needs
Page | 47

Six Important Characteristics for Patient Hand-Off Application in Inpatient Hospital Setting
Soleh U. Ayubi; Alexandra Pelletier
S42-1-1:
What is the most important reason paper checklist is not appropriate tool for hand-off
collaborative patient care?
a) Lack or limited of integration with electronic medical records (EMRs).
b) Security and privacy aspects.
c) May not be supported by local institutions.
d) Static, redundant, unstandardized, and unshared medium.
S42-1-2:
A few mobile checklist applications for patient care are available on application markets (Apple
App Store and Google Play). What is the strongest reason their adoptions are low?
a) Lack or limited of live-tracking components.
b) Lack or limited of integration with electronic medical records (EMRs).
c) Security and privacy aspects.
d) Redundant and unstandardized medium.
Page | 48

A Review and Analysis of Rounding and Handoff Document Content in Inpatient Resident
Physician Teams
Elliot G. Arsoniadis; Rohini Khatri; Jenna Marquard; Courtney Moors; Michael Kim; Genevieve B.
Melton
S42-2-1:
Of the following statements, which best encompasses the current status of standards related to
handoff document architecture?
a) Handoff document standards should conform to HL7 Clinical Document Architecture
Continuity of Care Document standard.
b) While no official standards for handoff documentation currently exist, the literature
does have some "common themes" that are broadly accepted. In addition, standards from
other clinical documents, such as HL7 Clinical Document Architecture Continuity of Care
Document, provide many overlapping standards that can be utilized by the handoff
document.
c) Standards for the Handoff document are not possible, given the many variations
present, even within a single institution's experience.
d) Standards should be dictated by functionality currently existing within proprietary
Electronic Health Record systems.
S42-2-2:
While the data needs of physician may vary by specialty and level of training, what common
themes have emerged from this initial study for constructing an ideal handoff document?
a) Highly granular data related to all organ systems should be included.
b) Only a broad summary of hospitalization is required by most in-training physicians.
c) Medication lists are highly important to enact care by on-call physicians and must be
included on all documents used for handoff/rounding purposes.
d) Pithy, concise patient summaries are valuable pieces of information. Time/Date
information for data, whether manually entered or automatically imported, is important
for ensuring trustworthiness of data by end users.
Page | 49
S42: Papers/Podium Presentations - Care Team Communication

Improving Care Team Communication: Early Experience at Implementing a Patient-centered
Microblog
Anuj K. Dalal; Jeffrey L. Schnipper; Anthony F. Massaro; Kelly McNally; Patricia C. Dykes; David
W. Bates
S42-3-1:
Key attributes of a microblog used to manage a patient-specific communication in a clinical
setting include all of the following EXCEPT:
a) Availability on web-based and mobile platforms
b) Identification of all members of the patients care team
c) Transparency and persistence of the electronic dialog
d) Asynchronous and real-time notifications when new content is posted
e) Limited to text posts alone
Vojtech Huser; Alexa McCray; Neil R. Smalheiser; Asba Tasneem; Chunhua Weng
S48-1:
Which of the following statements about ClinicalTrials.gov is correct?
a) We cannot identify the geographic distribution of open trials.
b) We cannot identify the common characteristics of target populations for a specific
disease domain, such as Type 2 diabetes.
c) We cannot identify the common patterns in clinical trial participant selection across
studies or across disease domains.
d) We cannot discover novel disease biomarkers using data from ClinicalTrials.gov.
S48-2:
ClinicalTrials.gov is a U.S. Government database. As such, which of the following statements
about the data contained in ClinicalTrials.gov is true?
a) All ClinicalTrials.gov data can be freely downloaded, used, and distributed with no
restrictions.
b) Data deposited by U.S. entities in ClinicalTrials.gov can be freely downloaded, used, and
distributed, but there is a charge for downloading data deposited by international
entities.
c) ClinicalTrials.gov data can be downloaded, used, and distributed with certain
restrictions.
d) Commercial entities may not download, use, and distribute ClinicalTrials.gov data.
Page | 50
S48-3:
ClinicalTrials.gov has now been in existence for over 15 years. Which of the following
statements is true about the genesis of ClinicalTrials.gov?
a) The Food and Drug Administration (FDA) and the National Institutes of Health (NIH),
sister agencies within the US Department of Health and Human Services, recognized
that there was a need to better track clinical trials data and determined that the best
way to do that would be to create a national database of clinical trials information.
b) In response to the requirement by the International Committee of Medical Journal
Editors (ICMJE) that investigators register their trials in a publicly available database as a
condition of publication, the National Library of Medicine created ClinicalTrials.gov.
c) In response to a statement by the World Health Organization (WHO) that all clinical
trials should be registered, the US Department of Health and Human Services, as a
multilateral partner of WHO, directed the National Library of Medicine to create
ClinicalTrials.gov.
d) In response to pressure from patient advocacy groups and others, Congress passed a
law to create ClinicalTrials.gov.
S48-4:
Full trial registration transparency means that all trials are registered. Full results transparency
means that all trials publish their results either by PubMed indexed article or summary results
deposition into a registry. Which of the following statements best fits the reality:
a) Approximately: 40 % of interventional trials are registered; 10% of observational trials
are registered; and 30% of results are published.
b) It is impossible to know with certainty about unregistered trials because many
unregistered trials never publish. Similarly it is difficult to know proportion of trials with
published results accurately.
c) AllTrials.net offers the best view of trials and it indicates that 55% of all trials are
registered and 35% of trials publish their results somehow.
d) Even though many countries do not have a legal mandate to register trials, the ICMJE
requirement to register any trial (including observational trials) is increasing the trial
transparency and it has increased from 10% to 42% since ICMJE introduction. Majority
of journals follow the ICMJE policy on trial registration.
Page | 51
S48-5:
What best describes how publications are linked to clinical trials:
a) Via a search engine (eg, Google Scholar), publications can be linked to trials because the
article will reference the trial name and abbreviation.
b) National Library of Medicine extracts trial IDs from article full text and creates a
structured link.
c) National Library of Medicine extracts trial IDs from two out of 10+ WHO primary
registries from article abstracts and creates a structured link for those two registries.
d) Publishers collect a structured data entry during manuscript submission for the related
clinical trial and submit this to PubMed in XML format (following the JATS schema
format).
S48-6:
Which format should be selected for downloading All Study and Results fields from
ClinicalTrials.gov?
a) XML
b) SQL
c) Plain Text
d) CSV

Conversational Agents for Automated Inpatient and Outpatient Health Counseling
Timothy Bickmore
S54 -1-1:
What patient populations respond especially well to conversational agents?
a) Children and young adults
b) Patients with low health, reading, or computer literacy
c) Patients who love technology and gadgets
d) Computer gamers
Page | 52
S54-1-2:
Why should conversational agents simulate social chat and other relational behavior?
a) To entertain patients
b) To comfort extroverted patients who like to engage in lengthy conversation
c) To acclimate patients to the technology
d) To build therapeutic alliance to increase intervention adherence and retention
S54-1-3:
What does simulated hand gesture add to a conversational agent's ability to communicate
health information?
a) Conveys the agent's personality
b) Maintains the patient's attention on the intervention content
c) Conveys additional semantic information and regulates the conversation
d) Conveys the agent's gender and race
OHDSI: An Open-Source Platform for Observational Data Analytics and Collaborative
Research
Jon Duke; Frank DeFalco; Chris Knoll; Vojtech Huser; Richard D. Boyce; Patrick B. Ryan
S54-2-1:
Which of the following is NOT a goal of the OHDSI initiative?
a) facilitate clinical characterization of electronic datasets
b) generate evidence based on population level estimation
c) provide clinical decision support
d) develop methodologies for patient level prediction
S54-2-2:
What are the requirements to submit an initial proposal to the OHDSI network?
a) Funding source and protocol
b) Protocol and distributable code
c) Funding, protocol, and distributable code
d) Community forum post and preliminary protocol
Page | 53
Tuesday, Nov. 17, 2015 - Questions & Answer Options

The Best of Imaging Informatics Research 2015
Charles E. Kahn; Bradley Erickson
S57-1:
In "Rethinking Radiology Informatics," Kohli et al. argue that:
a) Radiologists' role in health care IT should focus on managing imaging technology and
selecting appropriate IT products.
b) Radiologists will be expected to provide and manage information about the entire
imaging process.
c) Radiology should focus on clinical issues, and leave IT decisions to executive leadership.
d) There is no need to incorporate decision support or clinical recommendations into
radiology reports.
S57-2:
A study of radiology residents' use of structured reporting for chest radiographs found that:
a) Structured reports were more complete and more effective than traditional,
unstructured reports.
b) The use of structured reports was detrimental to resident education.
c) The specific implementation of reporting software has no effect on the accuracy and
completeness of radiology reports.
d) Imaging procedures such as chest radiography have too complex a vocabulary to permit
structured reporting.
S57-3:
The report by Hodge et al. on the ConnectomeDB project states that:
a) ConnectomeDB includes magnetic resonance imaging (MRI) data, but does not yet
include magnetoencephalograpy (MEG) or associated behavioral data.
b) Imaging and associated data are made available through bulk file transfer protocol (FTP)
download of the entire dataset.
c) The system is based on an extensible, open-source platform for managing and sharing
imaging and related data.
d) Users must perform validation of the imaging data, as the database only stores "raw"
information without any validation steps.
Page | 54
S57-4:
Deep Learning is being applied in medical imaging applications for many reasons. Which is
the LEAST significant factor in this growth:
a) Deep Learning neural networks take advantage of newer computing hardware (GPUs)
b) Lack of large labeled image data sets for training
c) Deep learning on images avoids the requirement for direct feature computation
d) Deep learning can find specific objects in complex photographs, and perhaps in medical
images
S57-5:
Significant advances for the use of Quantitative Imaging include all but:
a) Creation of large, curated, and annotated data sets with clinical data and outcomes
b) Careful evaluation of the reproducibility of quantitative imaging methods across
multiple centers for several modalities
c) Creation of a Network of investigators to study the appropriate use of QI methods in
specific diseases
d) Rapid adoption of quantitative imaging methods in most areas of radiological practice
Page | 55
Brian E. Dixon; Jamie Pina; Janise Richards; Hadi Kharrazi; Anne M. Turner
S58-1:
Informatics has struggled to amass a robust set of evidence on the effectiveness of electronic
health record systems to make demonstrable improvements in patient outcomes. The same is
true for public health informatics where governmental public health agencies have struggled to
evaluate the effectiveness of information systems. Major barriers preventing high quality
studies include the costs associated with evaluation as well as the complexity involved with
robust study designs such as large, randomized controlled trials (RCTs). Given the need for
more evidence but limited funding for evaluation, which of the following study designs best
balances robustness with complexity and costs making it the most palatable to public health
leaders or health care system administrators?
a) 5-year RCT linking the implementation of health information exchange on duplicate
imaging studies across a large accountable care organization.
b) 2-year before-after study with no control group to evaluate the implementation of an
electronic prescribing application in a series of public health clinics.
c) 3-year controlled before-after study to evaluate the introduction of preventative
reminders in safety net primary care clinics using late adopters as control sites for
earlier implementations.
d) A survey of provider satisfaction with updated computerized provider order entry
modules recently deployed across a large, integrated health system.
S58-2:
Significant implementation activity in the United States has been focused on systems that meet
the requirements of the Centers for Medicaid and Medicare Services (CMS) Meaningful Use
program. In the realm of public health informatics, this has meant a focus on systems that
connect public health agencies with health care providers. Which of the following clinical
information systems is an example of an implementation that would establish infrastructure to
move data to and/or from public health agencies?
a) Electronic laboratory reporting of clinical labs from the lab information systems to the
b) Master patient index to resolve and assign unique identifiers to the entire population of
patients covered by an accountable care organizations.
c) Emergency department information system interface to report chief complaints and
patient demographic information for all individuals registered for care.
d) Clinical decision support system to alert primary care physicians when a sub-population
of patients with diabetes report significant changes in weight.
Page | 56
S58-3:
Imagine a scenario in which nearly all of the ambulatory and inpatient facilities within a
community routinely share data as part of a health information exchange (HIE). The HIE, in turn,
reports a subset of data routinely to the local public health department, which monitors the
data for changes in disease trends (e.g., active surveillance). While reviewing routine
surveillance reports, an epidemiologist at the health department notices a spike in
gastrointestinal symptoms that correlates with an increased positivity rate in stool cultures
from the lab for Shigella. The epi quickly pushes an electronic alert via the HIE out to providers,
which stimulates increased ordering of stool cultures to monitor a suspected outbreak of
Shigella. Within a week, the epidemiologist not only confirms the outbreak but also identifies
that a subset of providers are treating the outbreak with antibiotics. Since Shigella is often
resistant to antibiotics, the epi distributes treatment guidelines from the Centers for Disease
Control and Prevention (CDC) using the HIE. Antibiotic rates decline and disease levels dissipate.
This example in which the health system was able to gather and analyze routine electronic
information to rapidly inform population health as well as clinical practice could be said to meet
the ideals of which vision espoused by the Institute of Medicine?
a) The Rapid Learning Network
b) Patient Centered Outcomes Research Network
c) Electronic Learning Network for Response to Disease Outbreaks
d) The Learning Health System
S58-4:
The major theme the Global Health Informatics (GHI) publications in 2013 and 2014 was:
a) Inclusion of artificial intelligence initiatives to determine trends in global disease
outbreaks
b) Telemedicine applications used to deliver healthcare in limited resource locations
c) Geographic information systems that assist environmental health researchers depict
global health issues
d) The deployment of cloud computing to assist low and middle income countries (LMIC) in
developing decision support systems
S58-5:
Changing the search strategy within PubMed between the 2013 and 2014 literature reviews led
to greater:
a) Sensitivity to the search that produced a larger pool of articles
b) Specificity to the search that produced a smaller pool of articles
c) Focus on LMICs (Low-to-Middle Income Countries) in the search that produced a smaller
pool of articles
d) Focus on themes in the search that produced a larger pool of articles
Page | 57
S58-6:
In examining the methodologies used in the 2013 and 2014 articles, we found most studies
were:
a) Random control trials
b) Cross-sectional descriptive studies
c) Longitudinal step-wise control trials
d) Systematic review studies
S59: Interactive Panel - What Could Go Wrong?
Migrating from One EHR to Another
Richard Schreiber; Ross Koppel; Catherine K. Craven; John D. McGreevey
S59-1:
What is the current approximate percentage of physicians who have installed second, third, or
more EMRs?
a) 10%
b) 20%
c) 30%
d) 40%
S59-2:
What is considered the most complex task involved in changing from one EMR product to
another?
a) Creating new interfaces
b) Data conversion
c) Upgrading hardware for required functions
d) Choosing the vendor in the first place
S59-3:
When switching from one EHR to another EHR, a key step is:
a) getting buy-in from most of the pharmacists
b) getting buy-in from most of the physicians
c) getting buy-in from most of the nurses
d) explaining the decision's processes and rationale.
Page | 58
S59-4:
If one converts from one instance EHR of vendor X to another of the same vendor, what are the
concerns about compatibility compared to switching vendors?
a) If one stays within vendors, one can be confident that the data will map reasonably well
to the new system, compared to switching vendors
b) If one shifts between vendors, one can be almost certain that the data will not map well,
compared to staying with the same vendors
c) Staying with the same vendor or not does not predict the certainty of data compatibility
d) Compatibility of the data are primarily determined by the implementation process, even
if one stays with the same vendor
S59-5:
Which of the following sequences represents the most appropriate approach to migrating from
one EHR to another?
a) Define current and future state approach to new EHR; Train on new EHR; optimization;
Align and standardize hospital/health system workflows and processes
b) Align and standardize hospital/health system workflows and processes; Define current
and future state approach to new EHR; Train on new EHR; optimization.
c) Define current and future state approach to new EHR; Align and standardize
hospital/health system workflows and processes; Train on new EHR; optimization.
d) Define current and future state approach to new EHR; Align and standardize
Page | 59
S59-6:
Of the following, which of the aging-related sociotechnical factors will impact future migration
to new electronic health record systems (EHRs) at Critical Access Hospitals?
1. Retiring CEOs
2. Retiring Directors of Nursing
3. New nurses who have not implemented EHRs before
4. The age-related health status of IT Directors
a) 1
b) 4
c) 1, 2, and 3
S59-7
CAHs typically had to pay at least what percent of their Total Cash-on-hand for their EHR
systems?
a) 100%
b) 80%
c) 30%
d) 25%
Jessica D. Tenenbaum; Greg Biggers; Bradley Malin; Lucila Ohno-Machado; Leslie Wolf
S60-1:
What is the difference between de-identified and anonymous datasets?
a) De-identified datasets cannot be re-identified, but anonymous datasets can
b) Anonymous datasets cannot be re-identified, but de-identified datasets can
c) Anonymous datasets may include dates and zip codes, but de-identified datasets may
not
d) De-identified data sets may include dates and zip codes, but anonymous datasets may
not
Page | 60
S60-2:
Which of the following statements is true under the "safe harbor" method for de-identification?
a) De-identified data sets may contain encounter dates, but not date of birth
b) De-identified data sets may not contain any part of dates or zip codes
c) De-identified data sets may contain the first 3 digits of a zip code if the corresponding
geographic unit contains more than 20,000 people
d) De-identified data sets may contain zip codes if an expert determines and documents
that the level of granularity implied could not result in identification of an individual.
S60-3:
Under the proposed changed to the Common Rule, what kind of consent is necessary for using
biospecimens for secondary purposes?
a) Specific consent for each secondary investigation
b) Specific consent for different classes of secondary investigations (classes to be defined
by the Department of Health and Human Services)
c) Broad consent for any purpose
d) No consent as long as no identifiers are included
S60-4:
Who can be members of the Data Access Committee at the NIH Database of Genotypes and
Phenotype?
a) Any U.S. citizen
b) Only members of the federal government
c) A mixture of research subject advocates and scientists
d) Members of a centralized institutional review board
Page | 61
S60-5:
Currently, research on existing biospecimens can be conducted without consent under existing
law. The Office for Human Research Protections considers biospecimens without identifiers or
biospecimens shared without identifiers and a promise not to share the identifies to fall within
the Common Rule because it is not human subjects research. Research with identifiable
biospecimens may fall within an exemption under the Common Rule, and, if not, may be
eligible for a waiver of consent under the Common Rule. Which of the following best explains
the reason for permitting research without consent under these circumstances?
a) It would be too difficult to obtain consent under these circumstances.
b) The research poses minimal risk to the people whose biospecimens are used.
c) The research will benefit society as a whole.
d) People do not have a right to control what happens to their biospecimens once they
have been collected.
S60-6:
Re-identification of individuals from so-called de-identified data sets is possible under certain
circumstances. An attacker may want to know whether a person of interest is included in a
particular data set of individuals with a certain condition. Assuming all safe harbor identifiers
(e.g., name, dates, SSN) are removed from a data set, what are possible ways to determine
whether a particular individual of interest is included in a disclosed data set (e.g., registry of
patients with disease X)?
a) Match phenotype information in the data set with external information about the
individual
b) Match rare variants in a genome sequence in the data set with external variant
information about the individual
c) Match demographic and administrative information with external information about the
individual
d) All of the above
Page | 62
Dean J. Karavite; Eric D. Shelov; Levon Utidjian; Jeremy Michel; Eli M. Lourie
S61-1:
You are working with clinical stakeholders to design an alert for sepsis, but are unsure of
whether it should be modal (interruptive) or non-modal (non-interruptive). Your clinical team
has done extensive chart review validating the triggering criteria and has gathered data
identifying patients who would and would not be identified by the criteria, along with those
who did and did not develop sepsis.
Which Statistical measure is the most helpful in determining how interruptive EHR-based
Clinical Decision Support should be?
a) Sensitivity
b) Specificity
c) Positive Predictive Value (i.e. True Positive Rate)
d) Negative Predicative Value (i.e. True Negative Rate)
S61-2:
You are tasked with improving your physicians rates of prescribing generic medications in your
outpatient clinic. You first decide to educate your clinicians with handouts and lunchtime talks.
After two months, you see that the number of generic prescriptions has not changed, so you
decide to implement a clinical decision support tool into your EMR to help with your project.
After another two months, you see that generic prescriptions have increased 20%.
The process described is an example of:
a) Failure modes and effects analysis (FMEA)
b) Root cause analysis (RCA)
c) Lean methodology
d) Plan, Do, Study, Act (PDSA) cycles
Page | 63
S61-3:
A colleague has conveyed to you something must be done to streamline our management
of a common clinical condition. You are presented with sufficient data to determine an
appropriate intervention point and have an idea for an intervention. You wish to gather further
user feedback from the clinical team before going too far with this idea to make sure you are on
the right track. What are the benefits of using a wireframe (i.e. low fidelity) mockup in this
scenario?
a) You can alter the look of the intervention in real time during a stakeholders meeting
b) You can assess the usability and internal workflow of the intervention
c) You can use test data to confirm that the interaction will work correctly with the EHR
d) You can gather feedback about how the system works in practice
S61-4:
You are working to develop a support intervention to aid clinicians in the efficient management
of a common clinical condition. After gathering requirements, identifying a workflow
intervention point, designing the intervention, testing the functionality of the decision support
it is finally time to implement the decision support into production. One of your team members
shouts, Success, whats next. You remind your team member that implementation is not the
end, but just the beginning. Which of the following is an important reason to continue testing a
decision support intervention after implementation?
a) The clinical knowledge used to underpin the guideline may have changed
b) To determine if the system performs as expected with real data
c) To identify additional features and future enhancements for the intervention
d) To correct peoples actions if they are using the intervention incorrectly
S61- 5:
Your team has developed a clinical decision support application. The team wants to validate the
systems usability, but does not have the time and resources to perform usability testing. Which
of the following methods is a valid low cost alternative to usability testing?
a) Focus group
b) Remote usability testing
c) Heuristic review
d) User survey
Page | 64
S61- 6:
You were tasked with developing a documentation tool to help improve rates of provider
documentation of patient tobacco use history to meet a Meaningful Use measure. A few weeks
after go-live, you hear from your data analyst that your hospitals rate of documentation of
tobacco use history has actually worsened. Investigating the issue with a small number of
clinicians uncovers that they had no way of easily knowing if the tool actually saved the history
data they had entered and they were sometimes closing charts and losing unsaved data.
Following which of the below Nielsen usability heuristics could have most easily avoided this
issue?
a) Flexibility and efficiency of use
b) Help and documentation
c) Match between system and the real world
d) Visibility of the system status
Glenn T. Gobbel; Ruth M. Reeves; Wendy Chapman; Dezon Finch; Jennifer H. Garvin
S62-1:
What is the best outcome measure to determine how well your NLP tool captures the true
cases of patients with a disease from a pool of patients?
a) Specificity
b) Sensitivity
c) Negative predictive value
d) Positive predictive value
S62-2:
The inter-annotator agreement scores in a concept recognition task are above 80% for some
concepts, but below 60% for others. Should you:
a) Retrain annotators on a new set of documents
b) Collect disagreeing annotations and obtain consensus among annotators, revising
guidelines with clarifications from consensus understanding
c) Determine whether the low-agreement concepts are prevalent in the test data; retrain
annotators only if these concepts are prevalent enough to significantly degrade the
performance of the NLP system
d) Remove all inconsistent annotations and leave only those where there is agreement
Page | 65
S62-3:
Identify a common characteristic of medical text that presents challenges to natural language
processing systems
a) Clinical text often includes medications with multiple trade names
b) A hospital may treat thousands of patients, and each patients record may contain
numerous documents with unstructured free text
c) The information content of clinical text is sparse compared to what is stored in
databases in structured format
d) Words and phrases in clinical text are often ambiguous
S62-4:
An initial step in designing and developing an NLP system for clinical use is:
a) Decide whether to use a rule-based or statistical NLP system
b) Create a concept schema for manual annotation followed by NLP system training and
testing
c) Define the question to be addressed and the information needed to address the
question
d) Find a source of documents that can be used for system development and training
S62-5:
Information Retrieval (IR) is a statistical technique that can be used to select relevant notes for
use in NLP. What is required before IR can be used to select the notes?
a) A term list must be developed representing the presence of the topic in the notes
b) A sample of the documents must be labeled (positive/negative) for the target concept
c) Notes must be first narrowed by clinic type and provider type
d) A sample of the notes must be stratified by patient demographics to insure an unbiased
selection
Page | 66
Catherine A. Smith; Fabienne C. Bourgeois; Pam Charney; Patricia F. Brennan
S70-1:
What is the principal difference, according to the speaker, between the academic
accommodations process in the kindergarten through grade 12 school system and the process
found in higher education?
a) In higher education settings, the student's parents must contact individual instructors to
request accommodations for the student.
b) In higher education settings, the student must self-identify and disclose the necessary
information to explain the need for the accommodation.
c) In higher education settings, individual instructors review medical documentation
supplied by the student to decide if a requested accommodation is warranted.
d) In higher education settings, no input is required from healthcare professionals; instead,
informal arrangements are made upon request from the student
S70-2:
What is the **largest** set of information from a particular patient's records that could
theoretically be considered "documentation" for the purposes of academic accommodations in
higher education?
a) All available data.
b) Office visit notes from the past year.
c) Hospitalizations in the past 7 years.
d) All the pediatrician's records for this patient from the age of 8.
S70-3:
Which of the following health information for an adolescent patient should be available for her
parents to view in a patient portal?
a) Chlamydia test result
b) Smoking history
c) Doxycycline allergy
d) Toxicology screen results
e) Oral contraceptive prescription
Page | 67
S70-4:
Which of the following PHR user access models allow both parents and adolescents access to
health information while still preserving patient confidentiality?
a) Full shared access control
b) Adolescent access only
c) Parent access only
d) Shared differential access
S70-5:
Which of the following is a major roadblock to receiving appropriate healthcare services for
adolescents who have chronic illness?
a) The HIPAA privacy rule
b) Technical inability to exchange information
c) Lack of clinician awareness regarding existence of roadblocks
d) Local policies regarding adolescent privacy rights
Jonathan M. Teich; Hamish S. Fraser; Eric Perakslis; Shefali Oza; Darius Jazayeri
S71-1:
The most significant special user-interface needs for epidemic disaster information systems
such as those in the Ebola crisis are generated by alterations in which sense, other than touch:
a) Vision
b) Smell
c) Hearing
d) Taste
S71-2:
Treatment centers for infectious epidemics such as Ebola are divided into units by
a) Patient residential location
b) Patient arrival date / length of stay
c) Culture/serology results
d) Patient severity
Page | 68
S71-3:
Which of the following information entry and management tasks can be most easily handled in
the green zone where working conditions are unencumbered by protective personal
equipment?
a) Lab test review
b) Patient assessment and vitals capture
c) Ordering
d) Medication administration
S71-4:
With regard to connectivity solutions for tablets in the red zone, which of these objectives is
better achieved with a continuous wireless/wifi solution compared to an intermittent docking
solution:
a) Reduced hardware requirement
b) Tolerance of intermittent main (AC) power availability
c) Instant two-way access to green zone personnel
d) Ability to work without Internet connectivity
S71-5:
Which of these user-interface controls favors greater speed of entry for entering orders and
parameters, such as entering an IV fluid order?
a) Multiple push-buttons
b) Dropdown menus
c) Number sliders
d) Keypad entry
S71-6:
Treatment protocols in fast-moving disasters can change rapidly. Among the best CDS methods
for ensuring continued best practice are:
a) Alerts
b) Infobuttons
c) Order sets
d) Analytics
Page | 69

Understanding the acceptance factors of an Hospital Information System: evidence from a
French University Hospital
Roxana Ologeanu Taddei; David Morquin; Hugues Domingo
S76-1-1:
A survey conducted in a French Hospital shows a low score of perceived utility and ease-of-use
of the Hospital Information System (HIS) for the clinical staff, which report several difficulties. A
main difficulty related to the HIS implementation and use are:
a) Understanding of the objectives
b) Computers cost
c) Information Systems Directors involvement
d) Insufficient compatibility of the HIS with the clinical workflow
S76-1-2:
The result of a survey conducted in a French Hospital is a strong correlation of acceptance
factors of HIS with clinical occupations. This may be explained by dissimilar
a) uses of the prescription module
b) work involvement
c) wages
d) uses of the billing management module.
Improving EHR Capabilities to Facilitate Stage 3 Meaningful Use Care Coordination Criteria
Dori Cross; Genna R. Cohen; Paige Nong; Anya-Victoria Day; Danielle Vibbert; Ramya
Naraharisetti; Julia Adler-Milstein
S76-2-1:
Which of the following represents a way in which EHRs are currently supporting care
coordination efforts within primary care?
a) Ability to attach ICD codes to problems and medications for easier reconciliation,
information sorting and decision support
b) Automatic generation of a Summary of Care Record to send to specialists upon referral,
with a compilation of relevant parts of the patient record needed for subsequent care
c) Integrated document tracking functionality to follow patients through the referral
process and ensure follow-up if gaps in care are identified
d) Enabling features to accommodate and integrate workflow and documentation needs of
ancillary team members, such as care managers
Page | 70
S76-2-2:
One common challenge in todays healthcare environment is managing the volume of
information generated in the provision of patient care. From a vendor perspective, which of
these solutions is the most challenging to implement?
a) Branching logic to determine what content is sent through the summary of care record
b) Document tagging to classify/sort information for easier retrieval
c) Automatic incorporation of referral reports and other incoming information in to
relevant section(s) of the patient record
d) Drag and drop functionality to more easily maintain accurate, up-to-date problem and
medication lists

Variability in Electronic Health Record Usage and Perceptions among Specialty vs. Primary
Care Physicians
Travis K. Redd; Julie W. Doberne; Daniel Lattin; Thomas R. Yackel; Carl O. Eriksson; Vishnu
Mohan; Jeffrey A. Gold; Joan S. Ash; Michael F. Chiang
S76-3-1:
Electronic health record (EHR) systems and the patient chart contained within them serve as an
important source of information used by physicians to inform clinical decision-making. Many of
these systems are designed for use across all medical specialties. However, recent evidence
suggests there are differences in the way physicians from different clinical disciplines
incorporate this chart review into clinical workflow, which may not always be accounted for.
When used as a source of patient information to inform clinical decision-making, in what ways
is EHR utilization different between physicians in primary care and specialty medical fields?
a) Primary care physicians spend more time reviewing the chart
b) Specialty physicians spend more time reviewing the chart
c) Timing of this initial chart review (i.e. before, during, or after the patient encounter)
d) Perceived importance of the chart as a source of patient information
Page | 71

Inferring Clinical Workflow Efficiency via Electronic Medical Record Utilization
You Chen; Wei Xie; Carl A. Gunter; David Liebovitz; Sanjay Mehrotra; He Zhang; Bradley Malin
S76-4-1:
Electronic health record (EHR) systems document the actions of care providers with respect to a
patients record in a temporal manner. This information can be leveraged to construct a
sequence that is representative of the actions invoked by care providers. In order to measure
the strength of an ordered relation from one action to another action, we make the following
assumption:
A sequence is a series of events (actions), where an event has an ordered relation with the two
subsequent events in a sequence, which are weighted by distance. Specifically, the
immediately following event has a stronger relationship than the next nearest event. For
instance, in a sequence e 1 e 2 e 3 e 4 , the event e 1 has an ordered relation with e 2 , which
is stronger than with e 3 , and no relation with e 4 .
Based on the assumption how can we calculate the strength of an ordered relation e 1 e 2 ?
Count(e i e j e k ) is a function which calculates the total number of times, sequence e i e j
e k appearing in all patient sequences.
a) Count(e 1 e 2 )
b) Count(e 1 e 2 ) / Count(e 1 e i ), where e i is the immediately following event of e 1
c) Count(e 1 e 2 ) + Count(e 1 e i e 2 ), where e i is any event following e 1
d) W 1 *Count(e 1 e 2 ) + W 2 *Count(e 1 e i e 2 ), where 0<W 2 < W 1 <1
S76-4-2:
A series of two ordered events such as e1 e2 and e3 e4 can appear in patient sequences for
varying reasons and durations. For instance, imagine e1 e2 appears four times, with
durations of {0.1, 0.2, 0.2, and 1} hour, while e3e4 appears five times and always for 0.1 hour.
Assume average and variance in duration are invoked as standard criteria for ordered-relation
categorization. In general, we can categorize ordered-relations into four types:
1.
Stable Efficient (SE): average duration 0.2 hours with variance 0.1 hours;
2.
Stable Inefficient (SI): average duration > 0.2 hours with variance 0.1 hours;
3.
Unstable Efficient (UE): average duration 0.2 hours with variance > 0.1 hours; and
4.
Unstable Inefficient (UI): average duration > 0.2 hours with variance > 0.1 hours
a. e 1 e 2 and e 3 e 4 are SE.
b. e 1 e 2 and e 3 e 4 are SI.
c. e 1 e 2 is SE, while e 3 e 4 is UE.
d. e 1 e 2 is UI and e 3 e 4 is SE
Page | 72
Farah Magrabi; Dean F. Sittig; Jean Scott; Peter Kilbridge
S80-1:
Your healthcare system wishes to examine evidence about the patient safety risks of HIT. It has
identified 125 HIT-related events that have been voluntarily reported by clinicians to the
organizational incident reporting system over the last 3-months. Which is the most appropriate
way to utilize the data provided by the event reports?
a) To assess the overall safety of HIT
b) To examine the different types of safety problems that can arise with HIT
c) To measure the frequency of HIT-related adverse events
d) To quantify the magnitude of patient harm
S80-2:
In your healthcare system, clinicians report most HIT problems to the IT service desk (or help
desk) which is available 24 hours a day, 7 days a week. Any incidents that are identified by
clinicians as impacting care delivery or patient safety are given the highest priority. Service desk
staff have been asked to escalate all large-scale adverse events for immediate investigation and
response. Which of the following events would be escalated?
a) All the computer systems in the emergency department have crashed.
b) A physician cannot login to the CPOE system to order a blood test for a patient who is
critically ill.
c) Data was lost and time was wasted when a desktop computer crashed during a
consultation.
d) A child had a full body x-ray. Some of the images went missing in a PACS when they
were digitized.
S80-3:
Your healthcare system is deploying a range of tools to support the operation of a new EHR.
Out-of-bandwidth tools refers to:
a) Dangerous applications that consume excessive bandwidth
b) Tools that gradually reduce network capacity
c) Tools that monitor network traffic from outside of the network
d) Standard network configuration tools
Page | 73
S80-4:
Your healthcare system is reviewing its data security practices. As resources to enhance existing
data security practices are limited, you have been asked to identify and prioritize only the most
important threats to the security of patient information stored in the EHR. Provider data
breaches today:
a) Remain comparatively rare events
b) Can be entirely prevented with sophisticated firewalls
c) Are less common than breaches in the financial sector
d) Should be expected and planned for
S80-5:
Once initially implemented, a healthcare system can expect to receive incremental updates for
defect remediation, data updates and new features. The original implementation was planned
prior to adoption of functionality into the workflows, and specific configurations were
established before go-live. As more systems are implemented, the effect of integration
becomes more complicated. In efforts to streamline updates, healthcare systems may adopt
features that are implemented at the enterprise level.
After an upgrade, a defect is discovered in which all care facilities within the healthcare system
are adversely affected in which the system is unavailable or no back-out mechanism is feasible.
Which of the following practices provides a strategy for risk mitigation?
a) Require increased software testing prior to release
b) Establish deployment approach to allow for monitoring of defects including ability to
respond and pause process
c) Report issues to the IT Service desk
d) Ensure all facilities are able to enact contingency plans such as having paper forms for
processing orders.
Page | 74
S80-6:
As healthcare delivery organizations become more dependent on EHRs the risks to patient care
during an extended system-wide downtime increase substantially. Over the next 3 years what is
the likelihood that any large healthcare delivery organization will experience a down time
lasting more than 8 hours?
a) Infinitesimally small. No need to worry.
b) Less than 15%, but you recently upgraded your air conditioning systems so you will be
fine.
c) About 75%, but with proper planning and preparation the risks to patient safety can be
greatly reduced.
d) Almost certainly one will occur, but there is nothing you can do to reduce your risk so no
need to worry.
S80-7:
Your healthcare system is using the SAFER guides. The Contingency Planning SAFER Guide
identifies recommended safety practices associated with planned or unplanned EHR
unavailabilityinstances in which clinicians or other end users cannot access all or part of the
EHR. Occasional temporary unavailability of EHRs is inevitable, due to failures of software and
hardware infrastructure, as well as power outages and natural and man-made disasters. Such
unavailability can introduce substantial safety risks to organizations that have not adequately
prepared.
Effective contingency planning addresses the causes and consequences of EHR unavailability,
and involves processes and preparations that can minimize the frequency and impact of such
events, ensuring continuity of care. Which of the following practices does not deal with
contingency planning?
a) A communication strategy that does not rely on the computing infrastructure exists for
down time and recovery periods.
b) Paper forms are available to replace key EHR functions during downtimes.
c) Policies and procedures are in place to ensure accurate patient identification when
preparing for, during, and after downtimes.
d) The organization has adopted a standard clinical vocabulary to record and report all
normal and abnormal laboratory test results.
Page | 75
Craig Kuziemsky; Madhu Reddy; Katie A. Siek; Sarah Collins
S81-1:
The predominant challenge facing design of HIT to support collaboration is:
a) The lack of explicit focus on collaboration in HIT studies
b) Poor understanding of workflow
c) Need to be more patient centered
d) Inability to support teamwork
S81-2:
Planned or spontaneous engagements that take place between individuals or teams of
individuals, whether in-person or mediated by technology, where information is exchanged in
some way (either explicitly, i.e., verbally or written, or implicitly, i.e., through shared
understanding of gestures, emotions, etc.), and often occur across different roles (i.e., physician
and nurse) to deliver patient care is defined as:
a) Communication
b) Coordination
c) Cooperation
d) Collaboration
S81-3:
The technologies, contexts, processes and outcomes that are involved in collaboration is
referred to as:
a) Agenda for studying collaboration
b) Collaboration space
c) Collaborative competencies
d) Patient-centered care
Page | 76
S81-4:
What are the best methods to study collaboration?
a) Qualitative methods
b) Quantitative methods
c) Mixed methods
d) Depends on the objectives of the study
S81-5:
The context of different collaborative settings is most closely linked to:
a) The collaborative structure
b) The collaborative behavior
c) The collaborative outcomes
d) The technology needed to support collaboration
Future)
Ozlem Uzuner; John Aberdeen; Stephane Meystre; Mehmet Kayaalp
S83-1:
What method of clinical narrative de-identification always results in 100% removal of
identifiers?
a) Rule-based systems that match patterns in narrative text.
b) Machine learning systems that are trained from examples.
c) Manual de-identification performed by human experts.
d) None of these methods results in 100% removal of identifiers.
Page | 77
S83-2:
For what reason(s) is the de-identification of clinical text more difficult than general English
text?
a) Clinical text is rich in ambiguous abbreviations
b) The structure of clinical text is often ungrammatical (e.g., telegraphic style)
c) Clinical notes frequently include lists and tables in the text
d) All of the above
S83-3:
Which one of these is always necessary for a HIPAA compliant data set to be shared with other
researchers?
a) Institutional Review Board approval by the institution that owns the data
b) Institutional Review Board approval by the home institution of the recipients of the data
c) Human subjects training by the recipients of the data
d) A commitment from the recipients of the data that they will destroy it properly.
S83-4:
In practice, clinical narrative texts are typically shared for secondary use purposes by what
methods?
a) Manual or automated de-identification, followed by open release
b) Manual de-identification only, followed by open release
c) Manual or automated de-identification followed by limited release
d) Clinical narrative texts are not shared for secondary use
Page | 78
S83-5:
Which one of these provides the most convincing argument for an IRB to approve the
distribution of automatically de-identified clinical text data?
a) Automatic systems are at least as accurate, if not more accurate and consistent, than
human de-identifiers.
b) In a task in which even humans do not perform 100% accurately, automatic deidentification systems give very close to human performance and can replace humans.
c)
Automatic de-identification will be accompanied with manual human review to get the best
de-identification possible before data distribution.
d) Automatic de-identification accompanied with appropriate data use agreements that put
limits on the use of the data minimize the risk of PHI leaks.

Adaptation of a Published Risk Model to Point-of-care Clinical Decision Support Tailored to
Local Workflow
Jeffrey L. Sobel; Craig C. Baker; David Levy; Carol Cain
S84-2-1:
In implementing electronic clinical decision support, one design question is whether to re-use
content that has been developed by someone else or build content inside the walls of an
organization. In this session, the organization describes their experience replicating a published
risk model. Why did the organization choose to replicate the model rather than link to an
external site?
a) The organization needed to customize some of the parameters to fit their clinical
practice
b) It was more reliable and integrated to reproduce the model inside their firewall
c) The published researchers were not willing to share their data model
d) The organization wanted to validate the model themselves
Page | 79
S84-2-2:
Many technical and organizational factors contribute to the success of a clinical decision
support initiative. Although all factors are important, successful clinical outcomes have been
demonstrated even without the inclusion of one of the factors below:
a) Endorsement of the model by the organization's subject matter experts
b) Sustained and regular use by frontline physicians
c) Automatic generation of EMR orders based on results of the model
d) Integration with existing clinical initiatives and improvement infrastructure

Challenges and Solutions in Optimizing Execution Performance of a Clinical Decision SupportBased Quality Measurement (CDS-QM) Framework
Tyler J. Tippetts; Phillip B. Warner; Polina Kukhareva; David E. Shields; Catherine J. Staes;
Kensaku Kawamoto
S84-3-1:
You are in charge of implementing a point-of-care reminder system for diabetes care, as well as
an electronic clinical quality measurement system for providing feedback to clinicians on their
quality of care for diabetes. Which of the following is a benefit of using the same framework
for implementing both the clinical decision support and electronic clinical quality measurement
in this scenario?
a) A clinical decision support-based quality measurement system will always perform
faster than a quality measurement system implemented in a different way.
b) There will be semantic consistency between what the clinician sees in terms of point-ofcare reminders and what they see as quality feedback reports.
c) The clinical decision support implementation team is almost always also responsible for
enterprise clinical quality measurement at any given health care organization.
d) Implementing the two functionalities using the same technical framework would follow
the typical approach taken at most health care organizations.
S84-3-2:
You have been using a clinical decision support-based quality measurement framework without
problem for inpatient quality measurement. When scaling to outpatient quality measurement,
however, you are having difficulty achieving the required throughput for large populations in a
Page | 80
timely manner. Which of the following is an approach you should consider for increasing the
throughput?
a) Query for patient data one patient at a time as opposed to for large batches of patients
at one time.
b) Make sure all processes are executed through a single machine/server.
c) Run processes in parallel rather than in sequence.
d) Use a standard data model rather than an ad hoc data model.

Using a Clinical Knowledge Base to Assess Comorbidity Interrelatedness Among Patients with
Multiple Chronic Conditions
Donna M. Zulman; Susana B. Martins; Yan Liu; Samson Tu; Brian B. Hoffman; Steven M. Asch;
Mary K. Goldstein
S84-4-1:
Of the following, which measure is specifically designed to capture the treatment complexity
that may arise when making clinical decisions for a patient with one or more comorbid
conditions?
a) Charlson Comorbdity Index
b) Comorbidity Interrelatedness Score
c) Elixhauser Comorbidity Measure
d) Diagnosis-Related Group (DRG)
S84-4-2:
Which of the following types of clinical decision support system is most likely adaptable to
generate comorbidity interrelatedness scores:
a) Alerts for critical lab values
b) Clinical reminders
c) Hospital admission order sets
d) Knowledge-based systems with an inference engine
Page | 81
S85: Papers - mHealth

Mining Twitter as a First Step toward Assessing the Adequacy of Gender Identification Terms
on Intake Forms
Amanda Hicks; William Hogan; Michael W. Rutherford; Bradley Malin; Mengjun Xie; Christiane
Fellbaum; Zhijun Yin; Daniel Fabbri; Josh Hanna; Jiang Bian
S85-1-1:
A clinic plans to start gathering patient data on gender identity after reading the Institute of
Medicines recommendation highlighting how this information will improve patient care and
lead to better understanding of the health needs and risks of gender minorities. In response to
this recommendation, Fenway Health conducted studies for determining the best way to collect
gender identity data to receive high response rates and accurate data. What format is
recommended for collecting gender-identity information on health intake forms?
a) Ask the patient to identify their gender. Do not provide options such as male and
female, but provide a place for the patient or intake personnel to write-in a response.
b) Ask the patient to identify their gender and provide the following options: male,
female, other please specify, with a write-in option for other please specify
c) Ask the patient two questions: 1) What sex were you assigned at birth? 2) What is your
current gender identity?
d) Ask the patient to identify their sex and give three options: male, female, other.
Do not include a free text option
S85-1-2:
Fenway Health, UCSF Center of Excellence for Transgender Health, and the Williams Institute
provide sample intake forms for asking about gender identity. Each form provides a different
set of options for specifying current gender identity.
What is a potential problem with these options from the point of view of collecting reliable
data?
a) These forms have too many options leading patients and researchers to be confused
about what the different options mean.
b) These forms do not adequately capture non-binary gender identities such as agender.
c) Men/women who are not trans* do not understand the options and frequently mark
the wrong one.
d) Patients frequently skip the questions and do not provide an answer.
Page | 82

Evaluating Consumer m-Health Services for Promoting Healthy Eating: A Randomized Field
Experiment
Yi-Chin Kato-Lin; Rema Padman; Julie Downs; Vibhanshu Abhishek
S85-2-1:
Which of the following activity can be fully achieved by simply providing a web app, as opposed
to a mobile app?
a) Help patients to monitor their exercise level by automatically tracking their active time
in minutes
b) Help patients to achieve a healthier eating habit by remotely calibrating patients
portion size misperception for each meal in real time
c) Help patients to monitor their sleeping quality at home by automatically tracking their
movements during the course of sleep
d) Help asthma patients to monitor their status by asking them to record the results of
peak flow testing
S85-2-2:
Mobile technologies have the potential to engage patients and change their healthy behaviors,
yet little evidence has been documented. Drawing on theoretical frameworks, the mobile app
study evaluates novel mobile-enabled interventions which upgrade three interventions
commonly used in practice for promoting healthy eating: self-monitoring, professional support,
and peer support. The impact of these interventions were examined via a randomized field
experiment. Suggested by the results of this study for healthy eating, which of the following
may best engage patients in managing their health by using an intervention tool?
a) Providing them a booklet with tips for better management and sheets for selfmonitoring
b) Providing them an online forum that allows them to ask questions and get answers from
healthcare professionals
c) Remotely connecting them with a healthcare professional who can give them
personalized feedback and suggestions that work specifically for them
d) Providing them an online community that allows them to compare and learn from other
patients like me
Page | 83

Public Perspectives of Mobile Phones Effects on Healthcare Quality and Medical Data
Security and Privacy: A 2-Year Nationwide Survey
Joshua E. Richardson; Jessica S. Ancker
S85-3-1:
The authors conducted a nationwide consumer survey over two years, asking three questions
about mobile health's potential impact on healthcare privacy, security, and quality. What, if
any, year-over-year change was there in consumers' perception about mobile health's impact
on healthcare quality?
a) Significantly improve
b) No significant change
c) Significantly worsen
d) No findings
S85-3-2:
about mobile health's potential impact on healthcare privacy, security, and quality. Which
answer best describes the primary takeaway from the survey's results?
a) Consumers likely see little benefit from mHealth due to overriding privacy and security
concerns
b) Consumers seem to have little to no understanding of mHealth
c) Consumers have strong concerns about mHealth privacy and security but increasingly
perceive mHealth as a means for improving healthcare quality
d) Consumer age groups significantly differed in their opinions about mHealth privacy and
security, and its effect on healthcare quality
Page | 84

Patient Engagement in Cancer Survivorship Care through mHealth: A Consumer-centered
Review of Existing Mobile Applications
Yimin Geng; Sahiti Myneni
S85-4-1:
Which of following statements about HIMSS Patient Engagement Framework is appropriate?
a) Patient Engagement Framework is a model that is used to create questionnaire for the
evaluation of patients self-management of chronic diseases.
b) Patient Engagement Framework consists of five distinct phases that begin with low-level
engagement and progress toward high-level engagement.
c) Patient Engagement Framework consists of five cumulative phases to engagement that
make up the framework, which include inform me, engage me, partner with me,
and support my e-community.
d) One of the fourteen Patient Engagement Framework categories identified, information
and way-finding, can only achieve inform me phase of patient engagement.
S85-4-2:
To engage patients in self-management of their own health, mHealth solutions development
can be based on
a) Traditional theories of behavior change solely as long as considering individuals, their
social and environment interactions.
b) Utilization of most advanced web-based and mobile platforms only.
c) Integration of behavior change strategies using technology development frameworks.
d) Integration of Engaging consumers in Health with Health Care (ECHC) framework and
Health belief model.
Page | 85

Human Factors of Health Information Exchange: Barriers and Facilitators to Use of the VAs
CPRS and a Regional Health Information Exchange
Andre W. Kushniruk; Elizabeth M. Borycki; Helen Monkman; Kenneth Boockvar
S88-1-1:
There is an increasing demand for interconnection between electronic health records and
regional health information exchanges. However, a number of human factors problems have
led to lower than expected adoption. Which of the following is the most likely reason for lack of
use of such interconnected systems from a human factors perspective?
a) lack of time for users to switch among systems
b) incompatible data formats
c) privacy laws, security breeches and malware attacks
d) file specification errors
S88-1-2:
Usability testing methods for analyzing user interaction with multiple systems in the study of
system integration involves:
a) simulating keystrokes
b) observing and video recording user interactions
c) applying heuristic design and evaluation guidelines
d) disseminating questionnaires
Page | 86

Uncovering the Cognitive Demands of EHR Use via Task Analysis
Mark S. Pfaff; Ozgur Eris; Amanda Anganes; Tina Crotty; Jonathan R. Nebeker; Merry Ward
S88-2-1:
Communication between clinicians on a care team is essential for maintaining common ground
about the patients diagnosis and treatment plan. The EHR provides functionality for clinicians
to communicate with one another, but often there are breakdowns in communication that can
potentially interfere with effective care delivery. In this Department of Veterans Affairs
qualitative study, which of the following did clinicians describe as the main communication
challenge they face when trying to maintain a shared understanding of the patients situation?
a) Clinicians described the communication functionality in the EHR as difficult to use
b) Clinicians were uncertain about what kinds of information was or was not required to be
exchanged with other clinicians
c) Clinicians reported that it was difficult to know whether someone had received or read a
message they had sent
d) Clinicians felt that it was difficult to keep on top of all the messages they received via
the EHR
S88-2-2:
Diagnosing a patient and developing a treatment plan can require clinicians to acquire and
comprehend large volumes of data about the patients history and current condition. Clinicians
also use information that provides framing and context, such as acceptable ranges of laboratory
values, which help the clinician understand the implications of certain pieces of patient data.
Unfortunately, the necessary pieces of data for making clinical decisions are often dispersed in
many different parts of the EHR (ie, separate screens or modules). According to this study,
which of the following is the most significant cognitive implication of fragmentation of data
across different parts of the EHR?
a) It increases the chances for duplication of effort in the EHR (e.g. redundant lab orders)
b) It requires slow and effortful thinking rather than enabling fast and associative thinking
c) It particularly affects novices interacting with the EHR
d) It increases the amount of training clinicians require with the EHR
Page | 87

Evaluating the Effects of Cognitive Support on Interpreting ICU Patient Data
Peter V. Killoran; Swaroop Gantela; Sahiti Myneni; Khalid Almoosa; Bela Patel; Thomas
Kannampallil; Vimla L. Patel; Trevor Cohen
S88-3-1:
Which of the following techniques of information organization has the greatest impact on
attention to Systemic Inflammatory Response (SIRS) criteria:
a) Organization of information according to expert defined intermediate constructs
b) Organization of information according to data source
c) Organization relative to reference ranges
d) Organization by relative value
S88-3-2:
Which of the following has the greatest impact on the sequence clinical information that is
viewed by clinicians when a diagnosis of Systemic Inflammatory Response (SIRS) is suspected?
a) The order in which it is presented
b) Relevance to the clinical scenario
c) Position of each variable relevant to SIRS
d) Deviation from the range of normal
Page | 88

Efficiency and Accuracy of Kinect and Leap Motion devices Compared to the Mouse for
Intraoperative Image Manipulation
Uchenna A. Uchidiuno; Yuanyuan Feng; Helena M. Mentis; Hamid Zahiri; Adrian Park; Ivan
George
S88-4-1:
Surgeries that involve the use and manipulation of medical images for referencing present
certain challenges for the surgeon. A few of these challenges reside in the use of the mouse and
keyboard to perform these operations. With the advent of touchless interaction devices such
the Microsoft Kinect and Leap Motion, the current method of interaction with these images are
presented with the opportunity for improvement. Which of the following is most likely the
motivation for implementing a touchless interaction device in place of the standard mouse?
a) current means of image interaction during surgery are too slow
b) use of the mouse and keyboard are cumbersome for the surgeons
c) touchless devices out-perform the mouse for all purposes
d) maintaining a sterile environment during surgery
S88-4-2:
The use of touchless interaction devices show potential for alleviating some challenges faced
with image manipulation during surgery. However, what is considered the best touchless
interaction device is not consistent. Which of the following accounts for this finding?
a) one touchless device is better suited for a particular surgery
b) device performance is affected by task type
c) one touchless device out performs the other for all tasks
d) appropriate touchless device is determined by surgeons preference
Page | 89

A Guideline for Assessing EHR Data Quality for Secondary Use
Nicole G. Weiskopf; Chunhua Weng
S90-1-1:
Reusing EHR data for clinical research has the potential to improve research efficiency and
generalizability. EHR data quality, however, is variable and often poor. Its important that
researchers be aware of the limitations of an EHR-derived dataset when conducting research.
Which of the following courses of action would be appropriate and sufficient for deriving this
knowledge about the data for a particular study?
a) Review existing literature on similar data elements from other institutions and
generalize their results to your own dataset.
b) Re-use an existing EHR-derived dataset that has already had its quality assessed for a
previous, unrelated study and cite the previous data quality findings.
c) Assess the quality and suitability of an EHR-derived dataset specifically within the
context of the current study.
d) Speak with providers in the appropriate clinical area to ascertain their views on the
quality and suitability of the data.
S90-1-2:
Many of the studies that report on the quality of EHR-derived data establish accuracy and
completeness through comparison to a gold standard. This gold standard is most commonly
derived through chart review or from administrative (e.g. billing) data. Which of the following is
a benefit of assessing EHR data quality through the use of a gold standard?
a) Gold standard data are always readily available for EHR-derived datasets.
b) Chart review and administrative data, which are often used to generate gold standards,
are generally sources of high-quality data.
c) Generating a gold standard through chart review is a straightforward and efficient
process.
d) Comparison to a gold standard allows for the calculation of statistics that are relatively
easy to interpret and understand, such as sensitivity and specificity.
Page | 90

Effects of HIE/HIT Implementation and Coordination of Care on Health Outcomes and Quality
Onyinyechi U. Enyia Daniel; Edward Mensah
S90-2-1:
Jack was admitted to the Emergency department last week for chest pain and shortness of
breath. During his visit the emergency department, doctors informed him that he also had high
blood pressure. Two weeks after discharge from the hospital, Jack saw Dr. Johnson, Jack's
primary care physician. Dr. Johnson was able to access Jack's clinical care summary
documentation and clinical discharge instructions electronically. Which of the following is a key
benefit of access to electronic clinical information?
a) Reduced hospital readmission
b) Reduced cost
c) Improved public health response
d) Decreased access to care
S90-2-2:
Dr. Jordan is working with hospital administrators to create a business case for participation in
a regional Health Information Exchange (HIE). A strong case for HIE can be made for which of
the following initiatives?
a) Cost reduction
b) Care coordination
c) Patient exclusion
d) Increased utilization
Impact of Electronic Health Records on Quality of Care: Evidence on Inpatient Mortality,
Readmissions, and Complications
Tina Hernandez-Boussard; Catherine Curtin; Doug Morrision; Swati Yanamadala; Katherine
McDonald
S90-3-1:
Evidence from this study suggests that adoption of EHRs is
a) is associated with improved care coordination and reduced 30-day readmissions
b) is associated with decreased inpatient mortality
c) is not associated with decreased inpatient mortality after controlling for important
confounders
d) is associated with improved patient safety
Page | 91
S90-3-2:
In this population based study, the cross-sectional analyses indicated that
a) Black patients were more likely to go to a hospital with no EHR system
b) Patients with public coverage were more likely to go to a hospital with an EHR-system
c) Older patients were more likely to go to hospitals with an EHR-system
d) none of the above
S90: Papers/Podium Presentations - EHR Usability and Quality

Usability Testing of an Ambulatory EHR Navigator
Gretchen Hultman; Elliot G. Arsoniadis; Jenna Marquard; Rubina F. Rizvi; Saif Khairat; Keri
Fickau; Genevieve B. Melton
S90-4-1:
During this study, eight participants completed tasks using two different versions of a navigator
embedded in a commercial EHR. One outcome examined was time to complete task. Which of
the following varied significantly?
a) Time to complete tasks across participants
b) Average time to complete tasks across navigators
c) Average time to complete tasks across cases
d) Perceived workload across cases
S90-4-2:
During this study, participants tested two different versions of an ambulatory navigator were
asked which of two different navigators in an EHR they preferred. Overall, which best describes
participants navigator preferences?
a) Participants had a strong preference for the new navigator
b) Participants had a strong preference for the old navigator
c) Preferences varied widely among participants
d) Participants did not express a strong preference for either navigator
Page | 92
Wednesday, Nov. 18, 2015- Questions & Answer Options

and Investigation
Patricia C. Dykes; Sarah Collins; Anuj K. Dalal; Ryan Greysen; Cindy Dwyer
S91-1:
Which methods of user authentication for enterprise sign-on access is not considered strong
(i.e., two-factor authentication)?
a) USB token (e.g., RSA Secure ID)
b) Finger print biometrics on personal mobile device
c) Username and password
d) One-Time-Password (OTP) tokens sent by SMS to personal mobile phone
S91-2:
Which of the following is true about older adults and general technology use?
a) Most seniors dont use the internet on a regular basis.
b) Most seniors dont use a cell phone.
c) Most seniors dont own a desktop or laptop computer.
d) Most seniors dont own a smartphone.
S91-3:
Which of the following is true about use of patient portals by older adults in the hospital?
a) Most older adults are not interested in accessing their electronic health records (EHR) in
the hospital.
b) Older adults are less able than younger patients to use hospital-provided devices (e.g.
tablets) to access their EHR in the hospital
c) Older adults are as likely as younger patients to bring their own mobile computing
devices to the hospital.
d) Older adults may require more detailed explanation and more time to successfully use
mobile devices to access their EHR in the hospital
Page | 93
S91-4:
Which of the following strategies is consistent with user-centered design?
a) The design team includes end-users with an understanding of tasks and environments.
b) The process is linear with a stage gate between each phase where requirements are
c) The design mainly addresses one aspect of the user experience.
S91-5:
Which of the following is not considered protected health information?
a) Patient problem list shared by the patient and care team
b) Medication list with infobuttons links for the patient
c) Educational videos about common diagnoses on a patient portal home page
d) Messages sent between a health care proxy and provider on a patient portal
S95: Papers All About Handoffs

Uncertainty, Case Complexity and the Content of Verbal Handoffs at the Emergency
Department
Jan Horsky; Edward H. Suh; Osman R Sayan; Vimla L. Patel
S95-2-1:
There seems to be little convergence in clinical research literature on the characterization of a
high-quality handoff or its primary purpose. A realistic objective may be ensuring that clinicians
taking over a patient can continue care without interruption and delay, do not need to
duplicate tests and procedures unnecessarily and are prepared to respond appropriately and
immediately to sudden or unexpected changes. Shared mental models of patient illness and
care between teams or individuals during a handoff are particularly relevant to preserving care
continuity, especially in emergency settings where adequate and reliable information may not
be available and uncertainty about symptoms, presentation and several possible diagnoses may
create ambiguities and different interpretations.
In the view of this goal, which characterization best describes the purpose of a handoff
specifically suited for emergency care?
Page | 94
a) The core objective is detailed, unambiguous, unidirectional transfer of information from

the outgoing to the incoming team. Accuracy, completeness and attention to detail are
the most optimal safeguards against information loss and misinterpretation.
b) The most important function is to relate sets of laboratory values, vital signs and results
of diagnostic testing, describe procedures, assessment and care trajectory and to specify
planned actions and contingency plans. The incoming clinician will be able to follow up
exactly as expected from written and verbal documentation.
c) A critical attribute of the process is to maintain standard and uniform structure and
content of handoff documentation across all patients, regardless of illness severity, case
complexity, diagnostic certainty or their care trajectory. High level of detail and minimal
deviations from a set protocol will make the process safer, more robust, reliable and will
contribute to higher quality.
d) Engage both parties in developing a shared mental model of each patients current state
and expected path that can be quickly adopted for care and create opportunities to
consider alternative viewpoints or correct errors. Highly structured processes do not
align well with uncertain diagnoses and indeterminate illness scripts.
S95-2-2:
Concerns about possible loss of information about patients and care between shifts in the
emergency department have motivated an initiative to revise physician verbal and written
handoffs to make the process more resilient toward misinterpretation and omission. Which of
the following interventions is most likely to have the desired effect and facilitate transitions of
care teams?
a) Clinicians will be required to attend several training sessions in which they will be
coached to emphasize completeness of documentation and detailed description of
illness severity, prior treatment, planned actions, contingency planning and other safe
handoff practices.
b) Highly structured forms with required fields will need to be filled out ahead of every
patient discussed during a handoff to ensure that a complete set of available
information is communicated to the incoming team.
c) A new electronic tool that mimics an existing handoff protocol successfully
implemented in the hospitals pediatric service will be made available to ED clinicians
who will be issued portable devices so that they can share and review the same
information during discussions.
d) A novel handoff process that would adapt an existing concept that is known to reduce
errors in inpatient settings (such as I-PASS) will be developed. The components are
congruent with characteristics and context of emergency care and what changes and
new interventions would be necessary will be investigated.
Page | 95

Physician handoffs: opportunities and limitations for supportive technologies
Katherine Blondon; Rolf Wipfli; Mathieu R. Nendaz; Christian Lovis
S95-3-1:
During evening handoffs, day team physicians transfer their roles and responsibilities to the
night physician. Which of the following statements is true?
a) The handoff process is a standardized process, often similar across medical centers
b) Mnemonics are commonly used to support verbal handoffs
c) Handoffs are particularly important in complex situations, or when unusual medical
management options have been chosen
d) Information provided during a handoff is similar at novice and expert levels
S95-3-2:
Let us consider how technology can provide better support for the handoff process. Please
select the correct answer:
a) Alerts are a simple, feasible solution to support handoffs, which just needs determining
who to alert
b) We should provide the night physician with a system that allows her to review all the
patients she is cross-covering and helps her easily to identify if new information (e.g.,
pending test results) comes in for one of the patients
c) A big data approach could help predict disease trajectories for patients, according to
comorbidities and severity of disease. We would then receive a notification if the
patient diverges from their disease trajectory
d) Abnormal laboratory results and vital signs are defined by the range of normal results,
and are more important than the changes in the parameter over time.
Improving Continuity of Care via the Discharge Summary
Farrant Sakaguchi; Leslie A. Lenert
S95-4-1:
Current efforts to improve the data content of the Discharge Care Summary (DCS) _____
a) will significantly improve the continuity of care for patients.
b) fail to emphasize medical decision making.
c) will make it easy to implement in most EHRs.
d) offer the best solution for HIE.
Page | 96
S95-4-2:
This research shows that the causal chain A) better discharge summaries increase B) continuity
of care which improves C) patient care _____
a) is straightforward to demonstrate with current qualitative research techniques and
measurement tools.
b) is likely to occur naturally with current discharge summaries.
c) exemplifies the challenges of proving increased health due to health information
technology alone.
d) ignores the complexity of the healthcare system and different, occasionally competing,
incentives.
Just One More Patient: Optimizing EMR Documentation in Ambulatory Care
Mark Pierce; Tammy Toscos
S96-1-1:
A cheese finder is a productive, efficient, high quality user of the EHR.
What is the benefit of putting resources into understanding "cheese finders" when rolling out a
new EHR?
a) Keeps your top performers happy
b) Less expensive approach to rolling out an EHR
c) Offers a way to identify successful strategies
d) This is the approach that most EHR companies recommend
S96-1-2:
What were the two differentiating behaviors that distinguished "cheese finders" from the other
providers studied?
a) Using speech recognition software for assessment/plan and for placing orders
b) Using speech recognition software for HPI documentation and placing orders while still
in the exam room
c) Using speech recognition software for placing orders and completing visit diagnosis
while still in exam room
d) Using speech recognition software for prescriptions and using a personal preference list
for ordering
Page | 97

Supporting Clinical Cognition: A Human-Centered Approach to a Novel ICU Information
Visualization Dashboard
Anthony Faiola; Preethi Srinivas; Jon Duke
S96-2-1:
Which of the following severely constrict clinicians analytical ability?
a) Time constraints, interruptions, alarm fatigue, cognitive filtering strategies
b) Distributed cognition
c) Collaboration with other clinicians
d) Technology usage
S96-2-2:
ICUs are identified as having the highest annual mortality rate of any hospital unit (12-22%).
Studies demonstrate that 80% of medical error is attributable to human factors and a majority
of user error is due to cognitive overload. Given the environment of an ICU, which of the
following could better enable rapid information assimilation, pattern recognition, and
diagnostic insights derived from examining large amounts of data in addition to existing
conventional bedside visual displays?
a) Visualization systems
b) More documentation
c) Definitive division of responsibilities
d) Task escalation
S96-2-3:
Which of the following does the current study propose as a possible solution to maximize the
clinicians ability to control data and narrow down diagnostic decision?
a) Bedside medical devices that display patient vital sign data
b) Critical decision support systems that provide recommendations through rules-based
alerts
c) Paper records
d) A visualization system that enables rapid recognition of essential changes in
physiological data over a designated time frame
Page | 98

User Frustration in HIT Interfaces: Exploring Past HCI Research for a Better Understanding of
Clinicians Experiences
Gloria Opoku-Boateng
S96-3-1:
One of the overarching goals of Human-Computer Interaction (HCI) research has been to ___
a) increase interaction with computers
b) increase the good experiences and possibly reduce or mitigate frustrating first
experiences with technology
c) make computers better than humans.
d) teach humans to be more like computers.
S96-3-2:
According to the conclusions drawn from this paper, what is the combined-approach to user
frustration research?
a) That a little bit of frustration is good so users need frustration to some extent
b) That frustration and emotion are equal
c) That it is essential to know which parts of an HCI for which humans take responsibility
versus which parts a computer should be left to complete
d) That the human-side research with the computer-side design approach should work
together to mitigate frustration.

Model Checking for Verification of Interactive Health IT Systems
Keith A. Butler; Eric Mercer; Ali Bahrami; Cui Tao
S96-4-1:
Why is model checking needed for clinical health IT?
a) The complexity of health IT combined with healthcare makes systems unpredictable.
b) Model checking is a type of system verification.
c) Health IT can disrupt appropriate workflows of care.
d) Clinicians are too busy to learn and stay current on multiple health IT systems.
Page | 99
S96-4-2:
What necessary principle for model checking interactive health IT does MATHflow satisfy?
a) It integrates information modeling with workflow.
b) It treats information as a type of resource.
c) It makes workflow explicit so it can be checked.
d) It is agnostic how either clinicians or computers can change information.

Evaluating Term Coverage of Herbal and Dietary Supplements in Electronic Health Records
Rui Zhang; Nivedha Manohar; Elliot G. Arsoniadis; Yan Wang; Terrence J. Adam; Serguei V.
Pakhomov; Genevieve B. Melton
S98-1-1:
Herbal and dietary supplement consumption has rapidly expanded in recent years. Due to
pharmacological and metabolic characteristics of some supplements, they can interact with
prescription medications, potentially leading to clinically important and potentially preventable
adverse reactions. A fundamental prerequisite is a functional understanding of supplement
documentation in electronic health records (EHRs) and associated supplement coverage in
major online databases.
Which statement is correct based on the investigation of herbal and dietary supplement term
coverage in the EHR?
a) The medication list in the EHR contains rich information about prescribed supplements.
b) Clinical notes mention supplement usage, which is not mentioned in the medication list
for patients.
c) Current standard terminologies cannot fully cover all supplement terms mentioned in
EHR.
d) All of the above.
Page | 100

2 Years Later: Follow-up to Analysis of Electronic Medication Orders with Large Overdoses
Judith Dexheimer; Eric Kirkendall; Michal Kouril; Thomas Minich; Philip Hagedorn; Cecilia Mahdi;
Stephen A. Spooner
S98-3-1:
Alert fatigue is a common problem in EHRs. We created a salience metric to measure the
effectiveness of the alerts on user behavior. Four alert rules (eRules) are evaluated for
effectiveness and demonstrate the following characteristics.
Which one has the highest Alert Salience Rate?
a) eRule 1
b) eRule 2
c) eRule 3
d) eRule 4
# Alerts Presented
# Alerts Overridden
# Alerts Heeded
100
200
100
200
90
190
70
170
10
10
30
30
Salience
Rate?
S98-3-2:
Vendor-supplied alerts for medication ordering are prevalent in EHRs. Custom-based dosing
rules for dosing can be implemented in place of these rules and are frequently more specific. In
this study, decreasing alert burden was associated with which of the following:
a) Increasing alert salience, and a decrease in alerts.
b) No change in alert salience, but a decrease in alerts.
c) Decreasing alert salience, but an increase in alerts
d) No change in alert salience, but an increase in alerts.
Page | 101

Pharmacy drug dispensing after physician discontinuation (cancel) orders
Tewodros Eguale; Aman D. Verma; Enrique Seoane-Vazquez; Rosa Rodriguez-Monguio; David
W. Bates; Robyn Tamblyn; Gordon Schiff
S98-4-1:
In an outpatient setting, the most common reason for physicians to discontinue drugs is
because they are no longer necessary. Which are the next most frequent reasons for drug
discontinuation or cancel orders?
a) Adverse drug event and ineffectiveness of the drug
b) Patient request and substitution for a less expensive drug
c) Error in prescribing and dispensing
d) Drug discontinued by another physician for an unknown reason and simplifying
treatment to increase adherence.
S98-4-2:
An electronic health record used in routine clinical practice which captures the reason for drug
treatment (treatment indication) and the reason for drug discontinuation (such as adverse drug
events or effectiveness) can effectively replace
a) case-control and cohort pharmacoepidemiology studies
b) pragmatic clinical trials
c) spontaneous reporting of adverse drug reactions
d) case-cross over studies
Electronic Health Management Platform (eHMP): The Next Phase of VAs EHR
Jonathan R. Nebeker; Walter P. Nichol; Shane McNamee; Jessica Murphy; James L. Hellewell;
Reese Omizo; Kristian Johnson; Margaret McDonald; Kensaku Kawamoto; Guilherme Del Fiol;
Emory Fry; Elaine Hunolt; Theresa Cullen; Scott D. Wood; Jennifer Herout; Charlene Weir
S101-1-1:
Condition-based worksheets in eHMP represent support what kind of workflows:
a) Business processes supported by underlying workflow engines.
b) Flow to track patient movement among various team members for care management.
c) Cognitive workflow to understand the status of a condition and facilitate decisions.
d) Wizards that step users through data entry.
Page | 102

Towards an Open EHR Platform: Porting a Complex Application using SMART on FHIR
Alistair Erskine; Marcy Stoots; David McCallie
S101-2-1:
Learning Health Systems aim to accomplish all of the following except:
a) Promote the progress of science
b) Generate new knowledge or insight
c) Analyze data produced as a by-product of care
d) Encourage continuous improvement
e) Avoid deviation from common practice
S101-2-2:
HL-7 sponsored FHIR (Fast Healthcare Interoperability Resource) is a(n):
a) Interface engine that facilitates the exchange of clinical data across Health Information
Exchanges
b) Internet protocol that improves the efficiency of transmitted clinical data
c) Draft standard and RESTful Application Programming Interface to transmit clinical data
d) Meaningful Use protocol adopted by the ONC for Stage 2 certification
S101-2-3:
In order to minimize the re-work needed to move functionality across EHRs in a timely manner,
one should:
a) Build an app using SMART Platform and FHIR data exchange resources that work with
both EHRs
b) Customize each EHR by configuring the respective system to take advantage of the
native EHRs
c) Submit an enhancement request to both EHR vendors for the new functionality sought
d) Join a local Health Information Exchange to move clinical data across EHRs
Page | 103
S101-2-4:
Which one of these needs is addressed by the SMART-on-FHIR approach to open platforms? As
a developer of a clinical application, I would like
a) to make my software available to users of different EHRs, without rewriting or
customizing for each new platform.
b) my softwares internal, private, data representation to be consistent with a standard.
c) to have a single public store in which to market my software.
d) to ensure that my users cannot easily switch to a competitors product.
S101-2-5:
Which one of these is NOT the responsibility of an EHR platform that supports SMART-on-FHIR
integration of clinical applications?
a) Implement FHIR services that can be used to access clinical data from the EHR.
b) Support the launching of SMART applications within the clinician's workflow.
c) Provide OAuth2-based security to ensure only authorized access to clinical data can
occur.
d) Capture usage information and bill the EHR user for the use of SMART-on-FHIR
applications.
S101-2-6:
Which of the following is an accurate description of FHIR APIs?
a) FHIR supports only "read" access to EHR data
b) FHIR assumes that the EHR uses a relational data model
c) FHIR uses common Internet standards including: HTTP, JSON, and XML
d) FHIR is based on HL7's Clinical Document Architecture
Page | 104
Richard H. Epstein; Karl Poterack; Patrick Guffey; Bala G. Nair; Bala G. Nair; Brian W. Pickering
S102-1:
Dr. Smith is performing a laparoscopic cholecystectomy, which the operating room scheduling
system has calculated to last 90 minutes from room entry to room exit. The scheduled duration
was based on a weighted estimate of the durations of the 20 times he has previously done this
procedure and his estimate for the current case. The case has been ongoing for 75 minutes.
Which of the following is a true statement regarding the expected median time remaining in
the case?
a) 15 minutes.
b) Less than 15 minutes.
c) More than 15 minutes.
d) Cannot be determined from the information provided
S102-2:
Surgical durations typically have a right skewed distribution, typically well described by a
lognormal distribution. What is the consequence of this observation for a given procedure
performed by Dr. Jones, assuming that estimated durations are calculated using the mean of
Dr. Joness prior cases?
a) Most (>50%) of such cases will take longer than the scheduled time.
b) Most (>50%) of such cases will take less than the scheduled time.
c) Half the cases will take the scheduled time.
S102-3:
What best describes the regulatory oversight on clinical decision support systems:
a) Clinical decision support systems are currently not regulated
b) Clinical decision support systems that perform therapeutic recommendations likely
require regulatory approval
c) All clinical decision support systems require clearance from regulatory agencies
d) Regulatory oversight is the same in the United States and European Union
Page | 105
S102-4:
Of the following, the most reliable source of data for decision support and reporting is:
a) ICD-9 codes entered manually at discharge
b) surgical start and stop times entered manually post hoc
c) Physiologic data retrieved directly at one minute intervals from monitors
d) Vital signs dictated into an operative note
S102-5:
Which of the following is an example of managerial decision support?
a) tracker board that identifies when patients preoperative preparation has been
completed
b) Pop-up reminder to administer antibiotics within one hour of surgical incision
c) Text alerts with glucose results in patients at risk for perioperative hyper- or hypoglycemia
d) Automated checklist to encourage adherence to a central line infection prevention
bundle
S102-6:
Most previous studies of clinical decision support systems have shown:
a) improvement in provider process adherence, but no improvements in any clinical
outcomes
b) no improvement in provider process adherence or any clinical outcomes
c) improvement in patient outcomes but not in provider process adherence
d) improvement in provider process adherence but no significant improvements in major
patient outcomes such as mortality
e) improvement in provider process adherence and in major patient outcomes such as
mortality
Page | 106
Courtney Lyles; Urmimala Sarkar; Neda Ratanawongsa; Danielle E. Oryn
S104-1:
a) 50% registered and 5% viewing, transmitting, or downloading information
S104-2:
than English?
a) Portals must be available to patients in the top 3 languages used within that healthcare
systems
c) There is no specific guidance about portals related to language
S104-3:
Which of the following is a top barrier to portal use among safety net patient populations?
a) Prefer WebMD instead
b) Computer/Internet access
c) Dislike their doctor
d) Inadequate health literacy/literacy skills
Page | 107
S104-4:
Which of the following is a top barrier to portal use among healthcare providers?
a) Prefer emailing patients through their personal email accounts instead
b) Lack of protected time for secure messaging
c) Dont think patients should have access to their medical information online
d) No need already have enough time with patients during in-person visits
S104-5:
What proportion of US hospitals are now offering patients access to secure messaging
functionality (as the beginning of Stage 2 Meaningful Use in 2014)?
a) 75% of hospitals
b) 50% of hospitals
c) 25% of hospitals
d) Virtually no hospitals

mobile Digital Access to a Web-enhanced Network (mDAWN): Assessing the Feasibility of
Mobile Health Tools for Self-Management of Type-2 Diabetes
Kendall Ho; Lana Newton; Allison Boothe; Helen Novak Lauscher
S105-1-1:
Surveys of North American population shows mobile phone use trends that demonstrate:
a) Disinterest of young adults in using smart phones for any type of health or disease
management
b) Older population of patients greater than 65 years old showing strong interest and
ability to use smart phones for health and illness management
c) Individuals living in rural and remote locations declining on line access to health services
d) Patients level of trust of peers with similar diseases is very low
Page | 108
S105-1-2:
A recent systematic review in using smart phone technologies to influence patient behavior in
physical activities suggests that current evidence in the literature indicates:
a) Self-monitoring of own data by itself is a powerful way to change behavior
b) Self-monitoring of own data together with external incentive or reward for positive
performance will convincing generate change
c) Setting goals, having expert advice, and peer support work synergistically with selfmonitoring of own data towards effective behavioral change
d) Self-monitoring of own data coupled with external supervision for accountability and
achievement of performance is essential in influencing behavioral change

Use of Patient Portals for Personal Health Information Management: The Older Adult
Perspective
Anne M. Turner; Katie Osterhage; Andrea L. Hartzler; Jonathan Joe; Lorelei Lin; Natasha
Kanagat; George Demiris
S105-2-1:
In the study by Turner et al., 20% of patients 60 years and older were using patient portals. The
reasons cited for using patient portals were:
a) A desire to share health information with family members.
b) Ability to make appointments and have contact with providers.
c) Greater security of health information.
d) A desire to gain new computer skills.
S105-2-2:
In Turner, et al. compared to non-portal users, older adult portal users typically
a) Lived in retirement communities
b) Had an increased need for personal health information management
c) Had less formal education
d) Reported using the computer 6-7 days/week
Page | 109

Long-Term Engagement with Health-Management Technology: a Dynamic Process in Diabetes
Predrag Klasnja; Logan Kendall; Wanda Pratt; Katherine Blondon
S105-3-1:
Which of the following statements is true?
a) Individuals with diabetes find it hard to find practical information about diabetes
management in websites and apps
b) An individual with diabetes should not give advice to another person with diabetes,
because each persons body has different responses
c) Health care providers are overall very knowledgeable about diabetes apps
d) The only way to have well-managed diabetes is to rigorously track foods and blood
glucose levels all the time.
S105-3-2:
Which of the following statements is true over time for individuals with diabetes?
a) Individuals have high daily needs for information about diabetes and its management all
throughout the disease
b) Insulin needs and blood glucose responses can change over time
c) All individuals with diabetes benefit from frequent self-monitoring of their blood
glucose levels
d) Individuals who have had diabetes for many years dont go out to eat at restaurants
because its too difficult to measure amounts of food and count carbs
Smartphone Data in Rheumatoid Arthritis - What Do Rheumatologists Want?
Phillip R. Say; Daniel Stein; Jessica S. Ancker; Cheng-Kang Hsieh; J.P. Pollak; Deborah Estrin
S109-1-1:
A new mobile health application allows patients to actively and passively input their personal
health data onto their smartphone. The application produces a report that the patient can
download for their physician. Which report would best facilitate decision making for a busy
practitioner?
a) A table of blood glucose levels linked with a daily diet journal
b) A data visualization demonstrating the patients blood glucose levels as they relate to
the number of steps they took that day
c) A medication adherence journal presented as a list
d) The patients daily blood glucose levels presented as an average with confidence
intervals
Page | 110
S109-1-2:
You recently discovered that a commonly available smartphone app passively tracks the user's
geographic location over time. You wonder whether this could be exploited for health-related
purposes. What should be the next step?
a) Running experiments to determine the precision and accuracy of the measurements
b) Choosing a data visualization
c) Finding out if tracking the users geographic location over time is a relevant measure of
any disease states
d) Interviewing physicians about how they would use this information when managing
patients
S109: Papers - The User Perspective on Informatics Tools: Organizational Uses of Health
Information Exchange to Change Cost and Utilization Outcomes: A Typology from a Multi-Site
Qualitative Analysis
Joshua R. Vest; Erika L. Abramson
S109-2-1:
In much of the research literature health information exchange is conceptualized as structural
feature only. What is the primary challenge this view has created?
a) Evaluations have found no significant effects.
b) It has limited adoption.
c) It has masked the actual process by which organizations apply information.
d) It has made it harder to measure health information exchange.
S109-2-2:
Health information exchange adoption might have the most immediate impact on which type
of activity?
a) Predictive modeling
b) Administrative, patient finding projects
c) Return on investment
d) Inpatient clinical care
Page | 111

Impact of Robotic Surgery on Decision Making: Perspectives of Surgical Teams
Rebecca Randell; Natasha Alvarado; Stephanie Honey; Joanne Greenhalgh; Peter Gardner;
Arron Gill; David Jayne; Alwyn Kotze; Alan Pearman; Dawn Dowding
S109-3-1:
A hospital has recently acquired a surgical robot. However, the surgeons have expressed
concern that their situation awareness is reduced when using the robot, which could have
implications for patient safety. Which of the following is the most appropriate response in
order to overcome this problem?
a) The team should be encouraged to communicate information to the surgeon
b) The surgeon should step away from the console intermittently
c) The surgeon should periodically ask the team for updates about the patient state
d) An individual should be identified who has responsibility for updating the surgeon about
changes in the patient state
S109-3-2:
It has been suggested that robotic surgery produces a sense of immersion, which might lead to
better decision making. In our interviews with surgeons, they suggested contexts in which the
surgeons sense of immersion is likely to be greater. Which of the following contexts is most
likely to lead to a greater sense of immersion?
a) The operating room is quiet
b) The surgeon is experienced in robotic surgery
c) The surgeon trusts the team to make him aware of changes in the operating room
d) The operation is straightforward
Page | 112

Challenges and Insights in Using HIPAA Privacy Rule for Clinical Text Annotation
Mehmet Kayaalp; Allen Browne; Pamela Sagan; Tyne McGee; Clement J. McDonald
S109-4-1:
A clinician who works in a clinic of a small town with a population size of 5,000 has observed an
interesting clinical case and decided to publish her observations in a clinical journal. In order to
fully describe the clinical case, she intends to list not only pertinent clinical manifestations of
the case, but also demographic information of her patients such as gender, age, size of
household, and annual household income.
Which of the following statements best describes what she can or cannot do in order to
preserve privacy and confidentiality of her patients?
a) She can provide both age and gender information but not the size or annual income of
the household
b) B. She can provide information about gender, age, and the size of the household
c) She can provide gender but no other demographic information
d) She can provide all such demographic information since they are not considered
personally identifying information
S109-4-2:
Which of the following statements best describes protected health information
a) Health information of particularly vulnerable subjects, such as children, prisoners,
pregnant women, mentally disabled persons, or economically or educationally
disadvantaged persons
b) Health information of subjects who belong to one of the five (physical, social,
psychological, legal, and economic) risk categories
c) Personally identifiable information such as personal name, medical record number of
the patient
d) Health information linked to the personal identifiers such as the photograph of the face
and drivers license number of an individual
Page | 113
AMIA 2015 Self Assessment and

Maintenance of Certification (MOC) Part II Q&A booklet
Correct answers, with feedback offering explanations and

suggested readings
Page | 114
Saturday, Nov. 14, 2015 Answers & Explanations

Herman Tolentino; Laura H. Franzke; Sridhar R. Papagari Sangareddy; Catherine Pepper
T02-1:
An applied definition is a statement of means and ends. Which of the following best describes
the ends in an applied definition of public health informatics?
a) Information value cycle
b) Problem solving
c) Systems thinking
d) Health system feedback loop
e) *Improving population health
Correct answer: e) Improving population health
Explanation: Public health agencies use information systems to provide data for decision
making and guide collective action for improving population health.
T02-2:
In the health system feedback loop, health consequences are affected by which of the following
components?
a) Information systems
b) *Health determinants and outcomes
c) The health system
d) Collective action
e) Data measurement challenges
Correct answer: b) Health determinants and outcomes
Explanation: The health system feedback loop mental model describes the connections
between its four components.
Page | 115
T02-3:
Generating value from data resources requires an iterative process. In the Information Value
Cycle (IVC) which step do we first consider the resources necessary to implement an
information system?
a) *Planning
b) Evaluation
c) Capture
d) Manage
e) Analyze
Correct answer: a) Planning
Explanation: The Information Value Cycle mental model helps problem solvers remember the
sequence of iterative steps in creating value from information systems. The first step is
planning.
T02-4:
In which part of the DISC Model will you find technology?
a) Data, information and knowledge
b) *Information systems
c) Context
d) Capture
e) Problem solving
Correct answer: b) Information systems
Explanation: The DISC Model (Data - Information System - Context) describes the different parts
of an information system and its context. Information systems consist of people, process and
technology.
Page | 116
T02-5:
In informatics problem-solving, which of the following is essential when identifying a problem?
a) Plot of trajectory of change from past to future state
b) *Knowledge of gaps between current and future state
c) List of prior trouble-shooting attempts
d) Identification of prior failure points
e) Description of past and current personnel involved in the project
Correct answer: b) Knowledge of gaps between current and future state.
Explanation: A problem is a recognized gap between current and future states.
T02-6:
Which components of the Problem Solving Framework focus on the problem solver?
a) Problem solving inputs and problem management
b) Problem management and performance improvement
c) Problem solving inputs and context
d) *Problem solving inputs and performance improvement
e) Context and performance improvement
Correct answer: d) Problem solving inputs and performance improvement
Explanation: Problem solving inputs includes an assessment of problem solver capacity and
capability to address the problem. Performance improvement requires the problem solver
looking back at the problem solving activity and reflecting on lessons learned that will help
improve future problem solving activities.
T02-7:
You get a request from the local health department to get help in solving a problem of
integrating community data into public health surveillance systems. Which step is the most
important to do first?
a) Start develop solutions based on what you think is going on
b) *Define the problem and its context
c) Make informed recommendations based on your experience in other settings
d) Contact a local IT company to do the job
e) Design a dashboard to show where the data is coming from and how the data are
integrated in the database backend.
Page | 117
Correct answer: b) Define the problem and its context

Explanation: Problem definition is the most important first step in problem solving. Defining the
problem well ensures that solutions developed will address root causes and leverage points
identified during problem definition.
T02-8:
In informatics problem solving, which of the following best describes the timing of a context
determination?
a) At the beginning of the project before establishing project scope
b) Mid-way through the project, once initial design specifications are outlined
c) As the project is coming to a close, so that context can be integrated into training efforts
d) *At multiple points, as the project unfolds
Correct answer: d) At multiple points, as the project unfolds
Explanation: Because problem solving usually occurs in a project setting, the problem solver has
to consider scope, time and resources. It may not always be possible to have a full appreciation
of context until one is underway in the subsequent steps of problem management. The steps in
the Problem Management component of the Problem Solving Framework are iterative in
nature.
T02-9:
Systems thinking in problem solving involves:
a) As a first step, identifying solutions that will solve the problem
b) *Viewing problems as part of a wider, dynamic system
c) Solving problems from one stakeholder perspective
d) Acting on surface features of a problem and assume certain root causes due to time
constraints
Correct answer: b) Viewing problems as part of a wider, dynamic system
Explanation: Systems thinking involves looking at components and their interconnections within
a system. The wider dynamic system that becomes context for problem solving may constrain
how problems can be approached and solved.
Page | 118
T02-10:
You are asked to solve a problem about integrating data from multiple information systems in a
state health department. You start analyzing the problem and learn the following:
Statement 1: Recently the State Senate has enacted legislation that mandates integration of all
injury surveillance systems.
Statement 2: The state health department asks what you can do to help link patient interview
and medical records abstraction datasets that are currently captured in separate databases in
their Medical Monitoring Program.
Select the right answer.
a) *Statement 1 is the context for the problem you have to solve.
b) Statement 1 is the problem statement.
c) Statement 2 is the context for the problem you have to solve.
Correct answer: a) Statement 1 is the context for the problem you have to solve.
Explanation: Statement 1 (context) constrains how Statement 2 (problem) can be approached
and managed.
Reference(s) for further study:

Sterman, JD. "Learning from evidence in a complex world." Am J Public Health. 2006
Mar;96(3):505-14. Epub 2006 Jan 31.
De Savigny D, Taghreed A. Systems thinking for health systems strengthening. World Health
Organization, 2009.
Kindig D, Stoddart G. What is population health? Am J Public Health. 2003 Mar;93(3):380-3.
Jonassen DH. "Toward a design theory of problem solving." Educational Technology Research
and Development 48.4 (2000): 63-85.
Meadows DH, Diana W. Thinking in systems: A primer. Chelsea Green Publishing, 2008.
Page | 119

Paul Fu; Richard Schreiber; Julie Hollberg; Joseph Kannry
Practical Approaches to EHR Governance - Dick Schreiber
T04-1:
The principles of EHR governance apply to:
a) Large university hospitals
b) Teaching hospitals
c) Small community hospitals
d) Ambulatory practices
e) *All of the above
Correct answer: e) All of the above.
Explanation: A fundamental learning objective of this discussion is that EHR governance applies
to healthcare environments universally.
T04-2:
The most critical success factor in EHR governance is:
a) assuring purchase of the newest technology
b) *managing and leading change transformation processes
c) establishing best infrastructure prior to and after EHR installation
d) rewarding staff for meeting or exceeding goals
Correct answer: b) managing and leading change transformation processes
Explanation: The best answer is b) because use of an EHR depends on supporting the workflow
requirements of users. The latest and greatest technology (a) is only as good as the ability of
personnel to use it. Although solid infrastructure is necessary (c), it is not sufficient. Rewarding
staff (d) has merit, but is also not sufficient.
T04-3:
Effective governance requires effective communication for EHR optimization and healthcare
transformation, as well as implementation. All of the following have been shown to be
successful forms of communication, but the least effective is:
a) Rich channels such as personal interaction and audiovisual aids are better than email
alone
b) *Encouraging leaders to disseminate information to users
c) Just-in-time trainingnot too soon, not too late; as well as at-the-elbow support
d) Involvement of clinicians as well as technical experts in creating the communication
plans.
Page | 120
Correct answer: b) Encouraging leaders to disseminate information to users

Explanation: Although most successful programs use all of the above, hoping for dissemination
of information is not reliable. It is critical to loop in clinicians and others who miss meetings.
Depending on top-down communication is risky. All the other answers involve face-to-face
interactive methods.
People, Process & Technology: Putting them all together to Optimize the Physician
Experience - Julie Hollberg
T04-4:
Physicians frustrations with EHRs are largely ignored for all of the following reasons EXCEPT:
a) Physicians who voice complaints are seen as technophobic, resistant, and uncooperative
b) * Healthcare IT is too fragile to withstand constructive criticism
c) Our government holds general belief that Health IT will improve care and reduce cost
d) Hospitals have a universal mandate to adopt a new technology before its effects are
fully understood
Correct answer: b) Healthcare IT is too fragile to withstand constructive criticism
Explanation: See:
Rosenbaum L, Transitional Chaos or Enduring Harm? The EHR and the Disruption of
Medicine. N Engl J Med 2015; 373:1585-1588 Oct. 22, 2015.
Koppel R, Patient safety and health information technology: learning from our mistakes.
AHRQ Web M&M. July 2012.
T04-5:
In order for healthcare IT to improve medical care, it must
a) Help us synthesize and analyze medical information
b) *Tell our patients stories
c) Acknowledge its mistakes and learn from them
d) All of the above
Correct answer: b) Tell our patients stories
Explanation: While the ability for health IT to aggregate medical information is important,
constructing a clear and interactive narrative is essential to the conduct of patient care. A
healthcare IT system that does not support clear presentation of patient information in a
coherent story is not effective healthcare IT.
Page | 120a
T04-6:
All of the following are examples of leadership failures which can damage your implementation,
except:
a) Lack of available time
b) Lack of political skills
c) Lack of communication skills
d) Lack of vision
e) * All of the above can contribute to system failures
Correct answer: e) All of the above can contribute to system failures
Explanation: See: Lorenzi & Riley. Managing Change: An Overview. J Am Med Inform
Assoc 2000; 7(2): 116-124.
T04-7:
Key leadership success factors for any project that involves people, processes, and technology
include all EXCEPT:
a) Executive leadership dedicated to removing barriers
b) Clearly identified and engaged local physician and clinical operation leaders to serve as
workflow change agents
c) * Decentralized command centers to address local concerns during go-live
d) Strong project managers with personal relationships with providers.
Correct answer: c) Decentralized command centers to address local concerns during go-live
Explanation: A central command center with easily identified responsible individuals allow for
the quick resolution of issues, which in turn, builds trust. Other success factors include
minimizing meetings wherever possible, and changing the focus of one-size-fits-all training to
targeted coaching, which acknowledges the diversity of users, proficiencies, and learning styles.
T04-8:
The following are all best practice myths EXCEPT:
a) Training is not necessary for computer-savvy providers
b) Ambulatory productivity should be reduced permanently to accommodate changed
provider documentation workflow
c) *Managers assigned to support implementation/conversion activities should not have
other responsibilities at go-live.
d) Additional staff is required to execute the new workflows.
Correct answer: c) Managers assigned to support implementation/conversion activities
should not have other responsibilities at go-live.
Explanation: A best practice for effective go-live support is to ensure that key managers and
superusers are able to focus on implementation/conversion activities during the go-live.
Training is essential for all providers, although one-size-fits-all training programs are not
optimal. Residents must also receive training, especially if working across multiple care settings
in which other EHR systems are used. Typically, with effective training, additional staff is not
required for new workflows; it is when users try to preserve old workflows in parallel with new
workflows that there may be increased staffing requests.
CMIO Surveys: Who are we? What do we do? - Dick Schreiber
T04-9:
Pressing challenges for the CMIO in 2016 include:
a) Data analytics and business intelligence
b) Population health
c) Meaningful Use
d) EHR optimization
Correct answer: e) All of the above
Explanation: See: 2015 Annual CMIO Survey conducted by SSi SEARCH. Dixon and Nilsen. June
2015. Accessed at http://www.ssi-search.com/images//pdfs/SSi-SEARCH-CMIO-Survey-publ2015.pdf; Shaffer. 11th Annual AMDIS-Gartner Survey of CMIOs. (Used by courtesy.)
CMIO Year-In-Review - Dick Schreiber/Paul Fu, Jr.
T04-10:
Which are the following statements are true about EHR adoption in the United States?
a) Adoption of a combination of Basic and Advanced EHR systems has increased
substantially in the past four years, reflecting a combination of the availability of
financial incentives and the impending penalties, at least from the Medicare part of the
meaningful-use program.
b) A majority of hospitals might not be ready to meet the stage 2 core objectives when
required to do so, especially Critical Access Hospitals and smaller hospitals which lag
behind in both EHR adoption and the ability to meet state 2 MU criteria.
c) More than 60% of US acute care hospitals with less than basic EHR are unable to
support online physician documentation.
d) Up-front capital costs, ongoing support costs, physician cooperation, and complexity of
meaningful-use criteria within the specified time-frame are key challenges for hospitals
able to meet and not able to meet MU Stage 2 core objectives.
Explanation: Although many hospitals have adopted EHR systems, many of these systems are
basic and lacking in core functionality that will allow the meeting of MU Stage 2 objectives.
Hospitals may be forced to convert EHR systems or accept Medicare penalties as a consequence
of failure to meet these objectives. Small and rural hospitals are at particular risk. Policies that
address these issues may disproportionately benefit these at-risk institutions. (ref. AdlerMilstein J, DesRoches CM, Kralovec P, Foster G, Worzala C, Charles D, Searcy T, Jha AK.
Electronic Health Record Adoption In US Hospitals: Progress Continues, But Challenges Persist.
Health Aff (Millwood). 2015 Nov 11)
T04-11:
A mismatch between clinician workflow and EHR functionality is a major and growing concern,
especially in the areas of usability and patient safety. Which of the following are
recommendations to improve EHR systems?
a) Optimize EHRs to facilitate longitudinal care including the ability to support teams of
clinicians and patients
b) Avoid checkboxes or similar repetitive data entry designs to minimize inaccurate date
entry.
c) Improve the designs of interfaces so that they support and build upon how people think.
d) Use public, standards-based APIs and data standards that enable EHRs to become more
open to innovators, researchers, and patients
Explanation: As consumer technology user-centered design far surpasses current EHR user
interfaces, the tension between a documentation system that has evolved to support billing
and legal protection and the requirement to improve patient care through new models of care
delivery (such as ACOs) has prompted an increasing number of calls for EHR design reform. The
AMIA EHR-2020 Task Force was released in early 2015 with the goal of identifying concrete,
actionable, and evidence-based recommendations for near-term EHR development.
(Kuhn T, Basch P, Barr M, Yackel T; Medical Informatics Committee of the American College
of Physicians. Ann Intern Med. 2015 Feb 17;162(4):301-3.
Payne TH, Corley S, Cullen TA, Gandhi TK, Harrington L, Kuperman GJ, Mattison JE, McCallie
DP, McDonald CJ, Tang PC, Tierney WM, Weaver C, Weir CR, Zaroukian MH. Report of the
AMIA EHR-2020 Task Force on the status and future direction of EHRs. J Am Med Inform
Assoc. 2015 Sep;22(5):1102-10. doi: 10.1093/jamia/ocv066. Epub 2015 May 28)
Page | 120d
T04-12:
Which of the following statements about Health Information Exchanges is true?
a) Studies of HIEs that used study designs having strong internal validity were significantly
more likely than studies not using such designs to find a benefit from HIEs for all
outcomes.
b) HIEs have spread across the US, are financially viable, and are very mature.
c) *Emerging vendor-driven interoperability initiatives may change the way we view and
value health information exchange
d) There are many rigorous studies that link HIE adoption and health outcomes.
Correct answer: c) Emerging vendor-driven interoperability initiatives may change the way we
view and value health information exchange
Explanation: Under the MU regulations, two-thirds of hospitals and almost half of physician
practices are now engaged in some type of HIE with outside organizations. Despite many
observational studies finding a beneficial relationship between HIE and outcomes, there are
few rigorous studies, likely because studying HIEs is very complex and it is difficult to define
who whom the costs and benefits accrue. (Rahurkar S, Vest JR, Menachemi N. Despite the
spread of health information exchange, there is little evidence of its impact on cost, use, and
quality of care. Health Aff (Millwood). 2015 Mar;34(3):477-83.)
T04-13:
The effective visual display of data and clinical decision support alerts is critical in supporting
safe, effective health care delivery. Which of the following statements is NOT true?
a) There are no regulations governing EHR usability.
b) Evidence-based design principles from existing EHR safety design guides (e.g. SAFER) are
not widely used by existing EHR systems
c) Clinical decision support rules frequently are integrated into the EHR without end-user
usability testing, resulting in poor adoption rates
d) Many current commercial EHRs have significant limitations in graphing capabilities of
laboratory test results and often display results in non-standardized fashion
e) *All of the above are true.
Correct answer: e) All of the above are true.
Explanation: With wider implementation of EHRs, more clinicians will rely on automatically
generated computerized displays that allow clear and accurate visual synthesis of data over
time. In addition, as health delivery models change and encourage multi-institutional
collaboration, differences in visual display of information may lead discordant interface designs.
See:
Page | 120e
Sittig DF, Murphy DR, Smith MW, Russo E, Wright A, Singh H. Graphical display of diagnostic
test results in electronic health records. J Am Med Inform Assoc. 2015 Jul;22(4):900-4;
Press A, et al. Usability testing of a complex clinical decision support tool in the emergency
department: lessons learned. JMIR Human Factors 2015;2(2): e14)
T04-14:
The drive for value- over volume-based reimbursement is leading to the greater adoption of
new models of care delivery. The patient-centered medical home (PCMH) model of primary
care is being implemented widely. Which of the following statements is FALSE regarding the
role of the EHR in PCMH?
a) The PCMH typically involves EHRs, organizational practice change, and payment reform,
although its effects on quality are unclear.
b) In a study of PCMH adoption vs non-PCMH adoption, with and without EHR support, the
PCMH group improved significantly more over time than either the paper group or the
EHR group in 4 out of 10 process measures
c) The aspects of the PCMH that drove improvement are distinct from, but may be enabled
by, the EHR.
d) *None of the above
Correct answer: d) None of the above.
Explanation: All of the statements are true. It is nearly impossible to do a randomized
controlled trial of a PCMH (or other health care delivery models) because of complexity and the
difficulty for any organization to stand up two parallel, but contrasting models of care; changing
health delivery models is a multicomponent intervention. (ref: Kern LM, Edwards A, Kaushal R.
The patient-centered medical home, electronic health records, and quality of care. Ann Intern
Med. 2014 Jun 3;160(11):741-9. doi: 10.7326/M13-1798.)
Patient Portals and Patient Empowerment - Paul Fu, Jr.
T04-15:
What is the role of the patient portal in patient empowerment?
a) Required by the EHR Meaningful Use
b) Change outcomes and behavior
c) None, there is no literature except diabetes
d) *All of the above
Correct answer: d) All of the above.
Explanation: While it is required for MU and there are conceptual models to support its use,
more study is needed to prove that patient engagement using HIT changes outcomes and
behavior.
Page | 120f
T04-16:
What is the most effective model for a patient portal?
a) Tethered to a sponsoring EHR vendor
b) Tethered to a sponsoring data source
c) *Tethered to a sponsoring provider organization
d) Untethered
Correct answer: c) Tethered to a sponsoring provider organization
Explanation: Patients prioritize data personalization and provider communication. Meaningful
Use has changed the dynamic so that patients are incentivized to use systems where there is
available data and they can interact with their providers.
T04-17:
What are the challenges and barriers involved in deploying a patient portal?
a) Policies
b) End-user support
c) Use case diversity
Correct answer: d) All of the above
Explanation: PHR or patient portal challenges and barriers include strategic issues (e.g. policies
and procedures, institutional ownership and alignment), implementation considerations (e.g.
diverse+complex use cases, misaligned workflow), and technical concerns (e.g. end-user
support, language support)
T04-18:
What factors predispose towards patient portal adoption?
a) Caregivers of elderly patients
b) Parents of young children
c) Patients with chronic conditions
d) Provider acceptance and promotion
Explanation: All the above, as well as frequent users of health care services, ease of
registration, and perceived privacy and security. Ethnic minorities (African American, Latino,
Asian), healthy adults, and those with lower education levels were less likely to adopt patient
portals.
Systems
Stanley M. Huff
T05-1:
a) *The complexity of modern medicine exceeds the limits of the unaided human mind.
b) Proper and rigorous medical education can lead to error-free medical management.
c) Physicians can reliably manage up to 10 parameters when making medical decisions.
d) Physicians actions match the facts that they have been taught.
Correct answer: a) The complexity of modern medicine exceeds the limits of the unaided
human mind.
Explanation: a) is true based on a quote from David Eddy.
b) is false. Clement McDonald published an article that showed education does not lead to
perfect decision making and its effects are only temporary.
c) is false, people can only use 5+/- two parameters in reliably making decisions.
d) is false because studies have shown that people are not perfect information processors and
they make errors even when they know the correct facts.
T05-2:
Which one of the following things is NOT needed to enable true data interoperability?
a) Detailed clinical information models.
b) *A knowledge repository for storing medical logic modules.
c) The detailed clinical information models need to be coupled (bound) to standard
terminologies.
d) Specification of Application Programmer Interfaces.
e) Open sharing of models, terminology, and APIs.
Correct answer: b) A knowledge repository for storing medical logic modules.
Explanation: All of the statements are true except b. A knowledge repository is not needed for
data interoperability.
Page | 121
T05-3:
Which of the following statements is true about SNOMED CT?
a) There is only one way to represent a given concept using SNOMED CT codes.
b) *SNOMED CT has many concepts that are not fully defined.
c) All needed medical concepts exist in SNOMED CT.
d) There is a clean division of labor between the content of SNOMED CT and LOINC codes.
Correct answer: b) SNOMED CT has many concepts that are not fully defined.
Explanation: A given concept can often be represented using different combinations of
SNOMED CT codes. Not all needed concepts exist in SNOMED CT, and concepts are still being
added. SNOMED CT observables overlap in meaning with LOINC codes. SNOMED CT has
primitive concepts and fully defined concepts. Primitive concepts are not fully defined. Other
concepts are only partially defined.
T05-4:
Which one of the following statements is NOT true?
a) Many countries and organizations worldwide are creating detailed clinical information
models.
b) Clinical information models need to be created for all aspects and domains of clinical
medicine.
c) Information models can be abstract and general, or very detailed and specific.
d) *We need to agree on a universal information model to enable interoperability.
Correct answer: d) We need to agree on a universal information model to enable
interoperability.
Explanation: HL7, openEHR, IHTSDO, Results for Care, CEN and ISO, and many other
organizations and countries are creating detailed clinical models. Models need to be created
for all aspects of medicine where information is to be exchanged. It is very useful to have a
core reference model from which more detailed models can be created by constraint. We dont
need to agree on one universal information model, since a single model is not convenient for all
purposes, and models can be converted into other isosemantic forms as needed.
Page | 122
T05-5:
Which one of the following is a goal of the Clinical Information Modeling Initiative (CIMI)?
a) *Create a shared repository of detailed clinical models.
b) Use any standard formal modeling language to represent the models.
c) Define standard application programmer interfaces (APIs) for using the models for
information sharing.
d) Use data types from standard programming languages as the elemental building blocks
of the models.
Correct answer: a) Create a shared repository of detailed clinical models.
Explanation: CIMI is concerned with making logical models that are bound to standard
terminologies, including the definition of primitive data types. CIMI does not plan to define
standard APIs.
T05-6:
Which one of the following statements about medical terminologies is true?
a) *Terminology tables or terminology services are needed for clinical information
modeling.
b) Terminologies are the same as ontologies.
c) Terminologies are organized according to self-evident hierarchies.
d) Terminologies assume an open world view.
Correct answer: a) Terminology tables or terminology services are needed for clinical
information modeling.
Explanation: The models reference sets of codes (value sets, reference sets) from standard
terminologies which are not directly included in the models. In order to understand the
information representation completely, there is a need to have terminology resources
available. Terminologies have some aspects of ontologies, but ontologies support more
sophisticated representations of knowledge than terminologies. Terminologies are organized
according to use case specific views of relationships. Terminologies have no standard approach
to open versus closed views of the world.
Page | 123
T05-7:
An evaluation style model would be more appropriate to use than an assertion style model
when:
a) *A particular attribute of the patient is being assessed
b) A condition or disease state that pertains to the whole individual is being assessed
c) A detailed statistical analysis will be performed
d) Very few measurements will be conducted
Correct answer: a) A particular attribute of the patient is being assessed
T05-8:
Which one of the following statements is true about representing subject of information in
detailed clinical information models?
a) Subject of information is always an independent attribute in models.
b) *Some modeling styles for subject of information could lead to combinatorial
explosion.
c) Representation of subject of information is unrelated to the representation of family
history information.
d) Subject of information is managed in one consistent style within the LOINC terminology.
Correct answer: b) Some modeling styles for subject of information could lead to
combinatorial explosion.
Explanation: Subject of information is often represented as part of a pre-coordinated concept
rather than as an independent attribute in the model. One kind of subject of information is
when the subject is a group of people who are the ancestors of the patient. In this case, the
representation of subject overlaps with the concept of family history. LOINC has codes that
pre-coordinate subject of information as well as codes that are meant for post-coordination of
subject.
T05-9:
Which one of the following statements about modeling strategies is true?
a) There is typically a best way to model a given type of clinical data.
b) *Two models can be isosemantic, that is, they have exactly the same information
content, but they have different representations.
c) Post coordination is better than precoordination.
d) The most popular medical modeling language is Abstract Syntax Notation 1 (ASN.1)
Page | 124
Correct answer: b) Two models can be isosemantic, that is, they have exactly the same
information content, but they have different representations.
Explanation: Many people argue that there is a single best model for a given kind of clinical
data. There are many ways to model a given kind of clinical data. Which style of model is best is
usually context dependent, not universal. Some models are easier for data entry while other
models are computationally more intuitive and efficient. Precoordination is often more efficient
for data entry. Archetype definition language is probably the most popular medical modeling
language. B is the definition of isosemantic models.
T05-10:
Which one of the following statements is NOT a requirement for good models?
a) Models are unambiguous.
b) *Models are independent of the terminology used for semantic binding.
c) Models are consistent across medical domains.
d) Models evolve gracefully with the addition of new knowledge.
Correct answer: b) Models are independent of the terminology used for semantic binding.
Explanation: The elimination of ambiguity is one of the primary purposes of detailed clinical
modeling. The consistent representation of the same data across different medical domains
makes the models easier to use and understand. Graceful evolution is important because
medical knowledge is always increasing and you want to preserve consistency of stored data
across long periods of time. B is false because the meaning of models is entirely dependent on
the terminology concepts that they are bound to.
Benson Munger; John Finnell
T06-1:
a) * None of the above is used as an option
b) Terms in the options are not stated consistently
c) The item contains superfluous information
d) The item contains tricky language in the stem
Correct answer: a) None of the above is used as an option
Page | 125
Explanation: Testwiseness allows exam takers who do not know the test content to guess the
correct answer or eliminate foils. None of the above is often used by test item writers as a
catch-all when they have no additional legitimate foils for the correct answer. None of the
above is also strongly discouraged as an item writing option.
T06 2:
a) An item is partially true
b) Language in the foils is not parallel
c) *The stem and a foil contain the same word
d) The item contains the word always
Correct answer: c) The stem and a foil contain the same word
Explanation: Testwiseness allows exam takers who do not know the test content to guess the
correct answer or eliminate foils. Item writers often fall into the trap of being connected to the
question content when constructing the correct answer. When this happens they often use the
same word or phrase.
T06 3:
In a multiple choice test item which one of the following is an example of Irrelevant Difficulty?
a) A foil does not logically follow from the stem
b) A subset of foils is collectively exhaustive
c) *The answer is connected to a previous item
d) The stem is negatively phrased
Correct answer: c) The answer is connected to a previous item
Explanation: Irrelevant difficulty occurs when a test item contains information that makes the
item more difficult without adding useful information or by creating connections that are unfair
to the exam taker. An example of this is connecting one item to another item in the exam. This
also reduces the number of independent testing points in the exam.
Page | 126
T06 4:
The length of an item stem should be which one of the following in relation to the responses?
a) Approximately the same
b) *Longer than the responses
c) Shorter than the responses
d) There is no standard
Correct answer: b) Longer than the responses
Explanation: Test item writers are strongly encouraged to put all the relevant information in the
stem of the item. The stem should contain a legitimate question and the foils should not force
the test taker to read through extensive and complex options.
T06 5:
A poorly constructed test exhibits which one of the following characteristics:
a) Content validity
b) Consistency
c) Reliability
d) *Standard length
Correct answer: d) Standard length
Explanation: A well constructed test should be reproducible in a similar population (reliability),
logically relate to the content being taught (content validity) and have items that are consistent
in format (consistency). There is no standard length for a test. There are minimums for
purposes of reliability but the total length can vary according to the demands of the content
and the time available.
Page | 127
T06 6:
Other things being equal, the highest level of test item discrimination would be achieved with
which one of the following levels of difficulty?
a) 90%
b) 75%
c) *50%
d) 25%
e) 10%
Correct answer: c) 50%
Explanation: An item difficulty between 40-60% provides the maximum amount of separation
between individuals who know the content and those that do not (discrimination). Most test
developers prefer to have items that fall between 30 and 70%.
T06 7:
Which one of the following steps should be part of developing a high quality test item?
a) Do a thorough literature search
b) Construct 5-6 plausible foils
c) Construct the foils first
d) *Write out the testing point
Correct answer: d) Write out the testing point
Explanation:
Writing out the testing point first define the content under consideration and the question
being tested. This guides the development of the stem and the options.
T06 8:
The City of Chicago
1.
Became a great trade center
2.
Became an early rail center
3.
Elected a second Mayor Daley
4.
Had a great fire in 1871
Page | 128
The test item above is poorly constructed because it,

a) *Fails to ask a definite question
b) Has foils that are too long
c) Has two foils that start with the same word
d) Lacks a definition in the stem
Correct answer: a) Fails to ask a definite question
Explanation:
The stem of a multiple choice item should contain a complete question that would allow the
test taker who is familiar with the content to develop an answer without accessing the item
options.
T06 9:
Where did American Slavery first start?
1.
Jamestown
2.
New York
3.
Pennsylvania
4.
Plymouth
The test item above is poorly constructed because the foils,
a) Are in alphabetical order
b) Are too specific
c) *Belong to different classes
d) Lack city/state names
Correct answer: c) Belong to different classes
Explanation:
The options for a multiple choice item (MCQ) should all reside in the same class. In this item
there are both city and state names. This requires the test taker to switch between different
concepts in reviewing the options.
Page | 129
T06 10:
How long do historians think the first Thanksgiving lasted?
1.
2.
3.
4.
One day
Three to four days
Two days
Two to three days
The test item above is poorly constructed because the foils are:
a) Independent
b) *Overlap times
c) Too short
d) Too specific
Correct answer: b) Overlap times
Explanation: The options for a multiple choice item (MCQ) must be independent. In this item
both three days and two days are contained in two options.
National Board of Medical Examiners, Constructing Written Test Questions for the Basic and
Clinical Sciences, Third Edition (revised), 2002.
Page | 130
Stephen J. Morgan; David M. Newman; Walton Sumner; Michael E. Kordek
WG09-1:
EHRs that use process control charts for population health measures usually do not provide
similar tools for time series of data in individual patient's records because :
a) Irregular intervals between observations compromise validity
b) *Physicians typically have managed patients without such charts
c) The statistical methods require aggregate data at each time point
d) Visual cues from an individuals data time series are often misleading
Correct answer: b) Physicians typically have managed patients without such charts
Explanation:
We are not aware of a reason to avoid using process control charts for analyzing individual
patients time series data, except that physicians never have done so. The methodology for
analyzing serial measures of an individual is well established. Irregular intervals are not an
issue, and the length of intervals is itself a valid measure to analyze with process control charts.
Visual cues are entirely valid, and may make a persuasive teaching tool when shown to the
patient (see, metformin does change your blood sugar).
Oniki, T. A., Clemmer, T. P., Arthur, L. K., & Linford, L. H. (1995). Using statistical quality control
techniques to monitor blood glucose levels. Proc Annu Symp Comput Appl Med Care, 586-590.
Mohammed, M. A., Worthington, P., & Woodall, W. H. (2008). Plotting basic control charts:
tutorial notes for healthcare practitioners. Qual Saf Health Care, 17(2), 137-145.
WG09-2:
The Systems Engineering Initiative for Patient Safety (SEIPS) 2.0 model added which of these
concepts to the original Structure-Process-Outcome model:
a) Accountability (documentation of data sources and decision makers)
b) *Engagement (stakeholders have separate but collaborative activities)
c) Fail-Safe (likely lapses and mistakes are automatically flagged for review)
d) Security (physical and keyword requirement for entering and retrieving data)
Page | 131
Correct answer: b) Engagement (stakeholders have separate but collaborative activities)

Explanation: SEIPS 2.0 added configuration, engagement, and adaptation. From the authors
2013 abstract: Engagement highlights the dynamic, hierarchical, and interactive properties of
sociotechnical systems, making it possible to depict how health-related performance is shaped
at a moment in time.
Engagement conveys that various individuals and teams can perform health-related activities
separately and collaboratively. Engaged individuals often include patients, family caregivers,
and other non-professionals. Adaptation is introduced as a feedback mechanism that explains
how dynamic systems evolve in planned and unplanned ways.
Holden, R. J., Carayon, P., Gurses, A. P., Hoonakker, P., Hundt, A. S., Ozok, A. A., et al. (2013).
SEIPS 2.0: a human factors framework for studying and improving the work of healthcare
professionals and patients. Ergonomics, 56(11), 1669-1686.
Carayon P, Schoofs Hundt A., Karsh, B. T., Gurses, A. P., Alvarado, C. J., Smith M., et al. (2006).
Work system design for patient safety: the SEIPS model. Qual Saf Health Care, 15 Suppl 1, i5058.
WG09-3:
Usability is:
a) unprofitable and/or unproductive
b) qualitative, never quantitative
c) a personal feeling
d) *an academic discipline
Correct answer: d) academic discipline
Explanation: With regard to the science of Usability as applied to electronic health record
systems, there are discreet methods and measures used but often the results are not precise.
Measuring usability can combine qualitative and quantitative methods and can include user
feelings about the product in question.
Smith, K Tara (2011). "Needs Analysis: Or, How Do You Capture, Represent, and Validate User
Requirements in a Formal Manner/Notation before Design". In Karwowski, W.; Soares, M.M.;
Stanton, N.A. Human Factors and Ergonomics in Consumer Product Design: Methods and
Techniques (Handbook of Human Factors in Consumer Product Design). CRC Press.
Page | 132
WG09-4:
The origin of style sheets can be credited to:
a) Steve Jobs and Apple
b) Allen & Gates: Microsoft
c) IBM
d) The New York Times
e) *"Hart's Rules" for the Oxford University Press
Correct answer: e) Hart's Rules" for the Oxford University Press
Explanation: Horace Hart published his "rules" for the Oxford University Press in 1893, initially
in 1893. It has been revised significantly from its single page many times, with the most recent
edition published in 2012.
Hart, H (1905). Rules for Compositors and Readers at the University Press, Oxford (19th ed.).
Retrieved 26 September 2012.
WG09-5:
Which of these organizations have not written about good Electronic Health Record design?
a) NIST
b) FHIR
c) AHRQ
d) *Apple
Correct answer: d) Apple
Explanation: There are many organizations that have published guidelines on EHR design.
Although Apple has produced several applications for health software design, they have not
specifically created documentation on EHR software.
Page | 133
WG09-6:
In the care of children, what is the most important reason for an electronic record system to
document the time of birth?
a) Parents like to share the time of birth on social media
b) Knowing the time of birth is required to generate an entry into the patient database
registry
c) *Several laboratory "normal reference ranges" widely vary based on the exact age of
the patient
d) Determining the time of birth is important to identify products of a multiple gestation
Correct answer: c) Several laboratory "normal reference ranges" widely vary based on the
exact age of the patient
Explanation: Although B and D are often used in EHR systems to develop a record within the
system's database, in the clinical care of infants the time of birth can greatly affect normal
reference ranges. One example of this is testing of serum bilirubin. A serum level of 10 in an
infant that is more than 72 hours old is considered normal, while that same level in an infant
that is only 3 hours old can lead to significant morbidity and mortality.
Spooner, S.A.; Council on Clinical Information Technology, American Academy of Pediatrics.
Special requirements of electronic health record systems in pediatrics. Pediatrics. 2007
Mar;119(3):631-7.
WG09-7:
The typical workflow for primary care physicians using an EHR is
a) An approximately linear progression from chief complaint to history to physical to
assessments and plans
b) Comparable to the subjective-objective-assessment-plan workflow without EHRs, then
after-visit documentation
c) Review of patient complaints and automated reminders, then focused history, physical,
assessment, and plan
d) *Multiple cycles of problem identification, varied data gathering, documentation,
assessment, and planning
Correct answer: d) Multiple cycles of problem identification, varied data gathering,
documentation, assessment, and planning
Explanation:
Page | 134
Holman et al report that 10 primary care physicians seeing 20 randomly selected patients
completed an average of 37 tasks (range 17-74) with zero overlap in workflow after the 12th
task. Whether or not the physicians had an EHR, a cycle of activity was observed: (1) physicians
learned of a problem from the EHR, the patient, or by examination; (2) a relevant task is
performed, often gathering information by asking questions, reading the chart, and/or
performing a focused examination, but possibly pursuing a plan without more data gathering.
After gathering information, physicians using EHR were more likely than non-EHR users to
recommend or discuss treatment options or document patient information.
The take home point is that there is not even a relatively common predictable workflow in
primary care, but there are cycles of problem identification and response. This workflow
requires flexible documentation processes to achieve real time documentation.
Holman, G. T., Beasley, J. W., Karsh, B. T., Stone, J. A., Smith, P. D., & Wetterneck, T. B. (2015).
The myth of standardized workflow in primary care. J Am Med Inform Assoc.
WG09-8:
The ideal EHR documentation process would be?
a) Anticipatory
b) Comprehensive
c) *Almost invisible
d) Rapid
Correct answer c) Almost invisible
Explanation: According to a critique of current EHR by Richard Byyny, The technology should
be invisible, helping instead of hindering the physician. Current EHR can anticipate
documentation needs by presenting documentation templates based on previously recorded
data, such as chief complaints, and can offer or demand exhaustively complete documentation.
Few if any current EHR provide genuinely rapid documentation, although some are more rapid
than others. This is a major grievance among physicians. Nevertheless, even EHR with rapid
documentation capabilities interpose between the physician and the patient, creating
conversations that lean toward the interaction of a clerk recording why a customer is returning
merchandise.
Byyny, R. L. (2015). The tragedy of the electronic health record. Pharos Alpha Omega Alpha
Honor Med Soc, 78(3), 2-
Page | 135
WG09-9:
Out of all the child health specific needs for electronic health record systems which is the most
challenging to implement?
a) Age specific growth charts
b) Weight based medication dosing
c) *Patient portals
d) Blood pressure percentile calculations
Correct answer text: c) Patient portals
Explanation:
There are many specific features that are needed to take adequate care of children and each
poses its unique challenges. Weight based dosing is difficult when considering the desire for
intelligent rounding and blood pressure percentile calculation is complex, however the
challenges of patient portals exceeds them all. This is especially true when considering the
confidentiality and privacy needs of adolescents. The guidelines vary widely by state, and
developers must consider how to suppress and display diagnoses, medications, laboratory
results and appointments scheduled. Further complications include
adolescents with complex and special health care needs such as developmental disorders
preventing the typical methods of informed consent.
Committee on Adolescence; Council on Clinical and Information Technology, Blythe MJ, Del
Beccaro MA. Standards for health information technology to ensure adolescent privacy.
Pediatrics. 2012 Nov;130(5):987-90
Page | 136
Sunday, Nov. 15, 2015 Answers & Explanations

Charles Jaffe; Douglas B. Fridsma; Stan Huff; Christopher Chute; John D. Halamka; William E.
Hammond
T07-1:
Which is not a goal of the Argonaut project?
a) Create an implementation guide for the query/response of the Meaningful Use
Common Data Set
b) Create an implementation guide for the secure RESTful transport of data
c) *Replace the CCDA
d) Create an implementation guide for the creation of a trust fabric using OAUTH
Correct answer: c) Replace the CCDA
T07-2:
What is not yet in scope for the Argonaut project?
a) Enable applications to exchange data within an organization
b) Enable applications to exchange data between organizations
c) Provide a mechanism for querying a provider directory
d) *Provide a mechanism for uniquely identifying a patient
Correct answer: d) Provide a mechanism for uniquely identifying a patient
T07-3:
Which of the following is not a component of FHIR?
a) Profile
b) *Bundle
c) Resource
d) Extension
e) Paradigm
Page | 137
Correct answer: b) Bundle
T07-4:
Which of the following statements about FHIR is incorrect?
a) FHIR permits transporting data at the lowest levels of granularity.
b) *FHIR access and use require membership in HL7 International.
c) FHIR is service driven.
d) FHIR permits sending only the data that is required.
e) FHIR is based on a stateless protocol.
Correct answer: b) FHIR access and use require membership in HL7 International.
T07-5:
Which of the following standards is most useful in creating Restful interoperable services?
a) HL7 Version 2.X
b) HL7 Version 3
c) HL7 Clinical Document Architecture (CDA)
d) *HL7 FHIR
Correct answer: d) HL7 FHIR
T07-6:
Which terminology is most useful for coding the names of clinical observations?
a) SNOMED CT
b) RxNorm
c) *LOINC
d) RadLex
Correct answer: c) LOINC
Page | 138
T07-7:
The most recent draft standard of FHIR (DSTU 2.0) provides all of these enhancements EXCEPT?
a) Extends search and versioning significantly
b) Increases the power and reach of the conformance resources and tools
c) Defines a terminology service
d) *Enhances human readability
e) Broadens functionality to cover new clinical, administrative and financial areas
Correct answer: d) Enhances human readability
T07-8:
The many attributes of FHIR include all of these benefits EXCEPT
a) Shortens development time and reduces development costs
b) *Replaces V3
c) Relies upon the rule of 80/20
d) Evolved from HL7 standards that are the keystone of meaningful use
e) Written in a manner understood by both clinicians and technology developers
Correct answer: b) Replaces V3
T07-9:
Which of the following FHIR resources are at the right level of granularity for a resource?
a) Resting HR after exercise
b) Intervention
c) *Allergy
d) Hospital
Correct answer: c) Allergy
Page | 139
T07-10:
Which of the following principles are NOT part of the guiding manifesto for FHIR?
a) Focus on implementers
b) *Develop healthcare specific web technologies
c) Require human readability as base level of interoperability
d) Make content freely available
Correct answer: b) Develop healthcare specific web technologies
T07-11:
What are the key innovations in the ICD 11 project from an informatics perspective?
a) ICD 11 includes many more codes than ICD 10
b) ICD 11 includes traditional medicine codes
c) *ICD 11 has an underpinning semantic network from which it can derive an arbitrary
number of coding linearizations
d) ICD 11 will be available in multiple languages
Correct answer: c) ICD 11 has an underpinning semantic network from which it can derive an
arbitrary number of coding linearizations
T07-12:
What is the common framework between the ICD and SNOMED?
a) ICD and SNOMED are both classifications
b) *ICD and SNOMED share a common ontology
c) ICD and SNOMED include shared information
d) ICD and SNOMED are multilingual
Correct answer: b) ICD and SNOMED share a common ontology
Page | 140

Joseph W. Hales; Christopher Cimino
Definition
T13-1:
Biomedical informatics refers to the field of study of
a) the use of biomedical information in the assessment and delivery of medical care.
b) the use of computers and related technologies to manipulate biomedical information.
c) *the optimal use of biomedical information for problem solving and decision making.
d) all aspects of electronic health record systems.
Correct Answer: c) the optimal use of biomedical information for problem solving and
decision making.
Explanation: Variations of the definition of Medical Informatics have appeared with the AMIA
Academic Forum consensus (J Am Med Inform Assoc 2012;19:931-938) and the definition
offered by Ted Shortliffe (Shortliffe EH, Perreault LE, Wiederhold G, Fagan LM, eds. Medical
Informatics: Computer Applications in Health Care. Reading, MA.: Addison-Wesley; 1990.) being
similar and both frequently cited. The delivery of health care as is only one facet of medical
informatics as is the electronic medical record. While computers have become essential to
carrying out informatics research and innovations, it is the manipulation of information, not the
use of computers, that is central to the discipline. The manipulation of biomedical information
for the purposes of problem solving and decision making in multiple fields (e.g. care delivery,
genetics research) is what medical informatics is about.
Terminology
T13-2:
Controlled vocabularies have advantages and disadvantages as compared to uncontrolled text.
Which options best captures the most positive and most negative aspects of controlled
vocabularies?
a) Pro: vocabulary controlled by experts; Con: licensing fees drive up healthcare costs.
b) Pro: expansion of the caregivers vocabulary; Con: require tedious menus for selections.
c) *Pro: allow automatic analysis of health information; Con: Limit the caregivers
expressiveness.
d) Pro: single terminology shared for many purposes; Con: Competition between
vocabulary vendors makes it hard to choose the most long lasting solution.
Page | 141
Correct Answer: c) Pro: allow automatic analysis of health information; Con: Limit the
caregivers expressiveness.
Explanation: There is no requirement that a controlled vocabulary be a commercial product nor
is there agreement that even among the commercial products that the expert management is
the ideal approach. The implementation of selection of terms from a controlled vocabulary
need not be tied to the vocabularys hierarchy structure nor does access to the terminology
mean the care giver understands the use of all the terms in that terminology. The use of a
controlled vocabulary for more than one purpose is always a compromise that grows greater as
the two purposes diverge. The control place on a vocabulary by definition constricts the
context in which its terms can be used and therefore constrains the expressiveness of the user.
However, that same constraint also allows for direct comparison of the use of those terms in
multiple instances.
T13-3:
The ideal controlled vocabulary
a) *provides coverage of the topic area with minimal overlap between terms.
b) provides an other or not otherwise specified category or modifier in all hierarchies.
c) places each term exactly once within the terminology hierarchy.
d) minimizes the depth of the terminology hierarchy.
Correct Answer: a) provides coverage of the topic area with minimal overlap between terms.
Explanation: The use of other and not otherwise specified renders a vocabulary flawed
since it can not change over time and still maintain compatibility since the definition of other
changes as more specific terms are added and therefore excluded from the meaning of other.
A vocabulary which restricts terms to a single hierarchy renders the vocabulary flawed since
many terms can logically be organized in multiple categories (e.g. an antibiotic is a chemical and
a treatment). A vocabulary with limited hierarchy depth is flawed since it places an arbitrary
limit on the granularity of the terms and their organization. An ideal vocabulary always has a
term that can be used to code the topic of interest (coverage) and always has exactly one such
term (minimal overlap).
Page | 142
T13-4:
The National Library of Medicines unified medical language system (UMLS) semantic network is
used
a) to compute the definition of medical terms for the purpose of decision support in the
information sources map.
b) to drive a natural language processing algorithm in the lexical matcher.
c) for research purposes but connects only minimally to the other UMLS components.
d) *to assist in matching new terms to existing concepts within the metathesaurus.
Correct Answer: d) to assist in matching new terms to existing concepts within the
metathesaurus.
Explanation: The National Library of Medicine Semantic Network is one of several knowledge
sources of the Unified Medical Language System (http://www.nlm.nih.gov/research/umls/). It
directs directly to the metathesaurus because every concept in the metathesaurus has at least
one and often two semantic network terms associated with it. It is relatively small and has a
very coarse granularity which makes it easy to select semantic types for new terms but it is a
poor tool for natural language processing or formal semantic type computations. A new
candidate vocabulary that can have semantic types attached becomes much easier to
incorporate into the metathesaurus because the domain of possible concept matches is
dramatically narrowed.
Medical Records
T13-5:
The primary driving force behind the development of electronic medical records has historically
been to
a) provide medico-legal documentation to address malpractice suits.
b) allow better communication between multiple care givers.
c) *provide faster submission of coded bills to insurance companies.
d) track and improve the quality of medical care.
Correct Answer: c) provide faster submission of coded bills to insurance companies.
Explanation: The medical record is used for multiple purposes including improved
communication between care providers, medico-legal documentation, and tracking procedures
and other information for the purposes of billing. The move from paper to electronic medical
records has been primarily justified to improve billing. This has a direct impact on revenues
which justified the expense. Although the move from paper to electronic records is a key step in
Page | 143
tracking quality of care, there is little evidence that there are cost savings sufficient to justify
the expenditure.
T13-6:
Health Level 7 (HL7) provides a standard that is supposed to increase the ease with which
different health information systems (ie pharmacy, lab, radiology) integrate into a single
medical record. Its primary approach to doing this is by defining standardized
a) table structures.
b) *messages.
c) vocabularies.
d) field types.
Correct Answer: b) messages.
Explanation: HL7 is a standardized messaging protocol. The 7 is a reference to the top layer of
the open system interconnection model of organizing protocol standards. It provides a
mechanism to identify any of many possible standardized vocabularies to be used in the
message contents. While the structure and format of these messages has implications for how
such information should be stored in a database (i.e. what field types and table structure to
use) it does not primarily define standards for fields or tables.
Decision Support
T13-7:
Over time, well structured rule-based reminder systems will
a) meet initial resistance to change physician behavior but eventually they will adopt the
new practices.
b) quickly produce a change in physician behavior that is sustained even after the
reminders are discontinued.
c) *quickly produce a change in physician behavior but their behavior returns to baseline
shortly after the reminders are discontinued.
d) meet initial resistance to change physician behavior and ultimately fail to create any
lasting change unless hospital policies are put in place and enforced to require the
change.
Correct Answer: c) quickly produce a change in physician behavior but their behavior returns
to baseline shortly after the reminders are discontinued.
Explanation: The classic study done by Clem McDonald (McDonald CJ. Protocol-based computer
reminders, the quality of care and the non-perfectability of man. N Engl J Med. 1976 Dec
Page | 144
9;295(24):1351-5.) showed that physicians were quickly responsive to appropriate alerts but
that their behaviors quickly returned to baseline when the alerts were removed. This should
not be confused with alert fatigue which is caused by excessive numbers of alerts, most of
which do not require immediate action.
T13-8:
The Health Information Technology for Economic and Clinical Health Act (HITECH) has become a
major driver of clinical decision support implementation because it
a) *provides incentives for providers who can demonstrate they have used electronic
records in a meaningful way that measures quantity and quality of health care
delivered.
b) requires all certified electronic health record systems to implement quality
improvement applications that can demonstrate a meaningful improvement in health
care quality over time.
c) provides funding for clinical decision support research.
d) requires Medicare and Medicare patients in programs receiving government funding to
allow their de-identified patient information to be used for research purposes.
Correct Answer: a) provides incentives for providers who can demonstrate they have used
electronic records in a meaningful way that measures quantity and quality of health care
delivered.
Explanation: At this time the U.S. government does not require providers to maintain electronic
medical records nor does it require patients to submit their clinical data for research purposes.
They have instead attempted to create incentives for providers to adopt electronic records. The
Affordable Care Act first provided such incentives but required providers to show meaningful
use of those records without providing clear definitions of what meaningful use meant. The
Health Information Technology for Economic and Clinical Health (HITECH) Act is part of the
American Recovery and Reinvestment Act (ARRA) further defines the mechanism by which
meaningful use will be defined. All of the core objectives required to meet the meaningful
use definition are aimed at improving and measuring the quantity and quality of health care
being delivered. This information directly feeds into clinical decision support making and one of
the objectives is specifically to implement clinical decision support rules. While some of the
funding can be construed is being for research of clinical decision support, the implementation
has occurred in advance of any research efforts.
Page | 145
T13-9:
The central dogma of biology describes the
a) basis for evolutionary development of life
b) *fundamental relationship between DNA and its expression in proteins (and
ultimately living organisms)
c) life cycle of living organisms, from birth or multiplication to death
Correct Answer: b) fundamental relationship between DNA and its expression in proteins (and
ultimately living organisms)
Explanation: The classic view of the central dogma of biology as defined by James Crick
describes the flow of genetic information, specifically sequence information, in living systems,
from DNA to RNA to protein. While evolutionary changes may be the result of mutations or
variations in the flow of genetic information, the dogma does not describe evolution or the
formation of life. The dogma is intended to describe the flow of information at a micro level,
not the entirety of the life-death cycle of living organisms.
T13-10:
The Gene Ontology is an initiative to
a) *unify the representation of genetic information across species.
b) create a controlled vocabulary of the human genome
c) name all of the genetic products of all possible combinations of gene sequences.
Correct Answer: a) unify the representation of genetic information across species.
Explanation: The Gene Ontology is an initiative that seeks to develop and maintain a controlled
vocabulary of the attributes of all genes and gene products across species; and to annotate
genes and gene products using a standardized markup language that makes the representation
machine readable. The Gene Ontology is not limited to human genetic data, but attempts to
unify the representation across all species. Further, it is not just a nomenclature or controlled
vocabulary of genes, but provides for standardized representation of the attributes of genes,
gene products and permits expressions for molecular functions and biologic processes.
Page | 146
Steve Demuth, Matthew Burton, Robert A. Greenes, Davide Sottara, Keith Toussaint
S01-1:
Considering an interoperable app deployment infrastructure, what are the essential features
that would benefit mostly from the curation, deployment and execution of a robust set of
knowledge resources externally to EHRs?
a) The resources can be edited in a common place.
b) The resources can be incorporated in services.
c) *Apps that deliver continuity of care, workflow support, and decision support must
bridge between EHRs.
d) There needs to be transparency of knowledge used in systems.
Correct answer: c) Apps that deliver continuity of care, workflow support, and decision
support must bridge between EHRs.
Explanation:
Although all of the answers reflect benefits from having an external set of knowledge
resources, answer C is the only one that reflects functionality that could not be delivered
without this capability. For example, care continuity across venues of care will need to have
workflow and business logic concerning tasks that must be performed in conjunction with data
from several possible EHRs and other data sources, and provide capabilities for sequencing of
events and decision support operating on that set of data.
S01-2:
Considering next-generation interoperable apps as the driving use cases, there are a number of
potential benefits to having a testbed/sandbox for communal development. However, what
would be the primary goal it could provide?
a) A place to assemble the needed resources, since they are hard to obtain.
b) A place to obtain needed technical support, since this is a new framework for app
deployment.
c) *A place to both evolve the infrastructure itself and support development of apps,
because the compelling use cases involve need for both.
d) A way to provide a bridge to deployment and commercialization, because the
ecosystem is not yet mature.
Correct answer: c) A place to both evolve the infrastructure itself and support development of
apps, because the compelling use cases involve need for both.
Explanation:
A testbed/sandbox will make it easier to do all of the functions listed. However, the advanced
Page | 147
functionality needed for continuity of care and the other use cases is not currently supported
fully by any reference implementation. Testbed environments are needed to both evolve the
next generation of reference architecture, and also support app development within currently
defined capabilities. It will in fact be the case that difficulties encountered with app
development for some of the use cases will help to drive and prioritize those features of future
iterations of the architecture.
S01-3:
Considering the characteristics of the prevalent service models in cloud computing, which
service models or combinations thereof must be present to enable open architectures
supporting the execution of clinical pathways and care coordination?
a) Software-as-a-Service only
b) Infrastructure-as-a-Service only
c) *Platform-as-a-Service and Infrastructure-as-a-Service
d) Software-as-a-Service and Infrastructure-as-a-Service
Correct answer: c) Platform-as-a-Service and Infrastructure-as-a-Service
Explanation:
In order to enable agility and flexibility, both platform and infrastructure services are required.
Infrastructure-only services do not offer the higher-level of domain-specific functionality
required for health care (or existing services like AWS and Azure would already be in
widespread use). Software-as-a-service functionality alone is merely another form of silo-ed,
proprietary capabilities - only deployed in a shared computing environment.
S01-4:
What are the central architectural principles employed in Mayo Clinics emerging knowledge
framework, as initially demonstrated by the EASE solution:
a) Deployment on public cloud computing infrastructure is a key aspect for scalability
b) Uses exclusively open-source implementations of architectural components
c) *Adequate separation of concerns at the data, knowledge, service and application
layers
d) Proves that the adoption of open interface standards alone is sufficient to enable
interoperability
Correct answer: c) Adequate separation of concerns at the data, knowledge, service and
application layers
Explanation:
Separation of concerns as represented in a three-tiered architecture (data services,
business/informatics services, app/presentation architecture), further refined, mediated by
Page | 148
open interfaces, knowledge and data representation formats and messaging standards enables
shared services to be used at each layer of the architecture while at the same time avoiding the
trap of silo-ed solutions that currently predominate. Moreover, it enables the reuse of content
and capabilities, simplifying their management, accelerating the time to delivery of applications
built on top of the architecture.
S01-5:
What are central tenets the Healthcare Services Platform Consortium (HSPC) seeks to enable
a) Enable only non-profit business models and transactions
b) *Facilitate clinical application interoperability and data sharing
c) Initiate full migration from existing EHRs and clinical systems
d) Establish a single reference architecture upon which all EHR systems are expected to
converge
Correct answer: b) Facilitate clinical application interoperability and data sharing
Explanation:
These aims can be achieved via a combination of for-profit or non-profit business models.
Further, this aim can still be achieved via multiple (reference) implementations as long as those
adhere to the same principles of separation of concerns, open interfaces, well-defined behavior
and a common underlying architectural pattern can be established. This architecture does not
need to be unique, but there has to be sufficient degree of interoperability to enable the
sharing of data, knowledge and functionalities.
Finally, existing solutions are welcome to leverage the modern techniques recommended
herein and are encouraged to do so.
S01-6:
A current limitation for the evolution of EHR solutions that this panel seeks to overcome is:
a) *Silos of data and proprietary architectures
b) For-profit EHR and other clinical and life systems information systems
c) Use of existing messaging and data format standards
d) Adherence to Meaningful Use-related EHR certification
Correct answer: a) Silos of data and proprietary architectures
Explanation:
This answer is taken directly from the abstract.
For-profit EHR solutions are fully embraced by this panels model as long as the architectural
separation of concerns and open interface tenets are adhered to.
Page | 149
HSPC explicitly leverages existing standards (HL7, OMG, W3C, etc.) in its solutions
Existing meaningful use requirements and testing standards are also openly embraced.
Context?
Emil Chiauzzi; Kevin Patrick; Cinnamon Bloss; Ernesto Ramirez
S03-1:
The Quantified Self community is a growing network of over 100 MeetUps around the world,
conference events, and focused meetings. At these different gatherings there are many ways
individuals expose their interest in self-tracking tools, the data they gather, and their
experiences. Of the following, what is the core focus of the community?
a) The gather and disseminate information about device accuracy and validity.
b) To develop methods to interface personal data with the healthcare system.
c) *To share real-world, personal examples of learning from the process of self-tracking
and self-experimentation.
d) To generate generalizable findings for research publication.
Correct answer: c) To share real-world, personal examples of learning from the process of
self-tracking and self-experimentation.
Explanation: The goal of the Quantified Self community is to develop and share real-world
examples of how personal data is shaping individuals lives. Each Quantified Self show&tell talk:
focuses on answering these three questions: What did you do? How did you do it? What did
you learn? This focus on personal learning helps to highlight the individual and role data have
played during the course of their experience.
S03-2:
There is increasing demand to incorporate patient-generated health data into the medical
record. Much of that data may have clinical significance. However, access to the data that apps
and devices create is unregulated, and implementation differs across companies.
Which of the following describes a best-case scenario for personal data access to data from
consumer health and wellness tools/apps?
a) Companies only integrate with personal medical records and individuals are allowed to
access it through their healthcare system.
b) Companies provide an option for the individual to export the data from their system.
c) Companies only share data with qualified researchers and research institutions.
d) *Companies provide access to the individual in a structured, machine-readable format
and allow data sharing through a well-documented API.
Page | 150
Correct answer: d) Companies provide access to the individual in a structured, machinereadable format and allow data sharing through a well-documented API.
Explanation: Access and control of the personal health data is an important feature that will
have important implications for individuals, researchers, and healthcare institutions. Companies
should adopt a user-centered vision of data access and provide well-structured data access and
patient-centered control of data sharing through APIs.
S03-3:
There has been much discussion of the use of wearable technologies as a means of assessing
the status of chronic disease patients in their natural environments. What principle should be
considered most in the application of wearable technologies to chronic disease management?
a) It is best to focus on younger, more active chronic disease populations
b) Wearable devices should only be used if they can account for all physical activity
c) Patients with chronic disease are unable to derive much benefit from devices that stress
social comparison and athletic competition
d) *The measurement need defined by patients should drive the decision to utilize
devices in chronic disease management.
Correct answer: d) The measurement need defined by patients should drive the decision to
utilize devices in chronic disease management.
Explanation: Personal data collection offers the potential for improved collaboration between
patients and their providers, but patients will need to be convinced about the value of their
data in medical decision making and treatment. If patients recognize this need, they may utilize
wearable devices to understand their disease progression and work with their providers to
integrate this data into the clinical workflow. Factors related to the age, a high level of
accuracy, and competitive features are less likely to be barriers if patients recognize that
wearables may provide a practical personal data source.
S03-4:
Many patients utilize wearable devices as a means of improving physical activity and improving
their self-management of chronic diseases. Although these devices often include behavioral
strategies such as virtual awards and ongoing feedback, they often lack the critical elements
required to enhance behavior change. Which behavior change element is frequently lacking in
the wearable activity experience?
a) Self-monitoring
b) *Behavioral instruction
c) Goal-setting
d) Feedback
Page | 151
Correct answer: b) Behavioral instruction

Explanation: Wearables are useful in measuring the outcome of health behaviors, thus are able
to provide feedback, self-monitoring, and goal-setting. Wearables are less useful in providing
users with behavioral instructions that assist in developing behavior change plans, overcoming
a lack of readiness, or problem-solving when plans go awry. Development of these capabilities
would expand the functions of wearables beyond measurement tools to behavior change tools.
S03-5:
The popularity of health self-tracking through the use of consumer health devices has primarily
been confined to fitness and wellness contexts, but there has been increasing interest in
leveraging this health data in clinical decision-making and care delivery. Why?
a) Personal tracking devices easily interface with electronic medical records
b) *Personal tracking devices provide a highly granular picture of health for an individual
c) Personal tracking devices are generally covered by insurance
d) Personal tracking devices are well understood by physicians
Correct answer: b) Personal tracking devices provide a highly granular picture of health for an
individual
Explanation: The use of personal tracking devices provides a passive means of gathering
personal health data in specific time segments for particular health behaviors - activity, sleep,
and even biometric measures such as heart rate. Such data allows providers to view an
individual's health at a highly granular level over time
S03-6:
Privacy is an issue that has been discussed at length in a range of fields, including law,
philosophy, and medicine. What is the agreed upon definition of privacy?
a) The well accepted definition of privacy is a legal one
b) Scholars agree that each individuals definition of privacy should be taken as accurate
c) The well accepted definition of privacy is in regards to informational privacy
d) *Scholars and experts do not have a universally agreed upon definition of privacy
Correct answer: d) Scholars and experts do not have a universally agreed upon definition of
privacy
Explanation: Privacy continues to be a difficult concept to define in many fields, and there is no
agreed upon definition that has been applied to the use of personal health data. although many
health consumers believe that sharing personal health data with their providers is acceptable,
Page | 152
there are concerns about the sharing of such data outside of clinical encounters, even in
aggregate form.
Chiauzzi E, Rodarte C, DasMahapatra P. Patient-centered activity monitoring in the selfmanagement of chronic health conditions. BMC Med. 2015 Apr 9;13:77. doi: 10.1186/s12916015-0319-2
Health Data Exploration Project. Personal Data for the Public Good.
http://www.rwjf.org/content/dam/farm/reports/reports/2014/rwjf411080, San
Diego: UCSD, 2014.
Capturing Preventive Care Services: Comparing Data Obtained from Manual Chart Review,
Automated EHR Extraction, and Insurance Claims
S.R. Bailey; M. Hoopes; H. Angier; R. Gold; J. Heintzman; M. Marino; J.P. O'Malley; J.E. DeVoe
S09-1-1:
data. Assuming their EMR is comparable to the one presented, if they chose to use claims data,
which preventive service would be a good choice to assess?
a) *Mammography
b) Smoking status assessment
c) Blood pressure screening
d) BMI assessment
Correct answer: a) Mammography
Explanation: Claims data often fail to capture assessments/screenings that are not typically
billed for, including smoking status, BMI, and blood pressure. Since mammography is often a
billed service, this is the best choice.
S09-1-2:
Page | 153
data Previously, the clinic used manual chart review for projects such as these. They are
wondering if an automated extraction will be as accurate for a given measure.
Assuming their EMR is comparable to the one presented, which measure might be the best first
choice because of the close agreement between manual and automated chart extraction?
a) Chlamydia screening
b) Cervical cancer screening
c) *BMI assessment
d) Cholesterol screening
Correct answer: c) BMI assessment
Explanation:
The automated extraction agreed perfectly with the manual chart review for BMI and blood
pressure assessments. This is likely due to both of these services being performed in the
primary care setting and not referred elsewhere for completion.
Heintzman J, Bailey SR, Hoopes MJ, Le T, Gold R, OMalley JP, et al. Agreement of Medicaid
claims and electronic health records for assessing preventive care quality among adults.
JAMIA.2014;21(4):720-724.
Comparing Weight Redistribution and Distance Imputation Methods for Missing Data in
Clear-text and Encrypted Record Linkage
Toan C. Ong; Lisa Schilling; Michael G. Kahn
S09-2-1:
Researchers are investigating the association between household income and diabetes
diagnoses among patients between ages 18 and 65. Since diagnoses are stored in the hospitals
electronic health record (EHR) system and income information resides in an external sociodemographic database (e.g. census data), the final dataset used in this study must be created
by linking these two datasets. After reviewing the data elements, four variables which appear in
both data sets are used to link these data sets. These variables are: first name, last name, date
of birth and gender. The personally identifiable information (PII) variables (name and DOB) in
both datasets have to be encrypted prior to data sharing and hence impact the linkage process.
Page | 154
The linkage result was very poor. Only 30% of the patients in the clinical data set have
matching records in the socio-demographic dataset. Which of following statements about the
cause of this result is INCORRECT?
a) Many records in both datasets have typographical errors and the linkage method is
deterministic.
b) *Many records in both datasets are missing gender and the linkage method is
probabilistic.
c) There were a lot of missing data in both datasets and the FRIL-0 linkage method was
used.
d) Most of patients in the socio-demographic database belong have age 17 or less
Correct answer: b) Many records in both datasets are missing gender and the linkage method
is probabilistic.
Explanation:
a) is correct because deterministic record linkage method will only consider two values a
match if they are exactly the same. Therefore, deterministic record linkage methods
do not work well with data with a lot of typographical errors.
b) is incorrect because gender would be a low-weight linkage variable since it isnt very
discriminating and therefore not likely to be the reason for so many non-matches
c) is correct because if FRIL-0 assigns 0 distance to fields with missing data this will
significantly reduce the overall matching score. Low matching score results in the
determination of a non-match.
d) is correct because if all patients in the EHR dataset have age between 18 and 65 as
stated above and most of patients in the socio-demographic dataset have age less than
17, the number of overlapping patients between these two datasets is small. So even
the linkage method is effective, the number of matches is still small.
S09-2-2:
Which of the following statements is correct regarding the approaches to improve the linkage
result in the scenario above?
a) The problem with typographical errors can be resolved entirely by using probabilistic
record linkage methods
b) Imputation methods can improve the linkage performance for missing clear-text data
only and will not work with encrypted data
c) *Deterministic record linkage method will be improved by improving the accuracy of
gender data.
Page | 155
Correct answer: c) Deterministic record linkage method will be improved by improving the
accuracy of gender data.
Explanation:
a) is incorrect because the effectiveness of probabilistic record linkage methods
depends on the amount of typographical errors in the data. Probabilistic record linkage
can mitigate minor (small edit distance) typographical errors.
b) is incorrect because imputation methods such as Distance Imputation work with
encrypted data also.
c) is correct because even though gender usually have small contribution to the
determination of linked records, having accurate gender data will improve linkage
performance, especially when gender is being used as blocking variable.
Rethinking Document Retrieval for Scientific Literature: A Learning to Rank Approach
Jesse M. Lingeman; Hong Yu
S09-3-1:
Previous work in learning to rank scientific documents uses traditional Information Retrieval
datasets to obtain relevance judgments. These traditional methods include ranking by a panel
of experts (as in TREC datasets) and ranking using pseudo-relevance feedback (i.e., by
document similarity). For the learning to rank sub-task of related document search, these
same methods are often used. Instead of relying on these traditional methods, we asked the
authors to rank documents related to their work for us. What did we find?
a) *The document ranking by the authors differed significantly from the document
b) The document ranking by the authors was the statistically the same as the document
c) Documents ranked by traditional IR systems appeared to be more relevant than the
authors ranking.
d) The authors ranking is not a good indicator of relevance.
Correct answer: a) The document ranking by the authors differed significantly from the
document ranking by traditional IR systems.
Explanation: We found that the rankings provided by traditional IR systems such as pseudorelevance feedback or by Google Scholar were very different from the way the authors
organized them. The authors were organizing their rankings according to information not based
on text similarity.
Page | 156
S09-3-2:
Learning to rank methods work by decomposing the relationship between a query and a
document into a list of features. These features can include text similarity, document length,
document age, etc. We found two features that were negatively correlated with the rankings
given by the authors. What were they?
a) *Citation age and number of citations in the literature
b) Text similarity between abstracts and text similarity between titles
c) The number of times a cited paper is referenced in the query and the text similarity of
the words around each reference with the abstract.
d) The text similarity of the discussion/conclusion with both the title and abstract.
Correct answer: a) Citation age and number of citations in the literature
Explanation: The citation age and number of citations in the literature are potential indicators
of how foundational a document is to a field of research. For example, an older, higher cited
paper is likely to be a reference that started a particular line of research the research paper
might be following. However, it is unlikely that the foundational paper directly inspired the
research at hand.
Lin J, Wilbur WJ. PubMed related articles: a probabilistic topic-based model for content
similarity. BMC bioinformatics. 2007 Jan 1;8:4233.
Ren X, Liu J, Yu X, Khandelwal U, Gu Q, Wang L, et al. ClusCite: effective citation
recommendation by information network-based clustering. The 20th ACM SIGKDD international
conference. New York, New York, USA: ACM; 2014. 10 p.
Page | 157
Monday, Nov. 16, 2015 Answers & Explanations

Alan L. Rector; James R. Campbell; Bedirhan T. Ustun; Christopher G. Chute; Harold R. Solbrig
S13-1:
What is the difference in primary purpose between SNOMED and ICD?
a) Aim for comprehensiveness vs aim for international standardization
b) *Use for clinical care vs use for statistical returns
c) Use for clinical care vs use for reimbursement
d) Use for decision support vs use for patient coding
Correct answer: b) Use for clinical care vs use for statistical returns
Explanation: The history of the ICD is as a means of international recording. The declared aim of
SNOMED on its website is to support clinical care. No claims are made by SNOMED concerning
use for decision support. Although ICDxCM is used for reimbursement and SNOMED aspires to
be used in some areas for reimbursement, that is not the primary purpose of either.
S13-2:
How are residual categories, such as "other" or "not elsewhere classified" dealt with in the new
ICD 11 architecture?
a) They are used throughout the system as has always been the case.
b) They have been eliminated in favor of using the parent category, i.e. the most specific
category that can be fully justified.
c) *They exist in the Linearizations in the usual form and are linked if required into the
Foundation Layer using queries.
d) Special legacy codes are provided for backwards compatibility, but they are otherwise
deprecated.
Correct answer: c) They exist in the Linearizations in the usual form and are linked if required
into the Foundation Layer using queries.
Explanation: Residual codes make sense only in the context of a specific (version of) a
Linrearization. They are "residual" with respect to the classes selected for that linearization.
However, other linearizations may select other classes and therefore have different residuals.
Page | 158
S13-3:
Why do the linearization codes have to be linked to the Foundation Model & Common
Ontology by queries rather than by description logic expressions? For example, why would class
for "Hypertension but NOT Pregnancy" expressed in a description logic for the Common
Ontology not capture the intended meaning of "Hypertension excluding Diseases of Pregnancy"
in ICD?
a) Because it could not be queried with the usual ICD tools.
b) Because the meaning in the description logic would be inconsistent (i.e. unsatisfiable)
because it is open world.
c) Because the intended meaning in SNOMED is "not classified under in SNOMED" which is a
closed world query.
d) *Because the intended meaning in ICD is "not classified under in this llinearization"
which is a =meaningless in the Common Ontology & SNOMED.
Correct answer: d) Because the intended meaning in ICD is "not classified under in this
llinearization" which is a =meaningless in the Common Ontology & SNOMED.
Explanation: There are many possible answers but only d) is actually correct. a) Is irrelevant b) is
just false, and c) is likewise wrong. The references to open and closed worlds although relevant
to related questions are just confounders in this context.
S13-4:
What is meant by for the SNOMED and Common Ontology hierarchies to be reconciled (i.e.,
coherent)?
a) Every corresponding concept has the corresponding parents and children
b) One is a sub graph of the other
c) *For every concept in one for which there is a corresponding concept in the other, no
ancestor of the original concept corresponds only to non-ancestors of the corresponding
concept
d) For every concept in one for which there is a corresponding concept in the other and for
which the ancestors correspond, the children also correspond
Correct answer: c) For every concept in one for which there is a corresponding concept in the
other, no ancestor of the original concept corresponds only to non-ancestors of the
corresponding concept.
Explanation: a) is too narrow; b) is sufficient but not necessary; d) is too strong.
Page | 159
S13-5:
What is the difference in the information contained in the "Content Model" and Common
Ontology of ICD-11?
a) The Common Ontology contains information that is generally true but to which there may
be exceptions The content model contains only information that is necessarily and always
true.
b) *The Common Ontology contains only information that is necessary and always true;
the content model contains information that is generally true but to which there may be
exceptions.
c) The Content Model is structured around signs and symptoms and the Common Ontology
around anatomy.
d) The Content Model is crowd-sourced while the Common Ontology is built by experts.
Correct answer: b) The Common Ontology contains only information that is necessary and
always true; the content model contains information that is generally true but to which there
may be exceptions.
Explanation: a) is backwards; c) might appear true at a glance but is not fundamental or always
true. d) may or may not occur in future. At the moment both are built largely by experts. c) Is
the fundamental difference between the necessarily true statements in the ontology and the
generally true statements in the Content Model. Most statements about signs and symptoms
are only generally true; this is not universal.
S13-6:
What is the primary reason that ICD, up to version 10, used a mono-hierarchy and SNOMED
uses a polyhierarchy?
a) Because early developers preferred mono-hierarchies. SNOMED was developed later.
b) *Because statistical requirements for ICD required that at each level the cases sum to
100% whereas in SNOMED the hierarchies were determined by logical inference that
could infer one concept to be child of several other concepts.
c) Because ICD was written on paper and SNOMED developed for computers.
d) Because SNOMED was concerned that it be easy to find, for example, all forms of heart
disease, whereas ICD was concerned to maintain traditional distinctions, e.g. between
congenital and degenerative heart diseases.
Correct answer: b) Because statistical requirements for ICD required that at each level the
cases sum to 100% whereas in SNOMED the hierarchies were determined by logical inference
that could infer one concept to be child of several other concepts.
Page | 160
Explanation: b) is clearly the fundamental and best answer although there are elements of truth
in the other three. The practice of a single hierarchy in ICD arguably began because it was
developed on paper but persists because of the insistence of those using international statistics
that everything always add up to 100% and aversion to "double counting" patients who have
more than one disease or whose disease might be classified in more than one way.
Polyhierarchies are intrinsic to any system developed, as is SNOMED, using a description
logic/OWL.
A Road Map for a National Health Information Technology Safety Center
Douglas Johnston; Andrew Gettinger; Kathy Kenyon; Stephanie Rizk; Colene Byrne; Linda
Dimitropoulos
S20-1-1:
Public and private sector organizations are involved in a range of efforts to ensure and improve
the safety of activities across industries such as transportation, agriculture, energy,
manufacturing, drug development, health care delivery and others. Historically, these efforts
have used a variety of organizational models: from governmental regulatory agencies to private
sector oversight bodies.
What does the Roadmap propose as the organizational model for a national Health IT Safety
Collaborative/Collaboratory?
a)
b)
c)
d)
A federal regulatory agency with rule-making and enforcement powers

A private, non-profit patient safety or health care accreditation organization
*A public-private partnership comprised of health IT safety stakeholders
An international standards development organization
Correct answer: c) A public-private partnership comprised of health IT safety stakeholders.

Explanation: The Roadmap proposes that a national Health IT Safety
Collaborative/Collaboratory be comprised of participants and members from a range of
different stakeholders who would benefit from the Collaboratives activities. These include:
Patients and family caregivers
Individual health care clinicians/providers
Health IT developers/vendors
Health care provider organizations
Health IT professionals
Health IT safety researchers and educators
Safety organizations
Accreditation organizations
Medical liability insurers and health insurers
Organizations that support electronic exchange of health information (HIE) and
interoperability
Government entities with responsibility for patient safety and health IT
Page | 161
Importantly, federal agencies, patient safety organizations, health care accreditors, SDOs, and
others are envisioned to be active members in a proposed national Health IT Safety
Collaborative/Collaboratory. Public-private partnerships have been used successfully by other
health care-related industries, in particular the pharmaceutical industry, to provide access to
information and solutions that benefit all stakeholders.
S20-1-2:
Since 2011, the Office of the National Coordinator for Health Information Technology (ONC) has
supported or led a range of initiatives to understand and respond to increasing evidence of
health IT safety-related events. The most recent of these efforts the Health IT Safety
Collaborative Roadmap defines objectives for a proposed Collaborative/Collaboratory. What
are these objectives? (Select all that apply.)
a)
b)
c)
d)
To regulate health IT vendors, and to investigate health IT-related safety events

*To improve the safety of health IT, and to use health IT to make care safer
To create safety-related criteria for the Meaningful Use incentive program
*To include health IT safety as part of a learning health system, and to improve safety
cultures
Correct Answers: b) To improve the safety of health IT, and to use health IT to make care
safer;
d) To include health IT safety as part of a learning health system, and to improve safety
cultures
Explanation: As described in the Roadmap, the main goals of a proposed Health IT Safety
Collaborative are to improve the safety and safe use of health IT. To achieve these goals, the
Roadmap Task Force recommended the development of a public-private partnership for
convening stakeholders, sharing evidence, and developing and disseminating solutions to
address emerging health IT safety issues. Moreover, through its convening, research, and
dissemination functions, the proposed Collaborative/Collaboratory would foster a learning
health system, and support health care organizations with developing cultures of safety around
health IT through participation in these activities.
To access the final Roadmap, visit: http://www.healthitsafety.org/
While prior reports have recommended the development of an investigatory body related to
health IT safety events, the main federal agencies currently supporting federal health IT
safety initiatives ONC and the Agency for Healthcare Research & Quality do not have the
authority to engage in direct investigation or surveillance related to such events. Moreover,
the Roadmap notes that any proposed Collaborative/Collaboratory will not duplicate the
functions of existing health IT Federal Advisory Committees, whose charge includes advising
CMS and ONC on the development of Meaningful Use and Health IT Certification Criteria.
Page | 162
However, separate from the Roadmap and its related components, ONCs final 2015 Health IT
Certification Rule features safety-enhanced design criteria that aim to improve patient safety
for example by applying enhanced user-centered design principles to health IT, enhancing
patient matching, requiring health IT to be capable of exchanging relevant patient information
(e.g., Unique Device Identifiers), improving the surveillance of certified health IT, making more
information about certified products publicly available and accessible, and other existing
criteria related to quality management systems.
S20-1-3:
In a study of data from the Pennsylvania Patient Safety Authority (See reference 1 below),
hospitals adopting advanced electronic health records (EHRs):
a)
b)
c)
d)
*Experienced a 27% overall reduction in reported patient safety events.

Found a 20% decline in medication errors
Found a 15% decline in procedure-related errors.
Did not show any improvements relative to basic EHRs.
Correct answer: a) Experienced a 27% overall reduction in reported patient safety events.
Explanation: There is good evidence supporting both the impact of EHRs in improving quality
and safety, as well as in raising new safety risks, hazards, and harms. Overall, the study of data
from the Pennsylvania Patient Safety Authority found that advanced EHRs produce safer care.
Specifically, this study also found that advanced EHRs produced a:
30% decline in medication errors not a 20% decline
25% decline in procedure-related errors not a 15% decline
There is also increasing evidence of risks, hazards, and harms associated with the use of EHRs.
Many different organizations have produced studies of health IT safety-related risks and
hazards including Patient Safety Organizations, accreditors, medical liability insurers, and
academic institutions using a range of data sources. A few notable studies include:
The ECRI Institute analyzed 171 health IT-related safety events reported to its PSO. The study
found that just over half of the report event types were computer-related, and just under 50
percent were more focused on the user or the user-computer interface. Common problems
included the wrong input, system interface issues, wrong record retrieval, and software
configuration problems.(Reference 2 below.)
The U.S. Food and Drug Administration (FDAs) MAUDE database was analyzed for health-IT
related reports submitted over a 30-month period, identifying 436 such reports. Almost all
(96%) of the problems reported were computer-related, mostly involving software issues, and
only 4 percent involved the machine-user interface. (Reference 3 below.)
The MEDMARX database of over 1 million reports of medication-related errors was searched by
researchers who identified some 63,000 problems related to computerized order entry (CPOE)
Page | 163
systems over 7 years. The authors identified 21 recurring error types and tested 13 of them on
16 different CPOE systems. None of the tested systems performed well; almost 80 percent of
the potentially dangerous order types could be submitted either easily or with minor
workarounds with no warnings.(Reference 4 below)
To access the Health IT Safety Roadmap, go to: www.healthitsafety.org
1. Hydari M, Telang R, Marella W. Saving Patient Ryan - Can advanced electronic medical
records make patient care safer? Social Science Research Network. Working Paper;
September 2014. Available at:
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2503702.
2. ECRI Institute. ECRI Institute PSO Deep Dive: Health Information Technology.
Pennsylvania; 2012.
3. Magrabi F, Ong M-S, MBiomedE, Runciman W, Coiera E. Patient Safety Problems
Associated with Healthcare Information Technology: an Analysis of Adverse Events
Reported to the US Food and Drug Administration. AMIA Annu Symp Proc.
2011;2011:853-7.
4. Schiff G, Amato M, Equale T, Boehne J, Wright A, Koppel R, et al. Computerized
physician order entry-related medication errors: Analysis of reported errors and
vulnerability testing of current systems. BMJ Quality and Safety. 2015;24(4):264-71.
Page | 164

Analysis and Classification of Patient Safety Reports in Computerized Prescriber Order Entry
(CPOE) Systems and Refinement of a New Taxonomy for Classification of CPOE-Related
Medication Errors
Mary G. Amato; Alejandra Salazar; Thu-Trang Hickman; Arbor J. Quist; Lynn A. Volk; Adam
Wright; Dustin S. McEvoy; Sarah P. Slight; David W. Bates; Gordon Schiff
S20-3-1:
Which of the following is an example in which the Computerized Provider Order Entry (CPOE)
system facilitated (actively contributed to) an error?
a) A nurse practitioner enters an order for a medication which dangerously interacts with
another medication and does not receive a warning or alert about the interaction.
b) *A physician in clinic enters a prescription and enters a new pharmacy requested by
the patient, but when the order is completed, the old pharmacy information autopopulates the pharmacy field and the prescription is electronically sent to the wrong
pharmacy.
c) When medication orders are entered electronically for a patient upon hospital
discharge, one of the pre-admission home medications the patient was taking is omitted
from the orders because the ordering resident overlooked it on the admission list.
d) A resident orders an extended release medication meant to be dosed once per day for
the patient to receive three times a day because they confused the medication with the
immediate release formulation.
Correct answer: b) A physician in clinic enters a prescription and enters a new pharmacy
requested by the patient, but when the order is completed, the old pharmacy information
auto-populates the pharmacy field and the prescription is electronically sent to the wrong
pharmacy.
Explanation: In the correct scenario, the physician entered the correct pharmacy information,
but the CPOE system reverted back to the previous pharmacy. In the other examples, a CPOE
system with better decision support and tools may have prevented the error, but it did not
directly contribute to the error.
Page | 165
S20-3-2:
Which of the following statements is true about what was found in this review of error reports
from several CPOE systems?
a) Report narratives generally followed a standardized format that was consistent across
sites.
b) A taxonomy category for what happened to the patient and what happened when
entering the order in the CPOE was able to be easily determined for each case.
c) *Review of the error reports enabled the iterative development of a taxonomy to
classify CPOE-related medication errors.
d) Wrong patient errors were the most common type of CPOE related error in this study.
Correct answer: c) Review of the error reports enabled the iterative development of a
taxonomy to classify CPOE-related medication errors.
Explanation: A taxonomy was developed iteratively and refined as new error reports were
reviewed. The error reports were highly variable in format and completeness and the reviewers
had difficulty assigning terms for some of the cases. The most common errors were errors in
prescription transmission, dose and duplicate orders.
Schiff GD, Amato MG, Eguale T et al. Computerised physician order entry-related medication
errors: analysis of reported errors and vulnerability testing of current systems. BMJ Qual Saf
2015;24:264-71.
Institute of Medicine (US).Committee on Patient Safety and Health Information Technology.
Health IT and Patient Safety: Building Safer Systems for Better Care. National Academies Press
2012.
Page | 166

Leveraging Health Information Exchange to Create Neighborhood Health Records for Public
Health Agencies
Brian E. Dixon; P. Joseph Gibson; Karen F. Comer; Jian Zou; Jennifer L. Williams; Marc Rosenman
S21-1-1:
Many health systems are either contemplating or investigating how to incorporate social
determinants of health into their electronic health record (EHR) systems. What are social
determinants of health?
a) Societal constructs designed to classify sub-populations such as race, ethnicity, and
sexual orientation.
b) *Conditions in the environments in which people are born, live, work, play, and age
that affect a wide range of health, functioning, and quality-of-life outcomes and risks.
c) Risk factors related to socially-oriented behaviors such as smoking, unprotected sex, or
binge drinking.
d) Environmental factors that influence health outcomes such as air quality, water quality,
and nearness to a food desert.
Correct answer: b) Conditions in the environments in which people are born, live, work, play,
and age that affect a wide range of health, functioning, and quality-of-life outcomes and
risks.
Explanation: Social determinants of health are conditions in the environments in which people
are born, live, learn, work, play, worship, and age that affect a wide range of health,
functioning, and quality-of-life outcomes and risks. Conditions (e.g., social, economic, and
physical) in these various environments and settings (e.g., school, church, workplace, and
neighborhood) have been referred to as place. In addition to the more material attributes of
place, the patterns of social engagement and sense of security and well-being are also
affected by where people live. Resources that enhance quality of life can have a significant
influence on population health outcomes. Examples of these resources include safe and
affordable housing, access to education, public safety, availability of healthy foods, local
emergency/health services, and environments free of life-threatening toxins.
S21-1-2:
Which national academy recommends incorporating social determinants of health into
electronic health record (EHR) systems?
a) Institute for Healthcare Improvement
b) Integrating the Healthcare Enterprise
c) *Institute of Medicine
Page | 167
d) Institute for Environmental Medicine

Correct answer text: c) Institute of Medicine
Explanation: Out of all the choices, only the IOM (Institute of Medicine) is part of the National
Academies of Science, Engineering and Business.
Reference(s) for further study: Bazemore AW, Cottrell EK, Gold R, Hughes LS, Phillips RL, Angier
H, et al. "Community Vital Signs": Incorporating geocoded social determinants into electronic
records to promote patient and population health. J Am Med Inform Assoc. 2015. Epub
2015/07/16.
The SDIDS System for Integrating Global Health Surveillance Data: An Example Application to
Malaria Surveillance in Uganda
Kate Zinszer; Anya Okhmatovskaia; Arash Shaban-Nejad; Lauren N. Carroll; Neil F. Abernethy;
David Buckeridge
S21-2-1:
The manager of Monitoring and Evaluation at a national malaria control program (NMCP) is
interested in comparing the timing and location of interventions deployed by partners in a
district that is currently experiencing an outbreak of malaria cases. Interventions include the
distribution of long-lasting insecticide treated nets (LLINs) to at-risk groups for malaria (e.g.,
children < 5, pregnant women) and indoor residual spraying (IRS), which is insecticide spraying
for mosquitoes in houses. In particular, the manager would like to understand the timing of the
interventions and to investigate if there has been a positive effect of the interventions on the
outbreak, to date. What is the most likely barrier for this analysis?
a) The national surveillance system for malaria does not permit analysis by place and time.
b) The partners of the NMCP are not willing to share the intervention data.
c) *The intervention data and malaria morbidity data are housed by separate systems
and it would be very difficult to share the data in a timely manner and also conduct
the analysis in a timely manner.
d) The effectiveness of LLINs and IRS is highly questionable.
Correct answer: c) The intervention data and malaria morbidity data are housed by separate
systems and it would be very difficult to share the data in a timely manner and also conduct
the analysis in a timely manner.
Page | 168
Explanation: The most likely barrier for this analysis would be data fragmentation. Malaria
control data can be siloed across organizations, institutions, and data sources (e.g., clinical,
laboratory). This fragmentation poses a barrier to analyses that could benefit from using
multiple data sources and also the data must be integrated manually for each analysis,
requiring a considerable amount of effort. Also, data access for data of different sources can be
a lengthy procedure thereby not allowing a timely analysis. National surveillance system usually
permit analysis by place and time and also, partners are often willing (and required) to share
their data to NMCP. Finally, LLINs and IRS have been proven to be effective interventions for
malaria control.
S21-2-2:
A central characteristic of the Scalable Data Integration for Disease Surveillance (SDIDS)
software platform is its ability to scale-up and integrate data from other geographical regions
and for other priority diseases. This means that a wide range of data sources can be mapped
once to SDIDS and then accessed and analyzed repeatedly by a wide range of global health
users. To enable such scalability, SDIDS relies on the use of a software ontology, which provides
a uniform data representation framework. What is the best definition of an ontology, in this
context?
a) An ontology is a convention for naming concepts/types, properties, and
interrelationships for a particular domain of knowledge.
b) The branch of metaphysics dealing with the nature of being.
c) An ontology can be viewed as a graph of information for a particular domain, with
relationships as nodes of the graph and concepts (terms) as links that connect the
relationships.
d) *An ontology is a formal taxonomy of terms augmented with the relationships
between concepts expressed in formal logic.
Correct answer: d) An ontology is a formal taxonomy of terms augmented with the
relationships between concepts expressed in formal logic.
Explanation: Software ontology is a formal, explicit specification of a shared conceptualization.
It is a formal way of defining concepts/types, properties, and interrelationships for a particular
domain of knowledge. An ontology can be viewed as a graph of information for a particular
domain, with terms (concepts) as nodes of the graph and relationships as links that connect the
terms. Compared to controlled vocabularies, which provide standard taxonomies of terms to be
shared by different communities, ontologies also have a well-defined semantics, where
relationships between terms are expressed in formal logic using one of existing ontology
languages, and can be processed by logical reasoning engines. Ontologies are, therefore, both
human- and machine-readable.
Page | 169
Quality Improvement
Kensaku Kawamoto; Marc J. Hadley; Thomas A. Oniki; Julia Skapik
S25-1:
You have been appointed Chief Medical Information Officer for your health system and have
been tasked with formulating your institutions strategy for clinical decision support. How
should you incorporate electronic clinical quality measurement into this overall strategy?
a) Do not include clinical quality measurement clinical decision support and clinical
quality measurement are completely unrelated.
b) Establish two independent, unrelated strategies: one for clinical decision support and
one for clinical quality measurement.
c) *Implement a single coordinated strategy across both clinical decision support and
clinical quality measurement, with harmonized standards and approaches used to the
extent possible.
d) Ensure that the clinical decision support team does not waste time implementing
related quality measures, since clinical decision support interventions always work and
their impact never need to be measured.
Correct answer: c) Implement a single coordinated strategy across both clinical decision
support and clinical quality measurement, with harmonized standards and approaches used
to the extent possible.
Explanation: Clinical decision support and clinical quality measurement are highly related.
Therefore, choices a) and b) are wrong.
Choice c) is correct because a unified approach to clinical decision support and clinical quality
measurement can improve efficiency and help ensure that clinicians obtain consistent
information regarding what they should do (clinical decision support) and how they have done
(clinical quality measurement).
Choice d) is wrong because clinical decision support interventions do not always work, making
evaluation and measurement a critical part of decision support-mediated quality improvement
efforts.
S25-2:
Your colleague just moved from the clinical quality measurement team at your institution to the
clinical decision support team. She noticed how the approaches taken and technologies used
Page | 170
are quite different between the two teams. She heard you went to a discussion on the topic of
decision support and quality measurement standards and asks you why the two approaches are
so different. What should you tell her which would be correct?
a) Clinical decision support and clinical quality measurement have fundamentally different
requirements related to the data model.
b) Clinical decision support and clinical quality measurement have fundamentally different
requirements related to the metadata.
c) Clinical decision support and clinical quality measurement have fundamentally different
requirements related to the expression language.
d) *Clinical decision support and clinical quality measurement standards have historically
been developed independent of one another.
Correct answer: d) Clinical decision support and clinical quality measurement standards have
historically been developed independent of one another.
Explanation: Choice d is correct the divergence of the two families of standards is due to
independent development rather than a fundamental difference in requirements related to the
data model, metadata, or expression logic.
S25-3:
An IT developer in your organization is really impressed with the HL7 FHIR standard and how
easy it is to use. When you mention that you went to a discussion on how people involved in
clinical decision support and clinical quality measurement are developing FHIR profiles, he asks
you why FHIR resources arent just being used without any profiles. What should you tell him
which would be correct?
a) Having a lot of models gives us choice and is therefore good for interoperability.
b) FHIR resource instances cannot communicate data attributes that are not explicitly
included in the resource definition.
c) *FHIR profiles allow for the definition of additional constraints compared to base FHIR
resources (e.g., the use of a LOINC code to identify a laboratory result), thereby
d) Only FHIR profiles can leverage a REST application programming interface.
Correct answer: c) FHIR profiles allow for the definition of additional constraints compared to
base FHIR resources (e.g., the use of a LOINC code to identify a laboratory result), thereby
Page | 171
Explanation: Choice c) is correct one benefit of FHIR profiles is the definition of additional
constraints such as vocabulary bindings to facilitate semantic interoperability.
Choice a) is incorrect because model proliferation without good reason hinders interoperability.
Choice b) is incorrect because FHIR resource instances can have extensions to communicate
data attributes not explicitly included in the resource definition.
Choice d) is incorrect because base FHIR resources can also leverage REST.
S25-4:
A colleague asks you what the core motivations are behind the Clinical Quality Framework
(CQF) initiative. What should you tell her that would be correct?
a) *The CQF initiative is seeking to reduce duplicate effort for clinical decision support
and clinical quality measurement implementation.
b) The CQF initiative is seeking to define a minimum data set for exchanging continuity of
care documents.
c) The CQF initiative is seeking to make it easier to integrate biometric device data into the
d) The CQF initiative is seeking to define a large corpus of clinical decision support rules for
clinical use.
Correct answer: a) The CQF initiative is seeking to reduce duplicate effort for clinical decision
support and clinical quality measurement implementation.
Explanation: Choice a) is correct a core motivation behind CQF is to reduce duplicate effort in
the areas of clinical decision support and clinical quality measurement. The other choices are
not core motivations of the CQF initiative.
S25-5:
A developer in your organization is implementing a clinical decision support execution
framework and asks you if you have head of any clinical expression language standards that
could be leveraged for such a framework. What could you tell her is a clinical expression
language standard developed through the Clinical Quality Framework initiative?
a) The GELLO standard
b) The Arden Syntax standard
c) The FHIR standard
d) *The Clinical Quality Language standard
Page | 172
Correct answer: d) The Clinical Quality Language standard

Explanation: Choice d) is correct the Clinical Quality Language (CQL) standard is the clinical
expression language standard developed through the Clinical Quality Framework initiative.
S25-6:
Your CMIO is tracking developments in the health IT standards space. She asks you how the
Clinical Quality Framework (CQF) initiative relates to the Health eDecisions (HeD) initiative.
What should you tell her that would be correct?
a) *The CQF initiative is taking the clinical decision support standards developed by the
HeD initiative and harmonizing those standards with electronic clinical quality
measurement standards.
b) The CQF initiative is taking the electronic clinical quality measurement standards
developed by the HeD initiative and harmonizing those standards with clinical decision
support standards.
c) The HeD initiative is taking the clinical decision support standards developed by the CQF
standards.
d) The HeD initiative is taking the electronic clinical quality measurement standards
developed by the CQF initiative and harmonizing those standards with clinical decision
support standards.
Correct answer: a) The CQF initiative is taking the clinical decision support standards
developed by the HeD initiative and harmonizing those standards with electronic clinical
quality measurement standards.
Explanation: Choice a) is correct HeD developed and validated clinical decision support
interoperability standards. CQF is harmonizing those standards with standards developed for
electronic clinical quality measurement.
Reference(s) for further study: U.S. Office of the National Coordinator for Health Information
Technology. Clinical Quality Framework initiative. 2013 [cited 17 Sept 2015]. Available from:
http://www.cqframework.info
Page | 173

Health Literacy, Education Levels, and Patient Portal Usage During Hospitalizations
Sharon E. Davis; Chandra Y. Osborn; Sunil Kripalani; Kathryn Goggins; Gretchen P. Jackson
S37-1-1:
hospitalized patients at Vanderbilt University Medical Center, which factor was associated with
portal use during hospitalization?
a) *Educational attainment
b) Health literacy
c) Socioeconomic status
d) Years of employment
Correct answer: a) Educational attainment
Explanation: Probability of portal registration and inpatient use increased with educational
attainment. Health literacy was associated with registration but not inpatient portal use. The
study was not able to explore associations with socioeconomic status, and thus could not draw
conclusions regarding associations between this factor and portal use.
S37-1-2:
hospitalized patients at Vanderbilt University Medical Center, use of which portal function was
associated with educational attainment?
a) Secure messaging with health care providers
b) Managing clinic appointments
c) *Viewing of electronic health information
d) Accessing targeted health education materials
Correct answer: c) Viewing of electronic health information
Page | 174
Explanation: Among admissions of hospitalized patients with portal use, the probability of
viewing electronic health record information through the portal steadily increased with years of
education.

Strategies for Managing Mobile Devices for Use by Hospitalized Inpatients
P.C. Dykes; D.L. Stade; A.K. Dalal; S. Collins; M.Clements; F.Y. Chang; A. Fladger; G. Getty; J.R.
Hanna; R. Kandala; L. Lehmann; K. Leone; A.F. Massaro; E. Mlaver; K. McNally; S.Ravindran; K.
Schnock; D.W. Bates
S37-2-1:
Which of the following strategies is not consistent with a socio-technical approach?
a) End-users with an understanding of tasks and environments are closely involved.
b) *The approach is linear with a stage gate between each phase where requirements
are reviewed and approved before continuing to the next phase.
c) The approach sheds new light on the potential roles of IT applications in health care
practices.
Correct answer: b) The approach is linear with a stage gate between each phase where
requirements are reviewed and approved before continuing to the next phase.
Explanation: The socio-technical approach is iterative (not linear).
S37-2-2:
Which of the following issues can be addressed after deploying mobile devices in hospital
(inpatient) settings?
a) Mobile device storage protocol
b) Mobile device charging strategy
c) Mobile device infection control protocol
d) *Providing individual patients with secure access
Correct answer: d) Providing individual patients with secure access.
Page | 175
Explanation: d) is correct because a-c must be addressed prior to deploying devices but d) must
be done for each individual patient after devices are deployed.
Reference(s) for further study: Berg M. Patient care information systems and health care work:
a sociotechnical approach. International journal of medical informatics. Aug 1999;55(2):87-101.

Content and Usability Evaluation of Patient Oriented Drug-Drug Interaction Websites
Terrence J. Adam; Joe Vang
S37-4-1:
The expert consensus on the level of evidence for the clinical risk for most drug-drug
interactions is?
a) High quality with randomized control trial data to estimate clinical risk for most drug-drug
interactions
b) Has a high degree of consistency no matter which reference source or drug database is
utilized
c) *Is often based on clinical case report data and drug class effects with limited
information on estimated actual risk of adverse events
d) Readily available in a comprehensive form in several public source databases
Correct answer: c) Is often based on clinical case report data and drug class effects with
limited information on estimated actual risk of adverse events
Explanation: The quality of clinical evidence to accurately describe the risk of adverse events
associated with drug-drug interaction exposures is limited due to the complexity of the clinical
problem. Prior studies have also shown variation in the interaction data included among
different available clinical databases.
S37-4-2:
Consumer oriented drug-drug interactions websites are characterized by?
a) Consistent reporting of clinical risk data on potential drug-drug interactions
b) Appropriate grade level content on the textual descriptions of drug-drug interaction risk
c) Comprehensive estimates of multiple drug interactions
d) *Variable use of patient friendly interface features and quantitative drug-drug
interaction risk communication
Correct answer: d) Variable use of patient friendly interface features and quantitative drugdrug interaction risk communication
Page | 176
Explanation: The available consumer facing drug-drug interaction websites have variation in the
risk reporting data provided to patients and typically occurs at too high of a grade level for most
patients when provided in narrative form. The consumer websites lack reporting on multiple
drug interactions, but do include some risk data on binary drug combinations.
Ayvaz, Serkan et al . Toward a complete dataset of drugdrug interaction information from
publicly available sources. Journal of Biomedical Informatics (2015): 55: 206 217
Understanding Challenges and Opportunities in Precision Medicine and Interoperability Using
Informatics Approaches
Jay Geronimo Ronquillo; Chunhua Weng; William T. Lester
S41-1-1:
An informaticist is designing a Precision Medicine reporting module that integrates patient
genomic results into their EHR system. They want to make sure that it is consistent with past
and/or existing certification criteria for EHRs. Which of the following
standards/implementation guides relating to genetic information exchange would be a good
starting point to making the reporting module more likely to be interoperable for Precision
Medicine?
a) HL7 Genetic Test Reporting
b) *HL7 Family Health History
c) Global Alliance for Genomics and Health
d) HL7 Genetic variation
e) Fast Health Interoperability Resources (FHIR)
Correct answer: b) HL7 Family Health History
Explanation: While the other HL7 standards mentioned above (genetic test reporting, genetic
variation) were also developed by the HL7 Clinical Genomics Working Group, Family Health
History remains the only one dealing with genetic information exchange that was actually
included in past and recent proposed Health IT/EHR Certification Criteria. Furthermore, while
genetic testing is currently not used in all clinical encounters, nearly all physicians have a strong
understanding of the clinical relevance of Family History, making it a good starting point for
Precision Medicine and gaining stakeholder buy-in. Both FHIR and the Global Alliance for
Genomics and Health represent APIs capable of exchanging genomic information and will likely
become powerful resources in the future, but existing health IT will require significant
enhancements in order to handle the size, scope, and complexity of genomic data in its current
state.
Page | 177
S41-1-2:
The Precision Medicine Initiative was announced at the beginning of 2015 to transform
healthcare by combining diverse patient data (clinical, genomic, etc) to provide innovative
and personalized insight into disease diagnose, treatment, and even prevention. Which of the
following clinical condition(s) represents the most important immediate priority for the
Precision Medicine Initiative?
a) Diabetes
b) *Cancer
c) Mendelian diseases
d) Diseases affecting underrepresented populations
e) Cardiovascular disease
Correct answer: b) Cancer
Explanation: The initially requested budget for the Precision Medicine Initiative was $215
million for the 2016 fiscal year, with $70 million planned specifically for the National Cancer
Institute to accelerate clinical research focused on the genetic/genomic aspects of cancer.
Furthermore, $130 million was budgeted for the NIH to build a one million patient cohort likely
composed of diverse sub-populations, including patients with rare monogenic disorders,
common conditions such as diabetes and cardiovascular disease, and patients frequently
underrepresented in clinical research.
Reference(s) for further study: Collins FS, Varmus H. A new initiative on precision medicine. N
Engl J Med. 2015 Feb 26;372(9):793-5
Early Experiences with Meaningful Use and Online Portal Implementation among
Providers/Staff and Patients/Caregivers in a Safety Net Healthcare System
Courtney Lyles; Lina Tieu; Dean Schillinger; Neda Ratanawongsa; Urmimala Sarkar
S41-2-1:
a) *50% registered and 5% viewing, transmitting, or downloading information
Correct answer: a) 50% registered and 5% viewing, transmitting, or downloading information
Page | 178
Explanation: The correct choice is a), from the Office of the National Coordinator of Health ITs
website.
S41-2-2:
than English?
a) Portals must be available to patients in the top three languages used within that
healthcare system
c) *There is no specific guidance about portals related to language
Correct answer: c) There is no specific guidance about portals related to language
Explanation: There is no official recommendation from the Office of the National Coordinator of
Health IT on this issue even though it will be a clear barrier for the millions of patients in the US
with limited English proficiency.
https://www.cms.gov/regulations-andguidance/legislation/ehrincentiveprograms/downloads/medicaid-ehr-guide.pdf
Physician Participation in Meaningful Use and Rehospitalization of Medicare Fee-for-Service
Enrollees
Mark A. Unruh; Hye-Young Jung; Joshua R. Vest; Lawrence Casalino; Rainu Kaushal
S41-3-1: If Meaningful Use is associated with lower overall rates of hospitalization among
patients of participating physicians, how would it affect our estimates (of the impact of
physician participation in Meaningful Use on the likelihood of readmission)?
a) It would likely not have any effect on the estimates.
b) It would suggest that specialists are more likely to participate in Meaningful Use
compared to primary care physicians.
c) *It would bias estimates towards the null.
d) It would bias estimates away from the null.
Correct answer: c) It would bias estimates towards the null.
Page | 179
Explanation: If Meaningful Use participation reduced the propensity to hospitalize in the first
place, this would bias our results towards the null since hospitalized patients of meaningful
users would be sicker, on average, than hospitalized patients of non-meaningful users.
S41-3-2:
Why was it important to also examine mortality in a study of physician participation in
Meaningful Use and readmissions?
a) *Differing mortality rates between patients of Meaningful Use participants and nonparticipants may bias estimates.
b) There is no relationship between mortality and readmission, but it is an interesting
outcome to examine.
c) Hospitals with high mortality rates are not allowed to participate in the Meaningful Use
incentive programs.
d) Meaningful Use specifically targeted physicians and hospitals with high mortality rates.
Correct answer: a) Differing mortality rates between patients of Meaningful participants and
non-participants may bias estimates.
Explanation: Patients who do not survive cannot be readmitted. If, for example, there were
higher 30-day mortality rates associated with physician participation in Meaningful Use, then
30-day readmission rates might appear lower for this group since some of the patients who did
not survive would likely have been readmitted had they not died.
Are Meaningful Use Requirements Really Meaningful for Medication Use? Experiences from
the Field and Future Opportunities
Sarah P. Slight; Eta S. Berner; William Galanter; Stanley M. Huff; Bruce L. Lambert; Carole
Lannon; Christoph U. Lehmann; Brian J. McCourt; Michael McNamara; Nir Menachemi; Thomas
Payne; Stephen A. Spooner; Gordon Schiff; Tracy Y. Wang; Ayse Akincigil; Stephen Crystal;
Stephen P. Fortmann; Meredith L. Vandermeer; David W. Bates
S41-4-1:
Which of the following EHR functionalities do Stage 3 Meaningful Use objectives specifically
mention or recommend?
a) EHR functionalities to assist with the prescribing of medications for children
b) *EHR functionalities to help maintain an up-to-date accurate medication list
c) EHR functionalities to obtain patient input into the reconciliation of problems on their
problem list
d) EHR functionalities to code contraindications
Correct answer: b) EHR functionalities to help maintain an up-to-date accurate medication list
Page | 180
Explanation: A Stage 3 recommendation was that EHR systems should provide functionality to
help maintain an up-to-date accurate medication list (SGRP 106).
Answer a) is incorrect because: Few of the Stage 3 requirements were aligned sufficiently with
the functionalities considered critical for the accurate prescribing of medications in children
Answer c) is incorrect because this has been proposed for a future stage.
Answer d) is incorrect because this has been proposed for a future stage.
S41-4-2:
Which one of the following statements about Stage 3 Meaningful Use Recommendations is
correct?
a) *The implementation of Meaningful Use capabilities can sometimes delay other EHR
development projects planned in organizations
b) All organizations should face the same levels of difficulty when implementing
Meaningful Use recommendations
c) The use of patient portals by patients living in rural areas is unlikely to be a challenge in
the future
d) No funding has been provided to prepare future professionals to meet emerging
workforce HIT needs
Correct answer: a) The implementation of Meaningful Use capabilities can sometimes delay
other EHR development projects planned in organizations
Explanation: In order to meet Meaningful Use requirements, work on other EHR development
projects was reportedly deferred at some organizations as clinical analysts, engineers, and
senior programming staff were redirected to work on implementing MU requirements.
Answer b) is incorrect because some organizations may find it easier to achieve MU objectives
than others. For example, well-established integrated delivery systems have organized,
coordinated, and collaborative networks that bring together various health care providers to
deliver coordinated care to a defined patient population.
Answer c) is incorrect because patient engagement is likely to be a challenge going forward
with rural residents, considering unreliable Internet connections, low health literacy, and lack of
resources. Although MU requirements currently set a low percentage of patients who are
expected to use the portals, systems must be scalable if more patients are to benefit, which will
likely entail the use of novel technologies such as mobile devices.
Answer d) is incorrect because awards totaling $10 million were collectively granted to five
domestic institutions to support HIT curriculum development in April 2010. Each of these
Curriculum Development Centers was given responsibility to develop, revise, and share
curriculum components covering a specific set of HIT content areas. The ultimate aim was to
prepare future professionals to meet emerging workforce needs.
Page | 181

Slight SP, Berner ES, Galanter W, Huff S, Lambert BL, Lannon C, Lehmann CU, McCourt BJ,
McNamara M, Menachemi N, Payne TH, Spooner SA, Schiff GD, Wang TY, Akincigil A, Crystal S,
Fortmann SP, Bates DW. Meaningful Use: Experiences From the Field and Future Opportunities.
JMIR Med Inform 2015 | vol. 3 | iss. 3 | e30 | p.1
Six Important Characteristics for Patient Hand-Off Application in Inpatient Hospital Setting
Soleh U. Ayubi; Alexandra Pelletier
S42-1-1:
What is the most important reason paper checklist is not appropriate tool for hand-off
collaborative patient care?
a) Lack or limited of integration with electronic medical records (EMRs).
b) Security and privacy aspects.
c) May not be supported by local institutions.
d) *Static, redundant, unstandardized, and unshared medium.
Correct answer: d) Static, redundant, unstandardized, and unshared medium.
Explanation:
A collaborative work requires a tool that facilitates recursive processes where two or more
users are able to work together on a shared task. The tool should facilitate a simple update on
the task and share this update to all the users in real time. A paper checklist doesnt meet these
requirements.
S42-1-2:
A few mobile checklist applications for patient care are available on application markets (Apple
App Store and Google Play). What is the strongest reason their adoptions are low?
a) Lack or limited of live-tracking components.
b) *Lack or limited of integration with electronic medical records (EMRs).
c) Security and privacy aspects.
d) Redundant and unstandardized medium.
Correct answer: b) Lack or limited of integration with electronic medical records (EMRs).
Explanation:
Physicians have to and have been using electronic medical records (EMRs) to record patient
diagnosis and cares so that when they are about to use another new system related to the
patient care, theyre expecting that this new system will seamlessly be integrated with the
EMRs.
Page | 182

A Review and Analysis of Rounding and Handoff Document Content in Inpatient Resident
Physician Teams
Elliot G. Arsoniadis; Rohini Khatri; Jenna Marquard; Courtney Moors; Michael Kim; Genevieve B.
Melton
S42-2-1:
Of the following statements, which best encompasses the current status of standards related to
handoff document architecture?
a) Handoff document standards should conform to HL7 Clinical Document Architecture
Continuity of Care Document standard.
b) *While no official standards for handoff documentation currently exist, the literature
does have some "common themes" that are broadly accepted. In addition, standards
from other clinical documents, such as HL7 Clinical Document Architecture Continuity of
Care Document, provide many overlapping standards that can be utilized by the handoff
document.
c) Standards for the Handoff document are not possible, given the many variations
present, even within a single institution's experience.
d) Standards should be dictated by functionality currently existing within proprietary
Electronic Health Record systems.
Correct answer: b) While no official standards for handoff documentation currently exist, the
literature does have some "common themes" that are broadly accepted. In addition,
standards from other clinical documents, such as HL7 Clinical Document Architecture
Continuity of Care Document, provide many overlapping standards that can be utilized by the
handoff document.
Explanation:
No official handoff document standard exists. However, common themes within the literature
have emerged in the past decade. In addition, the HL7 Clinical Document Architecture
Continuity of Care Document standard has emerged as a potential standard with which to build
off of for handoff document standards. While variation in what an ideal handoff document
would constitute varies between clinicians, this should not hinder efforts at working towards a
handoff document standard.
Page | 183
S42-2-2:
While the data needs of physician may vary by specialty and level of training, what common
themes have emerged from this initial study for constructing an ideal handoff document?
a) Highly granular data related to all organ systems should be included.
b) Only a broad summary of hospitalization is required by most in-training physicians.
c) Medication lists are highly important to enact care by on-call physicians and must be
included on all documents used for handoff/rounding purposes.
d) *Pithy, concise patient summaries are valuable pieces of information. Time/Date
information for data, whether manually entered or automatically imported, is
important for ensuring trustworthiness of data by end users.
Correct answer: d. Pithy, concise patient summaries are valuable pieces of information.
Time/Date information for data, whether manually entered or automatically imported, is
important for ensuring trustworthiness of data by end users.
Explanation: Handoff documents are most valued for providing succinct, accurate summaries of
a patient's current and anticipated issues. While the level of data beyond this is currently under
debate, it is clear from the literature and from our semi-structured interviews that data quality
is valued more that data quantity. In fact, high quantities of data may contribute to "noise" and
detract from the utility of the handoff tool. Data that has an associated date/time is most
valued and standards for handoff documents should acknowledge this point
S42: Papers/Podium Presentations - Care Team Communication
Improving Care Team Communication: Early Experience at Implementing a Patient-centered
Microblog
Anuj K. Dalal; Jeffrey L. Schnipper; Anthony F. Massaro; Kelly McNally; Patricia C. Dykes; David
W. Bates
S42-3-1:
Key attributes of a microblog used to manage a patient-specific communication in a clinical
setting include all of the following EXCEPT:
a) Availability on web-based and mobile platforms
b) Identification of all members of the patients care team
c) Transparency and persistence of the electronic dialog
d) Asynchronous and real-time notifications when new content is posted
e) *Limited to text posts alone
Correct answer: e) Limited to text posts alone
Page | 184
Explanation: A microblog is a type of web service that allows the subscribers to broadcast
short messages, pictures, or other types of media to other subscribers. Subscribers can read
messages and view other media content posted, request updates to be delivered in real time or
asynchronous as an instant message to their desktop or text message sent to a mobile device.
In the clinical setting, the patients care team members are the subscribers. Identified care
team members can post messages to update other care team members about the patients
plan of care, thus keeping everyone on-the-same page. Item E is correct response because
posted content can include all types of content (e.g., images, audio, etc.), not just text.
Vojtech Huser; Alexa McCray; Neil R. Smalheiser; Asba Tasneem; Chunhua Weng
S48-1:
Which of the following statements about ClinicalTrials.gov is correct?
a) We cannot identify the geographic distribution of open trials.
b) We cannot identify the common characteristics of target populations for a specific
disease domain, such as Type 2 diabetes.
c) We cannot identify the common patterns in clinical trial participant selection across
studies or across disease domains.
d) *We cannot discover novel disease biomarkers using data from ClinicalTrials.gov.
Correct answer: d) We cannot discover novel disease biomarkers using data from
ClinicalTrials.gov.
Explanation:
Text mining on ClinicalTrials.gov cannot lead to biomarker discovery but can identify research
design patterns and study distributions.
S48-2:
ClinicalTrials.gov is a U.S. Government database. As such, which of the following statements
about the data contained in ClinicalTrials.gov is true?
a) All ClinicalTrials.gov data can be freely downloaded, used, and distributed with no
restrictions.
b) Data deposited by U.S. entities in ClinicalTrials.gov can be freely downloaded, used, and
distributed, but there is a charge for downloading data deposited by international
entities.
c) *ClinicalTrials.gov data can be downloaded, used, and distributed with certain
restrictions.
d) Commercial entities may not download, use, and distribute ClinicalTrials.gov data.
Page | 185
Correct answer: c) ClinicalTrials.gov data can be downloaded, used, and distributed with
certain restrictions.
Explanation: While all ClinicalTrials.gov data can be downloaded, used, and distributed, there
are important Terms and Conditions for such use. These include appropriate attribution,
including the date the data were processed, any modifications made to the data, and,
importantly, maintaining the currency of the data.
There is no charge for using the data in ClinicalTrials.gov, but all use is subject to the Terms and
Conditions. Commercial entities may use ClinicalTrials.gov data as long as they adhere to the
Terms and Conditions.
S48-3:
ClinicalTrials.gov has now been in existence for over 15 years. Which of the following
statements is true about the genesis of ClinicalTrials.gov?
a) The Food and Drug Administration (FDA) and the National Institutes of Health (NIH),
sister agencies within the US Department of Health and Human Services, recognized
that there was a need to better track clinical trials data and determined that the best
way to do that would be to create a national database of clinical trials information.
b) In response to the requirement by the International Committee of Medical Journal
Editors (ICMJE) that investigators register their trials in a publicly available database as a
condition of publication, the National Library of Medicine created ClinicalTrials.gov.
c) In response to a statement by the World Health Organization (WHO) that all clinical
trials should be registered, the US Department of Health and Human Services, as a
multilateral partner of WHO, directed the National Library of Medicine to create
ClinicalTrials.gov.
d) *In response to pressure from patient advocacy groups and others, Congress passed a
law to create ClinicalTrials.gov.
Correct answer: d) In response to pressure from patient advocacy groups and others,
Congress passed a law to create ClinicalTrials.gov.
Explanation: Patient groups had long advocated for one-stop shopping for clinical trials
information. They were interested in having access to all trials that were recruiting patients,
regardless of who sponsored them, or where they were being conducted. Congress saw the
importance of making clinical trials information broadly and publicly available. Section 113 of
the FDA Modernization Act (FDAMA) of 1997 directed the U.S. Department of Health and
Human Services to create a database of clinical trials information.
Page | 186
While it is true that the FDA and NIH worked closely on the development of ClinicalTrials.gov,
their collaboration was not the impetus for creating the database.
While it is true that both ICMJE and WHO made statements about clinical trial registration,
these statements were made after ClinicalTrials.gov already existed.
S48-4:
Full trial registration transparency means that all trials are registered. Full results transparency
means that all trials publish their results either by PubMed indexed article or summary results
deposition into a registry. Which of the following statements best fits the reality:
a) Approximately: 40 % of interventional trials are registered; 10% of observational trials
are registered; and 30% of results are published.
b) *It is impossible to know with certainty about unregistered trials because many
unregistered trials never publish. Similarly it is difficult to know proportion of trials
with published results accurately.
c) AllTrials.net offers the best view of trials and it indicates that 55% of all trials are
registered and 35% of trials publish their results somehow.
d) Even though many countries do not have a legal mandate to register trials, the ICMJE
requirement to register any trial (including observational trials) is increasing the trial
transparency and it has increased from 10% to 42% since ICMJE introduction. Majority
of journals follow the ICMJE policy on trial registration.
Correct answer: b) It is impossible to know with certainty about unregistered trials because
many unregistered trials never publish. Similarly it is difficult to know proportion of trials
with published results accurately.
Explanation: The best statement is selected by eliminating the wrong answers.
The numbers presented at the lecture reject option a). AllTrials.net does not allow quantitative
analysis of any trial database. Journals compete and part of that competition is making
submission requirement simpler for authors. In addition, ICMJE policy has not the same effect
as legal mandate.
Page | 187
S48-5:
What best describes how publications are linked to clinical trials:
a) Via a search engine (eg, Google Scholar), publications can be linked to trials because the
article will reference the trial name and abbreviation.
b) National Library of Medicine extracts trial IDs from article full text and creates a
structured link.
c) *National Library of Medicine extracts trial IDs from two out of 10+ WHO primary
registries from article abstracts and creates a structured link for those two registries.
d) Publishers collect a structured data entry during manuscript submission for the related
clinical trial and submit this to PubMed in XML format (following the JATS schema
format).
Correct answer: c) National Library of Medicine extracts trial IDs from two out of 10+ WHO
primary registries from article abstracts and creates a structured link for those two registries.
Explanation: The link relies on ICMJE requirement to put trial registration into the article
abstract. NLM does not use full text to look for that link.
S48-6:
Which format should be selected for downloading All Study and Results fields from
ClinicalTrials.gov?
a) *XML
b) SQL
c) Plain Text
d) CSV
Correct answer: a) XML
Explanation: Plain Text and CSV formats are available for only 22 available fields.
ClinicalTrials.gov does not offer SQL format for download. XML is the only available format in
which all study and results fields can be downloaded.
Page | 188

Conversational Agents for Automated Inpatient and Outpatient Health Counseling
Timothy Bickmore
S54 -1-1:
What patient populations respond especially well to conversational agents?
a) Children and young adults
b) *Patients with low health, reading, or computer literacy
c) Patients who love technology and gadgets
d) Computer gamers
Correct answer: b) Patients with low health, reading, or computer literacy
Explanation: Patients who are unfamiliar or uncomfortable with technology respond well to
media that presents as a face-to-face conversation.
S54-1-2:
Why should conversational agents simulate social chat and other relational behavior?
a) To entertain patients
b) To comfort extroverted patients who like to engage in lengthy conversation
c) To acclimate patients to the technology
d) *To build therapeutic alliance to increase intervention adherence and retention
Correct answer: d) To build therapeutic alliance to increase intervention adherence and
retention.
Explanation: Establishing therapeutic alliance through simulated social behavior can increase
retention in automated longitudinal health interventions.
S54-1-3:
What does simulated hand gesture add to a conversational agent's ability to communicate
health information?
a) Conveys the agent's personality
b) Maintains the patient's attention on the intervention content
c) *Conveys additional semantic information and regulates the conversation
d) Conveys the agent's gender and race
Page | 189
Correct answer: c) Conveys additional semantic information and regulates the conversation
Explanation: Hand gestures, and other nonverbal conversational behavior, are used by people
to communicate information and regulate the flow of the conversation.
Bickmore T, Pfeifer L, Byron D, Forsythe S, Henault L, Jack B, et al. Usability of Conversational
Agents by Patients with Inadequate Health Literacy: Evidence from Two Clinical Trials. Journal
of Health Communication 2010;15(Suppl 2):197-210.
Bickmore T, Gruber A, Picard R. Establishing the computer-patient working alliance in
automated health behavior change interventions. Patient Educ Couns. 2005;59(1):21-30.
OHDSI: An Open-Source Platform for Observational Data Analytics and Collaborative
Research
Jon Duke; Frank DeFalco; Chris Knoll; Vojtech Huser; Richard D. Boyce; Patrick B. Ryan
S54-2-1:
Which of the following is NOT a goal of the OHDSI initiative?
a) facilitate clinical characterization of electronic datasets
b) generate evidence based on population level estimation
c) *provide clinical decision support
d) develop methodologies for patient level prediction
Correct answer: c) provide clinical decision support
S54-2-2:
What are the requirements to submit an initial proposal to the OHDSI network?
a) Funding source and protocol
b) Protocol and distributable code
c) Funding, protocol, and distributable code
d) Community forum post and preliminary protocol
Correct answer: d) Community forum post and preliminary protocol
Page | 190
Tuesday, Nov. 17, 2015 Answers & Explanations

The Best of Imaging Informatics Research 2015
Charles E. Kahn; Bradley Erickson
S57-1:
In "Rethinking Radiology Informatics," Kohli et al. argue that:
a) *Radiologists' role in health care IT should focus on managing imaging technology and
selecting appropriate IT products.
b) Radiologists will be expected to provide and manage information about the entire
imaging process.
c) Radiology should focus on clinical issues, and leave IT decisions to executive leadership.
d) There is no need to incorporate decision support or clinical recommendations into
radiology reports.
Correct answer: a) Radiologists' role in health care IT should focus on managing imaging
technology and selecting appropriate IT products.
Explanation: The authors argue that radiologists will provide information about the entire
imaging process, from time of examination ordering using shared decision making via point-ofcare (POC) clinical decision support, to POC decision support for protocoling and examination
interpretation, through optimized results communication. Kohli and colleagues state that
radiologists should become involved in the entire health care enterprise, including Information
Technology. (http://www.ajronline.org/doi/abs/10.2214/AJR.14.13840)
S57-2:
A study of radiology residents' use of structured reporting for chest radiographs found that:
a) *Structured reports were more complete and more effective than traditional,
unstructured reports.
b) The use of structured reports was detrimental to resident education.
c) The specific implementation of reporting software has no effect on the accuracy and
completeness of radiology reports.
d) Imaging procedures such as chest radiography have too complex a vocabulary to permit
structured reporting.
Correct answer: A. Structured reports were more complete and more effective than
traditional, unstructured reports.
Explanation:
Page | 191
Marcovici and Taylor (http://www.ncbi.nlm.nih.gov/pubmed/25415704) found structured

reports created by radiology residents to be more complete and effective than unstructured
reports. They identified potential benefits for radiology training, and pointed to references
suggesting that the ease of use of the reporting system can affect the completeness and quality
of reports.
S57-3:
The report by Hodge et al. on the ConnectomeDB project states that:
a) ConnectomeDB includes magnetic resonance imaging (MRI) data, but does not yet
include magnetoencephalograpy (MEG) or associated behavioral data.
b) Imaging and associated data are made available through bulk file transfer protocol (FTP)
download of the entire dataset.
c) *The system is based on an extensible, open-source platform for managing and
sharing imaging and related data.
d) Users must perform validation of the imaging data, as the database only stores "raw"
information without any validation steps.
Correct answer: c) The system is based on an extensible, open-source platform for managing
and sharing imaging and related data.
Explanation:
The database currently includes structural, task, resting state and diffusion MRI data, resting
state and task MEG data, and associated behavioral data. To facilitate high speed downloads,
ConnectomeDB uses a robust commercial UDP-based data transfer technology that allows
transfer rates much faster than those achievable by TCP-based technologies such as FTP and
HTTP. All imaging data undergo an extensive automated validation process, and some entail
manual QC processes. (http://www.ncbi.nlm.nih.gov/pubmed/25934470)
S57-4:
Deep Learning is being applied in medical imaging applications for many reasons. Which is
the LEAST significant factor in this growth:
a) Deep Learning neural networks take advantage of newer computing hardware (GPUs)
b) *Lack of large labeled image data sets for training
c) Deep learning on images avoids the requirement for direct feature computation
d) Deep learning can find specific objects in complex photographs, and perhaps in medical
images
Correct answer: b) Lack of large labeled image data sets for training
Explanation:
Page | 192
One area of research in deep learning is the use of unlabeled data to improve classification
results compared to labeled data alone.[Zhu Y, Zhang S, Liu W, Metaxas DN. Scalable
histopathological image analysis via active learning. Med Image Comput Comput Assist
Interv. 2014;17(Pt 3):369-76.]. The traditional approach of computing features from images
that are expected to be important and then feeding them to machine learning algorithms is
starting to be replaced by deep learning methods where this potential bias is avoided, and
appears to be very competitive with traditional methods. [Qiao H, Li Y, Li F, Xuanyang, Wu W.
Biologically Inspired Model for Visual Cognition Achieving Unsupervised Episodic and Semantic
Feature Learning. IEEE Trans Cybern. 2015 Sep 18. [Epub ahead of print] Recent examples of
this have been able to identify objects in complex images, leading to hope that such techniques
may be useful for medical imaging as well. Deep Learning has become feasible because of the
ability to leverage the computing power in GPUs, with several examples now being widely
available.
S57-5:
Significant advances for the use of Quantitative Imaging include all but:
a) Creation of large, curated, and annotated data sets with clinical data and outcomes
b) Careful evaluation of the reproducibility of quantitative imaging methods across
multiple centers for several modalities
c) Creation of a Network of investigators to study the appropriate use of QI methods in
specific diseases
d) *Rapid adoption of quantitative imaging methods in most areas of radiological
practice
Correct answer: d) Rapid adoption of quantitative imaging methods in most areas of
radiological practice
Explanation:
Recent activities to promote QI include the development of curated imaging-focused data sets
[http://www.cancerimagingarchive.net/], careful study of the reproducibility of quantitative
imaging methods [Yankeelov TE, Abramson RG, Quarles CC, Quantitative multimodality imaging
in cancer research and therapy. Nat Rev Clin Oncol. 2014 Nov;11(11):670-80. doi:
10.1038/nrclinonc.2014.134. Epub 2014 Aug 12.], and demonstrated value of QI in specific
diseases[A.B. Rosenkrantz, M. Mendiratta-Lala, B.J. Bartholmai, et al. Clinical utility of
quantitative imaging Acad Radiol, 22 (2015), pp. 3349]. However, while there are some
examples, it is not correct to say that at present most areas of radiological practice utilize QI
methods.
Page | 193
Brian E. Dixon; Jamie Pina; Janise Richards; Hadi Kharrazi; Anne M. Turner
S58-1:
Informatics has struggled to amass a robust set of evidence on the effectiveness of electronic
health record systems to make demonstrable improvements in patient outcomes. The same is
true for public health informatics where governmental public health agencies have struggled to
evaluate the effectiveness of information systems. Major barriers preventing high quality
studies include the costs associated with evaluation as well as the complexity involved with
robust study designs such as large, randomized controlled trials (RCTs). Given the need for
more evidence but limited funding for evaluation, which of the following study designs best
balances robustness with complexity and costs making it the most palatable to public health
leaders or health care system administrators?
a) 5-year RCT linking the implementation of health information exchange on duplicate
imaging studies across a large accountable care organization.
b) 2-year before-after study with no control group to evaluate the implementation of an
electronic prescribing application in a series of public health clinics.
c) *3-year controlled before-after study to evaluate the introduction of preventative
reminders in safety net primary care clinics using late adopters as control sites for
earlier implementations.
d) A survey of provider satisfaction with updated computerized provider order entry
modules recently deployed across a large, integrated health system.
Correct answer: c) 3-year controlled before-after study to evaluate the introduction of
preventative reminders in safety net primary care clinics using late adopters as control sites
for earlier implementations.
S58-2:
Significant implementation activity in the United States has been focused on systems that meet
the requirements of the Centers for Medicaid and Medicare Services (CMS) Meaningful Use
program. In the realm of public health informatics, this has meant a focus on systems that
connect public health agencies with health care providers. Which of the following clinical
information systems is an example of an implementation that would establish infrastructure to
move data to and/or from public health agencies?
a) Electronic laboratory reporting of clinical labs from the lab information systems to the
b) Master patient index to resolve and assign unique identifiers to the entire population of
patients covered by an accountable care organizations.
c) *Emergency department information system interface to report chief complaints and
patient demographic information for all individuals registered for care.
Page | 194
d) Clinical decision support system to alert primary care physicians when a sub-population
of patients with diabetes report significant changes in weight.
Correct answer: c) Emergency department information system interface to report chief
complaints and patient demographic information for all individuals registered for care.
S58-3:
Imagine a scenario in which nearly all of the ambulatory and inpatient facilities within a
community routinely share data as part of a health information exchange (HIE). The HIE, in turn,
reports a subset of data routinely to the local public health department, which monitors the
data for changes in disease trends (e.g., active surveillance). While reviewing routine
surveillance reports, an epidemiologist at the health department notices a spike in
gastrointestinal symptoms that correlates with an increased positivity rate in stool cultures
from the lab for Shigella. The epi quickly pushes an electronic alert via the HIE out to providers,
which stimulates increased ordering of stool cultures to monitor a suspected outbreak of
Shigella. Within a week, the epidemiologist not only confirms the outbreak but also identifies
that a subset of providers are treating the outbreak with antibiotics. Since Shigella is often
resistant to antibiotics, the epi distributes treatment guidelines from the Centers for Disease
Control and Prevention (CDC) using the HIE. Antibiotic rates decline and disease levels dissipate.
This example in which the health system was able to gather and analyze routine electronic
information to rapidly inform population health as well as clinical practice could be said to meet
the ideals of which vision espoused by the Institute of Medicine?
a) The Rapid Learning Network
b) Patient Centered Outcomes Research Network
c) Electronic Learning Network for Response to Disease Outbreaks
d) *The Learning Health System
Correct answer: d) The Learning Health System
S58-4:
The major theme the Global Health Informatics (GHI) publications in 2013 and 2014 was:
a) Inclusion of artificial intelligence initiatives to determine trends in global disease
outbreaks
b) *Telemedicine applications used to deliver healthcare in limited resource locations
c) Geographic information systems that assist environmental health researchers depict
global health issues
d) The deployment of cloud computing to assist low and middle income countries (LMIC) in
developing decision support systems
Correct answer: b) Telemedicine applications used to deliver healthcare in limited resource
locations
Page | 195
Explanation: The theme of telemedicine was included in the most identified GHI articles in
2013 (n=27) and 2014 (n=71).
S58-5:
Changing the search strategy within PubMed between the 2013 and 2014 literature reviews led
to greater:
a) *Sensitivity to the search that produced a larger pool of articles
b) Specificity to the search that produced a smaller pool of articles
c) Focus on LMICs (Low-to-Middle Income Countries) in the search that produced a smaller
pool of articles
d) Focus on themes in the search that produced a larger pool of articles
Correct answer: a) Sensitivity to the search that produced a larger pool of articles
Explanation: By including the individual LMIC (Low-to-Middle Income Country) names, along
with other designations for LMICs and the informatics-related keywords, the pool of articles in
2014 for review was larger.
S58-6:
In examining the methodologies used in the 2013 and 2014 articles, we found most studies
were:
a) Random control trials
b) Cross-sectional descriptive studies
c) Longitudinal step-wise control trials
d) *Systematic review studies
Correct answer: d) Systematic review studies
Explanation: Unfortunately, funding and the maturity level of information systems in LMICs are
two barriers to more rigorous studies such as RCT and others. Researchers in GHI are examining
the literature for few publications that have data to develop the rigor in the data to provide a
better depiction of the state of the art for the implementation of health information systems in
LMICs.
Suggested Reference:
Dixon BE, Pina J, Kharrazi H, Gharghabi F, Richards J. Whats Past Is Prologue: A Scoping Review
of Recent Public Health and Global Health Informatics Literature. Online J Public Health Inform.
2015; 7(2):e216. DOI: 10.5210/ojphi.v6i3.5931
Page | 196
S59: Interactive Panel - What Could Go Wrong?

Migrating from One EHR to Another
Richard Schreiber; Ross Koppel; Catherine K. Craven; John D. McGreevey
S59-1:
What is the current approximate percentage of physicians who have installed second, third, or
more EMRs?
a) 10%
b) 20%
c) 30%
d) *40%
Correct answer: d) 40%
Explanation: Most data on implementations, especially second or subsequent ones, comes from
survey data, not prospective studies.
This question was asked in this reference:
http://imaging.ubmmedica.com/all/editorial/physicianspractice/pdfs/2015-Tech-SurveyResults.pdf
S59-2:
What is considered the most complex task involved in changing from one EMR product to
another?
a) Creating new interfaces
b) *Data conversion
c) Upgrading hardware for required functions
d) Choosing the vendor in the first place
Correct answer: b) Data conversion
Explanation: Data conversion is expensive, time-consuming, fraught with data integrity
problems, and a great deal of the data is never really needed. All of the others are complex and
expensive, but not to the same extent.
Page | 197
S59-3:
When switching from one EHR to another EHR, a key step is:
a) getting buy-in from most of the pharmacists
b) getting buy-in from most of the physicians
c) getting buy-in from most of the nurses
d) *explaining the decision's processes and rationale
Correct answer: d) explaining the decision's processes and rationale
Explanation: There are so many countervailing forces in a medical facility, it's usually not
possible to reach buy-in from all about one EHR vendor -- and probably best to acknowledge
that. However, it's best to fully explain the logic, the process, and the players in the process.
S59-4:
If one converts from one instance EHR of vendor X to another of the same vendor, what are the
concerns about compatibility compared to switching vendors?
a) If one stays within vendors, one can be confident that the data will map reasonably well
to the new system, compared to switching vendors
b) If one shifts between vendors, one can be almost certain that the data will not map well,
compared to staying with the same vendors
c) *Staying with the same vendor or not does not predict the certainty of data
compatibility
d) Compatibility of the data are primarily determined by the implementation process, even
if one stays with the same vendor
Correct answer: c) Staying with the same vendor or not does not predict the certainty of data
compatibility
Explanation: New versions may reflect new software acquisitions by the vendor or new data
base systems. There are known cases where new versions required extensive conversion of the
databases and data standards.
Page | 198
S59-5:
Which of the following sequences represents the most appropriate approach to migrating from
one EHR to another?
a) Define current and future state approach to new EHR; Train on new EHR; optimization;
Align and standardize hospital/health system workflows and processes
b) *Align and standardize hospital/health system workflows and processes; Define
current and future state approach to new EHR; Train on new EHR; optimization.
c) Define current and future state approach to new EHR; Align and standardize
d) Define current and future state approach to new EHR; Align and standardize
Correct answer: b) Align and standardize hospital/health system workflows and processes;
Define current and future state approach to new EHR; Train on new EHR; optimization.
Explanation: It is not always been apparent that existing processes necessarily relate to the
EHR. For example, a variety of lab draw practices vary by unit and hospital. In some units only
phlebotomy draws; in others, only nursing. Most of the hospitals in one system have limited or
no phlebotomy; some use patient care techs to draw labs. In some situations (oncology patients
with ports or other standing intravenous access), phlebotomy can draw some kinds of labs such
as peripheral blood cultures but nurses must draw labs from the line.
All of this variation has made it challenging to build a next generation EHR system that is
standardized and coherent in terms of how labs are ordered, how orders are transmitted, how
appropriate lab or nursing personnel are notified, etc. Only during the discovery phase did we
realize the implications that this seemingly separate issue of blood draws would have on our
ability to develop a new EHR.
S59-6:
Of the following, which of the aging-related sociotechnical factors will impact future migration
to new electronic health record systems (EHRs) at Critical Access Hospitals?
1. Retiring CEOs
2. Retiring Directors of Nursing
3. New nurses who have not implemented EHRs before
4. The age-related health status of IT Directors
Page | 199
a) 1
b) 4
c) *1, 2, and 3
Correct answer: c) 1, 2, and 3
Explanation: Many CEOs and Directors of Nursing are either retiring imminently or in the next 510 years, so they will not be on hand to contribute the knowledge they acquired as key critical
care hospital (CAH) leaders and EHR team members, which they gained in the first wave of EHR
implementation at CAHs as a result of Meaningful Use. In addition, although the incoming
recently graduated nurses at CAHs will have learned to use EHRs during nursing school, and
they will have used them on the job from the start of their careers, they will not have
participated on the CAH EHRs team before.
S59-7
CAHs typically had to pay at least what percent of their Total Cash-on-hand for their EHR
systems?
a) *100%
b) 80%
c) 30%
d) 25%
Correct answer: a) 100%
Explanation: In many cases, it cost more than a CAHs cash-on-hand to pay for an EHR system
although vendors figured this out and started allowing them to pay in installments, with the
final installment due after they were reimbursed following successful attestation to Meaningful
Use Stage 1.
Jessica D. Tenenbaum; Greg Biggers; Bradley Malin; Lucila Ohno-Machado; Leslie Wolf
S60-1:
What is the difference between de-identified and anonymous datasets?
a) De-identified datasets cannot be re-identified, but anonymous datasets can
b) *Anonymous datasets cannot be re-identified, but de-identified datasets can
c) Anonymous datasets may include dates and zip codes, but de-identified datasets may
not
d) De-identified data sets may include dates and zip codes, but anonymous datasets may
not
Page | 200
Correct answer: b) Anonymous datasets cannot be re-identified, but de-identified datasets

can
Explanation:
De-identified datasets have identifiers removed or obscured, but the linkage may be
maintained in a number of different ways subject to IRB approval, e.g. a mapping key kept by
an honest broker. Either anonymous or de-identified datasets may contain dates or zip codes
only if the granularity of the information provided could not reasonably be used to identify
individuals, as determined and documented by an expert in the relevant statistical methods.
S60-2:
Which of the following statements is true under the "safe harbor" method for de-identification?
a) De-identified data sets may contain encounter dates, but not date of birth
b) De-identified data sets may not contain any part of dates or zip codes
c) *De-identified data sets may contain the first 3 digits of a zip code if the
corresponding geographic unit contains more than 20,000 people
d) De-identified data sets may contain zip codes if an expert determines and documents
that the level of granularity implied could not result in identification of an individual.
Correct answer: c) De-identified data sets may contain the first 3 digits of a zip code if the
corresponding geographic unit contains more than 20,000 people
Explanation: (a) is not correct because no distinction is made between different types of dates.
(b) is incorrect because the first 3 digits of the zip code may be allowed, as described by (c).
Item (d) could have been interpreted as true had the question not specified the safe harbor
method. Expert determination is valid but different from the safe harbor approach of removing
identifiers. See http://www.hhs.gov/ocr/privacy/hipaa/understanding/coveredentities/Deidentification/guidance.html
S60-3:
Under the proposed changed to the Common Rule, what kind of consent is necessary for using
biospecimens for secondary purposes?
a) Specific consent for each secondary investigation
b) Specific consent for different classes of secondary investigations (classes to be defined
by the Department of Health and Human Services)
c) *Broad consent for any purpose
d) No consent as long as no identifiers are included
Page | 201
Correct answer: c) Broad consent for any purpose

Explanation:
The goal is to strike the right balance between lowering obstacles to research while still
protecting participants rights to informed consent. Researchers may not know all future
studies that would wish to use samples when the participant is consented, and it would be
prohibitively expensive to go back and re-consent every participant for every future study.
S60-4:
Who can be members of the Data Access Committee at the NIH Database of Genotypes and
Phenotype?
a) Any U.S. citizen
b) *Only members of the federal government
c) A mixture of research subject advocates and scientists
d) Members of a centralized institutional review board
Correct answer: b) Only members of the federal government
Explanation:
DACs are composed of senior Federal employees with appropriate scientific, bioethics, and
human subjects research expertise. Members may be appointed for a specified term as defined
by the convening Institute. Consultants with specific expertise may be invited to meetings or to
provide written consultation. Consultants may be Federal or non-Federal employees, and are
not voting members of the DAC.
S60-5:
Currently, research on existing biospecimens can be conducted without consent under existing
law. The Office for Human Research Protections considers biospecimens without identifiers or
biospecimens shared without identifiers and a promise not to share the identifies to fall within
the Common Rule because it is not human subjects research. Research with identifiable
biospecimens may fall within an exemption under the Common Rule, and, if not, may be
eligible for a waiver of consent under the Common Rule. Which of the following best explains
the reason for permitting research without consent under these circumstances?
a) It would be too difficult to obtain consent under these circumstances.
b) *The research poses minimal risk to the people whose biospecimens are used.
c) The research will benefit society as a whole.
d) People do not have a right to control what happens to their biospecimens once they
have been collected.
Page | 202
Correct answer: b) The research poses minimal risk to the people whose biospecimens are
used.
Explanation:
Although (a) is some of the rationale for waiving consent (and the regulatory term is
impractical rather than difficult), it does not apply to the other circumstances. Similarly, (c)
is part of the reasons for the exceptions, it is not sufficient itself. Finally, although the law,
generally, has not recognized a right to control biospecimens outside the body, that has not
been part of the regulatory rationale, which also recognizes the need for consent when
biospecimens are collected prospectively.
See Wolf LE, Advancing Research on Stored Biological Materials: Reconciling Law, Ethics, and
Practice, Minnesota Journal of Law, Science and Technology, 11(1): 99-156 (2010).
S60-6:
Re-identification of individuals from so-called de-identified data sets is possible under certain
circumstances. An attacker may want to know whether a person of interest is included in a
particular data set of individuals with a certain condition. Assuming all safe harbor identifiers
(e.g., name, dates, SSN) are removed from a data set, what are possible ways to determine
whether a particular individual of interest is included in a disclosed data set (e.g., registry of
patients with disease X)?
a) Match phenotype information in the data set with external information about the
individual
b) Match rare variants in a genome sequence in the data set with external variant
information about the individual
c) Match demographic and administrative information with external information about the
individual
Explanation:
Although small, the potential for re-identification of individuals in de-identified data exists
and this risk may be potentially quantified for statistics disclosed from this data set (even when
the nature and number of external data sets is unknown). This can be done using methods
developed for differential privacy.
See The Algorithmic Foundations of Differential Privacy by Cynthia Dwork and Aaron Roth.
Foundations and Trends in Theoretical Computer Science. Vol. 9, no. 34, pp. 211407, Aug.
2014.
Page | 203
Dean J. Karavite; Eric D. Shelov; Levon Utidjian; Jeremy Michel; Eli M. Lourie
S61-1:
You are working with clinical stakeholders to design an alert for sepsis, but are unsure of
whether it should be modal (interruptive) or non-modal (non-interruptive). Your clinical team
has done extensive chart review validating the triggering criteria and has gathered data
identifying patients who would and would not be identified by the criteria, along with those
who did and did not develop sepsis.
Which Statistical measure is the most helpful in determining how interruptive EHR-based
Clinical Decision Support should be?
a) Sensitivity
b) Specificity
c) *Positive Predictive Value (i.e. True Positive Rate)
d) Negative Predicative Value (i.e. True Negative Rate)
Correct answer: c) Positive Predictive Value (i.e. True Positive Rate)
Explanation: Several factors can determine the format of a CDS intervention, but of the
statistical measures Positive Predictive Value (True Positives/All Positives) is most useful in
determining how interruptive and prescriptive CDS should be.
S61-2:
You are tasked with improving your physicians rates of prescribing generic medications in your
outpatient clinic. You first decide to educate your clinicians with handouts and lunchtime talks.
After two months, you see that the number of generic prescriptions has not changed, so you
decide to implement a clinical decision support tool into your EMR to help with your project.
After another two months, you see that generic prescriptions have increased 20%.
The process described is an example of:
a) Failure modes and effects analysis (FMEA)
b) Root cause analysis (RCA)
c) Lean methodology
d) *Plan, Do, Study, Act (PDSA) cycles
Correct answer: d) Plan, Do, Study, Act (PDSA) cycles
Page | 204
Explanation:
The scenario describes a process by which a quality improvement project is approached by
implementing a change, measuring its impact, making another change and measuring the
impact. This is best described as a Plan, Do, Study, Act (PDSA) cycle. 5 Whys, FMEA and RCA all
start from a point of failure and work backwards to find solutions. Lean methodology was
created by Toyota and attempts to improve a system by removing activities that are non-valueadded.
S61-3:
A colleague has conveyed to you something must be done to streamline our management
of a common clinical condition. You are presented with sufficient data to determine an
appropriate intervention point and have an idea for an intervention. You wish to gather further
user feedback from the clinical team before going too far with this idea to make sure you are on
the right track. What are the benefits of using a wireframe (i.e. low fidelity) mockup in this
scenario?
a) You can alter the look of the intervention in real time during a stakeholders meeting
b) *You can assess the usability and internal workflow of the intervention
c) You can use test data to confirm that the interaction will work correctly with the EHR
d) You can gather feedback about how the system works in practice
Correct answer: b) You can assess the usability and internal workflow of the intervention
Explanation:
Low fidelity mockups like wireframes allow for users to test how a system works before it is
fully developed.
This allows developers to make changes in response to previously unknown user needs before
the system is actually developed. Wireframe mockups take some time to develop, so they
cannot be changed on the fly at meetings. Projects in this stage would be better served by
sketches. They do not interact with data so cannot be used to perform white box testing with
test data. Wireframes do not function in practice so cannot be used under live conditions to
test a system. Finally, they do not support collection of clinical data and cannot be used to
assess the efficacy of an intervention.
S61-4:
You are working to develop a support intervention to aid clinicians in the efficient management
of a common clinical condition. After gathering requirements, identifying a workflow
intervention point, designing the intervention, testing the functionality of the decision support
Page | 205
it is finally time to implement the decision support into production. One of your team members
shouts, Success, whats next. You remind your team member that implementation is not the
end, but just the beginning. Which of the following is an important reason to continue testing a
decision support intervention after implementation?
a) The clinical knowledge used to underpin the guideline may have changed
b) *To determine if the system performs as expected with real data
c) To identify additional features and future enhancements for the intervention
d) To correct peoples actions if they are using the intervention incorrectly
Correct answer: b) To determine if the system performs as expected with real data
Explanation:
Post-implementation testing is important for determining if there are any unintended
consequences of the intervention. While guideline knowledge may have changed, this is
unlikely to be detected during post- implementation testing and would be better served by
environment scanning. Identifying new features can be done with post-implementation
testing, but this would be better classified as preparing for an upgrade and is not as important
as making sure the intervention is functioning appropriately in its current state. If people are
using the software incorrectly finding a way to resolve this will be important, but correcting
people actions may not be the best approach as it can build opposition toward the intervention.
Finding out why incorrect use is occurring and working with providers to figure out how to
better integrate the intervention into workflow is preferable.
S61- 5:
Your team has developed a clinical decision support application. The team wants to validate the
systems usability, but does not have the time and resources to perform usability testing. Which
of the following methods is a valid low cost alternative to usability testing?
a) Focus group
b) Remote usability testing
c) *Heuristic review
d) User survey
Correct answer: c) Heuristic review
Explanation:
Heuristics are general principles for interaction design. In 1990, Nielsen and Molich
demonstrated that the heuristic review, when performed by three to five reviewers, is an
Page | 206
effective alternative to formal usability testing in identifying usability issues/problems with a

system.
S61- 6:
You were tasked with developing a documentation tool to help improve rates of provider
documentation of patient tobacco use history to meet a Meaningful Use measure. A few weeks
after go-live, you hear from your data analyst that your hospitals rate of documentation of
tobacco use history has actually worsened. Investigating the issue with a small number of
clinicians uncovers that they had no way of easily knowing if the tool actually saved the history
data they had entered and they were sometimes closing charts and losing unsaved data.
Following which of the below Nielsen usability heuristics could have most easily avoided this
issue?
a) Flexibility and efficiency of use
b) Help and documentation
c) Match between system and the real world
d) *Visibility of the system status
Correct answer: d) Visibility of the system status
Explanation:
The documentation tool provided a new way to document tobacco use history, but clinicians
could not to tell if the system had successfully saved this documentation. Following the Nielsen
heuristic focusing on visibility of the system status could have avoided this problem by
keeping users informed about what is going on in the system through timely, appropriate
feedback, such as an indication of whether or not the documentation was saved. The help and
documentation heuristic emphasizes providing a place for users to get answers to their
questions, but it is better if the system can be used without documentation. Flexibility and
efficiency of use focuses on providing shortcuts to more experienced users and would not
have helped this issue. Match between system and the real world means the system uses
words and concepts familiar to the user and follow real world conventions.
Page | 207
Glenn T. Gobbel; Ruth M. Reeves; Wendy Chapman; Dezon Finch; Jennifer H. Garvin
S62-1:
What is the best outcome measure to determine how well your NLP tool captures the true
cases of patients with a disease from a pool of patients?
a) Specificity
b) *Sensitivity
c) Negative predictive value
d) Positive predictive value
Correct answer: b) Sensitivity
Explanation: Sensitivity is equivalent to recall, and it measures how well a tool performs with
respect to finding true positives. Positive predictive value, also known as precision, measures
performance in terms of the fraction of positive cases found by the tool, which may consist of
both true and false positives, that are actually positive.
S62-2:
The inter-annotator agreement scores in a concept recognition task are above 80% for some
concepts, but below 60% for others. Should you:
a) Retrain annotators on a new set of documents
b) *Collect disagreeing annotations and obtain consensus among annotators, revising
guidelines with clarifications from consensus understanding
c) Determine whether the low-agreement concepts are prevalent in the test data; retrain
annotators only if these concepts are prevalent enough to significantly degrade the
performance of the NLP system
d) Remove all inconsistent annotations and leave only those where there is agreement
Correct answer: b) Collect disagreeing annotations and obtain consensus among annotators,
revising guidelines with clarifications from consensus understanding
Explanation: Low inter-annotator agreement on a particular concept generally suggests that the
annotation guidelines are not adequate to clearly define the concept. It may also indicate a lack
of consensus among the annotators as to the concept definition.
S62-3:
Identify a common characteristic of medical text that presents challenges to natural language
processing systems
Page | 208
a) Clinical text often includes medications with multiple trade names

b) A hospital may treat thousands of patients, and each patients record may contain
numerous documents with unstructured free text
c) The information content of clinical text is sparse compared to what is stored in
databases in structured format
d) *Words and phrases in clinical text are often ambiguous
Correct answer: d) Words and phrases in clinical text are often ambiguous
Explanation: Medical text often contains acronyms and phrases whose meaning may depend on
the provider writing the text, the clinical specialty, and the medical facility.
Answer a) is incorrect because, although medications do go by multiple trade names, such
names are generally very specific and available via various databases (e.g. RxNorm and NDFRT).
Answer b) is incorrect in that NLP systems are generally created specifically to handle large
volumes of text and reduce the burden of manual review.
Lastly, clinical text is a rich source of information. Commonly, NLP is used to capture the
information in clinical text and convert it to structured form for storage in databases.
S62-4:
An initial step in designing and developing an NLP system for clinical use is:
a) Decide whether to use a rule-based or statistical NLP system
b) Create a concept schema for manual annotation followed by NLP system training and
testing
c) *Define the question to be addressed and the information needed to address the
question
d) Find a source of documents that can be used for system development and training
Correct answer: c) Define the question to be addressed and the information needed to
address the question
Explanation: While finding a document source and creating a schema are important steps in
NLP development, potential users should first explicitly define the question being addressed.
Doing this will assist in identifying the information needed to address the question, the
documents containing that information, and the NLP methods most useful for extracting that
information.
Page | 209
S62-5:
Information Retrieval (IR) is a statistical technique that can be used to select relevant notes for
use in NLP. What is required before IR can be used to select the notes?
a) A term list must be developed representing the presence of the topic in the notes
b) *A sample of the documents must be labeled (positive/negative) for the target
concept
c) Notes must be first narrowed by clinic type and provider type
d) A sample of the notes must be stratified by patient demographics to insure an unbiased
selection
Correct answer: b) A sample of the documents must be labeled (positive/negative) for the
target concept
Explanation:
Information retrieval (IR) distinguishes a positive from a negative document based on
established statistical differences between the two document types. Therefore, before using
IR, some examples of positive and negative documents must be provided to estimate the
statistical properties associated with each document label. The alternative answers to the
question describe elements that often occur when generating a document corpus for
investigation, but none are required for information retrieval.
Stetler CB, Damschroder LJ, Helfrich CD, Hagedorn HJ. A guide for applying a revised version of
the PARIHS framework for implementation. Implementation Science 2011 Dec; 6:99-108. 2.
Kitson AL, Rycroft-Malon J, Harvey G, McCormack B, Titchen A. Evaluating the successful
implementation of evidence into practice using the PARiHS framework: theoretical and practical
challenges. Implementation Science 2008 Dec; 3:1-12. 3.
Goldstein MK, Coleman RW, Tu SW, Shankar RD, OConnor MJ, Musen MA, et al. Translating
research into practice: organizational issues in implementing automated decision support for
hypertension in three medical centers. JAMIA 2014 Dept-Oct, 11(5)1:368-376.
Page | 210
Catherine A. Smith; Fabienne C. Bourgeois; Pam Charney; Patricia F. Brennan
S70-1:
What is the principal difference, according to the speaker, between the academic
accommodations process in the kindergarten through grade 12 school system and the process
found in higher education?
a) In higher education settings, the student's parents must contact individual instructors to
request accommodations for the student.
b) *In higher education settings, the student must self-identify and disclose the
necessary information to explain the need for the accommodation.
c) In higher education settings, individual instructors review medical documentation
supplied by the student to decide if a requested accommodation is warranted.
d) In higher education settings, no input is required from healthcare professionals; instead,
informal arrangements are made upon request from the student
Correct answer: b) In higher education settings, the student must self-identify and disclose
the necessary information to explain the need for the accommodation.
Explanation: The fact that a college student, unlike a high school or younger student, must selfidentify as having a disability, and self-negotiate with the higher education system, makes the
ability of that student to so identify and manage their own health information (among other
types of information) a key element of the accommodations process.
S70-2:
What is the **largest** set of information from a particular patient's records that could
theoretically be considered "documentation" for the purposes of academic accommodations in
higher education?
a) *All available data.
b) Office visit notes from the past year.
c) Hospitalizations in the past 7 years.
d) All the pediatrician's records for this patient from the age of 8.
Correct answer: a) All available data.
Explanation: There is no real limit on the type, age, format or content of the documentation
required to support a request for academic accommodations. This is entirely dependent on the
nature of the disability for which the student is seeking these accommodations, as well as
situational factors such as the type of higher education institution, state, etc.
Page | 211
S70-3:
Which of the following health information for an adolescent patient should be available for her
parents to view in a patient portal?
a) Chlamydia test result
b) Smoking history
c) *Doxycycline allergy
d) Toxicology screen results
e) Oral contraceptive prescription
Correct answer: c) Doxycycline allergy
Explanation:
Doxycycline is not a sensitive/confidential medication. In general, allergies should not be
suppressed in the patient portal, given the potential of harm if the parent is unaware of a
serious drug allergy.
Smoking history and toxicology results are generally suppressed, as these relate to a patients
substance use/abuse and are considered confidential.
Oral contraceptive prescriptions are also generally suppressed as they are often indicated in the
management of an adolescents reproductive health and are thus considered confidential.
S70-4:
Which of the following PHR user access models allow both parents and adolescents access to
health information while still preserving patient confidentiality?
a) Full shared access control
b) Adolescent access only
c) Parent access only
d) *Shared differential access
Correct answer: d) Shared differential access
Explanation:
Only the shared differential access option allows access to both the parent and the
adolescent and preserves patient confidentiality. This model allows different information to be
access based on the user type, thus filtering confidential patient information from the parent
view.
Page | 212
The Full shared access provides the same information to both the parent and the patient and
creates a high risk of inadvertent disclosures, as confidential information may not be filtered.
The adolescent access only model gives exclusive access to the adolescent patient; the parent is
not able to access any patient health information.
The parent access only model gives exclusive access to the parent; usually until the patient
becomes and adolescent, at which point all access is terminated
S70-5:
Which of the following is a major roadblock to receiving appropriate healthcare services for
adolescents who have chronic illness?
a) The HIPAA privacy rule
b) Technical inability to exchange information
c) Lack of clinician awareness regarding existence of roadblocks
d) *Local policies regarding adolescent privacy rights
Correct answer: d) Local policies regarding adolescent privacy rights
Explanation:
The HIPAA privacy rule and currently available technology are not intended to act as roadblocks
to healthcare for adolescents who have chronic illness. The majority of clinicians providing care
for adolescents with chronic illness are very aware of the roadblocks they face. In most cases,
confusing and contradictory state and local rules and regulations surrounding adolescent
privacy rights act as major roadblocks for adolescents who have chronic illness.
Jonathan M. Teich; Hamish S. Fraser; Eric Perakslis; Shefali Oza; Darius Jazayeri
S71-1:
The most significant special user-interface needs for epidemic disaster information systems
such as those in the Ebola crisis are generated by alterations in which sense, other than touch:
a) *Vision
b) Smell
c) Hearing
d) Taste
Correct answer: a) Vision
Page | 213
Explanation: It is well known that the personal protective equipment (PPE) worn by staff in the
red zone, including multiple layers and multiple gloves, affect manual dexterity as well as
personal comfort, requiring large buttons and efficient operation so staff can do their work
while minimizing discomfort and potential exposure.
Staff also wear goggles and complete face covering, and the treatment areas are not always
well lit. The combination of goggles, perspiration, and dim lighting mean that it is difficult to see
text on the computer screen at small fonts that would be usable without the PPE. As well as
testing and adjusting the size of touch controls, it is necessary to adjust the font size of text
meant for display in the red zone.
S71-2:
Treatment centers for infectious epidemics such as Ebola are divided into units by
a) Patient residential location
b) Patient arrival date / length of stay
c) *Culture/serology results
d) Patient severity
Correct answer: c) Culture/serology results
Explanation: Because preventing contagion is paramount in an infectious epidemic, the Ebola
treatment centers were divided into units for those patients requiring treatment based on their
Ebola results: one unit for patients who are Ebola negative (these patients may have other
diseases such as malaria), one for patients whose Ebola status is not known, one for patients
with confirmed Ebola. Both of the last two units require protective equipment for staff. Patient
severity can be used to separate intensive care patients from those with reduced needs, but
this separation is more for work efficiency and is not a requirement for preventing contagion.
Patient location might be a separating point in an epidemic where different serotypes were
readily identified from different residential areas.
S71-3:
Which of the following information entry and management tasks can be most easily handled in
the green zone where working conditions are unencumbered by protective personal
equipment?
a) *Lab test review
b) Patient assessment and vitals capture
c) Ordering
d) Medication administration
Page | 214
Correct answer: a) Lab test review

Explanation: When rounding on patients in the red zone, staff quickly capture the patients
symptoms and vital signs, order new changes in medications and IV fluids, and administer those
medications and fluids. Patient assessment and medication administration must be done at the
bedside, and it is advantageous for ordering to happen there too, so that the treatment
changes can happen immediately. The staff may also review lab test results at the bedside, but
(1) lab results are generated in the lab and not directly by the patient, and (2) the results can
come in anytime, not just during rounds. Before each round, the clinical staff can review the
patients prior assessments and lab results to make a preliminary plan.
S71-4:
With regard to connectivity solutions for tablets in the red zone, which of these objectives is
better achieved with a continuous wireless/wifi solution compared to an intermittent docking
solution:
a) Reduced hardware requirement
b) Tolerance of intermittent main (AC) power availability
c) *Instant two-way access to green zone personnel
d) Ability to work without Internet connectivity
Correct answer: c) Instant two-way access to green zone personnel
Explanation: The wifi technology is always connected, making certain urgent tasks and
communications faster and more seamless. Examples of these are requests from the pharmacy,
access to older information and certain types of decision support. Hardware needs are about
the same in both solutions: the wifi requires a wifi server, the docked solution requires docking
stations. The docked model is probably better if power supply to the connecting technology is
very unreliable, because the data can still be collected locally in the docked model; the wifi
model needs to be continuously connected to function. Both solutions work without Internet
connectivity, as the connections are all internal to the treatment center.
S71-5:
Which of these user-interface controls favors greater speed of entry for entering orders and
parameters, such as entering an IV fluid order?
a) *Multiple push-buttons
b) Dropdown menus
c) Number sliders
d) Keypad entry
Page | 215
Correct answer: a) Multiple push-buttons

Explanation: Using dedicated buttons with assigned labels tends to provide the fastest
interaction, as only one tap is needed to record the intended action. Dropdown menus take up
less space on the screen, but usually require two, three or more actions to record an action.
Number sliders can take several steps to use, and using them with low-dexterity protective
equipment can be problematic. The same is true for straight keypad entry; Text entry should be
avoided when possible, not only because of dexterity and usability issues in these settings, but
also because the data recorded is often non-structured.
S71-6:
Treatment protocols in fast-moving disasters can change rapidly. Among the best CDS methods
for ensuring continued best practice are:
a) Alerts
b) Infobuttons
c) *Order sets
d) Analytics
Correct answer: c) Order sets
Explanation: Order sets are often the best for conveying changes in practice, because:
they provide support before the decision is actually made, compared to an alert which
only fires after the wrong decision has been made;
they support an important and required task (ordering) so they are very likely to be used
frequently; and,
the order set with the new protocol can simply replace the order set from the previous
protocol, so the clinician who is used to using the older order set will be presented with
the new information automatically.
Infobuttons are a valuable form of CDS and can offer the most comprehensive information
about the change; however, the user must first recognize that there is valuable new
information and choose to open the infobutton. An infobutton can be included in an order set
to make this deeper information available. Analytics are vital after the fact to measure the
processes, outcomes, and impacts of changes.
Page | 216

Understanding the acceptance factors of an Hospital Information System: evidence from a
French University Hospital
Roxana Ologeanu Taddei; David Morquin; Hugues Domingo
S76-1-1:
A survey conducted in a French Hospital shows a low score of perceived utility and ease-of-use
of the Hospital Information System (HIS) for the clinical staff, which report several difficulties. A
main difficulty related to the HIS implementation and use are:
a) Understanding of the objectives
b) Computers cost
c) Information Systems Directors involvement
d) *Insufficient compatibility of the HIS with the clinical workflow
Correct answer: d) Insufficient compatibility of the HIS with the clinical workflow.
Explanation:
Various authors have already identified insufficient compatibility of the HIS with the clinical
workflow as a difficulty in the implementation and meaningful use of an information
technology. This challenge is a major finding of the survey conducted in a French Hospital, for
various clinical occupations. The insufficient compatibility of the HIS with the clinical workflow
may result in workarounds, which is time consuming and may lead to medical errors.
Understanding the objectives of the implementation is a necessary step but previous studies
show that clinicians agree with the expected benefits of HIS implementation, as reducing
information redundancy. Computers cost is a well-defined cost, which is early considered by
managers. Information Systems Directors involvement is important but not a crucial factor
because the ergonomics itself and the organizational process have to be adapted to support the
everyday clinical workflow, although there are several differences between the occupations, for
example between anesthesiologists and medical secretaries.
S76-1-2:
The result of a survey conducted in a French Hospital is a strong correlation of acceptance
factors of HIS with clinical occupations. This may be explained by dissimilar
a) *uses of the prescription module
b) work involvement
c) wages
d) uses of the billing management module.
Page | 217
Correct answer: a) uses of the prescription module

Explanation: The survey conducted in a French Hospital highlights the crucial issue of
prescription for physicians, surgeons and especially for anesthesiologists. Anesthesiologists
assess the lowest score for ease-of-use because they are the clinical occupation for which
medical error (especially prescription error) may have the most rapid dramatic consequences.
Wages and work involvement are not enough to explain HIS acceptance, especially as surgeons
and anesthesiologists have the highest wages but a low score of ease-of-use. Billing
management is not a core interest for medical occupations.
Saleem JJ, Patterson ES, Militello L, Render ML, Orshansky G, Asch SM. Exploring barriers and
facilitators to the use of computerized clinical reminders. J. Am. Med. Inform. Assoc.
2005;12(4):438-447.
Park SY, Lee SY, Chen Y. The effects of EMR deployment on doctors' work practices: a
qualitative study in the emergency department of a teaching hospital. Int J Med Inf.
2012;81(3):204-217.
Improving EHR Capabilities to Facilitate Stage 3 Meaningful Use Care Coordination Criteria
Dori Cross; Genna R. Cohen; Paige Nong; Anya-Victoria Day; Danielle Vibbert; Ramya
Naraharisetti; Julia Adler-Milstein
S76-2-1:
Which of the following represents a way in which EHRs are currently supporting care
coordination efforts within primary care?
a) Ability to attach ICD codes to problems and medications for easier reconciliation,
information sorting and decision support
b) *Automatic generation of a Summary of Care Record to send to specialists upon
referral, with a compilation of relevant parts of the patient record needed for
subsequent care
c) Integrated document tracking functionality to follow patients through the referral
process and ensure follow-up if gaps in care are identified
d) Enabling features to accommodate and integrate workflow and documentation needs of
ancillary team members, such as care managers
Page | 218
Correct answer: b) Automatic generation of a Summary of Care Record to send to specialists

upon referral, with a compilation of relevant parts of the patient record needed for
subsequent care
Explanation: b) (generation of the SCR) is the only feature listed that is currently available
within most commercial EHRs to support care coordination practices. All other features listed
highlight existing challenges that providers face in providing well-coordinated care, and would
be welcome innovations.
S76-2-2:
One common challenge in todays healthcare environment is managing the volume of
information generated in the provision of patient care. From a vendor perspective, which of
these solutions is the most challenging to implement?
a) Branching logic to determine what content is sent through the summary of care record
b) Document tagging to classify/sort information for easier retrieval
c) *Automatic incorporation of referral reports and other incoming information in to
relevant section(s) of the patient record
d) Drag and drop functionality to more easily maintain accurate, up-to-date problem and
medication lists
Correct answer: c) Automatic incorporation of referral reports and other incoming
information in to relevant section(s) of the patient record
Explanation: All other answers provided present vendor-specific solutions that can be
implemented to improve information management in the absence of EHR interoperability. For
information to come in to a practice and automatically sort in to a patients record, systems
have to be able to read the incoming information (i.e. no flat PDF files) and parse out discrete
pieces of information to update different sections of the patient record. This level of
interoperability is beyond the scope of a single EHR vendor, and requires significant policy-level
engagement.
Page | 219

Variability in Electronic Health Record Usage and Perceptions among Specialty vs. Primary
Care Physicians
Travis K. Redd; Julie W. Doberne; Daniel Lattin; Thomas R. Yackel; Carl O. Eriksson; Vishnu
Mohan; Jeffrey A. Gold; Joan S. Ash; Michael F. Chiang
S76-3-1:
Electronic health record (EHR) systems and the patient chart contained within them serve as an
important source of information used by physicians to inform clinical decision-making. Many of
these systems are designed for use across all medical specialties. However, recent evidence
suggests there are differences in the way physicians from different clinical disciplines
incorporate this chart review into clinical workflow, which may not always be accounted for.
When used as a source of patient information to inform clinical decision-making, in what ways
is EHR utilization different between physicians in primary care and specialty medical fields?
a) Primary care physicians spend more time reviewing the chart
b) Specialty physicians spend more time reviewing the chart
c) *Timing of this initial chart review (i.e. before, during, or after the patient encounter)
d) Perceived importance of the chart as a source of patient information
Correct answer: c) Timing of this initial chart review (i.e. before, during, or after the patient
encounter)
Explanation:
There are numerous differences in the way physicians from different clinical disciplines utilize
the EHR.
One of these is how these systems are incorporated into clinical workflow. Specifically, the
primary care physicians tend to review the chart prior to a clinical encounter, whereas specialty
physicians are more likely to review the chart during or after the patient interview. This has
implications for EHR design elements that may be customizable to suit different practice
environments. See Grinspan ZM, et al. Physician specialty and variations in adoption of
electronic health records. Appl Clin Inform. 2013 May 22;4(2):225-40.
Inferring Clinical Workflow Efficiency via Electronic Medical Record Utilization
You Chen; Wei Xie; Carl A. Gunter; David Liebovitz; Sanjay Mehrotra; He Zhang; Bradley Malin
S76-4-1:
Electronic health record (EHR) systems document the actions of care providers with respect to a
patients record in a temporal manner. This information can be leveraged to construct a
sequence that is representative of the actions invoked by care providers. In order to measure
Page | 220
the strength of an ordered relation from one action to another action, we make the following
assumption:
A sequence is a series of events (actions), where an event has an ordered relation with the two
subsequent events in a sequence, which are weighted by distance. Specifically, the
immediately following event has a stronger relationship than the next nearest event. For
instance, in a sequence e 1 e 2 e 3 e 4 , the event e 1 has an ordered relation with e 2 , which
is stronger than with e 3 , and no relation with e 4 .
Based on the assumption how can we calculate the strength of an ordered relation e 1 e 2 ?
Count(e i e j e k ) is a function which calculates the total number of times, sequence e i e j
e k appearing in all patient sequences.
a) Count(e 1 e 2 )
b) Count(e 1 e 2 ) / Count(e 1 e i ), where e i is the immediately following event of e 1
c) Count(e 1 e 2 ) + Count(e 1 e i e 2 ), where e i is any event following e 1
d) W 1 *Count(e 1 e 2 ) + W 2 *Count(e 1 e i e 2 ), where 0<W 2 < W 1 <1
Correct Answer: d) W 1 *Count(e 1 e 2 ) + W 2 *Count(e 1 e i e 2 ), where 0<W 2 < W 1 <1
Explanation: Answers a) and b) only consider the relations with the nearest neighboring event
only, which neglects the subsequent neighbor. Answer c) does not discount the relation
between events with respect to their distance.
S76-4-2:
A series of two ordered events such as e1 e2 and e3 e4 can appear in patient sequences for
varying reasons and durations. For instance, imagine e1 e2 appears four times, with
durations of {0.1, 0.2, 0.2, and 1} hour, while e3e4 appears five times and always for 0.1 hour.
Assume average and variance in duration are invoked as standard criteria for ordered-relation
categorization. In general, we can categorize ordered-relations into four types:
1.
Stable Efficient (SE): average duration 0.2 hours with variance 0.1 hours;
2.
Stable Inefficient (SI): average duration > 0.2 hours with variance 0.1 hours;
3.
Unstable Efficient (UE): average duration 0.2 hours with variance > 0.1 hours; and
4.
Unstable Inefficient (UI): average duration > 0.2 hours with variance > 0.1 hours
a. e 1 e 2 and e 3 e 4 are SE.
b. e 1 e 2 and e 3 e 4 are SI.
c. e 1 e 2 is SE, while e 3 e 4 is UE.
d. e 1 e 2 is UI and e 3 e 4 is SE
Page | 221
Correct Answer: d) e 1 e 2 is UI and e 3 e 4 is SE

Explanation: The average time duration and corresponding variance of the ordered relation e1
e2 is 0.37 hours and 0.42 hours, respectively. These are within the range for UI. The
average time duration and variance of the ordered relation e3
e4 is 0.1 hours and 0 hour,
respectively. These are within the range for SE.
Zhang H, Mehotra S, Liebovitz D, Gunter CA, Malin B. Mining deviations from patient care
pathways via electronic medical record system audits. ACM Transactions on Management
Information Systems. 2013; 4(4): 17.
You Chen, Wei Xie, Carl Gunter, David Liebovitz, Sanjay Mehrotra, He Zhang, Bradley Malin.
Inferring Clinical Workflow Efficiency via Electronic Medical Record Utilization. AMIA 2015
Annual Symposium.
Farah Magrabi; Dean F. Sittig; Jean Scott; Peter Kilbridge
S80-1:
Your healthcare system wishes to examine evidence about the patient safety risks of HIT. It has
identified 125 HIT-related events that have been voluntarily reported by clinicians to the
organizational incident reporting system over the last 3-months. Which is the most appropriate
way to utilize the data provided by the event reports?
a) To assess the overall safety of HIT
b) *To examine the different types of safety problems that can arise with HIT
c) To measure the frequency of HIT-related adverse events
d) To quantify the magnitude of patient harm
Correct answer: b) To examine the different types of safety problems that can arise with HIT
Explanation: The purpose of analyzing event reports is to identify some of the categories or
error types so that we can develop interventions to reduce them, not to assess the overall
safety of HIT. Many incidents may go undetected, or be detected only after patients have been
harmed. As reports do not represent a systematic sample they cannot be used to examine the
frequency of safety problems. Event reports are one source among an array of information
repositories (e.g. the literature, existing registries for equipment failure and hazards, medical
record review, complaints, and medico-legal investigations) that need to be brought together
Page | 222
to provide a more comprehensive understanding about the nature, causes, consequences, and
outcomes of HIT problems.
S80-2:
In your healthcare system, clinicians report most HIT problems to the IT service desk (or help
desk) which is available 24 hours a day, 7 days a week. Any incidents that are identified by
clinicians as impacting care delivery or patient safety are given the highest priority. Service desk
staff have been asked to escalate all large-scale adverse events for immediate investigation and
response. Which of the following events would be escalated?
a) *All the computer systems in the emergency department have crashed.
b) A physician cannot login to the CPOE system to order a blood test for a patient who is
critically ill.
c) Data was lost and time was wasted when a desktop computer crashed during a
consultation.
d) A child had a full body x-ray. Some of the images went missing in a PACS when they
were digitized.
Correct answer: a) All the computer systems in the emergency department have crashed.
Explanation: Large-scale adverse events are individual HIT events which harm or increase the
risk of harm to numerous patients.
S80-3:
Your healthcare system is deploying a range of tools to support the operation of a new EHR.
Out-of-bandwidth tools refers to:
a) Dangerous applications that consume excessive bandwidth
b) Tools that gradually reduce network capacity
c) Tools that monitor network traffic from outside of the network
d) Standard network configuration tools
Correct answer: c) Tools that monitor network traffic from outside of the network
Explanation: Monitoring of IT system performance can detect hazards associated with
infrastructure (e.g. power supply, data network availability) and specific software systems (e.g.
availability of the EHR, order entry and PACS). In computer networks, out-of-band management
involves the use of a dedicated channel for managing network devices.
Page | 223
S80-4:
Your healthcare system is reviewing its data security practices. As resources to enhance existing
data security practices are limited, you have been asked to identify and prioritize only the most
important threats to the security of patient information stored in the EHR. Provider data
breaches today:
a) Remain comparatively rare events
b) Can be entirely prevented with sophisticated firewalls
c) Are less common than breaches in the financial sector
d) *Should be expected and planned for
Correct answer: d) Should be expected and planned for
Explanation: The risk of data breach is an unintended consequence arising from the
digitalization of the medical records. The number of medical data breaches has increased
dramatically in recent years. As of September 2015, there were 1,326 data breaches reported
to the U.S. Department of Health and Human Services since August 2009, involving more than
143 million patients the most common forms of data breach were thefts, unauthorized access
or disclosure, and data loss. In 2012, a bi-annual survey of 250 U.S. healthcare organizations
found that 27% of respondents had at least one security breach over the past year, compared
to 19% in 2010 and 13% in 2008. Based on these figures data breaches should be expected and
there is a need for strategies to rapidly identify and mitigate their risks.
S80-5:
Once initially implemented, a healthcare system can expect to receive incremental updates for
defect remediation, data updates and new features. The original implementation was planned
prior to adoption of functionality into the workflows, and specific configurations were
established before go-live. As more systems are implemented, the effect of integration
becomes more complicated. In efforts to streamline updates, healthcare systems may adopt
features that are implemented at the enterprise level.
After an upgrade, a defect is discovered in which all care facilities within the healthcare system
are adversely affected in which the system is unavailable or no back-out mechanism is feasible.
Which of the following practices provides a strategy for risk mitigation?
a) Require increased software testing prior to release
b) *Establish deployment approach to allow for monitoring of defects including ability to
respond and pause process
c) Report issues to the IT Service desk
d) Ensure all facilities are able to enact contingency plans such as having paper forms for
processing orders.
Page | 224
Correct answer: b) Establish deployment approach to allow for monitoring of defects

including ability to respond and pause process
Explanation: While testing is a method to identify issues prior to deployment, this cannot
control for varied configurations of HIT. Reporting the issues and enacting contingency
operations only provide means for addressing continuity of operations. However, using a
staged deployment will provide opportunity for identification of issues before a large-scale
event occurs due to software and/or data updates. The staged deployment is a continuous
method that limits the impact while the defect is being corrected.
S80-6:
As healthcare delivery organizations become more dependent on EHRs the risks to patient care
during an extended system-wide downtime increase substantially. Over the next 3 years what is
the likelihood that any large healthcare delivery organization will experience a down time
lasting more than 8 hours?
a) Infinitesimally small. No need to worry.
b) Less than 15%, but you recently upgraded your air conditioning systems so you will be
fine.
c) *About 75%, but with proper planning and preparation the risks to patient safety can
be greatly reduced.
d) Almost certainly one will occur, but there is nothing you can do to reduce your risk so no
need to worry.
Correct answer: c) About 75%, but with proper planning and preparation the risks to patient
safety can be greatly reduced.
Explanation: Unexpected downtimes related to EHRs are fairly common. In recent survey, 50 US
based healthcare institutions reported at least one unplanned downtime (of any length) in the
last 3 years and 70% had at least one unplanned downtime greater than 8 h in the last 3 years.
Three institutions reported that one or more patients were injured as a result of either a
planned or unplanned downtime.
S80-7:
Your healthcare system is using the SAFER guides. The Contingency Planning SAFER Guide
identifies recommended safety practices associated with planned or unplanned EHR
unavailabilityinstances in which clinicians or other end users cannot access all or part of the
EHR. Occasional temporary unavailability of EHRs is inevitable, due to failures of software and
hardware infrastructure, as well as power outages and natural and man-made disasters. Such
Page | 225
unavailability can introduce substantial safety risks to organizations that have not adequately
prepared.
Effective contingency planning addresses the causes and consequences of EHR unavailability,
and involves processes and preparations that can minimize the frequency and impact of such
events, ensuring continuity of care. Which of the following practices does not deal with
contingency planning?
a) A communication strategy that does not rely on the computing infrastructure exists for
down time and recovery periods.
b) Paper forms are available to replace key EHR functions during downtimes.
c) Policies and procedures are in place to ensure accurate patient identification when
preparing for, during, and after downtimes.
d) *The organization has adopted a standard clinical vocabulary to record and report all
normal and abnormal laboratory test results.
Correct answer: d) The organization has adopted a standard clinical vocabulary to record and
report all normal and abnormal laboratory test results.
Explanation: Communication is key during an EHR downtime. The ability to communicate using
methods other than those that rely on the computing infrastructure is key. Paper forms are
necessary to allow clinicians to order new tests and medications and record vital patientrelated information and the care process. Policies are necessary to help employees understand
what is expected of them before, during and after downtimes. While a standard clinical
vocabulary for laboratory results is important, it does NOT help an organization when the
computer system is not working.
Magrabi F, Baker M, Sinha I, Ong MS, Harrison S, Kidd MR, et al. Clinical safety of England's
national programme for IT: A retrospective analysis of all reported safety events 2005 to 2011.
Int J Med Inform.2015;84(3):198-206.
Coiera E, Magrabi F. Information system safety in The Guide to Health Informatics. Boca Raton,
FL, USA: CRC Press, Taylor & Francis Group; 2015. p.195-220.
Kilbridge P. Computer crash--lessons from a system failure. N Engl J Med. 2003;348(10):881-2.
Meeks DW, Smith MW, Taylor L, Sittig DF, Scott JM, Singh H. An analysis of electronic health
record-related patient safety concerns. J Am Med Inform Assoc. 2014;21(6):1053-9.
Sittig DF, Gonzalez D, Singh H. Contingency planning for electronic health record-based care
continuity: a survey of recommended practices. Int J Med Inform. 2014;83(11):797-804.
Sittig DF, Ash JS, Singh H. The SAFER guides: empowering organizations to improve the safety
and effectiveness of electronic health records. Am J Manag Care. 2014;20(5):418-23.
Page | 226
Craig Kuziemsky; Madhu Reddy; Katie A. Siek; Sarah Collins
S81-1:
The predominant challenge facing design of HIT to support collaboration is:
a) *The lack of explicit focus on collaboration in HIT studies
b) Poor understanding of workflow
c) Need to be more patient centered
d) Inability to support teamwork
Correct answer: a) The lack of explicit focus on collaboration in HIT studies.
Explanation: Answers b), c), and d) are all related to a lack of explicit focus on collaboration.
S81-2:
Planned or spontaneous engagements that take place between individuals or teams of
individuals, whether in-person or mediated by technology, where information is exchanged in
some way (either explicitly, i.e., verbally or written, or implicitly, i.e., through shared
understanding of gestures, emotions, etc.), and often occur across different roles (i.e., physician
and nurse) to deliver patient care is defined as:
a) Communication
b) Coordination
c) Cooperation
d) *Collaboration
Correct answer: d) Collaboration
Explanation: Direct definition from the abstract
S81-3:
The technologies, contexts, processes and outcomes that are involved in collaboration is
referred to as:
a) Agenda for studying collaboration
b) *Collaboration space
c) Collaborative competencies
d) Patient-centered care
Page | 227
Correct answer: b) Collaboration space

Explanation: Direct definition
S81-4:
What are the best methods to study collaboration?
a) Qualitative methods
b) Quantitative methods
c) Mixed methods
d) *Depends on the objectives of the study
Correct answer: d) Depends on the objectives of the study
Explanation: All methods can be used to study collaboration but the correct one depends on
the objectives of the study
S81-5:
The context of different collaborative settings is most closely linked to:
a) The collaborative structure
b) *The collaborative behavior
c) The collaborative outcomes
d) The technology needed to support collaboration
Correct answer: b) The collaborative behavior
Explanation: The behavior of a setting is ultimately defined by the context of a setting and
subsequently shapes the other parameters.
Reference(s) for further study: Eikey EV, Reddy MC, Kuziemsky CE. Examining the role of
collaboration in studies of health information technologies in biomedical informatics: A
systematic review of 25 years of research. J Biomed Inform. 2015; 57;263-277.
Page | 228
Future)
Ozlem Uzuner; John Aberdeen; Stephane Meystre; Mehmet Kayaalp
S83-1:
What method of clinical narrative de-identification always results in 100% removal of
identifiers?
a) Rule-based systems that match patterns in narrative text.
b) Machine learning systems that are trained from examples.
c) Manual de-identification performed by human experts.
d) *None of these methods results in 100% removal of identifiers.
Correct answer: d) None of these methods results in 100% removal of identifiers.
Explanation: No clinical narrative de-identification process finds all instances of personal health
identifiers in all cases. All processes can leave behind some number of residual identifiers.
S83-2:
For what reason(s) is the de-identification of clinical text more difficult than general English
text?
a) Clinical text is rich in ambiguous abbreviations
b) The structure of clinical text is often ungrammatical (e.g., telegraphic style)
c) Clinical notes frequently include lists and tables in the text
Explanation: Clinical text frequently contains ambiguous abbreviations, lists, and tables, and it
rarely uses formal grammatical structures.
S83-3:
Which one of these is always necessary for a HIPAA compliant data set to be shared with other
researchers?
a) *Institutional Review Board approval by the institution that owns the data
b) Institutional Review Board approval by the home institution of the recipients of the data
c) Human subjects training by the recipients of the data
d) A commitment from the recipients of the data that they will destroy it properly.
Page | 229
Correct answer: a) Institutional Review Board approval by the institution that owns the data
Explanation:
While institutions vary in their requirements imposed on the recipients of the data, one
component that is always necessary for any HIPAA compliant data distribution is an IRB
approval by the institution that owns the data. The IRB protocol usually details the process by
which the data is rendered HIPAA compliant, how the distribution is going to be carried out,
and any protocols in place for handling PHI leaks that may be discovered. Therefore, it is the
key component for making any data set available outside of its home institution.
The requirements on the recipients of the data is usually detailed in the IRB protocol. But given
the potentially diverse backgrounds of these groups, and the fact that they may come from
anywhere in the world, with varied institutional governance rules for handling sensitive data, an
IRB review by the home institution of these groups, human subjects training, and other
requirements for handling the data may be different or completely absent.
S83-4:
In practice, clinical narrative texts are typically shared for secondary use purposes by what
methods?
a) Manual or automated de-identification, followed by open release
b) Manual de-identification only, followed by open release
c) *Manual or automated de-identification followed by limited release
d) Clinical narrative texts are not shared for secondary use
Correct answer: c) Manual or automated de-identification followed by limited release
Explanation: In practice, releasing clinical narrative data for secondary use involves both
technological (de-identification), and policy (limited release governed by data use agreements)
protections.
S83-5:
Which one of these provides the most convincing argument for an IRB to approve the
distribution of automatically de-identified clinical text data?
a) Automatic systems are at least as accurate, if not more accurate and consistent, than
human de-identifiers.
b) In a task in which even humans do not perform 100% accurately, automatic deidentification systems give very close to human performance and can replace humans.
c) *Automatic de-identification will be accompanied with manual human review to get the
best de-identification possible before data distribution.
d) Automatic de-identification accompanied with appropriate data use agreements that put
limits on the use of the data minimize the risk of PHI leaks.
Page | 230
Correct answer: c) Automatic de-identification will be accompanied with manual human

review to get the best de-identification possible before data distribution.
Explanation: While automatic de-identification systems can give close to human performance
or even perform better in places where humans may be inconsistent, the most convincing
argument to a risk-averse IRB would be to couple automatic de-identification with manual
human review, so as to get the best of both worlds and to minimize the risk of PHI leaks. Any
data use agreements that accompany the distribution of the data usually aim to address the
residual risk, after the risk has been minimized by automatic and manual review.

Adaptation of a Published Risk Model to Point-of-care Clinical Decision Support Tailored to
Local Workflow
Jeffrey L. Sobel; Craig C. Baker; David Levy; Carol Cain
S84-2-1:
In implementing electronic clinical decision support, one design question is whether to re-use
content that has been developed by someone else or build content inside the walls of an
organization. In this session, the organization describes their experience replicating a published
risk model. Why did the organization choose to replicate the model rather than link to an
external site?
a) The organization needed to customize some of the parameters to fit their clinical
practice
b) *It was more reliable and integrated to reproduce the model inside their firewall
c) The published researchers were not willing to share their data model
d) The organization wanted to validate the model themselves
Correct answer: b) It was more reliable and integrated to reproduce the model inside their
firewall
Explanation: The original computer model was available on the Internet, and the organization
agreed with its findings. The need to reproduce it was really driven from desire to integrate the
model with data from the EMR, as well as the risk (later realized) that the model would stop
being provided on the Internet.
Page | 231
S84-2-2:
Many technical and organizational factors contribute to the success of a clinical decision
support initiative. Although all factors are important, successful clinical outcomes have been
demonstrated even without the inclusion of one of the factors below:
a) Endorsement of the model by the organization's subject matter experts
b) Sustained and regular use by frontline physicians
c) *Automatic generation of EMR orders based on results of the model
d) Integration with existing clinical initiatives and improvement infrastructure
Correct answer: c) Automatic generation of EMR orders based on results of the model
Explanation: Ensuring widespread model adoption and use was a complex sociotechnical effort
involving thought leader endorsement and alignment with the other strategic work of the
organization. We would have loved to populate the order set with model data as this would
have sped the workflow, but this wasn't available through the API with the EMR. We were still
able to achieve improvements in clinical outcomes without this factor.
Reference(s) for further study: Sittig DF, Singh H. A new sociotechnical model for studying
health information technology in complex adaptive healthcare systems. Qual Saf Health Care.
2010 Oct;19 Suppl 3:i68-74.
Challenges and Solutions in Optimizing Execution Performance of a Clinical Decision SupportBased Quality Measurement (CDS-QM) Framework
Tyler J. Tippetts; Phillip B. Warner; Polina Kukhareva; David E. Shields; Catherine J. Staes;
Kensaku Kawamoto
S84-3-1:
You are in charge of implementing a point-of-care reminder system for diabetes care, as well as
an electronic clinical quality measurement system for providing feedback to clinicians on their
quality of care for diabetes. Which of the following is a benefit of using the same framework
for implementing both the clinical decision support and electronic clinical quality measurement
in this scenario?
a) A clinical decision support-based quality measurement system will always perform
faster than a quality measurement system implemented in a different way.
b) *There will be semantic consistency between what the clinician sees in terms of pointof-care reminders and what they see as quality feedback reports.
c) The clinical decision support implementation team is almost always also responsible for
enterprise clinical quality measurement at any given health care organization.
Page | 232
d) Implementing the two functionalities using the same technical framework would follow
the typical approach taken at most health care organizations.
Correct answer: b) There will be semantic consistency between what the clinician sees in
terms of point-of-care reminders and what they see as quality feedback reports.
Explanation: There are two main benefits to implementing clinical decision support-based
electronic clinical quality measurement. One of them is semantic consistency (choice b),
because the underlying data analysis uses the same clinical logic. The other benefit is
implementation efficiency, which is not one of the choices provided.
Choice a) is incorrect because there is no guarantee that execution speed will be faster.
Choices c) and d) are incorrect because clinical decision support and quality measurement are
implemented in separate ways by separate groups in a typical organization.
S84-3-2:
You have been using a clinical decision support-based quality measurement framework without
problem for inpatient quality measurement. When scaling to outpatient quality measurement,
however, you are having difficulty achieving the required throughput for large populations in a
timely manner. Which of the following is an approach you should consider for increasing the
throughput?
a) Query for patient data one patient at a time as opposed to for large batches of patients
at one time.
b) Make sure all processes are executed through a single machine/server.
c) *Run processes in parallel rather than in sequence.
d) Use a standard data model rather than an ad hoc data model.
Correct answer: c) Run processes in parallel rather than in sequence.
Explanation: Choice c) is correct parallelization is a key aspect of performance optimization.
Choice a) is incorrect because database queries are less efficient on a per-patient basis when
data are pulled for a single patient rather than for many patients.
Choice b) is incorrect because using multiple machines/servers can improve performance.
Choice d) is incorrect because the use of standards may be beneficial for other reasons, but not
necessarily for performance optimization.
Page | 233

Kukhareva P, Kawamoto K, Shields DE, et al. (2014). Clinical Decision Support-based Quality
Measurement (CDS-QM) framework: prototype implementation, evaluation, and future
directions. AMIA Annu Symp Proc, 825-34.
Using a Clinical Knowledge Base to Assess Comorbidity Interrelatedness Among Patients with
Multiple Chronic Conditions
Donna M. Zulman; Susana B. Martins; Yan Liu; Samson Tu; Brian B. Hoffman; Steven M. Asch;
Mary K. Goldstein
S84-4-1:
Of the following, which measure is specifically designed to capture the treatment complexity
that may arise when making clinical decisions for a patient with one or more comorbid
conditions?
a) Charlson Comorbdity Index
b) *Comorbidity Interrelatedness Score
c) Elixhauser Comorbidity Measure
d) Diagnosis-Related Group (DRG)
Correct answer: b) Comorbidity Interrelatedness Score
Explanation:
Charlson and Elixhauser are well-known and widely-used measures of comorbidity designed to
predict outcomes such as length of stay, hospital charges, and mortality, but were not designed
to assess the complexity of clinical decision-making. Diagnosis-Related Group (DRG) is a system
to classify hospital cases into one of a set of defined groups expected to use a similar level of
hospital resources. The Comorbidity Interrelatedness Score is designed to capture the extent to
which comorbid conditions interact with the primary condition to affect the degree of
complexity in clinical decision-making.
S84-4-2:
Which of the following types of clinical decision support system is most likely adaptable to
generate comorbidity interrelatedness scores:
a) Alerts for critical lab values
b) Clinical reminders
c) Hospital admission order sets
d) *Knowledge-based systems with an inference engine
Page | 234
Correct answer: d) Knowledge-based systems with an inference engine

Explanation:
Alerts regarding critical lab values are usually based on the lab value reference standards; in
some cases they may vary with certain clinical conditions, but they are not designed to detect
interactions among multiple clinical conditions concurrently. Clinical reminders are generally
rule-based algorithms that may include logic regarding one or more comorbidities but are not
designed to include knowledge about multiple comorbidities and their potentially interacting
treatments. Hospital admission order sets are generally designed to include standard orders
appropriate to a single admission diagnosis. Knowledge-based systems generally encode
extensive clinical knowledge including comorbidity information together with treatment
information and so are more likely to contain the clinical knowledge necessary to generate
comorbidity interrelatedness scores.
Zulman DM, Asch SM, Martins SB, Kerr EA, Hoffman BB, Goldstein MK. Quality of care for
patients with multiple chronic conditions: The role of comorbidity interrelatedness. J Gen Intern
Med. 2014;29(3):529-37.
Safford MM, Allison JJ, Kiefe CI. Patient complexity: more than comorbidity. The vector model
of complexity. J Gen Intern Med. 2007;22 Suppl 3:382-90.
Goldstein M, Coleman R, Tu S, et al. Translating Research into Practice: Organizational Issues in
Implementing Automated Decision Support for Hypertension in Three Medical Centers. J Am
Med Inform Assoc. 2004;11:368-76.
Tinetti ME, Fried TR, Boyd CM. Designing health care for the most common chronic condition-multimorbidity. JAMA. 2012;307(23):2493-4.
Steinman MA, Lee SJ, Peterson CA, Fung KZ, Goldstein MK. A clinically guided approach for
improving performance measurement for hypertension. Medical care. 2012;50(5):399-405.
Mining Twitter as a First Step toward Assessing the Adequacy of Gender Identification Terms
on Intake Forms
Amanda Hicks; William Hogan; Michael W. Rutherford; Bradley Malin; Mengjun Xie; Christiane
Fellbaum; Zhijun Yin; Daniel Fabbri; Josh Hanna; Jiang Bian
S85-1-1:
A clinic plans to start gathering patient data on gender identity after reading the Institute of
Medicines recommendation highlighting how this information will improve patient care and
lead to better understanding of the health needs and risks of gender minorities. In response to
this recommendation, Fenway Health conducted studies for determining the best way to collect
gender identity data to receive high response rates and accurate data. What format is
recommended for collecting gender-identity information on health intake forms?
Page | 235
a) Ask the patient to identify their gender. Do not provide options such as male and
female, but provide a place for the patient or intake personnel to write-in a response.
b) Ask the patient to identify their gender and provide the following options: male,
female, other please specify, with a write-in option for other please specify
c) *Ask the patient two questions: 1) What sex were you assigned at birth? 2) What is
your current gender identity?
d) Ask the patient to identify their sex and give three options: male, female, other.
Do not include a free text option
Correct answer: c) Ask the patient two questions: 1) What sex were you assigned at birth? 2)
What is your current gender identity?
Explanation:
The two-step format is recommended for intake forms as well as surveys. By distinguishing
between sex assigned at birth and current gender identity, patients with a transgender history
who identify as simply a man or a woman rather than transgender man or transgender
woman are able to indicate their gender identity as well as their transgender history. This
enables more accurate data collection.
Furthermore, cis-gender patients those who are not trans, non-binary, or gender variant
understand this format and are less likely to inadvertently indicate that they are transgender.
The two-step format only specifies the questions and the order to ask them in. It does not
specify which options should be presented to the patient.
a Providing only a write-in option increases the difficulty for structuring the data in a machine
readable format, thereby making it difficult to identify even simple cohorts such as all of the
adult, biological males seen by a particular clinic.
b and c The one question format does not provide accurate information compared to the two
question format. Furthermore, the word other ought to be avoided on surveys and intake
forms since it has negative connotations for many individuals.
S85-1-2:
Fenway Health, UCSF Center of Excellence for Transgender Health, and the Williams Institute
provide sample intake forms for asking about gender identity. Each form provides a different
set of options for specifying current gender identity.
What is a potential problem with these options from the point of view of collecting reliable
data?
a) These forms have too many options leading patients and researchers to be confused
about what the different options mean.
Page | 236
b) *These forms do not adequately capture non-binary gender identities such as

agender.
c) Men/women who are not trans* do not understand the options and frequently mark
the wrong one.
d) Patients frequently skip the questions and do not provide an answer.
Correct answer: b) These forms do not adequately capture non-binary gender identities such
as agender.
Explanation: While genderqueer is a broad term, its meaning may have different
connotations in different contexts, ranging from somewhat binary interpretations (not
exclusively male or female) to decidedly not binary. Either way, not all people who do not
identify with a binary gender identify as genderqueer. A Gender Not Listed Here discusses
specific health risks and experiences of discrimination faced by non-binary people and those
whose gender-identity were not listed in the National Transgender Discrimination Survey. In
brief, the current options do not capture data for certain marginalized and high-risk gender
groups.
Re response a): One of the virtues of these forms is their tractable number of answers. While
we do not know what the ideal and most comprehensive list of gender identities would look
like, we do want to keep the options to a relatively small number while providing an accurate
option to the greatest number of people.
Re responses c) and d): Fenway Healths research on the two-step format showed that c and d
are both false.
Cahill S, Makadon H. Sexual orientation and gender identity data collection in clinical settings
and in electronic health records: A key to ending LGBT health disparities. LGBT Health.
2014;1(1):34-41.
Harrison, Jack, Grant, Jaime, & Herman, Jody L. A gender not listed here: Genderqueers, gender
rebels, and otherwise in the National Transgender Discrimination Survey. LGBTQ Public Policy
Journal at the Harvard Kennedy School. 2012; 2(1).
Harrison, Jack, Grant, Jaime, & Herman, Jody L. A gender not listed here: Genderqueers, gender
rebels, and otherwise in the National Transgender Discrimination Survey. LGBTQ Public Policy
Journal at the Harvard Kennedy School. 2012; 2(1).
Kuper LE, Nussbaum R, Mustanski B. Exploring the diversity of gender and sexual orientation
identities in an online sample of transgender individuals. Journal of Sex Research. 2012;49(23):244-54.
Page | 237

Evaluating Consumer m-Health Services for Promoting Healthy Eating: A Randomized Field
Experiment
Yi-Chin Kato-Lin; Rema Padman; Julie Downs; Vibhanshu Abhishek
S85-2-1:
Which of the following activity can be fully achieved by simply providing a web app, as opposed
to a mobile app?
a) Help patients to monitor their exercise level by automatically tracking their active time
in minutes
b) Help patients to achieve a healthier eating habit by remotely calibrating patients
portion size misperception for each meal in real time
c) Help patients to monitor their sleeping quality at home by automatically tracking their
movements during the course of sleep
d) *Help asthma patients to monitor their status by asking them to record the results of
peak flow testing
Correct answer: d) Help asthma patients to monitor their status by asking them to record the
results of peak flow testing
Explanation:
What makes a mobile app different from a web app is the mobile apps ability to activate a
built-in camera, to control sensors, to create and send multimedia files in few clicks, to
automatically collect data, and to be used without any spatial or time constraint whereas usage
of web apps are constrained by time and space. Option D is selected because manually
recording the results of peak flows, usually measured by another device, can be achieved using
a web app. Recording the peak flows at anytime and anywhere using a mobile app is certainly
more convenient, but not necessary in order to perform the activity described in d).
Option a) can only be achieved using a mobile app, because automatically tracking patients
active time cannot be achieved unless the patients can bring a mobile device with a biometric
sensor when they are exercising. Option b) can only be achieved using a mobile app. To enable
a dietitian to calibrate patients misperceptions, it is essential to provide the dietitian both
the perception and the fact. To do this remotely, the patient has to either send a food
image or a food video for the dietitians evaluation in real-time. This action would require the
use of a mobile device which can trigger the built-in camera and send the image or video
instantly during their mealtimes. Option c) can only be achieved using a mobile app, because
tracking the users movement during the course of sleep cannot be achieved unless the user
can put a device on his/her bedside which can trigger the movement sensor and continuously
collect data.
Page | 238
Reference 1 below provides further studies about why mobile phones are a promising platform
for health interventions and what intervention strategies have been used for various health
conditions.
S85-2-2:
Mobile technologies have the potential to engage patients and change their healthy behaviors,
yet little evidence has been documented. Drawing on theoretical frameworks, the mobile app
study evaluates novel mobile-enabled interventions which upgrade three interventions
commonly used in practice for promoting healthy eating: self-monitoring, professional support,
and peer support. The impact of these interventions were examined via a randomized field
experiment. Suggested by the results of this study for healthy eating, which of the following
may best engage patients in managing their health by using an intervention tool?
a) Providing them a booklet with tips for better management and sheets for selfmonitoring
b) Providing them an online forum that allows them to ask questions and get answers from
healthcare professionals
c) *Remotely connecting them with a healthcare professional who can give them
personalized feedback and suggestions that work specifically for them
d) Providing them an online community that allows them to compare and learn from other
patients like me
Correct answer: c) Remotely connecting them with a healthcare professional who can give
them personalized feedback and suggestions that work specifically for them
Explanation:
The most significant effect found in our study was the effect of our dietitian support on users
engagement in self-monitoring. What makes our dietitian support different from the existing
support provided by healthcare professionals is our ability to make the feedback personalized
and contextualized. By providing feedback on users food images, the users can feel that the
feedback can apply only to them, but not just a general suggestion. In fact, in our survey, our
respondents also indicate that this is what makes them stay engaged in the study. Therefore,
option C is selected as the correct answer.
On the contrary, a booklet with tips for better management described in option a) certainly
cannot provide personalized suggestions. In addition, as mentioned in the abstract, a paperbased diary usually cannot keep users engaged. Therefore, option a) is not selected. An online
forum indeed provides a channel to connect healthcare professionals and consumers. However,
due to its public nature, the messages exchanged cannot be really personalized, thus limiting
the level of engagement. Therefore, option b) is not selected. Option d) is not selected, because
as shown in our study, peer support may not work for everyone. While some people can be
more engaged by interacting with their peers, others might be intimidated by the peers or may
Page | 239
not value their feedback sufficiently. Reference 2 below provides further studies about how to
develop interventions or policies to engage patients or families.
1. Klasnja P, Pratt W. Healthcare in the pocket: Mapping the space of mobile-phone health
interventions. Journal of Biomedical Informatics. 2012;45(1):184-98.
2. Carman KL, Dardess P, Maurer M, et al. Patient And Family Engagement: A Framework
For Understanding The Elements And Developing Interventions And Policies. Health Aff
(Millwood). 2013 February 1, 2013;32(2):223-31.
Public Perspectives of Mobile Phones Effects on Healthcare Quality and Medical Data
Security and Privacy: A 2-Year Nationwide Survey
Joshua E. Richardson; Jessica S. Ancker
S85-3-1:
about mobile health's potential impact on healthcare privacy, security, and quality. What, if
any, year-over-year change was there in consumers' perception about mobile health's impact
on healthcare quality?
a) *Significantly improve
b) No significant change
c) Significantly worsen
d) No findings
Correct answer: a) Significantly improve
S85-3-2:
about mobile health's potential impact on healthcare privacy, security, and quality. Which
answer best describes the primary takeaway from the survey's results?
a) Consumers likely see little benefit from mHealth due to overriding privacy and security
concerns
b) Consumers seem to have little to no understanding of mHealth
c) *Consumers have strong concerns about mHealth privacy and security but increasingly
perceive mHealth as a means for improving healthcare quality
d) Consumer age groups significantly differed in their opinions about mHealth privacy and
security, and its effect on healthcare quality
Page | 240
Correct answer: c) Consumers have strong concerns about mHealth privacy and security but
increasingly perceive mHealth as a means for improving healthcare quality
Patient Engagement in Cancer Survivorship Care through mHealth: A Consumer-centered
Review of Existing Mobile Applications
Yimin Geng; Sahiti Myneni
S85-4-1:
Which of following statements about HIMSS Patient Engagement Framework is appropriate?
a) Patient Engagement Framework is a model that is used to create questionnaire for the
evaluation of patients self-management of chronic diseases.
b) Patient Engagement Framework consists of five distinct phases that begin with low-level
engagement and progress toward high-level engagement.
c) *Patient Engagement Framework consists of five cumulative phases to engagement
that make up the framework, which include inform me, engage me, partner with
me, and support my e-community.
d) One of the fourteen Patient Engagement Framework categories identified, information
and way-finding, can only achieve inform me phase of patient engagement.
Correct answer: c) Patient Engagement Framework consists of five cumulative phases to
engagement that make up the framework, which include inform me, engage me, partner
with me, and support my e-community.
Explanation:
The Patient Engagement Framework is a model created to guide healthcare organizations in
developing and strengthening their patient engagement strategies through the use of eHealth
tools and resources. Patient Engagement Framework consists of five cumulative phases to
engagement that make up the framework include inform me, engage me, partner with
me, and support my e-community. Patient engagement level is embedded as a cumulative
dimension of the framework, i.e. engagement categories in a given phase are specified in
addition to the ones available in previous phase (e.g. Empower me engagement = Empower me
+ Engage me). For the fourteen Patient Engagement Framework categories identified, sample
elements can achieve different patient engagement phases. For example, if a sample element
of information and way-finding only provide physician directory, its patient engagement level
is at inform me phase. If it provides quality and safety reports on providers and healthcare
organizations, its patient engagement level will be at empower me phase.
Page | 241
S85-4-2:
To engage patients in self-management of their own health, mHealth solutions development
can be based on
a) Traditional theories of behavior change solely as long as considering individuals, their
social and environment interactions.
b) Utilization of most advanced web-based and mobile platforms only.
c) *Integration of behavior change strategies using technology development
frameworks.
d) Integration of Engaging consumers in Health with Health Care (ECHC) framework and
Health belief model.
Correct answer: c) Integration of behavior change strategies using technology development
frameworks.
Explanation:
Integrating behavior change theories is vital to engage patients in self-management of health.
However, the validity of traditional theories in digital health era has been questioned. Existing
behavior change theories maybe insufficient to inform mobile intervention development.
Integration of behavior strategies using technology development frameworks that consider the
role of human cognition and socialtechnical factors surrounding users health-related
behaviors can be an approach for effective mHealth intervention development.
Human Factors of Health Information Exchange: Barriers and Facilitators to Use of the VAs
CPRS and a Regional Health Information Exchange
Andre W. Kushniruk; Elizabeth M. Borycki; Helen Monkman; Kenneth Boockvar
S88-1-1:
There is an increasing demand for interconnection between electronic health records and
regional health information exchanges. However, a number of human factors problems have
led to lower than expected adoption. Which of the following is the most likely reason for lack of
use of such interconnected systems from a human factors perspective?
a) *lack of time for users to switch among systems
b) incompatible data formats
c) privacy laws, security breeches and malware attacks
d) file specification errors
Correct answer: a) lack of time to switch among systems
Page | 242
Explanation: Results of our work indicated that negative impact on workflow (i.e. in particular
lack of time for busy physicians to switch between systems) was cited by end users as being the
most significant reason for lack of use of interconnected systems.
S88-1-2:
Usability testing methods for analyzing user interaction with multiple systems in the study of
system integration involves:
a) simulating keystrokes
b) *observing and video recording user interactions
c) applying heuristic design and evaluation guidelines
d) disseminating questionnaires
Correct answer: b) observing and video recording user interactions
Explanation: Usability testing approaches involve observing and typically video recording end
users as they interact with systems under study (as was carried out in the study described in the
presentation).
Gadd CS, Ho YX, Cala CM, Blakemore D, Chen Q, Frisse ME, Johnson KB. User perspectives on
the usability of a regional health information exchange. JAMIA 2011;18:711-716.
Borycki EM, Kushniruk AW, Nohr C, Takeda H, Kuwata S, Carvalho C, Bainbridge M, Kannry J.
Usability methods for ensuring health information technology safety. Yearb Med Inform
2013;8(1)20-27.

Uncovering the Cognitive Demands of EHR Use via Task Analysis
Mark S. Pfaff; Ozgur Eris; Amanda Anganes; Tina Crotty; Jonathan R. Nebeker; Merry Ward
S88-2-1:
Communication between clinicians on a care team is essential for maintaining common ground
about the patients diagnosis and treatment plan. The EHR provides functionality for clinicians
to communicate with one another, but often there are breakdowns in communication that can
potentially interfere with effective care delivery. In this Department of Veterans Affairs
qualitative study, which of the following did clinicians describe as the main communication
challenge they face when trying to maintain a shared understanding of the patients situation?
Page | 243
a) Clinicians described the communication functionality in the EHR as difficult to use

b) Clinicians were uncertain about what kinds of information was or was not required to be
exchanged with other clinicians
c) *Clinicians reported that it was difficult to know whether someone had received or
read a message they had sent
d) Clinicians felt that it was difficult to keep on top of all the messages they received via
the EHR
Correct answer: c) Clinicians reported that it was difficult to know whether someone had
received or read a message they had sent.
Explanation: Maintaining common ground requires not only the sharing of information, but a
mutual understanding between team members that everyone has the information they need to
know. Put another way, I know that you know I know. Not knowing whether a message had
been received and read makes it difficult for one clinician to know whether another has a
shared understanding of a particular issue, which undermines the process of attaining common
ground among the care team Thus, answer c is the best answer to this question. .In our study,
clinicians did not report many difficulties with using the EHRs communication functions. In
addition, clinicians did not report ambiguity about what kinds of information was supposed to
be shared this relies more on clinical best practices rather than EHR functionality. Clinicians
did report often being overwhelmed by the quantity of messages they received from the EHR,
but this is primarily a problem with managing workload rather than achieving common ground.
S88-2-2:
Diagnosing a patient and developing a treatment plan can require clinicians to acquire and
comprehend large volumes of data about the patients history and current condition. Clinicians
also use information that provides framing and context, such as acceptable ranges of laboratory
values, which help the clinician understand the implications of certain pieces of patient data.
Unfortunately, the necessary pieces of data for making clinical decisions are often dispersed in
many different parts of the EHR (ie, separate screens or modules). According to this study,
which of the following is the most significant cognitive implication of fragmentation of data
across different parts of the EHR?
a) It increases the chances for duplication of effort in the EHR (e.g. redundant lab orders)
b) *It requires slow and effortful thinking rather than enabling fast and associative
thinking
c) It particularly affects novices interacting with the EHR
d) It increases the amount of training clinicians require with the EHR
Page | 244
Correct answer: b) It requires slow and effortful thinking rather than enabling fast and
associative thinking
Explanation: In the time-pressured environment of patient care, clinicians aim to make wellinformed and accurate decisions as quickly as possible. Research shows that experienced
clinicians are able to make high-quality decisions based on rapid cognitive processes (referred
to as Type 1 thinking) that involve pattern matching between the patients symptoms and
extensive knowledge of prior cases. However, when fragmentation of information makes it
difficult to acquire and integrate patient information on a single display, clinicians are forced to
piece together information incrementally from multiple sources. This slows the decision-making
process and increases the load on the clinicians working memory, causing a change to more
effortful and deliberative thought processes known as Type 2 thinking. If the clinician cannot
easily find needed information, like a laboratory result, he or she may assume the test was not
performed and will re-order it. However, that is primarily a problem of workflow, not cognition.
Additional training on the EHR may help clinicians know where to look for information, but does
not address the increased cognitive load of comprehending fragmented information that the
EHR could present in a unified display.
Clark HH, Brennan SE. Grounding in communication. In: Resnick LB, Levine J, Teasley SD, editors.
Perspectives on Socially Shared Cognition. Washington, DC: American Psychological Association;
1991. p. 12749.
Croskerry P. Clinical cognition and diagnostic error: Applications of a dual process model of
reasoning. Adv Heal Sci Educ Theory Pract. Springer Netherlands; 2009;14 Suppl (1):2735.
Evaluating the Effects of Cognitive Support on Interpreting ICU Patient Data
Peter V. Killoran; Swaroop Gantela; Sahiti Myneni; Khalid Almoosa; Bela Patel; Thomas
Kannampallil; Vimla L. Patel; Trevor Cohen
S88-3-1:
Which of the following techniques of information organization has the greatest impact on
attention to Systemic Inflammatory Response (SIRS) criteria:
a) *Organization of information according to expert defined intermediate constructs
b) Organization of information according to data source
c) Organization relative to reference ranges
d) Organization by relative value
Page | 245
Correct answer: a) Organization of information according to expert defined intermediate

constructs
Explanation: The experimental arm of our study demonstrated that presenting information
according to expert defined intermediate constructs (clusters of findings that are diagnostically
or therapeutically meaningful) significantly improved resident physician attention to SIRS
criteria. This approach was superior to our control arm which organized information according
to data source. Displaying relative to a reference range could assist with determining if
individual values are abnormal, but does not address complex concepts such as SIRS, which
require interpretation of multiple abnormal values. Similarly, ordering by relative value (i.e.,
largest to smallest) would not be helpful in assessing multi-factorial concepts.
S88-3-2:
Which of the following has the greatest impact on the sequence clinical information that is
viewed by clinicians when a diagnosis of Systemic Inflammatory Response (SIRS) is suspected?
a) *The order in which it is presented
b) Relevance to the clinical scenario
c) Position of each variable relevant to SIRS
d) Deviation from the range of normal
Correct answer: a) The order in which it is presented
Explanation: Our study found that the order in which information was presented had the
greatest impact on which data elements were viewed by clinicians when considering a diagnosis
of SIRS. The strongest viewing pattern that we observed was to view information from the top
of the screen to the bottom. This pattern was not affected by the clinical scenario or the degree
to which results were abnormal. Organization of data relevant to SIRS according to
intermediate constructs did result in significantly greater attention to SIRS criteria, but was a
smaller effect than the order in which each element was presented.
Efficiency and Accuracy of Kinect and Leap Motion devices Compared to the Mouse for
Intraoperative Image Manipulation
Uchenna A. Uchidiuno; Yuanyuan Feng; Helena M. Mentis; Hamid Zahiri; Adrian Park; Ivan
George
S88-4-1:
Surgeries that involve the use and manipulation of medical images for referencing present
certain challenges for the surgeon. A few of these challenges reside in the use of the mouse and
keyboard to perform these operations. With the advent of touchless interaction devices such
Page | 246
the Microsoft Kinect and Leap Motion, the current method of interaction with these images are
presented with the opportunity for improvement. Which of the following is most likely the
motivation for implementing a touchless interaction device in place of the standard mouse?
a) current means of image interaction during surgery are too slow
b) use of the mouse and keyboard are cumbersome for the surgeons
c) touchless devices out-perform the mouse for all purposes
d) *maintaining a sterile environment during surgery
Correct answer: d) maintaining a sterile environment during surgery
Explanation: One of the motivations behind the use of touchless interaction devices is to
eliminate the need for a surgeon to have to de-glove in order to interact with a keyboard and
mouse when the need for image referencing comes up during a surgery. With current research
highlighting the issues of sterility related to use of keyboard and mouse, the use of touchless
devices enables surgeons to avoid making physical contact with the system and eliminates the
need to take off gloves during surgery, this facilitates the ability for a surgeon to maintain
sterility.
S88-4-2:
The use of touchless interaction devices show potential for alleviating some challenges faced
with image manipulation during surgery. However, what is considered the best touchless
interaction device is not consistent. Which of the following accounts for this finding?
a) one touchless device is better suited for a particular surgery
b) *device performance is affected by task type
c) one touchless device out performs the other for all tasks
d) appropriate touchless device is determined by surgeons preference
Correct answer: b) device performance is affected by task type
Explanation: In our study results, we found that all devices had varying performances in term of
efficiency and accuracy as a direct consequence of the type of task. For example, for the tasks
zoom, circle measure, and line measure, there was little or no difference in performance in all
devices. However, this was not the case for other tasks such as step-through and pan.
Reference(s) for further study: OHara K, Gonzalez G, Sellen A, Penney G, Varnavas A, Mentis H,
et al. Touchless Interaction in Surgery. Commun ACM. 2014 Jan;57(1):707.
Page | 247

A Guideline for Assessing EHR Data Quality for Secondary Use
Nicole G. Weiskopf; Chunhua Weng
S90-1-1:
Reusing EHR data for clinical research has the potential to improve research efficiency and
generalizability. EHR data quality, however, is variable and often poor. Its important that
researchers be aware of the limitations of an EHR-derived dataset when conducting research.
Which of the following courses of action would be appropriate and sufficient for deriving this
knowledge about the data for a particular study?
a) Review existing literature on similar data elements from other institutions and
generalize their results to your own dataset.
b) Re-use an existing EHR-derived dataset that has already had its quality assessed for a
previous, unrelated study and cite the previous data quality findings.
c) *Assess the quality and suitability of an EHR-derived dataset specifically within the
context of the current study.
d) Speak with providers in the appropriate clinical area to ascertain their views on the
quality and suitability of the data.
Correct answer: c) Assess the quality and suitability of an EHR-derived dataset specifically
within the context of the current study.
Explanation: EHR data have been shown to vary in quality across departments and institutions.
It has also been demonstrated that EHR data quality is task-dependent. In other words, while
existing findings about EHR data quality can inform research using these data, these findings
cannot be generalized across institutions, from one setting to another (e.g. clinical to research),
or even from one study to another. A full understanding of the quality and suitability of a
dataset can only be derived within the context of the research task of interest.
S90-1-2:
Many of the studies that report on the quality of EHR-derived data establish accuracy and
completeness through comparison to a gold standard. This gold standard is most commonly
derived through chart review or from administrative (e.g. billing) data. Which of the following is
a benefit of assessing EHR data quality through the use of a gold standard?
a) Gold standard data are always readily available for EHR-derived datasets.
b) Chart review and administrative data, which are often used to generate gold standards,
are generally sources of high-quality data.
c) Generating a gold standard through chart review is a straightforward and efficient
process.
d) *Comparison to a gold standard allows for the calculation of statistics that are
relatively easy to interpret and understand, such as sensitivity and specificity.
Page | 248
Correct answer: d) Comparison to a gold standard allows for the calculation of statistics that
are relatively easy to interpret and understand, such as sensitivity and specificity.
Explanation: Gold standards derived through chart review and administrative data have many
limitations. Chart review is often extremely time-consuming, which can counteract the
efficiency of reusing existing EHR data for research. Moreover, once an EHR-derived dataset has
been deidentified, theres no way to compare those data to other sources at the patient level,
meaning that using a gold standard for data quality assessment may in fact be impossible.
Lastly, using chart review and administrative data for gold standard development also assumes
that the chart and administrative data are themselves of high quality, which is not a reasonable
assumption. One benefit of using a gold standard is that it allows for straightforward
calculations related to data quality, while other methods, though possibly more pragmatic or
appropriate, can yield results that require more complex interpretation or analysis.
Effects of HIE/HIT Implementation and Coordination of Care on Health Outcomes and Quality
Onyinyechi U. Enyia Daniel; Edward Mensah
S90-2-1:
Jack was admitted to the Emergency department last week for chest pain and shortness of
breath. During his visit the emergency department, doctors informed him that he also had high
blood pressure. Two weeks after discharge from the hospital, Jack saw Dr. Johnson, Jack's
primary care physician. Dr. Johnson was able to access Jack's clinical care summary
documentation and clinical discharge instructions electronically. Which of the following is a key
benefit of access to electronic clinical information?
a) *Reduced hospital readmission
b) Reduced cost
c) Improved public health response
d) Decreased access to care
Correct answer: a) Reduced hospital readmission
Explanation: Clinical care summary documentation and clinical discharge instructions were
associated with reductions in hospital readmission. Access to critical patient information can
assist primary care providers and other members of the health care team make clinical
decisions appropriate to the patients medical history.
Page | 249
S90-2-2:
Dr. Jordan is working with hospital administrators to create a business case for participation in
a regional Health Information Exchange (HIE). A strong case for HIE can be made for which of
the following initiatives?
a) Cost reduction
b) *Care coordination
c) Patient exclusion
d) Increased utilization
Correct answer: b) Care coordination
Explanation: HIEs allow for enhanced care coordination among disparate providers. This
coordination allows for more effective population health management, and may also result in
improved health outcomes.
Impact of Electronic Health Records on Quality of Care: Evidence on Inpatient Mortality,
Readmissions, and Complications
Tina Hernandez-Boussard; Catherine Curtin; Doug Morrision; Swati Yanamadala; Katherine
McDonald
S90-3-1:
Evidence from this study suggests that adoption of EHRs is
a) is associated with improved care coordination and reduced 30-day readmissions
b) is associated with decreased inpatient mortality
c) *is not associated with decreased inpatient mortality after controlling for important
confounders
d) is associated with improved patient safety
Correct answer: c) is not associated with decreased inpatient mortality after controlling for
important confounders
Explanation: After controlling for important confounders, such as disease severity, the
implementation of a full or partial EHR system was not associated with improved inpatient
mortality compared to hospitals with no EHR system.
Page | 250
S90-3-2:
In this population based study, the cross-sectional analyses indicated that
a) Black patients were more likely to go to a hospital with no EHR system
b) *Patients with public coverage were more likely to go to a hospital with an EHRsystem
c) Older patients were more likely to go to hospitals with an EHR-system
d) none of the above
Correct answer: b) Patients with public coverage were more likely to go to a hospital with an
EHR-system
Explanation: Our data suggested that a larger portion of patients with Medicaid or Medicare
coverage went to hospitals with a fully implemented or partially implemented EHR system
compared to patients with private insurance coverage
S90: Papers/Podium Presentations - EHR Usability and Quality
Usability Testing of an Ambulatory EHR Navigator
Gretchen Hultman; Elliot G. Arsoniadis; Jenna Marquard; Rubina F. Rizvi; Saif Khairat; Keri
Fickau; Genevieve B. Melton
S90-4-1:
During this study, eight participants completed tasks using two different versions of a navigator
embedded in a commercial EHR. One outcome examined was time to complete task. Which of
the following varied significantly?
a) *Time to complete tasks across participants
b) Average time to complete tasks across navigators
c) Average time to complete tasks across cases
d) Perceived workload across cases
Correct answer: a) Time to complete tasks across participants
Explanation: During this study, high level of individual variation was observed across
participants regardless of which navigator was used. This indicated that individual patterns and
preferences are important when designing EHR based tools.
S90-4-2:
During this study, participants tested two different versions of an ambulatory navigator were
asked which of two different navigators in an EHR they preferred. Overall, which best describes
participants navigator preferences?
a) Participants had a strong preference for the new navigator
b) Participants had a strong preference for the old navigator
c) *Preferences varied widely among participants
Page | 251
d) Participants did not express a strong preference for either navigator

Correct answer: c) Preferences varied widely among participants
Explanation: Among the participants in this study, preferences between the two navigators
varied widely with some preferring the new navigator, some preferring the old navigator, and
some not expressing a strong preference. This demonstrates the difficulty in finding one
optimal design for an EHR based tool such as a navigator.
Page | 252
Wednesday, Nov. 18, 2015 Answers & Explanations

and Investigation
Patricia C. Dykes; Sarah Collins; Anuj K. Dalal; Ryan Greysen; Cindy Dwyer
S91-1:
Which methods of user authentication for enterprise sign-on access is not considered strong
(i.e., two-factor authentication)?
a) USB token (e.g., RSA Secure ID)
b) Finger print biometrics on personal mobile device
c) *Username and password
d) One-Time-Password (OTP) tokens sent by SMS to personal mobile phone
Correct answer: c) Username and password
Explanation: Two-factor authentication (also known as 2FA) is a technology that provides
identification of users by means of the combination of two different components. The use of
two-factor authentication to prove ones identity is based on the premise that an unauthorized
user is unlikely to supply both factors required for access. These components may be something
that the user knows (a password), something that the user possesses, or something that is
inseparable from the user. To authenticate ones identity, the user must be able to
demonstrate Something they know - a password, for example. Something they are - a physical
characteristic such as a fingerprint, handprint, voice print or eye pattern. Something they have typically a token, ID card or pass card. Entering the username and password alone is not
considered strong two-factor authentication because it addresses only one factor, user
knowledge.
S91-2:
Which of the following is true about older adults and general technology use?
a) Most seniors dont use the internet on a regular basis.
b) Most seniors dont use a cell phone.
c) Most seniors dont own a desktop or laptop computer.
d) *Most seniors dont own a smartphone.
Correct answer: d) Most seniors dont own a smartphone.
Explanation:
a) Incorrect: about 60% of seniors use the internet; 70% of those do so every day.
b) Incorrect: about 80% of seniors own a cell phone.
Page | 253
c) Incorrect: about 2/3 of seniors own a desktop or laptop computer.

d) Correct: only about 20% of seniors own smartphones.
S91-3:
Which of the following is true about use of patient portals by older adults in the hospital?
a) Most older adults are not interested in accessing their electronic health records (EHR) in
the hospital.
b) Older adults are less able than younger patients to use hospital-provided devices (e.g.
tablets) to access their EHR in the hospital
c) Older adults are as likely as younger patients to bring their own mobile computing
devices to the hospital.
d) *Older adults may require more detailed explanation and more time to successfully
use mobile devices to access their EHR in the hospital
Correct answer: d) Older adults may require more detailed explanation and more time to
successfully use mobile devices to access their EHR in the hospital
Explanation:
a) Incorrect: older adults are just as eager as younger patients to gain electronic access
b) Incorrect: older adults are just as capable of using tablets and can complete assigned tasks in
their EHR with rates similar to younger patients
c) Incorrect: only 1 in 4 older adults bring their own device to the hospital (compared to 4 in 5
patients under 65)
d) Correct: while older adults are just as able to access their EHR and complete tasks, they often
require more assistance perhaps as much as double (30 vs. 15 minutes)
S91-4:
Which of the following strategies is consistent with user-centered design?
a) The design team includes end-users with an understanding of tasks and environments.
b) The process is linear with a stage gate between each phase where requirements are
c) The design mainly addresses one aspect of the user experience.
d) *The design team includes multidisciplinary skills and perspectives.
Correct answer: d) The design team includes multidisciplinary skills and perspectives.
Explanation:
Page | 254
a) This answer is not correct because while end user requirements need to be considered from
the beginning and included into the whole product cycle, the design team does not need to
include end users as members.
b) This answer is not correct because the UCD process is iterative (not linear).
c) This answer is not correct because UCD should address all aspects of user experience.
d) This answer is correct because a core requirement of UCD is that the design team includes
multidisciplinary skills and perspectives.
S91-5:
Which of the following is not considered protected health information?
a) Patient problem list shared by the patient and care team
b) Medication list with infobuttons links for the patient
c) Educational videos about common diagnoses on a patient portal home page
d) Messages sent between a health care proxy and provider on a patient portal
Correct answer: c) Educational videos about common diagnoses on a patient portal home
page
Explanation:
a) This answer is not correct because any information about health status or provision of
healthcare that can be linked to a specific patient is PHI, this includes clinical problem lists.
b) This answer is not correct because any information about health status or provision of
healthcare that can be linked to a specific patient is PHI, this includes medication lists. A
medication list may link out to external resources through the infobutton standard. Those
external resources are not PHI, but the medication list itself can be attributed to the patient
and is therefore PHI.
c) This answer is correct because the provision of educational videos about common diagnoses
on a website or patient portal is not PHI, as long as those videos are not selected for display
based on specific patients diagnoses. If a patient portal uses a patient diagnoses to suggested
educational videos that process is using PHI.
d) This answer is not correct because any information about health status or provision of
healthcare that can be linked to a specific patient is PHI, this includes messages between care
team members and a patient or health care proxy.
Dalal AK, Dykes PC, Collins S, Lehmann LS, Ohashi K, Rozenblum R, Stade D, McNally K, Morrison
CR, Ravindran S, Mlaver E, Hanna J, Chang F, Kandala R, Getty G, Bates DW. A web-based,
patient-centered toolkit to engage patients and caregivers in the acute care setting: a
preliminary evaluation. J Am Med Inform Assoc. 2015 Aug 2.
Page | 255

Uncertainty, Case Complexity and the Content of Verbal Handoffs at the Emergency
Department
Jan Horsky; Edward H. Suh; Osman R Sayan; Vimla L. Patel
S95-2-1:
There seems to be little convergence in clinical research literature on the characterization of a
high-quality handoff or its primary purpose. A realistic objective may be ensuring that clinicians
taking over a patient can continue care without interruption and delay, do not need to
duplicate tests and procedures unnecessarily and are prepared to respond appropriately and
immediately to sudden or unexpected changes. Shared mental models of patient illness and
care between teams or individuals during a handoff are particularly relevant to preserving care
continuity, especially in emergency settings where adequate and reliable information may not
be available and uncertainty about symptoms, presentation and several possible diagnoses may
create ambiguities and different interpretations.
In the view of this goal, which characterization best describes the purpose of a handoff
specifically suited for emergency care?
a) The core objective is detailed, unambiguous, unidirectional transfer of information from
the outgoing to the incoming team. Accuracy, completeness and attention to detail are
the most optimal safeguards against information loss and misinterpretation.
b) The most important function is to relate sets of laboratory values, vital signs and results
of diagnostic testing, describe procedures, assessment and care trajectory and to specify
planned actions and contingency plans. The incoming clinician will be able to follow up
exactly as expected from written and verbal documentation.
c) A critical attribute of the process is to maintain standard and uniform structure and
content of handoff documentation across all patients, regardless of illness severity, case
complexity, diagnostic certainty or their care trajectory. High level of detail and minimal
deviations from a set protocol will make the process safer, more robust, reliable and will
contribute to higher quality.
d) *Engage both parties in developing a shared mental model of each patients current
state and expected path that can be quickly adopted for care and create opportunities
to consider alternative viewpoints or correct errors. Highly structured processes do
not align well with uncertain diagnoses and indeterminate illness scripts.
Correct answer: d) Engage both parties in developing a shared mental model of each patients
current state and expected path that can be quickly adopted for care and create
opportunities to consider alternative viewpoints or correct errors. Highly structured
processes do not align well with uncertain diagnoses and indeterminate illness scripts.
Page | 256
Explanation: Research evidence suggests that handoffs engender misinterpretations and

omissions that may lead to medical errors and adverse events, but that these discussions also
afford the opportunity to consider alternative viewpoints, correct mistakes and propose new
tests or interventions. Efforts to analyze and improve the process generally assume that its
primary goal is a unidirectional and complete transfer of information between individuals and
teams on adjoining shifts. However, it is also an opportunity for a dialog in which participants
co-construct and negotiate a shared understanding of existing clinical issues and check the
accuracy of medical reasoning and prioritize next steps. Recognizing that there are multiple
purposes for handoffs that need to be addressed simultaneously is a critical precursor to quality
improvement. Standardized handoff tools and templates will likely improve human
performance and reduce the occurrence of errors for relatively routine and predictable illness
scripts. Highly structured, static forms of support will not align well with uncertain diagnoses,
less reliable information and illness trajectories that cannot be clearly determined.
S95-2-2:
Concerns about possible loss of information about patients and care between shifts in the
emergency department have motivated an initiative to revise physician verbal and written
handoffs to make the process more resilient toward misinterpretation and omission. Which of
the following interventions is most likely to have the desired effect and facilitate transitions of
care teams?
a) Clinicians will be required to attend several training sessions in which they will be
coached to emphasize completeness of documentation and detailed description of
illness severity, prior treatment, planned actions, contingency planning and other safe
handoff practices.
b) Highly structured forms with required fields will need to be filled out ahead of every
patient discussed during a handoff to ensure that a complete set of available
information is communicated to the incoming team.
c) A new electronic tool that mimics an existing handoff protocol successfully
implemented in the hospitals pediatric service will be made available to ED clinicians
who will be issued portable devices so that they can share and review the same
information during discussions.
d) *A novel handoff process that would adapt an existing concept that is known to
reduce errors in inpatient settings (such as I-PASS) will be developed. The components
are congruent with characteristics and context of emergency care and what changes
and new interventions would be necessary will be investigated.
Correct answer: d) A novel handoff process that would adapt an existing concept that is
known to reduce errors in inpatient settings (such as I-PASS) will be developed. The
components are congruent with characteristics and context of emergency care and what
changes and new interventions would be necessary will be investigated.
Page | 257
Explanation:
Safety science often emphasizes standardization, consistency, reproducibility and reduction of
variability as fundamental concepts for error reduction. Application of these principles to
handoffs has produced guidelines for clinicians such as SBAR (Situation, Background,
Assessment, Recommendation), ANTICipate (Administrative data, New clinical information,
Tasks, Illness severity, Contingency plans) or I-PASS (Illness severity, Patient summary, Action
list, Situational awareness and contingency planning, and Synthesis or read-back). Some have
clearly improved handoffs in inpatient settings and committees developing safer processes
appropriately prioritize training and the use of checklists or structured forms as best practices.
However, a methodical and systematic investigation of implementation settings is needed to
understand the effects of clinical context and differences in patient populations on the way
clinicians reason about illness and care and how they develop and share their mental models.
These insights are crucial for guiding the development of protocols and interventions. For
example, uncertainty in emergency care is relatively high compared to inpatient or ambulatory
care and clinicians are more likely to use narrative form of reasoning. Effective handoff tools
will therefore need to adapt to the dynamics and differences encountered in emergency care
where a strict standardization approach may be incongruent with environment characteristics.
Physician handoffs: opportunities and limitations for supportive technologies
Katherine Blondon; Rolf Wipfli; Mathieu R. Nendaz; Christian Lovis
S95-3-1:
During evening handoffs, day team physicians transfer their roles and responsibilities to the
night physician. Which of the following statements is true?
a) The handoff process is a standardized process, often similar across medical centers
b) Mnemonics are commonly used to support verbal handoffs
c) *Handoffs are particularly important in complex situations, or when unusual medical
management options have been chosen
d) Information provided during a handoff is similar at novice and expert levels
Correct answer: c) Handoffs are particularly important in complex situations, or when unusual
medical management options have been chosen.
Explanation: Our study helped identify patient cases for which handoffs are particularly
important for quality and efficiency of care and for patient safety. Such situations included
unstable patients, those with anticipated problems (nocturnal agitation, for example), and end
of life care.
Page | 258
S95-3-2:
Let us consider how technology can provide better support for the handoff process. Please
select the correct answer:
a) Alerts are a simple, feasible solution to support handoffs, which just needs determining
who to alert
b) *We should provide the night physician with a system that allows her to review all the
patients she is cross-covering and helps her easily to identify if new information (e.g.,
pending test results) comes in for one of the patients
c) A big data approach could help predict disease trajectories for patients, according to
comorbidities and severity of disease. We would then receive a notification if the
patient diverges from their disease trajectory
d) Abnormal laboratory results and vital signs are defined by the range of normal results,
and are more important than the changes in the parameter over time.
Correct answer: b) We should provide the night physician with a system that allows her to
review all the patients she is cross-covering and helps her easily to identify if new information
(e.g., pending test results) comes in for one of the patients
Explanation: Although well-designed alerts can play an important role for patient safety, many
issues still remain unresolved: threshold for alert, whom to alert, and how to alert (on phone, in
the medical chart, etc). Trends or changes in status are probably more important than a single
value.
Improving Continuity of Care via the Discharge Summary
Farrant Sakaguchi; Leslie A. Lenert
S95-4-1:
Current efforts to improve the data content of the Discharge Care Summary (DCS) _____
a) will significantly improve the continuity of care for patients.
b) *fail to emphasize medical decision making.
c) will make it easy to implement in most EHRs.
d) offer the best solution for HIE.
Correct answer: b) fail to emphasize medical decision making.
Explanation: The presence of medical decision making was one of the focuses of the research.
One of the limitations of this study is that we did not link weaknesses of these discharge
Page | 259
summaries with patient outcomes. While most practitioners will agree that a clearer handover
makes it simpler to care for a patient, they also have learned to provide good care even in the
absence of poor information flow. While many EHR's are improving with tools to automatically
insert data into documentation forms, an efficient handover that includes relevant and less
obvious reasoning and goals still requires a clinician. One of the challenges of HIE is the transfer
of relevant information. Improving discharge summaries may serve to concentrate relevant
information. However, improving documentation is separate from the challenges of a strong
business case for HIE in a fee-for-service world.
S95-4-2:
This research shows that the causal chain A) better discharge summaries increase B) continuity
of care which improves C) patient care _____
a) is straightforward to demonstrate with current qualitative research techniques and
measurement tools.
b) is likely to occur naturally with current discharge summaries.
c) *exemplifies the challenges of proving increased health due to health information
technology alone.
d) ignores the complexity of the healthcare system and different, occasionally competing,
incentives.
Correct answer: c) exemplifies the challenges of proving increased health due to health
information technology alone.
Explanation: Even when continuity of care is interrupted, there are many factors that may
improve or worsen the health outcomes of an individual. Improving the information content of
documentation does not ensure effective communication about medical reasoning, health
goals, etc., between patients and their multiple care providers. One of the limitations of this
study was the qualitative evaluation by a single reviewer. Further, there are limited measures
available regarding what is a good handover in a discharge summary. Many clinicians agree that
discharge summaries can be useful to improve handovers, which is an intermediate step to
increasing continuity of care. While, clinicians have spent years honing their skills to quickly
summarize the inpatient stay, it is less habitual to convey the information that the receiving
provider will need. This research did not focus on the complexity and competing priorities and
incentives of the healthcare system as a whole. However a more efficient transfer of relevant
clinical information is likely to be appreciated by care providers.
Lenert LA, Sakaguchi FH, Weir CR. Rethinking the discharge summary: a focus on handoff
communication. Acad Med J Assoc Am Med Coll. 2014 Mar;89(3):3938.
Page | 260
Bergkvist A, Midlv P, Hglund P, Larsson L, Bondesson A, Eriksson T. Improved quality in the

hospital discharge summary reduces medication errors--LIMM: Landskrona Integrated
Medicines Management. Eur J Clin Pharmacol. 2009 Oct;65(10):103746.
Jack BW, Chetty VK, Anthony D, Greenwald JL, Sanchez GM, Johnson AE, et al. A reengineered
hospital discharge program to decrease rehospitalization: a randomized trial. Ann Intern Med.
2009 Feb 3;150(3):17887.
Moore C, Wisnivesky J, Williams S, McGinn T. Medical Errors Related to Discontinuity of Care
from an Inpatient to an Outpatient Setting. J Gen Intern Med. 2003;18(8):64651.
Moore C, McGinn T, Halm E. Tying up loose ends: discharging patients with unresolved medical
issues. Arch Intern Med. 2007 Jun 25;167(12):130511.

Just One More Patient: Optimizing EMR Documentation in Ambulatory Care
Mark Pierce; Tammy Toscos
S96-1-1:
A cheese finder is a productive, efficient, high quality user of the EHR.
What is the benefit of putting resources into understanding "cheese finders" when rolling out a
new EHR?
a) Keeps your top performers happy
b) Less expensive approach to rolling out an EHR
c) *Offers a way to identify successful strategies
d) This is the approach that most EHR companies recommend
Correct answer: c) Offers a way to identify successful strategies
Explanation:
Looking for the bright spots and uncovering exactly what efforts make an office visit successful
offers a way to identify successful strategies. Taking this alternative approach (the Switch
model) to allocate resources toward the bright spots vs. toward those who were struggling
allows an EHR implementation team to uncover differentiating behaviors that make 'cheese
finders' more efficient and effective in office visits.
Page | 261
S96-1-2:
What were the two differentiating behaviors that distinguished "cheese finders" from the other
providers studied?
a) Using speech recognition software for assessment/plan and for placing orders
b) *Using speech recognition software for HPI documentation and placing orders while
still in the exam room
c) Using speech recognition software for placing orders and completing visit diagnosis
while still in exam room
d) Using speech recognition software for prescriptions and using a personal preference list
for ordering
Correct answer: b) Using speech recognition software for HPI documentation and placing
orders while still in the exam room
Explanation:
To uncover these behaviors, we conducted an observational pilot developing a checklist of 35
items (see Figure 1 of our paper) based on discussions with cheese finders as to what they think
makes them successful and also based on our observations of those doing well. We synthesized
these finding with the information captured with the EHR company's 'user action log' (UAL). The
UAL tool allowed the capture of keystrokes, mouse clicks, menu interaction, and navigation
sequences. So, this provided more detailed analysis of cheese finder behaviors than what was
done in the observational pilot.
Supporting Clinical Cognition: A Human-Centered Approach to a Novel ICU Information
Visualization Dashboard
Anthony Faiola; Preethi Srinivas; Jon Duke
S96-2-1:
Which of the following severely constrict clinicians analytical ability?
a) *Time constraints, interruptions, alarm fatigue, cognitive filtering strategies
b) Distributed cognition
c) Collaboration with other clinicians
d) Technology usage
Correct answer: a) Time constraints, interruptions, alarm fatigue, cognitive filtering strategies
Explanation: ICU is a very complex environment and clinicians work with various interruptions
under severe time constraints that affect their cognitive capabilities. The workflow is non-linear
Page | 262
and requires the clinicians to attend to alarms from patient bedside devices and order entry
systems. Oftentimes, false alarms cause alert fatigue and require the clinicians to cognitively
filter which alarms to attend to.
S96-2-2:
ICUs are identified as having the highest annual mortality rate of any hospital unit (12-22%).
Studies demonstrate that 80% of medical error is attributable to human factors and a majority
of user error is due to cognitive overload. Given the environment of an ICU, which of the
following could better enable rapid information assimilation, pattern recognition, and
diagnostic insights derived from examining large amounts of data in addition to existing
conventional bedside visual displays?
a) *Visualization systems
b) More documentation
c) Definitive division of responsibilities
d) Task escalation
Correct answer: a) Visualization systems
Explanation:
Clinicians need to process patient information from various sources at critical times of
diagnostic decision-making. Analyzing non-electronic and device-generated data which is both
numeric and textual results in excessive cognitive strain and irregular thinking patterns which
impact the quality of care and patient safety. Therefore, critical care teams should be
supported by appropriate visualization systems to ease clinicians working memory and
concentration.
S96-2-3:
Which of the following does the current study propose as a possible solution to maximize the
clinicians ability to control data and narrow down diagnostic decision?
a) Bedside medical devices that display patient vital sign data
b) Critical decision support systems that provide recommendations through rules-based
alerts
c) Paper records
d) *A visualization system that enables rapid recognition of essential changes in
physiological data over a designated time frame
Correct answer: d) A visualization system that enables rapid recognition of essential changes
in physiological data over a designated time frame
Page | 263
Explanation:
The current study proposes a novel EMR dashboard technology that uses a visualization engine
to optimize diagnosis speed and accuracy by rapid recognition of essential changes in
physiological data over a designated time frame, e.g., several minutes, hours, days, or weeks.
Using selection menus, ICU clinicians control the necessary data sources and density, time
periods, and time resolutions to narrow down their diagnostic target of a patients condition. It
was designed to maximize the clinicians ability to control and compare what data is visualized
during a specific context-related patient episode or general diagnosis, e.g., during daily rounds.
Patel VL, Arocha JF, Kaufman DR. A primer on aspects of cognition for medical informatics. J Am
Med Inform Assoc. 2001 Jul-Aug;8(4):324-43.
Ahmed A, Chandra S, Herasevich V, Gajic O, Pickering BW. The effect of two different electronic
health record user interfaces on intensive care provider task load, errors of cognition, and
performance. Crit Care Med.2011,39(7);1626-34.
Khairat S, and Gong Y. Enhancing Patient Safety Through Clinical Communication Knowledge
Representation. e-Health Networking Applications and Services (Healthcom), 2011 13th IEEE
International Conference. 2011:130-3.
Effken J, Loeb R, Johnson K, Johnson S, Reyna S. Using cognitive work analysis to design clinical
displays, In: V.L. Patel, R. Rogers, R. Haux (Eds), Proceedings of MedInfo-2001, American
Medical Information Association, London. 2001,127131.
Cohen T, Blatter B, Almeida C, Shortliffe E, Patel V. A cognitive blueprint of collaboration in
context: distributed cognition in the psychiatric emergency department. Artif Intell Med. 2006
Jun;37(2):73-83.
User Frustration in HIT Interfaces: Exploring Past HCI Research for a Better Understanding of
Clinicians Experiences
Gloria Opoku-Boateng
S96-3-1:
One of the overarching goals of Human-Computer Interaction (HCI) research has been to ___
a) increase interaction with computers
b) *increase the good experiences and possibly reduce or mitigate frustrating first
experiences with technology
c) make computers better than humans.
d) teach humans to be more like computers.
Page | 264
Correct answer: b) Increase the good experiences and possibly reduce or mitigate frustrating
first experiences with technology.
Explanation: In HCI research, the goal is to learn from user experience with technology and
enhance the experience for man.
S96-3-2:
According to the conclusions drawn from this paper, what is the combined-approach to user
frustration research?
a) That a little bit of frustration is good so users need frustration to some extent
b) That frustration and emotion are equal
c) That it is essential to know which parts of an HCI for which humans take responsibility
versus which parts a computer should be left to complete
d) *That the human-side research with the computer-side design approach should work
together to mitigate frustration.
Correct answer: d) That the human-side research with the computer-side design approach
should work together to mitigate frustration.
Explanation: The combined approach proposes that efforts to mitigate end user frustration
should be combined with efforts to make computers more intuitive to user frustration.
Magsamen-Conrad, K., & Checton, M.Technology and Health Care: Efficiency, Frustration, and
Disconnect in the Transition to Electronic Medical Records.Journal on Media & Communications
(JMC), (2014). 1(2).
Ceaparu, I., Lazar, J., Bessiere, K., Robinson, J., & Shneiderman, B.Determining causes and
severity of enduser frustration. International journal of human-computer interaction,. (2004)
17(3), 333-356 3. Bickmore, T. W., & Picard, R. W. Establishing and maintaining long-term
human-computer relationships. ACM Transactions on Computer-Human Interaction (TOCHI),
(2005) 12(2), 293-327
Page | 265

Model Checking for Verification of Interactive Health IT Systems
Keith A. Butler; Eric Mercer; Ali Bahrami; Cui Tao
S96-4-1:
Why is model checking needed for clinical health IT?
a) *The complexity of health IT combined with healthcare makes systems unpredictable.
b) Model checking is a type of system verification.
c) Health IT can disrupt appropriate workflows of care.
d) Clinicians are too busy to learn and stay current on multiple health IT systems.
Correct answer: a) The complexity of health IT combined with healthcare makes systems
unpredictable
Explanation: Unpredictable system behavior risks patient safety.
S96-4-2:
What necessary principle for model checking interactive health IT does MATHflow satisfy?
a) It integrates information modeling with workflow.
b) It treats information as a type of resource.
c) It makes workflow explicit so it can be checked.
d) *It is agnostic how either clinicians or computers can change information.
Correct answer: d) It is agnostic how either clinicians or computers can change information.
Explanation: Verifying the change of information, instead of the performance of tasks and
functions, allows model checking to avoid the vast differences of human cognition and
computer functions.
Evaluating Term Coverage of Herbal and Dietary Supplements in Electronic Health Records
Rui Zhang; Nivedha Manohar; Elliot G. Arsoniadis; Yan Wang; Terrence J. Adam; Serguei V.
Pakhomov; Genevieve B. Melton
S98-1-1:
Herbal and dietary supplement consumption has rapidly expanded in recent years. Due to
pharmacological and metabolic characteristics of some supplements, they can interact with
Page | 266
prescription medications, potentially leading to clinically important and potentially preventable

adverse reactions. A fundamental prerequisite is a functional understanding of supplement
documentation in electronic health records (EHRs) and associated supplement coverage in
major online databases.
Which statement is correct based on the investigation of herbal and dietary supplement term
coverage in the EHR?
a) The medication list in the EHR contains rich information about prescribed supplements.
b) Clinical notes mention supplement usage, which is not mentioned in the medication list
for patients.
c) Current standard terminologies cannot fully cover all supplement terms mentioned in
EHR.
d) *All of the above.
Correct answer: d) All of the above.
Explanation: Drug supplement interactions are attracting more attention recently due to their
known and potential adverse effects on patient safety. The electronic health record (EHR)
system provides a valuable source from which drug-supplement interactions can be mined and
assessed for their clinical effects.
Overall, about 40% of listed medications are supplements, most of which are included in
medication lists as nutritional or miscellaneous products. In our study, all supplements in the
medication lists were also mentioned in the clinical notes. A gap was found between standard
terminologies and supplements in the EHR and not a single standard terminology can cover all
supplements in the EHR. Moreover, clinical notes contain additional information such as
suggestion or discussion of supplements existing in the medication lists as well as those
supplements not mentioned in the medication lists.
2 Years Later: Follow-up to Analysis of Electronic Medication Orders with Large Overdoses
Judith Dexheimer; Eric Kirkendall; Michal Kouril; Thomas Minich; Philip Hagedorn; Cecilia Mahdi;
Stephen A. Spooner
S98-3-1:
Alert fatigue is a common problem in EHRs. We created a salience metric to measure the
effectiveness of the alerts on user behavior. Four alert rules (eRules) are evaluated for
effectiveness and demonstrate the following characteristics.
Which one has the highest Alert Salience Rate?
Page | 267
# Alerts Presented
a. eRule 1
b. eRule 2
*c. eRule 3
d. eRule 4
100
200
100
200
# Alerts
Overridden
90
190
70
170
# Alerts Heeded
10
10
30
30
Salience
Rate?
10%
5%
30%
15%
Correct answer: c. eRule 3.

Explanation: The Alert Salience Rate is defined as the number of alerts that were heeded
(changed prescribing behavior) divided by the total number of alerts presented to users
(opportunities to change behavior). When calculated, eRule 1 has a salience rate of 10%, eRule
2 is 5%, eRule 3 is 30%, and eRule 4 is 15%.
S98-3-2:
Vendor-supplied alerts for medication ordering are prevalent in EHRs. Custom-based dosing
rules for dosing can be implemented in place of these rules and are frequently more specific. In
this study, decreasing alert burden was associated with which of the following:
a) *Increasing alert salience, and a decrease in alerts.
b) No change in alert salience, but a decrease in alerts.
c) Decreasing alert salience, but an increase in alerts
d) No change in alert salience, but an increase in alerts.
Correct answer: a) Increasing alert salience, and a decrease in alerts.
Explanation: The goal of customizing medication rules is to decrease the total number of alerts
and increase the alert salience. We saw an increase in alert salience and a decrease in the total
number of alerts. By incorporating more custom rules into the medication alerting system, we
were able to decrease the total number of alerts displayed to users, increase the number of
alerts that fired off of the custom rules, and increase the total salience of our medication alerts.
Pharmacy drug dispensing after physician discontinuation (cancel) orders
Tewodros Eguale; Aman D. Verma; Enrique Seoane-Vazquez; Rosa Rodriguez-Monguio; David
W. Bates; Robyn Tamblyn; Gordon Schiff
S98-4-1:
In an outpatient setting, the most common reason for physicians to discontinue drugs is
because they are no longer necessary. Which are the next most frequent reasons for drug
discontinuation or cancel orders?
Page | 268
a) *Adverse drug event and ineffectiveness of the drug

b) Patient request and substitution for a less expensive drug
c) Error in prescribing and dispensing
d) Drug discontinued by another physician for an unknown reason and simplifying
treatment to increase adherence.
Correct answer: a) Adverse drug event and ineffectiveness of the drug
Explanation: This study found that many drugs were discontinued because they were no
longer necessary (27.3%), ineffective (18.7%) or caused adverse drug events (18.2%). Other
reasons included error in prescribing (7%), error in dispensing (1%), patient request (5%), and
substitution for less expensive drugs (2%).
S98-4-2:
An electronic health record used in routine clinical practice which captures the reason for drug
treatment (treatment indication) and the reason for drug discontinuation (such as adverse drug
events or effectiveness) can effectively replace
a) case-control and cohort pharmacoepidemiology studies
b) pragmatic clinical trials
c) *spontaneous reporting of adverse drug reactions
d) case-cross over studies
Correct answer: c) spontaneous reporting of adverse drug reactions
Explanation: Spontaneous reporting of adverse drug events as the primary method of postmarket surveillance is plagued by systematic under-reporting of adverse events by physicians,
lack of accurate numerators and denominators to estimate incidence, delay in detection,
incomplete data on the treatment indication, and the lack of a relationship between reporting
and prescribing. Broad-scale adoption of electronic documentation of treatment indication
nationally and internationally, coupled with information on drug discontinuations, would allow
the creation of data in real-time to evaluate the safety and effectiveness of drugs and may
eventually replace spontaneous reporting of adverse drug events.
Schiff GD, Rucker TD. Computerized prescribing: building the electronic infrastructure for better
medication usage. JAMA 1998 Apr 1;279(13):1024-9.
Eguale T, Tamblyn R, Winslade N, Buckeridge D. Detection of adverse drug events and other
treatment outcomes using an electronic prescribing system. Drug Saf 2008;31(11):1005-16.
Page | 269
Eguale T, Winslade N, Hanley JA, Buckeridge DL, Tamblyn R. Enhancing pharmacosurveillance

with systematic collection of treatment indication in electronic prescribing: a validation study in
Canada. Drug Saf 2010 Jul 1;33(7):559-67.

Electronic Health Management Platform (eHMP): The Next Phase of VAs EHR
Jonathan R. Nebeker; Walter P. Nichol; Shane McNamee; Jessica Murphy; James L. Hellewell;
Reese Omizo; Kristian Johnson; Margaret McDonald; Kensaku Kawamoto; Guilherme Del Fiol;
Emory Fry; Elaine Hunolt; Theresa Cullen; Scott D. Wood; Jennifer Herout; Charlene Weir
S101-1-1:
Condition-based worksheets in eHMP represent support what kind of workflows:
a) Business processes supported by underlying workflow engines.
b) Flow to track patient movement among various team members for care management.
c) *Cognitive workflow to understand the status of a condition and facilitate decisions.
d) Wizards that step users through data entry.
Correct answer: c) Cognitive workflow to understand the status of a condition and facilitate
decisions.
Explanation: Workflow" with respect to EHRs is implemented in many different ways. Most
rarely is workflow used in the context of cognitive flow. Most common workflow tools are a
sort of wizard or layout that guides the user through steps in data entry and actions. Ironically,
design for support of cognition is often at odds with design for support of railroaded or guided
workflows. Support for cognition around a medical condition must enhance the ability to see
relationships among a few factors to determine desirableness of current state and reveal
factors affecting current state and which factors need to change, if any, to get to the desired
state. This type of understanding requires sequential relational compositions of drugs, other
conditions, and observations to arrive at the series of inferences underlying the assessment and
plan. Excessive guidance though these inferences often manifests as narrow perspective on
information that constrains information foraging required to support non-linear thought
processes.
Page | 270

Towards an Open EHR Platform: Porting a Complex Application using SMART on FHIR
Alistair Erskine; Marcy Stoots; David McCallie
S101-2-1:
Learning Health Systems aim to accomplish all of the following except:
a) Promote the progress of science
b) Generate new knowledge or insight
c) Analyze data produced as a by-product of care
d) Encourage continuous improvement
e) *Avoid deviation from common practice
Correct answer: e) Avoid deviation from common practice
Explanation: The Institute of Medicine defines the Learning Health System (LHS) as progress in
science, informatics, and care culture align to generate new knowledge as an ongoing, natural
byproduct of the care experience, and seamlessly refine and deliver best practices for
continuous improvement in health and healthcare. The entire series from the IOM is available
online for free PDF download at
http://www.nap.edu/catalog/13301/the-learning-health-system-series.
The LHS tends to question common practice, rather than avoid deviation from it, in favor of
developing practice-based evidence practice through a series of continuous feedback.
S101-2-2:
HL-7 sponsored FHIR (Fast Healthcare Interoperability Resource) is a(n):
a) Interface engine that facilitates the exchange of clinical data across Health Information
Exchanges
b) Internet protocol that improves the efficiency of transmitted clinical data
c) *Draft standard and RESTful Application Programming Interface to transmit clinical
data
d) Meaningful Use protocol adopted by the ONC for Stage 2 certification
Correct answer: c) Draft standard and RESTful Application Programming Interface to transmit
clinical data
Explanation: FHIR resources are draft HL-7 standards that use web-based language or RESTful
API to transmit clinically relevant data between software applications. Details accessible at
http://www.hl7.org/fhir. FHIR resources are not an interface engine but rather a set of
Page | 271
standards and APIs; the use of FHIR resources does not increase the transmission speed of the
transaction; and FHIR standards were not part of Meaning Use Stage 2 certification, in
particular because Stage 2 certification criteria emerged before FHIR standards were released
and FHIR standards remain in draft.
S101-2-3:
In order to minimize the re-work needed to move functionality across EHRs in a timely manner,
one should:
a) *Build an app using SMART Platform and FHIR data exchange resources that work
with both EHRs
b) Customize each EHR by configuring the respective system to take advantage of the
native EHRs
c) Submit an enhancement request to both EHR vendors for the new functionality sought
d) Join a local Health Information Exchange to move clinical data across EHRs
Correct answer: a) Build an app using SMART Platform and FHIR data exchange resources that
work with both EHRs
Explanation: The JASON Report presented to the ONC offered that the use of public APIs and
non-proprietary interfaces between EHRs would help interoperability. In addition, the JASON
report also described adding functionality in the form of apps developed once and made
useful across multiple EHRs with minimal modifications. The ONC funded a series of SHARP
grants, including the SMART Platform project. SMART Platform offers software developers a
non-proprietary way to exchange authentication, user, patient and encounter information
between app and EHR. The latest iteration of HL-7 interface standards, while still in draft/test
use, provides a simpler and web-based (RESTful API) means to exchange clinical information
between app and EHR. Submitting enhancement requests to EHR vendors while feasible does
not guarantee that the EHR vendor will take on the enhancement, nor it such an approach
consistent with the desired timelines to production. The customization of each EHR requires
subspecialized skills and re-work as the developer goes across different EHR vendors. Joining a
HIE only moves clinical data itself, not functionality or clinical workflow.
S101-2-4:
Which one of these needs is addressed by the SMART-on-FHIR approach to open platforms? As
a developer of a clinical application, I would like
a) *to make my software available to users of different EHRs, without rewriting or
customizing for each new platform.
b) my softwares internal, private, data representation to be consistent with a standard.
c) to have a single public store in which to market my software.
d) to ensure that my users cannot easily switch to a competitors product.
Page | 272
Correct answer: a) to make my software available to users of different EHRs, without

rewriting or customizing for each new platform.
Explanation: The purpose of SMART-on-FHIR is to allow an ecosystem for substitutable apps.
A SMART-on-FHIR application can usually be integrated into any compliant EHR without
custom, EHR-specific development. To reverse this plug-and-play model, within an EHR, one
SMART-on-FHIR app can easily be substituted for another. This creates an open model that is
the very opposite of answer d); users can choose the components they want to use, and easily
substitute one app for another if it is better suited to their needs.
Answer b) is incorrect because the SMART and FHIR specifications only dictate how an
application is invoked and how data is exchanged between applications. The application
developer is free to choose any internal data representation, data storage, programming
language, as long as the app respects the application programming interface (API) when
communicating to other applications.
Answer c) suggests a business model in which all SMART-on-FHIR applications are sold through
a single store. While the easy plug-ability of SMART and FHIR apps may help to enable an
app store (or many stores), the technical frameworks are independent of business models
that might emerge, and certainly dont limit application exchange to a single source.
S101-2-5:
Which one of these is NOT the responsibility of an EHR platform that supports SMART-on-FHIR
integration of clinical applications?
a) Implement FHIR services that can be used to access clinical data from the EHR.
b) Support the launching of SMART applications within the clinician's workflow.
c) Provide OAuth2-based security to ensure only authorized access to clinical data can
occur.
d) *Capture usage information and bill the EHR user for the use of SMART-on-FHIR
applications.
Correct answer: d) Capture usage information and bill the EHR user for the use of SMART-onFHIR applications.
Explanation: For an EHR to support SMART-on-FHIR applications, it must provide the three
functions described in answers A, B, and C: a method for launching a SMART application with
patient and user context, FHIR services that let the application access the patient clinical data it
needs, and lastly the security mechanism that protects those data services from inappropriate
access. These technical specifications do not address a billing model for SMART-on-FHIR
applications, as per answer D. In fact, they allow many business models, ranging from free
Page | 273
applications, to applications sold by an EHR vendor, to applications marketed by

entrepreneurial software developers directly to clinical users.
S101-2-6:
Which of the following is an accurate description of FHIR APIs?
a) FHIR supports only "read" access to EHR data
b) FHIR assumes that the EHR uses a relational data model
c) *FHIR uses common Internet standards including: HTTP, JSON, and XML
d) FHIR is based on HL7's Clinical Document Architecture
Correct answer: c) FHIR uses common Internet standards including: HTTP, JSON, and XML
Explanation: FHIR services use proven Internet standards that are already familiar to many
software developers. FHIR services are based on a RESTful model that allows access to
individual data elements as distinct resources, not a single summary document as in the CDA
(answer D). Read (GET) and write (PUT or POST) functions are included in the FHIR
standard. There is no limitation to read only, as stated in answer A. The FHIR standard
addresses the exchange of data between applications, but makes no requirements or
assumptions about how data is handled within an application; answer B is incorrect.
Mandl KD, Mandel JC, Kohane IS. Driving Innovation in Health Systems through an Apps-Based
Information Economy. Cell Systems. 2015 Jul 29. Available at:
http://dx.doi.org/10.1016/j.cels.2015.05.001
Page | 274
Richard H. Epstein; Karl Poterack; Patrick Guffey; Bala G. Nair; Bala G. Nair; Brian W. Pickering
S102-1:
Dr. Smith is performing a laparoscopic cholecystectomy, which the operating room scheduling
system has calculated to last 90 minutes from room entry to room exit. The scheduled duration
was based on a weighted estimate of the durations of the 20 times he has previously done this
procedure and his estimate for the current case. The case has been ongoing for 75 minutes.
Which of the following is a true statement regarding the expected median time remaining in
the case?
a) 15 minutes.
b) Less than 15 minutes.
c) *More than 15 minutes.
Correct answer: c) More than 15 minutes.
Explanation: Because the median estimated time was 90 minutes, this means that some cases
were completed in less time (say, by 70 minutes). It is a mathematical consequence that the
median duration of the cases lasting longer than 70 minutes will be more than 90 minutes.
Thus, the expected median time remaining in the case is more than 20 minutes. The implication
of this finding is that operating room dashboards that display the progress and estimated end
times of ongoing cases should incorporate Bayesian methods to adjust those times.
S102-2:
Surgical durations typically have a right skewed distribution, typically well described by a
lognormal distribution. What is the consequence of this observation for a given procedure
performed by Dr. Jones, assuming that estimated durations are calculated using the mean of
Dr. Joness prior cases?
a) Most (>50%) of such cases will take longer than the scheduled time.
b) *Most (>50%) of such cases will take less than the scheduled time.
c) Half the cases will take the scheduled time.
Correct answer: b) Most (>50%) of such cases will take less than the scheduled time.
Explanation: Because procedure durations are right skewed, this means that the mean duration
will be greater than the median duration. Thus, most (i.e., >50%) cases will take less than the
estimated duration.
Page | 275
S102-3:
What best describes the regulatory oversight on clinical decision support systems:
a) Clinical decision support systems are currently not regulated
b) *Clinical decision support systems that perform therapeutic recommendations likely
require regulatory approval
c) All clinical decision support systems require clearance from regulatory agencies
d) Regulatory oversight is the same in the United States and European Union
Correct answer: b) Clinical decision support systems that perform therapeutic
recommendations likely require regulatory approval
Explanation: Regulation of clinical decision support systems in the US is currently in a state of
transition. Latest guidance from the FDA suggests that systems that provide guidance on
evidence-based best practice protocols and redisplay of medical device data will not be
regulated. However, systems that make therapeutic recommendations or immediate clinical
actions are likely regulated, though specifics are unclear at this time.
S102-4:
Of the following, the most reliable source of data for decision support and reporting is:
a) ICD-9 codes entered manually at discharge
b) surgical start and stop times entered manually post hoc
c) *Physiologic data retrieved directly at one minute intervals from monitors
d) Vital signs dictated into an operative note
Correct answer: c) Physiologic data retrieved directly at one minute intervals from monitors
Explanation: Physiologic data retrieved directly from monitors is not subject to human
intermediaries. ICD-9 codes entered manually, times entered post hoc, and data dictated into
an operative note are subject to the vagaries of human intermediaries and subject to error;
additionally, data entered manually is of much greater latency than data extracted at frequent
intervals from physiologic monitors.
Page | 276
S102-5:
Which of the following is an example of managerial decision support?
a) *tracker board that identifies when patients preoperative preparation has been
completed
b) Pop-up reminder to administer antibiotics within one hour of surgical incision
c) Text alerts with glucose results in patients at risk for perioperative hyper- or hypoglycemia
d) Automated checklist to encourage adherence to a central line infection prevention
bundle
Correct answer: a) tracker board that identifies when patients preoperative preparation
has been completed
Explanation: Perioperative decision support can be categorized as: managerial, process of care,
or outcomes based. Managerial decision support provides data to facilitate the efficient use of
resources. Process of care decision support provides information to improve adherence to
clinical protocols, guidelines, and standards of care. Outcome based decision support is
designed to facilitate care that leads to better patient outcomes. Since the other answer
options, b, c, and d, all are aimed at improving adherence to clinical guidelines and protocols,
they are examples of process of care decision support.
S102-6:
Most previous studies of clinical decision support systems have shown:
a) improvement in provider process adherence, but no improvements in any clinical
outcomes
b) no improvement in provider process adherence or any clinical outcomes
c) improvement in patient outcomes but not in provider process adherence
d) *improvement in provider process adherence but no significant improvements in
major patient outcomes such as mortality
e) improvement in provider process adherence and in major patient outcomes such as
mortality
Correct answer: d) improvement in provider process adherence but no significant
improvements in major patient outcomes such as mortality
Explanation: Most investigations of clinical decision support systems typically show noticeable
improvements in provider adherence to processes and proposed workflows. While some
improvements in less than major patient outcomes have been demonstrated, significant
Page | 277
improvements in major outcomes such as mortality have been difficult to demonstrate. In the
perioperative realm, this is in part due to the fact that some of these significant outcomes occur
well downstream of the immediate perioperative period.
Dexter F, Epstein RH, Lee JD, Ledolter J. Automatic updating of times remaining in surgical cases
using bayesian analysis of historical case duration data and "instant messaging" updates from
anesthesia providers. Anesthesia & Analgesia 108:929-940, 2009
Wanderer JP, Sandberg WS, Ehrenfeld JM. Real-Time Alerts and Reminders Using Information
Systems. Anesthesiology clinics. 2011;29(3):389-396.
Courtney Lyles; Urmimala Sarkar; Neda Ratanawongsa; Danielle E. Oryn
S104-1:
a) *50% registered and 5% viewing, transmitting, or downloading information
Correct answer: a) 50% registered and 5% viewing, transmitting, or downloading information
Explanation: The correct choice is A from the Office of the National Coordinator of Health ITs
website.
S104-2:
than English?
a) Portals must be available to patients in the top 3 languages used within that healthcare
systems
c) *There is no specific guidance about portals related to language
Correct answer: c) There is no specific guidance about portals related to language
Page | 278
Explanation: The correct choice is c) as there is no official recommendation from the Office of
the National Coordinator of Health IT on this issue even though it will be a clear barrier for the
millions of patients in the US with limited English proficiency.
S104-3:
Which of the following is a top barrier to portal use among safety net patient populations?
a) Prefer WebMD instead
b) Computer/Internet access
c) Dislike their doctor
d) *Inadequate health literacy/literacy skills
Correct answer: d) Inadequate health literacy/literacy skills
Explanation: The correct choice is D health communication barriers because of literacy/health
literacy are a major barrier in our research to date. Basic access to the computer/Internet
seems to be less of a challenge because of the shrinking digital divide in the US, but the skills to
be able to use a sophisticated and often complex portal website are often insufficient.
S104-4:
Which of the following is a top barrier to portal use among healthcare providers?
a) Prefer emailing patients through their personal email accounts instead
b) *Lack of protected time for secure messaging
c) Dont think patients should have access to their medical information online
d) No need already have enough time with patients during in-person visits
Correct answer: b) Lack of protected time for secure messaging
Explanation: The correct answer is b) since providers are most worried about the amount of
time that it will take for messaging their patients through a portal (especially since this time is
not reimbursed like an in-person visit is).
S104-5:
What proportion of US hospitals are now offering patients access to secure messaging
functionality (as the beginning of Stage 2 Meaningful Use in 2014)?
a) 75% of hospitals
b) *50% of hospitals
c) 25% of hospitals
d) Virtually no hospitals
Page | 279
Correct answer: b) 50% of hospitals

Explanation: Meaningful Use has had a dramatic impact of the portal use features that have
been offered across US healthcare systems. In 2012, there were virtually no hospitals offering
secure messaging to patients in the US, and this has increased to about 50% in 2014 based on
the latest numbers published by CMS and the Office of the National Coordinator for Health IT.
This demonstrates that MU is a powerful tool for patient engagement, and could likely be
leveraged further to think about patient support for being able to use the secure messaging
functionality.
https://www.cms.gov/Regulations-andGuidance/Legislation/EHRIncentivePrograms/Stage_2.html
mobile Digital Access to a Web-enhanced Network (mDAWN): Assessing the Feasibility of
Mobile Health Tools for Self-Management of Type-2 Diabetes
Kendall Ho; Lana Newton; Allison Boothe; Helen Novak Lauscher
S105-1-1:
Surveys of North American population shows mobile phone use trends that demonstrate:
a) Disinterest of young adults in using smart phones for any type of health or disease
management
b) *Older population of patients greater than 65 years old showing strong interest and
ability to use smart phones for health and illness management
c) Individuals living in rural and remote locations declining on line access to health services
d) Patients level of trust of peers with similar diseases is very low
Correct answer: b) Older population of patients greater than 65 years old showing strong
interest and ability to use smart phones for health and illness management
Explanation: Surveys suggest that young people much prefer on line interactions to obtain their
health services, older population wanting to use telehealth to work with their health
professionals to monitor their health, individuals living in rural and remote locations wanting
telehealth and mobile health to obtain improved access to services, and patients having a
strong level of trust of their peers with same diseases in exchanging health tips and sharing
issues related to their diseases.
Page | 280
S105-1-2:
A recent systematic review in using smart phone technologies to influence patient behavior in
physical activities suggests that current evidence in the literature indicates:
a) Self-monitoring of own data by itself is a powerful way to change behavior
b) Self-monitoring of own data together with external incentive or reward for positive
performance will convincing generate change
c) *Setting goals, having expert advice, and peer support work synergistically with selfmonitoring of own data towards effective behavioral change
d) Self-monitoring of own data coupled with external supervision for accountability and
achievement of performance is essential in influencing behavioral change
Correct answer: c) Setting goals, having expert advice, and peer support work synergistically
with self-monitoring of own data towards effective behavioral change
Explanation: The article by Bort-Roig et al Measuring and influencing physical activity with
smartphone technology: a systematic review uncovered modest evidence that physical
activity profiles, goal setting, real-time feedback, social support networking, and online expert
consultation were identified as the most useful strategies to encourage physical activity
change. However, the authors noted that intervention effects reported in the extant
literature are modest at best, and future studies need to utilize randomized controlled
trial research designs, larger sample sizes, and longer study periods to better explore the
physical activity measurement and intervention capabilities of smartphones. Therefore, it is
important to continue to follow the literature and also generate more evidence, such as some
of the findings in our mDAWN study, to further tease out how different personal motivational
strategies and social support with expert interventions can work synergistically to promote
effective behavioral change.
Use of Patient Portals for Personal Health Information Management: The Older Adult
Perspective
Anne M. Turner; Katie Osterhage; Andrea L. Hartzler; Jonathan Joe; Lorelei Lin; Natasha
Kanagat; George Demiris
S105-2-1:
In the study by Turner et al., 20% of patients 60 years and older were using patient portals. The
reasons cited for using patient portals were:
a) A desire to share health information with family members.
b) *Ability to make appointments and have contact with providers.
c) Greater security of health information.
d) A desire to gain new computer skills.
Page | 281
Correct answer: b) Ability to make appointments and have contact with providers.
Explanation: In in-depth interviews participants cited a number of reasons why they were using
patient portals and reported what they liked about their patient portal. The facilitators
mentioned included the ease of use, the ability to make appointments with their providers, and
offering a way to contact their provider directly.
S105-2-2:
In Turner, et al. compared to non-portal users, older adult portal users typically
a) Lived in retirement communities
b) Had an increased need for personal health information management
c) Had less formal education
d) *Reported using the computer 6-7 days/week
Correct answer: d) Reported using the computer 6-7 days/week
Explanation: Among those interviewed, portal users in general used computers more
frequently, lived independently, had more formal education and reported having to manage
their own health information less.
Long-Term Engagement with Health-Management Technology: a Dynamic Process in Diabetes
Predrag Klasnja; Logan Kendall; Wanda Pratt; Katherine Blondon
S105-3-1:
Which of the following statements is true?
a) *Individuals with diabetes find it hard to find practical information about diabetes
management in websites and apps
b) An individual with diabetes should not give advice to another person with diabetes,
because each persons body has different responses
c) Health care providers are overall very knowledgeable about diabetes apps
d) The only way to have well-managed diabetes is to rigorously track foods and blood
glucose levels all the time.
Correct answer: a) Individuals with diabetes find it hard to find practical information about
diabetes management in websites and apps
Explanation: Patients reported difficulties in finding hands-on information about food choices
and meals, particularly for ethnic foods. Patients typically found that their healthcare providers
Page | 282
did not know apps or websites well, particularly with all the rapidly changing range of available
tools. Social support in diabetes is a precious resource, and even if individual responses to
insulin may vary, the basic management principles are applicable to all. Individuals who have
experience with diabetes management do not need to track everything as rigorously, as they
get to know their bodys responses and needs.
S105-3-2:
Which of the following statements is true over time for individuals with diabetes?
a) a) Individuals have high daily needs for information about diabetes and its management
all throughout the disease
b) *Insulin needs and blood glucose responses can change over time
c) All individuals with diabetes benefit from frequent self-monitoring of their blood
glucose levels
d) Individuals who have had diabetes for many years dont go out to eat at restaurants
because its too difficult to measure amounts of food and count carbs
Correct answer: b) Insulin needs and blood glucose responses can change over time
Explanation: Diabetes is a hormone disease which can have variations in the course of the
disease, such as higher insulin resistance over time. Individuals have higher needs for
information at the time of diagnosis, and for finer adjustments in the relearning phase, and
depend less on disease information and tracking in the stabilization and expertise phases.
Individuals with type 2 diabetes without insulin treatment may not necessarily need to track
their blood glucose levels regularly. Individuals who have had diabetes over many years
develop the ability to eye-ball foods and estimate carbs, and can adjust to many new
experiences.
Smartphone Data in Rheumatoid Arthritis - What Do Rheumatologists Want?
Phillip R. Say; Daniel Stein; Jessica S. Ancker; Cheng-Kang Hsieh; J.P. Pollak; Deborah Estrin
S109-1-1:
A new mobile health application allows patients to actively and passively input their personal
health data onto their smartphone. The application produces a report that the patient can
download for their physician. Which report would best facilitate decision making for a busy
practitioner?
a) A table of blood glucose levels linked with a daily diet journal
b) A data visualization demonstrating the patients blood glucose levels as they relate to
the number of steps they took that day
c) A medication adherence journal presented as a list
d) *The patients daily blood glucose levels presented as an average with confidence
intervals
Page | 283
Correct answer: d) The patient's daily blood glucose levels presented as an average with
confidence intervals
Explanation:
Smartphone data that patients share with their physicians should not exacerbate clinician
information overload but facilitate clinical decisions.
Option a) suggesting a table of blood sugars and a daily diet journal would likely be very long
and too much information for a physician to review during an office visit.
Regarding option b), the relationship between blood glucose levels and the number of steps
that a person takes a day is interesting but irrelevant to most physician decision making.
As for option c), an evaluation of medication adherence is very important to decision making
but may not be easy to interpret in the form of a list.
Option d) is the correct answer because an average with confidence intervals gives the
physician a snapshot of how well the patient has been doing over the past month. This type of
information gives the practitioner a good understanding of the patients blood glucose levels
over time, which can be easily compared to the previous month. Data in this format can
effectively guide a physicians decision making for changes in therapy.
S109-1-2:
You recently discovered that a commonly available smartphone app passively tracks the user's
geographic location over time. You wonder whether this could be exploited for health-related
purposes. What should be the next step?
a) Running experiments to determine the precision and accuracy of the measurements
b) Choosing a data visualization
c) *Finding out if tracking the users geographic location over time is a relevant measure
of any disease states
d) Interviewing physicians about how they would use this information when managing
patients
Correct answer: c) Finding out if tracking the users geographic location over time is a relevant
measure of any disease states
Explanation:
The efforts that are necessary to ensure that smartphone data facilitates clinical decisions by
physicians begin with finding out if the data is a relevant measure of a disease state.
Page | 284
Reference(s) for further study: Nundy S, Lu CY, Hogan P, Mishra A, Peek ME. Using patientgenerated health data from mobile technologies for diabetes self-management support:
provider perspectives from an academic medical center. J Diabetes Sci Technol. 2014 Jan
1;8(1):74-82.
S109: Papers - The User Perspective on Informatics Tools: Organizational Uses of Health
Information Exchange to Change Cost and Utilization Outcomes: A Typology from a Multi-Site
Qualitative Analysis
Joshua R. Vest; Erika L. Abramson
S109-2-1:
In much of the research literature health information exchange is conceptualized as structural
feature only. What is the primary challenge this view has created?
a) Evaluations have found no significant effects.
b) It has limited adoption.
c) *It has masked the actual process by which organizations apply information.
d) It has made it harder to measure health information exchange.
Correct answer: c) It has masked the actual process by which organizations apply information.
Explanation: Measuring health information exchange as structural feature does not account for
different user types and different use cases within the same organization.
S109-2-2:
Health information exchange adoption might have the most immediate impact on which type
of activity?
a) Predictive modeling
b) *Administrative, patient finding projects
c) Return on investment
d) Inpatient clinical care
Correct answer: b) Administrative, patient finding projects
Explanation: The risk of non-acceptance of technology tends to run higher among clinicians
than administrative staff and administrative usage requires minimal information like addresses
or telephone numbers to be effective in reaching previously lost patients.
Page | 285

Impact of Robotic Surgery on Decision Making: Perspectives of Surgical Teams
Rebecca Randell; Natasha Alvarado; Stephanie Honey; Joanne Greenhalgh; Peter Gardner;
Arron Gill; David Jayne; Alwyn Kotze; Alan Pearman; Dawn Dowding
S109-3-1:
A hospital has recently acquired a surgical robot. However, the surgeons have expressed
concern that their situation awareness is reduced when using the robot, which could have
implications for patient safety. Which of the following is the most appropriate response in
order to overcome this problem?
a) *The team should be encouraged to communicate information to the surgeon
b) The surgeon should step away from the console intermittently
c) The surgeon should periodically ask the team for updates about the patient state
d) An individual should be identified who has responsibility for updating the surgeon about
changes in the patient state
Correct answer: a) The team should be encouraged to communicate information to the
surgeon
Explanation:
Our interviews revealed that the team plays an important role in maintaining the surgeons
situation awareness, with the team communicating not only information about the patient but
also about the robot, for example, the robot arms clashing or coming too close for the patient.
Teams are more likely to communicate this information if there is a positive relationship
between the surgeon and the team, which may be helped by undertaking whole team training
and/or having a dedicated robotic team.
S109-3-2:
It has been suggested that robotic surgery produces a sense of immersion, which might lead to
better decision making. In our interviews with surgeons, they suggested contexts in which the
surgeons sense of immersion is likely to be greater. Which of the following contexts is most
likely to lead to a greater sense of immersion?
a) The operating room is quiet
b) The surgeon is experienced in robotic surgery
c) *The surgeon trusts the team to make him aware of changes in the operating room
d) The operation is straightforward
Correct answer: c) The surgeon trusts the team to make him aware of changes in the
operating room
Page | 286
Explanation: Our interviews revealed that when the surgeon trusts the team to make him
aware of changes outside of his field of view, he feels confident to remain in the console,
increasing the sense of immersion. Attitudes about the role of experience varied; while some
surgeons thought that their sense of immersion would increase as the experience increased,
others felt that it would decrease.
Healey A, Benn J. Teamwork enables remote surgical control and a new model for a surgical
system emerges. Cognition, Technology & Work. 2009;11(4):255-65.
Challenges and Insights in Using HIPAA Privacy Rule for Clinical Text Annotation
Mehmet Kayaalp; Allen Browne; Pamela Sagan; Tyne McGee; Clement J. McDonald
S109-4-1:
A clinician who works in a clinic of a small town with a population size of 5,000 has observed an
interesting clinical case and decided to publish her observations in a clinical journal. In order to
fully describe the clinical case, she intends to list not only pertinent clinical manifestations of
the case, but also demographic information of her patients such as gender, age, size of
household, and annual household income.
Which of the following statements best describes what she can or cannot do in order to
preserve privacy and confidentiality of her patients?
a) She can provide both age and gender information but not the size or annual income of
the household
b) B. She can provide information about gender, age, and the size of the household
c) *She can provide gender but no other demographic information
d) She can provide all such demographic information since they are not considered
personally identifying information
Correct answer: c) She can provide gender but no other demographic information
Explanation: Although HIPAA Privacy Rule does not consider demographic information as
personally identifying information, the clinician cannot provide such information since the size
of population she serves is smaller than what HIPAA Privacy Rule considers as non-identifying.
The only exception here can be gender information, because gender (male or female)
information alone cannot be used to identify the person individually, but other demographic
information when combined together may be used to identify her patients.
Page | 287
S109-4-2:
Which of the following statements best describes protected health information
a) Health information of particularly vulnerable subjects, such as children, prisoners,
pregnant women, mentally disabled persons, or economically or educationally
disadvantaged persons
b) Health information of subjects who belong to one of the five (physical, social,
psychological, legal, and economic) risk categories
c) Personally identifiable information such as personal name, medical record number of
the patient
d) *Health information linked to the personal identifiers such as the photograph of the
face and drivers license number of an individual
Correct answer: d) Health information linked to the personal identifiers such as the
photograph of the face and drivers license number of an individual
Explanation: Protected Health Information (PHI) is health information that is linked to the
personally identifiable information (PII) of any individual (i.e., PHI = Health Information + PII).
HIPAA Privacy Rule specifies 18 such identifiers, which include personal names, medical record
numbers, drivers license numbers, full face photography and biometric identifiers. Health
information dissociated from PII is no longer considered protected information.
Kayaalp M, Browne AC, Dodd Z.A., Sagan P, McDonald CJ. (2013) Clinical Text De-Identification
Research. Technical Report to the LHNCBC Board of Scientific Counselors. Available at
http://lhncbc.nlm.nih.gov/system/files/tr2013002.pdf
Kayaalp, M., Browne, A.C., Dodd, Z.A., Sagan, P., McGee, T., McDonald, C.J.
(2015). Challenges and Insights in Using HIPAA Privacy Rule for Clinical Text Annotation.
Proceedings of the Annual American Medical Informatics Association Fall Symposium.
Page | 288

AMIA 2015 Self Assessment MOC II Booklet FINAL

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

AMIA 2015 Self Assessment MOC II Booklet FINAL

Uploaded by

Copyright:

Available Formats

AMIA 2015 Self Assessment and

Maintenance of Certification (MOC) Part II Q&A booklet

Email attestation text for American Board of Pathology-boarded

Sessions Designated as Available for MOC-II Credit

Saturday, November 14, 2015

Sunday, November 15, 2015

Monday, November 16, 2015

Tuesday, November 17, 2015

Wednesday, November 18, 2015

Saturday, Nov. 14, 2015 Questions & Answer Options

T04: AMIA 2015 CMIO Workshop

CMIO Surveys: Who are we? What do we do? - Dick Schreiber

CMIO Year-In-Review - Dick Schreiber/Paul Fu, Jr.

Patient Portals and Patient Empowerment - Paul Fu, Jr.

Sunday, Nov. 15, 2015 - Questions & Answer Options

T13: Introduction to Biomedical Informatics

S09: Podium Presentations: Information Retrieval and Data Capture

S09: Podium Presentations: Information Retrieval and Data Capture

S09: Podium Presentations: Information Retrieval and Data Capture

Monday, Nov. 16, 2015- Questions & Answer Options

S20: Podium Presentations - Are We Safer Yet?

S20: Podium Presentations - Are We Safer Yet?

S21: Systems Demonstrations - Mapping Public Health

S21: Systems Demonstrations - Mapping Public Health

S37: Papers - Patients Want to Know

S37: Papers - Patients Want to Know

S41: Podium Presentations - Government Initiatives in Health IT

S41: Podium Presentations - Government Initiatives in Health IT

S42: Podium Presentations - Care Team Communication

S42: Podium Presentations - Care Team Communication

S42: Papers/Podium Presentations - Care Team Communication

S54: Systems Demonstrations - From Patients to Research

Tuesday, Nov. 17, 2015 - Questions & Answer Options

S76: Papers - Making EHRs Useful

S76: Papers - Making EHRs Useful

S76: Papers - Making EHRs Useful

S84: Papers - Clinical Decision Support II

S84: Papers - Clinical Decision Support II

S84: Papers - Clinical Decision Support II

S85: Papers - mHealth

S85: Papers - mHealth

S85: Papers - mHealth

S85: Papers - mHealth

S88: Podium Presentations - Human Factors are Key

S88: Podium Presentations - Human Factors are Key

S88: Podium Presentations - Human Factors are Key

S88: Podium Presentations - Human Factors are Key

S90: Podium Presentations - EHR Usability and Quality

S90: Podium Presentations - EHR Usability and Quality

S90: Papers/Podium Presentations - EHR Usability and Quality

Wednesday, Nov. 18, 2015- Questions & Answer Options

S95: Papers All About Handoffs

a) The core objective is detailed, unambiguous, unidirectional transfer of information from

S95: Papers All About Handoffs

S96: Papers - Human-computer Interaction

S96: Papers - Human-computer Interaction

S96: Papers - Human-computer Interaction

S98 Papers - Taken as Directed

S98 Papers - Taken as Directed

S98 Papers - Taken as Directed

S101: Systems Demonstrations - EHRs of the Future

S105: Papers - Patients in Control: Self-management Tools

S105: Papers - Patients in Control: Self-management Tools

S105: Papers - Patients in Control: Self-management Tools