Professional Documents
Culture Documents
November 2009
Contents
Executive Summary.................................................................................................................................................iii
Introduction...............................................................................................................................................................v
Acknowledgements..................................................................................................................................................ix
Current Hard Problems in INFOSEC Research
1. Scalable Trustworthy Systems ...................................................................................................................1
2. Enterprise-Level Metrics (ELMs) ...........................................................................................................13
3. System Evaluation Life Cycle....................................................................................................................22
4. Combatting Insider Threats . ...................................................................................................................29
5. Combatting Malware and Botnets ..........................................................................................................38
6. Global-Scale Identity Management ........................................................................................................50
7. Survivability of Time-Critical Systems . .................................................................................................57
8. Situational Understanding and Attack Attribution ..............................................................................65
9. Provenance .................................................................................................................................................76
10. Privacy-Aware Security ...........................................................................................................................83
11. Usable Security . .......................................................................................................................................90
Appendices
Appendix A. Interdependencies among Topics . .............................................................................................A1
Appendix B. Technology Transfer ..................................................................................................................... B1
Appendix C. List of Participants in the Roadmap Development..................................................................C1
Appendix D. Acronyms....................................................................................................................................... D1
i
Executive Summary
Executive Summary
The United States is at a significant decision point. We must continue to defend our
current systems and networks and at the same time attempt to “get out in front” of
our adversaries and ensure that future generations of technology will position us to
better protect our critical infrastructures and respond to attacks from our adversaries.
The term “system” is used broadly to encompass systems of systems and networks.
For each of these hard problems, the roadmap identifies critical needs, gaps in
research, and research agenda appropriate for near, medium, and long term
attention.
DHS S&T assembled a large team of subject matter experts who provided input
into the development of this research roadmap. The content was developed over
the course of 15 months that included three regional multi-day workshops, two
virtual workshops for each topic, and numerous editing activities by the participants.
iii
Introduction
Introduction
Information technology has become pervasive in every way—from our phones and
other small devices to our enterprise networks to the infrastructure that runs our
economy. Improvements to the security of this information technology are essential
for our future. As the critical infrastructures of the United States have become more
and more dependent on public and private networks, the potential for widespread
national impact resulting from disruption or failure of these networks has also
increased. Securing the nation’s critical infrastructures requires protecting not only
their physical systems but, just as important, the cyber portions of the systems on
which they rely. The most significant cyber threats to the nation are fundamentally
different from those posed by the “script kiddies” or virus writers who tradition-
ally have plagued users of the Internet. Today, the Internet has a significant role
in enabling the communications, monitoring, operations, and business systems
underlying many of the nation’s critical infrastructures. Cyberattacks are increas-
ing in frequency and impact. Adversaries seeking to disrupt the nation’s critical
infrastructures are driven by different motives and view cyberspace as a possible
means to have much greater impact, such as causing harm to people or widespread
economic damage. Although to date no cyberattack has had a significant impact on
our nation’s critical infrastructures, previous attacks have demonstrated that exten-
sive vulnerabilities exist in information systems and networks, with the potential for
serious damage. The effects of a successful attack might include serious economic
consequences through impacts on major economic and industrial sectors, threats
to infrastructure elements such as electric power, and disruptions that impede the
response and communication capabilities of first responders in crisis situations.
The United States is at a significant decision point. We must continue to defend our
current systems and networks and at the same time attempt to “get out in front”
of our adversaries and ensure that future generations of technology will position
us to better protect our critical infrastructures and respond to attacks from our
adversaries. It is the opinion of those involved in creating this research roadmap that
government-funded research and development (R&D) must play an increasing role
to enable us to accomplish this goal of national and economic security. The research
topics in this roadmap, however, are relevant not only to the federal government
but also to the private sector and others who are interested in securing the future.
v
Historical background research programs. The original list has mixes of legacy systems), and the pres-
proven useful in guiding INFOSEC ence of significant, asymmetric threats.
The INFOSEC Research Council (IRC) research, and policy makers and planners
is an informal organization of govern- may find the document useful in evalu- The area of cybersecurity and the associ-
ment program managers who sponsor ating the contributions of ongoing and ated research and development activities
information security research within the proposed INFOSEC research programs. have been written about frequently over
U.S. Government. Many organizations However, the significant evolution of the past decade. In addition to both
have representatives as regular members technology and threats between 1999 the original IRC HPL in 1999 and the
of the IRC: Central Intelligence Agency, and 2005 required an update to the list. revision in 2005, the following reports
Department of Defense (including the Therefore, an updated version of the have discussed the need for investment
Air Force, Army, Defense Advanced HPL was published in November 2005. in this critical area:
Research Projects Agency, National This updated document included the
Toward a Safer and More Secure
Reconnaissance Office, National Secu- following technical hard problems from
rity Agency, Navy, and Office of the the information security perspective: Cyberspace
Secretary of Defense), Department Federal Plan for Cyber Security
1. Global-Scale Identity Management and Information Assurance
of Energy, Department of Homeland
Security, Federal Aviation Administra- 2. Insider Threat Research and Development
tion, Intelligence Advanced Research 3. Availability of Time-Critical Cyber Security: A Crisis of
Projects Activity, National Aeronautics Systems
Prioritization
and Space Administration, National 4. Building Scalable Secure Systems
Hardening the Internet
Institutes of Health, National Institute 5. Situational Understanding and
of Standards and Technology, National Attack Attribution Information Security
Science Foundation, and the Technical 6. Information Provenance Governance: A Call to Action
Support Working Group. In addition, The National Strategy to Secure
7. Security with Privacy
the IRC is regularly attended by partner Cyberspace
organizations from Canada and the 8. Enterprise-Level Security Metrics
Cyber Security Research and
United Kingdom.
Development Agenda
These eight problems were selected
The IRC developed the original Hard as the hardest and most critical chal-
Problem List (HPL), which was com- lenges that must be addressed by the These reports can be found at http://
posed in 1997 and published in draft INFOSEC research community if trust- www.cyber.st.dhs.gov/documents.html
form in 1999. The HPL defines desir- worthy systems envisioned by the U.S.
able research topics by identifying a set Government are to be built. INFOSEC Current context
of key problems from the U.S. Govern- problems may be characterized as “hard”
ment perspective and in the context of for several reasons. Some problems are On January 8, 2008, the President
IRC member missions. Solutions to hard because of the fundamental techni- issued National Security Presiden-
these problems would remove major cal challenges of building secure systems, tial Directive 54/Homeland Security
barriers to effective information secu- others because of the complexity of Presidential Directive 23, which for-
rity (INFOSEC). The Hard Problem information technology (IT) system malized the Comprehensive National
List was intended to help guide the applications. Contributing to these Cybersecurity Initiative (CNCI) and a
research program planning of the IRC problems are conflicting regulatory and series of continuous efforts designed to
member organizations. It was also hoped policy goals, poor understanding of establish a frontline defense (reducing
that nonmember organizations and operational needs and user interfaces, current vulnerabilities and preventing
industrial partners would consider these rapid changes in technology, large het- intrusions), defending against the full
problems in the development of their erogeneous environments (including spectrum of threats by using intelligence
vi
and strengthening supply chain security, influence in networking and IT systems, interagency coordination to ensure cov-
and shaping the future environment by components, and standards among U.S. erage of all the topics.
enhancing our research, development, competitors. Federal agencies with
and education, as well as investing in mission-critical needs for increased Each of the following topic areas is
“leap-ahead” technologies. cybersecurity, which includes informa- treated in detail in a subsequent section
tion assurance as well as network and of its own, from Section 1 to Section 11.
The vision of the CNCI research com- system security, can play a direct role
1. Scalable trustworthy systems
munity over the next 10 years is to in determining research priorities and
(including system architectures and
“transform the cyber-infrastructure so assessing emerging technology proto-
requisite development methodol-
that critical national interests are pro- types. Moreover, through technology
ogy)
tected from catastrophic damage and transfer efforts, the federal government
2. Enterprise-level metrics (including
our society can confidently adopt new can encourage rapid adoption of the
measures of overall system trust-
technological advances.” results of leap-ahead research. Technol-
worthiness)
ogy breakthroughs that can curb or
Two components of the CNCI deal break the resource-draining cycle of 3. System evaluation life cycle (in-
cluding approaches for sufficient
with cybersecurity research and develop- security patching will have a high likeli-
assurance)
ment—one focused on the coordination hood of marketplace implementation.
of federal R&D and the other on the 4. Combatting insider threats
development of leap-ahead technologies. As stated previously, this Cybersecu- 5. Combatting malware and botnets
rity Research Roadmap is an attempt 6. Global-scale identity management
No single federal agency “owns” the to begin to address a national R&D 7. Survivability of time-critical
issue of cybersecurity. In fact, the agenda that is required to enable us to systems
federal government does not uniquely get ahead of our adversaries and produce 8. Situational understanding and
own cybersecurity. It is a national and the technologies that will protect our attack attribution
global challenge with far-reaching con- information systems and networks into 9. Provenance (relating to informa-
sequences that requires a cooperative, the future. The topics contained in this tion, systems, and hardware)
comprehensive effort across the public roadmap and the research and develop-
10. Privacy-aware security
and private sectors. However, as it has ment that would be accomplished if the
11. Usable security
done historically, U.S. Government roadmap were implemented are, in fact,
R&D in key technologies working in leap-ahead in nature and address many
close cooperation with private-sector of the topics that have been identified Eight of these topics (1, 2, 4, 6, 7, 8,
partners can jump-start the necessary in the CNCI activities 9, 10) are adopted from the November
fundamental technical transformation. 2005 IRC Hard Problem List [IRC05]
Document format and are still of vital relevance. The
The leap-ahead strategy aligns with the other three topics (3, 5, 11) represent
consensus of the nation’s networking The intent of this document is to additional areas considered to be of
and cybersecurity research communi- provide detailed research and develop- particular importance for the future.
ties that the only long-term solution to ment agendas for the future relating to
the vulnerabilities of today’s network- 11 hard problem areas in cybersecurity, The order in which the 11 topics are
ing and information technologies is to for use by agencies of the U.S. Govern- presented reflects some structural simi-
ensure that future generations of these ment and anyone else that is funding larities among subgroups of the topics
technologies are designed with secu- or doing R&D. It is expected that each and exhibits clearly some of their major
rity built in from the ground up. The agency will find certain parts of the interdependencies. The order proceeds
leap-ahead strategy will help extend document resonant with its own needs roughly from overarching system con-
U.S. leadership at a time of growing and will proceed accordingly with some cepts to more detailed issues—except
vii
for the last topic—and has the following Background What R&D is evolutionary and
structure: what is more basic, higher risk,
What is the problem being game changing?
a. Topics 1–3 frame the overarching addressed?
Resources
problems. What are the potential threats?
Measures of success
b. Topics 4–5 relate to specific major Who are the potential
beneficiaries? What are their What needs to be in place for test
threats and needs.
respective needs? and evaluation?
c. Topics 6–10 relate to some of the
What is the current state of the To what extent can we test real
“ilities” and to system concepts
practice? systems?
required for implementing the
previous topics. What is the status of current Following the 11 sections are three
research? appendices:
Topic 11, usable security, is different
from the others in its cross-cutting Future Directions Appendix A: Interdependencies among
nature. If taken seriously enough, it Topics
On what categories can we
can influence the success of almost all
the other topics. However, some sort subdivide the topics? Appendix B: Technology Transfer
of transcendent usability requirements What are the major research
need to be embedded pervasively in all gaps? Appendix C: List of Participants in the
the other topics. What are some exemplary Roadmap Development
problems for R&D on this topic?
Each of the 11 sections follows a
What are the challenges that
similar format. To get a full picture of
must be addressed?
the problem, where we are, and where
we need to go, we ask the following What approaches might be
questions: desirable?
References
[IRC2005] INFOSEC Research Council Hard Problem List, November 2005
http://www.cyber.st.dhs.gov/docs/IRC_Hard_Problem_List.pdf.
[USAF-SAB07] United States Air Force Scientific Advisory Board, Report on Implications of Cyber Warfare. Volume 1:
Executive Summary and Annotated Brief; Volume 2: Final Report, August 2007. For Official Use Only.
Additional background documents (including the two most recent National Research Council study reports on cybersecurity)
can be found online. (http://www.cyber.st.dhs.gov/documents.html).
viii
Acknowledgements
Acknowledgements
The content of this research roadmap was developed over the course of 15 months
that included three workshops, two phone sessions for each topic, and numer-
ous editing activities by the participants. Appendix C lists all the participants.
The Cyber Security program of the Department of Homeland Security (DHS)
Science and Technology (S&T) Directorate would like to express its appre-
ciation for the considerable amount of time they dedicated to this effort.
DHS S&T would also like to acknowledge the support provided by the staff of SRI
International in Menlo Park, CA, and Washington, DC. SRI is under contract with
DHS S&T to provide technical, management, and subject matter expert support for
the DHS S&T Cyber Security program. Those involved in this effort include Gary
Bridges, Steve Dawson, Drew Dean, Jeremy Epstein, Pat Lincoln, Ulf Lindqvist,
Jenny McNeill, Peter Neumann, Robin Roy, Zach Tudor, and Alfonso Valdes.
Of particular note is the work of Jenny McNeill and Peter Neumann. Jenny
has been responsible for the organization of each of the workshops and phone
sessions and has worked with SRI staff members Klaus Krause, Roxanne Jones,
and Ascencion Villanueva to produce the final document. Peter Neumann
has been relentless in his efforts to ensure that this research roadmap rep-
resents the real needs of the community and has worked with roadmap
participants and government sponsors to produce a high-quality product.
ix
Current Hard Problems in INFOSEC Research
1. Scalable Trustworthy Systems
BACKGROUND
1
Growing interconnectedness among follows: (1) trustworthiness, (2) com- computing base that would provide a
existing systems results, in effect, in posability, and (3) scalability. Thus, the suitable foundation for such computing.
new composite systems at increasingly challenge addressed here is threefold: However, this assumption has not been
large scales. Existing hardware, operat- (a) to provide a sound basis for compos- justified. In the future, we must be able
ing system, networking, and application ability that can scale to the development to develop scalable trustworthy systems
architectures do not adequately account of large and complex trustworthy effectively.
for combined requirements for security, systems; (b) to stimulate the develop-
performance, and usability—confound- ment of the components, analysis tools, Who are the potential
ing attempts to build trustworthy and testbeds required for that effort;
beneficiaries? What are their
systems on them. As a result, today the and (c) to ensure that trustworthiness
respective needs?
security of a system of systems may be evaluations themselves can be composed.
drastically less than that of most of its Large organizations in all sectors—for
components. What are the potential example, government, military, com-
mercial, financial, and energy—suffer
threats?
In certain cases, it may be possible the consequences of using large-scale
to build systems that are more trust- Threats to a system in operation include computing systems whose trustworthi-
worthy than some (or even most) everything that can prevent critical appli- ness either is not assured or is potentially
of their components—for example, cations from satisfying their intended compromised because of costs that
through constructive system design and requirements, including insider and out- outweigh the perceived benefits. All
meticulous attention to good software sider misuse, malware and other system stakeholders have requirements for
engineering practices. Techniques for subversions, software flaws, hardware confidentiality, integrity, and availabil-
building more trustworthy systems out malfunctions, human failures, physical ity in their computing infrastructures,
of less trustworthy components have damage, and environmental disruptions. although the relative importance of
long been known and used in practice Indeed, systems sometimes fail without these requirements varies by application.
(e.g., summarized in [Neu2004], in the any external provocation, as a result Achieving scalability and evolvability of
context of composability). For example, of design flaws, implementation bugs, systems without compromising trust-
error-correcting codes can overcome misconfiguration, and system aging. worthiness is a major need. Typical
unreliable communications and storage Additional threats arise in the system customers include the following:
media, and encryption can be used to acquisition and code distribution pro-
Large-system developers (e.g., of
increase confidentiality and integrity cesses. Serious security problems have
despite insecure communication chan- also resulted from discarded or stolen operating systems, database
nels. These techniques are incomplete by systems. For large-scale systems consist- management systems, national
themselves and generally ignore many ing of many independent installations infrastructures such as the power
security threats. They typically depend (such as the Domain Name System, grid)
on the existence of some combination DNS), security updates must reach and Application developers
of trustworthy developers, trustwor- be installed in all relevant components Microelectronics developers
thy systems, trustworthy users, and throughout the entire life cycle of the
System integrators
trustworthy administrators, and their systems. This scope of updating has proven
trustworthy embedding in those systems. to be difficult to achieve. Large- and small-scale users
Purveyors of potential exemplar
The primary focus of this topic area is Critical systems and their operating envi- applications for scalable
scalability that preserves and enhances ronments must be trustworthy despite a trustworthiness
trustworthiness in real systems. The per- very wide range of adversities and adver-
ceived order of importance for research saries. Historically, many system uses Several types of systems suggest the
and development in this topic area is as assumed the existence of a trustworthy importance of being able to develop
Development Unprincipled systems, Principles not Built-in assured Fewer flaws and risks;
methodologies unsafe languages, sloppy experientially scalably composable Simplified evaluations
and software programming practices demonstrated; Good trustworthiness
engineering programming language
theory widely ignored
Analytic tools Ad-hoc, piecemeal tools with Tools need sounder bases Rigorously based Eliminating many flaws
limited usefulness composable tools
Whole-system Impossible today for large Top-to-bottom, end-to- Formal methods, Scalable incremental
evaluations systems end analyses needed hierarchical staged evaluations
reasoning
Operational Enormous burdens on User and administrator Dynamic self-diagnosis Simplified, scalable
practices administrators usability are often ignored and self-healing operational management
What are the challenges that of high assurance information technol- could compromise the trustworthi-
must be addressed? ogy. Time-consuming evaluations of ness of the entire system. Designing
trustworthy systems today create long complex secure systems from the ground
The absence of sound systemwide delays when compared with conven- up is an exceptionally hard problem,
architectures designed for trustworthi- tional system developments with weaker particularly since large systems may
ness and the relatively large costs of evaluations. Consequently, development have catastrophic flaws in their design
full verification and validation (V&V) of trustworthy systems can be expected and implementation that are not dis-
have kept any secure computing base to take longer than is typically planned covered until late in development, or
from economically providing the req- for COTS systems. In addition, the even after deployment. Catastrophic
uisite assurance and functionality. (The performance of trustworthy systems software flaws may occur even in just
sole exception is provided by “high- typically lags the performance of COTS a few lines of mission-critical code,
consequence” government applications, systems with comparable functions. and are almost inevitable in the tens
in which cost is a secondary concern of millions of lines of code in today’s
to national security.) This situation is One of the most pressing challenges systems. Given the relatively minuscule
exacerbated by the scale and complexity involves designing system architectures size of programs and systems that have
often needed to provide required func- that minimize how much of the system been extensively verified and the huge
tionality. In addition, the length of must be trustworthy—i.e., minimiz- size of modern systems and applica-
the evaluation process can exceed the ing the size and extent of the trusted tions, scaling up formal approaches to
time available for patches and system computing base (TCB). In contrast, for production and verification of bug-free
upgrades and retarded the incorporation a poorly designed system, any failure systems seems like a Herculean task. Yet,
Vision: Make the development of trustworthy systems of systems (TSoS) practical; ensure that even very large and complex systems
can be built with predictable scalability and demonstrable trustworthiness, using well-understood composable architectures and well-
designed, soundly developed, assuredly trustworthy components.
Challenges: Most of today’s systems are built out of untrustworthy legacy systems using inadequate architectures, development
practices, and tools. We lack appropriate theory, metrics of trustworthiness and scalability, sound composable architectures, synthesis and
analysis tools, and trustworthy building blocks.
Goals: Sound foundations and supporting tools that can relate mechanisms to policies, attacks to mechanisms, and systems to
requirements, enabling facile development of composable TSoS systematically enhancing trustworthiness (i.e., making them more
trustworthy than their weakest components); documented TSoS developments, from specifications to prototypes to deployed systems.
MILESTONES
Tech transfer: Publish composition methodologies for developing TSoS with mix-and-match components. Release open-source tools
for creating, configuring, and maintaining TSoS. Release open-source composable, trustworthy components. Publish successful, well-
documented TSoS developments. Develop profitable business models for public-private TSoS development partnerships for critical
applications, and pursue them in selected areas.
formally inspired approaches may be components is almost certainly an even of the executable code has not been
more promising than any of the less harder problem. compromised and (b) that the code
formal approaches attempted to date. resides in memory in a manner that it
In addition, considerable progress is As one example, securing the bootload can be neither read nor altered, but only
being made in analyzing system behav- process would be very valuable, but the executed. Firmware residing in ROM,
ior across multiple layers of abstraction. underlying general principle is that every when ROM updating is cryptographi-
On the other hand, designing complex module of executable software within cally protected for integrity, meets these
trustworthy systems and “compromise- a system should be backed by a chain criteria. Software that is cryptographi-
resilient” systems on top of insecure of trust, assuring (a) that the integrity cally protected for integrity, validated
Finally, efficiently creating provably More basic, higher-risk, game-changing To this end, many resources will be
trustworthy systems will require R&D broadly includes various topics essential. The most precious resource is
creation of secure but flexible com- under the umbrella of composability, undoubtedly the diverse collection of
ponents, and theories and tools for because it is believed that only effec- people who could contribute. Also vital
combining them. Without a secure tive composability for trustworthiness are suitable languages for requirements,
computing foundation, developers will can achieve true scalability (just as specification, programming, and so on,
References
[Can2001] Ran Canetti. Universally composable security: A new paradigm for cryptographic protocols
(http://eprint.iacr.org/2000/067), 2005. An extended version of the paper from the 42nd
Symposium on Foundations of Computer Science (FOCS’01) began a series of papers
applying the notion of universal composability to cryptography. Much can be learned
from this work regarding the more general problems of system composability.
[Neu1995] Peter G. Neumann. Computer-Related Risks, Addison-Wesley/ACM Press, New York, 1995. See also an
annotated index to online sources for the incidents noted here, as well as many more recent cases
(http://www.csl.sri.com/neumann/illustrative.html).
[Sal+2009] J.H. Saltzer and F. Kaashoek. Principles of computer design. Morgan Kauffman, 2009. (Chapters
1-6; Chapters 7-11 are online at: http://ocw.mit.edu/ans7870/resources/system/index.htm).
BACKGROUND
Along with the systems- and component-level metrics that are discussed elsewhere
in this document and the technology-specific metrics that are continuing to emerge
with new technologies year after year, it is essential to have a macro-level view of
security within an organization. A successful research program in metrics should
define a security-relevant science of measurement. The goals should be to develop
metrics to allow us to answer questions such as the following:
13
Enterprise-level metrics (ELMs) address environment. Note that this definition quantifiable, feasible to measure, and
the security posture of an organization incorporates a specification of system repeatable. They provide relevant trends
and complement the component-level objectives and a specification of the over time and are useful in tracking
metrics examined elsewhere in the system environment, which would performance and directing resources
roadmap topics. “Enterprise” is a term include some notion of a threat model. to initiate performance improvement
that encompasses a wide range. It could Although this type of probability metric actions.” [http://www.itl.nist.gov/lab/
in principle apply to the Internet as a has been computed for system reliability bulletns/bltnaug03.htm]
whole, but realistically it is intended and for certain system risk assessments,
here to scale in scope from a large cor- the potential accuracy of such assess- Most organizations view the answers to
poration or department of the federal ments with respect to security seems the questions listed above in the short
government down to the small office/ to be extremely questionable, given the term from a financial mind-set and
home office (SOHO). For our purposes, rapidly changing threat environment for attempt to make cost-benefit trade-
an enterprise has a centralized decision IT systems. For example, a presumed off analyses. However, in the absence
making authority to ensure the use of high probability of meeting security of good metrics, it is unclear whether
ELMs to rationally select among alterna- objectives essentially goes to zero at the those analyses are addressing the right
tives to improve the security posture of instant security exploits are announced problems. Decisions resulting from
that enterprise. ELMs can support deci- and immediately perpetrated. such analyses will frequently be detri-
sions such as whether adoption of one mental to making significant security
technology or another might improve Security metrics are difficult to develop improvements in the long term and
enterprise security. ELMs also provide because they typically try to measure thus eventually require costly new
the basis for accurate situational aware- the absence of something negative (e.g., developments.
ness of the enterprise’s security posture. lack of any unknown vulnerabilities in
systems and lack of adversary capabilities What are the potential
In this discussion, we define metrics rel- to exploit both known and unknown
threats?
evant to systems and networking within vulnerabilities). This task is difficult
an enterprise, and consider composing because there are always unknowns in Lack of effective ELMs leaves one in the
host-level and other lower-layer mea- the system and the landscape is dynamic dark about cyberthreats in general. With
surements up to an enterprise level. In and adversarial. We need better defini- respect to enterprises as a whole, cyber-
other words, the goals of ELMs are to tions of the environment and attacker security has been without meaningful
understand the security of a large-scale models to guide risk-based determi- measurements and metrics throughout
system—enabling us to understand nation. These are difficult areas, but the history of information technol-
enterprise security as a whole, with a progress is achievable. ogy. (Some success has been achieved
goal of using these measurements to with specific attributes at the compo-
guide rational investments in security. The following definition from NIST nent level.) This lack seriously impedes
If these ELM goals are met, then exten- may provide useful insights. the ability to make enterprise-wide
sions to other related cases, such as informed decisions of how to effectively
Internet service providers (ISPs) and “IT security metrics provide a practical avoid or control innumerable known
their customers, should be feasible. approach to measuring information and unknown threats and risks at every
security. Evaluating security at the system stage of development and operation.
Security itself is typically poorly defined level, IT security metrics are tools that
in real systems, or is merely implicit. facilitate decision making and account- Who are the potential
One view might be to define it as the ability through collection, analysis, and
beneficiaries? What are their
probability that a system under attack reporting of relevant performance data.
respective needs?
will meet its specified objectives for a Based on IT security performance goals
specified period of time in a specified and objectives, IT security metrics are In short, everyone who is affected by an
14 ENTERPRISE-LEVEL METRICS
automated IT system has the potential caused by cyber attacks, which might short-term economic losses caused by
to benefit from better security metrics, be enhanced with the existence of mean- system outages. Potential beneficiaries,
especially at the enterprise level. Spon- ingful metrics. However, that market challenges, and needs are summarized
sors of security R&D require such is perhaps undercut not by the lack in Table 2.1.
metrics to measure progress. With such of suitable metrics, but more by the
metrics, decision makers, acquisition prevalence of insecure systems and their What is the current state of
managers and investors in security tech- exploitations and by a historical lack of
the practice?
nology could make a better business case consistent actuarial data.
for such technology, and guide intel- At present, the practice of measuring
ligent investment in such technology. Metrics defined relative to a mission security is very ad hoc. Many of the
This demand of course would guide threat model are necessary to understand processes for measurement and metric
the market for development of mea- the components of risk, to make risk selection are mostly or completely sub-
surably more secure systems. Metrics calculations, and to improve decision jective or procedural, as in evaluation
can be applied not just to technol- making in response to perceived risk. of compliance with Sarbanes-Oxley,
ogy, but to practices as well, and can A risk model must incorporate threat HIPAA, and so on. New approaches
provide management with an incentive information, the value of the enterprise are introduced continually as the old
structure oriented toward security per- information being protected, poten- approaches prove to be ineffective. There
formance improvement. Robust metrics tial consequences of system failure, are measurements such as size and scope
would enhance the certification and operational practices, and technology. of botnets, number of infections in a
accreditation process, moving toward More specifically, risk assessment needs a set of networks, number of break-ins,
quantitative rather than qualitative pro- threat model (encompassing intent and antivirus detection rates over time, and
cesses. Metrics also can be used to assess capabilities), a model of actual protective numbers of warrants served, crimi-
the relative security implications of measures, a model of the probability that nal convictions obtained, and national
alternative security measures, practices, the adversary will defeat those protective security letters issued (enforcement).
or policies. measures, and identification of the con- These are not related to fundamental
sequences of concern or adversary goals. characteristics of systems, but are more
Administrators require metrics to guide These consequences of concern are typi- about what can be measured about
the development of optimal network cally specific to each enterprise, although adversaries. Examples include websites
configurations that explicitly consider many commonalities exist. For critical that attempt to categorize the current
security, usability, cost, and perfor- infrastructures, loss of system availability state of the Internet’s health, the current
mance. There seems to be a potential may be the key concern. For commercial state of virus infections world wide, or
market in insurance and underwriting enterprises, loss of proprietary infor- the number and sizes of botnets cur-
for predicting and reducing damages mation may be a greater concern than rently active.
ENTERPRISE-LEVEL METRICS 15
Numerous initiatives and projects are on some sort of thermometer). Measures of effectiveness. The
being undertaken to improve or develop However, password strength is a Institute for Defense Analyses
metrics for all or a specific portion of the rather vacuous concept in systems (IDA) developed a methodology
security domain. Included in these are with inherently weak security in for determining the effectiveness
the following: other areas. of cybersecurity controls based on
Security implementation its well-used and -documented
Several government documents methodology for determining the
and efforts (for example, NIST metrics, which might be used
to assess how many systems in effectiveness of physical security
SP800-55) that describe an controls. Using a modified
approach to defining and an enterprise install a newly
announced patch, and how Delphi technique, the measures
implementing IT security of effectiveness of various
metrics. Although some of the quickly.
components and configurations
measures and metrics are useful, Initiatives in security processes, were determined, which then
they are not sufficient to answer which might define metrics allowed for a security “ranking”
the security questions identified relating to the adoption of those of the potential effectiveness
earlier in this section. processes and require extensive of various architectures and
documentation. However, operating modes against different
Methods that assess security
such approaches typically are classes of adversaries [IDA2006].
based on system complexity
about process and not actual
(code complexity, number Ideal-based metrics. The Idaho
performance improvement with
of entry points, etc.). These National Laboratory (INL) took
respect to security.
may give some indication of a vastly different approach to
vulnerability, but in the absence This section focuses on metrics for developing metrics. It chose to
of data on attack success rates or cybersecurity issues. However, it is also specify several best-case outcomes
the efficacy of mitigation efforts, useful to consider existing metrics and
of security and then attempt to
these methods prove very little. design techniques for physical security
develop real-world measures of
systems, and the known limitations
Red Teaming, which provides those “ideals.” The resulting set of
of those techniques. This informa-
some measure of adversary work 10 system measurements covering
tion would help advance cybersecurity
factor and is currently done 7 ideals is being tested in the
research. It will also be required as
in security assessments and field to determine how well they
our logical and physical cybersecurity
penetration testing. One can can predict actual network or
systems become ever more intertwined
apply penetration testing, using system security performance
and interdependent. Similarly, tech-
a variety of available tools and/ [McQ2008].
niques for financial risk management
or hiring a number of firms that Goal-oriented metrics. Used
may also be applicable to cybersecurity.
provide this as a service. For primarily in the software
example, this can provide metrics development domain, the
on adversary work factor and What is the status of current goal-oriented paradigm seeks to
residual vulnerabilities before and research? establish explicit measurement
after implementation of a security goals, define sets of questions
There are initiatives aimed at developing
plan. that relate to achieving the goals,
new paradigms for identifying measures
Heuristic approaches to provide and metrics. Some of them attempt to and identify metrics that help to
metrics in a number of security- apply tools and techniques from other answer those questions.
related areas. For example, disciplines; others attempt to approach Quality of Protection (QoP).
systems often report a measure the problem from new directions. These This is a recent approach that
of “password strength” (usually initiatives include the following: is in early stages of maturity. It
16
ENTERPRISE-LEVEL METRICS
has been the subject of several answering questions such as the degree Analysis
workshops but is still relatively to which one system is more secure than Analysis focuses on determining how
qualitative [QoP2008]. another or the degree to which adop- effectively the metrics describe and
Adversary-based metrics. MIT tion of security technology or practice predict the performance of the system.
Lincoln Laboratory chose to makes a system more secure. However, The prediction should include both
explore the feasibility and effort as noted above, these measurements are current and postulated adversary
required for an attacker to break relative to assumed models for adver- capabilities. There has been relatively
into network components, by sary capabilities and goals, and to our little work on enterprise-level analy-
examining reachability of those knowledge of our systems’ vulnera- ses, because a foundation of credible
components and vulnerabilities bilities—and therefore are potentially metrics and foundational approaches
present or hypothesized to be limited by shortcomings in the models, for deriving enterprise-level evaluations
present. It and others have built requirements, knowledge, assumptions, from more local evaluations have been
tools employing attack graphs to and other factors. lacking.
model the security of networks. Composition
While this section is focused on enter-
prise-level metrics (ELMs), we must Since security properties are often best
FUTURE DIRECTIONS also consider definitions of metrics for viewed as total-system or enterprise-level
interconnected infrastructure systems, emergent properties, research is required
as well as for non-enterprise devices. in the composability of lower-level
On what categories can we
We must also anticipate the nature of metrics (for components and subsys-
subdivide this topic?
the enterprise of the future; for example, tems) to derive higher-level metrics
For the purposes of this section, we technology trends imply that we should for the entire system. This “compos-
divide the topic of enterprise-level consider smart phones as part of the able metrics” issue is a key concern for
metrics into five categories: definition, enterprise. Infrastructure systems may developing scalable trustworthy systems.
collection, analysis, composition, and be thought of as a particular class of In addition, the composability of enter-
adoption. enterprise-level systems. However, the prise-level metrics into meta-enterprise
interrelationships among the differ- metrics and the composability of the
Definition ent infrastructures also suggest that resulting evaluations present challenges
Definition identifies and develops the we must eventually be able to consider for the long-term future.
models and measures to create a set meta-enterprises.
of security primitives (e.g., for confi- Adoption
dentiality, integrity, availability, and Collection Adoption refers to those activities that
others). NIST SP 800-55 provides a Collection requirements may inspire transform ELM results into a useful
useful framework for metrics definition. new research in hardware and software form (such as a measurement paradigm
This publication proposes development for systems that enable the collection of or methodology) that can be broadly
of metrics along the dimensions of data through meaningful metrics, ideally used—taking systems, processes, organi-
implementation (of a security policy), in ways that cannot be compromised by zational constraints, and human factors
effectiveness/efficiency, and mission adversaries. This includes conditioning into account. Monetary and financial
impact. the data via normalization, categoriza- considerations may suggest adoption of
tion, prioritization, and valuation. It metrics such as the number of records
Ideally, metrics would be defined to might also include system developments in a customer database and a cost per
quantify security, but such definitions with built-in auditability and embedded record if those records are disclosed.
have been difficult to achieve in prac- forensics support, as well as other topic We may also consider financial metrics
tice. At the basic level, we would like areas, such as malware defense and situ- retrospectively (the cost of a particular
to quantify the security of systems, ational understanding. compromise, in terms of direct loss,
ENTERPRISE-LEVEL METRICS 17
reputation, remediation costs, etc.). This Composition models of metrics as system survivability under threats
retrospection would be useful for system to determine enterprise values that are not addressed, human safety,
designers and for the insurance under- from subsystem metrics and so on.
writing concept mentioned previously. Scalability of sets of metrics
Adapting approaches to metrics from
Developing or identifying metric
What are the major research other disciplines is appropriate, but the
hierarchies result is not complete and often not
gaps?
Measures and metrics for security sufficiently applicable (as in the case of
In spite of considerable efforts in primitives probability metrics for component and
the past, we do not have any univer- Appropriate uses of metrics system reliability). We should consider
sally agreed-upon methodologies to (operations, evaluation, risk connections with other fields, while
address the fundamental question of management, decision making) remaining aware that their techniques
how to quantify system security. At a may not be directly applicable to cyber-
Ability to measure operational
minimum, an evaluation methodol- security because of intelligent adversaries
security values
ogy would support hypothesis testing, and the fluid nature of the attack space.
benchmarking, and adversary models. Measuring human-system
Hypothesis testing of various degrees of interaction (HSI) Many disciplines (such as financial
formality, from simple engagements to Tools to enhance and automate metrics and risk management practices;
formal, well-instrumented experiments, the above areas in large balanced scorecard, six-sigma, and insur-
is needed to determine the viability of enterprises ance models; complexity theory; and
proposed security measures. Bench- data mining) operate in environments of
marking is needed to establish a system decision making under uncertainty, but
effectiveness baseline, which permits the
What R&D is evolutionary, most have proven methods to determine
progress of the system to be tracked as and what is more basic, risk. For example, the field of finance
changes are made and the threat envi- higher risk, game changing? has various metrics that help decision
ronment evolves. Finally, evaluation Composability advances (for multi- makers understand what is transpiring
must include well-developed adver- ple metrics) could be game-changing in their organizations. Such metrics
sary models that predict how a specific advances. Hierarchical composition can provide insight into liquidity, asset
adversary might act in a given context of metrics should support frameworks management, debt management, prof-
as systems react to that adversary’s intru- such as argument trees and security cases itability, and market value of a firm.
sions or other exploits. (analogous to safety cases in complex Capital budgeting tools determining
mechanical systems, such as aircraft). net present-values and internal rates of
What are some exemplary return allow insights into the returns
Identifying comprehensive metrics, or that can be expected from investments
problems for R&D on this
a different set of measurement dimen- in different projects. In addition, there
topic?
sions, might provide a leap forward. The are decision-making approaches, such
The range of requirements for metrics in well-known and well-used confidential- as the Capital Asset Pricing Model
security is broad. R&D may be focused ity, integrity, availability (CIA) model and options pricing models, that link
in any of the following areas: is good for discussing security, but may risk and return to provide a perspec-
not be easily or directly measured in tive of the entire financial portfolio
Choosing appropriate metrics large enterprises. It is also inherently under a wide range of potential market
Methods for validating metrics incomplete. For example, it ignores conditions. These methodologies have
requirements relating to accountability, demonstrated some usefulness and have
Methods for metric computation auditing, real-time monitoring, and been applied across industries to support
and collection other aspects of trustworthiness, such decision making. A possible analog for
18 ENTERPRISE-LEVEL METRICS
IT security would be sound systems Economic or market analysis of metrics and evaluation methodologies
development frameworks that support adversary actions may provide for security of the information domain
enterprise-level views of an organiza- an indirect metric for security with the metrics and evaluation meth-
tion’s security. Research is needed to effectiveness. If the cost to odologies for physical, cognitive, and
identify system design elements that exploit a vulnerability on a social domains.
enable meaningful metrics definition critical and widely used server
and data collection. Research is also system increases significantly, we
might surmise that the system Resources
needed on issues in collection logistics,
such as the cost of collection and its is becoming more secure over Industry trends such as exposure to data
impact on the metric being used (e.g., time or that the system has breaches are leading to the development
whether the collection compromises become more valuable to its of tools to measure the effectiveness
adversaries. This approach can be
security). of system implementations. Industry
confounded by, for example, the
mandates and government regulations
monetary assets accessible to the
Research on metrics related to adversary such as the Federal Information Secu-
adversary by compromising the
behaviors and capabilities needs to be rity Management Act (FISMA) and
service. (A very secure system not
conducted in several key areas, such as Sarbanes-Oxley require the govern-
widely used in an attractive target
the following: space may discourage a market ment and private-sector firms to become
for high-priced vulnerabilities.) accountable in the area of IT security.
The extent of an adversary’s These factors will lead industry and
opportunity to affect hardware It is also not obvious that this
is an enterprise-level metric. government to seek solutions for the
and software needs to be studied. improvement of security metrics.
Nonetheless, the assembled
This may lead to research into,
experts considered market
for example, global supply-chain Government investment in R&D is still
analysis a novel and interesting
metrics that account for potential avenue of research. required to address the foundational
adversarial influence during questions that have been discussed,
acquisition, update, and remote Metrics relating to the impact of
such as adversary capabilities and threat
management cycles. cybersecurity recommendations
measurements.
on public- and private-sector
Metrics in the broad area of enterprise-level systems.
adversary work factor have Measures of success
been considered for some time.
The simple example is the Metrics can guide root-cause analysis in The ability to accurately and confi-
increase in the recommended the case of security incidents. Research dently predict the security performance
length of cryptographic keys using existing events should compile a of a component, network, or enter-
as computational power has list of metrics that might have avoided prise is the ultimate measure of success
increased. This work should the incident if they had been known for metrics R&D. Interim milestones
continue, but there is a question before the incident. include better inputs for risk calculation
as to the repeatability of the and security investment decisions. The
A stretch objective in the long term is extent to which the evaluation of local
obtained metric.
the development of metrics and data metrics (e.g., see the other sections)
Research related to an adversary’s collection schemes that can provide can be combined into enterprise-level
propensity to attempt a actuarial-quality data with respect to metrics would be a significant measure
particular attack, in response security. This is needed for a robust of success.
to a defensive posture adopted market for insurance against cybersecu-
by the enterprise, needs to be rity-related risks. Another long-range
conducted. stretch goal would be to unify the
ENTERPRISE-LEVEL METRICS 19
What needs to be in place for assessment of “time to compromise” To what extent can we test
test and evaluation? experimental metrics, possibly consider- real systems?
ing systems that are identical except for
Testbeds and tools within the testbeds some security enhancement. An enterprise is a testbed of sorts to
are needed to evaluate the descriptive glean insights on usability, organiza-
and predictive value and effectiveness of Evaluation and experimentation are tional behavior, and response to security
proposed measures and models, particu- essential to measure something that is practices. Much of the initial collection
larly for potentially destructive events. relevant to security. Evaluation method- and verification must be done on real
Repositories of measurement “baselines” ology goes hand in hand with metrics, systems to ensure applicability of the
to compare new metric methods and and tools that accurately measure and measurements and derived metrics.
models will also be required. Virtualiza- do not distort quantities of interest also
tion and honeynet environments permit have direct influence on metrics.
References
[And2008] R. Anderson. Security Engineering: A Guide to Building Dependable Distributed Systems. Wiley, Indianapolis,
Indiana, 2008.
[Avi+2004] A. Avizienis, J.-C. Laprie, B. Randell, and C. Landwehr. Basic concepts and taxonomy of dependable and
secure computing. IEEE Transactions on Dependable and Secure Computing, 1(1):11-33, January-March 2004.
[Che2006] E. Chew, A. Clay, J. Hash, N. Bartol, and A. Brown. Guide for Developing Performance Metrics for Information
Security. NIST Special Publication 800-80, National Institute of Standards and Technology, Gaithersburg,
Maryland, May 2006.
[CRA2003] Four Grand Challenges in Trustworthy Computing: Second in a Series of Conferences on Grand Research
Challenges in Computer Science and Engineering. Computing Research Association, Washington, D.C.,
2006 (http://www.cra.org/reports/trustworthy.computing.pdf ).
[Jaq2007] A. Jaquith. Security Metrics. Addison Wesley Professional, Upper Saddle River, New Jersey, 2007.
[IDA2006] Institute for Defense Analysis. National Comparative Risk Assessment Pilot Project. Draft Final,
September 2006, IDA Document D-3309.
[McQ2008] M.A. McQueen, W.F. Boyer, S. McBride, M. Farrar, and Z. Tudor. Measurable control system security through
ideal driven technical metrics. In Proceedings of the SCADA Scientific Security Symposium, January 2008.
[Met2008] Metricon 3.0, July 29, 2008, with copious URLs (http://www.securitymetrics.org/content/Wiki.
jsp?page=Metricon3.0).
[NIS2009] Information Security Training Requirements: A Role- and Performance-Based Model. NIST Special
Publication 800-16 Revision 1, National Institute of Standards and Technology, Gaithersburg, Maryland,
March 20, 2009 (http://csrc.nist.gov/publications/PubsDrafts.html).
20
ENTERPRISE-LEVEL METRICS
[QoP2008] 4th Workshop on Quality of Protection (Workshop co-located with CCS-2008), October 2008 (http://
qop-workshop.org/)
[Swa+2003] M. Swanson, N. Bartol, J. Sabato, J. Hash, and L. Graffo. Security Metrics Guide for Information Technology
Systems. NIST Special Publication 800-55, National Institute of Standards and Technology, Gaithersburg,
Maryland, July 2003.
21
Current Hard Problems in INFOSEC Research
3. System Evaluation Life Cycle
BACKGROUND
22
Developing system evaluation Such understanding is needed to evalu- of security products (because they need
processes whereby incremental ate the likelihood of human acceptance reliable means to evaluate what they
changes can be tracked and of proposed security artifacts and to buy); the creators of these products, such
rapidly reevaluated without simulate human actions during evalu- as software and hardware companies;
having to repeat the entire ation (e.g., browsing patterns during and researchers (because they need to
process. evaluation of a web server defense). measure their success). Having effective
evaluation methods opens the door to
In each case, independent assessment What are the potential the possibility of standardization of
of a product could reduce reliance on security and to formation of attestation
threats?
vendor claims that might mask serious agencies that independently evaluate
problems. On the other hand, embed- Threats against information and infor- and rank security products. The poten-
ded self-assurance techniques (such as mation systems are at the heart of the tial beneficiaries, challenges, and needs
proof-carrying code) could also be used need for robust system evaluation. In are summarized in Table 3.1.
to demonstrate that certain properties addition to the threats to operational
were satisfied. systems, adversaries have the potential to What is the current state of
affect the security of artifacts at numer-
the practice?
Systematic, realistic, easy-to-use and ous points within the development life
standardized evaluation methods are cycle. The complexity of systems, modi- Evaluation of security artifacts is ad hoc.
needed to objectively quantify perfor- fications, constant changes to supply Current methodologies, such as these
mance of any security artifacts and the chains, remote upgrades and patches, discussed in NIST SP800-64 (Security
security of environments where these and other factors give rise to numerous Considerations in the System Development
artifacts are to be deployed, before and new threat vectors. Life Cycle) [NIS2008] and Microsoft’s
after deployment, as well as the per- The Security Development Life cycle
formance of proposed solutions. The Who are the potential [How+2006], merely reorder or reem-
evaluation techniques should objectively phasize many of the tools and methods
beneficiaries? What are their
quantify security posture throughout the that have been unsuccessful in creating
respective needs?
critical system life cycle. This evaluation security development paradigms. There
should support research, development, With regard to the system life cycle, are neither standards nor metrics for
and operational decisions, and maximize system architects, engineers, develop- security evaluation. Product developers
the impact of the investment. ers, and evaluators will benefit from and vendors evaluate their merchandise
enhanced methods of evaluation. Bene- in-house, before release, via different
Finally, evaluation must occur in a ficiaries of improved security evaluations tests that are not disclosed to the public.
realistic environment. The research range from large and small enterprises Often, real evaluation takes place in
community lacks data about realis- to end users of systems. Although customer environments by product
tic adversarial behavior, including the beneficiaries’ needs are generally the vendors collecting periodic statistics
tactics and techniques that adversar- same—to prevent security incidents and about threats detected and prevented
ies use to disrupt and deny normal to respond quickly to those that evade during live operation. Although this is
operations, as well as normal system prevention and minimize damage, while the ultimate measure of success—how a
use patterns and business relation- protecting privacy—environments that product performs in the real world—it
ships, to create a realistic environment they seek to protect may be very different, does not offer security guarantees to
for evaluation that resembles current as are their needs for reliability, correct- customers prior to purchase. There have
environments in which systems are ness of operation, and confidentiality. been many incidents when known secu-
deployed. We also lack understanding Direct beneficiaries of better evaluation rity devices have failed (e.g., the Witty
of human behavior as users interact with methods are system developers, system worm infected security products from a
the system and with security artifacts. users and administrators; the customers well-known security product vendor). In
24
SYSTEM EVALUATION LIFE CYCLE
Incorporate relevant (current and specifications. Concerns about that are ill-documented, are time-
anticipated) threats models in insider threats inside the dependent, and occur only when
the requirements phase so that development process also need to all of the subsystems have been
the final specification can be be addressed. integrated.
evaluated against those threats. Pursue verification that a system Conduct Red Team exercises in
Specify what constitutes secure is implemented in a way that a structured way on testbeds to
operation of systems and security claims can be tested. bring realism. Expand the Red
environments. Consider new programming Team concept to include all
Establish requirement languages, constraints on or phases of the life cycle.
specification languages that subsets of existing languages, Establish evolvable testbeds
express security properties, so and hardware design techniques that are easily upgradeable as
that automated code analysis that express security properties, technology, threat, and adversary
can be used to extract what the enforce mandatory access models change.
code means to do and what its controls, and specify interfaces,
Improve techniques for
assumptions are. so that automated code analysis
combined performance, usability,
Design can be used to extract what the
and security testing. This
Be able to share data with code means to do and what its
includes abnormal environments
adequate privacy, including data assumptions are.
(e.g., extreme temperatures) and
on attacks, and with emphasis on Testing operating conditions (e.g., misuse
economics of data sharing. Select and evaluate metrics for by insiders) that are relevant for
Develop a richer process to evaluation of trustworthiness security testing but may exceed
develop data used to validate requirements. the system’s intended range of
security claims. Select and use evaluation operation.
Develop frameworks for threat methods that are well suited to Deployment and Operations
prediction based on data about the anticipated ranges of threats Establish and use evaluation
current attacks and trends. and operational environments. methods that can compare actual
Develop simulations of (unusual Develop automated techniques operational measurements with
or unanticipated) system states for identifying all accessible design specifications to provide
that are critical for security, as system interfaces (intentional, feedback to all life cycle phases.
opposed to simulation of steady unintentional, and adversary- Develop methods to identify
states. induced) and system system, threat, or environment
Development and Implementation dependencies. For example, changes that require reevaluation
Pursue evaluation methods able exploitation of a buffer overflow to validate compliance with
to verify that an implementation might be considered a simple evolving security requirements.
example of an unintended system
follows requirements precisely Define and consistently deploy
interface.
and does not introduce anything certification and accreditation
not intended by requirements. Develop and apply automated methods that provide
If specifications exist, this can tools for testing all system realistic values regarding the
be done in two steps: verifying dependencies under a wide range trustworthiness of a system with
consistency of specifications of conditions. As an example, respect to its given requirements.
with requirements and then some adversaries may exploit Decommissioning
consistency of software with hardware-software interactions Develop end-of-life evaluation
26
SYSTEM EVALUATION LIFE CYCLE
Evolutionary, relatively short-term success and for security based on metrics for evaluation, including joint
R&D challenges include the following: the models of correct operation. design of realism criteria for evaluation
Developing benchmarks to environments.
Defining verifiable parametric
sets of requirements for standardize testing.
Government should help in mandat-
trustworthiness and improved Developing understanding about
ing, regulating, and promoting this
models for assessing advantages and limitations of collaboration, especially with regard
requirements. various evaluation methods to data sharing. Legal barriers to data
Devising methods to recreate (simulation, emulation, pilot sharing must be addressed. Some
realism in testbeds and deployment, model checking, industry sectors may be reluctant to
simulations while providing etc.) when related to specific share vulnerability data because of legal
flexible trade-offs between cost, threats. liability concerns. There may also be
scalability, and accuracy. (These Managing risky test privacy and customer relations concerns.
include better methods for environments (such as those An example would be data sharing by
designing experiments for large containing malware). common carriers where the shared data
testbeds). Developing better techniques for uniquely identify individual customers.
Developing methods and security testing across all domains The government should also provide
representations such as of conflict. more complete threat and adversary
abstraction models to describe capability models for use in developing
Developing integrated, cost-
threats, so that designers can evaluation and testing criteria.
effective methodologies and
develop detailed specifications. tools that systemically address
Other potential government activities
Developing user interfaces, tools, all of the above desiderata,
include the following:
and capabilities to allow complex including facilitation of scalable
evaluations to be conducted. trustworthiness (Section 1), Propose evaluation methods that
survivability (Section 7), are proven correct as national
Developing tool sets that
resistance to tampering and or international standards for
can grow with technology
other forms of insider misuse tech transfer. They also should
(e.g., 64‑bit words, IPv6).
by developers (Section 4), rapid be implemented in current
Creating better techniques for reevaluation after incremental popular simulations and testbeds.
testing combined performance, changes, and suitable uses of Industry should be encouraged
usability, and security. formal methods where most to use these methods, perhaps via
Developing understanding of usefully applicable—among market incentives.
how much realism matters and other needs. The potential Form attestation agencies that
what type of realism is possible utility of formal methods has would evaluate products on the
and useful. increased significantly in the past market, using evaluation methods
four decades and needs to be that are ready for tech transfer,
Long-term, high-risk R&D challenges
considered whenever it can be and rank those products publicly.
include the following:
demonstrably effective. Create a National CyberSecurity
Developing models of correct and Safety Board that would
operation for various network Resources
collect attack reports from
elements and networks at and Academia and industry should col- organizations and share them in
across all levels of protocol laborate to share data about traffic, a privacy-safe manner. The board
models. attacks, and network environments could also mandate sharing.
Developing metrics for attack and to jointly define standards and Another way is establishing
References
[Ade2008] S. Adee. The hunt for the kill switch. IEEE Spectrum, 45(5):32-37, May 2008
(http://www.spectrum.ieee.org/may08/6171).
[DSB2005] Defense Science Board Task Force on High Performance Microchip Supply, February 2005
(http://www.acq.osd.mil/dsb/reports/2005-02-HPMS_Report_Final.pdf ).
[How+2006] M. Howard and S. Lipner. The Security Development Life Cycle. Microsoft Press, Redmond, Washington, 2006.
[NIS2008] Security Considerations in the System Development Lhife Cycle. NIST Special Publication 800-64 Revision 2
(Draft), National Institute of Standards and Technology, Gaithersburg, Maryland, March 2008.
Trusted insiders are among the primary sources of many losses in the
commercial banking industry.
Well-publicized intelligence community moles, such as Aldrich Ames,
Robert Hanssen, and Jonathan Pollard, have caused enormous and
irreparable harm to national interests.
Many insiders involved in misuses were hired as system
administrators, became executives, or held other kinds of privileges
[Cap2008.1, Cap2008.2].
This section focuses on insider threats to cyber systems and presents a roadmap for
high-impact research that could aggressively curtail some aspects of this problem. At
a high level, opportunities exist to mitigate insider threats through aggressive profil-
ing and monitoring of users of critical systems, “fishbowling” suspects, “chaffing”
data and services users who are not entitled to access, and finally “quarantining”
confirmed malevolent actors to contain damage and leaks while collecting action-
able counter-intelligence and legally acceptable evidence.
There are many proposed definitions of the insider threat. For the purposes of this
discussion, an insider threat is one that is attributable to individuals who abuse
granted privileges. The scope of consideration here includes individuals masquerad-
ing as other individuals, traitors abusing their own privileges, and innocents fooled
by malevolent entities into taking adverse actions. Inadvertent and intentional
misuse by privileged users are both within the scope of the definition. Although an
insider can have software and hardware acting on his or her behalf, it is the indi-
vidual’s actions that are of primary concern here. Software proxies and other forms
of malevolent software or hardware—that is, electronic insiders—are considered in
Section 5 on combatting malware and botnets.
The insider threat is context dependent in time and space. It is potentially relevant
at each layer of abstraction. For example, a user may be a physical insider or a
logical insider, or both. The threat model must be policy driven, in that no one
description will fit all situations.
Unlike unauthorized outsiders and insiders who must overcome security controls to
access system resources, authorized insiders have legitimate and (depending on their
positions) minimally constrained access to computing resources. In addition, highly
29
trusted insiders who design, maintain, [Noo+2008], integrity, availability, and brought forward by the research commu-
or manage critical information systems total system survivability are of highest nity are multilevel security (MLS), an
are of particular concern because they priority and can be compromised by example of mandatory access controls
possess the skills and access necessary to insiders. (MAC) that prevents highly sensitive
engage in serious abuse or harm. Typical information from being accessed by less
trusted insiders are system administra- Beneficiary needs may include tools privileged users. Some work has also
tors, system programmers, and security and techniques to prevent and detect been done on multilevel integrity (MLI
administrators, although ordinary users malicious insider activity throughout [Bib1977]), which prevents less trusted
may have or acquire those privileges the entire system life cycle, approaches entities from affecting more trusted
(sometimes as a result of design flaws to minimize the negative impact of entities. However, these are typically
and implementation bugs). Thus, there malicious insider actions, education and too cumbersome to be usable in all but
are different categories of insiders. training for safe computing technology the most extreme environments; even
and human peer detection of insider in such environments, the necessary
What are the potential abuses, and systems that are resilient systems are not readily available. Access
and can effectively remediate detected controls that are used in typical business
threats?
insider exploits. Of particular interest environments tend to be discretionary,
The insider threat is often discussed will be the ability to deal with multiple meaning that the individual or group of
in terms of threats to confidentiality colluding insiders—including detect- individuals who are designated as owners
and privacy (such as data exfiltration). ing potential abuses and responding to of an object can arbitrarily grant or deny
However, other trustworthiness require- them. others access to the object. Discretion-
ments, such as integrity, availability, ary access controls (DAC) typically do
and accountability, can also be com- What is the current state of not prevent anyone with read access to
promised by insiders. The threats span an object from copying it and sharing
the practice?
the entire system life cycle, including the copy outside the reach of that user’s
not only design and development but The insider threat today is addressed access control system. They also do not
also operation and decommissioning mostly with procedures such as aware- ensure sufficient protection for system
(e.g., where a new owner or discov- ness training, background checks, good and data integrity. Further background
erer can implicitly become a de facto labor practices, identity management on these and other security-related issues
insider). and user authentication, limited audits can be found in [And08,Bis02,Pfl03].
and network monitoring, two-person
Who are the potential controls, application-level profiling and File and disk encryption may have some
monitoring, and general access con- relevance to the insider threat, to the
beneficiaries? What are their
trols. However, these procedures are extent that privileged insiders might
respective needs?
not consistently and stringently applied not be able to access the encrypted data
The beneficiaries of this research range because of high cost, low motivation, of other privileged users. Also of pos-
from the national security bodies operat- and limited effectiveness. For example, sible relevance might be secret splitting,
ing the most sensitive classified systems large-scale identity management can k-out-of-n authorizations, and possibly
to homeland security officials who accomplish a degree of nonrepudiation zero-knowledge proofs. However, these
need to share Sensitive But Unclassified and deterrence but does not actually would need considerable improvement
(SBU) information/Controlled Unclas- prevent an insider from abusing granted if they were to be effective in commercial
sified Information (CUI), and to health privileges. products.
care, finance, and many other sectors
where sensitive and valuable informa- Technical access controls can be applied
tion is managed. In many systems, such to reduce the insider threat but not elim-
as those operating critical infrastructures inate it. The technologies traditionally
Develop better methods to advances in natural language might be able to hinder insider misuse.
combat insiders acting alone. understanding). (Protect) For example, what might be the rela-
(Protect) tive merits of cryptographically based
Develop insider prediction
authentication, biometrics, and so on,
Pursue the relevance and techniques for users, agents, and
with respect to misuse, usability, and
effectiveness of deception actions. (React)
effectiveness? To what extent would
techniques. (Protect) various approaches to differential
Incorporate integrity protection What R&D is evolutionary and access controls hinder insider misuse?
into authorization and system what is more basic, higher Detectability of insider misuse and the
architectures. (Protect) risk, game changing? inviolability of audit trails would also be
amenable to useful metrics.
Develop behavior-based security,
Intelligent uses of authentication, exist-
for example, advanced decoy ing access-control and accountability The extent to which such localized
networking. (Protect) mechanisms, and behavior monitor- metrics might be composable into
Develop and apply various risk ing would generally be incremental enterprise-level metrics is a challenge of
indicators. (React) improvements. However, in the long particular interest here.
term, significantly new approaches are
Long Term
desirable. To what extent can we test
Establish effective methods
to apply the principle of least real systems?
privilege. (Protect)
Resources
There is a strong need for
Develop methods to address Research, experimental testbeds, and realistic data for evaluation of
multiple colluding insiders. evaluations will be essential. technologies and policies that
(Protect) counter insider threats. This
Measures of success must be done operationally in
Pursue the architecture of insider-
a relatively noninvasive way.
resilient systems. (Protect) Various metrics are needed with respect Testbeds are needed, as well
Pursue applications of to the ability of systems to cope with as exportable databases of
cryptography that might limit insiders. Some will be generic: others anonymized data (anonymization
insider threats. (Protect) will be specific to given applications and
is generally a complicated
given systems. Metrics might consider
Develop automated decoy problem).
the extent to which various approaches
generation (may require to authentication and authorization Red teaming is needed to identify
References
[And2008] R. Anderson. Security Engineering: A Guide to Building Dependable
Distributed Systems. Wiley, Indianapolis, Indiana, 2008.
[Bib1977] K.J. Biba. Integrity Considerations for Secure Computer Systems. Technical Report MTR 3153,
The MITRE Corporation, Bedford, Massachusetts, June 1975. Also available from USAF
Electronic Systems Division, Bedford, Massachusetts, as ESD-TR-76-372, April 1977.
[Bis2002] M. Bishop. Computer Security: Art and Science. Addison-Wesley Professional, Boston, Massachusetts, 2002.
[Bra2004] Richard D. Brackney and Robert H. Anderson. Understanding the Insider Threat: Proceedings of a
March 2004 Workshop. RAND Corporation, Santa Monica, California, 2004
(http://www.rand.org/pubs/conf_proceedings/2005/RAND_CF196.pdf ).
[Cap2008.1] D. Capelli, T. Conway, S. Keverline, E. Kowalski, A. Moore, B. Willke, and M. Williams. Insider Threat
Study: Illicit Cyber Activity in the Government Sector, Carnegie Mellon University, January 2008
(http://www.cert.org/archive/pdf/insiderthreat_gov2008.pdf ).
[Cap2008.2] D. Capelli, E. Kowalski, and A. Moore. Insider Threat Study: Illicit Cyber Activity in the Information
Technology and Telecommunications Sector. Carnegie Mellon University, January 2008
(http://www.cert.org/archive/pdf/insiderthreat_it2008.pdf ).
[FSS2008] Financial Services Sector Coordinating Council for Critical Infrastructure Protection and
Homeland Security, Research and Development Committee. Research Agenda for the Banking
and Finance Sector. September 2008 (https://www.fsscc.org/fsscc/reports/2008/RD_Agenda-
FINAL.pdf ). Challenge 4 of this report is Understanding the Human Insider Threat.
[HDJ2006] IT Security: Best Practices for Defending Against Insider Threats to Proprietary Data, National Defense
Journal Training Conference, Arlington, Virginia. Homeland Defense Journal, 19 July 2006
[Kee2005] M. Keeney, D. Cappelli, E. Kowalski, A. Moore, T. Shimeali, and St. Rogers. Insider Threat
Study: Computer System Sabotage in Critical Infrastructure Sectors. Carnegie Mellon
University, May 2005 (http://www.cert.org/archive/pdf/insidercross051105.pdf ).
[Moo2008] Andrew P. Moore, Dawn M. Cappelli, and Randall F. Trzeciak. The “Big Picture” of IT Insider Sabotage
Across U.S. Critical Infrastructures. Technical Report CMU/SEI-2008-TR-009, Carnegie Mellon
University, 2008 (http://www.cert.org/archive/pdf/08tr009.pdf ). This report describes the MERIT model.
[Neu2008] Peter G. Neumann. Combatting insider misuse with relevance to integrity and accountability in elections
and other applications. Dagstuhl Workshop on Insider Threats, July 2008
(http://www.csl.sri.com/neumann/dagstuhl-neumann.pdf ). This position paper expands on the
fuzziness of trustworthiness perimeters and the context-dependent nature of the concept of insiders.
[Noo+2008] Thomas Noonan and Edmund Archuleta. The Insider Threat to Critical Infrastructures. National
Infrastructure Advisory Council, April 2008
(http://www.dhs.gov/xlibrary/assets/niac/niac_insider_threat_to_critical_infrastructures_study.pdf ).
[Ran04] M.R. Randazzo, D. Cappelli, M. Keeney, and A. Moore. Insider Threat Study: Illicit Cyber Activity in the
Banking and Finance Sector, Carnegie Mellon University, August 2004
(http://www.cert.org/archive/pdf/bankfin040820.pdf ).
[Sto+08] Salvatore Stolfo, Steven Bellovin, Shlomo Hershkop, Angelos Keromytis, Sara Sinclair, and Sean
Smith (editors). Insider Attack and Cyber Security: Beyond the Hacker. Springer, New York, 2008.
Malware infects systems via many vectors, including propagation from infected
machines, tricking users to open tainted files, or getting users to visit malware-
propagating websites. Malware may load itself onto a USB drive inserted into
an infected device and then infect every other system into which that device is
subsequently inserted. Malware may propagate from devices and equipment that
contain embedded systems and computational logic. An example would be infected
test equipment at a factory that infects the units under test. In short, malware can
be inserted at any point in the system life cycle. The World Wide Web has become
a major vector for malware propagation. In particular, malware can be remotely
injected into otherwise legitimate websites, where it can subsequently infect visitors
to those supposedly “trusted” sites.
There are numerous examples of malware that is not specific to a particular operat-
ing system or even class of device. Malware has been found on external devices (for
example, digital picture frames and hard drives) and may be deliberately coded into
systems (life cycle attacks). Increasingly intelligent household appliances are vulner-
able, as exemplified by news of a potential attack on a high-end espresso machine
[Thu2008]. Patching of these appliances may be difficult or impossible. Table 5.1
summarizes malware propagation mechanisms.
Potentially victimized systems include end user systems, servers, network infra-
structure devices such as routers and switches, and process control systems such as
Supervisory Control and Data Acquisition (SCADA).
A related policy issue is that reasonable people may disagree on what is legitimate
commercial activity versus malware. In addition, ostensibly legal software utilities
(for example, for digital rights management [DRM]) may have unintended conse-
quences that mimic the effects of malware [Sch2005, Hal2006].
38
It is likely that miscreants will develop systems until attribution can be consequences of botnets and malware
new infection mechanisms in the future, accomplished. Honeypots can include spam, distributed denials of
either through discovery of new security also be useful in this regard.) service (DDoSs), eavesdropping on
gaps in current systems or through new traffic (sniffing), click fraud, loss of
exploits that arise as new communi- The NSA/ODNI Workshop on system stability, loss of confidentiality,
cation and computation paradigms Computational Cyberdefense in loss of data integrity, and loss of access
emerge. Compromised Environments, Santa to network resources (for example,
Fe, NM, August 2009, was an being identified as a bot node and
The technical challenges are, wherever example of a step in this direction then blocked by one’s ISP or network
possible, to do the following: (http://www.c3e.info). administrator, effectively a DoS inflicted
by one victim on another). An increas-
Avoid allowing malware onto a
What are the potential ing number of websites (such as popular
platform.
social networking systems, web forums,
Detect malware that has been threats?
and mashups) permit user-generated
installed. Malware has significant impact in many content, which, if not properly checked,
Limit the damage malware can aspects of the information age and can allow attackers to insert rogue
do once it has installed itself on a underlies many of the topics discussed content that is then potentially down-
platform. elsewhere in this document. Impacts loaded by many users.
Operate securely and effectively can be single-host to networkwide, nui-
in the presence of malware. sance to costly to catastrophic. Negative Beyond its nuisance impact, malware
Determine the level of risk consequences include degraded system can have serious economic and national
based on indications of detected performance and data destruction or security consequences. Malware can
malware. modification. Spyware permits adver- enable adversary control of critical com-
Remove malware once it has saries to log user actions (to steal user puting resources, which in turn may
been installed (remediation), and credentials and facilitate identity theft, lead, for example, to information com-
monitor and identify its source for example), while bot malware enables promise, disruption and destabilization
(attribution). (Remediation an adversary to build large networks of of infrastructure systems (“denial of
may sometimes be purposefully compromised machines and amplify an control”), and manipulation of financial
delayed on carefully monitored adversary’s digital firepower. Negative markets.
while new A/V signatures take time Web-based A/V services have entered Vendors of operating systems and
to produce, test, and distribute. In the market, some offering a service applications have developed mecha-
addition, it takes time for the user com- whereby a security professional can nisms for online updating and patching
munity to develop, test, and deploy submit a suspicious executable to see software for bugs, including bugs that
patches for the underlying vulnerability whether it is identified as malicious by affect security. Other defenses include
that the malware is exploiting. Further- current tools. This mechanism most antispyware, whitelists of trusted web-
more, the malware developers can test likely functions also as a testbed for sites and machines, and reputation
their software against the latest A/V malware developers (VirusTotal). mechanisms.
versions. [Vir].
Research in malware detection and The U.S. National Institute of Stan- Current detection and remediation
prevention is ongoing. For example, dards and Technology (NIST) Security approaches are losing ground, because
see the Cyber-Threat Analytics project Content Automation Protocol (SCAP) it is relatively easy for an adversary
(http://www.cyber-ta.org). Also worth is a method for using specific stan- (whether sophisticated or not) to
noting is the Anti-Phishing Working dards to enable automated vulnerability alter malware to evade most existing
Group (APWG): management, measurement, and policy detection approaches. Given trends in
http://www.antiphishing.org. compliance evaluation. malware evolution, existing approaches
46
COMBATTING MALWARE AND BOTNETS
in this domain range from privacy con- that must be detected in order to claim It would be beneficial to have reliable
cerns, legal aspects of data sharing, and effectiveness at some level. metrics that estimate the vulnerability
the sheer volume of data itself. Research of particular systems to corruption by
in generating adequate metadata and We can define measures of success at malware, and how well they are able
provenance is required to overcome a high level by answering the follow- to withstand other kinds of malware-
these hurdles. ing questions and tracking the answers enabled attacks, such as DDoS attacks.
over time: Similarly, metrics that suggest the ben-
Techniques to capture and analyze efits that will accrue with the use of
How many machines do we
malware and propagate defenses faster particular malware prevention or reme-
are essential in order to contain epidem- know about that serve malware? diation strategies would be helpful.
ics. Longer-term research should focus What is the rate of emergence of
on inherently secure, monitorable, and new malware?
auditable systems. Threat analysis and Since spam is a primary botnet What needs to be in place for
economic analysis of adversary markets output, what fraction of e-mail is test and evaluation?
should be undertaken in pilot form in spam?
the near term, and pursued more vigor- Beyond reverse engineering of malware,
What is the industry estimate of
ously if they are shown to be useful. the most effective studies of malicious
hosts serving malware? code have taken place on network test-
What is the trend in malware beds. These testbeds have included
Measures of success severity (on a notional simple virtual machines “networked”
We require baseline measurements of continuum, say from nuisance to on an analyst’s computer, testbeds
the fraction of infected machines at any adware, spyware, bot capture)? consisting of tens or hundreds of real
time; success would be a reduction in What fraction of known attacks (nonvirtualized) nodes, such as DETER
this fraction over time. is successful, and what fraction is [DET], and simulated networks created
thwarted? within network simulation tools. The
Some researchers currently track the research community has yet to approach
emergence of malware. In this way, they We may also consider cost-based mea- studies of malware in Internet-scale
are able to identify trends (for example, sures (from the defender point of view), emulated environments. The infrastruc-
the number of new malware samples per such as: ture and tools do not currently exist to
month). A reversal of the upward trend build emulation environments on the
What is our cost of searching for
in malware emergence would indicate order of 10,000,000 nodes or more.
success. malware propagators?
What is the cost to identify As malware sophistication improves to
Time between malware capture and botnets and their bot command include detection of virtual environ-
propagation of defense (or, perhaps and control infrastructures? ments, the realism of the virtualization
more appropriately, implementation What is the cost to increase environment (for example, virtual
of the defense on formerly vulnerable sharing of malware host lists? machine or honeynet) testbed presents
systems) tracks progress in human and a challenge.
automated response time. Economic analysis of adversary markets
may allow definition of metrics as to Tools and environments to study
With reference to the repository, we effectiveness of particular defenses. malware need to evolve as the malware
may define a minimal set of exemplars evolves. In particular, the community
References
[Ant2008] A.M. Antonopoulos. Georgia cyberwar overblown. Network World, August 19, 2008
(http://www.pcworld.com/businesscenter/article/150021/georgia_cyberwar_overblown.html).
[CAT2009] Conference for Homeland Security 2009 (CATCH ’09), Cybersecurity Applications and Technology,
March 3–4, 2009. The IEEE proceedings of this conference include relevant papers on detection and
mitigation of botnets, as well as correlation and collaboration in cross-domain attacks, from the University
of Michigan and Georgia Tech, as well as Endeavor, HBGary, Milcord, and Sonalyst (among others).
[Dai2006] Dino Dai Zovi, Vitriol: Hardware virtualization rootkits. In Proceedings of the Black Hat
USA Conference, 2006.
[Fra2007] J. Franklin, V. Paxson, A. Perrig, and S. Savage. An inquiry into the nature and
causes of the wealth of Internet miscreants. Proceedings of ACM Computer and
Communications Security Conference, pp. 375-388, October 2007.
[GAO2007] CYBERCRIME: Public and Private Entities Face Challenges in Addressing Cyber Threats. Report
GAO-07705, U.S. Government Accountability Office, Washington, D.C., July 2007.
[Hal2006] J.A. Halderman and E.W. Felten. Lessons from the Sony CD DRM episode. In
Proceedings of the 15th USENIX Security Symposium, August 2006.
48
COMBATTING MALWARE AND BOTNETS
[Hol2008] T. Holz, C. Gorecki, K. Rieck, and F. Freiling. In Proceedings of the 15th Annual
Network & Distributed System Security (NDSS) Symposium, February 2008.
[Kim2004] Hyang-Ah Kim and Brad Karp, Autograph: Toward automated, distributed worm signature
detection, In Proceedings of the 13th USENIX Security Symposium, August 2004.
[IW2007] L. Greenemeier. Estonian attacks raise concern over cyber ‘nuclear winter.’ Information Week, May 24,
2007 (http://www.informationweek.com/news/internet/showArticle.jhtml?articleID=199701774).
[LAT2008] J.E. Barnes. Cyber-attack on Defense Department computers raises concerns. Los
Angeles Times, November 28, 2008 (http://www.latimes.com/news/nationworld/
iraq/complete/la-na-cyberattack28-2008nov28,0,230046.story).
[Mes2003] Ellen Messmer. Welchia Worm Nails Navy Marine Corps, Network World Fusion, August 19,
2003. (http://pcworld.com/article/112090/welchia_worm_nails_navy_marine_corps.html).
[Pou2003] Kevin Poulsen. Slammer worm crashed Ohio nuke plant network. SecurityFocus, August 19, 2003
(http://www.securityfocus.com/news/6767).
[Sha2004] H. Shacham, M. Page, B. Pfaff, E.-J. Goh, N. Modadugu, and D. Boneh. On the
effectiveness of address-space randomization. In Proceedings of the 11th ACM Computer
and Communications Security Conference, Washington, D.C., pp. 298-307, 2004.
[Sha2008] M. Sharif, V. Yegneswaran, H. Saidi, P. Porras, and W. Lee. Eureka: A framework for
enabling static malware analysis. In Proceedings of the 13th European Symposium on Research
in Computer Security (ESORICS), Malaga, Spain, pp. 481-500, October 2008.
[Sch2005] Bruce Schneier. Real story of the rogue rootkit. Wired, November 17, 2005 (http://
www.wired.com/politics/security/commentary/securitymatters/2005/11/69601).
[Thu2008] R. Thurston. Coffee drinkers in peril after espresso overspill attack. SC Magazine, June 20, 2008 (http://
www.scmagazineuk.com/coffee-drinkers-in-peril-after-espresso-overspill-attack/article/111458).
Our concern here is mainly the IT-oriented aspects of the broad problems of
identity and credential management, including authentication, authorization, and
accountability. However, we recognize that there will be many trade-offs and privacy
implications that will affect identity management. In particular, global-scale identity
management may require not only advances in technology, but also open standards,
social norms, legal frameworks, and policies for the creation, use, maintenance,
and audit of identities and privilege information (e.g., rights or authorizations).
Clearly, managing and coordinating people and other entities on a global scale
also raises many issues relating to international laws and regulations that must be
considered. In addition, the question of when identifying information must be
provided is fundamentally a policy question that can and should be considered. In
all likelihood, any acceptable concept of global identity management will need to
incorporate policies governing release of identifying information. Overall, countless
critical systems and services require authenticated authorization for access and use,
50
and global-scale identity management fronts by a wide range of potential of integrity, confidentiality, and system
will be a critical enabler of future IT attackers with diverse motivations, survivability, as well as denial-of-service
capabilities. Furthermore, it is essential within large-scale organizations and attacks.
to be able to authorize on the basis of across multiple organizations. Insider
attributes other than merely supposed and outsider misuses are commonplace. Threats described in other topic areas
identities. Identity management needs Because of the lack of adequate iden- can also affect global-scale identity
to be fully integrated with all the systems tity management, it is often extremely management, most notably defects in
into which it is embedded. difficult to identify the misusers. For trustworthy scalable systems. In addi-
example, phishing attacks have become tion, defects in global-scale identity
Identity management systems must a pervasive problem for which identify- management can have negative impacts
enable a suite of capabilities. These ing the sources and the legitimacy of the on provenance and attack attribution.
include control and management of cre- phishers and rendering them ineffective
dentials used to authenticate one entity where possible are obvious needs. Who are the potential
to another, and authorization of an
beneficiaries? What are their
entity to adopt a specific role and assert Identity-related threats exist throughout
respective needs?
properties, characteristics, or attributes the development cycle and the global
of entities performing in a role. Global- supply chain, but the runtime threats Governmental agencies, corporations,
scale identity management must also are generally predominant. Misuse of institutions, individuals, and particu-
support nonrepudiation mechanisms identities by people and misuse of flawed larly the financial communities [FSSCC
and policies; dynamic management of authentication by remote sites and com- 2008] would benefit enormously from
identities, roles, and properties; and promised computers (e.g., zombies) are the existence of pervasive approaches
revocation of properties, roles, and iden- common. The Internet itself is a source to global identity management, with
tity credentials. Identity management of numerous collateral threats, including greater convenience, reduction of
systems must provide mechanisms for coordinated, widespread denial-of-ser- administrative costs, and possibili-
two-way assertions and authentica- vice attacks, such as repeated failed ties for better oversight. Users could
tion handshakes building mutual trust logins that result in disabling access by benefit from the decreased likelihood of
among mutually suspicious parties. legitimate users. Various threats arise impersonation, identity and credential
All the identities and associated asser- when single-sign-on authentication fraud, and untraceable misuse. Although
tions and credentials must be machine of identities occurs across boundaries the needs of different individuals and
and human understandable, so that all of comparable trustworthiness. This different organizations might differ
parties are aware of the identity interac- is likely to be a significant concern in somewhat, significant research in this
tions and relationships between them highly distributed, widespread system area would have widespread benefits for
(e.g., what these credentials are, who environments. Additional threats arise all of them.
issued them, who has used them, and with respect to the misuse of identities
who has seen them). The lifetimes of and authentication, especially in the What is the current state of
credentials may exceed human lifetimes presence of systems that are not ade-
the practice?
in some cases, which implies that pre- quately trustworthy. Even where systems
vention of and recovery from losses are have the potential for distinguishing There are many current approaches to
particularly difficult problems. among different roles associated with identity management. Many of these
different individuals and where fine- are not yet fully interoperable with
What are the potential grained access controls can be used, other required services, not scalable,
operational considerations and inade- only single-use, or limited in other
threats?
quate user awareness can tend to subvert ways. They do, however, collectively
Identification and authentication (I&A) the intended controls. In particular, exhibit pointwise examples that can lead
systems are being attacked on many threats are frequently aimed at violations toward enabling a global-scale identity
References
[FSS2008] Financial Services Sector Coordinating Council for Critical Infrastructure. Protection and Homeland
Security, Research and Development Committee. Research Agenda for the Banking and Finance
Sector. September 2008, (https://www.fsscc.org/fsscc/reports/2008/RD_Agenda-FINAL.pdf ).
[IDT2009] 8th Symposium on Identity and Trust on the Internet (IDtrust 2009), NIST, April 14-16, 2009
(http://middleware.internet2.edu/idtrust). The website contains proceedings of previous
years’ conferences. The 2009 proceedings include three papers representing team members
from the I3P Identity Management project (which includes MITRE, Cornell, Georgia
Tech, Purdue, SRI, and the University of Illinois at Urbana-Champaign).
Time-critical systems, generally speaking, are systems that require response on non-
human timescales to maintain survivability (i.e., continue to operate acceptably)
under relevant adversities. In these systems, human response is generally infeasible
because a combination of the complexity of the required analysis, the unavailabil-
ity and infeasibility of system administrators in real time, and the associated time
constraints. This section uses the following definition:
Of particular interest here are systems for which impaired survivability would have
large-scale consequences, particularly in terms of the number of people affected.
Examples of such systems include electric power grids and other critical infrastruc-
ture systems, regional transportation systems, large enterprise transaction systems,
and Internet infrastructure such as routing or DNS. Although impaired survivability
for some other types of systems may have severe consequences for small numbers of
users, they are not of primary relevance to this topic. Examples of such systems are
medical devices, individual transportation systems, home desktop computers, and
isolated embedded systems. Such systems are not always designed for an adequate
level of survivability, but the problem is less challenging to address for them than for
large and distributed systems. However, common-mode failures of large numbers
of small systems (for example, a vulnerability in a common type of medical device)
could have large-scale consequences. (Note that personal systems are not actually
ignored here, in that certain major advances in survivability of large-scale time-
critical systems may be applicable to smaller systems.)
57
Figure 7.1: Examples of Systems With Different Time-Criticality Requirements and
Different User Populations
Time-criticality
Pacemaker
Avionics
Enterprise/sector
critical transac-
tion system
Ad-hoc emergency
response system
Corporate
office server
Home PC
of systems categorized with respect to failures, and accidents. Rather than enu- educators and students, standards
relative time criticality and size of the merate a long list, we refer throughout bodies, and so on. These categories of
user population they serve. The systems to “all relevant adversities” for which beneficiaries have very different needs.
on the right side of the diagonal line survivability is required. End users need to have a working system
are considered in primary scope for this whenever they need to use it (avail-
discussion, while systems to the left of Who are the potential ability), and they need the system to
the line are of secondary interest, as continue working correctly once they
beneficiaries? What are their
indirect beneficiaries. have started using it (reliability). System
respective needs?
owners have many additional needs;
What are the potential Beneficiaries include the ultimate end for example, they need to have situ-
users of critical infrastructure systems ational awareness so that they can be
threats?
(the public), system owners and opera- warned about potential problems in
As noted in the definition of survivabil- tors, system developers and vendors, the system and manage system load,
ity, the threats include system attacks, regulators and other government bodies, and they need to be able to react to an
[DIS2003] 3rd DARPA Information Survivability Conference and Exposition (DISCEX-III 2003), 22-24
April 2003, Washington, DC, USA. IEEE Computer Society 2003, ISBN 0-7695-1897-4.
[Ell+1999] R.J. Ellison, D.A. Fisher, R.C. Linger, H.F. Lipson, T. Longstaff, and N.R.
Mead. Survivable Network Systems: An Emerging Discipline. Technical Report
CMU/SEI-97-TR-013, Carnegie Mellon University, May 1999.
[Hai+2007] Yacov Y. Haimes, Joost R. Santos, Kenneth G. Crowther, Matthew H. Henry, Chenyang Lian, and
Zhenyu Yan. Analysis of Interdependencies and Risk in Oil & Gas Infrastructure Systems. I3P Research
Report No. 11, June 2007 (http://www.thei3p.org/docs/publications/researchreport11.pdf ).
[Ker+2008] Peter Kertzner, Jim Watters, Deborah Bodeau, and Adam Hahn. Process Control System Security
Technical Risk Assessment Methodology & Technical Implementation. I3P Research Report No.
13, March 2008 (http://www.thei3p.org/docs/publications/ResearchReport13.pdf ).
[Neu2000] P.G. Neumann. Practical Architectures for Survivable Systems and Networks. SRI International,
Menlo Park, California, June 2000 (http://www.csl.sri.com/neumann/survivability.html).
Situational understanding includes the state of one’s own system from a defensive
posture irrespective of whether an attack is taking place. It is critical to understand
system performance and behavior during non-attack periods, in that some attack
indicators may be observable only as deviations from “normal behavior.” This
understanding also must include performance of systems under stress that are
not caused by attacks, such as a dramatic increase in normal traffic due to sudden
popularity of a particular resource.
Situational understanding also encompasses both the defender and the adversary.
The defender must have adversary models in order to predict adversary courses of
action based on the current defensive posture. The defender’s system-level goals
are to deter unwanted adversary actions (e.g., attacking our information systems)
and induce preferred courses of action (e.g., working on socially useful projects as
opposed to developing crimeware, or redirecting attacks to a honeynet).
65
attack. Accurate attribution supports and how our decision makers interpret, Adversaries may be able to exfiltrate
improved situational understanding and react to, and mitigate those attacks. Of sensitive data over periods of time,
is therefore a key element of research in special concern are attacks on informa- again without actually taking down
this area. Appropriate attribution may tion systems with potentially significant the targeted systems. Here, situational
often be possible only incrementally, strategic impact, such as wide-scale understanding should clearly include
as situational understanding becomes power blackouts or loss of confidence understanding of government threat
clearer through interpretation of avail- in the banking system. Attacks may models and concerns. Sharing such
able information. come from insiders, from adversaries understanding is particularly impor-
using false credentials, from botnets, or tant—and sensitive in the sense that it is
Situational understanding is larger than from other sources or a blend of sources. likely to lead to recognition of additional
one user, or possibly even larger than one Understanding the attack is essential for weaknesses and vulnerabilities.
administrative domain, and addresses defense, remediation, attribution to the
what is happening through consider- true adversary or instigator, hardening In addition, the more serious attacks
ation of a particular area of interest at of systems against similar future attacks, now occur at two vastly different time-
a granularity that is appropriate to the and deterring future attacks. Attribution scales. The classic fear is cyber attacks
administrator(s) or analyst(s). In partic- should also encompass shell companies, that occur faster than human response
ular, situational understanding of events such as rogue domain resellers whose times. Those attacks are still of concern.
within infrastructures spanning multiple business model is to provide an enabling However, another concern is “low and
domains may require significant coor- infrastructure for malfeasance. There slow” and possibly stealthy attacks
dination and collaboration on multiple are numerous areas of open research that break the attack sequences into
fronts, such as decisions about when/ when it comes to these larger questions a series of small steps spread over a
whether to share data, how to depict the of attribution. For example, we have long time period. Achieving situational
situation as understanding changes over not adequately addressed digital finger- awareness for these two ends of the con-
time, and how to interpret or respond printing of rogue providers of hosting tinuum is likely to require very different
to the information. Attribution is a key services. (See also Section 9.) approaches.
element of this process, since it is con-
cerned with who is doing what and what There have been numerous widely pub- Who are the potential
should be done in response. licized large-scale attacks launched for
beneficiaries? What are their
a variety of purposes, but recently there
respective needs?
What are the potential is a consensus that skilled nonstate
actors are now primarily going after Although all computer users and all
threats?
financial gain [GAO2007, Fra2007]. consumers of information systems
Situational understanding addresses a Click fraud, stock “pump and dump,” products are potential victims of the
broad range of cyber attacks, specifically and other manipulations of real-time broad range of attacks we address, and
including large-scale and distributed markets prove that it is possible to profit would benefit from improved situational
attacks, where it is felt that adversary from cybercrime without actually taking awareness, we are primarily seeking tools
capabilities are outstripping our ability down the systems that are attacked. In and techniques to help the communities
to defend critical systems. Inability to this context, situational understanding whose challenges and needs are given
attribute sophisticated attacks to the should clearly encompass law enforce- in Table 8.1—although this is not an
original perpetrator leads to a growing ment threat models and priorities, as comprehensive set.
asymmetry in cyber conflict. well as how financial gains can accrue.
Because of time criticality for respond-
In this topic area, we are concerned For state actors, the current concern ing to certain cyber attacks, and hence
chiefly with the universe of cyber attacks is targeting of our critical infrastruc- the need to tie these to situational aware-
within the information systems domain tures and key government systems. ness, we consider developers and users
of autonomic response systems as part but is currently accomplished via ad not know and trust each other. (For
of the customer base for advances in this hoc and informal relationships. In a example, how can an administrator in
topic area. few instances, data is shared across Domain A prove that a customer of
organizations, but normally the kinds Domain B is an attacker, and thereby
What is the current state of of information shared are limited persuade an administrator in that domain
(e.g., only network packet headers). to take corrective action?)
the practice?
Situational understanding currently Intrusion detection/prevention tech- Industry has made significant progress
is addressed within administrative nology is widely deployed, but many in the area of event/data correlation,
domains through intrusion detection/ question how much longer it will be with several security information and
prevention systems and security event effective as traffic volumes grow, attacks event management (SIEM) commercial
correlation systems, with much of the get more subtle, signature bases grow products widely deployed in the market.
analysis still done through manual correspondingly larger and unable to These offer considerable value in timely
perusal of log files. There have been cope with new attacks, and attackers data reduction and alarm management.
efforts to provide visualizations and use encryption, which makes packet However, with respect to visualization
other analytical tools to improve the payload signature analysis difficult. and presentation on a massive data scale,
ability to comprehend large amounts of Response to large-scale attacks remains these systems are inadequate and do not
data. These are largely special purpose to a large degree informal, via personal have scope well beyond organizational
and found within research laboratories trust relationships and telephone com- boundaries.
rather than being used widely within munications. This situation makes it
the field. Sharing security-relevant infor- difficult or impossible to achieve very We need to consider the viewpoint of
mation across domains is essential for rapid response or cooperation between the defender (end host, infrastructure
large-scale situational understanding domains where the administrators do component, enterprise, Internet). An
References
[Fra2007] J. Franklin, V. Paxson, A. Perrig, and S. Savage. An inquiry into the nature and
causes of the wealth of Internet miscreants. Proceedings of ACM Computer and
Communications Security Conference, pp. 375-388, October 2007.
[GAO2007] CYBERCRIME: Public and Private Entities Face Challenges in Addressing Cyber Threats. Report
GAO-07-705, U.S. Government Accountability Office, Washington, D.C., July 2007.
[Hol2008] T. Holz, C. Gorecki, K. Rieck, and F. Freiling. Measuring and detecting fast-flux service networks. In
Proceedings of the 15th Annual Network & Distributed System Security (NDSS) Symposium, February 2008.
BACKGROUND
Provenance is also concerned with the original sources of any subsequent changes
or other treatment of information and resources throughout the life cycle of data.
That information may be in any form, including software, text, spreadsheets, images,
audio, video, proprietary document formats, databases, and others, as well as meta-
level information about information and information transformations, including
editing, other forms of markup, summarization, analysis, transformations from one
medium to another, formatting, and provenance markings. Provenance is generally
concerned with the integrity and reliability of the information and meta-information
rather than just the information content of the document.
76
To determine provenance accurately, we scientific fields are examples where prov- What is the current state of
must have trustworthy systems that reli- enance markings are beginning to be practice?
ably track both usage and modification used. Other fields that can benefit from
of information and other resources. As provenance maintenance systems include Physical provenance markings in jewelry
with all computer systems, security of critical infrastructure providers (e.g., in (e.g., claiming your diamond is from a
provenance tracking cannot be absolute, SCADA and other control systems), blood-free mining operation, your silver
and trustworthiness of provenance track- emergency responders, military person- or gold is pure, and the style is not a
ing systems will be relative to the value nel, and other decision makers. Users in knockoff copy of a designer’s), explo-
of the provenance to the users of the all these areas need reliable information sive components (e.g., nitrates), and
information and resources. For example, obtained from many sources, commu- clothing have historically added value
a simple change-tracking mechanism nicated, aggregated, analyzed, stored, and enabled tracing of origin. Docu-
in a document preparation system may and presented by complex information ment markings such as wax seals and
provide adequate provenance track- processing systems. Information sources signatures have been used to increase
ing from the point of view of a small must be identified, maintained, and assurance of authenticity of high-value
group of authors collaborating in the tracked to help users make appropriate documents for centuries. More recently
publication of an article, even though decisions based on reliable understand- the legal, auditing, and medical fields
the document change history might ing of the provenance of the data used have begun to employ first-level authen-
not be protected from unauthorized as input to critical decisions. ticated provenance markings.
modification. On the other hand, the
same mechanism may be inadequate in In addition, new techniques are needed The current practice is rather rudimen-
the context of legal discovery, precisely that will allow management of prov- tary compared with what is needed to
because the change-tracking mechanism enance for voluminous data. Part of be able to routinely depend on prov-
does not guarantee the authenticity of what has made provenance easier to enance collection and maintenance.
the change history. manage up to now is its small volume. The financial sector (in part driven
Now, geospatial information-gathering by Sarbanes-Oxley requirements) has
What are the potential systems are being planned that will have developed techniques to enable track-
the capability of handling gigabytes of ing of origins, aggregations, and edits of
threats?
data per second, and the challenges of data sets. Users of document production
Without trustworthy provenance track- these data volumes will be exacerbated software may be familiar with change-
ing systems, there are threats to the by collection via countless other sensor tracking features that provide a form of
data and to processes that rely on the networks. Within 20 years, the govern- provenance, although one that cannot
data, including, for example, unattrib- ment will hold an exabyte of potentially necessarily be considered trustworthy.
uted sources of software and hardware; sensitive data. The systems for handling
unauthorized modification of data and establishing provenance of such As an example of provenance in which
provenance; unauthorized exposure of volumes of information must function security of the provenance has not been
provenance, where presumably pro- autonomously and efficiently with infor- a direct concern, software development
tected; and misattribution of provenance mation sources at these scales. teams have relied for decades on version
(intentional or otherwise). control systems to track the history of
Note that situations are likely to arise changes to code and allow for historical
Who are the potential where absence of provenance is impor- versions of code to be examined and
beneficiaries? What are their tant—for example, where information used. Similar kinds of systems are used
respective needs? that needs to be made public must not in the scientific computing community.
The legal, accounting, medical, and be attributable.
PROVENANCE 77
What is the status of current Provenance-aware storage Pedigree management. The
research? systems. A provenance- Pedigree Management and
Current research appears to be driven aware storage system supports Assessment Framework (PMAF)
largely by application- and domain-spe- automatic collection and [SPI2007] enables a publisher
cific needs. Undoubtedly, these research maintenance of provenance of information in a network-
efforts are seen as vital in their respective metadata. The system creates centric intelligence gathering and
communities of interest. provenance metadata as new assessment environment to record
objects are created in the system standard provenance metadata
Examples of active, ongoing research and maintains the provenance about the source, the manner
areas related to information and resource just as it maintains ordinary file- of collection, and the chain of
provenance include the following areas: system metadata. See [PAS]. The modification of information as it
Lineage File System [LFS] records is passed through processing and
Data provenance and
the input files, command-line assessment.
annotation in scientific
options, and output files when a
computing. Chimera [Fos2002] For further background, see the proceed-
program is executed; the records
allows a user to define a are stored in an SQL database ings of the first USENIX workshop on
workflow, consisting of data sets that can be queried to reconstruct the theory and practice of provenance
and transformation scripts. The the lineage of a file. [TAP2009].
system then tracks invocations,
Chain of custody in computer
annotating the output with
forensics and evidence and FUTURE DIRECTIONS
information about the runtime
change control in software
environment. The myGrid On what categories can we
development. The Vesta
system [Zha2004], designed subdivide the topic?
[Hey2001] approach uses
to aid biologists in performing
provenance to make software Provenance may be usefully subdivided
computer-based experiments, builds incremental and
allows users to model their along three main categories, each of
repeatable. which may be further subdivided, as
workflows in a Grid environment.
Open Provenance Model. The follows:
CMCS [Pan2003] is a toolkit for
Open Provenance Model is a
chemists to manage experimental Representation: data models
recently proposed abstract data
data derived from fields such and representation structures
model for capturing provenance.
as combustion research. ESSW for provenance (granularity and
The model aims to make it easier
[Fre2005] is a data storage system access control).
for provenance to be exchanged
for earth scientists; the system Management (creation; access;
between systems, to support
can track data lineage so that development of provenance tools, annotation [mark original
errors can be traced, helping to define a core set of inference documents/resources with
maintain the quality of large rules that support queries on provenance metadata]; editing
data sets. Trio [Wid2005] is a provenance, and to support [provenance-mark specific
data warehouse system that uses a technology-neutral digital fine-grained changes through
data lineage to automatically representation of provenance the life cycle]; pruning [delete
compute the accuracy of the for any object, regardless of provenance metadata for
data. Additional examples can be whether or not it is produced performance, security, and
found in the survey by Bose and by a computer system. See privacy reasons]; assurance; and
Frew [Bos2005]. [OPM2007]. revocation)
78 PROVENANCE
Presentation (query [request In the following itemization of gaps, the Pruning provenance, deleting
provenance information]; present letters R, M, P annotating each point and sanitizing extraneous item
[display provenance markings]; refer to the main categories—represen- for privacy and purpose of
alert [notify when provenance tation, management, and presentation, performance. (RMP)
absence, compromise, or fraud is respectively—where uppercase denotes Efficiently representing
detected]) high relevance (R, M, P), and lowercase provenance. An extreme
denotes some relevance (r, m, and p). goal would be to efficiently
Other useful dimensions to consider represent provenance for every
that are cross-cutting with respect to the Appropriate definitions and
means for manipulating bit, enabling bit-grained data
following dimensions: transformations, while requiring
meaningful granularity of
System engineering (human- a minimum of overhead in time
information provenance
computer interfaces; workflow and space. (RMp)
markings. Taxonomy of
implications; and semantic webs) provenance. (R) Scale: the need for solutions that
Legal, policy, and economic scale up and down efficiently. (R)
Given trends in markup
issues (regulation; standards; Dealing with heterogeneous data
languages, the metadata and
enforcement; market incentives) types and data sensors, domain
the underlying data are often
These are summarized in Table 9.1. intermixed (as in XML), specificity, and dependency
thus presenting challenges tracking. (Rm)
What are the major research in appropriate separation of Partial or probabilistic
gaps? concerns with data integrity and provenance (when the chain of
integrity of the provenance. (R) custody cannot be stated with
Numerous gaps in provenance and absolute certainty). (RMp)
Confidential provenance
tracking research remain to be filled,
and anonymous or partially Coping with legacy systems.
requiring a much broader view of the
anonymous provenance, (RM)
problem space and cross-disciplinary
to protect sources of Intrinsic vs. extrinsic provenance
efforts to capture unifying themes and
information. (R) and the consistency between
advance the state of the art for the
benefit of all communities interested in Representing the trustworthiness them when both are available.
provenance. of provenance. (R) (RMp)
PROVENANCE 79
Developing and adopting tools Teams (CERTs) need to be What R&D is evolutionary,
based on existing research results. able to prove from where and what is more basic,
(RMP) they got information about higher risk, game changing?
Centralized versus distributed vulnerabilities and fixes; when Information provenance presents a
provenance. (M) they publish alerts, they should large set of challenges, but significant
be able to reliably show that impact may be made with relatively
Ensuring the trustworthiness of
the information came from an modest technical progress. For example,
provenance (integrity through the
appropriate, credible source—for it may be possible to develop a coarse-
chain of custody). (M)
example, to avoid publishing grain information provenance appliance
Tracking: where did the an alert based on incorrect that marks documents traversing an
information/resources go; how information submitted by a
intranet or resting in a data center and
were they used? (M) competitor. They also need
makes those markings available to deci-
Usable provenance respecting their customers to believe that
sion makers. Although this imagined
security and privacy concerns. the information being sent is
appliance may not have visibility into
(Mp) not from an imposter (although
all the inputs used to create a docu-
certificates are supposed to take
Information provenance systems ment, it could provide relatively strong
care of this problem).
should be connected to chain of assurances about certain aspects of the
Law enforcement forensics provenance of the information in ques-
custody, audit, and data forensic
for computer-based evidence, tion. It is important to find methods
approaches. Provenance should
surveillance data, and other to enable incremental rollout of prove-
connect and support, not repeat
computer artifacts, of sufficient nance tools and tags in order to maintain
functionality of these related
integrity and oversight to compliance with existing practices and
services. (MP)
withstand expert counter- standards. Another incremental view is
User interfaces. When dealing testimony. to consider provenance as a static type
with massive amounts of data Crime statistics and analyses from system for data. Static type systems exist
from many sources with massive which patterns of misuse can be for many programming languages and
communication processes, how is deduced. frameworks that help prevent runtime
the end user informed and about
Medical and health care errors. By analogy, we could create an
what aspects of the information
information, particularly with information provenance system that is
integrity? (P)
respect to data access and data able to prevent certain types of misuse of
Users of aggregated information modification. data by comparing the provenance infor-
need to be able to determine mation with policies or requirements.
Identity-theft and identity-fraud
when less reliable information
detection and prevention.
is interspersed with accurate Resources
information. It is of critical Financial sector—for example, With respect to the extensive list of
importance to identify and with respect to insider research gaps noted above, resources will
propagate the source and information, funds transfers, and be needed for research efforts, experi-
derivation (or aggregation) of partially anonymous transactions. mental testbeds, test and evaluation, and
the chain of custody with the Provenance embedded within technology transition.
information itself. (P) digital rights management.
Measures of success
What are some exemplary In many of the above examples, some One indicator of success will be the
problem domains for R&D in of the provenance may have to be ability to track the provenance of infor-
this area? encrypted or anonymized—to protect mation in large systems that process and
Computer Emergency Response the identity of sources. transform many different, heterogeneous
80 PROVENANCE
types of data. The sheer number of dif- also provide measures of success. Effi- In medical systems, personally
ferent kinds of sensors and information ciency of representations might also identifiable information
systems involved and, in particular, the be a worthwhile indicator, as would be connected with embarrassing or
number of legacy systems developed measures of overhead attributable to insurance-relevant information
without any attention to maintenance maintaining and processing provenance. may be used to make life-critical
of provenance present major challenges Metrics that consider human usability health care decisions.
in this domain. of provenance would be very appropri-
An emergency responder system
ate—especially if they can discern how
might be considered that could
Red Teaming can give added analysis— well people actually are able to distin-
provide more reliable provenance
for example, assessing the difficulty of guish authentic and bogus information
information to decision makers
planting false content and subverting based on provenance.
provenance mechanisms. (e.g., who must be evacuated,
who has been successfully
What needs to be in place for evacuated from a building).
Also, confidence-level indicators are
test and evaluation?
desirable—for example, assessing the A provenance system for the legal
estimated accuracy of the information Testing and evaluating the effectiveness profession.
or the probability that information of new provenance systems is challeng- Credit history and scoring—for
achieves a certain accuracy level. ing because some of the earliest adopters example, provenance on credit
of the technology are likely to be in
history data might help reduce
More generally, analytic tools can evalu- domains where critical decisions depend
delays involved in getting a
ate (measure) metrics for provenance. on provenance data. Thus, the impact
mortgage despite errors in credit
of mistaken provenance could be large.
reports.
Cross-checking provenance with
archived file modifications in environ- Potential testbed applications should be Depository services; title history;
ments that log changes in detail could considered, such as the following: personnel clearance systems.
References
[Bos2005] R. Bose and J. Frew. Lineage retrieval for scientific data processing: a survey. ACM Computing Surveys,
37(1):1-28, 2005.
[Fos2002] I.T. Foster, J.-S. Voeckler, M. Wilde, and Y. Zhao. Chimera: A virtual data system for representing, querying,
and automating data derivation. In Proceedings of the 14th Conference on Scientific and Statistical Database
Management, pp. 37-46, 2002.
[Fre2005] J. Frew and R. Bose. Earth System Science Workbench: A data management infrastructure for earth science
products. In Proceedings of the 13th Conference on Scientific and Statistical Database Management, p. 180,
2001.
[Hey2001] A. Heydon, R. Levin, T. Mann, and Y. Yu. The Vesta Approach to Software Configuration Management.
Technical Report 168, Compaq Systems Research Center, Palo Alto, California, March 2001.
PROVENANCE 81
[OPM2007] L. Moreau, J. Freire, J. Futrelle, R.E. McGrath, J. Myers, and P. Paulson. The Open Provenance Model.
Technical report, ECS, University of Southampton, 2007 (http://eprints.ecs.soton.ac.uk/14979/).
[Pan03] C. Pancerella et al. Metadata in the collaboratory for multi-scale chemical science. In Proceedings of the
2003 International Conference on Dublin Core and Metadata Applications, 2003.
[SPI2007] M.M. Gioioso, S.D. McCullough, J.P. Cormier, C. Marceau, and R.A. Joyce. Pedigree management and
assessment in a net-centric environment. In Defense Transformation and Net-Centric Systems 2007. Proceedings
of the SPIE, 6578:65780H1-H10, 2007.
[TAP2009] First Workshop on the Theory and Practice of Provenance, San Francisco, February 23, 2009 (http://www.
usenix.org/events/tapp09/).
[Wid2005] J. Widom. Trio: A system for integrated management of data, accuracy, and lineage. In Proceedings of the
Second Biennial Conference on Innovative Data Systems Research, Pacific Grove, California, January 2005.
[Zha2004] J. Zhao, C.A. Goble, R. Stevens, and S. Bechhofer. Semantically linking and browsing provenance logs for
e-science. In Proceedings of the 1st International Conference on Semantics of a Networked World, Paris, 2004.
82 PROVENANCE
Current Hard Problems in INFOSEC Research
10. Privacy-Aware Security
BACKGROUND
The need to prove things about oneself (for example, proof of residence)
Various degrees of anonymity (protection of children online, victims of
crime and disease, cash transactions, elections)
Enabling limited information disclosure sufficient to guarantee security,
without divulging more information than necessary
Identity escrow and management
Multiparty access controls
Privacy-protected sharing of security and threat information, as well as
audit logs
Control of secondary reuse
Remediation of incorrect information that is disclosed, especially if done
without any required user approval
Effective, appropriate access to information for law enforcement and
national security
Medical emergencies (for example, requiring information about allergic
reactions to certain medications)
83
available to external media (via printers, information bureaus) is also bidding on a job, engaging in a
e-mail, wireless emanations, and so on), needed. collaborative venture, pursuing
and has come primarily outside the Organizations do not want mergers, and the like.
purview of authentication, computer proprietary information disclosed Social networks need means
access controls, audit trails, and other for other than specific agreed to share personal information
monitoring on the originating systems. purposes. within a community while
Research communities (e.g., protecting that information from
The central problem in privacy-aware abuse (such as spear-phishing).
security is the tension between compet- in medical research and social
ing goals in the disclosure and use of sciences) need access to accurate, Governments need to collect
private information. This document specific, and complete data for and selectively share information
takes no position on what goals should such purposes as analysis, testing for such purposes as census,
be considered legitimate or how the hypotheses, developing potential disease control, taxation, import/
tension should be resolved. Rather, treatments/solutions. export control, and regulation of
the goal of research in privacy-aware Law enforcement requires commerce.
security is to provide the tools necessary access to personal information to
to express and implement trade-offs conduct thorough investigations. What is the current state of
between competing legitimate goals National security/intelligence practice?
in the protection and use of private needs to detect and prevent
information. terrorism and hostile activity by Privacy-aware security involves a complex
nation-states and nonstate actors mix of legal, policy, and technological
Who are the potential while maintaining the privacy considerations. Work along all these
of U.S. persons and coalition dimensions has struggled to keep up
beneficiaries? What are their
partners. with the pervasive information sharing
respective needs?
that cyberspace has enabled. Although
Financial sector organizations
The beneficiaries for this topic are many the challenges have long been recog-
need access to data to analyze for
and widely varied. They often have nized, progress on solutions has been
indicators of potential fraud.
directly competing interests. An exhaus- slow, especially on the technology side.
tive list would be nearly impossible to Health care industries need At present, there are no widely adopted,
produce, but some illustrative examples access to private patient uniform frameworks for expressing and
include the following: information for treatment enforcing protection requirements for
purposes, billing, insurance, and private information while still enabling
Individuals do not generally reporting requirements. sharing for legitimate purposes. On the
want to reveal any more private technology side, progress has been made
Product development and
information than absolutely in certain application areas related to
marketing uses data mining
necessary to accomplish a privacy. Examples of privacy-enhancing
to determine trends, identify
specific goal (transaction, technologies in use today include the
potential customers, and tune
medical treatment, etc.) and following:
product offerings to customer
want guarantees that the
needs.
information disclosed will be Access controls (e.g., discretion-
used only for required and Business development, ary and mandatory, role-
authorized purposes. The ability partnerships, and based, capability-based, and
to detect and correct erroneous collaborations need to selectively database management system
data maintained by other reveal proprietary data to a authorizations) attempt to limit
organizations (such as credit limited audience for purposes of who can access what information,
84 PRIVACY-AWARE SECURITY
but they are difficult to configure Minimizing data retention time parties, or providing the ability later
to achieve desired effects, are appropriately to identify the misusers. A significant
often too coarse-grained, and challenge to the DRM approach is the
Protecting data in transmission
may not map well to actual development of an indisputable defini-
and storage (e.g., with
privacy and data use policies. tion of who controls the distribution.
encryption)
Encrypted storage and For example, should medical informa-
Conducting sensible risk analyses tion be controlled by the patient, by
communications can prevent
wholesale loss or exposure of Auditing of access audit logs doctors, by nurses, by hospitals, or by
sensitive data but do very little to (actually examining them, not insurance companies, or by some com-
prevent misuse of data accessed bination thereof? Each of them may be
just keeping them)
within allowed privileges or the originator of different portions of
Privacy policy negotiation and the medical information. Information
within flawed system security.
management provenance (Section 9) interacts with
Anonymous credential systems
privacy in defining the trail of who did
may enable authorization without what with the medical information, and
necessarily revealing identity (for What is the status of current
research? both interact with system and informa-
example, Shibboleth [Shib]). tion integrity.
Anonymization techniques, Security with privacy appears to require
such as mix networks, onion establishment of fundamental trust Many examples of ongoing or planned
routing, anonymizing proxy structures to reflect demands of privacy. privacy-related research are of interest
servers, and censorship-resistant It also requires means for reducing the here. For example, the following are
access technology, attempt risks of privacy breaches that can occur worth considering. NSF Trustworthy
to mask associations between (accidentally or intentionally) through Computing programs have explicitly
identities and information the use of technologies such as data included privacy in recent solicitations
content. mining. Ideas for reconciling such (http://www.nsf.gov/funding/). Some
One-time-use technologies, technologies in this context include research projects funded by the National
such as one-time authenticators privacy-aware, distributed association- Research Council Canada are also
and smart cards, can also rule mining algorithms that preserve relevant (http://iit-iti.nrc-cnrc.gc.ca/
contribute. privacy of the individual sites, queries on r-d/security-securite_e.html), as are
encrypted data without decrypting, and British studies of privacy and surveil-
At the same time, there are known best a new formulation to address the impact lance, including a technology roadmap
practices that, if consistently adopted, of privacy breaches that makes it possible (http://www.raeng.org.uk/policy/
would also advance the state of the prac- to limit breaches without knowledge of reports/pdf/dilemmas_of_privacy_and_
tice in privacy-preserving information original data distribution. surveillance_report.pdf ).
sharing. These include
Digital rights management (DRM) Other privacy related research includes
Use of trustworthy systems and techniques, while not currently applied the following:
sound system administration, for privacy protection, could be used
with strong authentication, Microsoft Research database
to protect information in such diverse
settings as health care records and cor- privacy: (http://www.research.
differential access controls, and
porate proprietary data, allowing the microsoft.com/jump/50709
extensive monitoring
originator of the information to retain and http://www.microsoft.com/
Adherence to the principle of mscorp/twc/iappandrsa/research.
some degree of access control even after
least privilege the information has been given to third mspx)
PRIVACY-AWARE SECURITY 85
Project Presidio: collaborative See also (http://www.itaa.org/ Oxley; HIPAA), and economics
policies and assured information infosec/faith.pdf ) and (http:// and security (e.g., http://www.
sharing www.schneier.com/blog/ cl.cam.ac.uk/~rja14/econsec.
(http://www.projectpresidio.com) archives/2007/03/security_ html).
plus_p.html)
Stanford University Web Security
Research: private information What are the major research
retrieval (http://crypto.stanford. FUTURE DIRECTIONS gaps?
edu/websec/)
Following are some of the gaps in pri-
Security with Privacy ISAT On what categories can we vacy-aware security that need to be
briefing (http://www.cs.berkeley. subdivide the topic? addressed.
edu/~tygar/papers/ISAT-final- For purposes of a research and devel-
briefing.pdf ) opment roadmap, privacy-aware Selective disclosure and
information sharing can be usefully privacy-aware access
Naval Research Lab: Reputation
Sound bases are needed for
in Privacy Enhancing divided along the following categories,
directly mirroring the gaps noted above. selective disclosure through
Technologies (http://chacs.
See Table 10.1. techniques such as attribute-
nrl.navy.mil/publications/
based encryption, identity-based
chacs/2002/2002dingledine-
Selective disclosure and encryption, collusion-resistant
cfp02.pdf )
privacy-aware access to data: broadcast encryption, private
ITU efforts related to security, information retrieval (PIR), and
theoretical underpinnings and
privacy, and legislation: (http:// oblivious transfer.
system engineering.
www.itu.int/ITU-D/cyb/
How do we share data sets
publications/2006/research- Specification frameworks for
while reducing the likelihood
legislation.pdf ) providing privacy guarantees:
that arbitrary users can infer
DHS report on the ADVISE languages for specifying privacy
individual identification? (The
program (http://www.dhs.gov/ policies, particularly if directly
U.S. Census Bureau has long
xlibrary/assets/privacy/privacy_ implementable; specifications been concerned about this
rpt_advise.pdf ) for violations of privacy; and problem.)
UMBC Assured privacy detecting violations of privacy.
Data sanitization techniques are
preserving data mining, recipient Policy issues: establishing needed that are nonsubvertible
of DoD’s MURI award (http:// privacy policies, data correction, and that at the same time do not
ebiquity.umbc.edu/blogger/tag/ propagation of updates, privacy render analysis useless.
muri/) implications of data integrity. More generally, data quality
Anonymous communication This also includes legal (aspects must be maintained for research
(http://freehaven.net/anonbib) of current law that constrain purposes while protecting
Statistics research community, as technology development; privacy, avoiding profiling or
in the Knowledge Discovery and aspects of future law that could temporal analysis to deanonymize
Data Mining (KDD) conferences enable technology development; source data.
(http://sigkdd.org) questions of jurisdiction), Irreversible transformations
Framework for privacy metrics standards (best practices; privacy of content are needed that
[Pfi+2001]) standards analogous to Sarbanes- exhibit statistical characteristics
86 PRIVACY-AWARE SECURITY
consistent with the original data Policy issues Policies are needed for dealing
without revealing the original Distinctions between individual with privacy violations, detection
content. and group privacy are unclear. of violations, consequences of
Privacy and security for very large Release of bogus information violations, and remediation of
data sets does not scale easily— about individuals is poorly damage.
for example, maintaining privacy handled today. However, with
of individual data elements is stronger protection it becomes
What are some exemplary
difficult. more difficult to check validity of
information. problems for R&D on this
Associations of location with topic?
users and information may Information gathered from some
Several problem domains seem par-
require privacy protection, persons can allow probabilistic
ticularly relevant, namely, data mining
particularly in mobile devices. inference of information about
for medical research, health care
others.
Low-latency mix networks can records, data mining of search queries,
provide anonymization, but need Policies for data collection and census records, and student records at
further research. sharing with regard to privacy universities.
are needed, especially relating
Mechanisms to enforce retention
to what can be done with the
limits are lacking. What R&D is evolutionary and
private data. For example, who
what is more basic, higher
Sharing of security information are the stakeholders in genetic risk, game changing?
such as network trace data needs information? What policies are
privacy controls. needed for retention limits? Near term
Deriving requirements for
Specification frameworks Communications create further automating privacy policies:
Specification frameworks for privacy problems relating to learning from P3P
expressing privacy guarantees are identification of communication
Policy language development
weak or missing. In particular, sources, destinations, and
specification and enforcement of patterns that can reveal Implement best practices
context-dependent policies for information, even when other Research into legal issues in
data sharing and use are needed. data protections are in place. communications privacy
PRIVACY-AWARE SECURITY 87
Medium term considerable commitment from govern- have worked on this, as in
Anonymous credentials ment funding agencies, corporations, determining statistical similarity
Role-based Access Control and application communities such as of purposely fuzzed data sets.)
(RBAC) health care to ensure that the research is How many queries are needed
relevant and that it has adequate testbeds to get to specific data items
Attribute-based encryption
for practical applications. It will also for individuals in databases
Distributed RBAC: no central that purport to hide such
engender considerable scrutiny from
enforcement mechanism required information?
the privacy community to ensure that
Protection against excess the approaches are adequately privacy Adversary work factors to violate
disclosure during inference and preserving. privacy.
accumulation
Risk analysis: This has been
Application of DRM techniques
Measures of success applied to security (albeit
for privacy
somewhat haphazardly). Can risk
Searching encrypted data A goal for addressing concerns regarding
analysis be effectively applied to
without revealing the query; both data mining and identity theft is to
privacy?
more generally, computation on quantify users’ ability to retain control
encrypted data of sensitive information and its dissemi- Costs for identity-fraud
nation even after it has left their hands. insurance.
Long term
For data mining, quantitative measures Black market price of stolen
Private information retrieval
of privacy have been proposed only identity.
(PIR)
recently and are still fairly primitive. For
Multiparty communication example, it is difficult to quantify the
effect of a release of personal informa-
What needs to be in place for
Use of scale for privacy
tion without knowing the full context test and evaluation?
Resistance to active attacks for
deanonymizing data with which it may be fused and within Access to usable data sets is important,
which inferences may be drawn. Evalu- for example,
Developing measures of privacy ation and refinement of such metrics are
Census data (see http://www.
Game changing certainly in order.
Limited data retention
fedstats.gov)
Useful realistic measures are needed for Google Trends
Any two databases should be
capable of being federated evaluating privacy and for assessing the PREDICT (e.g., network traffic
without loss of privacy (privacy relative values of information. data; http://www.predict.org)
composability) Medical research data
Possible measures of progress/success
Low-latency private include the following: E-mail data (e.g., for developing
communications resistant to spam filters)
timing attack Rate of publication of privacy-
breach stories in the media. Possible experimental testbeds include
Resources Database measures: Can we the following:
simulate a database without
real data? How effective would Isolated networks and their users
This topic is research-intensive, with
considerable needs for testbeds demon- approaches be that cleanse data Virtual societies
strating effectiveness and for subsequent by randomization? Can we
technology transfer to demonstrate the use such approaches to derive In addition, privacy Red Teams could
feasibility of the research. It will require metrics? (Statistical communities be helpful.
88 PRIVACY-AWARE SECURITY
References
[Bri+1997] J. Brickell, D.E. Porter, V. Shmatikov, and E. Witchell. Privacy-preserving remote diagnostics, CCS ’07,
October 29 – November 2, 2007.
[Pfi+2001] A. Pfitzmann and M. Köhntopp. Anonymity, unobservability, and pseudonymity: A proposal for terminology.
In Designing Privacy Enhancing Technologies, pp. 1-9, Springer, Berlin/Heidelberg, 2001.
[Rab1981] M. Rabin. How to exchange secrets by oblivious transfer. Technical Report TR-81, Aiken Computation
Laboratory, Harvard University, 1981.
Many additional references can be found by browsing the URLs noted above in the text of this section.
PRIVACY-AWARE SECURITY 89
Current Hard Problems in INFOSEC Research
11. Usable Security
BACKGROUND
People use systems to perform various tasks toward achieving some goal. Unless the
tasks at hand are themselves security related, having to think about security inter-
feres with accomplishing the user’s main goal. Security as it is typically practiced in
today’s systems increases complexity of system use, which often causes confusion and
frustration for users. When the relationship between security controls and security
risks is not clear, users may simply not understand how best to interact with the
system to accomplish their main goals while minimizing risk. Even when there is
some appreciation of the risks, frustration can lead users to disregard, evade, and
disable security controls, thus negating the potential gains of security enhancements.
Security must be usable by persons ranging from nontechnical users to experts and
system administrators. Furthermore, systems must be usable while maintaining
security. In the absence of usable security, there is ultimately no effective security.
The need for usable security and the difficulties inherent in realizing adequate
solutions are increasingly being recognized. In attempting to address the chal-
lenges of usability and security, several guiding principles are worth considering.
Furthermore, when we refer here to usable security, we are really concerned with
trustworthy systems whose usability has been designed into them through proactive
requirements, constructive architectures, sound system and software development
practices, and sensible operation. As observed in previous sections, almost every
system component and every step in the development process has the potential to
compromise trustworthiness. Poor usability is a huge potential offender.
90
Security issues must be made as trans- it or switch to an alternative system that productivity. Security is poorly
parent as possible. For example, security is more user friendly but less secure. understood by nonexperts, and the
mechanisms, policies, and controls must consequences of disabled or weakened
be intuitively clear and perspicuous to What are the potential security controls are often indirect and
all users and appropriate for each user. not immediately felt; and the worst
threats?
In particular, the relationships among effects may be felt by those not directly
security controls and security risks must The threats from the absence of usable involved (e.g., credit card fraud), leading
be presented to users in ways that can be security are pervasive and mostly noted users to question the value of having
understood in the context of system use. in the above discussion. However, these security technology at all.
threats are somewhat different from
Users must be considered as fundamen- those in most of the other 10 topics—in At the same time, consciousness of secu-
tal components of systems during all that the threats are typically more likely rity issues is becoming more widespread,
phases of the system life cycle. Different to arise from inactions, inadvertence, and technology developers are paying
assumptions and requirements pertain- and mistakes by legitimate users. On increasing attention to security in their
ing to users’ interactions with systems the other hand, threats of misuse by products and systems. However, usabil-
must be made explicit to each type outsiders and insiders similar to those ity in general appears not to be much
of user—novices, intermittent users, in the other topics can certainly arise as better understood by software practi-
experts, and system administrators, to a result of the lack of usability. tioners than security is. This situation
name a few. In general, one-size-fits-all makes the problem of usable security
approaches are unlikely to succeed. Who are the potential even more challenging, since it com-
bines two problems that are difficult to
beneficiaries? What are their
Relevant education about security prin- solve individually.
respective needs?
ciples and operational constraints must
be pervasive. Security issues can never Although the problem of achieving Usability of systems tends to decrease as
be completely hidden or transparent. usable security is universal—it affects attempts are made to increase security
There will always be the possibility of everyone, and everyone stands to benefit and, more broadly, trustworthiness.
conflict between what users might want enormously if we successfully address Many current security systems rely on
to accomplish most easily and the secu- usability as a core aspect of security—it humans performing actions (such as
rity risks involved in doing so. Helping affects different users in different ways, typing passwords) or making decisions
users to understand these trade-offs must depending on applications, settings, (such as whether or not to accept an
be a key component of usable security. policies, and user roles. The guiding SSL certificate). For example, one e-mail
principles may indeed be universal, but system requires that users reauthenticate
Security metrics must take usability into as suggested above there is certainly every 8 hours to assure that they are
account. Although one might argue that no general one-size-fits-all solution. actually the authorized person. This
a system with a certain security control Examples of different categories of users requirement is a direct counter to system
is in principle more secure than an oth- and ways in which they are affected by usability. For example, some web brows-
erwise equivalent system without that problems in usable security are shown ers warn users before any script is run.
control—for example, a web browser in Table 11.1. But users may still browse onto a web
that supports client/server authentica- server that has scripts on every page,
tion vs. one that does not—the real What is the current state of causing pop-up alerts to appear on each
security may in fact be no greater (and page.
practice?
possibly even less) in a system that
implements that security control, if its Although the importance of secu- Many of the potential impacts of security
introduction compromises usability to rity technology is widely recognized, that is not usable involve increased sus-
the point that users are driven to disable it is often viewed as a hindrance to ceptibility to social-engineering attacks.
USABLE SECURITY 91
TABLE 11.1: Beneficiaries, Challenges, and Needs
This might be an adversary sending an A few illustrative examples from the was cumbersome to configure, even
e-mail “this configuration change makes current state of the practice may help for experts, and imposed significant
your system more usable” to “this patch illuminate challenges in usable security system overhead. Key management
must be manually installed”. But it also and identify some promising directions was typically either cumbersome, or
involves attackers who gain the trust of from which broader lessons may be reduced to one key or perhaps just a
users by helping those users cope with drawn. few. Many newer operating systems
difficult-to-use systems. Thus, resistance now offer ready-to-use full-disk
to social engineering must be built into Somewhat positive examples of usable encryption out of the box, requiring
systems, and suitable requirements and security might include transparent little more than a password from the
metrics included from the outset of any file-system encryption. When first user, while imposing no noticeable
system development. introduced, file encryption technology performance penalty.
92 USABLE SECURITY
Other, more mixed examples illustrate understanding, leading to the Net Trust). Although this seem
how security technology still falls short frustration effects noted earlier. to enhance usability, many users
in terms of usability: Mail authentication. There may not adequately understand
are mechanisms to authenticate the implications of accepting
Passwords. Security pitfalls of trust information from systems
poorly implemented password senders of valid e-mails, such
as SPF (sender permitted that may be unknown to those
schemes have been extensively users. They are also unlikely to
documented over the years. from). DomainKeys Identified
Mail (DKIM) is an e-mail understand fully what factors
When users must resort to might be helpful, harmful, or
writing them on slips of paper authentication technology that
allows e-mail recipients to verify some of each.
or storing them unencrypted
on handheld devices, the risk whether messages that claim to CAPTCHA systems. A
of password exposure may have been sent from a particular CAPTCHA (Completely
outweigh the increased security of domain actually originated there. Automated Public Turing test
strong passwords. Nevertheless, It operates transparently for end to tell Computers and Humans
passwords are often simplistically users and makes it easier to detect Apart) is a challenge-response
believed to be a usable security possible spam and phishing mechanism intended to ensure
mechanism, and elaborate attacks, both of which often rely that the respondent is a human
procedures are promulgated on domain spoofing. Some large and not a computer. CAPTCHAs
purporting to define sensible e-mail service providers now are familiar to most web users
password practices (with respect support DKIM. as distorted images of words or
to frequency of changing, not Client-side certificates. Most other character sequences that
using dictionary words, including must be input correctly to gain
web browsers and e-mail
nonalphabetic characters, etc.). access to some service (such
applications in widespread use
Tools that help users select good as a free e-mail account). To
today support user authentication
passwords and manage their make a CAPTCHA effective
via certificates based on public-
passwords have been touted for distinguishing humans from
key cryptography. However, the
to enhance both usability and computers, solving it must be
technology is not well understood
security. However, to make difficult for computers but
by nonexpert users, and typically
passwords more effective for relatively easy for humans. This
the integration of client-side balance has proven difficult to
stronger security, they must be so certificate authentication into
long and so complex that users achieve, resulting in CAPTCHAs
applications makes the use and that are either breakable by
cannot remember them, which
management of these certificates computers or too difficult for
seriously compromises usability.
opaque and cumbersome for humans. Another challenge is
Security pop-up dialogs. No users. to produce CAPTCHAs that
matter how much effort is put accommodate users with special
The SSL lock icon. This
into making security controls needs.
approach gives the appearance
automated and transparent,
of security, but its limitations Not accounting for cultural
there are inevitably situations
are not generally understood. differences and personal
that require users to make
For example, it may be totally disabilities. For example,
security-related decisions. Today,
unfortunately, user involvement spoofed. Its presence or absence people of one ethnic group tend
appears to be required too may also be ignored. to have difficulty recognizing
often and usually in terms that “Web of trust”-like approaches different faces of people in
nontechnical users have difficulty to certificate trust (e.g., Google, other ethnic groups, which
USABLE SECURITY 93
could cause usability differences Overloading of security this area. An example of a new
in authentication. Similarly, attributions in the context direction might be making Tor
CAPTCHAs could be culture of domain-validation more usable for administration.
dependent. In addition, people certificates. People tend to trust Highlighting important changes
with a prosopagnosia disorder certificates too much or else are
to systems (e.g., operating
have difficulty distinguishing overwhelmed by their presence.
systems, middleware, and
between different people by sight. Revocation. Dealing with
This would seriously impair their applications) that could improve
change is typically difficult, but security and usability (rather than
ability to distinguish among usability may be impaired when
different pictorial authenticators just one).
revocation is required. If not
and CAPTCHAs. Reevaluating decisions/trade-offs
carefully designed into systems
Policies and centralized in advance with usability and made in past systems. A sense of
administration. Lack of user understandability in mind, history in cybersecurity is vital
flexibility is common. On the mechanisms for revocation but is too often weak.
other hand, it is generally unwise are likely to have unintended
to expect users to make security/ One Laptop Per Child Bitfrost
consequences.
usability trade-off evaluations. security model.
Federated identity What is the status of current Integration of biometrics with
management. Cross-domain research? laptops (e.g., fingerprint, facial
access is complex. Simplistic recognition); this is in practice
Following is a brief summary of some
approaches such as single sign-
current research, along with gaps. For today, for better or worse. It
on can lead to trust violations.
background, see [SOU2008]. may be good for administration,
Conversely, managing too
many passwords is unworkable. but perhaps not so good from
Usable authentication. For
More work is needed on access the point of view of user
example, visual passwords and
cards such as the CAC system, understanding.
various other authentication
DoD’s Common Access Card,
approaches exist but need much
(which combines authentication,
encryption of files and e-mail, further work to determine
whether they can be used FUTURE DIRECTIONS
and key escrow) and other such
systems to identify security effectively. At present, they are
vulnerabilities. In all such often very difficult to use and On what categories can we
systems, usability is critical. seem unlikely to scale well to subdivide the topic?
PGP, S/MIME, and other large numbers of passwords. We consider the following three cat-
approaches to secure e-mail. User security. Currently funded egories as a useful subdivision for
Many past attempts to security-related usability research formulating a research roadmap for
encapsulate encryption into mail includes the CMU CyLab Usable usability and security:
environments have been hindered Privacy and Security Laboratory
by the lack of seamless usability. Interface design (I)
(CUPS), and Stanford University
Links. Phishing, cross-site Science of evaluation for usable
work on Web integrity. A list of
scripting, and related problems security (E)
CUPS projects with descriptions
with bogus URLs are laden with and papers can be found at Tool development (T)
risks. URLs may seem to increase
http://cups.cs.cmu.edu.
usability, but malicious misuse The following are second-level bins, with
of them can seriously diminish Ease of administration. descriptors defining their relevance to I,
security. Relatively little research exists in E, and T:
94 USABLE SECURITY
Principles of usable security; a designing for and evaluating usability security of novel approaches and out-
taxonomy of usable security (E) of computer systems. However, only of-the-box thinking in usable security.
Understanding users and their a small fraction of this research has
interactions with security controls focused on usability specifically as it There is a need to increase knowledge of
(IET) relates to security. At the same time, usability among security practitioners.
security research tends to focus on spe- A common lament in industry is that
Usable authentication and
cific solutions to specific problems, with programmers are too rarely taught how
authorization technology (IT) little or no regard for how those solu- to create secure programs, but even
Design of usable interfaces for tions can be made practical and, most those who do receive such training are
security, with resistance to social importantly, transparent to users and unlikely to be taught how to provide
engineering (I) system administrators. To the extent that both security and usability simultane-
Development tools that assist in security practitioners do consider the ously. Just as with security, usability is
the production of systems that practical implications of their proposed not a property that can easily be added
are both more secure and more solutions, the result is often a new or to existing systems, and it is not a prop-
usable (T) modified user interface component for erty that one member of a large team can
configuring and controlling the security provide for everyone else. The implica-
Adapting legacy systems
technology, which does little to address tion is that a large body of designers,
Building new systems the fundamental problem that most programmers, and testers needs to have a
Usable security for embedded users cannot and do not want to be much deeper understanding of usability.
and mobile devices (IET) responsible for understanding and man- Adding usability to existing curricula
aging security technology; they simply would be a good start but could not
Evaluation approaches and
want it to do the right thing and stay be expected to pay dividends for years
metrics for usability and
out of the way. to come. Methods to increase under-
security (E)
standing of usability among software
User education and In short, usable security is not funda- developers already working in industry
familiarization with security mentally about better user interfaces to are equally necessary.
issues and technology (IE) manage security technology; rather, it is
User feedback, experience (e.g., about evaluating security in the context We need to identify a useful framework
usability bug reports) (E) of tasks and features and of the user, and for discussing usability as it relates to
Security policies (especially, rearchitecting it to fit into that context. security, such as the following:
implementation of them) that
It is important to note the inherently Research on usable security
increase both usability and
interdisciplinary nature of usability “out of the box” (security
security (ET)
and security. Security researchers and transparency).
Tools for evaluating security
practitioners cannot simply expect that Identification of the most useful
policies
the HCI experts will fix the usabil- points in the R&D pipeline at
Market creation for usable ity problem for trustworthy systems.
security technology which to involve users in the
Addressing the problem adequately
development of trustworthy
will require close collaboration between
systems.
members of the security and usabil-
What are the major research
ity research communities. One goal Research into the question of
gaps?
is to develop the science of usability how to evaluate usability as it
Human-computer interaction (HCI) as applied to security. For example, relates to security. Here we would
research has made strides in both we need to have ways to evaluate the expect significant contributions
USABLE SECURITY 95
from HCI research that has Lessons from the automotive effects they want to achieve but are not
already developed methodologies industry experts in system administration. In
for evaluating usability. addition, if a user decides to modify the
What are some exemplary access configuration, how could that be
System architectures that starkly
problems for R&D on this done in a usable way, while achieving
reduce the size and complexity only the desired modifications (e.g., not
topic?
of user interfaces, perhaps by making access to sensitive data either
simplifying the interface, hiding One exemplary problem is protecting more or less restrictive than intended)?
the complexity within the users against those who pose as someone
interface, providing compatible else on the Internet. Techniques like What R&D is evolutionary and
interfaces for different types of certificates have not worked. Alerts from
what is more basic, higher
users (such as administrators), or browsers and toolbars and other add-ins
risk, game changing?
various other strategies, without about suspicious identities of websites or
losing the ability to do what must e-mail addresses do not work, because In the short term, the situation can be
be done especially in times of users either do not understand the alerts significantly improved by R&D that
system or component failures. or do not bother using the tools. Note focuses on making security technology
that, if used properly, these techniques work sensibly “out of the box”—ideally
The ability to reflect physical- could be effective. The failure is in their with no direct user intervention. More
world security cues in computer lack of easy usability. The goal here basic, higher-risk, game-changing
systems. should be not just to find any alternative research would be to identify funda-
Consideration of usability approach, but rather to find approaches mental system design principles for
from a data perspective; for that can work well for ordinary users. trustworthy systems that minimize
example, usability needs can direct user responsibility for trustwor-
drive collection of data that can Another exemplary problem is the secure thy operation.
lead to security problems (PII as handling of e-mail between an arbitrary
authenticators, for example) sender and an arbitrary receiver in a Near term
usable way. Judging from the limited Informing the security research
Hard problems use of encrypted e-mail today, existing community on the results
Usable security on mobile devices approaches are not sufficiently usable. obtained in the usable security
Yet, users are regularly fooled into believ-
Usable mutual authentication community on the design and
ing that forged e-mail is actually from
execution of usability studies
Reusable “clean” abstractions for the claimed sender. It is only a matter
[Cra+2005]
usable security of time before serious problems are
encountered because of e-mail traveling Developing a bibliography of
Usable management of access
across its entire path unencrypted and best practices and developing
controls a community expectation that
unauthenticated. For a general discus-
Usable secure certificate services sion on why cryptography is typically security researchers will use them
Resistance to social engineering not very easily used, see [Whi+1999]. in their work
Identifying the common
Other areas we might draw on Another possibility is configuring an characteristics of “good” usable
Usability in avionics: reducing office environment so that only the
security (and also common
people who should have access to sensi-
the cognitive load on pilots characteristics of usability done
tive data can actually access it—so that
Lessons from safety in general, badly)
such a configuration can be accom-
especially warnings science plished by users who understand the Developing a useful framework
96 USABLE SECURITY
for discussing usability (in the Measures of success as part of all applicable research
context of security) Meaningful metrics for usable security in other areas.
Developing interdisciplinary must be established, along with Guidelines/How-Tos for
connections between the security generic principles of metrics. These usability studies. (See Garfinkel
and HCI communities (relates to must then be instantiated for specific & Cranor [Cra+2005].)
the first bullet above) systems and interfaces. We need to A “Usable Security 101” course,
Identifying ways of involving measure whether and to what extent including how to develop and
users in the security technology increased usability leads to increased
evaluate usable systems.
R&D process security, and to be able to find “sweet
spots” on the usability and security Standardized testbed for
Medium term curves. Usable security is not a black- conducting usability studies
Usable access control mechanisms and-white issue. It must also consider (perhaps learning from DETER
(such as a usable form of RBAC) returns on investment. and PlanetLab).
Usable authentication Anonymous reporting system
We do not have metrics that allow
Developing a common within a repository for usability
direct comparison of the usability of
framework for evaluating problems (perhaps learning from
two systems (e.g., we cannot say defini-
usability and security the avionics field).
tively that system A is twice as usable as
Long term system B), but we do perhaps have some
Composability of usable
To what extent can we test
well-established criteria for what consti-
components: can we put together tutes a good usability evaluation. One real systems?
good usable components for possible approach would be to develop Usability studies need to be based on real
particular functions and get a usable solution for one of the exemplar systems. They need not be live systems
something usable in the total problems and demonstrate both that used to conduct actual business, but
system?
users understand it and that its adoption they need to be real in the sense that they
Tools, frameworks, and standards reduces the incidence or severity of the offer the same interfaces and operate in
for usable security associated attack. For example, demon- the same environments as such systems.
strate that a better anti-phishing scheme
Resources reduces the frequency with which users Usability competitions might be con-
follow bogus links. Admittedly, this sidered (e.g., who can come up with
Designing and implementing systems would demonstrate success on only a the most usable system for application/
with usable security is an enormously single problem, but it could be used to function X that satisfies security require-
challenging problem. It will necessitate show that progress is both possible and ments Y). A possible analogy would
embedding requirements for usability demonstrable, something that many be to the challenge of creating a more
in considerable detail throughout the people might not otherwise believe is usable shopping cart. Building test and
development cycle, reinforced by exten- true about usable security. evaluation into the entire research and
sive evaluation of whether it was done development process is essential.
adequately. If those requirements are What needs to be in place for
incomplete, it could seriously impair test and evaluation?
the resulting usability. Thus, significant
resources—people, processes, and soft- Several approaches could help:
ware development—need to be devoted
Test and evaluation for usability
to this challenge.
USABLE SECURITY 97
References
[Cra+2005] L.F. Cranor and S. Garfinkel, editors. Security and Usability: Designing Secure Systems That People Can Use.
O’Reilly Media, Inc., Sebastopol, California, 2005
(http://www.oreilly.com/catalog/securityusability/toc.html).
[Joh2009] Linda Johansson. Trade-offs between Usability and Security. Master’s thesis in computer
science, Linkoping Institute of Technology Department of Electrical Engineering,
LiTH-ISY-EX-3165, 2001 (http://www.accenture.com/xdoc/sv/locations/sweden/
pdf/Trade-offs%20Between%20Usiability%20and%20Security.pdf ).
[Sun+09] J. Sunshine, S. Edelman, H. Almuhimedi, N. Atri, and L.F. Cranor. Crying Wolf:
An empirical study of SSL warning effectiveness. USENIX Security 2009.
[Whi+1999] Alma Whitten and J.D. Tygar. Why Johnny can’t encrypt: A usability evaluation of PGP 5.0.
In Proceedings of the 8th USENIX Security Symposium, Washington, D.C., August 23–26, 1999,
pp. 169–184 (http://www.usenix.org/publications/library/proceedings/sec99/whitten.html).
98 USABLE SECURITY
Appendix A
Appendix A. Interdependencies Among Topics
Y:
X: Topic 1 2 3 4 5 6 7 8 9 10 11 H M L
1: Scalable
- H H H H H H H H H H 10 0 0
Trustworthiness
2: Enterprise
M - H H H H H H H H H 9 1 0
Metrics
3: Evaluation
H M - H H H H H H M H 8 2 0
Life Cycle
4: Combatting
H M M - H M M H M M H 4 6 0
Insider
5: Combatting
H M M M - M H H M M H 4 6 0
Malware
6: Global ID
H M M H H - M H H H H 7 3 0
Management
7: System
H M M H M M - M M L H 3 6 1
Survivability
8: Situational
M M M H H M H - M M H 4 6 0
Awareness
9: Provenance M M M M H M M H - H H 4 6 0
10: Privacy-
M M L H L H M H M - H 4 4 2
Aware Security
11: Usable
M M M M M M M M M M - 0 10 0
Security
H 5 1 2 7 7 4 5 8 4 4 10 *57
M 5 9 7 3 2 6 5 2 6 5 0 *50
L 0 0 1 0 1 0 0 0 0 1 0 *3
A1
Almost every topic area has some poten- other topic areas, most obviously evolution must also be driven by
tial influence and/or dependence on the including enterprise-level metrics feedback from those other topics.
success of the other topics, as summa- and the system evaluation life Combatting Insider Threats
rized in the table. The extent to which cycle (which together could drive (topic 4) will share some
topic X can contribute to topic Y is rep- the definitions and assessments common benefits with
resented by the letter H, M, or L, which of trustworthiness), global-scale Combatting Malware and
indicate that Topic X can make high, identity management, system Botnets (topic 5), particularly
medium, or low contribution to the survivability, and usable security,
with respect to the development
success of Y. These ratings, of course are but also including work on
and systematic use of fine-
very coarse and purely qualitative. On combatting insider misuse and
the other hand, any finer-grained ratings grained access controls and
combatting malware.
are not likely to be useful in this context. audit trails. However, note
Enterprise-Level Metrics that combatting insider threats
The purpose of the table is merely to
(ELMS) (topic 2) is particularly can contribute highly (H) to
illustrate the pervasive nature of some
interesting. It is one topic to combatting malware, although
relatively strong interdependencies.
which all other topic areas the reverse contributions may
A preponderance of H in a row indicates must contribute to some extent, be somewhat less (M). Both
that the corresponding row topic is of because each other topic area of these topics have significant
fundamental importance to other topics. must explicitly include metrics benefits for the other topics.
That is, it can contribute strongly to the specific to that area. In the other Also, Situational Understanding
success of most other topics. direction of dependence, the (topic 8) is fundamental to both,
mere existence of thorough and and clearly is relevant to both
Examples: rows 1 (SCAL: all H), well-conceived enterprise-level insider threats and malware.
2 (METR: 9 H), 3 (EVAL: 8 H). metrics would drive R&D in Thus, the potential synergies here
the individual topic areas to will be very important.
The preponderance of H in a column help them contribute to the
Global-Scale Identity
indicates that the corresponding column satisfaction of the enterprise-
Management (topic 6) and
topic is a primary beneficiary of the level metrics. This can also
Provenance (topic 9) can be
other topics. inspire the composability of the
mutually beneficial: the former
evaluation of topic metrics into
can significantly enhance the
Examples: columns 11 (USAB: 10 H), the evaluation of the enterprise-
latter (H), whereas the latter can
8 (SITU: 8 H), 4 (INSI: 7 H), level metrics, which is a major
enhance the former somewhat
5 (MALW: 7 H). research need. The enterprise-
less (M), although it can increase
level metrics topic area thus
the assurance of the former.
Not surprisingly, the table is not sym- interacts bidirectionally with all
metric. However, there are numerous the other topics, as exhibited by Survivability of Time
potential synergies here, such as the the H entries in that row and the Critical Systems (topic 7) is
following: M entries in that column. strongly linked with Scalable
Trustworthy Systems (topic 1),
Scalable Trustworthy Systems The System Evaluation Life
because survivability is one
(topic 1) is the one topic that Cycle (topic 3) is similar to of the fundamental aspects of
can highly enhance all the other Enterprise-Level Metrics (topic 2) trustworthiness. In addition,
topics. However, its success could in this context. It is fundamental it is particularly relevant to
also derive significant benefits to trustworthiness in almost all combatting insider threats and
from advances in some of the the other topic areas, but its malware.
A2
INTERDEPENDENCIES AMONG TOPICS
Situational Understanding and development, and operation. Failure to following topics can also contribute to
Attack Attribution (topic 8) is satisfy any of these requirements can advances in this topic area.
important throughout. potentially undermine the trustworthi-
Enterprise-level metrics (that
Privacy-Aware Security ness of entire systems and indeed entire
enterprises. is, measures of trustworthiness
(topic 10) is somewhat of an that apply to systems and
outlier with respect to strong systems of systems as a whole):
dependence in both directions. To illustrate the pervasiveness of the
interdependencies summarized in Evaluation methodologies must
It is only moderately dependent allow composability of lower-
on other topics, and most other Table A.1, we consider the 11 topics,
in greater detail. For each topic, we layer metrics and the resulting
topics are only moderately evaluations. Formalization of
dependent on it. Nevertheless, consider first how success in the other
topic areas might contribute to that the ways in which metrics and
it is a very important and often evaluations can compose should
neglected broad topic area—one particular topic (that is, represented by
the corresponding column of the table), contribute to the composability
that is becoming increasingly of scalable systems and their
important as more applications and then consider how success in that
particular topic might benefit the other ensuing trustworthiness.
become heavily dependent on the
need for trustworthy computer 10 topics (represented by the corre- System evaluation life cycle:
systems. sponding rows of the table). These more Methodologies for evaluating
detailed descriptions are intended to be security should be readily
Usable Security (topic 11) is beneficial for readers who are interested applicable to trustworthy system
fundamental throughout. It can in a particular column or row. They also developments; evaluations must
strongly influence the success of amplify some of the concepts raised in themselves be composable and
almost all the other topics but is the 11 sections of this report. scalable. Similar to the enterprise-
also a critical requirement of each
level metrics topic, advances in
of those topics. Generic gains
evaluation methodologies can
in achieving usability will have Topic 1: Scalable Trustworthy contribute to the composability
enormous impact throughout, in Systems of trustworthy systems of systems.
both directions. This is one of We consider first how success in the
many examples of an iterative Combatting insider threats:
other topic areas could contribute to
symbiotic feedback loop, where Various advances here could
scalable trustworthy systems, and then
advances in usability will help benefit scalable trustworthy
how success in scalable trustworthy
other topics, and advances in systems, including policy
systems might benefit the other topic
other topics will help usability. development, access control
areas.
mechanisms and policies,
The low incidence of low-order inter-
containment and other forms of
dependencies in Table A.1 may at first What capabilities from other topic areas
isolation, compromise-resistant
seem odd. However, it may actually are required or would be particularly desir-
and compromise-resilient
be a testament to the relative impor- able for effective progress in this topic area?
operation, and composable
tance of each of the 11 topic areas
metrics and evaluations
and the mutual synergies among the Research on the theory and practice
topics, as well as the inherently holistic of scalable trustworthiness is essen- applicable to insider threats.
nature of trustworthiness [Neu2006], tial. Although some of that research Combatting malware: Advances
which ultimately requires serious atten- must result from the pursuit of scalable such as those in the previous
tion to all the critical requirements trustworthy systems per se, research topic relating to malware
throughout system architecture, system and development experience from the detection and prevention can
A4
INTERDEPENDENCIES AMONG TOPICS
Scalable trustworthy systems over insider misuse can also
Topic 2: Enterprise-Level
would help address remote help prevent or at least limit the
Metrics (ELMs)
access by logical insiders as deleterious effects of malware.
What capabilities from other topic areas
are required for effective progress in this well as local access by physical The prevention aspects are closely
topic area? insiders, by virtue of distributed related.
authentication, authorization,
Each of the other topic areas is expected Life cycle protection must
and accountability.
to define local metrics relevant to its account for the insider threat.
own area. Those local metrics are likely Situational understanding and
Survivability of systems can
to influence the enterprise-level metrics. attack attribution must apply to
be aided by knowledge of the
insiders as well as other attackers.
presence of potential malware or
How does progress in this area support This dependency implies that
of insiders who may have been
advances in others? synergy is required between
detected in potential misuse.
Proactive establishment of sensible misuse detection systems and the
enterprise-level metrics would natu- access controls used to minimize
Topic 5: Combatting Malware
rally tend to drive refinements of the insider misuse.
and Botnets
local metrics.
Identity management relates What capabilities from other topic areas
to the accountability aspects of are required for effective progress in this
Topic 3: System Evaluation the insider threat, as well as to topic area?
Life Cycle remote access by insiders.
What capabilities from other topic areas Malware is a principal mechanism
are required for effective progress in this Malware can be used by insiders whereby machines are taken over for
topic area? or could act as an insider on botnets. Significant progress in the
Advances in scalability, composability, behalf of an outside actor. Thus, malware area will go far toward enabling
and overall system trustworthiness are malware prevention can help effective botnet mitigation. Economic
likely to contribute to the development combat insider threats. analysis of adversary markets supports
of scalable, composable evaluation meth- Provenance can also help combat this area, as well as botnet defense, and
odologies, and suggest some synergistic insider threats. For example, may provide background intelligence
evolution. Metrics that facilitate evalu- strong information provenance in support of situational understanding.
ation will also contribute significantly. can help detect instances where
How does progress in this area support
insiders improperly altered
How does progress in this area support advances in others?
critical data.
advances in others? Progress in the area of inherently secure
Effective evaluation methodologies can Privacy-aware security requires systems that can be thoroughly moni-
provide major benefits to all the other knowledge of insiders who were tored and audited will benefit other
topics. Otherwise, the absence of such detected in misuse, as well as topics, especially situational understand-
methodologies leaves significant doubts. mechanisms for privacy. ing. Attribution also links this topic to
situational understanding. Advances in
Topic 4: Combatting Insider How does progress in this area support detection enable malware repositories,
Threats advances in others?
which can be mined to identify families
What capabilities from other topic areas Progress in combatting insider and histories of malware, which in turn
are required for effective progress in this threats will support advances may make attribution possible.
topic area? in privacy and survivability for
Several dependencies on other topic time-critical systems, as well as Collaborative detection may depend
areas are particularly relevant: conventional systems. Controls on progress in global-scale identity
A6
INTERDEPENDENCIES AMONG TOPICS
that can be tolerated while still mitigation draw on advances in surviv- can completely undermine would-be
being able to follow the adversary ability, for example. solutions. Global-scale identity man-
assets. agement is essential for enterprise-wide
We should examine metrics Topic 9: Provenance privacy. Usability is essential, because
related to human factors to assess What capabilities from other topic areas otherwise mechanisms tend to be
effectiveness of presentation misused or bypassed and policies tend
would facilitate progress in this topic
approaches. to be flouted. Situational understand-
area?
ing and attack attribution, as well as
We should explore metrics
Provenance is dependent on most of the the ability to combat malware, may be
for information sharing—for other topics and most of the other topics somewhat less important but still can
example, the tradeoff between are dependent on provenance, but a few contribute to the detection of privacy
how much the sharer reveals topics have more direct connections. violations.
versus how actionable the Global-scale identity management
community perceives the shared How does progress in this area support
is required to track authorship as well
data to be. This issue may touch advances in others?
as chain-of-custody through informa-
on sharing marketplaces and tion processing systems. Privacy-aware Global-scale identity management
reputation systems. security is highly relevant to the dis- can benefit—for example, by being
The current state of metrics with semination of provenance information. shown how to build identity manage-
respect to adversary nets and fast Scalable trustworthiness is essential ment systems that protect privacy. The
flux are not adequately known. to trustworthy provenance. Usability system evaluation life cycle can benefit
We should examine how SANS would be important as well. from provenance. To some extent, this
and similar organizations collect topic can influence requirements for
measurement data. How does progress in this area support how scalable trustworthy systems are
advances in others? designed and developed.
How does progress in this area support Trustworthy provenance would con-
advances in others? tribute significantly to combatting Topic 11: Usable Security
malware and to situational under- What capabilities from other topic areas
For many attack situations of inter- standing. It could also contribute are required for effective progress in this
est, advances in analysis and attack to privacy-aware security. It would topic area?
taxonomy would also support malware provide considerable improvements in Identity management: Large-
defense and therefore mitigate botnets. system usability overall. scale identity management
Systems that are intrinsically monitor- systems could solve one of the
able and auditable would presumably Topic 10: Privacy-Aware most vexing security problems
be easier to defend and less prone to users face today—namely, how
Security
malware. What capabilities from other topic areas to establish trust between
are required for effective progress in this and among users and systems,
Advances in attribution to the ultimate topic area? particularly within systems
attack source would support advances and networks that are easy to
in defense against botnets and other Information provenance is needed
use by ordinary users and by
attacks where the immediate launch for many different privacy mechanisms
administrators.
point of the attack is itself a victimized applied to data. Scalable trustworthy
machine. systems are needed to ensure the integ- Survivability of time-
rity of the privacy mechanisms and critical systems: Advances in
This topic and the survivability area policies. Combatting insider threats availability directly enhance
are mutually reinforcing. Reaction and is essential, because otherwise insiders usability, especially whenever
Reference
[Neu2006] Peter G. Neumann. Holistic systems. ACM SIGSOFT Software Engineering Notes 31(6):4-5, November 2006.
A8
INTERDEPENDENCIES AMONG TOPICS
Appendix B
Appendix B. Technology Transfer
This appendix considers approaches for transitioning the results of R&D on the
11 topic areas into deployable systems and into the mainstream of readily available
trustworthy systems.
B.1 Introduction
R&D programs, including cyber security R&D, consistently have difficulty in
taking the research through a path of development, testing, evaluation, and tran-
sition into operational environments. Past experience shows that transition plans
developed and applied early in the life cycle of the research program, with prob-
able transition paths for the research products, are effective in achieving successful
transfer from research to application and use. It is equally important, however, to
acknowledge that these plans are subject to change and must be reviewed often.
It is also important to note that different technologies are better suited for differ-
ent technology transition paths; in some instances, the choice of the transition
path will mean success or failure for the ultimate product. Guiding principles for
transitioning research products involve lessons learned about the effects of time/
schedule, budgets, customer or end-user participation, demonstrations, testing and
evaluation, product partnerships, and other factors.
There are at least five canonical transition paths for research funded by the
Federal Government. These transition paths are affected by the nature of the
technology, the intended end-user, participants in the research program, and
B1
other external circumstances. Success B.2 Fundamental Issues for concepts discussed in this topic area into
in research product transition is often Technology Transition the mainstream of education, training,
accomplished by the dedication of the experience, and practice will be essential.
program manager through opportunistic What are likely effective ways to trans-
channels of demonstration, partnering, fer the technology? B.3 Topic-Specific
and occasional good fortune. However, Considerations
no single approach is more effective than There is no one-size-fits-all approach In this section, certain issues that are
a proactive technology champion who to technology transfer. Each of the 11 specific to each of the 11 topics are
is allowed the freedom to seek potential topic areas will have its own special considered briefly.
utilization of the research product. The considerations for effective transitioning.
five canonical transition paths can be For example, effective transitioning will Topic 1: Scalable Trustworthy Systems
identified simply, as follows: depend to some extent on the relevant
customer bases and the specific applica- Easy scalability, pervasive trustworthi-
Department/Agency direct to tions. However, this section considers ness, and predictable composability all
Acquisition (Direct) what might be common to most of the require significant and fundamental
Department/Agency to 11 topics. A few issues that are specific changes in how systems are developed,
Government Lab (Lab) to each topic are discussed subsequently. maintained, and operated. Therefore,
Department/Agency to Industry this topic clearly will require consider-
(Industry) It will be particularly important that able public-private collaboration among
the results (such as new systems, mecha- government, industry, and academia,
Department/Agency to Academia
nisms, policies, and other approaches) with some extraordinary economic,
to Industry (Start-up) be deployable incrementally, wherever social, and technological forcing func-
Department/Agency to Open appropriate. tions (see Section B.4). The marketplace
Source Community (Open has generally failed to adapt to needs for
Source) Technologies that are to be deployed trustworthiness in critical applications.
on a global scale will require some
Many government agencies and com- innovative approaches to licensing and Topic 2: Enterprise-Level Metrics
mercial companies use a measure known sharing of intellectual property, and (ELMs)
as a Technology Readiness Level (TRL). serious planning for test, evaluation,
The TRL is a term for discussing the and incremental deployment. They will This is perhaps a better-mousetrap
maturity of a technology, to assess the also require extensive commitments to analogy: if enterprise-level metrics
maturity of evolving technologies (mate- sound system architectures, software were well developed and able to be
rials, components, devices, etc.) prior engineering disciplines, and commit- readily evaluated (topic 3), we might
to incorporating that technology into ment to adequate assurance. presume the world would make a beaten
a system or subsystem. Whereas this path to their door. Such metrics need
mechanism is primarily used within Carefully documented worked examples to be experimentally evaluated and
the DoD, it can be considered a rea- would be enormously helpful, especially their practical benefits clearly demon-
sonable guideline for new technologies if they are scalable. Clearly, the concepts strated, initially in prototype system
for almost any department or agency. addressed in this document need to environments and ultimately in realistic
Table B.1 lists the various technology become a pervasive part of education large-scale applications.
readiness levels and descriptions from and training. To this end, relevant R&D
a systems approach for both hardware must be explicitly oriented toward real
and software. applicability. Furthermore, bringing the
B2
TECHNOLOGY TRANSFER
Table B1: Typical Technology Readiness Levels
4. Component and/or breadboard validation in laboratory Basic technological components are integrated to establish that they
environment. will work together. This is relatively “low fidelity” compared to the
eventual system. Examples include integration of “ad hoc” hardware in
the laboratory.
5. Component and/or breadboard validation in relevant Fidelity of breadboard technology increases significantly. The basic
environment. technological components are integrated with reasonably realistic
supporting elements so it can be tested in a simulated environment.
Examples include “high fidelity” laboratory integration of components.
6. System/subsystem model or prototype demonstration in a Representative model or prototype system, which is well beyond that
relevant environment. of TRL 5, is tested in a relevant environment. Represents a major step
up in a technology’s demonstrated readiness. Examples include testing
a prototype in a high-fidelity laboratory environment or in simulated
operational environment.
7. System prototype demonstration in an operational Prototype near, or at, planned operational system. Represents a major
environment. step up from TRL 6, requiring demonstration of an actual system
prototype in an operational environment such as an aircraft, vehicle, or
space. Examples include testing the prototype in a test bed aircraft.
8. Actual system completed and qualified through test and Technology has been proven to work in its final form and under
demonstration. expected conditions. In almost all cases, this TRL represents the end of
true system development. Examples include developmental test and
evaluation of the system in its intended weapon system to determine if
it meets design specifications.
9. Actual system proven through successful mission operations. Actual application of the technology in its final form and under
mission conditions, such as those encountered in operational test
and evaluation. Examples include using the system under operational
mission conditions.
TECHNOLOGY TRANSFER B3
Topic 3: System Evaluation Life system survivability requires an encourage and fund research and
Cycle overarching commitment to system development relating to all of the
Similarly, if effective evaluation meth- trustworthiness that must transcend topics considered here, with particular
odologies could be developed, their what has been done in the past. emphasis on trustworthy systems, com-
usefulness would need to be clearly dem- posability, scalability, and evolutionary
onstrated on real systems, as in topic 2. Topic 8: Situational Understanding system architectures. It also needs to
Thoroughly specified and relatively and Attack Attribution encourage the incorporation of source-
complete requirements would also be R&D in this area has been slow to available and nonproprietary systems
required. Given a few well-documented find its way into commercial products. that can demonstrably contribute to
demonstrations of effectiveness, the Recognition of the pervasive needs for trustworthiness.
incentives for technology transfer would monitoring and accountability would
be greatly increased. be of great value. Academic research needs to pursue theo-
ries and supporting tools that enable
Topic 4: Combatting Insider Threats Topic 9: Provenance systematic development of composable
Once again, the proof is in the pudding. Provenance would be very useful in and scalable trustworthy systems and all
Demonstrations of the effectiveness finance, government, health care, and the other topics discussed here.
of approaches that combat insider many other application areas, and
misuse would encourage adoption of would facilitate forensics. Commercial developers need to instill a
the techniques. more proactive discipline of principled
Topic 10: Privacy-Aware Security system developments that allow interop-
Topic 5: Combatting Malware and Advances in this topic could be particu- erability among different systems and
Botnets larly useful in many application areas, subsystems, that employ much better
As noted in Appendix A, the com- such as health care, financial records, software engineering practices, that
monalities among insider threats and communication logs, and so on. result in trustworthy systems that are
malware suggest that demonstrations more composable and scalable, and that
of the effectiveness of approaches that Topic 11: Usable Security provide cost-effective approaches for all
combat malware are likely to be rapidly Almost anything that significantly the topics discussed here.
and widely adopted in practice. increased the usability of security and
helped manage its inherent complex- Topic 4: Combatting Insider Threats
Topic 6: Global-Scale Identity Man- ity would be likely to find its way into Governments need to establish base-
agement practice fairly readily. lines and standards. Legal issues
It will be important to design mech- relating to trap-based defensive strate-
anisms and policies that can be gies and entrapment law should be
incrementally deployed. Technologies B.4 Forcing Functions (Some addressed. Applying these to the many
that are to be deployed on a global scale Illustrative Examples) real situations in government activity
will require some innovative approaches For several of the 11 topics, this section where insider behavior is a genuine
to licensing and sharing intellectual addresses the question What are the threat would be beneficial. Current
properties, and serious planning for test, appropriate roles for government, aca- government efforts to standardize on
evaluation, and incremental deployment. demia, industry, and markets? Many authentication and authorization (e.g.,
of the suggested forcing functions are the Common Access Card) are worth-
Topic 7: Survivability of Time Criti- applicable in other topics as well. while despite their potential limitations,
cal Systems particularly in helping combat insider
R&D communities have long under- Topic 1: Scalable Trustworthy Sys- misuse. Academia needs to pursue
stood how to take advantage of tems R&D that is realistically relevant to the
fault-tolerance mechanisms. However, The federal government needs to insider threat. Industry research needs
B4
TECHNOLOGY TRANSFER
to be more closely allied with the needs eat its own dog food, establishing sound Provide suitable funding for basic
of practical systems with fine-grained identity management mechanisms and research in usable security.
access controls and monitoring facilities. policies, and adhering to them. Encourage interdisciplinary
Industry is also the most likely source of research in usable security.
data sets that contain instances of insider Academia needs to recognize more
Adopt usability reviews for
misbehavior, or at least more detailed widely the realistic problems of global
security research.
knowledge of some kind on how real identity management and to embed
insider misbehavior tends to manifest more holistic and realistic approaches Establish appropriate standards,
itself. The marketplace needs to be into research. criteria, and best practices.
responsive to customers demanding Pervasively embed usability
better system solutions. Note also the Industry needs to recognize the enor- requirements into the
possible relevance of HSPD-12 PIV-I mous need for interoperability within procurement process.
and PIV-II. multivendor and multinational feder-
Reconsider security policies from
ated systems.
a usability perspective.
Various incentive structures might be
considered: The marketplace needs to anticipate Ensure that usable security is
long-term needs and somehow inspire a criteria for evaluating NSA
Business cases as incentive governments, academia, and industry centers of academic excellence.
(investment vs. potential cost) to realize the importance of realistic (This will provide an incentive
Insurance as financial protection approaches. to get usability into the
against insiders curriculum.)
TECHNOLOGY TRANSFER B5
Appendix C
Appendix C. List of Participants in the Roadmap Development
We are very grateful to many people who contributed to the development of this roadmap for cybersecurity research,
development, test, and evaluation. Everyone who participated in at least one of the five workshops is listed here.
C1
Appendix D
Appendix D. Acronyms
A/V antivirus
AMI Advanced Metering Infrastructure
BGP Border Gateway Protocol
C2 command and control
CAC Common Access Card
CAPTCHA Completely Automated Public Turing test to tell Computers and Humans Apart
CASSEE computer automated secure software engineering environment
CERTs Computer Emergency Response Teams
CMCS Collaboratory for Multi-scale Chemical Science
COTS commercial off-the-shelf
CUI Controlled Unclassified Information
CVS Concurrent Versions System
DAC discretionary access controls
DARPA Defense Advanced Research Projects Agency
DDoS distributed denial of service
DETER cyber-DEfense Technology Experimental Research
DHS Department of Homeland Security
DKIM DomainKeys Identified Mail
DNS Domain Name System
DNSSEC DNS Security Extensions
DoS denial of service
DRM digital rights management
ESSW Earth System Science Workbench
EU European Union
FIPS Federal Information Processing Standards
FISMA Federal Information Security Management Act
GPS Global Positioning System
HDM Hierarchical Development Methodology
HIPAA Health Insurance Portability and Accountability Act
HSI human-system interaction
HVM hardware virtual machine
I&A identification and authentication
I3P Institute for Information Infrastructure Protection
IDA Institute for Defense Analyses
IDE integrated development environment
IDS intrusion detection system
INL Idaho National Laboratory
IPS intrusion prevention system
D1
IPsec Internet Protocol Security
IPv4 Internet Protocol Version 4
IPv6 Internet Protocol Version 6
IRB institutional review board
ISP Internet service provider
IT information technology
LPWA Lucent Personalized Web Assistant
MAC mandatory access controls
MIT Massachusetts Institute of Technology
MLS multilevel security
MTBF mean time between failures
NIST National Institute of Standards and Technology
NOC network operations center
OODA Observe, Orient, Decide, Act
OS operating system
OTP one-time password
P2P peer-to-peer
P3P Platform for Privacy Preferences
PDA personal digital assistant
PGP Pretty Good Privacy
PII personally identifiable information
PIR private information retrieval
PKI public key infrastructure
PL programming language
PMAF Pedigree Management and Assessment Framework
PSOS Provably Secure Operating System
PREDICT Protected Repository for the Defense of Infrastructure against Cyber Threats
QoP Quality of Protection
RBAC role-based access control
RBN Russian Business Network
RFID radio frequency identification
ROM read-only memory
SBU Sensitive But Unclassified
SCADA Supervisory Control and Data Acquisition
SCAP Security Content Automation Protocol
SIEM security information and event management
SOHO small office/home office
SPF sender permitted from
SQL Structured Query Language
SRS Self-Regenerative Systems
SSL Secure Sockets Layer
T&E test and evaluation
TCB trusted computing base
TCP/IP Transmission Control Protocol/Internet Protocol
TLD top-level domain
TPM Trusted Platform Module
TSoS trustworthy systems of systems
UI user interface
UIUC University of Chicago at Urbana-Champaign
USB universal serial bus
US-CERT United States Computer Emergency Readiness Team
VM virtual machine
VMM Virtual Machine Monitor