Establishing a Security

Metrics Program
Tiger Team Final Report
Chris Cain & Erik Couture
October 2011

SANS Technology Institute - Candidate for Master of Science Degree

Introduction

Team Members
Mandate
Overall project aim
Methodology

SANS Technology Institute - Candidate for Master of Science Degree

Security Metrics Overview

How secure are we?
Are our security investments making a
difference?
Where can we have the most impact on
our security posture?"

SANS Technology Institute - Candidate for Master of Science Degree

Why Metrics?
Metrics vs Measurement
The importance of context and knowledge,
not just data
The challenge of what to measure

SANS Technology Institute - Candidate for Master of Science Degree

Goal/Scope
Paint a clear picture of our security posture
Identify areas of greatest risk
Help educate resource allocation towards
areas of greatest security gain
Educate senior management on possible
business impacts of our security posture
Provide a method to monitor the
effectiveness of our policy and technological
changes over time
SANS Technology Institute - Candidate for Master of Science Degree

Example 1
Secure Firewalls, Routers, and Switches
Aim
Visibility of the
ground truth
Ensure minimal
ports/services exposed
Input Data
Network Device Threat Level
Average days to fix configuration issues
Total insecure configurations found
Visualization
Horizontal bar charts give a good sense of progress over
several reporting periods and between each device type
SANS Technology Institute - Candidate for Master of Science Degree

Example 2
Boundary Defense
Aim
Reduce by 80% the number of internet entry points
Achieve 100% of hosts pointed at secure DNS servers
Achieve 100% physical network verification.
Input Data
Total quantity of defenses scored Score from 1 to 5
Boundary Defense Threat Level (subjectively assigned)
Visualization
Line graph comparing boundary
device types against their scores

SANS Technology Institute - Candidate for Master of Science Degree

Example 3
Incident Response Capability
Aim
Assess ability to detect and respond
Fuse/visualize end-to-end IH timelines
Input Data
Mean time to incident recovery
Number of Lessons Learned
as a result of the incident.
Mean time to incident eradication
Mean time to incident detection/identification
Visualization
Stacked Bar Chart allows reader to quickly compare the
relative time involved in each phase of incident handling
SANS Technology Institute - Candidate for Master of Science Degree

Visualization / Dashboard (1)

SANS Technology Institute - Candidate for Master of Science Degree

Visualization / Dashboard (2)

SANS Technology Institute - Candidate for Master of Science Degree

Recommendations
The establishment of an enterprise-wide security
metrics program.
The adoption of the SANS Top 20 Security Controls
framework as a basis for the ongoing gathering
and reporting of security metrics.
The institution of a security metrics board which will
regularly assess the effectiveness and adjust the
security metrics program.

SANS Technology Institute - Candidate for Master of Science Degree

References

Twenty Critical Security Controls for Cyber Defense: SANS/CAG

NIST Special Publication 800-61
Beautiful Security Metrics by Elizabeth Nichols
Twenty Most Important Controls and Metrics for Effective Cyber Defense
and Continuous FISMA Compliance by John Gilligan
Seven Myths about Information Security Metrics by Dr. Gary Hinson
Security Metrics, Replacing Fear, Uncertainty and Doubt, Gary McGraw
FISMA FY2011 - CIO Reporting Metrics by US DHS
IT Security Metrics, A Practical Framework for Measuring Security &
Protecting Data, Lance Hayden, Ph.D.
A Guide to Security Metrics (SANS Reading Room), Shirley C. Payne
CSO Security and Risk by Scott Berinato

SANS Technology Institute - Candidate for Master of Science Degree