International Telecommunication Union

IP Performance Metrics:
Definitions and Implementation Examples

Al Morton
AT&T Labs

Workshop on End-to-End Quality of Service.What is it? How do we get it?

Geneva, 1-3 October 2003
 Outline
ITU-T

o Performance Management Framework

Relationship to the E2E QoS goal
o IP Parameters/Metrics Summary
o In-progress Metric Development
o Implementations
1. Service Providers
2. Customers
3. 3rd Parties
1. Performance for MPLS-enabled IP Nets

Workshop on End-to-End Quality of Service. What is it? How do we get it? 2

1-3 October 2003
 Network Performance
ITU-T Management Framework

o Fault Monitoring -- failure detection

o Passive Info Collection (single point)
Read MIB counters or control data
Sample Traffic
o Active Measurements
Synthetic Traffic Dedicated to meas.
o Customer Measurements
Live or Synthetic traffic
Workshop on End-to-End Quality of Service. What is it? How do we get it? 3
1-3 October 2003
 Relationship to E2E QoS:
ITU-T Provide answers to ...
o Network Provider
Is the design meeting requirements for
various traffic classes or applications?
How can I demonstrate the superior
performance of my service offering?
o Customer
Is Network Performance Agreement?
o 3rd Parties
What does the net look like? Hot
spots?
Workshop on End-to-End Quality of Service. What is it? How do we get it? 4
1-3 October 2003
 ingress MP egress MP

IPRE1
IPRE2
Valid header and Successful
tTmax error-free payload IP packet outcome
ITU-T
IPRE1
Packet Perf. Parameters

IPRE2
Corrupted header or Errored
tTmax errored payload IP packet outcome

IPRE2 Spurious
(Note) IP packet outcome

IPRE1

Never delivered or
delivered to an unpermitted
Lost
egress MP
IP packet outcome

IPRE1

t >Tmax IPRE2 Lost

(Note) IP packet outcome

NOTE Outcome occurs independent of IP packet contents

Workshop on End-to-End Quality of Service. What is it? How do we get it? 5

1-3 October 2003
 Metric/Parameter Definition
ITU-T Summary
IETF IPPM RFCs ITU-T Recs.
Framework 2330 Y.1540 cl 1 thru 5
Sampling 2330 Poisson (future work in
3432 Periodic SG4 ?)
Loss 2680 Y.1540 cl 5.5.6
Delay 2679 (1-way) Y.1540 cl 6.2
2681 (Round Trip)
Delay Variation 3393 Y.1540 cl 6.2.2

Availability 2678 Y.1540 cl 7

Bulk Transfer Cap 3148
Loss Patterns 3357 Possibly in G.IPP
Workshop on End-to-End Quality of Service. What is it? How do we get it? 6
1-3 October 2003
 Comparison of IETF and ITU-T
ITU-T Delay Variation Metrics
IETF IPDV is a measure of transfer Tx Rcv Playout
1
delay variation w.r.t. previous packet.
For Packet n,
IPDV(n) = Delay(n) - Delay(n-1)
or = R(n) - R(n-1) - T(n) - T(n-1) 2
Inter packet
If the nominal transfer time is 1 arrival time,
longer than
=10msec, and packet 2 is delayed in send interval
transit for an additional 5 msec, then 3 t

two IPDV values will be affected. 2

IPDV(2) = 15 - 10 = 5 msec
IPDV(3) = 10 - 15 = -5 msec 4
IPDV(4) = 10 - 10 = 0 msec 3

ITU-T SG 13 PDV is delay w.r.t. a

reference, usually minimum delay. 4
PDV(n) = Delay(n) - Min[Delay(*)]
PDV(1,3,4)=0 PDV(2)=5

Time spent in: Transit Rcv Buffer

Workshop on End-to-End Quality of Service. What is it? How do we get it? 7

1-3 October 2003
 Transient Delay Variation
ITU-T caused by burst traffic
100
90
80
70
Delay or Jitter (ms)

60
50
40
30
20
10
0
-10
-20
120 220 320 420 520 620 720 820 920 1020 1120
Time in ms
ITU PDV IETF
delta ms IPDV ms RT-delay ms
Workshop on End-to-End Quality of Service. What is it? How do we get it? 8
1-3 October 2003
 Packet Metrics for VoIP and
ITU-T other voiceband applications

o *new* metrics in G.IPP

Consecutive Packet Loss
Degraded Seconds
Short-term Delay Variation
Overall VoIP Parameters
o Alan Clarks Presentation

Workshop on End-to-End Quality of Service. What is it? How do we get it? 9

1-3 October 2003
 What is Packet Reordering?
ITU-T
Packets arrive at Dst, but not in send order.

1, 2, 3, 7, 8, 9, 10, 11,... Loss,no reordering

1, 2, 3, 7, 8, 9, 4, 5, 6, 10, 11,...reordering

In the world of order all these packets are of

interest.

1, 2, 3, 7, 8, 9, 4, 5, 6, 10, 11,...
| Early | Late |

No reordering until Late Packets Arrive

# of Early Packets => Reordering Extent

Workshop on End-to-End Quality of Service. What is it? How do we get it? 10

1-3 October 2003
 Affect of Reordered Packets on
ITU-T most applications

o Receivers must perform work to restore order

1, 2, 3, 7, 8, 9,10, 4, 5, 6, 11, 12,...

| Buffered ||Reordered|

Dst Time axis

1 2 3 7 8 9 10 4 5 6 11
Higher
layers
1 2 3 4 5 6, 11
(& 7 to 10)

Workshop on End-to-End Quality of Service. What is it? How do we get it? 11

1-3 October 2003
 Definition of Reordered Packet
ITU-T o Packet n is designated reordered when
its sequence number is less than the
Next Expected threshold (set by the
arrival of a previous packet).
12
11
Next Expected
10 10
9
Send Order (s)

8 8
7
6 6
5
4 4
3
2 2
1
0
1 2 3 4 5 6 7 8 9 10 11

Arrival Order (i)

Workshop on End-to-End Quality of Service. What is it? How do we get it? 12

1-3 October 2003
 Failure Recovery Time
ITU-T

o When recovery was a simple outage,

characterization was simple, too.
o IETF Benchmarking Methodology WG has
identified 5 possible recovery scenarios:

Lost packets Induced delay

7 6 2 1 4 3 2 1

Duplicate packets Out-of-order packets Errored packets

6 5 4 4 3 2 1 7 5 6 3 4 2 1 7 6 2 1

Workshop on End-to-End Quality of Service. What is it? How do we get it? 13

1-3 October 2003
 Implementations: Customers,
ITU-T Service Providers & 3rd Parties
$ ping R2 (or R3) R3
R1 R2 or
or R4

o Select Ping Target - make Round-trip

connectivity and RTT measurement
o Accuracy Issues include path through
router, path through net (asymmetries),
response time at target, sampling rates
o Compare to current perf. to normal
Workshop on End-to-End Quality of Service. What is it? How do we get it? 14
1-3 October 2003
 Beyond ping: ICMP Timestamp
ITU-T or Timestamp Reply Message
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type | Code | Checksum |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Identifier | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Originate Timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Receive Timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Transmit Timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
R3
Originate R2
R1 Receive
Code=13 R4
Transmit
Code=14
o Time spent processing packet at target
can be removed, for more accurate RTT.
Workshop on End-to-End Quality of Service. What is it? How do we get it? 15
1-3 October 2003
 Implementations:
ITU-T AT&T Global IP Measurements
AT&T GLOBAL IP BACKBONE INFRASTRUCTURE
BR BR
(City 1) BR (City n)
(City 2)

MEASUREMENT
COLLECTION Measurement Probes
SERVER (MCS)

Measurement
probe
AGGREGATED http://www.att.com/ipnetwork
MEASUREMENT
DATA

MEASUREMENT WEB
AGGREGATION CLIENTS
& REPORTING (for report
SERVER (MRS) viewing)

Workshop on End-to-End Quality of Service. What is it? How do we get it? 16

1-3 October 2003
 AT&Ts IP Measurement
24 hours
Design
...
ITU-T 15 minutes

o Poisson Sequence o Periodic Sequence

(RFC2330) (RFC3432)
15 minute duration 1 minute duration
= 0.3 pkts/sec Random Start Time
Type UDP, IPv4 20 ms packet spacing
278 bytes total Type UDP, IPv4
~300 packets sent 60 bytes total
unbiased sample ~3000 packets sent
Workshop on End-to-End Quality of Service. What is it? How do we get it? 17
1-3 October 2003
 Technical Collaborators at AT&T
ITU-T

o Len Ciavattone
o George Holubec
o Madhukar Kshirsagar
o Ron Kulper
o Arvind Ramarajan
o Gomathi Ramachandran

Workshop on End-to-End Quality of Service. What is it? How do we get it? 18

1-3 October 2003
 New Measurement Challenges
ITU-T for MPLS-enabled IP Networks
o Most (all?) IP/Packet Network challenges
o Two main categories of MPLS Domains:
LDP-based, connection-less
Traffic Engineering, connection oriented
o Label Switched Paths are Unidirectional
o point to point and multi-point to point
o Many options for Failure Recovery
o LSP identity optionally removed (PHP)
o Work in progress in SG 13 = Y.MPLSperf
Workshop on End-to-End Quality of Service. What is it? How do we get it? 19
1-3 October 2003
 New Measurement Challenges
ITU-T for MPLS-enabled IP Networks
Scope of OA&M Measurements: MPLS Domain
single Network Section or MPLS Domain
MPLS Domain

Network Network
section section

Exchange
link Exchange
link
Exchange Network
link section

Exchange
link

Exchange
link
MPLS Edge Node, or
MPLS Ingress Node, or
LSR if both IP and MPLS
MPLS are enabled Node Label Switched Paths

Network Section Ensemble (NSE)

MPLS Network
Workshop on End-to-End Quality of Service. What is it? How do we get it? 20
1-3 October 2003
 New Measurement Challenges
ITU-T for MPLS-enabled IP Networks

o New Protocols = New Opportunities to

Blackhole Traffic
o Detect this new class of failures with
Y.1711 MPLS OA&M Connectivity Verific.
First version approved, adding fast failure
detection
LSP-Ping, Like ICMP Echo Request, plus
One-way Delay measurement possible
LSP Traceroute possible

Workshop on End-to-End Quality of Service. What is it? How do we get it? 21

1-3 October 2003
 New Measurement Challenges
ITU-T for MPLS-enabled IP Networks
o New Availability Definition? Crossroad:
Connection-Oriented Transport has used
a 10 second sliding window
Connection-Less Packet Transport has
used a 5 minute fixed window
o MPLS Networks => both transport types
When Connection-oriented Services use
a Connection-less transport, which
precedent should the Availability
Definition follow?
Workshop on End-to-End Quality of Service. What is it? How do we get it? 22
1-3 October 2003
 Summary
ITU-T

o Performance Management Framework

Measurement Systems are a key step
toward the goal of E2E QoS
o Summary of existing Parameters/Metrics
o In-progress Metric Development
o Active Measurement Implementations
Ping for connectivity and ...
Dedicated Measurement Systems
o Parameter Framework for MPLS has new
challenges
Workshop on End-to-End Quality of Service. What is it? How do we get it? 23
1-3 October 2003
 Resources and References
ITU-T
o L. Ciavattone, A. Morton and G.
Ramachandran, "Standardized Active
Measurements on a Tier 1 IP Backbone,"
IEEE Communications Magazine, June
2003.
o Geoff Huston, Measuring IP Network
Performance, The Internet Protocol
Journal, vol 6, no.1, March 2003
http://www.cisco.com/ipj
o X.Xiao, et al., A Practical Approach for
Providing QoS in the Internet Backbone,
IEEE Communications Magazine,
December 2002.
o D. Meyer, et al., Trends in Measurement
and Monitoring of Internet Backbones,
Panel at NANOG 26, slides etc. at
http://www.nanog.org/mtg-
0210/measurement.html
Workshop on End-to-End Quality of Service. What is it? How do we get it?
1-3 October 2003
o ITU-T Rec. Y.1540, Internet Protocol Data
Communication Service IP Packet
Transfer and Availability Performance
Parameters, 2003.
o IETF IP Performance Metrics Working
Group, links to RFC 2330, other IPPM
RFCs and Internet Draft on Reordering:
http://www.ietf.cnri.reston.va.us/html.chart
ers/ippm-charter.html
o Draft New Recommendation Y.MPLSperf,
Performance and Availability Parameters
for MPLS Networks
o Draft New Recommendation G.IPP,
Performance Parameter Definitions for
Quality of Speech and other Voiceband
Applications Utilising IP Networks
o RFC 792, Internet Control Message
Protocol, J. Postel, September 1981.
24