Building a Highly Adaptive, Resilient, and Scalable MPLS Backbone

Ning So, Verizon Business Hao-Hsin Huang, WANDL, Inc. Feb. 9, 2007 MPLS World Congress 2007 Paris, France

2006 Verizon. All Rights Reserved. PT00000. 00/00/06

Agenda
About Verizon Business and WANDL Verizon Business MPLS core network MPLS TE Fast Reroute (FRR) design on VzB MPLS core Design challenges and solutions Solving LSP meshing scalability Q&A

Verizon Business
Verizon Business is the premier communications solutions provider for global businesses, government agencies, and educational institutions. Data and IP services Security IT solutions Managed networks Premises equipment Contact centers Conferencing Voice

WANDL
WANDL is the leading supplier of software solutions for advanced network planning, management, design, and optimization 20 years expertise in software development for network optimization, planning, design, OSS/automation Work with Cisco, Juniper, Tellabs, Alcatel, Nortel, Lucent, Huawei, etc. Manage technologies such as IP, MPLS, VoIP, transport (SONET/SDH), FR/ATM ,TDM Customers include carriers, ISPs, telcos, PTTs, service providers, enterprise, and government organizations

MPLS Core Network Overview

Verizon Business Private MPLS Core Network
U.S.-based Backbone trunks are OC48 and OC192 packet-over-SONET Provide Layer 2 (MPLS-TE) transport services for Verizon Business private data networks including IP, Ethernet, and Frame Relay networks

Drawing of the MPLS Core Network

Key Objectives of the Network

Network Capacity Provide sufficient backbone capacity to carry traffic from all feeder networks Network Resiliency 100% traffic re-route during single hardware and/or transmission facility failure Network Performance Provide the minimal latency routes available between any two network elements Network Efficiency Best practices in network architecture and traffic engineering techniques to optimize network routing and resource usage
7

Why Implement MPLS TE FRR?

Underlying transmission facilities supporting the MPLS core network are gradually shifting from a 1+1 SONET ring-based infrastructure to 1+0 linear-based infrastructure.
Transmission equipment moving from conventional DWDM to Ultra Long Haul (ULH). ULH favors linear over ring infrastructures.
ULH has fewer optical regions compared with conventional systems, resulting in a less efficient ring design (larger rings) and no impact on linear design ULH has a much higher availability rate, reducing the need for the ring system for transmission facility maintenance and repair work

The core network with linear transmission facility relies more on the Layer 2 reroute for service restoration. MPLS TE FRR is the only proven technology that provides service restoration at a speed comparable to ring-based restoration.

FRR Implementation Design Challenges

FRR design can be highly complicated with implementation and management presenting bigger challenges to service providers. FRR design challenges 1. Bundled FRR design for ease of implementation and management 2. FRR bandwidth and pre-emption design 3. FRR design with built-in facility failures knowledge base 4. Do not re-route LSP on a FRR-enabled network environment

FRR Design Challenge #1

Description When Juniper FRR is engineered and signaled on a per LSP basis, service providers lose the control of FRR route selection Design and management of individual FRR quickly becomes too time consuming for service providers with a large MPLS core network (such as the Verizon Business MPLS Core network). Solution Utilize the node/link protection scheme

10

Challenge #1 Solution Illustration NLP Discovery and Design Parameters

Auto FRR design parameters

NLP flag on LSP

Primary tunnel path in yellow NLP bypass tunnel in green

11

Challenge #1 Solution Bypass Tunnel and Its Protected Primary Tunnel Paths

Bypass tunnel path

Protected path

All tunnels being protected by this bypass tunnel

12

FRR Design Challenge #2

Description How to set up the bypass tunnel bandwidth and preemption? Problems occur if bypass tunnel bandwidth is set up according to the primary LSP bandwidth
Additional bandwidth on the backbone has to be reserved for FRR, causing inefficient use of valuable network resources FRR bandwidth and preemption design quickly becomes too complicated when multiple FRR paths are set up to account for multiple network failures Multiple network element failure can cause domino effect on FRR reroute due to preemption which magnifies the problem and causes network instability Service provider loses performance predictability due to the massive amount of combinations and permutations of the re-route scenarios

13

FRR Design Challenge #2

Solution
Set bypass tunnel bandwidth to zero, effectively turning off the preemption All traffic on the FRR route share the pain during the outages Rely on the LSP re-signal to restore the LSP primary route, based on the new network topology Allow queuing at the physical interface level handle the traffic discard during congestion

14

Challenge #2 Solution Option 1: Both Design and RSVP BW=0

All bypass tunnels are routed based on 0 bw requirement for protection. Shortest path results. An efficient network resource usage results due to 0 rsvp bw. Queuing delay could occur during congestion.

15

Challenge #2 Solution Option 2: Full Design BW and 0 RSVP BW

All bypass tunnels are routed based on 100% primary tunnel bw requirement for protection. Best paths for full protection will result. An efficient network resource usage results due to 0 rsvp bw. Propagation delay could be increased for some bypass tunnels.
16

FRR Design Challenge #2

Alternative Solution
Create multiple bypass tunnels Set maximum bw and subscription ratio for bypass tunnels. Load balance among all available network resources Minimize congestion Prevent losing all tunnels during multiple failure scenarios

17

Challenge #2 Alternative Solution Multiple Bypass Tunnels

Full BW for bypass tunnel design Low RSVP BW to conserve network resource

Set maximum protection BW for bypass tunnels Set maximum number of bypass tunnels allowed

A possible scenario is to use multiple (4) OC48 trunks to protect a single OC192 trunk Load balancing is desired among 4 OC48 trunks This scheme could preserve 75% of the traffic even when the OC192 and one OC48 are down at the same time
18

FRR Design Challenge #3

Description
Current MPLS network does not have any knowledge of the physical layer topology, so the auto-configured FRR may be put on the same physical transmission facilities as the primary route of the LSP
Manual configuration of individual FRR, even using node/link protection scheme, can be error prone Admin Group function is the ideal and logical choice for marking the facilities. However, existing RSVP TE extension does not include Admin Group

19

FRR Design Challenge #3

Solution
Use fate sharing function to group trunks based on physical topology Fate sharing allows for manual categorization of trunks When constructing bypass tunnels, router will avoid links in the same group as the protected link Juniper fate-sharing and Cisco Share Risk Link Group (SRLG) features are vendor-specific implementation Create explicit bypass paths using WANDL tool to ensure facility diversity

20

Challenge #3 Solution Facility Grouping

Three OC48 ports on the same PIC

Automatically constructed facility grouping during parsing of the configuration files User may add new facility grouping to study its impact on bypass tunnel design Specify facility diversity for FRR auto design
21

Challenge #3 Solution FRR Design with Facility Diversity

Primary tunnel path in yellow and NLP bypass tunnel in green
With facility diversity Bypass tunnel avoids link in the same facility due to added constraint

W/o facility diversity Bypass tunnel uses link in the same facility due to CSPF

22

FRR Design Challenge #4

Description
How to design LSP that resists re-routing in a FRR-enabled network environment
Customers with ultra-delay sensitive applications who do not want to be placed on sub-optimal route Customers prefer to be notified of the failure and switch to alternative network/provider

Solution
Dedicate a set of do not re-route LSPs Disable the FRR at the LSP head end while maintaining the node/link protection scheme network wide

23

Challenge #4 Solution LSP Tunnels Configlet Generation

Mix of LSPs with NLP and without NLP Link-protection on rsvp interfaces

LSP with and without NLP

24

LSP Scalability Challenge

Designing a large MPLS core network that converges multiple feeder networks with thousands of trunking LSPs for each feeder network Designing and managing a network with tens of thousands of LSPs is not practical for service providers Network performance and operator ability to troubleshoot are negatively impacted as the number of LSPs increases

25

A Practical Solution for LSP Consolidation

Establish a full mesh of LSPs as the base LSP set in the MPLS core network. Set up feeder network logical trunking mesh as pseudowires within the base LSPs Establish each set of PW with its own Class of Service queue for traffic management and policing Build more than one set of base LSPs depending on special requirements, e.g., do not re-route LSPs

26

LSP Scalability illustration

Tunnels routed thru a core link

Pseudowires riding on a LSP tunnel

One fully meshed set of LSP tunnels Many PWs riding on a LSP tunnel All PWs passing thru a link are protected by FRR NLP

27

Conclusions
FRR is a necessary and powerful tool in the MPLS TEenabled network; however, it is also highly complicated to implement MPLS TE is still a rapidly developing and evolving technology with many gaps in the current standards Although most major service providers have deployed MPLS TE-enabled networks, the learning curve is still steep, and there are many challenges ahead A comprehensive FRR design and simulation tool like MPLSView is needed to ensure proper BW protection and efficient use of network resources

28

Q&A

Thank You!
Ning So Verizon Business 2400 N. Glenville Richardson, Texas ning.so@verizonbusiness.com Hao-Hsin Huang WANDL, Inc 88 Centennial Ave. Piscataway, NJ haohsin@wandl.com

29