You are on page 1of 54

Core Network Evolution and IPoDWDM: 40G 100G and Beyond

Stefan Kollar Consulting Systems Engineer CCIE #10668 skollar@cisco.com

2010 Cisco and/or its affiliates. All rights reserved.

OTN Technical Foundation OTN Properties and Impact on IP Layer MPLS-TP Technical Foundation MPLS-TP Forwarding and OAM MPLS-TP Deployments 40G/100G Design Considerations 40G/100G Deployment Considerations in 10G Optical Networks 100G - and beyond

2010 Cisco and/or its affiliates. All rights reserved.

Global IP Traffic Growth

IP traffic will increase fivefold from 2008 2013


IP packets will Dominate 87% Consumer Traffic Video IP Routed Services L2 Packet Services Other Services SONET/SDH Data Centre Private Line
2010 Cisco and/or its affiliates. All rights reserved. Source: Cisco Visual Networking IndexForecast, 2008-2013 3

OTN Technical Foundation

Presentation_ID

2010 Cisco and/or its affiliates. All rights reserved.

Cisco Public

OTN defined a fixed hierarchy

of payloads: from OTU1 (2.5G) to OTU3 (40G). Now ODU0 (1G) and OTU4 (100G) are being added
OTN started as a pure wrapper

Frame ODU0 (coming) OTU1 OTU2 OTU3 OTU4 (coming)

Payload (OPU) 1,238,954 kbit/s 2,488,320 kbit/s 9,995,276 kbit/s 40,150,519 kbit/s 104,355,975 kbit/s

around WDM client signals to improve reach and manageability


Recently it has developed into a

complex multiplexing structure that enables a service layer as well as TDM bandwidth mgmt

2010 Cisco and/or its affiliates. All rights reserved.

2010 Cisco and/or its affiliates. All rights reserved.

The OTN Multiplexing Hierarchy


OTN Hierarchy
ODU0: ~1.22Gbps : GE ODU1: ~ 2.7Gbps : STM-16 ODU2 : ~10.7Gbps : STM-64 ODU3 : ~43Gbps : STM-256 ODU4: ~112Gbps : 100GE ODU2e:~11.1Gbps : 10GE LAN PHY ODU3e:~40Gbps : 10GE onto 40G

ODU-FLEX
Similar concept to VCATs in SONET/SDH but uses OTN containers Increments are variable (10Gbps, 1Gbps) but only one increment size per link 1.25Gbps increments is the most commonly discussed ODU-FLEX service .

ODU-FLEX Switch

ODU-FLEX Switch

TS
ODU-FLEX GMP
GFP

TS
ODU-FLEX GMP GMP

TS TS TS TS
HO ODU

TS TS TS TS
HO ODU GFP GFP GFP GFP + idles

ODU-FLEX GMP

Client

GFP Classifier GFP GFP + idles

TS TS TS
HO ODU

TS TS TS
HO ODU

Client

GMP stands for the Generic Mapping Procedure that is currently under definition in Q11

2010 Cisco and/or its affiliates. All rights reserved.

Sub-lambda HO interfaces (SONET, OTN, Ethernet, ESCON)

OTN HO Electrical Cross Connect Grooming and aggregation

Ethernet Switching Fabric

G.709 formatted signal, OTUm

Ethernet interfaces (e.g. OC-3/STM-1)

OADM / Mesh DWDM node

Multi-degree ROADM Cross Connecting Lambdas Dropping full lambdas

Supported Ethernet Service Types per G.8011.x Point-to-Point


Ethernet Private Line (EPL) Type 1 Ethernet Private Line (EPL) Type 2 Ethernet Private Line (EPL) Type 2 timing transparent Ethernet Virtual Private Line (EVPL) Type 1, 2, 3
2010 Cisco and/or its affiliates. All rights reserved.

Supported Ethernet Service Types per G.8011.x Point-to-Multipoint


Ethernet Private Tree (EPT) Ethernet Private LAN (EPLAN) Ethernet Virtual Private Tree (EVPT), Type 1, 2, 3 Ethernet Virtual Private LAN (EVPLAN), Type 1, 2, 3
8

OTN Properties and Impact on IP Layer

Presentation_ID

2010 Cisco and/or its affiliates. All rights reserved.

Cisco Public

OTN
OEO

OTN
OOO

OTN
OEO

Encapsulation and OAM&P over optical spans (G.709 transponders)

OTN
OEO

OTN
OOO

OTN
OEO

Point to Point Multiplexing (G.709 muxponders)

OTN
OEO

OTN
OOO

OTN
OEO

OTN
OEO

OTN Grooming and Switching (OEO Cross Connects)

OTN
OEO
2010 Cisco and/or its affiliates. All rights reserved. 10

Individual IFs Works Cost


2010 Cisco and/or its affiliates. All rights reserved.

Channelized POS/OTN No chOTN Cost

Ethernet w/ VLANs Shaping Metrics


11

Primary Optical Path

Back Optical Path Primary Path failure

Characteristics Optical level over-provisioning : 100% IP level over-provisioning : 0% Overall over-provisioning : 100% Complexity : High need hold timers Complete IP level protection: No Span Failure IP working : Yes

Primary Optical Path

Patch Cord, transponder or router card failure IP working : No

Back Optical Path


2010 Cisco and/or its affiliates. All rights reserved. 12

The Contrasting Topologies


Hierarchical Solution
Physical and Logical Topology the same
Video and Internet cache End User Business Data Centre IP/TV Head End Internet Internet

L3 Routing Nodes

Business Data Centre Internet

Bypass Solution
Video and Internet cache IP/TV Head End

Internet End User Business Data Centre

OTN ODU-FLEX switch


2010 Cisco and/or its affiliates. All rights reserved.

Business 13 Data Centre

Whats the Impact at the Network Level ?


Links and Over Provisioning
1Gbps Initial Demand 10Gbps increments
Link Level Over provisioning Link Level Over Provisioning

1Gbps Initial Demand 1Gbps increments

Network Efficiency and provisioning needs to account for total number of links, traffic growth, provisioning efficiency and upgrade frequency.
OTN Technical Foundation

Network Wide Implications 100 Node network Worse over-provisioning : Links * b/w increment Network wide link upgrades = Links * Link upgrades Worse case over-provisioning =200 * 10Gbps = 2000Gbps Number network wide physical link upgrades = 200 * 0 = 0
2010 Cisco and/or its affiliates. All rights reserved.

Network Wide Implications 100 Node network Worse over-provisioning : Links * b/w increment Network wide link upgrades = Links * Link upgrades Worse case over-provisioning =5000 * 1Gbps = 5000Gbps Number network wide logical link upgrades = 5000 * 7 = 35000
14 Note : This does not take into account physical link upgrades

Photonics Bypass
Client to OEO with SR optics Transport OTN OEO Approach

Client with SR Optics


Optical Approach

Photonic
Client with Integrated Optics


Short Reach
15

Major IP demands are very large often the driving force behind DWDM upgrades Full interfaces directly to the photonic layer of NG/ROADM Intelligent DWDM system Cut-through / bypass at the photonics layer Transponder Eliminates expensive high bandwidth opti-electrical components More pronounced with 40Gbps IPoDWDM Optical and in particularand 100Gbps is the most
2010 Cisco and/or its affiliates. All rights reserved.

cost effective high

b/w inter router connection

MPLS-TP Technical Foundation

Presentation_ID

2010 Cisco and/or its affiliates. All rights reserved.

Cisco Public

16

Evolution of SONET/SDH transport networks to packet switching

driven by
Growth in packet-based services (L2/L3 VPN, IPTV, VoIP, etc) Desire for bandwidth/QoS flexibility New packet transport networks need to retain same operational

model

An MPLS transport profile being defined at IETF (in collaboration

with ITU-T)

2010 Cisco and/or its affiliates. All rights reserved.

17

IETF and ITU-T agreed to work together and bring transport requirements into the IETF and extend IETF MPLS forwarding, OAM, survivability, network management, and control plane protocols to meet those requirements through the IETF Standards Process.[RFC5317]1 ITU-T withdrawal of T-MPLS draft G.8114 in Jan. 2008.
Definition of MPLS Transport Profile (MPLS-TP) protocols, based on ITU-T requirements Note: IETF decided to support single MPLS-TP OAM solution. IETF Chair stated at IETF 79 (11/2010) and IETF 80 (3/2011) Derive packet transport requirements Integration of IETF MPLS-TP definition into transport network recommendations

1: [RFC 5317]: Joint Working Team (JWT) Report on MPLS Architectural Considerations for a Transport Profile, Feb. 2009.
2010 Cisco and/or its affiliates. All rights reserved.

18

18

18

MPLS Solution Space


ECMP MP2Pt LDP, IP

MPLS-TP Solution Space PHP default disabled

MPLS-TP Only Solution Space

MPLS-TP is proper subset of MPLS proper So far, no MPLS-TP only functionality standardized
2010 Cisco and/or its affiliates. All rights reserved. 19

Connection-oriented packet switching model No modifications to MPLS data plane Interoperates/interworks with existing MPLS and pseudowire control and data

planes

No LSP merging or PHP LSPs may be point to point (unidirectional, co-routed bidirectional or associated

bidirectional)
LSPs may be point to multipoint (unidirectional) Networks created and maintained using static provisioning or a dynamic control

plane: LDP for PWs and RSVP-TE (GMPLS) for LSPs


In-band OAM (fate sharing) Protection options: 1:1, 1+1 and 1:N Network operation similar to existing transport networks

See RFC 5654


2010 Cisco and/or its affiliates. All rights reserved. 20

MPLS-TP Forwarding Plane

Presentation_ID

2010 Cisco and/or its affiliates. All rights reserved.

Cisco Public

21

Static Bidirectional Co-routed (same forward and reverse paths) In-band Generic Associated Channel (G-ACh) Ultimate hop popping (no explicit/implicit null) No ECMP Contained within a tunnel

MPLS-TP LSP G-ACh MPLS-TP Tunnel

2010 Cisco and/or its affiliates. All rights reserved.

22

No IP routing required in control and

forwarding planes

Node will still source/terminate IP packets

(e.g. SNMP, NTP) interface

MPLS-TP

Link numbers required on each MPLS-TP Two interface configuration models


IP-enabled (uses ARP) IP-less (no ARP)

IP-enabled requires interface configuration

for

Local IPv4 address Remote IPv4 (next-hop) address

IP-less requires configuration of


Destination MAC address

Same OAM messages for IP-enabled and

IP-less interface configurations

2010 Cisco and/or its affiliates. All rights reserved.

23

MPLS-TP tunnels abstracted as Tunnel-tp

interface LSP

Tunnel holds a working LSP and a protected


Working Protect (optional)

Tunnel may be configured with a bandwidth

allocation

Tunnel operationally UP if at least one LSP

operationally UP (and not locked out) session operationally UP

Working LSP G-ACh

Protect LSP

MPLS-TP Tunnel

LSP operationally UP if OAM (Continuity Check) LSP requires static configuration of LSP label

G-ACh

imposition (output label and output link) disposition (input label)

LSP requires static configuration of LSP label LSP must be co-routed (no embedded check)

2010 Cisco and/or its affiliates. All rights reserved.

24

Same OAM functions for LSPs, pseudowires and sections In-band OAM packets (fate sharing) OAM functions can operate on an MPLS-TP network without a

control plane

Extensible framework with current standardization focus on

fault and performance management


Independent of underlying technology Independent of PW emulated service

2010 Cisco and/or its affiliates. All rights reserved.

25

IETF

Label PW Label ACH OAM Payload Label GAL ACH OAM Payload

0 0 0 1 Version

Reserved

Channel Type RFC 5085

PW Associated Channel Header (ACH)

G-ACh

PW LSP G-ACh

Generic Associated Channel Label (GAL) Associated Channel Header RFC 5586

OAM capabilities extended using a generic associated channel (G-ACh) based on

RFC 5085 (VCCV)


A G-ACh Label (GAL) acts as exception mechanism to identify maintenance

packets
GAL not required for pseudowires (first nibble as exception mechanism) G-ACh used to implement FCAPS (OAM, automatic protection switching (APS),

signaling communication channel, management communication channel, etc)

2010 Cisco and/or its affiliates. All rights reserved.

26

Before Failure

Relies on a disjoint working and a disjoint


Working LSP (Up, Active) PE1 Protect LSP (Up, Standby) P1 Working LSP (Up, Active) PE2

protect path between two nodes LSP) in revertive mode IP/MPLS

Provides 1:1 protection (only one active Functionally similar to path protection in Protection switching can be triggered by

P2

Protect LSP (Up, Standby)

During Failure
Working LSP (Down, Standby) PE1 Protect LSP (Up, Active) Working LSP (Down, Standby) PE2

Detected defect condition (AIS/LDI, LKR) Administrative action (lockout) Far end request (lockout) Server layer defect indication (LOS) Revertive timer (wait-to-restore)
P2 Protect LSP (Up, Active) P1

2010 Cisco and/or its affiliates. All rights reserved.

27

IETF

Function
Continuity Check Connectivity Verification Diagnostic Tests Route Tracing Lock Instruct Lock Reporting Alarm Reporting Remote Defect Indication Client Failure Indication Packet Loss Measurement Packet Delay Measurement

Description
Checks ability to receive traffic Verifies that a packet reaches expected node General diagnostic tests (e.g. looping traffic) Discovery of intermediate and end points Instruct remote MEPs to lock path (only test/OAM traffic allowed) Report a server-layer lock to a client-layer MEP Report a server-layer fault to a client-layer MEP Report fault to remote MEP Client failure notification between MEPs Ratio of packets not received to packets sent One-way / two-way delay (first bit sent to last bit received)

Tool
BFD BFD (proactive) LSP Ping (on-demand) New LSP Ping New New New BFD PW Status New New

2010 Cisco and/or its affiliates. All rights reserved.

28

MPLS-TP deployements

Presentation_ID

2010 Cisco and/or its affiliates. All rights reserved.

Cisco Public

29

Flexible Edge and Services Architecture


Aggregation! Edge! Core! Multiservice Core!

Static MPLS-TP Access Ethernet Access IP/MPLS Access (L2 and CE only) Pseudo-Wire Switching MPLS-TPIP/MPLS

IP/MPLS + Dynamic MPLS-TP

IP/MPLS

IP/MPLS

L3 IP Edge and Service Placement Circuit Emulation + Ethernet

Flexible IP Edge and Service Placement Agg, Edge or Core IP/MPLS in Core / Edge / Aggregation Use MPLS-TP toolbox to enhance dynamic IP/MPLS domain Variety of Access options static MPLS-TP, Ethernet, IP/MPLS Common protocols and control plane aggregation to aggregation
2010 Cisco and/or its affiliates. All rights reserved. 30

Centralised Edge and Transport Architecture


Aggregation! Edge! Core! Multiservice Core!

MPLS-TP Access Ethernet Access IP/MPLS Access (L2 and CE only)

IP/MPLS (L2 and CE only)

IP/MPLS

IP/MPLS

L3 IP + Services Placement Circuit Emulation + Ethernet

Centralised IP Edge and Service placement : Edge, Core IP/MPLS in Core / Edge / Aggregation Variety of Access options static MPLS-TP, Ethernet, IP/MPLS Common protocols and control plane aggregation to aggregation Ethernet Access / IP/MPLS L2 Aggregation : Widely deployed model
2010 Cisco and/or its affiliates. All rights reserved. 31

Centralised Edge and Transport Architecture


Aggregation! Edge! Core!

Static MPLS-TP Access

Ethernet Access Static MPLS-TP IP/MPLS IP/MPLS

L3 IP + Services Placement Circuit Emulation + Ethernet

Centralised IP Edge and Service placement - Edge, Core IP/MPLS in Core / Edge Static MPLS-TP in Aggregation Variety of Access options : Ethernet / Static MPLS-TP Common forwarding protocols aggregation to aggregation End to end transport operations using pseudo-wire switching
2010 Cisco and/or its affiliates. All rights reserved. 32

40G/100G Design Considerations

Presentation_ID

2010 Cisco and/or its affiliates. All rights reserved.

Cisco Public

33

Solutions for Implementing 100G DWDM


100G SR 100G lambda
OTN

100G lambdas
OTN

10G lambda 10G and 100G DWDM Coexistence


10G and 100G lambdas co-exist on same fibre Packet uses 100G, everything else 10G Advantages Only high demand clients upgraded to 100G Protects existing 10G DWDM investment Lowest cost per bit (100G TXPs>10 x10G TXPs) Disadvantages Need a guard band between 10G and 100G frequencies Not appealing in ULH environments

10G SR OTN Multiplexing


All lambdas upgraded to 100Gbps Sub-100G services provided by OTN OEO

Advantages All lambdas on a fibre are 100G

Disadvantages 100TXP investment upfront Need an additional OTN OEO All 10G TXPs are obsolete
34

2010 Cisco and/or its affiliates. All rights reserved.

40 Gig and above rates must meet minimum requirements:


Target 10 Gig distances1500 Km reach Not simply a Greenfield technology, but plug and play over existing 10Gig networks Must be as open as possible, operate over third party DWDM networks Must operate over both 100GHz as well as 50GHz spacings Power and footprint must be reasonable, can not redesign Router/ transport shelf due to blade

To achieve must leverage/control:


1. Optical Impairments 2. Modulations schemes

2010 Cisco and/or its affiliates. All rights reserved.

35

Attenuation
Loss of signal strength Limits transmission distance

Chromatic Dispersion (CD)


Distortion of pulses Limits transmission distance Proportional to bit rate

Optical Signal to Noise

Ratio (OSNR)

Effect of noise in transmission Caused by amplifier Limits number of amplifier


2010 Cisco and/or its affiliates. All rights reserved. 36

Polarization Mode Dispersion

(PMD)

Caused by nonlinearity of fiber geometry Effective for higher bit rates (10G)

Spreaded Pulse as It Leaves the Fiber

Four Wave Mixing (FWM)


Effects in multichannel systems Effects for higher bit rates

Self/Cross Phase Modulation

(SPM, XPM)

Effected by high channel power Effected by neighbor channels


2010 Cisco and/or its affiliates. All rights reserved. 37

Cisco expects 100G Deployments to: Target 10G distances1500 - 2000 Km reach Plug-and-play over existing 10Gig networks Must be spectrally efficient- 50GHz Grid Power / density / cost / performance trade off As Bit Rate increases the above becomes more challenging

Simplify deployment of 100Gig into 10Gig Systems


2010 Cisco and/or its affiliates. All rights reserved. 38

Transmitter
Decrease Speed reduce $$ Increase Modulation - Increase spectral efficiency Increase Optical efficiency Increase spectral efficiency

Receiver
Move from Direct Detection to Coherent detection Compensate for Optical impairments in Electrical Domain(DSP) reduce $$

Forward Error Correction (FEC)


Move to Higher coding gain FECs Increase reach

100Gig was our first challenge overcame with PM-QPSK and new FECs
2010 Cisco and/or its affiliates. All rights reserved. 39

Need to go slower Optical impairments are directly related to signaling rates Need to increase modulation efficiency Signaling speed decreases & Information Rate increases NRZ to ODB to (D)PSK to (D)QPSK Need to increase optical efficiency Split signal over two polarizations (PM Mod Scheme)

1 bit/symbol

1 bit/symbol

2 bits/symbol 00 01 11 QPSK
40

0 NRZ
2010 Cisco and/or its affiliates. All rights reserved.

-1 PSK

10

RX Laser behaves as Local Oscillator to provide a Polarization reference 90 Hybrid:

Converts Phase modulation in Amplitude modulation


Signal Processor:

Recovers Polarization Compensates CD and PMD electronically

2010 Cisco and/or its affiliates. All rights reserved.

41

100G PM-QPSK At and / or exceeds 10Gig System Performance

Utilizing Ciscos advanced Optics and FEC Distances up to 2000Km and beyond Operates over existing Infrastructure at both 50 / 100GHz

PM-QPSK allows 100G operation over existing and greenfield networks at 10Gig distances
2010 Cisco and/or its affiliates. All rights reserved. 42

40G/100G Deployment Considerations in 10G Optical Networks

Presentation_ID

2010 Cisco and/or its affiliates. All rights reserved.

Cisco Public

43

3rd PARTY DWDM SYSTEM MUST SUPPORT ALIEN WAVELENGTHS!

Alien/foreign wavelength is any 3rd party ITU wavelength operating over an existing DWDM infrastructure. G698.2 Standard for Alien/Foreign waves defines: properties for signal sources and sinks properties for DWDM links for black links (i.e. alien wavelengths)

2010 Cisco and/or its affiliates. All rights reserved.

44

Design Considerations

40G receiver differs from 10G

Noise and Impairment Limits

40G IPoDWDM Transponder (DPSK+) Launch Powers Rx Windows OSNR (.1nm) CD PMD
2010 Cisco and/or its affiliates. All rights reserved.

10G Transponder 0 dBm 0 to 23 dBm ~ 15 dB +/- 2000ps/nm 10ps


45

0 dBm 5 to 18 dBm ~ 14.5 dB +/- 750ps/nm 2.5ps

One of the biggest challengers in Poland (~350k broadband users) Implemented XR12000 core 1.5 year ago (one of the first

XR12000 production networks in the world)


BB explosion has made their traffic grow quicker than expected =>

need to upgrade main nodes


Customer A expected big CapEx/OpEx savings

2010 Cisco and/or its affiliates. All rights reserved.

46

IPoDWDM CapEx/OpEx reduction Eliminates Transponder Shelf 4:1 Capacity Savings Conducted successful tests on 40G IPoDWDM

on Warsaw -> Pozna link (614 km) link designed for 10G optical

2010 Cisco and/or its affiliates. All rights reserved.

47

100G Where are we today?

Presentation_ID

2010 Cisco and/or its affiliates. All rights reserved.

Cisco Public

48

IEEE 802.3ba: 40Gb/s and 100Gb/s Ethernet Task Force


40G and 100G Ethernet Physical interfaces for Backplane, Copper, Fiber PMDs

Ratified

IEEE 802.3bg: 40Gb/s SMF Ethernet Task Force


40G Serial PMD optimized for carrier applications

Ratified

ITU Study Group 15: Optical and Transport Networks


OTU4 frame format Single mapping for 40GE/100GE into OTU3/OTU4 OTL protocol enabling OTU3/4 over multi-lane (low cost) optics

Ratified

OIF: 100G Long-distance DWDM Transmission


Industry consolidation around a single 100G DWDM solution

Ratified

2010 Cisco and/or its affiliates. All rights reserved.

49

Customer demands are driving the need for 100Gig and beyond Video HD / 3D
Video Conferencing HD / 3D Gaming HD / 3D

Packet will dominate


28% of population connected 14% of population broadband

100Gig is deploying NOW


Content Providers Tier One SPs and MSOs

Mass deployments 2nd Half 2012 early 2013

Question is not When 100Gig? but rather What is after 100Gig?

2010 Cisco and/or its affiliates. All rights reserved.

50

Higher data rates 200Gig, 400Gig, 1T? Need to investigate other modulation techniques PM-16QAM, PM-64QAM, . or CO-OFDM ? Need deeper look at FEC Advanced FEC What other algorithms are there Need of intelligent DWDM layer Flex spectrum Control plane Advanced operations, troubleshooting and protection mechanisms
Must a channel really fit into 50GHz spacing?

Or should it be gridless?

2010 Cisco and/or its affiliates. All rights reserved.

51

Information distributed over a few Sub-Carriers spaced as closely as possible forming a 1,000Gbps Super-Channel Each Sub-Carrier transporting a lower Bit Rate, compatible with current ADCs and DSPs

|Sch(f)| 2
-200-150-100 -50 0

50 100 150 200

f [GHz]

10x 100Gbit/s SubCarriers close-to-Baud-rate spaced


Super-Channel #1 Super-Channel #2 Super-Channel #3

2010 Cisco and/or its affiliates. All rights reserved.

52

Sub-Carrier spacing: 1.2 times the

Baud Rate

Different approaches for 1,000Gb/s: CP-QPSK: 10 Sub-Carriers at 111 Gbit/s each


back-to-back sensitivity 12 dB

1.8 1.44

CP-QPSK 1.12 1.2 1.08 1.04


1.8 1 1.44

CP-8QAM: 8 Sub-Carriers at 138.75 Gbit/ s each


back-to-back sensitivity 16.1 dB

1.8

CP-16QAQM: 5 Sub-Carriers at 222 Gbit/s each


back-to-back sensitivity 19.1 dB

CP-8QAM 1.20 1.12 1.08 1.04 1.2 CP-16QAM 1.44


1.12 1.08 1.04

with subcarrier spacings

System Configuration: Span of 90km each (ITU-T G.652) Span Insertion Loss: 25dB

2010 Cisco and/or its affiliates. All rights reserved.

53

Thank you.

You might also like