You are on page 1of 103

Understanding, Integrating, and

T bl h ti F C i ti i Troubleshooting Fax Communications in


Today's Enterprise Networks
BRKUCC-2021
Agenda Agenda
Fundamental Concepts p
Fax Transport Methods
QoS Considerations QoS Considerations
Design Best Practices
Troubleshooting Troubleshooting
Fax Servers
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
2
Fundamental Concepts
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
3
Fax Communications Fax Communications
Fax invented by Alexander Bain in
1843 (patented 30 years before
the telephone)
Fax is a ubiquitous form of Fax is a ubiquitous form of
communication today and Fax
over IP (FoIP) is often overlooked
i IP T l h in IP Telephony
Group 3 (G3) is todays fax
standard (speeds up to 14 4 kbps) standard (speeds up to 14.4 kbps)
Super G3 is an optional extension
of G3 that allows for speeds up to
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
4
Alexander Bains Facsimile Apparatus,
British Patent 9745
p p
33.6 kbps
G3 Fax Protocols G3 Fax Protocols
Modern fax communication is based on the Group 3
(G3) fax standard, which is composed of the ITU-T
specifications of T.30, T.4, and T.6
T30 details the signaling T4 specifies the Modified T.30 details the signaling, T.4 specifies the Modified
Huffman (MH) and Modified Read (MR) encoding
methods, and T.6 covers Modified Modified Read (MMR)
T.4/T.6Specifies the Fax Page Encoding Algorithm
PSTN
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
5
T.30Signaling and Negotiation Between Fax Machines
Basic G3 Fax Transaction Basic G3 Fax Transaction
The optional Called
Subscriber Identification
Sending Fax Receiving Fax
Offhook then dial
Ringing, Offhook,
Ans er
Subscriber Identification
(CSI) and Non-Standard
Facilities (NSF) messages
may be transmitted along
with the DIS message
Offhook, then dial
Calling Tone (CNG)
Answer
Called Tone (CED)
with the DIS message
The optional Transmitting
Subscriber Identification
(TSI) message may be
transmitted along with the
Digital Identification Signal (DIS)
T i i Ch k (TCF)
Digital Command Signal (DCS)
transmitted along with the
DCS message
Blue messages indicate
low speed T.30 signaling
and gold messages
Training Check (TCF)
Confirmation (CFR)
Fax Page Transmission and gold messages
represent higher speed
modulations used in training
and page transmission
End of Procedure (EOP)
Message Confirmation (MCF)
Fax Page Transmission
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
6
Disconnect (DCN)
Super G3 Fax
G3 Fax Call Initiation
Super G3 Fax
Super G3 uses the V.34
d l ti t hi
G3 Fax Call Initiation
CNG
CED
Originating
Fax
Terminating
Fax
modulation to achieve
page transmission
speeds up to 33.6 Kbps
CED
DIS
compared to the
maximum G3 speed of
14.4 Kbps
Super G3 Fax Call Initiation
CNG
Originating
F
Terminating
F
p
The signals used by SG3
are different than G3 and
tl
Fax Fax
ANSam
Calling Menu (CM)
are currently
incompatible with T.38
and Cisco fax relay.
Joint Menu (JM)
CJ Terminator
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
7
DIS
Transporting Fax Over IP Transporting Fax Over IP
The default behavior for Cisco voice gateways is to g y
initially handle all calls as voice calls so all fax calls
start as voice calls
C d i d t d i d t Compressed voice codecs are not designed to
handle modulated data so alternative, real-time
transport methods must be utilized
There are two principal methods of transporting fax
communications
1. Passthrough
2 Relay
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
8
2. Relay
Transporting G3 Fax Protocols Over IP Transporting G3 Fax Protocols Over IP
T.4/T.6Specifies the Fax Page Encoding Algorithm
PSTN
How Do Standard Fax Communications Integrate With IP Networks?
T.30Signaling and Negotiation Between Fax Machines
How Do Standard Fax Communications Integrate With IP Networks?
T.4/T.6Specifies the Fax Page Encoding Algorithm
Passthrough or Relay
Transport
IP
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
9
T.30Signaling and Negotiation Between Fax Machines
Passthrough Passthrough
Modulated fax data information is sampled and encoded as
standard Pulse Code Modulation (PCM) (i.e., G.711) and
encapsulated in Real Time Protocol (RTP) for transport
over IP just like a voice codec does for human speech
From the gateway perspective, this is more or less a G.711
voice call
Commonly referred to as Voice Band Data (VBD)
A l A l
End-to-End Modulated Data
Analog
Modulated Data
RTP Packet with PCM Payload
RTP RTP RTP
Analog
Modulated Data
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
10
IP
Relay Relay
The analog modulated fax data is demodulated by a Digital
Signal Processor (DSP) on the gateway and the binary Signal Processor (DSP) on the gateway and the binary
information is extracted
Binary information is passed over IP using one of several y p g
available relay protocols
A DSP on the destination gateway takes the binary
information from the relay packets and re modulates it into a information from the relay packets and re-modulates it into a
fax data signal on the telephony side
Analog Analog Relay Protocol
g
Modulated Data
g
Modulated Data
Relay Relay Relay
Relay Protocol
01101010 11000111 10100101 00101011
Relay
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
11
IP
The Switchover The Switchover
All modulated calls
TGW
OGW
begin as voice calls
The process that
transitions a gateway transitions a gateway
from a VoIP call to the
final fax media stream is
k th it h
VoIP Call
known as the switchover
There are a variety of
different switchover
Switchover
different switchover
mechanisms that we will
discuss throughout this
presentation
Fax Call
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
12
presentation
Fax/Modem Media
Fax Transport Methods
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
13
Fax Transport Methods Fax Transport Methods
Fax Fax
over IP
Passthrough Relay
Fax
Passthrough
(NSE)
Fax
Pass-through
(Protocol)
T.38
Fax Relay
(Protocol)
T.38
Fax Relay
(NSE)
Cisco
Fax Relay
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
14
( ) (Protocol) (Protocol) (NSE)
Fax Passthrough Switchover Methods Fax Passthrough Switchover Methods
Voice Voice
Mode
NSE-Based Protocol-Based NSE Based
Switchover
Protocol Based
Switchover
Fax
Passthrough
Fax
Pass through
Passthrough Switchover:
1. Codec upspeed
Passthrough Pass-through
2. VAD disabled on DSP
3. Jitter buffer transitions from
adaptive to a fixed optimum value
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
15
Fax Passthrough
Passthrough Terminology Passthrough Terminology
Fax Passthrough generically refers to passing fax calls over the
G 711 codec G.711 codec
Fax Passthrough can also refer to the more specific
passthrough transport method using an NSE switchover to pass
faxes over G.711
Modem Passthrough is a general term used to refer to an NSE-
based switchover to passthrough for both fax and modem calls
because it is the syntax used by the configuration command for
this feature this feature
The underlying operation of modem passthrough is different
depending on whether it is a fax or a high speed modem call, so
to avoid confusion we will refer to modem passthrough for a fax
call as fax passthrough and modem passthrough for a modem call as fax passthrough and modem passthrough for a modem
call as modem passthrough
Fax pass-through details a switchover to passthrough using the
call control protocol
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
16
Passthrough Switchover Mechanisms Passthrough Switchover Mechanisms
Tone
Detection
1
RTP RTP RTP NSE NSE NSE
NSE-Based
Switchover
2
IP
3
Protocol-Based
Switchover
H.323 SIP
OGW
TGW
Terminating Gateway (TGW) initiates the switchover to
passthrough upon detection of fax signals played by the
1
answering fax device
2
If NSE-based passthrough is configured, the TGW sends
NSE switchover packets in the RTP media stream upon NSE switchover packets in the RTP media stream upon
detection of CED (G3 fax) or ANSam (SG3 fax) tones
3
If fax pass-through is configured, the TGW initiates the
it h i th H 323 SIP ll t l t l
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
17
switchover in the H.323 or SIP call control protocols upon
detection of fax message flags
Payload Types Used for Modulated Data Payload Types Used for Modulated Data
The Payload Type eld identies the type of data being carried in the
RTP k t
Payload Type
Payload Encoding
RTP packet
0
8
18
G.711 -law
G.711 a-law
G 729
Static
18
90
96
G.729
RFC 2198 Passthrough Redundancy
Cisco Fax Relay Switchover
97
100
y
Cisco Fax Relay Switchover ACK
Named Signaling Event (NSE)
Dynamic
And
Unassigned
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
18
119
122
Cisco Text Relay
Cisco Fax Relay
g
Named Signaling Events (NSE) Named Signaling Events (NSE)
Cisco proprietary message sent as part of the RTP p p y g p
stream to signal an event, such as a switchover
This form of signaling is call control independent
RTP P l d RTP H d
The Event ID field uses Cisco-defined event
numbers to signal a specific task in-band
NSE
RTP Payload RTP Header
PT
Payload Type =100
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
19
R Volume E Event ID Duration
NSE Events Used for Modulated Data NSE Events Used for Modulated Data
The NSE Event ID eld uses Cisco-dened values
NSE
Description
to signal in-band a variety of tasks
192
193
Triggered by 2100Hz tone. Signals switchover to passthrough
Triggered by ANSam tone detection. Signal to disable ECANs
194
199
200
Triggered by 4s silence or carrier loss. Signals return to voice mode
Triggered by ANSam tone. Signals support of Cisco Modem Relay
T i d b f V21 P bl Si l it h t T38
200
201
202
Triggered by fax V.21 Preamble. Signals switchover to T.38
ACK to NSE-200 confirming peer gateway switchover to T.38
NACK to NSE-200 implying peer gateway cannot switchover to T38
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
20
202
203
NACK to NSE 200 implying peer gateway cannot switchover to T.38
Triggered by V.8 CM detection. Signals transition to Modem Relay
Fax Passthrough Switchover (G3) Fax Passthrough Switchover (G3)
TGW
OGW
Th i iti l V IP ll
VoIP Call
OGW: Upspeed codec
d i h
The initial VoIP call
can be setup by any
of the common call
control protocols
(H 323 SIP MGCP
and switch to
Passthrough mode
NSE-192 CED
(H.323, SIP, MGCP, or
SCCP)
NSE-192
TGW: Upspeed codec
and switch to
The underlying call
control protocol is
f th
and switch to
Passthrough mode
Fax Passthrough Call
unaware of the
transition to fax
passthrough due to the
protocol independent
t f th NSE
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
21
nature of the NSE
switchover
Fax Passthrough Switchover (SG3) Fax Passthrough Switchover (SG3)
TGW
OGW
The initial VoIP call
VoIP Call
The initial VoIP call
can be setup by any
of the common call
control protocols
(H 323 SIP MGCP or
ANSam NSE-192
(H.323, SIP, MGCP, or
SCCP)
NSE-192
TGW: Detect 2100 Hz
tone. Upspeed codec
and switch to
Passthrough mode
OGW: Upspeed
codec and switch to
Passthrough mode
NSE-193
NSE-193
SG3 Fax Call
Passthrough mode
OGW: Disable echo
cancellers
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
22
SG3 Fax Call
TGW: Detect phase
reversal of ANSam.
Disable ECAN
Confirming Fax Passthrough
Switchover Switchover
Before Switchover: show call active voice brief Before Switchover: show call active voice brief
11F1 : 10 2924510ms.1 +10530 pid:1 Originate 200 active
dur 00:00:07 tx:99/1903 rx:253/4993
IP 1 1 1 2:17932 SRTP: off rtt:0ms pl:4810/0ms lost:0/0/0 delay:60/60/70ms g729r8 IP 1.1.1.2:17932 SRTP: off rtt:0ms pl:4810/0ms lost:0/0/0 delay:60/60/70ms g729r8
VoIP Call
After Switchover: show call active voice brief After Switchover: show call active voice brief
11F1 : 10 2924510ms.1 +10530 pid:1 Originate 200 active
dur 00:00:22 tx:877/125447 rx:1040/128809
IP 1.1.1.2:17932 SRTP: off rtt:1ms pl:40/0ms lost:0/0/0 delay:60/60/60ms g711ulaw p y g
TextRelay: off
media inactive detected:n media contrl rcvd:n/a timestamp:n/a
long duration call detected:n long duration call duration:n/a timestamp:n/a MODEMPASS
nse buf:0/0 loss 0% 0/0 last 1359s dur:0/0s
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
23
Fax Passthrough Call
nse buf:0/0 loss 0% 0/0 last 1359s dur:0/0s
Fax Passthrough Call
Fax Passthrough Configuration Fax Passthrough Configuration
SIP/H.323:
dial-peer voice 1 voip
modem passthrough nse codec g711ulaw
(Note: Can be configured globally under voice service voip or the dialpeer for
SIP and H.323. The configuration under the dialpeer takes precedence.)
IP RTP RTP RTP
NSE NSE NSE
Feature: fax passthrough
MGCP:
mgcp modem passthrough voip mode nse
mgcp modem passthrough voip codec g711ulaw
SCCP:
voice service voip
mgcp modem passthrough voip codec g711ulaw
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
24
p
modem passthrough nse codec g711ulaw
(Note: Can only be configured globally under voice service voip for SCCP.)
Fax Pass-through Switchover (G3) Fax Pass through Switchover (G3)
TGW
OGW
Th i iti l V IP ll i
VoIP Call - SIP Signaling
P th h
CED T
The initial VoIP call is
setup using the SIP
call control protocol
g g
Pass-through
parameters for the
upcoming session
are established in
th SIP INVITE d
INVITE for G.711 Pass-through
CED Tone
V.21
Preamble
100 Trying
200 OK
the SIP INVITE and
200 OK messages
The V.21 Preamble of
the first T30 message
ACK
Fax Pass through Call
the first T.30 message
following the CED Tone
(i.e. DIS) is the trigger
for the transition to pass-
through
An H.245 Request
Mode for G.711 is used
to signal the switchover
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
25
Fax Pass-through Call
through
g
in H.323 to fax pass-
through mode
Confirming Fax Pass-through
Switchover Switchover
Before Switchover: show call active voice brief
126C : 58 11411290ms.1 +10520 pid:1 Originate 200 active
dur 00:00:08 tx:112/2154 rx:307/6092
IP 1.1.1.2:17620 SRTP: off rtt:1ms pl:5930/0ms lost:0/0/0 delay:60/60/70ms g729r8
VoIP Call
After Switchover: show call active voice brief
126C : 58 11411290ms.1 +10520 pid:1 Originate 200 active
dur 00:00:41 tx:1677/252554 rx:1953/258112
IP 1 1 1 2 17620 SRTP off rtt 4ms pl 40/0ms lost 1/0/0 dela 60/60/60ms g711 la IP 1.1.1.2:17620 SRTP: off rtt:4ms pl:40/0ms lost:1/0/0 delay:60/60/60ms g711ulaw
TextRelay: off
media inactive detected:n media contrl rcvd:n/a timestamp:n/a
long duration call detected:n long duration call duration:n/a timestamp:n/a
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
26
Fax Passthrough Call
long duration call detected:n long duration call duration:n/a timestamp:n/a
Fax Pass-through Call
Fax Pass-through Configuration Fax Pass through Configuration
SIP/H.323:
dial-peer voice 1 voip
fax protocol pass-through g711ulaw
Note: Can be configured globally under voice service voip or the dialpeer for
SIP and H.323. The configuration under the dialpeer takes precedence.
IP
Feature: fax pass-through
H.323 RM
SIP
INVITE
MGCP:
Fax pass-through is NOT supported for MGCP
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
27
SCCP:
Fax pass-through is NOT supported for SCCP
Fax Transport Methods Fax Transport Methods
Fax Fax
over IP
Passthrough Relay
Fax
Passthrough
(NSE)
Fax
Pass-through
(Protocol)
T.38
Fax Relay
(Protocol)
T.38
Fax Relay
(NSE)
Cisco
Fax Relay
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
28
( ) (Protocol) (Protocol) (NSE)
Fax Relay Switchover Methods Fax Relay Switchover Methods
Voice Voice
Mode
Protocol-Based Switchover
(Standards-Based)
RTP PT RTP PT
Switchover
NSE-Based
Switchover
T38 Fax
C
T38 Fax
T.38 Fax
Relay
Cisco Fax
Relay
T.38 Fax
Relay
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
29
Fax Relay
Cisco Fax Relay (CFR) Cisco Fax Relay (CFR)
Pre-standard and proprietary fax relay implementation for y y
G3 fax transmissions (no SG3 support)
Default fax transport method on most Cisco voice gateways
Uses a unique Payload Type switchover that is not used by
any other modulated communication transport method
RTP Payload
Type is set to
122 for Cisco
Demodulated fax HDLC
data carried in the Cisco
Fax Relay packet
Fax Relay data
UDP
Header
IP
Header
RTP
Header
Cisco Fax Relay Packet Cisco Fax Relay Packet
y p
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
30
Cisco Proprietary Formatting
Cisco Fax Relay Switchover (G3) Cisco Fax Relay Switchover (G3)
TGW
OGW
VoIP Call
The TGW noties the
OGWof the impending
V.21
Preamble
PT-96 CFR Switchover
OGW of the impending
switchover with a
special RTP message
using a PT value of 96
PT-96 CFR Switchover
The detection of fax
PT-97 CFR Switchover ACK
PT-97 CFR Switchover ACK
Cisco Fax Relay Call
flags (V.21 Preamble) at
the TGW triggers the
transition to CFR.
The CFR switchover
packet is ACKd with
PT-97 message
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
31
Cisco Fax Relay Call
g
Confirming Cisco Fax Relay Switchover Confirming Cisco Fax Relay Switchover
Before Switchover: show call active voice brief
121C : 26 1027355900ms.1 +10510 pid:1 Originate 200 active
dur 00:00:11 tx:117/2244 rx:424/8432
IP 1.1.1.2:18274 SRTP: off rtt:2ms pl:7810/0ms lost:0/0/0 delay:60/60/70ms g729r8
VoIP Call
After Switchover: show call active voice brief
121C : 26 1027355900ms.1 +10510 pid:1 Originate 200 active
dur 00:00:45 tx:1113/22164 rx:1001/19972
IP 1 1 1 2 182 4 SRTP ff 2 l 0/0 l 0/0/0 d l 0/0/0 i T R l ff IP 1.1.1.2:18274 SRTP: off rtt:2ms pl:0/0ms lost:0/0/0 delay:0/0/0ms cisco TextRelay: off
media inactive detected:n media contrl rcvd:n/a timestamp:n/a
long duration call detected:n long duration call duration:n/a timestamp:n/a
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
32
Fax Passthrough Call
Cisco Fax Relay Call
Cisco Fax Relay Configuration Cisco Fax Relay Configuration
SIP/H.323:
dial-peer voice 1 voip
fax protocol cisco fax protocol cisco
Note: CFR is the default fax configuration on most IOS gateways.
IP RTP RTP
PT-97 PT-96
Feature: Cisco Fax Relay
MGCP:
ccm-manager fax protocol cisco
mgcp fax t38 inhibit
SCCP:
voice service voip
fax protocol cisco
Note: Like CFR, T.38 is also enabled by default for
MGCP. However T.38 takes precedence over CFR and
must be explicitly disabled with mgcp fax t38 inhibit.
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
33
Note: CFR can only be configured under voice service voip for SCCP starting
in IOS Release 12.4(11)T.
T.38 Fax Relay T.38 Fax Relay
T.38 is the de facto standard for handling fax transmissions today
Cisco has always supported the 1998 version (known as version 0) Cisco has always supported the 1998 version (known as version 0)
of the ITU-T T.38 specification with UDP/UDPTL encapsulation
In IOS version 15.1(1)T Cisco started supporting SG3 fax over T.38
( ) (version 3)
Cisco products can use either NSEs (proprietary) or the call control
protocol (standards-based) to switch a call to T.38 mode p ( )
UDP Transport Layer
(UDPTL) header is simply
T.38 Internet Fax Packets (IFP)
and optional redundancy packets
UDP IP UDPTL
UDPTL Payload (T38 UDPTL Payload (T38 IFPs IFPs))
(UDPTL) header is simply
a 2 byte sequence number
and optional redundancy packets
are carried in the UDPTL payload
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
34
Header Header Header
UDPTL Payload (T.38 UDPTL Payload (T.38 IFPs IFPs))
UDP Encapsulated T.38
T.38 SwitchoverNSE (G3) T.38 Switchover NSE (G3)
TGW
OGW
The initial VoIP call
VoIP Call
OGW: Transition from
voice mode to T.38
can be setup by any
of the common call
control protocols
(H.323, SIP, MGCP, or
NSE-200
V.21
Preamble
(
SCCP)
NSE-201
The detection of fax
flags (V.21 Preamble) at
the TGW triggers the
t iti t T38
T.38 Fax Relay Call
transition to T.38
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
35
TGW: T.38 ACK
received, start T.38
session
T.38 SwitchoverH.323 (G3) T.38 Switchover H.323 (G3)
TGW
OGW
Th i iti l V IP ll i
VoIP CallH.323 Signaling
T.38 parameters for the
upcoming fax relay
The initial VoIP call is
setup using the H.323
call control protocol
g g
V.21
Preamble
H.245 Request Mode T.38
upcoming fax relay
session are negotiated
in the H.245 Request
Mode messages
H.245 Request Mode T.38 ACK
Closing of Voice Logical Channels
Each gateway closes
its voice media
T38 Fax Relay Call
The detection of fax
flags (V.21 Preamble)
at the TGW triggers the
transition to T.38
Opening of T.38 Logical Channels
channel and opens a
T.38 fax relay media
channel while
acknowledging the
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
36
T.38 Fax Relay Call
same events by the
other side
T.38 SwitchoverSIP (G3) T.38 Switchover SIP (G3)
TGW
OGW
Th i iti l V IP ll i
VoIP Call - SIP Signaling
T.38 parameters for the
upcoming fax relay
session are negotiated
The initial VoIP call is
setup using the SIP
call control protocol
g g
INVITE for T.38 mode
V.21
Preamble
session are negotiated
in the SIP INVITE and
200 OK messages
100 Trying
200 OK
The detection of fax
flags (V.21 Preamble)
at the TGW triggers the
T38 Fax Relay Call
gg
transition to T.38 ACK
T.38 fax relay
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
37
T.38 Fax Relay Call
switchover process
using the SIP protocol
stack is complete
T.38 SwitchoverMGCP (G3) T.38 Switchover MGCP (G3)
TGW
OGW
Call Agent
Th i iti l V IP ll i
VoIP Call - MGCP Signaling
The initial VoIP call is
setup using the MGCP
call control protocol
MGCP NTFY
V.21
Preamble
200 OK
200 OK
TGW notifies the CA
that fax V.21 flags are
detected
MDCX - T.38
CA instructs OGW to
switchover to T.38
with an MDCX
message 200 OK
200 OK
CA instructs TGW to
switchover to T38
MDCX- T.38
message
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
38
T.38 Fax Relay Call
200 OK
switchover to T.38
with an MDCX
message
T.38 SwitchoverSIP (SG3) T.38 Switchover SIP (SG3)
TGW
OGW
The initial VoIP call is
setup using the SIP
VoIP Call - SIP Signaling
ANSam is sent across in
the RTP media stream
ANS ANS
setup using the SIP
call control protocol
INVITE for T.38 mode
T.38 fax relay
switchover process
ANSam
(ANSam tunneled in voice codec)
CM
ANSam
100 Trying
200 OK
The detection of
using the SIP protocol
stack is complete
T38 SG3 Fax Relay Call
Calling Menu (CM) at
the OGW triggers the
transition to T.38
ACK
Note: CM is squelched
by the OGW DSP and
sent over T.38. The
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
39
T.38 SG3 Fax Relay Call
CM/JM exchange is
done entirely over T.38
prior to T.30 negotiation
Confirming T.38 Fax Relay Switchover Confirming T.38 Fax Relay Switchover
Before Switchover: show call active voice brief
11E2 : 4 2956390ms.1 +10500 pid:1 Originate 200 active
dur 00:00:11 tx:117/2244 rx:436/8643
IP 1.1.1.2:18496 SRTP: off rtt:3ms pl:4800/0ms lost:0/0/0 delay:60/60/70ms g729r8
VoIP Call
Aft S it h h ll ti i b i f After Switchover: show call active voice brief
11E2 : 4 2956390ms.1 +10500 pid:1 Originate 200 active
dur 00:00:46 tx:707/13770 rx:914/12339
IP 1 1 1 2:18496 SRTP: off rtt:3ms pl:4800/0ms lost:0/0/0 delay:60/60/70ms t38 IP 1.1.1.2:18496 SRTP: off rtt:3ms pl:4800/0ms lost:0/0/0 delay:60/60/70ms t38
TextRelay: off
media inactive detected:n media contrl rcvd:n/a timestamp:n/a
long duration call detected:n long duration call duration:n/a timestamp:n/a
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
40
Fax Passthrough Call
long duration call detected:n long duration call duration:n/a timestamp:n/a
T.38 Fax Relay Call
NSE-Based T.38 Configuration NSE Based T.38 Configuration
SIP/H.323:
dial-peer voice 1 voip
fax protocol t38 nse ls-redundancy 0 hs-redundancy 0 fallback cisco fax protocol t38 nse ls redundancy 0 hs redundancy 0 fallback cisco
Note: Configured globally under voice service voip or the dialpeer for SIP and
H.323. The dialpeer configuration takes precedence.
IP
Feature: NSE-Based T.38
RTP RTP RTP
NSE NSE NSE
SCCP
MGCP:
Enabled by default
SCCP:
voice service voip
fax protocol t38 nse ls-redundancy 0 hs-redundancy 0 fallback cisco
N t C l b fi d d i i i f SCCP Ci IOS t
Note: no mgcp fax t38 inhibit is required but is default.
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
41
Note: Can only be configured under voice service voip for SCCP. Cisco IOS gateway
support for NSE-based T.38 fax relay with the SCCP voice signaling protocol is
introduced in Cisco IOS Release 12.4(11)T.
Protocol-Based T.38 Configuration Protocol Based T.38 Configuration
SIP/H.323:
dial-peer voice 1 voip
fax protocol t38 version [0|3] ls-redundancy 0 hs-redundancy 0 fallback cisco
Note: version 0 configures legacy T.38 G3 fax support, whereas version 3 enables
the newly supported SG3 over T.38 feature [as of 15.1(1)T]
IP MGCP
NTFY
Feature: Protocol-Based T.38
H.323 RM
SIP INVITE
MGCP:
mgcp package-capability fxr-package
mgcp default package fxr package
SCCP:
SCCP does not support
mgcp default-package fxr-package
Note: These two commands are needed to go from
the default configuration of NSE-based T.38 to
protocol-based T.38. The command no mgcp fax t38
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
42
pp
protocol-based (version 0
or version 3) T.38
g
inhibit is required for both NSE and protocol-based
T.38, but it is the default.
MGCP does not offer any version 3 support.
QoS Considerations
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
43
Are VoIP QoS Policies Sufficient for
Fax? Fax?
In many cases, fax
Headquarters
devices are located where
VoIP is already installed
You can piggyback You can piggyback
fax traffic onto the same
QoS policy implemented
f V IP for VoIP
As a rule of thumb, if
good voice quality exists
IP
good voice quality exists
between locations, then
fax communications
should also work
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
44
should also work
Branch 1 Branch 2 Branch 3
QoS Network Factors QoS Network Factors
Terminating
Gateway
Originating
Gateway
Packet Loss Jitter
Gateway Gateway
T.38 T.38 T.38 T.38 T.38
Delay
Delay or latency: the amount of time it takes a packet to
Delay
travel from source to destination
Packet loss: the amount of packets that are
f l i i i t th d ti ti unsuccessful in arriving at the destination
Jitter: the measure of the variability over time of the
latency across a network
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
45
latency across a network
FoIP and Packet Loss FoIP and Packet Loss
Originating
Gateway
Terminating
Gateway
T.38 Fax Packets
Gateway Gateway
IFP 2
(Primary)
IFP 1
(Secondary)
IFP 3
(Primary)
IFP 2
(Secondary)
IFP 1
(Primary)
Fax over IP (FoIP) is generally more affected by packet
T.38 Fax Relay With Redundancy Level Set to 1
loss than VoIP
Ideally no packet loss should occur for a fax call
If packet loss is present then use T.38 with redundancy
Enabling T.38 redundancy requires more bandwidth
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
46
FoIP and Delay FoIP and Delay
Multiple IP and PSTN hops are
prime sources of additional delay
PSTN
IP
p y
Satellite links
cause large
Delay is not as impacting to FoIP compared to VoIP
PSTN
IP
IP
cause large
amounts of delay
Delay is not as impacting to FoIP compared to VoIP
FoIP calls have been known to handle network delays of 1 second
or more
However, as a best practice it is still recommended to minimize
network delays as much as reasonably possible because too
much delay will cause FoIP calls to fail
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
47
Watch out for multiple IP and PSTN hops and satellite links
FoIP and Jitter FoIP and Jitter
Variably spaced T.38
packets arrive at the
playout buffer and
Packets are re-sequenced
if necessary and placed in
Evenly spaced
packets are played
out to the DSP for p y
some may even be
out of sequence
if necessary and placed in
the required order for
playout
transmission on
the PSTN
DSP
IP
11
Fax
8
Fax
10
Fax
9
Fax
7
Fax
6
Fax
5
Fax
4
Fax
3
Fax
2
Fax
1
Fax
DSP
Codec
(T.38)
T.38 and Cisco fax relay use 300 ms fixed jitter or playout buffers
300 ms Fixed Playout Buffer
Fax passthrough uses the last voice mode setting before the
switchover
With large playout buffers, FoIP can handle larger amounts of jitter
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
48
g p y g j
than VoIP but as a best practice it is still recommended to keep
jitter to a minimum
QoS Design Parameters for Fax QoS Design Parameters for Fax
Delay Jitter Packet Loss
< 150 ms < 30 ms < 1%
Voice
(one-way,
(mouth to ear)
(average,
one-way)
< 1000 ms < 300 ms for None* unless
Fax
< 1000 ms < 300 ms for
fax relay,
< 30 ms for
passthrough
None , unless
using T.38
with
redundancy p g y
*Fax passthrough is very sensitive to packet loss and may be able to handle 0 1%0 2%loss
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
49
Fax passthrough is very sensitive to packet loss and may be able to handle 0.1% 0.2% loss
depending on when in the fax transaction the loss occurs and if it is consecutive packets. Cisco
fax relay can handle more loss than passthrough but T.38 with redundancy is still the best choice
for fax calls when packet loss is occurring.
QoS Marking Configuration on
Gateways Gateways
IP
T.38 T.38
Fax Gateway
dial-peer voice 100 voip
destination-pattern 100
session target ipv4:10.1.1.1
fax protocol t38 ls redundancy 0 hs redundancy 0
cae-dh-3660#show dial-peer voice 100
<snip>
type = voip, session-target = `ipv4:10.1.1.1',
technology prefix:
F d d t t Q S ki i DSCP b i l
fax protocol t38 ls-redundancy 0 hs-redundancy 0
fallback cisco
ip qos dscp cs3 signaling
technology prefix:
settle-call = disabled
ip media DSCP = ef, ip signaling DSCP = cs3,
<snip>
Fax, modem, and text QoS marking via DSCP can be simply
added at the dial-peer level for H.323 and SIP or globally for
MGCP with the command mgcp ip qos dscp
Dial peer QoS markings can be viewed with the command show Dial-peer QoS markings can be viewed with the command show
dial-peer voice
Default dial-peer QoS markings are af31 for signaling and ef
for media
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
50
for media
Cisco recommended best practice is to use cs3 for signaling
QoS Marking and Shaping Example QoS Marking and Shaping Example
CLASSIFICATION & MARKING QUEUEING & DROPPING POST-QUEUING OPERATIONS
Fax Fax
OGW TGW
I
P
dial-peer voice 919 voip
destination-pattern 919.......
class-map match-all fax_voice
match ip dscp ef
class-map match-any call_signaling
h i d 3
PC Phone Phone PC
p
session target ipv4:10.0.0.19
fax protocol t38 ls-redundancy 0 hs-redundancy 0
fallback cisco
ip qos dscp cs3 signaling
no vad
match ip dscp cs3
match ip dscp af31
!
policy-map WAN
class fax_voice
priority percent 33
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
51
no vad
priority percent 33
class call_signaling
bandwidth percent 5
class class-default
fair-queue
Voice and T.38 fax traffic can share the same
QoS marking and shaping.
Design Best Practices
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
52
Design Best Practices Design Best Practices
Unified Communications Integration Methods g
Cisco Unified Communications Manager, Gateway-
Controlled
Cisco Unified Communications Manager CA-Controlled Cisco Unified Communications Manager, CA-Controlled
Design considerations
Bandwidth utilization Bandwidth utilization
Super G3
Protocol and product support
SIP Trunking and FoIP Support
T.38: de facto standard
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
53
Gateway to Gateway Gateway to Gateway
Gateway Gateway
H.323/SIP Voice Signaling
H.323/SIP Fax Signaling
IP
Fax
Gateway
Fax
Gateway
Voice Media
Voice Media
NSE/PT Switchover
All signaling and media packets are exchanged between the voice
Fax Media
All signaling and media packets are exchanged between the voice
gateways
NSE-based switchovers (fax passthrough and NSE-based T.38) and
payload type switchovers (Cisco fax relay) happen within the voice media
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
54
p y yp ( y) pp
stream while protocol-based switchovers (fax pass-through and protocol-
based T.38) happen within the call control protocol
Cisco Unified Communications
Manager, Gateway-Controlled Manager, Gateway Controlled
H.323/SIP/MGCP/SCCP H.323/SIP/MGCP/SCCP
Unified CM
IP
Fax
Gateway
Fax
Gateway
Voice Media
NSE/PT Switchover
Switchover for fax
passthrough, NSE-
based T.38, and Cisco
fax relay occurs within
Voice media is
established by Unified
CM using the H.323,
SIP, MGCP, or SCCP
Fax Media
fax relay occurs within
the voice media stream
SIP, MGCP, or SCCP
protocols
Cisco Unified Communications Manager (Unified CM) controls the setup
of the initial voice call between the gateways using standard call control
signaling such as H.323, SIP, MGCP, and SCCP
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
55
The switchover to a fax media stream is handled by the gateways
themselves without the knowledge of Unified CM
Cisco Unified Communications
Manager, Gateway Controlled Manager, Gateway Controlled
Use Cisco Unified Communications Manager, Gateway-
C t ll d h Ci U ifi d C i ti Controlled, when a Cisco Unified Communications
Manager version does not support a certain fax media
negotiation in the call control protocol (like T.38)
NSE-based T.38 allows for T.38 to occur even if a Cisco
Unified Communications Manager version does not offer
T.38 support within the call control protocol
This integration method must also be used with non-
standard switchover mechanisms such as fax
passthrough and Cisco fax relay
Remember that NSE and PT switchovers are Cisco
proprietary and this integration method only works with
Cisco devices and not third-party equipment
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
56
p y q p
Cisco Unified Communications
Manager, CA-Controlled Manager, CA Controlled
Unified CM
IP
Fax
Gateway
Fax
Gateway
Voice Media
Switchover for protocol-
based T.38 is handled by
Unified CM within the
Voice media is
established by Unified
CM using the H.323,
Cisco Unified Communications Manager controls the setup of the initial
i ll b t th i t d t l th it h t T38
T.38 Media
call control protocol SIP, or MGCP protocols
voice call between the voice gateways and controls the switchover to T.38
fax relay
Instead of the gateways independently handling the switchover, Cisco
Unified Communications Manager coordinates the switchover using the
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
57
Unified Communications Manager coordinates the switchover using the
call control protocol
Cisco Unified Communications
Manager, CA-Controlled Manager, CA Controlled
Recommended as a best practice!
Only protocol-based T.38 (using H.323, SIP, or MGCP)
is capable of this integration method
This integration method is standards based and allows This integration method is standards-based and allows
for interoperability with third-party devices such as
gateways and fax servers
If Ci U ifi d C i ti M d i If Cisco Unified Communications Manager and voice
gateways support this integration method then this is
recommended compared to the Cisco Unified
Communications Manager Gateway-Controlled method Communications Manager, Gateway-Controlled method
which depends on proprietary switchover mechanisms
T.38 support within Cisco Unified Communications
Manager varies depending on software version
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
58
Manager varies depending on software version
Cisco Unified Communications
Manager and T.38 Support Manager and T.38 Support
Call Control Protocol
Support for T.38
Cisco Unified
Communications Manager
S f Software Release
H.323 4.1(1), 4.2(3), 5.0(1), and
6.0(1)
H.323 and MGCP 4.2(3) and 6.0(1)
H.323 and SIP 5.0(1) and 6.0(1)
H 323 SIP and MGCP 6 0(1) and later
Only release 6.0(1) and later offers support for
T38 ithi th ll t l t l f H 323 SIP
H.323, SIP, and MGCP 6.0(1) and later
T.38 within the call control protocols of H.323, SIP,
and MGCP
The SCCP protocol only handles T38 using an
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
59
The SCCP protocol only handles T.38 using an
NSE-based switchover
Design Best Practices Design Best Practices
Unified Communications Integration Methods g
Cisco Unified Communications Manager, Gateway-
Controlled
Cisco Unified Communications Manager CA-Controlled Cisco Unified Communications Manager, CA-Controlled
Design considerations
Bandwidth utilization Bandwidth utilization
Super G3
Protocol and product support
SIP Trunking and FoIP Support
T.38: de facto standard
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
60
FoIP Bandwidth Utilization FoIP Bandwidth Utilization
Different FoIP transports
use varying amounts of
Codec Bandwidth
1
use varying amounts of
bandwidth
On links where saving
G.711 (64 Kbps) 83 Kbps
G.729 (8 Kbps) 27 Kbps
bandwidth is a priority then
relay is a better choice
T.38 redundancy handles
G.723 (6.3 Kbps) 19 Kbps
Fax passthrough/
pass-through (G.711)
83 Kbps
y
packet loss much better
than fax passthrough/ pass-
through with redundancy
Fax passthrough (G.711)
with redundancy
170 Kbps
T.38 (no redundancy) 25 Kbps
2
( y) p
T.38 (redundancy level 1) 41 Kbps
2
T.38 (redundancy level 2) 57 Kbps
2
1
Values are approximate with Ethernet or
Frame Relay headers
2
Values are peak and only occur during the
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
61
( y ) p
Cisco fax relay 48 Kbps
2
Values are peak and only occur during the
sending of a page at 14.4 Kbps; gateways
can force lower fax speeds for additional
bandwidth savings
Methods for Handling SG3 Fax Calls Methods for Handling SG3 Fax Calls
Sending Fax Receiving Fax
OGW TGW
CNG CNG
ANSam
CM
Sending fax
i i i SG3 ll
Sg3-to-g3 spoofing
f OGW
Absence of SG3 CM causes
f G CM
DIS
initiates SG3 call
by sending CM
feature on OGW
blocks CM message
receiving fax to initiate G3
fax procedures (DIS message)
The sg3-to-g3 spoofing feature forces Super G3 fax calls to negotiate down to
G3 speeds for compatibility with T.38 or Cisco fax relay
Modem passthrough will handle SG3 calls at their native speeds Modem passthrough will handle SG3 calls at their native speeds
Fax pass-through is not compatible with SG3 because of its dependency on
the V.21 flags found in G3 negotiations for initiating the switchover
Starting in IOS 15 1(1)T SG3 over T38 is supported and these other methods
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
62
Starting in IOS 15.1(1)T SG3 over T.38 is supported and these other methods
for handling SG3 faxes are not necessary
Error Correction Mode (ECM) Error Correction Mode (ECM)
Sending Fax Receiving Fax
OGW TGW
CNG CNG
DIS (ECM enabled)
CED
DIS (ECM disabled)
DCS (ECM disabled)
( ) ( )
ECM support indicated
in DIS message
TGW overwrites
ECM setting to disabled
Sending fax believes receiving
fax does not support ECM
DCS (ECM disabled)
ECM ensures error-free transmission of fax pages
When ECM is disabled, fax pages can proceed with minor errors that may When ECM is disabled, fax pages can proceed with minor errors that may
or may not be noticeable
Cisco gateways can force ECM to be disabled when configured for T.38 or
Cisco fax relay
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
63
Disabling ECM may be useful for scenarios where network impairments
make it difficult for ECM to be efficient
T.38 Fallback T.38 Fallback
Receiving Fax
3845 TGW
dial-peer voice 100 voip
fax protocol t38 ls-redundancy 0
hs-redundancy 0 fallback cisco
Sending Fax
OGW
IP
T.38 supported by 3845
and call is successful
6608 TGW
Receiving Fax
T.38 call fails to 6608
but fallback to Cisco
fax relay is successful
For increased interoperability, T.38 fax relay can fall
back to Cisco fax relay or fax pass-through if the initial
T.38 negotiation fails
T.38 also has an automatic fallback mechanism that
tries a protocol based T38 fax relay switchover if the
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
64
tries a protocol-based T.38 fax relay switchover if the
configured NSE-based switchover fails
FoIP Call Control Protocol Support FoIP Call Control Protocol Support
Transport
M th d
H.323 SIP MGCP SCCP
Method
Cisco fax
relay
Yes* Yes* Yes* Yes*
NSE-based
T.38
Yes Yes Yes Yes
Protocol- Yes Yes Yes No
based T.38
Fax pass-
through
Yes Yes No No
Modem
passthrough
Yes Yes Yes Yes
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
65
*IOS platforms such as the 5350, 5400, and 5850 utilizing the
Nextport DSP cards do not support Cisco fax relay
FoIP Product Support Caveats FoIP Product Support Caveats
Cisco ATA only supports NSE-based passthrough
( d th h) (modem passthrough)
SCCP voice gateways (IOS or VG248) do not support
any protocol-based switchovers(like fax pass-through y p ( p g
and protocol-based T.38), so NSE-based T.38 and
modem passthrough must be used
MGCP gateways do not support fax pass-through, only MGCP gateways do not support fax pass through, only
modem passthrough can be utilized
Legacy Nextport DSP platforms (AS5350, AS5400, and
AS5850) do not support Cisco fax relay AS5850) do not support Cisco fax relay
Legacy 6608 and 6624 cards only support Cisco fax
relay and modem passthrough
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
66
SIP Trunks and FoIP SIP Trunks and FoIP
SIP Trunks typically use three different methods to transport
standard G3 fax calls
G.711 Voice - All calls (voice and fax) are always sent using the G.711 voice
codec
Pass-through - A SIP RE-INVITE message is used to up-speed to G.711 when a
fax tone is detected
T.38 Fax Relay Call is switched to T.38 fax relay using a SIP RE-INVITE when
a fax tone is detected
Fax Method T38 Fax Relay G 711 Voice Pass through Fax Method T.38 Fax Relay G.711 Voice Pass-through
Pros Highest fax success rates
can be achieved
Cleanest solution from
signaling and media point
f i
Most widely deployed
Simplest solution
Provides bandwidth savings as
G.729 voice call only upspeeds
to G.711 if call is fax
of view
Use less bandwidth
than G.711
Cons Interoperability between
different vendors
Consumes a large amount of
bandwidth for all calls
Interoperability between
different vendors
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
67
Not offered by many
Service Providers
Sensitive to impairments, no
redundancy
Not supported by Cisco Unified
Communications Manager
Sensitive to impairments, no
redundancy
T.38: De Facto Standard for FoIP T.38: De Facto Standard for FoIP
T.38 CFR
Fax Pass-
through
Fax
Passthrough
Standards-Based,
Third-Party
Integration
Low Bandwidth
Redundancy
Fallback
Mechanisms
SRTP and CRTP
Support
Legacy Platform
Support
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
68
T.38 Should Always Be Your First Choice When Handling FoIP Traffic!
Troubleshooting
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
69
Call Troubleshooting Stages Call Troubleshooting Stages
Any fax call can be
Gateway Gateway
broken down into three
troubleshooting stages
1. Initial Voice Call
H.323/SIP/MGCP/SCCP Call Setup
1. Initial Voice Call
2. Switchover
3. Fax Media
Voice Media
Once the problem has
been narrowed down to
a specific stage then
NSE or Protocol-Based Switchover
a specific stage, then
the appropriate
troubleshooting strategy
can be implemented
Fax Media
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
70
can be implemented
Packet Captures
Capture media
and call control
Packet Captures
Unified CM
Capture media
and call and call control
Voice
Gateway
Capture
call control
Fax
Voice
Gateway
and call
control
Fax
Gateway
IP
Gateway
Fax
Packet captures are one of the best tools for
troubleshooting fax problems
Ideally captures should be made at both endpoints to Ideally captures should be made at both endpoints to
ensure that both signaling and media are seen
Cisco Unified Communications Manager only
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
71
g y
participates in the call control signaling
Packet Captures Packet Captures
Packet capture software such as Wireshark is one of
th b t t l f t bl h ti ith NSE
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
72
the best tools for troubleshooting either NSE or
protocol-based switchovers as well as the fax media
Packet Captures: VoIP Call Analysis Packet Captures: VoIP Call Analysis
For protocol-based
switchovers like fax pass- switchovers like fax pass
through and T.38, the VoIP
Call Analysis feature in
Wireshark offers a graphical
means for analyzing the means for analyzing the
switchover
VoIP Call Analysis also
breaks down the T.38 media
stream, showing the T.30
signaling and page
transmission
S l S i i f Select Statistics from top
of Wireshark window, VoIP
Calls, click on a call, and
then click Graph
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
73
Troubleshooting NSE-Based
Switchovers Switchovers
debug voip rtp session named-event
Jan 10 22:01:58.471: s=DSP d=VoIP payload 0x64 ssrc 0x1EF2 sequence 0x1FBD
timestamp 0x20631C26
Jan 10 22:01:58.471: Pt:100 Evt:192 Pkt:00 00 00 <Snd>>>
Jan 10 22:01:58.483: s=VoIP d=DSP payload 0x64 ssrc 0x9A20101 sequence 0x18C2
timestamp 0xADA80F0A
Jan 10 22:01:58.483: <<<Rcv> Pt:100 Evt:192 Pkt:00 00 00
Use the command debug voip rtp session
named-event to view NSE-related switchovers and
events on IOS gateways, such as fax passthrough and
NSE b d T38 NSE-based T.38
Most common cause of problems with NSE-based
switchovers are an incorrect configuration or a
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
74
switchovers are an incorrect configuration or a
transcoder or proxy in the call path
Packet Captures: NSE Switchover Packet Captures: NSE Switchover
The default RTP payload type
for NSE messages is 100
The NSE Event ID is shown in
the payload as a hex value. In
this case 0xC8 is 200 in decimal
(NSE-200), which is a T.38 fax
relay switchover request
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
75
Troubleshooting PT-Based Switchovers Troubleshooting PT Based Switchovers
debug voip rtp session named-event
Jan 10 16:03:32 838: s=VoIP d=DSP payload 0x60 ssrc 0x0 sequence 0x0 Jan 10 16:03:32.838: s=VoIP d=DSP payload 0x60 ssrc 0x0 sequence 0x0
timestamp 0x0
Jan 10 16:03:32.838: <<<Rcv> Pt:96 Evt:0 Pkt:03 00 02
Jan 10 16:03:32.838: s=DSP d=VoIP payload 0x61 ssrc 0xFFEB sequence 0x0 p y q
timestamp 0x0
Jan 10 16:03:32.838: Pt:97 Evt:0 Pkt:07 00 02 <Snd>>>
Jan 10 16:03:32.842: s=DSP d=VoIP payload 0x60 ssrc 0xFFE2 sequence 0x0
timestamp 0x0
Jan 10 16:03:32.842: Pt:96 Evt:0 Pkt:07 00 02 <Snd>>>
Jan 10 16:03:32.842: s=VoIP d=DSP payload 0x61 ssrc 0x0 sequence 0x0 timestamp 0x0
Jan 10 16:03:32 842: <<<Rcv> Pt:97 Evt:1 Pkt:B8 01 E0
Like NSE-based switchovers, confirm RTP payload
type (PT) switchovers as utilized by Cisco fax relay
i th d d b i t i d
Jan 10 16:03:32.842: <<<Rcv> Pt:97 Evt:1 Pkt:B8 01 E0
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
76
using the command debug voip rtp session named-
event
Troubleshooting Protocol-Based
SwitchoversH.323 and T.38 Switchovers H.323 and T.38
H.245 Request Mode message
(from the terminating side)
Voice
Gateway
Voice
Gateway
( g )
initiates the T.38 switchover
Gateway
*Feb 3 04:34:53.070: H245 MSC OUTGOING PDU ::=
value MultimediaSystemControlMessage ::= request :
tM d
Gateway
Voice Media
requestMode :
<SNIP>
type dataMode :
{
application t38fax :
{
a
s
n
1
H 245 R t M d d ACK t38FaxProtocol udp : NULL
t38FaxProfile
{
fillBitRemoval FALSE
transcodingJBIG FALSE
transcodingMMR FALSE
b
u
g

h
2
4
5

H.245 Request Mode and ACK
Closing of G.711 Logical Channels
Opening of T38 Logical Channels
T.38
g
version 0
t38FaxRateManagement transferredTCF : NULL
t38FaxUdpOptions
{
t38FaxMaxBuffer 200
t38FaxMaxDatagram 72
d
e
b
Opening of T.38 Logical Channels
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
77
t38FaxMaxDatagram 72
t38FaxUdpEC t38UDPRedundancy : NULL
<SNIP>
Troubleshooting Protocol-Based
SwitchoverSIP and T.38 Switchover SIP and T.38
SIP re-INVITE message (from
the terminating side) initiates the
Voice
G t
Voice
G t
g )
switchover to T.38
*Feb 3 22:32:18.514: //-1/xxxxxxxxxxxx/SIP/Msg/ccsipDisplayMsg:
Sent:
i 1 2 18 110 86 060 S /2 0
Gateway Gateway
INVITE
INVITE sip:172.18.110.86:5060 SIP/2.0
Via: SIP/2.0/UDP 14.1.97.15:5060;branch=z9hG4bK618FF
From: <sip:100@14.1.97.15>;tag=4320068C-1F6B
<SNIP>
v=0
o=CiscoSystemsSIP-GW-UserAgent 8190 1299 IN IP4 14.1.97.15
e
s
s
a
g
e
s
Voice Media
re-INVITE
ACK
200 OK
s=SIP Call
c=IN IP4 14.1.97.15
t=0 0
m=image 16582 udptl t38
c=IN IP4 14.1.97.15
a=T38FaxVersion:0
u
g

c
c
s
i
p
m
ACK
a=T38MaxBitRate:14400
a=T38FaxFillBitRemoval:0
a=T38FaxTranscodingMMR:0
a=T38FaxTranscodingJBIG:0
a=T38FaxRateManagement:transferredTCF
a=T38FaxMaxBuffer:200
d
e
b
u
T.38
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
78
a T38FaxMaxBuffer:200
a=T38FaxMaxDatagram:72
a=T38FaxUdpEC:t38UDPRedundancy
Troubleshooting Protocol-Based
SwitchoverMGCP and T.38 Switchover MGCP and T.38
A NTFY containing
FXR/t38(start) initiates
*Feb 4 17:01:36.597: MGCP Packet sent to 172.18.110.85:2427--->
NTFY 935850485 S2/SU0/DS1-0/23@CAE-DH-3845-2 MGCP 0.1
FXR/t38(start) initiates
the MGCP T.38
switchover
Unified CM
NTFY 935850485 S2/SU0/DS1 0/23@CAE DH 3845 2 MGCP 0.1
N: ca@172.18.110.85:2427
X: 17
O: FXR/t38(start)
<---
CAE-DH-3845-2#show ccm-manager
MGCP Domain Name: CAE-DH-3845-2
Priority Status Host
==============================================
Primary Registered CAE-DH-7845 (172 18 110 85) Primary Registered CAE-DH-7845 (172.18.110.85)
First Backup None
Second Backup None
Current active Call Manager: 172.18.110.85
<SNIP>
Voice Media
TIP: Omission of the configuration
command mgcp package-capability fxr-
package is the most common cause of
Voice
Gateway
(Sending)
Voice
Gateway
(Receiving)
T.38
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
79
package is the most common cause of
MGCP T.38 failing
Troubleshooting Protocol-Based
Switchovers With Cisco RTMT Switchovers With Cisco RTMT
Ci R l Ti M it i T l (RTMT) l i
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
80
Cisco Real Time Monitoring Tool (RTMT) log viewer
can show protocol-based switchover information
Troubleshooting the Fax/Modem Media:
Gateway DSP Statistics Gateway DSP Statistics
CAE-DH-3845-2#terminal monitor
CAE-DH-3845-2#show voice call 0/1/1
0/1/1
t l l 0 t t S CONNECT l l 1 t t FXSLS CONNECT vtsp level 0 state = S_CONNECTvpm level 1 state = FXSLS_CONNECT
vpm level 0 state = S_UP
calling number , calling name unavailable, calling time 02/03 04:20
CAE-DH-3845-2# ***DSP FAX RELAY STATISTICS***
Max Jit Depth: 36, Max Nwk RxQ Depth 3, Jitter Overflow Pkt Drops: 0 p , Q p , p
Nwk RxQ Overflow: 0, Tx Pkts: 247, Tx Pkts Drops(Nwk Busy): 0
Rx Pkts: 677, Rx Pkts Loss: 0, Rx Invalid Pkts: 0, Rx Pkts Out Of Seq: 0
Recent Hi-Speed Modulation: V.17/long/14400TX Pages: 2
Max SendInQ Depth 2, Max RecvOutQ Depth 2
Max Hi Speed Buf Usage 54 SendInQ Overflow 0 RecvOutQ Overflow 0
DSP packet statistics on Cisco voice gateways provide IP network
information about the active, incoming media stream for
fax/modem passthrough Cisco fax relay and T38
Max Hi-Speed Buf Usage 54, SendInQ Overflow 0, RecvOutQ Overflow 0
fax/modem passthrough, Cisco fax relay, and T.38
Errors in these stats typically indicate packet loss and/or jitter
problems
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
81
The command show call active voice brief also provides some
DSP statistics and information about the media stream
Fax Relay Message Debugging Fax Relay Message Debugging
Voice
Gateway
fr-msg-det
Message received by DSP
Fax
y
g y
fr-msg-tx
IP
The command debug fax relay t30 all-level-1 displays
fr-msg-tx
Message transmitted by DSP
The command debug fax relay t30 all level 1 displays
T.30 messages for T.38 and Cisco fax relay from the
DSPs perspective on the POTS/PSTN side of the
gateway ga e ay
Inbound messages are flagged fr-msg-det and
messages sent by the DSP out the telephony interface
are indicated by fr msg tx
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
82
are indicated by fr-msg-tx
Sample Output From
debug fax relay t30 all-level-1 debug fax relay t30 all level 1
Nov 2 17:19:22.928: 0/0/0 (42) 626612205 fr-entered=10(ms)
timestamp=626613325 fr-msg-det NSF timestamp 626613325 fr msg det NSF
timestamp=626614305 fr-msg-det CSI
timestamp=626615005 fr-msg-det DIS
timestamp=626617385 fr msg tx TSI
Output from the Cisco
timestamp=626617385 fr-msg-tx TSI
timestamp=626618185 fr-msg-tx DCS
timestamp=626623295 fr-msg-det CFR
Output from the Cisco
voice gateway command:
debug fax relay t30 all-level-1
timestamp=626648365 fr-msg-tx EOP
timestamp=626649985 fr-msg-det MCF
timestamp=626652165 fr-msg-tx DCN
This debug is only valid for T.38 and Cisco fax relay as
the gateway plays an active role in modulating and
Nov 2 17:20:03.208: 0/0/0 (42) 626652485 fr-end-dcn
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
83
the gateway plays an active role in modulating and
demodulating the T.30 fax messages
Tips for Using
debug fax relay t30 all-level-1 debug fax relay t30 all level 1
Primary debug used for troubleshooting T.38 and Cisco
f l bl Ci i t fax relay problems on a Cisco voice gateway
Only low-speed T.30 messages are displayed so high
speed communications such as trainings and page p g p g
transmissions are not shown
If there are multiple fax calls occurring at once you can
specify a called or calling number to only view the debug specify a called or calling number to only view the debug
messages for a specific call
The level-2 option for this debug is only readable by
the DSP vendor and it should not be enabled the DSP vendor and it should not be enabled
This debug (as well as packet captures) is useful for
viewing unique T.30 problem signatures
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
84
Unique T.30 Debug Signature: Delay Unique T.30 Debug Signature: Delay
T.30 Negotiation
Failure With a Likely
Cause of Too Much
Voice
G t
Voice
G t
IP
PSTN
Cause of Too Much
Delay Between the
Fax Endpoints
Gateway Gateway
NSF/CSI/DIS
fr-entered (10ms)
fr-msg-det NSF
DCS (train)
debug fax
relay t30 all-level-1
T.30 messages are not being
received before previous
messages are retransmitted
Common ca ses of dela
g
fr-msg-det CSI
fr-msg-det DIS
fr-msg-tx DCS
fr-msg-det CSI
NSF/CSI/DIS
DCS (train)
Common causes of delay
are satellite links or multiple VoIP
hops
Adjusting the playout buffer on
fr msg det CSI
fr-msg-det DIS
fr-msg-tx DCS
fr-msg-det NSF
fr-msg-det CSI
NSF/CSI/DIS
DCS (train)
Adjusting the playout buffer on
Cisco voice gateways may help
For example, configure playout-
delay fax 100 under the
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
85
fr-msg-det CSI
fr-msg-det DIS
fr-msg-tx DCS
DCS (train) delay fax 100 under the
appropriate dial-peer to lower the
playout buffer to 100 ms
Unique T.30 Debug Signature:
Corruption Corruption
IP
PSTN
T.30 Negotiation
Failure With a Likely
C f C t d
Voice
Gateway
Voice
Gateway
IP
PSTN Cause of a Corrupted
Training Signal
NSF/DIS
debug fax
l t30 ll l l1
FTT
TSI/DCS (train)
Fax machine is receiving a
corrupted training signal and
sending an FTT message
fr-entered=10(ms)
fr-msg-det NSF
fr-msg-det DIS
f t TSI
relay t30 all-level1
FTT
TSI/DCS (train)
g g
Most common cause of a
corrupted training is PSTN
impairments, such as clock slips
fr-msg-tx TSI
fr-msg-tx DCS
fr-msg-det FTT
fr-msg-tx TSI
f t DCS
TSI/DCS (train)
First step is to always confirm
that any digital circuits are free
of errors
fr-msg-tx DCS
fr-msg-det FTT
fr-msg-tx TSI
fr-msg-tx DCS
FTT
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
86
fr-msg-det FTT
Viewing Digital Circuit Errors Viewing Digital Circuit Errors
T1 0/1/0 is up.
Applique type is Channelized T1
Output from the Cisco
voice gateway command:
Applique type is Channelized T1
Cablelength is long gain36 0db
No alarms detected.
alarm-trigger is not set
S ki ti 3 Cl ti 10
voice gateway command:
show controller t1
Soaking time: 3, Clearance time: 10
AIS State:Clear LOS State:Clear LOF State:Clear
Version info Firmware: 20060711, FPGA: 13, spm_count = 0
Framing is ESF, Line Code is B8ZS, Clock Source is Line.
CRC Threshold is 320. Reported from firmware is 320.
Data in current interval (729 seconds elapsed):
0 Line Code Violations, 0 Path Code Violations
24 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins p , , , g
24 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Total Data (last 24 hours)
0 Line Code Violations, 0 Path Code Violations,
2989 Slip Secs 0 Fr Loss Secs 0 Line Err Secs 0 Degraded Mins
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
87
2989 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins,
2989 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Confirming the Training in T.38 Confirming the Training in T.38
The TCF or training signal is a pattern of all 0s
Other hex values shown in the TCF decode can cause
the training to fail
This training pattern can be checked within Wireshark
to confirm its integrity
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
88
to confirm its integrity
Scan Line Corruption Scan Line Corruption
Snippet from a corrupted fax page
In addition to training problems, PSTN errors and impairments can
also cause scan lines to become corrupted
Problem will not be viewable with Error Correction Mode (ECM)
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
89
Problem will not be viewable with Error Correction Mode (ECM)
faxes but instead faxes will take a long time to complete or fail
completely
Audio Trace of Scan Line and
Training Corruption Training Corruption
DCS-train
DCN
Page
EOP
DIS CFR MCF
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
90
Preventing Corruption of the
Training and Scan Lines Training and Scan Lines
Ensure that all digital circuits in the fax path are
error free
Configure the clocking correctly on Cisco voice
gateways to avoid slips on digital PSTN interfaces gateways to avoid slips on digital PSTN interfaces,
which are notorious for causing training and scan
line corruption
M k th t th IP th i f f jitt d Make sure that the IP path is free from jitter and
packet loss
Major corruption problems prevent the fax Major corruption problems prevent the fax
endpoints from training while less severe problems
may allow training to occur but fax page data is
affected
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
91
Fax Servers
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
92
What Is a Fax Server? What Is a Fax Server?
H.323/SIP Call Setup
G 711 Voice
Fax Cisco Voice
G.711 Voice
T.38 Fax
A software application running on a server that
Server Gateway
A software application running on a server that
processes fax calls over a PSTN or IP connection
A Cisco Unified Communications Manager for g
handling fax traffic
Provides secure, automated, and efficient handling of
i ti f t ffi
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
93
an organizations fax traffic
Fax Server Components Fax Server Components
Server
Fax
Server
Hardware
Application Software
Fax
Engine
Server hardware requirements are defined by fax
server vendors
Application software is produced by a variety of
vendors and it handles administration configuration vendors and it handles administration, configuration,
and integration functions
Fax engine is either hardware (typically a PCI fax board
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
94
Fax engine is either hardware (typically a PCI fax board
installed into the server) or software-based
Why Migrate to a Fax Server? Why Migrate to a Fax Server?
Security
Faxes can be directed to a specific user or
storage system with limited access g y
Workflow
Integration
Fax servers can help automate process
management procedures and specific
workflow applications workflow applications
Archiving
Faxes can efficiently stored and indexed to
comply with industry mandates such as
Sarbox or SOx and HIPAA
Accountability
All faxes sent or received along with
confirmation of their success or failure can
be handled with a fax server solution
MFP Integration
MFPs integrate directly with fax servers for
increased cost savings and flexibility
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
95
Email Integration
Users have the ability to send/receive faxes
directly via email
Fax Server Access Methods Fax Server Access Methods
Fax
Server
IP
Web
Browser
API
IP
Mail
Server
Client
Software
Application
MFP
Application
Users can access a fax server through interfaces such as a
web browser, email, an MFP, and client software
Other devices and automated processes can communicate
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
96
Other devices and automated processes can communicate
directly with a fax server through different APIs
Fax Server Integration With Cisco
Unified Communications Manager Unified Communications Manager
Fax
Server
Unified CM
H.323/SIP
H.323/SIP
H.323/SIP
Server
Cisco Voice
G t
Fax servers communicate with Cisco Unified Communications
Manager using H.323 or SIP
Gateways
Regardless of the call control protocol between the fax server and
Cisco Unified Communications Manager, H.323, SIP, or MGCP can be
used between Cisco Unified Communications Manager and the
gateway
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
97
g y
T.38 media flows between the fax server and the voice gateways
Integrating a Fax Server with a SIP
Trunk Trunk
T.38 fax relay is the best choice for sending faxes over SIP
trunks
No special modifications or configurations are typically
necessary on a fax server to utilize SIP trunks, especially
when a border element is present when a border element is present
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
98
Cisco Fax Server: Centralized Solution Cisco Fax Server: Centralized Solution
Fax
Server
Fax
Client
Email
Client Fax
Voice
Gatewayy
Unified CM
MFP
Cisco fax server is the Open Text Fax Server (RightFax)
application on a Cisco MCS server
A full featured, scalable enterprise solution that is centralized and
supports both hardware and software-based fax engines to handle
various integration solutions
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
99
Deployed internally by Cisco in conjunction with Multifunction
Peripherals (MFPs)
AXP-XMediusFAX: Branch Solution AXP XMediusFAX: Branch Solution
Branches typically have little IT infrastructure and yp y
staff to maintain server and application hardware
Integrate fax server directly into the local ISR
t gateway
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
100
Cisco AXP: XmediusFAX Solution Cisco AXP: XmediusFAX Solution
XMediusFAX
APPRE
Module
AXP
XMediusFAX
Cisco ISR Gateway
Module
XmediusFAX server on an ISR blade
AXP runs a hardened Linux OS and provides a virtualized hosting
environment
XmediusFAX integrates directly with ISR architecture and is able g y
to be configured and managed through the IOS CLI
Functionality of a standalone fax server with no footprint
Add th d f ffi i tl h dli f t ffi t th l l
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
101
Addresses the need for efficiently handling fax traffic at the local
branch level
Complete Your Online
Session Evaluation Session Evaluation
Give us your feedback and you
ld i f b l i could win fabulous prizes.
Winners announced daily.
Receive 20 Cisco Preferred
A i t f h i Access points for each session
evaluation you complete.
Complete your session
evaluation online now (open a evaluation online now (open a
browser through our wireless
network to access our portal)
or visit one of the Internet
Dont forget to activate your
stations throughout the
Convention Center.
g y
Cisco Live and Networkers Virtual
account for access to all session
materials, communities, and on-demand
and live activities throughout the year.
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Presentation_ID
102
g y
Activate your account at any internet
station or visit www.ciscolivevirtual.com.

You might also like