You are on page 1of 84

UC Troubleshooting

Instructor: Goldman Lee, Triple CCIE #8269 (RS/SP/Sec)


company email: goldman.lee@globalknowledge.com.sg facebook email: goldmanlee@ymail.com add me today! :)

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

Agenda
What this Course is? An environment where student can learn by doing (25% brief concise theory, 75% hands on labs)

An advanced course, enterprise-focused

What this Course is Not?

A course that teaches fundamental


A course that teaches design in detail

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

Troubleshooting Signaling Protocol Overview


Protocol Description ITU standard protocol for interactive conferencing; evolved from H.320 ISDN standard; flexible, complex IETF standard for PSTN gateway control; thin device control IETF protocol for interactive and noninteractive conferencing; simpler, but less mature, than H.323 Cisco proprietary protocol used between Cisco Unified Communications Manager and Cisco VoIP phones

H.323
MGCP SIP

SCCP or Skinny

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

Application of H.323
H.323 suite:
Peer-to-peer protocol Gateway configuration necessary because gateway must maintain dial plan and route pattern. Examples: Cisco VG224 Analog Phone Gateway (FXS only) and, Cisco 2800 Series and, Cisco 3800 Series routers.

PSTN H.323 Q.921 Q.931


UC Troubleshooting 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Application of MGCP
MGCP:
Works in a client/server architecture Simplified configuration

Cisco Unified Communications Manager maintains the dial plan


Examples: Cisco VG224 Analog Phone Gateway (FXS only) and, Cisco 2800 Series and , Cisco 3800 Series routers Cisco Catalyst operating system MGCP example: Cisco Catalyst 6000 WS-X6608-T1 and Catalyst 6000 ws-X6608-E1

PSTN MGCP
Q.931
UC Troubleshooting 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Q.921

Application of SIP
SIP:
Peer-to-peer protocol. Gateway configuration is necessary because the gateway must maintain a dial plan and route pattern. Examples: Cisco 2800 Series and Cisco 3800 Series routers.

PSTN
SIP Q.921

Q.931
UC Troubleshooting 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Application of SCCP
SCCP
Works in a client/server architecture. Simplified configuration. Cisco Unified Communications Manager maintains a dial plan and route patterns. Examples: Cisco VG224 (FXS only) and, Cisco VG248 Analog Voice Gateways, Cisco ATA 186, and Cisco 2800 Series with routers FXS ports.
PSTN

SCCP
SCCP Endpoint
7

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

MGCP Registration
The gateway boots up.

A TCP connection is opened. Restart in Progress (RSIP) informs Cisco Unified CallManager that gateway is being brought into service. Acknowledgment (ACK)

TCP RSIP

TCP

A TCP connection is opened.

OK AUEP OK RQNT

Acknowledgment (ACK) Cisco Unified CallManager sends an Audit Endpoint (AUEP) per endpoint. Cisco Unified CallManager sends a Request Notify (RQNT) per endpoint.

Acknowledgment (ACK)

OK

Endpoints are now registered in Cisco Unified CallManager


UC Troubleshooting 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential

MGCP Gateway Registration Issues


Cisco IOS MGCP gateway registration issues:
Network connectivity issues Gateway issues Cisco Unified CallManager issues

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

Rundown of Cisco IOS MGCP Gateway Communications


Understanding the following Cisco IOS MGCP gateway behaviors is critical to effective troubleshooting:
Endpoints on the gateway will register with Cisco Unified CallManager server. Cisco Unified CallManager must have the gateway added to its configuration database. Endpoints are under the control of Cisco Unified CallManager. MGCP configuration on the gateway must be performed initially. The ability to download configuration files to the MGCP gateway enables adds, moves, and changes to MGCP endpoints to be performed in the Cisco Unified CallManager web interface.

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

10

Verify MGCP Gateway Status


Pod1HQ#show ccm-manager MGCP Domain Name: Pod1HQ Priority Status Host ============================================================ Primary Registered 10.1.1.2 First Backup Backup Ready 10.1.1.1 Second Backup None Current active Call Manager: Backhaul/Redundant link port: Failover Interval: Keepalive Interval: Last keepalive sent: Last MGCP traffic time: Last failover time: Last switchback time: Switchback mode: MGCP Fallback mode: Last MGCP Fallback start time: Last MGCP Fallback end time: MGCP Download Tones: Configuration Error History: FAX mode: cisco 10.1.1.2 2428 30 seconds 15 seconds 10:52:49 UTC 12:33:37 UTC 12:33:37 UTC 12:33:07 UTC Graceful Not Selected None None Disabled

Jun Jun Jun Jun

17 17 17 17

2006 2006 2006 2006

(elapsed time: 1:41:10) (elapsed time: 00:00:22) from (10.1.1.2) from (10.1.1.1)

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

11

Verify MGCP Gateway Status

To validate the configuration of an MGCP gateway:


Choose Device > Gateway > Find. Click See Endpoints on the desired gateway. Select the desired endpoint to validate the configuration and status.

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

12

Viewing Cisco Unified CallManager Event Log Messages


Enum Definitions -Device type. [Optional] 120 MODEL_MGCP_STATION 121 MODEL_MGCP_TRUNK 125 MODEL_TRUNK 254 MODEL_UNKNOWN_MGCP_GATEWAY 255 MODEL_UNKNOWN

Enum Definitions -Reason Code [Optional]. Value 1 Unknown 2 NoEntryInDatabase 3 DatabaseConfigurationError 4 DeviceNameUnresolveable 5 MaxDevRegExceeded 6 ConnectivityError 7 InitializationError 8 DeviceInitiatedReset 9 CallManagerReset 10
UC Troubleshooting

Definition

DeviceUnregistered
2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential

13

Cisco IOS MGCP Gateway Commands


Pod1HQ#

show ccm-manager

Displays a list of Cisco Unified CallManager servers and their current status and availability
Pod1HQ#

show mgcp endpoint

Displays information for endpoints that are controlled using MGCP


Pod1HQ#

show mgcp connection

Displays information for active connections that are controlled using MGCP
UC Troubleshooting 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential

14

Cisco IOS MGCP Gateway Commands (Cont.)


Pod1HQ#

show mgcp statistics

Displays MGCP statistics regarding received and transmitted network messages


Pod1HQ#

show mgcp srtp {detail |summary}

Displays information for active SRTP connections controlled using MGCP


Pod1HQ#

debug mgcp [all | errors | endpoint endpoint-name | media | nas | packets | parser | src | voipcac]

Enables debugging messages for MGCP errors, endpoints, events, media, packets, parser, and CAC
UC Troubleshooting 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential

15

MGCP IOS Gateway Troubleshooting


Problem report:
When deploying a new Cisco IOS MGCP gateway, it shows up as not registered in Cisco Unified CallManager.

Consider the possibilities:


Network connectivity problems. IP addressing problems with DHCP or manual settings. Cisco Unified CallManager service is not running. Cisco IOS Software version is not compatible with Cisco Unified CallManager version.

Missing or incorrect configuration in Cisco Unified CallManager configuration.


Missing or incorrect configuration on the MGCP gateway.
UC Troubleshooting 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential

16

Cisco IOS MGCP Gateway Troubleshooting Topology


Cisco Unified CallManager Nodes First Node/TFTP = 10.1.1.1 Second Node = 10.1.1.2

MGCP Gateway Ethernet IP = 10.1.1.101 Loopback = 10.1.250.101 Hostname = HQ-1

PRI

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

17

Cisco IOS MGCP Gateway Troubleshooting


Pod1HQ#ping 10.1.1.1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.1.1.1, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/3/8 ms

Pod1HQ#show version
Cisco IOS Software, 3700 Software (C3725-IPVOICEK9-M), Version 12.4(6)T5, RELEASE SOFTWARE (fc1) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2006 by Cisco Systems, Inc. Compiled Fri 06-Oct-06 20:16 by kellythw

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

18

Cisco IOS MGCP Gateway Troubleshooting (Cont.)


Host Name of Router Slot 0/Subunit0/Port 0

Verify settings in Cisco Unified CallManager under the gateway.


UC Troubleshooting 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential

19

Cisco IOS MGCP Gateway Troubleshooting (Cont.)


Hostname of gateway First choice Cisco Unified CM
HQ-1#show run | begin mgcp mgcp mgcp call-agent 10.1.1.2 mgcp dtmf-relay voip codec all mode out-of-band mgcp rtp unreachable timeout 1000 action notify mgcp modem passthrough voip mode cisco mgcp package-capability rtp-package mgcp package-capability sst-package no mgcp timer receive-rtcp ! ccm-manager redundant-host 10.1.1.1 ccm-manager mgcp ccm-manager config server 10.1.1.1 ccm-manager config ! controller T1 0/1/0 framing esf linecode b8zs pri-group timeslots 1-24 service mgcp ! interface Serial0/1/0:23 no ip address isdn switch-type primary-ni isdn incoming-voice voice isdn T306 30000 isdn bind-l3 ccm-manager no cdp enable ! dial-peer voice 999200 pots service mgcpapp port 2/0/0
2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Second choice Cisco Unified CM TFTP server to get configuration from

Enables MGCP for PRI

Enables PRI backhauling of the Q931 signaling Enabled MGCP for the analog port
UC Troubleshooting

20

Cisco IOS MGCP Gateway Troubleshooting (Cont.)


HQ-1# Dec 10 01:45:49.343: MGCP Packet sent to 10.1.1.1:2427---> RSIP 604725784 *@HQ-1 MGCP 0.1 RM: graceful <--HQ-1# Dec 10 01:47:02.287: MGCP Packet sent to 10.1.1.1:2427---> RSIP 604725786 *@HQ-1 MGCP 0.1 RM: restart <--Dec 10 01:47:02.487: MGCP Packet received from 10.1.1.1:2427---> 500 604725786 <---

debug mgcp packets

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

21

Cisco IOS MGCP Gateway Troubleshooting (Cont.)


Detailed Trace Output for MGCP
MGCPRestartInProgress | (1,100,120,1).31-(*:10.1.1.101) | *@HQ-1 portNum= 0 RSIP 604725826 - RM: restart MGCPRestartInProgress | (1,100,120,1).32-(*:10.1.1.101) | [R:NP - HP: 0, NP: 0, LP: 0, VLP: 0, LZP: 0 DBP: 0]*@HQ-1 portNum= 0 RSIP 604725826 - RM: restart MGCPRestartInProgress | (1,100,120,1).32-(*:10.1.1.101) | [R:NP - HP: 0, NP: 0, LP: 0, VLP: 0, LZP: 0 DBP: 0]*@HQ-1 portNum= 0 RSIP 604725826 - RM: restart MGCPRestartInProgress | (1,100,120,1).32-(*:10.1.1.101) | *@HQ-1 portNum= 0 RSIP 604725826 - RM: restart | Destination process is not Active:[1,100,3,1]->[1,100,117,11] - [MGCPKeepAliveTimer] | Destination process is not Active:[1,100,3,1]->[1,100,117,11] - [MGCPocTimer] | Destination process is not Active:[1,100,117,11]->[1,100,3,1] - [SdlProcessNE] | Destination process is not Active:[1,100,117,11]->[1,100,3,1] - [SdlProcessNE] * Output has been edited to fit on the screen

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

22

Point to Note H.323/SIP Gateway


Understanding the following H.323/SIP gateway behaviors is critical to effective troubleshooting:
H.323/SIP gateways will never register to Cisco Unified CallManager.

In Cisco Unified CallManager, their status will always show as unknown.


Cisco Unified CallManager requires destinations reached through these gateways to be included in the dial plan.

The H.323/SIP gateway requires that dial peers be configured to create the dial plan on the gateway.

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

23

Gather Facts: Network Connectivity


Gatekeeper

Endpoint

Endpoint

Verify connectivity between the endpoints and the gatekeeper.

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

24

Gather Facts: Verify Gatekeeper Configuration That Can Affect Registration


Verify the zone name and IP address to use on the local gatekeeper. If the endpoint settings do not match, registration can fail. Verify that the endpoint is allowed to register for the zone. gatekeeper zone local HQ-1 cisco.com 10.1.250.101 zone remote DGK cisco.com 10.0.0.99 1719 no zone subnet HQ-1 default enable zone subnet HQ-1 10.1.250.0/24 enable zone prefix HQ-1 1... zone prefix HQ-1 501* zone prefix DGK * gw-type-prefix 1#* default-technology bandwidth interzone default 256 bandwidth session default 256 bandwidth session zone HQ-1 256 no shutdown

Verify the technology prefix. Registration issues can result if this does not match on the endpoint.

Verify that the gatekeeper is enabled with a no shutdown command.

Verify the gatekeeper configuration.


UC Troubleshooting 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential

25

Gather Facts: Verify Gateway Configuration That Can Affect Registration


Device > Gatekeeper
IP address of the gatekeeper Will send an RRQ every 60 seconds as a form of keepalive Wait 5 minutes on a failed registration attempt to try to register again Terminal type should be Gateway

The configured gatekeeper to use when sending call across the trunk Technology prefix matches technology prefix on gatekeeper Zone name must match configured zone on gatekeeper; case matters

Device > Trunk

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

26

Gather Facts: Verify Gateway Configuration That Can Affect Registration (Cont.)
interface Loopback 0
Verify that the IP address of the gatekeeper is correct.

ip address 10.1.250.101 255.255.255.0 h323-gateway voip interface h323-gateway voip id HQ-1 ipaddr 10.1.250.101 1718 h323-gateway voip h323-id HQGateway-1 h323-gateway voip tech-prefix 1# h323-gateway voip bind srcaddr 10.1.250.101

Verify that the technology prefix matches the gatekeeper.

Verify IP address that H.323 messages will be sourced from is allowed to register with the gatekeeper.

dial-peer voice 1 voip ! dial-peer voice 2 pots destination-pattern 1611 port 1/1/1 no register e164 destination-pattern .... session target ras

Verify that the gateway function is enabled.


UC Troubleshooting

gateway
2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential

27

Gather Facts: Debugging Gatekeeper Discovery


Gatekeeper# or Gateway#

debug h225 asn1

Displays RAS messages with ASN1 content

value RasMessage ::= registrationReject : { requestSeqNum 5 protocolIdentifier { 0 0 8 2250 0 4 } rejectReason discoveryRequired : NULL

}.

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

28

Gather Facts: Debugging Gatekeeper Registration


Reject reasons:
DuplicateAlias: duplicate E.164 or H323 ID TerminalExcluded: not authorized to register SecurityDenial: security parameters do not match InvalidAlias: zone name does not match or is not configured
value RasMessage ::= registrationReject : {

requestSeqNum 2829
protocolIdentifier { 0 0 8 2250 0 3 } rejectReason REJECT REASON:Duplicate Alias

gatekeeperIdentifier {HQ-1"}
}

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

29

Gather Facts: Cisco Unified CallManager Trace

ARQ Digits dialed

IP address in Hex = 10.1.1.1

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

30

Problem Report: Endpoint Calls Fail


Gatekeeper Endpoint Endpoint

X
gatekeeper#show gatekeeper endpoint GATEKEEPER ENDPOINT REGISTRATION ================================ CallSignalAddr 10.1.250.101 10.1.1.1 10.1.1.2 Port 1720 1720 1720 RASSignalAddr 10.1.250.101 10.1.1.1 10.1.1.2 Port Zone Name Type Flags VOIP-GW VOIP-GW VOIP-GW --------------- ----- --------------- ----- --------- ---- ----50890 HQ-1 50890 HQ-1 50890 HQ-1 H323-ID: HQGateway-1

Problem report: Endpoints are registered but not able to make calls.

H323-ID: CCM1_1
H323-ID: CCM1_2

Total number of active registrations = 3

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

31

Possible Causes: Endpoint Calls Fail


Possible causes of failed calls when the endpoints are registered:
Connectivity between endpoints Insufficient bandwidth due to CAC Dial plan misconfiguration Incorrect IP address returned Called party is not registered

Gather facts:
Verify connectivity between endpoints show gatekeeper endpoint show gatekeeper zone status debug h225 asn1 debug ras
UC Troubleshooting 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential

32

Gather Facts: Debugging Gatekeeper Admission


Reject reasons:
calledPartyNotRegistered: The called party either has never registered or has timed out requestDenied: Bandwidth is not sufficient for the call

value RasMessage ::= admissionReject : { requestSeqNum 34

rejectReason REJECT REASON : NULL


}

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

33

Where to start?
1.
2.

Obtain Detailed Problem Description: Time of events, calling/called #, IP phones Device name(SEPMacAddress) In the CCM trace, use the MAC or IP address to find the IP phones keepalive message, this will yield the TCP handle for this device.
12/29/2003 14:49:01.602 CCM|StationInit - InboundStim - KeepAliveMessage Send KeepAlive to Device Controller. DeviceName=SEP003094C37F47, TCPHandle=000000046, IPAddr=172.16.240.11, Port=49888, Device Controller=[1,89,40]|<CLID::EDPUB-Cluster><NID::172.16.240.119> <CT::1,100,90,1.85832><IP::172.16.240.11><DEV::SEP003094C37F47>

3.

FYI: StationInit is an inbound skinny msg to CM, while StationD is an outbound msg from CM. Follow the TCP handle throughout the CCM trace for all events on this phone.

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

34

Common Skinny Call States


OffHook = 1 OnHook = 2 RingOut = 3 RingIn = 4 Connected = 5 Busy = 6 Congestion = 7 Hold = 8 CallWaiting = 9 CallTransfer = 10 CallPark = 11 Proceed = 12 CallRemoteMultiline = 13 InvalidNumber = 14

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

35

Common Skinny Tones


InsideDialTone=0x21, Silence = 0, Dtmf1 = 1, Dtmf2 = 2, Dtmf3 = 3, Dtmf4 = 4, Dtmf5 = 5, Dtmf6 = 6, Dtmf7 = 7, Dtmf8 = 8, Dtmf9 = 9, Dtmf0 = 0xa, DtmfStar = 0xe, DtmfPound = 0xf, OutsideDialTone=0x22, LineBusyTone=0x23, AlertingTone=0x24, ReorderTone=0x25, RecorderWarningTone=0x26, RecorderDetectedTone=0x27, RevertingTone=0x28, ReceiverOffHookTone=0x29, PartialDialTone=0x2A, NoSuchNumberTone=0x2B, BusyVerificationTone=0x2C, CallWaitingTone=0x2D, ConfirmationTone=0x2E, CampOnIndicationTone=0x2F, RecallDialTone=0x30,

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

36

Skinny Softkey Events

Redial = 1 NewCall = 2 Hold = 3 Trnsfer = 4 CFwdAll = 5 CFwdBusy = 6

Resume = 10 Answer = 11 Info = 12 Confrn = 13 Park = 14 Join = 15 MeetMeConfrn = 16 CallPickUp = 17

CFwdNoAnswer = 7
BackSpace = 8 EndCall = 9

GrpCallPickUp = 18

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

37

Softkey Indexes (Part I)


0. Call State = Onhook, Key Set =0
0. Redial 1. NewCall

1. Call State = Connected, Key Set =1


0. Hold 1. EndCall 2. Trnsfer 3. Park 4. Confrn

2. Call State = OnHold, Key Set =2


0. Resume 1. NewCall

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

38

Softkey Indexes (Part II)


3. Call State = Ring In, Key Set =3
0. Answer

4. Call State = Offhook, Key Set = 4


0. Redial 1. EndCall 2. CFwdAll 3. MeetMeConfrn 4. CallPickUp 5. GrpCallPickUp

5. Call State = Connected w/ Transfering, Key Set = 5


0. 1. EndCall 2. Transfer

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

39

Softkey Indexes (Part III)

6. Call State = More Digits, Key Set =6


0. BackSpace 1. EndCall

7. Call State = Connected w/ Conferencing, Key Set = 7


0. 1. EndCall 2. Confrn

8. Call State = Ring Out, Key Set = 8


0. 1. EndCall

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

40

Skinny Media Payload Types


Media_Payload_G729AnnexB = 15, Media_Payload_NonStandard = 1, Media_Payload_G711Alaw64k = 2, Media_Payload_G711Alaw56k = 3, Media_Payload_G711Ulaw64k = 4, Media_Payload_G711Ulaw56k = 5, Media_Payload_G722_64k = 6, Media_Payload_G722_56k = 7, Media_Payload_G722_48k = 8, Media_Payload_G7231 = 9, Media_Payload_G728 = 10, Media_Payload_G729 = 11, Media_Payload_G729AnnexA = 12, Media_Payload_Is11172AudioCap = 13, Media_Payload_Is13818AudioCap = 14, Media_Payload_G729AnnexAwAnnexB = 16,

Media_Payload_GSM_Full_Rate = 18,
Media_Payload_GSM_Half_Rate = 19, Media_Payload_GSM_Enhanced_Full_Rate = 20, Media_Payload_Wide_Band_256k = 25, Media_Payload_Data64 = 32, Media_Payload_Data56 = 33, Media_Payload_GSM = 80, Media_Payload_ActiveVoice = 81, Media_Payload_G726_32K = 82, Media_Payload_G726_24K = 83, Media_Payload_G726_16K = 84,

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

41

Call Flow #1

2000 calls 2001 and the call established. 2001 puts 2000 on hold. 2000 hears MOH. 2001 takes 2000 off hold. 2001 blind transfers the call to 2002. 2002 answers and the call is established.

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

42

Call Flow #1 (Contd)


CCM

46 42 Call

44

MOH x2000

Hold x2001 Resume Transfer

x2002

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

43

Basic Skinny Call Flows


Cisco IP Phone 1
Station OffHook Station Display Text Station Set Lamp (Steady) Station Start Tone (Dial Tone) Station Keypad Button Station Stop Tone Station Keypad Button Station Keypad Button Station Keypad Button Station Call Info Station Start Tone (Ringback) Station Open Receive Channel Station Stop Tone Station Call Info Station Open Receive Channel Ack Station Start Media Xmission Station Call Info Station Set Lamp (Blink) Station Set Ringer Station Off Hook Station Set Ringer (Off) Station Set Lamp (Steady) Station Open Receive Channel Station Start Media Xmission Station Open Receive Channel Ack

Cisco CM

Cisco IP Phone 2

User Information Exchange Station On Hook Station Set Lamp (Off) Station Close Receive Channel Station Stop Media Xmission

IPT Bootcamp Day 5 e 96

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

44

CCM Trace for Call Flow #1


Cisco IP Phone Cisco CallManager Cisco IP Phone

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

45

CCM Trace for Call Flow #1


- IP phone device SEP003094C37F47 and DN 2000 went offhook, CM instructs the phone to display appropriate info, and play a dialtone. The TCP Handle for this phone is 000000046 StationInit: 000000046 OffHook.|<CLID::EDPUBCluster><NID::172.16.240.119><CT::1,100,90,1.85834><IP::172.16.240.11><DEV::SEP003094C37F47> StationD: 000000046 SetRinger ringMode=1(RingOff) StationD: 000000046 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=2(LampOn) StationD: 000000046 CallState callState=1 lineInstance=1 callReference=16777264 StationD: 000000046 DisplayPromptStatus timeOutValue=0 promptStatus=' ' content='Enter Number' lineInstance=1 callReference=16777264 ver=0x80000004StationD: 6991ad8 ActivateCallPlane lineInstance=1. StationD: 000000046 SelectSoftKeys instance=1 reference=16777264 softKeySetIndex=4 validKeyMask=-1 StationD: 000000046 ActivateCallPlane lineInstance=1 StationD: 000000046 StartTone tone=33(InsideDialTone) User dials and CM informs the ip phone to stop the dialtone and change the softkeys. StationInit: 000000046 KeypadButton kpButton=2 StationD: 000000046 StopTone StationD: 000000046 SelectSoftKeys instance=1 reference=16777264 softKeySetIndex=6 validKeyMask=-1

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

46

CCM Trace for Call Flow #1


Cisco CallManager Cisco IP Phone

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

47

CCM Trace for Call Flow #1, continued


- 2000 continues to dial and CM does digit analysis to rule out potential matches. Note pss= are the partitions this DN has access to:
Digit analysis: match(fqcn="2000", cn="2000", pss="Line 1:Line 2:Local_Dial:Long_Distance_Dial:International_Dial:Internal", dd="2") Digit analysis: potentialMatches=PotentialMatchesExist StationInit: 000000046 KeypadButton kpButton=0 Digit analysis: match(fqcn="2000", cn="2000", pss="Line 1:Line 2:Local_Dial:Long_Distance_Dial:International_Dial:Internal", dd="20") Digit analysis: potentialMatches=PotentialMatchesExist StationInit: 000000046 KeypadButton kpButton=0.

Digit analysis: match(fqcn="2000", cn="2000", pss="Line 1:Line 2:Local_Dial:Long_Distance_Dial:International_Dial:Internal", dd="200") Digit analysis: potentialMatches=PotentialMatchesExist
StationInit: 000000046 KeypadButton kpButton=1 Digit analysis: match(fqcn="2000", cn="2000", pss="Line 1:Line 2:Local_Dial:Long_Distance_Dial:International_Dial:Internal", dd="2001")

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

48

CCM Traces for Call Flow #1, continued


- Last digit came in the CM, Digit analysis found a match. NoPotentialMatchesExist means an exact match was found.
|CallingPartyNumber=2000 |DialingPartition=Line 1 |DialingPattern=2001 |DialingRoutePatternRegularExpression=(2001) |DialingWhere= |PatternType=Enterprise |PotentialMatches=NoPotentialMatchesExist |DialingSdlProcessId=(1,34,20) |IndexOfAnalyzedPattern=0 |PretransformDigitString=2001 |PretransformTagsList=SUBSCRIBER |PretransformPositionalMatchList=2001 |CollectedDigits=2001 |UnconsumedDigits= |TagsList=SUBSCRIBER
|PositionalMatchList=2001 |VoiceMailboxMask= |VoiceMailCallingSearchSpace=

|VoiceMailPilotNumber=
|DisplayName= |RouteBlockFlag=RouteThisPattern |InterceptPartition= |InterceptPattern= |InterceptWhere= |InterceptSdlProcessId=(0,0,0) |InterceptSsType=0 |InterceptSsKey=0

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

49

CCM Traces for Call Flow #1, continued


Cisco IP Phone Cisco CallManager Cisco IP Phone

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

50

CCM Trace for Call Flow #1, continued


- CM located the device with extension 2001 with TCP handle 000000044; instructs the phone to ring and more.
StationD: StationD: StationD: 000000044 CallState callState=4 lineInstance=1 callReference=16777265 000000044 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=5(LampBlink) 000000044 SetRinger ringMode=2(InsideRing)

StationD: 000000044 DisplayPromptStatus timeOutValue=0 promptStatus=' 2000' content='From 2000' lineInstance=1 callReference=16777265 ver=0x80000004
StationD: 000000044 SelectSoftKeys instance=1 reference=16777265 softKeySetIndex=3 validKeyMask=-1

- In the mean time, CM tells the calling phone (2000) to change softkeys, play ringback tone, and more.
StationD: StationD: StationD: StationD: StationD: 000000046 CallState callState=12 lineInstance=1 callReference=16777264 000000046 CallInfo callingPartyName='' callingParty=2000 cgpnVoiceMailbox= 000000046 DialedNumber dialedNumber=2001 lineInstance=1 callReference=16777264 000000046 StartTone tone=36(AlertingTone), direction=0 000000046 CallState callState=3 lineInstance=1 callReference=16777264

StationD: 000000046 SelectSoftKeys instance=1 reference=16777264 softKeySetIndex=8 validKeyMask=-1

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

51

CCM Trace for Call Flow #1, continued


Cisco IP Phone Cisco CallManager Cisco IP Phone

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

52

CCM Trace for Call Flow #1, continued


- 2001 went offhook.
StationInit: 000000044 OffHook StationD: 000000044 ClearNotify StationD: 000000044 SetRinger ringMode=1(RingOff) StationD: 000000044 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=2(LampOn) StationD: 000000044 CallState callState=1 lineInstance=1 callReference=16777265 StationD: 000000044 ActivateCallPlane lineInstance=1

- CM tells 2000 to stop ringback and 2001 to stop ringing. Then sends OpenReceiveChannel to both phones.
StationD: 000000046 StopTone StationD: 000000046 OpenReceiveChannel conferenceID=0 passThruPartyID=1000201 millisecondPacketSize=20 compressionType=4(Media_Payload_G711Ulaw64k) qualifierIn=?. myIP: bf010ac (172.16.240.11) StationD: 000000044 StopTone StationD: 000000044 OpenReceiveChannel conferenceID=0 passThruPartyID=1000211 millisecondPacketSize=20 compressionType=4(Media_Payload_G711Ulaw64k) qualifierIn=?. myIP: 57f010ac (172.16.240.87)

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

53

CCM Trace for Call Flow #1, continued


- CM changes the call state phones to Connected and changes softkeys and diaplays.
StationD: 000000044 CallState callState=5 lineInstance=1 callReference=16777265 StationD: 000000044 SelectSoftKeys instance=1 reference=16777265 softKeySetIndex=1 validKeyMask=-1 StationD: 000000044 DisplayPromptStatus timeOutValue=0 promptStatus=' ' content='Connected' lineInstance=1 callReference=16777265 ver=0x80000004 StationD: 000000046 CallState callState=5 lineInstance=1 callReference=16777264 StationD: 000000046 SelectSoftKeys instance=1 reference=16777264 softKeySetIndex=1 validKeyMask=-1 StationD: 000000046 DisplayPromptStatus timeOutValue=0 promptStatus=' ' content='Connected' lineInstance=1 callReference=16777264 ver=0x80000004

- CM receives OpenReceiveChannelAck from both phones, cuts through audio rtp streams between the phones.
StationInit: 000000046 OpenReceiveChannelAck Status=0, IpAddr=0xbf010ac, Port=27686, PartyID=16777729 StationD: 000000044 StartMediaTransmission conferenceID=0 passThruPartyID=1000211 remoteIpAddress=bf010ac(172.16.240.11) remotePortNumber=27686 milliSecondPacketSize=20 compressType=4(Media_Payload_G711Ulaw64k) qualifierOut=?. myIP: 57f010ac (172.16.240.87) StationInit: 000000044 OpenReceiveChannelAck Status=0, IpAddr=0x57f010ac, Port=30432, PartyID=16777745 StationD: 000000046 StartMediaTransmission conferenceID=0 passThruPartyID=1000201 remoteIpAddress=57f010ac(172.16.240.87) remotePortNumber=30432 milliSecondPacketSize=20 compressType=4(Media_Payload_G711Ulaw64k) qualifierOut=?. myIP: bf010ac (172.16.240.11)

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

54

CCM Trace for Call Flow #1, continued


- 2001 puts the call on hold, CM tears down the rtp media stream on both phones. StationInit: 000000044 SoftKeyEvent softKeyEvent=3(Hold) lineInstance=1 callReference=16777265 StationD: 000000046 CloseReceiveChannel conferenceID=0 passThruPartyID=1000201. myIP: bf010ac (172.16.240.11) StationD: 000000046 StopMediaTransmission conferenceID=0 passThruPartyID=1000201. myIP: bf010ac (172.16.240.11) StationD: 000000044 CloseReceiveChannel conferenceID=0 passThruPartyID=1000211. myIP: 57f010ac (172.16.240.87) StationD: 000000044 StopMediaTransmission conferenceID=0 passThruPartyID=1000211. myIP: 57f010ac (172.16.240.87) - CM instructs 2001 to change call state, softkeys, displays, etc. StationD: 000000044 CallState callState=8 lineInstance=1 callReference=16777265 StationD: 000000044 SelectSoftKeys instance=1 reference=16777265 softKeySetIndex=2 validKeyMask=-1 StationD: 000000044 DisplayPromptStatus timeOutValue=0 promptStatus=' ' content='Hold' lineInstance=1 callReference=16777265 ver=0x80000004 - CM tries to allocated MOH to stream music to held phone with DN 2000. MediaResourceManager::waiting_MrmAllocateMohResourceReq MediaResourceCdpc::sortDeviceGivenList Name=MOH_EDPUB DeviceManager::findDpidGivenKey - Name=MOH_EDPUB Pid=(1,66,2) MohDControl - AllocateMohResourceReq successfully - AudioSourceID and MuticastFlag = 1 MRM::updateMohCounter MRL allocateCounter=1

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

55

CCM Trace for Call Flow #1, continued


- CM sets up the music rtp stream between 2000 and MOH server.
StationD: 000000046 OpenReceiveChannel conferenceID=0 passThruPartyID=1000202 millisecondPacketSize=20 compressionType=4(Media_Payload_G711Ulaw64k) qualifierIn=?. myIP: bf010ac (172.16.240.11) StationInit: 000000046 OpenReceiveChannelAck Status=0, IpAddr=0xbf010ac, Port=28264, PartyID=16777730 MohDControl - stationOutputStartMediaTransmission tcpHandle=0x8 myIP: 77f010ac (172.16.240.119) MohDControl - RemoteIpAddr: bf010ac (172.16.240.11) RemoteRtpPortNumber: 28264 msecPacketSize: 20 compressionType: 4

- 2001 takes the call off hold by pressing the Resume softkey.
StationInit: 000000044 SoftKeyEvent softKeyEvent=10(Resume) lineInstance=1 callReference=16777265

- CM tears down the music stream from MOH server to 2000


StationD: 000000046 CloseReceiveChannel conferenceID=0 passThruPartyID=1000202. myIP: bf010ac (172.16.240.11) MohDControl - stationOutputStopMediaTransmission tcpHandle=0x8 myIP: 77f010ac (172.16.240.119) MediaResourceManager::waiting_MrmDeallocateMohResourceReq MRM::updateMohCounter MRL allocateCounter=0

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

56

CCM Trace for Call Flow #1, continued

- CM reconnects the IP phones


StationD: 000000046 OpenReceiveChannel conferenceID=0 passThruPartyID=1000203 millisecondPacketSize=20 compressionType=4(Media_Payload_G711Ulaw64k) qualifierIn=?. myIP: bf010ac (172.16.240.11) StationD: 000000044 OpenReceiveChannel conferenceID=0 passThruPartyID=1000212 millisecondPacketSize=20 compressionType=4(Media_Payload_G711Ulaw64k) qualifierIn=?. myIP: 57f010ac (172.16.240.87) StationInit: 000000046 OpenReceiveChannelAck Status=0, IpAddr=0xbf010ac, Port=29378, PartyID=16777731 StationD: 000000044 StartMediaTransmission conferenceID=0 passThruPartyID=1000212 remoteIpAddress=bf010ac(172.16.240.11) remotePortNumber=29378 milliSecondPacketSize=20 compressType=4(Media_Payload_G711Ulaw64k) qualifierOut=?. myIP: 57f010ac (172.16.240.87) StationInit: 000000044 OpenReceiveChannelAck Status=0, IpAddr=0x57f010ac, Port=32122, PartyID=16777746 StationD: 000000046 StartMediaTransmission conferenceID=0 passThruPartyID=1000203 remoteIpAddress=57f010ac(172.16.240.87) remotePortNumber=32122 milliSecondPacketSize=20 compressType=4(Media_Payload_G711Ulaw64k) qualifierOut=?. myIP: bf010ac (172.16.240.11)

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

57

CCM Trace for Call Flow #1, continued


- 2001 initiated a Transfer by pressing the Transfer softkey
StationInit: 000000044 SoftKeyEvent softKeyEvent=4(Trnsfer) lineInstance=1 callReference=16777265.

- CM puts the call on 2001s line 1 on hold first.


StationD: StationD: StationD: 000000044 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=3(LampWink) 000000044 CallState callState=8 lineInstance=1 callReference=16777265 000000044 SelectSoftKeys instance=1 reference=16777265 softKeySetIndex=2 validKeyMask=-1

StationD: 000000044 DisplayPromptStatus timeOutValue=0 promptStatus=' ' content='Hold' lineInstance=1 callReference=16777265 ver=0x80000004

- CM tears down the media stream between the phones again.


StationD: 000000046 CloseReceiveChannel conferenceID=0 passThruPartyID=1000203. myIP: bf010ac (172.16.240.11) StationD: 000000046 StopMediaTransmission conferenceID=0 passThruPartyID=1000203. myIP: bf010ac (172.16.240.11) StationD: 000000044 CloseReceiveChannel conferenceID=0 passThruPartyID=1000212. myIP: 57f010ac (172.16.240.87) StationD: 000000044 StopMediaTransmission conferenceID=0 passThruPartyID=1000212. myIP: 57f010ac (172.16.240.87)

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

58

CCM Trace for Call Flow #1, continued


- CM sets up MOH to 2000 again.
StationD: 000000046 OpenReceiveChannel conferenceID=0 passThruPartyID=1000204 millisecondPacketSize=20 compressionType=4(Media_Payload_G711Ulaw64k) qualifierIn=?. myIP: bf010ac (172.16.240.11) StationInit: 000000046 OpenReceiveChannelAck Status=0, IpAddr=0xbf010ac, Port=29672, PartyID=16777732 MohDControl - stationOutputStartMediaTransmission tcpHandle=0x8 myIP: 77f010ac (172.16.240.119) MohDControl - RemoteIpAddr: bf010ac (172.16.240.11) RemoteRtpPortNumber: 29672 msecPacketSize: 20 compressionType: 4

- CM opens the 2nd call on 2001s line 1 to transfer the call, provides dial -tone.
StationD: 000000044 CallState callState=1 lineInstance=1 callReference=16777268

StationD: 000000044 DisplayPromptStatus timeOutValue=0 promptStatus=' ' content='Enter Number' lineInstance=1 callReference=16777268 ver=0x80000004 StationD: StationD: StationD: 000000044 SelectSoftKeys instance=1 reference=16777268 softKeySetIndex=9 validKeyMask=-1 000000044 ActivateCallPlane lineInstance=1 000000044 StartTone tone=33(InsideDialTone), direction=0

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

59

CCM Trace for Call Flow #1, continued


- User started to input number (2002) to transfer the call followed by Digit Analysis Results.
StationInit: 000000044 KeypadButton kpButton=2 StationD: 000000044 StopTone StationD: 000000044 SelectSoftKeys instance=1 reference=16777268 softKeySetIndex=6 validKeyMask=-1 Digit analysis: match(fqcn="2001", cn="2001", pss="Internal:Line 2:Line 1:Local_Dial:Long_Distance_Dial", dd="2") Digit analysis: potentialMatches=PotentialMatchesExist StationInit: 000000044 KeypadButton kpButton=0 Digit analysis: match(fqcn="2001", cn="2001", pss="Internal:Line 2:Line 1:Local_Dial:Long_Distance_Dial", dd="20") Digit analysis: potentialMatches=PotentialMatchesExist StationInit: 000000044 KeypadButton kpButton=0 Digit analysis: match(fqcn="2001", cn="2001", pss="Internal:Line 2:Line 1:Local_Dial:Long_Distance_Dial", dd="200") Digit analysis: potentialMatches=PotentialMatchesExist StationInit: 000000044 KeypadButton kpButton=2 Digit analysis: match(fqcn="2001", cn="2001", pss="Internal:Line 2:Line 1:Local_Dial:Long_Distance_Dial", dd="2002")

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

60

CCM Trace for Call Flow #1, continued

Digit analysis: analysis results


||CallingPartyNumber=2001 |DialingPartition=Line 1 |DialingPattern=2002 |DialingRoutePatternRegularExpression=(2002) |DialingWhere= |PatternType=Enterprise |PotentialMatches=NoPotentialMatchesExist |DialingSdlProcessId=(1,34,17) |IndexOfAnalyzedPattern=0 |PretransformDigitString=2002 |PretransformTagsList=SUBSCRIBER |PretransformPositionalMatchList=2002 |CollectedDigits=2002 |UnconsumedDigits= |TagsList=SUBSCRIBER |PositionalMatchList=2002 |VoiceMailboxMask= |VoiceMailCallingSearchSpace=

|VoiceMailPilotNumber= |DisplayName= |RouteBlockFlag=RouteThisPattern |InterceptPartition= |InterceptPattern= |InterceptWhere= |InterceptSdlProcessId=(0,0,0) |InterceptSsType=0 |InterceptSsKey=0 |WithTags= |WithValues= |CgpnPresentation=NotSelected |CallManagerDeviceType=UserDevice |AlternateMatches= Information Not Available

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

61

CCM Trace for Call Flow #1, continued


- CM locates the phone with DN of 2002, rings it.
StationD: 000000042 CallState callState=4 lineInstance=1 callReference=16777269 StationD: 000000042 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=5(LampBlink) StationD: 000000042 SetRinger ringMode=2(InsideRing)

StationD: 000000042 DisplayPromptStatus timeOutValue=0 promptStatus=' 2001' content='From 2001' lineInstance=1 callReference=16777269 ver=0x80000004 StationD: 000000042 SelectSoftKeys instance=1 reference=16777269 softKeySetIndex=3 validKeyMask=-1

- CM changes call state, tone, softkeys, on 2001.


StationD: 000000044 SelectSoftKeys instance=1 reference=16777265 softKeySetIndex=2 validKeyMask=-3 StationD: 000000044 CallState callState=12 lineInstance=1 callReference=16777268 StationD: 000000044 DialedNumber dialedNumber=2002 lineInstance=1 callReference=16777268 StationD: 000000044 StartTone tone=36(AlertingTone), direction=0 StationD: 000000044 CallState callState=3 lineInstance=1 callReference=16777268 StationD: 000000044 DisplayPromptStatus timeOutValue=0 promptStatus=' ' content='Ring Out' lineInstance=1 callReference=16777268 ver=0x80000004

- 2001 completes the transfer by pressing the transfer softkey a second time.
StationInit: 000000044 SoftKeyEvent softKeyEvent=4(Trnsfer) lineInstance=1 callReference=16777268

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

62

CCM Trace for Call Flow #1, continued...


- CM tears down MOH to 2000, then tells it to play ringback tone.
StationD: 000000046 CloseReceiveChannel conferenceID=0 passThruPartyID=1000204. myIP: bf010ac (172.16.240.11) MohDControl - stationOutputStopMediaTransmission tcpHandle=0x8 myIP: 77f010ac (172.16.240.119) MediaResourceManager::waiting_MrmDeallocateMohResourceReq StationD: 000000046 StartTone tone=36(AlertingTone), direction=0

- CM clears the 1st and 2nd call on line 1 of 2001.


StationD: StationD: StationD: StationD: StationD: StationD: 000000044 ClearPromptStatus lineInstance=1 callReference=16777265 000000044 CallState callState=2 lineInstance=1 callReference=16777265 000000044 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=2(LampOn) 000000044 ClearPromptStatus lineInstance=1 callReference=16777268 000000044 CallState callState=2 lineInstance=1 callReference=16777268 000000044 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=1(LampOff)

- Finally, 2002 accepts the incoming call by pressing the answer softkey.
StationInit: 000000042 OffHook

- The media setup between 2000 and 2002 is similar to the media setup shown earlier in the initial call between 2000 and 2001.

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

63

CCM Trace for Call Flow #1, continued


- 2000 hangs up to disconnect the call so CM sends a close receive channel and stop media transmission then changes the display on the phone.
StationInit: 000000046 OnHook. StationD: 000000046 CloseReceiveChannel conferenceID=0 passThruPartyID=1000205. myIP: bf010ac (172.16.240.11) StationD: 000000046 StopMediaTransmission conferenceID=0 passThruPartyID=1000205. myIP: bf010ac (172.16.240.11) StationD: 000000046 ClearPromptStatus lineInstance=1 callReference=16777264. StationD: 000000046 SelectSoftKeys instance=0 reference=0 softKeySetIndex=0 validKeyMask=-1 StationD: 000000046 DisplayPromptStatus timeOutValue=0 promptStatus=' ' content='Your current options' lineInstance=0 callReference=0 ver=0x80000004

- CM sends similar messages to 2002.


StationD: 000000042 CloseReceiveChannel conferenceID=0 passThruPartyID=1000241. myIP: 5bf010ac (172.16.240.91) StationD: 000000042 StopMediaTransmission conferenceID=0 passThruPartyID=1000241. myIP: 5bf010ac (172.16.240.91) StationD: 000000042 ClearPromptStatus lineInstance=1 callReference=16777269 StationD: 000000042 SelectSoftKeys instance=0 reference=0 softKeySetIndex=0 validKeyMask=-1 StationD: 000000042 DisplayPromptStatus timeOutValue=0 promptStatus=' ' content='Your current options' lineInstance=0 callReference=0 ver=0x80000004

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

64

QUIZZ TIME!
Question 1.
Which Call Trace provide more detailed information? A) SDI

B) SDL

Question 2.

Gateway send RSIP to CM to .


A) Update the status idle, busy and out of service B) Notify its endpoints are remove or back in service
UC Troubleshooting 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential

65

QUIZZ TIME!
Question 3.
Which is the command to troubleshoot gatekeeper? A) Debug h225 asn1

B) Debug mgcp event

Question 4.

How to identify Skinny Message send to CCM via?


A) StationInit B) StationD
UC Troubleshooting 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential

66

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

67

Appendix: Further Reading Call Flow #2

Inbound call from the PSTN over PRI on an H323 gateway. The calling number is (408) 555-1212 and the called number is 2001 (IP Phone). 2001s TCP handle is 000000007. Make note of the callref so you can follow the H225 messages in the trace. Note: the call reference increases by 8 depending on the direction (ie. inbound is 0x0008 so outbound is 0x8008 or inbound is 00 07 and outbound is 80 07).

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

68

H.323 Client to Skinny Client Call Flow


H.323 Gateway
H.225 Setup H.225 Setup Ack

Cisco CallManager
Station Call Info Station Set Lamp (Blink) Station Set Ringer (On) Station Off Hook Station Set Lamp (Steady) Station Set Ringer (Off)

Cisco IP Phone

H.225 Alerting H.225 Connect H.245 Master/Slave Determination

H.245 Master/Slave Determination Ack


H.245 Terminal Capabilities Set H.245 Terminal Capabilities Set Ack H.245 Open Logical Channel H.245 Open Logical Channel Ack Station Start Media Reception Station Start Media Transmission

Conversation
Station On Hook Station Stop Media Transmission Station Stop Media Reception Station Set Lamp (Off)

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

69

Trace and Debug for Call Flow #2


H.323 Gateway
H.225 Setup

Cisco CallManager

Cisco IP Phone

H.225 Setup Ack

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

70

Trace and Debug for Call Flow #2

- Inbound Q931 setup message on the IOS voice gateway:


00:42:12: ISDN Se0/0:23: RX <- SETUP pd = 8 callref = 0x0008 00:42:12: 00:42:12: 00:42:12: Bearer Capability i = 0x9090A2 Channel ID i = 0xA98397 Progress Ind i = 0x8183 - Origination address is non-ISDN

00:42:12:
00:42:12:

Calling Party Number i = 0x00, 0x80, '4085551212', Plan:Unknown, Type:Unknown


Called Party Number i = 0x81, '2001', Plan:ISDN, Type:Unknown

- In the CCM trace, we see the H225 setup message:


In Message -- H225SetupMsg -- Protocol= H225Protocol Ie - H225BearerCapabilityIe -- IEData= 04 03 90 90 A3 Ie - H225CallingPartyIe -- IEData= 6C 0C 00 80 34 30 38 35 35 35 31 32 31 32 Ie - Q931CalledPartyIe -- IEData= 70 05 81 32 30 30 31

IsdnMsgData1= 08 02 00 07 05 04 03 90

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

71

Trace and Debug for Call Flow #2, cont


- CM then performs digit analysis on the incoming call:
Digit analysis: match(fqcn="", cn="4085551212", pss="", dd="2001") |CallingPartyNumber=4085551212 |DialingPartition= |DialingPattern=2001 |DialingRoutePatternRegularExpression=(2001) |DialingWhere= |PatternType=Enterprise |PotentialMatches=NoPotentialMatchesExist |DialingSdlProcessId=(1,34,3) |PretransformDigitString=2001 |PretransformTagsList=SUBSCRIBER |PretransformPositionalMatchList=2001 |CollectedDigits=2001 |UnconsumedDigits= |TagsList=SUBSCRIBER |PositionalMatchList=2001 |VoiceMailbox= |VoiceMailCallingSearchSpace= |VoiceMailPilotNumber= |DisplayName= |RouteBlockFlag=RouteThisPattern |InterceptPartition= |InterceptPattern= |InterceptWhere= |InterceptSdlProcessId=(0,0,0) |InterceptSsType=0 |InterceptSsKey=0 |WithTags= |WithValues=

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

72

Trace and Debug for Call Flow #2, cont


H.323 Gateway
H.225 Setup H.225 Setup Ack

Cisco CallManager
Station Call Info Station Set Lamp (Blink) Station Set Ringer (On)

Cisco IP Phone

H.225 Alerting

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

73

Trace and Debug for Call Flow #2, cont


- CM rings phone with DN 2001 and changes the display.
StationD: 000000007 CallState callState=4 lineInstance=1 callReference=16777218 StationD: 000000007 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=5(LampBlink) StationD: 000000007 SetRinger ringMode=3(OutsideRing) StationD: 000000007 DisplayPromptStatus timeOutValue=0 promptStatus=' 4085551212' content='From 4085551212' lineInstance=1 callReference=16777218 ver=0x84000005 StationD: 000000007 SelectSoftKeys instance=1 reference=16777218 softKeySetIndex=3 validKeyMask=-1

- CM sends a call proceeding message.


Out Message -- H225CallProceedingMsg -- Protocol= H225Protocol IsdnMsgData2= 08 02 80 07 02 7E

- This is followed by an alerting message from CM.


Out Message -- H225AlertMsg -- Protocol= H225Protocol IsdnMsgData2= 08 02 80 07 01

- On the IOS voice gateway, we see the following Q931 messages.


00:42:12: ISDN Se0/0:23: TX -> CALL_PROC pd = 8 callref = 0x8008 00:42:12: Channel ID i = 0xA98397 00:42:12: ISDN Se0/0:23: TX -> ALERTING pd = 8 callref = 0x8008

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

74

Trace and Debug for Call Flow #2, cont


H.323 Gateway
H.225 Setup H.225 Setup Ack

Cisco CallManager
Station Call Info Station Set Lamp (Blink) Station Set Ringer (On) Station Off Hook Station Set Lamp (Steady) Station Set Ringer (Off)

Cisco IP Phone

H.225 Alerting H.225 Connect H.245 Master/Slave Determination

H.245 Master/Slave Determination Ack


H.245 Terminal Capabilities Set H.245 Terminal Capabilities Set Ack

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

75

Trace and Debug for Call Flow #2, cont


- IP phone with DN 2001 goes offhook to answer the call and CM changes the display. StationInit: 000000007 OffHook StationD: 000000007 SetRinger ringMode=1(RingOff) StationD: 000000007 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=2(LampOn) StationD: 000000007 CallState callState=1 lineInstance=1 callReference=16777218 StationD: 000000007 ActivateCallPlane lineInstance=1 StationD: 000000007 SelectSoftKeys instance=1 reference=16777218 softKeySetIndex=1 validKeyMask=-1 StationD: 000000007 DisplayPromptStatus timeOutValue=0 promptStatus=' ' content='Connected' lineInstance=1 callReference=16777218 ver=0x84000005 - Connect message is send to the PSTN. Out Message -- H225ConnectMsg -- Protocol= H225Protocol IsdnMsgData2= 08 02 80 07 07 7E - The following Q931 messages are seen on the IOS voice gateway. 00:42:19: ISDN Se0/0:23: CALL_PROGRESS: CALL_CONNECTED call id 0x8, bchan 22, dsl 0 00:42:19: ISDN Se0/0:23: TX -> CONNECT pd = 8 callref = 0x8008 00:42:19: ISDN Se0/0:23: RX <- CONNECT_ACK pd = 8 callref = 0x0008 - At this point, the capabilities exchange occurs (codec, silence suppression, etc.). I have not included the full capabilities exchanges. H245ASN - TtPid=(1,100,108,1) -Incoming -value MultimediaSystemControlMessage H245ASN - TtPid=(1,100,108,1) -Outgoing -value MultimediaSystemControlMessage

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

76

Trace and Debug for Call Flow #2, cont


H.323 Gateway
H.225 Setup H.225 Setup Ack

Cisco CallManager
Station Call Info Station Set Lamp (Blink) Station Set Ringer (On) Station Off Hook Station Set Lamp (Steady) Station Set Ringer (Off)

Cisco IP Phone

H.225 Alerting H.225 Connect H.245 Master/Slave Determination

H.245 Master/Slave Determination Ack


H.245 Terminal Capabilities Set H.245 Terminal Capabilities Set Ack H.245 Open Logical Channel H.245 Open Logical Channel Ack Station Start Media Reception Station Start Media Transmission

Conversation

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

77

Trace and Debug for Call Flow #2, cont


- CM and Gateway send Open Logical Channel to the gateway and OpenReceiveChannel to the IP phone to establish RTP between the two devices.
H245ASN - TtPid=(1,100,108,1) -Outgoing -value MultimediaSystemControlMessage ::= request : openLogicalChannel : H245ASN - TtPid=(1,100,108,1) -Incoming -value MultimediaSystemControlMessage ::= request : openLogicalChannel : StationD: 000000007 StopTone StationD: 000000007 OpenReceiveChannel conferenceID=0 passThruPartyID=11 millisecondPacketSize=20 compressionType=11(Media_Payload_G729) qualifierIn=?. myIP: 6c00000a (10.0.0.108) H245ASN - TtPid=(1,100,108,1) -Incoming -value MultimediaSystemControlMessage ::= response : openLogicalChannelAck : H245Interface(1) OLC outgoing confirm ip = 300010a, port = 18590 StationInit: 000000007 OpenReceiveChannelAck Status=0, IpAddr=0x6c00000a, Port=28816, PartyID=17 StationD: 000000007 StartMediaTransmission conferenceID=0 passThruPartyID=11 remoteIpAddress=300010a(10.1.0.3) remotePortNumber=18590 milliSecondPacketSize=20 compressType=11(Media_Payload_G729) qualifierOut=?. myIP: 6c00000a (10.0.0.108) H245ASN - TtPid=(1,100,108,1) -Outgoing -value MultimediaSystemControlMessage ::= response : openLogicalChannelAck : H245Interface(1) paths established ip = 6c00000a, port = 28816

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

78

Trace and Debug for Call Flow #2, cont


H.323 Gateway
H.225 Setup H.225 Setup Ack

Cisco CallManager
Station Call Info Station Set Lamp (Blink) Station Set Ringer (On) Station Off Hook Station Set Lamp (Steady) Station Set Ringer (Off)

Cisco IP Phone

H.225 Alerting H.225 Connect H.245 Master/Slave Determination

H.245 Master/Slave Determination Ack


H.245 Terminal Capabilities Set H.245 Terminal Capabilities Set Ack H.245 Open Logical Channel H.245 Open Logical Channel Ack Station Start Media Reception Station Start Media Transmission

Conversation
Station On Hook Station Stop Media Transmission Station Stop Media Reception Station Set Lamp (Off)

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

79

Trace and Debug for Call Flow #2, cont


- IP phone 2001 hangs up so CM tears down the call and changes the phone display.
StationInit: 000000007 OnHook StationD: 000000007 CallState callState=2 lineInstance=1 callReference=16777218 StationD: 000000007 SelectSoftKeys instance=0 reference=0 softKeySetIndex=0 validKeyMask=-1 StationD: 000000007 DisplayPromptStatus timeOutValue=0 promptStatus=' ' content='Your current options' lineInstance=0 callReference=0 ver=0x84000005 StationD: 000000007 CloseReceiveChannel conferenceID=0 passThruPartyID=11. myIP: 6c00000a (10.0.0.108) StationD: 000000007 StopMediaTransmission conferenceID=0 passThruPartyID=11. myIP: 6c00000a (10.0.0.108)

- CM sends a H245 request to close logical channel to the voice gateway and receives a close logical channel ack.
H245ASN - TtPid=(1,100,108,1) -Outgoing -value MultimediaSystemControlMessage ::= request : closeLogicalChannel : H245ASN - TtPid=(1,100,108,1) -Incoming -value MultimediaSystemControlMessage ::= response : closeLogicalChannelAck :

- CM sends a H245 disconnect.


H245ASN - TtPid=(1,100,108,1) -Outgoing -value MultimediaSystemControlMessage ::= command : endSessionCommand : disconnect : H245ASN - TtPid=(1,100,108,1) -Incoming -value MultimediaSystemControlMessage ::= command : endSessionCommand : disconnect :

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

80

Trace and Debug for Call Flow #2, cont

- CM then sends out a ReleaseComplete message.


Out Message -- H225ReleaseCompleteMsg -- Protocol= H225Protocol IsdnMsgData2= 08 02 80 07 5A

- The following Q931 message is seen on the IOS gateway.


00:42:24: ISDN Se0/0:23: TX -> DISCONNECT pd = 8 callref = 0x8008 00:42:24: Cause i = 0x8290 - Normal call clearing

00:42:25: ISDN Se0/0:23: RX <- RELEASE pd = 8 callref = 0x0008 00:42:25: ISDN Se0/0:23: TX -> RELEASE_COMP pd = 8 callref = 0x8008

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

81

Forwarding Calls
What do I look for when calls are forwarded to another extension?
- Look for ForwardManager activities, right after the initial digit analysis of the original called number. Then you should see what kind of forwarding it is (CFA, CFNA, CFB) with forwarded number. Lastly you should see CM performing digit analysis again for the forwarded number. In this case, 2000 called 2001, 2001 has CFA set to 2002.
Digit analysis: match(fqcn="2000", cn="2000", pss="Line 1:Line 2:Local_Dial:Long_Distance_Dial:International_Dial:Internal", dd="2001") ForwardManager - wait_SsInterceptInd Received - Party= 0x1000038, InterceptKey= 0x2 ForwardManager - findInterceptTableEntry(ssKey) - Found Intercept table entry for dn= 2001:Line 1, InterceptKey= 0x2,0x2 ForwardManager::findActivationEntryBySsParty ForwardManager::getNextForwardActiveCallIndex ForwardManager - wait_SsInterceptInd - New Forward or Call Pickup started for Dn= 2001:Line 1. Party= 0x1000038, InterceptKey= 0x2, CallKey= 0x1 Forwarding::callForwardAllSet Forwarding - awaitForwardInitiation_SsInterceptInd - 1 CFA attempted. CallKey= 0x1 Forwarding::callForwardAllDestinationAvailable Forwarding::redirectCallToForwardAllDestination Forwarding - redirectCallToForwardAllDestination - Last Forward - Dn= 2002, CallKey= 0x1

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

82

Forwarding Calls cont

- Digit analysis is then performed on the forwarded call. The trace is similar to call flow 1 presented earlier. Digit analysis: match(fqcn="2000", cn="2000", pss="Internal:Line 2:Line 1:Local_Dial:Long_Distance_Dial", dd="2002")

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

83

UC Troubleshooting

2010 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

84