3G Performance Engineer Soc Classification level 2 Nokia Siemens Networks DNO Optimization Processes / JC Ejarque / 15.09.09 CSSR VOICE ACCESSIBLITY OPTIMIZATION COUNTERS 3 Nokia Siemens Networks DNO Optimization Processes / JC Ejarque Perform data analysis and assessment 1. Assess weekly average PLMN KPI performance to identify KPIs below targets Start from bigger picture assessment (PLNM -> RNC -> Cluster -> WCEL ) Weekly average will smooth the performance and gives better accuracy of performance assessment as daily performance varies a lot especially in unloaded network
2. Assess network performance at RNC/Area level to check if bad performance happens across network or only particular RNC/Area Compare different RNC/Regions performance
3. Apply drill down approach to assess bad performing KPIs in WCEL level Identify the failure call phases Categorizing failure ratio or distribution of each counters (radio, AC, transmission, BTS, RNC) Identify main cause of underperforming KPIs
4. Prioritize analysis Filter using high number of failures or high number failure ratio (weighting) Identify top 50 worst cells
4 Nokia Siemens Networks DNO Optimization Processes / JC Ejarque Perform data analysis and assessment Using smart 3G RAN Reports, Weekly KPI ( PLMN) < X % No action needed No Yes Yes No System Program (PLMN) Weekly KPI (RNC) < X % ?
System Program (RNC) Daily KPI (WCEL) < X%
Identify Call failure phases of bad performing KPIs, for example CSSR
Identify failures root- causes and failure distribution of bad KPIs Identify Top50 Worst Cells based on highest number of root causes failures System Program (WCEL) Others 3G RAN Reports Others 3G RAN Reports Yes Mataching failure distribution into network topology System Program (WCEL)
Solution Proposal Mapinfo 5 Nokia Siemens Networks DNO Optimization Processes / JC Ejarque CSSR CS/PS Analysis Failure counters 6 Nokia Siemens Networks DNO Optimization Processes / JC Ejarque CSSR definition 7 Nokia Siemens Networks DNO Optimization Processes / JC Ejarque Low in CSSR? Identify call setup failure phases Call Setup Phases CSSR affected if any of the followings take place. 1. RRC Conn. Setup Fail 2. RRC Conn. Access Fail 3. RAB Setup Fail 4. RAB Setup Access Fail Call Setup Phase RRC Conn. Req RRC Conn. Setup RRC Conn. Acc RRC Conn. Act RAB Setup RAB Drop Call Setup Success Call Drop Connected 8 Nokia Siemens Networks DNO Optimization Processes / JC Ejarque Low in CSSR? RRC/RAB Setup & Access Analysis Process Flow Chart
Sites OK ? Cell and Neighbour Cells availability Alarms/ Tickets Setup / Access Yes Setup / Access Setup Failure Cause? Capacity Optimisation BTS/ TRANS/FROZBS UL/ DL Interference (DL codes) AC Troubleshooting RNC RF Optimisation Top (N) RRCSetup and Access failures Top (N) RAB Setup and Access failures Coverage/ I nterf erence setup setup I nterference Coverage 3Gcell at inter-RNC border ? SRNSRelocation troubleshooting Yes NO Access Access Sites OK ? Cell and Neighbour Cells availability Alarms/ Tickets Setup / Access Yes Setup / Access Setup Failure Cause? Capacity Optimisation BTS/ TRANS/FROZBS UL/ DL Interference (DL codes) AC Troubleshooting RNC RF Optimisation Top (N) RRCSetup and Access failures Top (N) RAB Setup and Access failures Coverage/ I nterf erence setup setup I nterference Coverage 3Gcell at inter-RNC border ? SRNSRelocation troubleshooting Yes NO Access Access 9 Nokia Siemens Networks DNO Optimization Processes / JC Ejarque RRC Part Performance Signaling and Trigger points positive [RACH] RRC:RRC Connection Request UE Node B RNC ALCAP:ERQ NBAP: RL Setup Request [DCH] RRC: RRC Connection Setup Complete L1 Synchronisation Start TX/RX Start TX/RX [FACH] RRC: RRC Connection Setup NBAP: RL Setup Response AC to check to accept or reject RRC Connection Request ALCAP:ECF NBAP: Synchronisation Indication RRC Connection Setup phase RRC Connection Access phase RRC Connection Active phase Allocation of UTRAN resources Waiting for UE reply RRC active complete Pre-Emption SRNS Relocation drop M1001C0 M1001C1 M1001C8 10 Nokia Siemens Networks DNO Optimization Processes / JC Ejarque RRC Part Connection Setup & Access and Active negative BTS UE RNC CN RRC: RRC connection Request RRC: RRC connection Setup RRC SETUP phase (Resource Reservation in RNC, BTS, Transport) RRC ACCESS phase (RNC waits for Reply from UE) RRC: RRC connection Setup Complete R R C
S e t u p
t i m e
RRC: Initial Direct Transfer RANAP: Initial UE Message RANAP: Iu Release Command UE-CN Signalling (E.g. RAB Establishment and Release) RRC: RRC connection Release RRC: RRC connection Release Complete Release RRC resources in RNC, BTS, Transport RRC ACTIVE phase RRC ACCESS fails if the UE does not reply to RRC: RRC CONNECTION SETUP message with the RRC: RRC CONNECTION SETUP COMPLETE message in given time, if the BTS reports a radio link synchronisation failure or in an RNC internal failure occurs RRC SETUP fails if some of the needed resources (RNC, BTS, AIR, Transport) are not available. When an RRC setup failure occurs the RNC sends an RRC: RRC CONNECTION REJECT message to UE RRC ACTIVE fails when an interface related (Iu, Iur , Iub, or Radio) or RNC internal failure occurs, and the failure causes the release of the RRC Connection. When an RRC active failure occurs, the RRC send a RANAP: IU RELEASE REQUEST to all involved CNs and waits for RANAP: IU RELEASE COMMAND message (s) RRC ACTIVE release cause can be either ISHO, IFHO, SRSN relocation or pre-emption 11 Nokia Siemens Networks RRC SETUP PHASE -- DNO Optimization Processes / JC Ejarque 12 Nokia Siemens Networks RAB SETUP PHASE DNO Optimization Processes / JC Ejarque 13 Nokia Siemens Networks RANOP2_M1/January 2011 UE RRC: Connection Request RRC: RRC Connection Request Setup RRC: RRC Connection Completed RRC: Initial Direct Transfer cm service request RANAP: Initial UE Message DIRECT TRANSFER (Call Proceeding) RANAP: RAB Assignment Request RRC: Radio Bearer Set-up RRC: Radio Bearer Setup Complete RANAP: RAB Assignment Response DIRECT TRANSFER (Alerting) BTS RNC CN DIRECT TRANSFER (Connect) DIRECT TRANSFER (Connect Acknowledge) RAB Connection Access phase RNC waits reply from UE Call Set- up Success Rate Call Set- up Time Definition of Call Set-Up Success Rate (CSSR) RRC Connection Access phase RNC waits reply from UE RRC Connection Setup phase Resource Reservation in RNC, BTS, Transmission RAB Connection Setup phase Resource Reservation in RNC, BTS, Transmission For Internal Use Only 14 Nokia Siemens Networks Failure analysis for low CSSR/SSSR cells RRC, RAB Setup, PS Setup Failure due AC RRC, RAB, HSPA Setup Performance KPI/counters Follow-Up Yes No Air Interface RRC, RAB Setup, PS Setup Failure due BTS Yes No BTS RRC, RAB Setup, PS Setup Failure due Iub Yes No Iub RRC, RAB Setup, PS Setup Failure due RNC, DMCU Yes RNC RRC, RAB Setup failure due to Frozen BTS No Yes High Traffic Event RANOP2 - M2/January 2011 For Internal Use Only 15 Nokia Siemens Networks RANOP2 - M2/January 2011 Starting criteria for setup failure analysis RRC or RAB setup failures could be due issues related to Coverage or interference Inter RNC problem Capacity Configuration & SW issues
Capacity issues could results to the RRC/RAB setup failures related to BTS for Channel Element (WSP & FSM) capacity issues TRANS for Iub capacity issues AC for radio capacity issues (UL Load, DL Load and DL spreading codes)
Check UL Interference, DL Power & Code occupancy if there is need to upgrade radio capacity Check over shooting, cell availability of own and target site , Special Events High Traffic , Missing Neighbour Check the alarms for the serving Node B and Neighboring sites in order to identify any hardware faults
RRC_CONN_STP_FAIL_BTS
Evaluate NBAP counters (radio link reconf. Failures) and KPIs for troubleshooting BTS resources Check BTS configuration in terms of CE allocation Check the Available CE in comparison with CE utilization( Module might be down ) Expand the Capacity or decrease the traffic offered to the site until the problem is solved In case BTS is not responding delete and re-create COCO Upgrade the Module to FSME :MAX CE element
17 Nokia Siemens Networks Low in CSSR: Failure Counters-II DNO Optimization Processes / JC Ejarque RRC_CONN_STP_FAIL_RNC Typically RNC fault or Incoming SRNC Relocation Failure (inter-RNC border) Required ICSU log tracing if no RNC fault or SRNC relocation problem Communicate the problem to Care department CHECK alarm BFD Alarm if exist DMPG problem in RNC
RRC_CONN_STP_FAIL_TRANS Evaluate Number of reconfiguration failure due the transmission Check COCO Configuration Cross check PCR value & verify AAL2 Connection failure . Expand the capacity or decrease the traffic offered to the site until the problem is solved For IP based Iub Check IP based route parameters Delete and Recreate the COCO : Reset the Site if sleeping cells alarm exist . Temporary decrease the Traffic or shift to GSM if the problem continuous exist with High Failure
RRC_CONN_STP_FAIL_IUB_AAL2_TRANS
Updated when there is shortage or blocking of AAL2 resource Cross check the PCR value & change as per Recommendation Cross check the Transmission Alarms . Reduce the Traffic or shift to GSM if the problem persist continuous with high failures .
If RRC setup/access failure due to Radio/MS, it is also possible to check whether UE s are located at distance area or close to cell edge area
Propagate delay counters from RRC measurement M1006C128-C148 reports call setup distance during RRC connection request or cell update This give hints that either cells has large coverage area (tall sites with over-shooting) or non- optimum cell coverage from neighbouring cells
0 100000 200000 300000 400000 500000 600000 P R A C H _ D E L A Y _ C L A S S _ 0 P R A C H _ D E L A Y _ C L A S S _ 1 P R A C H _ D E L A Y _ C L A S S _ 2 P R A C H _ D E L A Y _ C L A S S _ 3 P R A C H _ D E L A Y _ C L A S S _ 4 P R A C H _ D E L A Y _ C L A S S _ 5 P R A C H _ D E L A Y _ C L A S S _ 6 P R A C H _ D E L A Y _ C L A S S _ 7 P R A C H _ D E L A Y _ C L A S S _ 8 P R A C H _ D E L A Y _ C L A S S _ 9 P R A C H _ D E L A Y _ C L A S S _ 1 0 P R A C H _ D E L A Y _ C L A S S _ 1 1 P R A C H _ D E L A Y _ C L A S S _ 1 2 P R A C H _ D E L A Y _ C L A S S _ 1 3 P R A C H _ D E L A Y _ C L A S S _ 1 4 P R A C H _ D E L A Y _ C L A S S _ 1 5 P R A C H _ D E L A Y _ C L A S S _ 1 6 P R A C H _ D E L A Y _ C L A S S _ 1 7 P R A C H _ D E L A Y _ C L A S S _ 1 8 P R A C H _ D E L A Y _ C L A S S _ 1 9 P R A C H _ D E L A Y _ C L A S S _ 2 0 0.00% 20.00% 40.00% 60.00% 80.00% 100.00% 120.00% average CDF Low in CSSR: Additional counters 19 Nokia Siemens Networks DNO Optimization Processes / JC Ejarque Low in CSSR: Failure Counters RAB setup & Access Fail Root Cause Analysis: RT and NRT (I/B) domains
RAB_STP_FAIL_XXX_AC Check UL Interference, DL Power & Code occupancy if there is need to upgrade radio capacity using the already distributed. Check PrxTarget / PrxOffset / PrxMeasFilterCoeff PtxTarget / PtxOffset / PtxMeasFilterCoeff of the affected cells
RAB_STP_FAIL_XXX_BTS Evaluate NBAP counters (radio link reconf. Add failures) and KPIs for troubleshooting BTS resources Check BTS configuration in terms of CE allocation Use Channel Element (5001) Counters in order to evaluate lack of Channel Elements Expand the Capacity or decrease the traffic offered to the site until the problem is solved. In case BTS is not responding delete and re-create COCO
RAB_STP_FAIL_XXX_TRANS Evaluate Number of reconfiguration failure due the transmission Check M1005C128 CANC_ADD_SRNC_TRAN_STP_FAIL Check RAB_STP_FAIL_XXX_IUB_AAL2, M1001C531-C533 Check COCO Configuration check IUB Capacity
RAB_STP_FAIL_XXX_RNC Typically RNC fault or Incoming SRNC Relocation Failure (inter-RNC border) Required ICSU log tracing if no RNC fault or SRNC relocation problem Communicate the problem to Care department
20 Nokia Siemens Networks DNO Optimization Processes / JC Ejarque Low in CSSR: Failure Counters RAB setup & Access Fail Root Cause Analysis: RT and NRT (I/B) domains
RAB_ACC_FAIL_XXX_UE
Evaluate Cell resource Prx and Ptx parameters (same as RAB_STP_FAIL_XXX_AC Case)
RAB_ACC_FAIL_XXX_RNC
Typically RNC fault or Incoming SRNC Relocation Failure (inter-RNC border) Required ICSU log tracing if no RNC fault or SRNC relocation problem Communicate the problem to Care department.
21 Nokia Siemens Networks RRC CONN STP FAIL AC DNO Optimization Processes / JC Ejarque 22 Nokia Siemens Networks RRC_CONN_STP_FAIL_BTS
DNO Optimization Processes / JC Ejarque 23 Nokia Siemens Networks RRC CONN STP FAIL _ TRANS DNO Optimization Processes / JC Ejarque 24 Nokia Siemens Networks RRC CONN STP FAIL TRANS DNO Optimization Processes / JC Ejarque 25 Nokia Siemens Networks RRC CONN STP FAIL _RNC DNO Optimization Processes / JC Ejarque BFD ALARM 26 Nokia Siemens Networks RRC CONN STP FAIL FROZBS DNO Optimization Processes / JC Ejarque 27 Nokia Siemens Networks FINAL IMP COUNTER DNO Optimization Processes / JC Ejarque 28 Nokia Siemens Networks CRITICAL & MAJ OR WCDMA ALARMS-I
Base Station generated alarms from HW problems: 61022 BFD down, 10.68.18.241 61605 BFD 1 down, ingress, 10.68.18.241 to 10.68.121.9
3117-- FAILURE IN SAAL UNI SIGNALLING LINK ACTIVATION
1.7650 BASE STATION FAULTY A critical fault has occurred in the Base Station. None the cells of this Base Station are capable of transmitting traffic and thus also HSDPA is unavailable in this Base Station. 2.7651 BASE STATION OPERATION DEGRADED A major fault (or faults) has occurred in the Base Station. Depending on the fault details this alarm may indicate problems with HSDPA (see also alarm 7776 below). 3.7653 CELL FAULTY Hardware fault in Base Station has an effect for a Cell. The Cell is unavailable for all traffic. 4.7654 CELL OPERATION DEGRADED Hardware fault in Base Station has an effect for a Cell. Depending on the fault details this alarm may indicate problems with HSDPA (see also alarm 7776 below). 5. 7772 PHYSICAL SHARED CHANNEL CONFIGURATION FAILED HSDPA channel configuration or reconfiguration to Base Station has failed. HSDPA is not available in given Cell.
DNO Optimization Processes / JC Ejarque 29 Nokia Siemens Networks CRITICAL & MAJ OR WCDMA ALARMS-II
6. 7776 HSDPA FAILURE IN WCEL Base Station has informed RNC that it cannot provide HSDPA. The reason for the fault is either a hardware failure or a lack of capacity in Base Station. See Base Station generated alarms for fault details. . 7665 BASE STATION TRANSMISSION ALARM 10. 61058 Synchronization lost 11. 7650 BASE STATION FAULTY 12.7651 BASE STATION OPERATION DEGRADED . There is no limit to how many 7651 alarms can be active at the same time 13. 7652 BASE STATION NOTIFICATION . There is no limit to how many 7652 alarms can be active at the same time 14. 7653 CELL FAULTY . Start, the affected cell state is set disabled . Cancel, the affected cell state is set enabled 15. 7654 CELL OPERATION DEGRADED . There is no limit to how many 7654 alarms can be active at the same tim
19. 7661 BASE STATION LICENCE NOTIFICATION . Licence alarms do not have dependencies to other BTS alarms.
DNO Optimization Processes / JC Ejarque 30 Nokia Siemens Networks CRITICAL & MAJ OR WCDMA ALARMS-III
DNO Optimization Processes / JC Ejarque 16. 7655 CELL NOTIFICATION . There is no limit to how many 7655 alarms can be active at the same time 17. 7665 BASE STATION TRANSMISSION ALARM . In case of faulty transmission equipment or transmission software, this alarm is reported towards the OMS or NetAct. . Transmission alarms do not have dependencies to other BTS alarms.
18. 7660 BASE STATION LICENCE EXPIRED . Licence alarms do not have dependencies to other BTS alarms.
31 Nokia Siemens Networks Thank you DNO Optimization Processes / JC Ejarque