Measurements and QoS Analysis of Live-World Mobile Telecommunication Networks

Charalampos N. Pitas, Eleni D. Avgeri, Marilena P. Kallenou, and Philip Constantinou

Mobile Radiocommunications Laboratory School of Electrical and Computer Engineering, National Technical University of Athens e-mail : {chpitas, lena, marilena, fkonst}@mobile.ntua.gr
Abstract The scope of this paper is to study the performance of contemporary live-world mobile telecommunication networks. The survey is focused on High Speed Packet Access technology which is based on Wideband CDMA. The measurement platform allows QoS assessment, radio access networks audit and benchmarking. Specifically, services are evaluated via extensive mobile and static measurements campaigns in both indoor and various outdoor environments. Emphasis is given to QoS categorization and characterization. Accessibility, retainability and integrity are thoroughly examined by Key Performance Indicators. Enhanced drill down analysis, measurements filtering and reporting is executed on a spatial relational database management system. Index TermsMobile Telecommunication Systems (GSM / GPRS / UMTS / HSPA), Quality of Service (QoS), Key Performance Indicators (KPIs).

presented as well as the tests which are carried out. Analysis and measurements post-processing follow in Section V and finally, SECTION VI, several conclusions are exposed. II. HIGH SPEED PACKET ACCESS NETWORKS. HSPA, which is known as 3.5G, is composed of two technologies, High Speed Downlink Packet Access (HSDPA) and High Speed Uplink Packet Access (HSUPA) [1]. Services of 3G wireless cellular networks, such as video telephony, voice over IP (VoIP), gaming, file downloading and video/picture sharing, require different Quality of Service (QoS) with respect to data rate, delay, error rate, capacity and coverage. HSDPA is targeting increased capacity, reduced round trip delay, higher peak data rates up to 10 Mbps, higher user/cell throughput and lower latency. For this scope, the 3GPP has developed the HSDPA technology, an evolution of UMTS based on Wideband Code Division Multiple Access (WCDMA). It is important to note that HSDPA is a pure access evolution without any core networks impacts, except for minor changes due to the higher bandwidth access. For example, in the 3GPP Rel. 5 the maximum throughput set into the signaling protocol has been increased from 2 Mbps to 16 Mbps in order to support the theoretical maximum limit of HSDPA data rate (14.4 Mbps). It follows that the deployment of HSDPA is very cost effective since the incremental cost is mainly due to Node B and Radio Network Controller (RNC) hardware/software upgrade while the operator cost to provide data services is significantly reduced. This is due to the high improvements in spectral efficiency introduced by HSDPA. In order to achieve these goals, HSDPA technology has introduced new channels and new techniques. HSDPA optimizes downlink performance. The HSDPA functionalities define three new channel types: High-Speed Downlink Shared Channel (HS-DSCH), High-Speed Shared Control Channel (HS-SCCH) and High-Speed Dedicated Physical Control Channel (HS-DPCCH). Three new features of HSDPA are Adaptive Modulation and Coding (AMC), Hybrid Automatic Repeat Request (HARQ) and Fast Scheduling. High Speed Uplink Packet Access (HSUPA) uses the Enhanced Dedicated Channel (E-DCH) and constitutes a set of improvements that optimize uplink performance. Among

I. INTRODUCTION Nowadays 3rd generation wireless cellular networks have already met a great success and have been spread to several new services. Some of these services require higher bit rates, reduced latency, and increased spectral efficiency. High Speed Packet Access represents a new high-speed data transfer feature released by the 3rd Generation Partnership Project (3GPP) with the aim of empower UMTS downlink and uplink data rates and improve support of high dada-rate packet-switched services. Commercial mobile telecommuni-cation networks in Greece have already introduced HSPA technology in coexistence with UMTS R.99 and GSM/GPRS radio access networks.
GSM GPRS WCDMA R.99 HSDPA Release 5 HSUPA Release 6

Fig. 1 Mobile Telecommunication Systems evolution up to 3.5G.

At the beginning of this paper, SECTION II, a short introduction of the HSPA technology is presented with new channel types details. Then, in SECTION III, QoS KPIs (Key Performance Indicators) are defined for network evaluation. Afterwards, in SECTION IV, the measurement equipment is

these improvements are higher throughputs, reduced latency, and increased spectral efficiency. HSUPA is standardized in Release 6. Performance gains of HSUPA are achieved through the following approaches an enhanced dedicated physical channel, a short TTI, as low as 2 ms, which allows faster responses to changing radio conditions and error conditions, fast Node-B-based scheduling, which allows the base station to efficiently allocate radio resources and fast Hybrid ARQ, which improves the efficiency of error processing.
TABLE I 2G TO 3G SUPPORTED BIT RATES EVOLUTION UMTS HSPA GSM GPRS (R.99) 7.2 Mbps Theoritical 14.4 171 2 Mbps Up to peak data Kbps Kbps 14.4Mbps rate Typical 9.6 384 downlink 40 Kbps 5.8 Mbps Kbps Kbps peak data rate Typical 9.6 uplink 20 Kbps 64 Kbps 2Mbps Kbps peak data rate

3.

received. The KPI does not include the duration of PDP Context activation. Setup Failure Rate: Tests percentage that failed at the setup phase.

B. Retainability 1. Download Time: The interval of time between the first and the last byte. 2. RTT (Round Trip Time): The interval of time between a packet is sent and received. 3. Drop Rate: Tests percentage that failed and had already started normally. C. Integrity 1. Throughput: The sum of bytes that had been sent or received to the duration it took. 2. Overall Service Success: Tests percentage that have been successfully completed. It is calculated by excluding from the sum of tests the ones that failed either at the setup phase either they dropped. Furthermore, voice quality is of the first consideration for packet switched mobile telephony. Speech Quality indicator is the major factor determining customer satisfaction, since it represents the quality of listeners reception. It is generally expressed as a Mean Opinion Score (MOS) which is valued between 1 (bad level) and 5 (excellent level). Hereupon speech quality is measured and assessed by relative algorithm as well as statistics are presented. IV. MEASUREMENT SYSTEM A. System Configuration The measurements and analysis have been conducted using SwissQual Diversity equipment.

III. QUALITY OF SERVICE KEY PERFORMANCE INDICATORS According to ETSIs standards [2] QoS KPIs can be categorized in five phases during service use from the customer's point of view: 1. Network Availability: Probability that the mobile services are offered to a user. 2. Network Accessibility: Probability that the user performs a successful registration on the PLMN. 3. Service Accessibility: If the user wants to use a service, the network operator should provide him as fast as possible access to the service. Includes KIPs that describe how successful is the service access (call setup rate) as well as KPIs that describe setup time (call setup time). 4. Service Integrity: This describes the Quality of Service during service use. Includes KPIs that describe the quality of the service (MOS for voice quality). 5. Service Retainability: Service retainability describes the termination of services (in accordance with or against the will of the user).Includes KPIs that count the time that a service can retain (Drop Call Rate or call holding time). Analysis is concentrated on QoS for PS data. Crucial KPIs are comprehensively studied: A. Accessibility 1. PDP Context Activation Duration and Success Rate: indicates the time required for the service access as well as the success rate. 2. Setup Time: The interval of time from the moment the user service requests until the first byte is sent or

Control & Analysis

Test Manager NQDI SQL Server

Backend
CS Network Voice Server Mobile Network IP Network Media Server

Probes
Diversity Benchmarker

Fig. 2. Measurement System Architecture

Diversity is a portable, highly scalable platform for drivetest systems that allows the user to measure more than one cellular network of different technology. The system is mobile

in automotive setting or carried by a pedestrian. The interface to the mobile network is through special models of cellular phones that give access to internal network parameters [4],[5]. The measurement system sets up calls, records cellular network parameters, measures voice quality calls using Voice Server as well as the quality of other payload such as data, email, ping and video using Media Server. Diversity provides QoS evaluation, benchmarking, and RF and network optimization. It is designed for both wireless and fixed networks. The quality of service in voice and data tests is performed through related speed and audio quality algorithms. Tests execution is configured and controlled by Test Manager and also measurements data can be displayed live on NQView. Test Manager and NQView are software applications running on the laptop during the measurements. In this scenario, laptop user in the vehicle monitors the process of measurements. B. Measurement Campaign The measurement system is placed in a car while executing a measurement campaign [6]. In the campaign a session of tests is consecutively repeated. The data session contains the following actions and tests: 1. Activation Packet Data Protocol (PDP Context Activation) 2. Reception of 10 MB file (FTP Download) from a public server broadband connected to the internet. 3. Sent a 3 MB file to the public server (FTP Upload). 4. Web browsing at NTUA (www.ntua.gr) website of size of 1 MB. 5. 10 Ping with 400 Bytes Packet Size. 6. Deactivation Packet Data Protocol (PDP Context Deactivation).

The session of voice tests contains both mobile phone to mobile phone test as well as mobile to PSTN phone test. In both cases A-side and B-side are defined and exchange short voice recordings alternately in order to emulate a call conversation. The conversation schema is presented at Fig.2.

Fig. 4 Quality of voice measurement tasks configuration.

V. ANALYSIS POST PROCESSING RESULTS A. Analysis Tool Measurements are imported in the post processing application based on SwissQuals NQDI (NetQual Database Interface) which is connected with a relational time-spatial database. NQDI offers enhanced drill down analysis, filtering and data mining, KPI statistics, map plotting, database querying/mining and reporting. Using NQDI the measurement data can be filtered by using complicated queries in order to analyze specific part of all the data located in the database. A typical structure of a query based filter applied in the data base aims to show up performance parameter values and compute statistics from a set of well organized measurements or/and network information tables where crucial criteria and combined conditions take place. A general query form is presented: SELECT <List_of_Parameters>, <Statistics_Functions> FROM <List_of_Measurements_Tables> OR/AND <List_of_Radio_Network_Parameters> WHERE <List_of_Combined_Criteria/Conditions> The QoS statistics analysis is focused on significant KPIs, especially, achieved download and upload throughput, overall http success rate and round-trip-time [7].

Fig. 3 Measurements Scenario

In Table II, a statistics analysis of the achieved download bitrates is presented in details. A percentage of 42.59% is remarked between 1.8Mbps and 3.6Mbps when a worthy 28.71% is provided up to 7.2Mbps. It is obvious that 14.4Mbps technology has not introduced yet. Download throughput statistics functions are plotted at the following figure. It is remarked that only 8,33% of download measurements are below 384kbps while 91,67% are fine supported by HSDPA.

Fig. 6. FTP Download Throughput PDF and CDF graph.

Fig. 5. Analysis process diagram.

B. Results 1) Packet Switched Services It is known that the maximum typical throughput for HSDPA 7.2Mbps technology is 5.8Mbps while for HSDPA 3.6Mbps technology is 3Mbps. As it is arisen from the following diagram, HSDPA 7.2Mbps is applied in the network even though 3.6Mbps technology is used in more samples.
TABLE II FTP DOWNLOAD THROUGHPUT STATISTICS

In Table III, a statistics analysis of the achieved upload bitrates is presented in details. A worthy percentage of 77.19% is remarked between 384kbps and 2Mbps. It is clear that HSUPA technology is well introduced and is traced in the majority of the measurement samples. Besides, 21,02% of the relevant samples are detected between 128kbps and 384kbps. So, broadband access is well supported for demanding upload use.
TABLE III FTP UPLOAD THROUGHPUT STATISTICS

FUNCTION

PDF

FUNCTION

PDF

CDF

DL throughput (kbps) >7200 3600<DL<=7200 1800<DL<=3600 384<DL<=1800 DL<=384 >7200 3600<DL<=7200 1800<DL<=3600 384<DL<=1800 DL<=384

PERCENTAGE 0,00% 28,70% 42,59% 20,37% 8,33% 0,00% 28,70% 71,30% 91,67% 100,00%

CDF

UL throughput (kbps) >2000 384<UL<=2000 128<UL<=384 64<UL<=128 UL<=64 >2000 384<UL<=2000 128<UL<=384 64<UL<=128 UL<=64

PERCENTAGE 0,00% 77,19% 21,02% 1,66% 0,13% 0,00% 77,19% 98,21% 99,87% 100,00%

Upload throughput statistics functions are plotted at the following figure. It is mentioned that 98,21% of the upload measurement samples are located above 128kbps and up to 2Mbps owing to HSUPA technology. Less than 2% of the samples are lower than 128kbps.

following diagram (Fig.8) we present a total time distribution of each technology.

Fig. 7. FTP Upload Throughput PDF and CDF graph.

Download time is used to evaluate HTTP performance, which is considered to be more efficient than throughput. In the diagram, the average and minimum download time is presented, as well as the Overall Service Success, tests that have successfully completed without drop or setup failure. The average download time 7,1sec is capable for loading a website of 1 MB.

Fig. 9. Data Serving Technology distribution.

2) Circuit Switched Services Telephony Speech Quality indicator is the major factor determining customer satisfaction, since it represents the quality of listeners reception. It is generally expressed as a Mean Opinion Score (MOS) which is valued between 1, bad level, and 5, excellent level. Most objective algorithms [8][9][10] are based on a comparison between a reference sample and a coded version of the reference. The most widely used models of algorithms are ITU-T P.862 standardized in 2001. In contrast with subjective experiments, this model enables extensive testing to be performed over short periods. The results of ITU-T P.862 algorithm are displayed in the following figures.

Fig. 8. Total HTTP session time and Success Rate. Fig. 10 CDF of Listening Quality (P.862.1)

Based on the results of the Round Trip Time KPI and the following table, the network performance considered to be very good with average RTT 194msec.
TABLE III
PING RTT STATISTICS

Count % Avg [ms] Min [ms] Max [ms] StDev [ms] 95% Cut-Off

528 99,62% 194 101 3454 165 345

Fig. 11 Listening Quality distribution.

For each test, Diversity stored the duration of the provided data service technology. Combining all types of tests, in the

3) KPIs Map Plotting Finally, Fig.12 and Fig.13 display crucial QoS parameters

Channel Quality Indicator and Listening Quality - having applied a spatial filter on a polygon which contains a route of the NTUA Campus. The result of the spatial query has been plotted on a GIS Viewer (Google Earth) giving us an opportunity to locate problematic areas regarding performance or RF coverage.

Consequently, it is necessary and obligatory (by independent national regulatory authority [11]) for mobile operators to perform continuously measurements so as to audit and optimize the efficiency of radio access networks as well as for benchmarking reasons. Open and frequent publication of crucial KPIs will lead mobile operators to robust competition introducing innovative and qualitative services. Contemporary technologies like Long Term Evolution UMTS, also known as HSPA+, will explicitly uplift the provided bitrates. Future work may be performing large scale measurements campaigns in differentiated environments of interest in order to extensively examine QoS parameters. REFERENCES
H.Holma and A.Toskala, HSDPA/HSUPA FOR UMTS, High Speed Radio Access for Mobile Communications, Ed. John Wiley & Sons, LTD, 2006. [2] ETSI TS 102 250-1, Speech Processing, Transmission and Quality Aspects (STQ); QoS aspects for popular services in GSM and 3G networks; Part 1: Identification of Quality of Service aspects, V1.2.1, Technical Specification, 2007. [3] ETSI TS 102 250-2, Speech Processing, Transmission and Quality Aspects (STQ); QoS aspects for popular services in GSM and 3G networks; Part 2: Definition of Quality of Service parameters and their computation, V1.6.2, Technical Specification, 2008. [4] ETSI TS 102 250-3, "Speech Processing, Transmission and Quality Aspects (STQ); QoS aspects for popular services in GSM and 3G networks; Part 3: Typical procedures for Quality of Service measurement equipment", V1.3.2, Technical Specification, 2005. [5] ETSI TS 102 250-4, Speech Processing, Transmission and Quality Aspects (STQ); QoS aspects for popular services in GSM and 3G networks; Part 4: Requirements for Quality of Service measurement equipment, V1.2.1, Technical Specification, 2007. [6] ETSI TS 102 250-5, Speech Processing, Transmission and Quality Aspects (STQ); QoS aspects for popular services in GSM and 3G networks; Part 5: Definition of typical measurement profile, V1.5.1, Technical Specification, 2008. [7] ETSI TS 102 250-6, "Speech Processing, Transmission and Quality Aspects (STQ); QoS aspects for popular services in GSM and 3G networks; Part 6: Post processing and statistical methods", V1.2.1, Technical Specification, 2004. [8] ITU-T Rec. P.862, Perceptual evaluation of speech quality (PESQ): An objective method for end-to-end speech quality assessment of narrowband telephone networks and speech codecs, 2/2001. [9] ITU-T Rec. P.862.1, Mapping function for transforming P.862 raw result scores to MOS-LQO, 11/2003. [10] ITU-T Rec. P.862.2, Wideband extension to Recommendation P.862 for the assessment of wideband telephone networks and speech codecs, 11/2007. [11] Hellenic Telecommunications and Post Commission (EETT), Desicion, No.480/017, "Definition of quality indicators of public provided electronic communication services and specification of the content and the form of the reported information as well as the method and the time of the publication by the electronic communication services providers", Official Government Gazette of Hellenic Republic, OGG/1153/B/2008, 2008. [1]

Fig. 12 Channel Quality Indicator (CQI) plotted on Google Earth

Fig. 13 Listening Quality (LQ P.862.1) plotted on Google Earth.

VI. CONCLUSIONS In this paper, it is concluded by performance measurements that nowadays mobile telecommunication networks meet a great evolution, integration and interoperability in Greece. Mobile broadband access supports ubiquitously QoS demanding services which are well described by KPIs.