34th Meeting of the Quality of Service Development Group

QSDG
Johannesburg, South Africa, 26-27 July 2017

Strategies to Establish Quality Measurement Frameworks

(Draft Rec. E.RQUAL)

Tiago Sousa Prado

ANATEL/Brazil
tiago.prado@anatel.gov.br
 AGENDA
Motivation to E.RQUAL

Approved scope

Proposed structure

Proposed KRI definition

Proposed quality regulation principles

 Motivation to E.RQUAL

Creating an international reference to be

considered by governments aiming to establish
national or regional quality regulatory
frameworks
Motivation to E.RQUAL

E.RQUAL - Strategies to Establish Quality

Measurement Frameworks

ITU-T E.800-series Supl. 9 Guidelines on

regulatory aspects of QoS

ITU-T Rec. G.1000 - Communications quality of

service: A framework and definitions
 Approved Scope
Principles High-level quality regulatory approaches and
and enforcement strategies, for example there are
several different approaches to attain quality
enforcement improvements where needed and to help
consumers make informed choices;
strategies
Strategies to empower end-users with useful
Strategies to information based on results of the
aforementioned key regulatory indicators
empower measurements, such as standardized service
end-users labels that tabulate key aspects of service
quality.
 Approved Scope
Key regulatory indicators to assess and compare
QoS quality of the services achieved by operators, in
the context of subscription voice, video and IP-
regulatory based communications services, (these
assessment indicators are more useful to regulators than to
operators)

Key regulatory indicators to assess and compare

QoE quality as perceived by end-users of differing

operators, categorized by the applications used (these

regulatory indicators are more useful to users than to operators);

Key regulatory indicators to assess and compare
telecommunication users satisfaction for differing
assessment operators in the context of the applications they use,
possibly through participation in questionnaires;
 Proposed Structure
Section 1 Section 2 Section 3
Scope References Definitions

Section 4 Section 5 Section 6

Abbreviations Conventions Principles for
and acronyms quality regulation

Section 7 Section 8 Section 9

Assessing and Assessing and Strategies to
comparing QoS comparing QoE empower users
 PKey Regulatory Indicators:
proposed definition

A quantifiable measure defined by the

regulator as a representation of the service
quality delivered or perceived by end-users,
including their overall satisfaction.
 Proposed quality regulation principles

1 - Quality regulation better fits retail market

Information asymmetry between operators and end-

users (availability, coverage, QoS, QoE, repair time, etc)
Wholesale contracts usually include SLAs
Quality regulation should focus on operators with offers
in telecom retail markets
 Proposed quality regulation principles

2 - Free competition fosters quality improvement

Competition pushes operators to differentiate their

services by price and quality
Quality regulation is an entrance barrier in the market
Avoiding to create costly measurement frameworks
 Proposed quality regulation principles
3 - Quality regulation overburdens small operators
and newcomers

Small operators and newcomers already have market incentives to

be better than traditional telcos
Data collection, consolidating and publication can be impracticable
for them (more entrance barrier and less competition)
Example: exempting operator less than 5000 subscribers to comply
with quality regulation
 Proposed quality regulation principles
4 - Responsive regulation avoid litigation and raise
regulation effectiveness
Quality improvement pushed by stimulus triggered according to assessment
results
Creating steps of increasing pressure over the operators with poor quality
Keeping fines as the last alternative
Examples:
Starting a special vigilance regime over an operator and ask an action plan to improve quality
Highlighting to end-users the poor results of an specific operator (naming and shame)
Providing end-users compensation
Suspension of sales
 Proposed quality regulation principles
5 - Convergent networks and services limit the
number of KRIs needed
Economies of scope brought by network convergence
Centralized customer service and billing infrastructure
Necessity to avoid KRIs duplication over differing services
Focus on
Availability
Level of complaints from end-users
Level of compliance with the capacity offered (ex. speed, latency, etc)
 Proposed quality regulation principles
6 - KRIs targets reliable choices demands knowledge
on current quality delivered
Avoiding defining targets beforehand
Preferable to measure the KRIs in a first cycle and then
defining reasonable minimum and reference targets
Setting targets in a secondary instrument helps to expand
regulation lifetime and to evolve the targets when needed
(end-users quality requirements always increase)
 Proposed quality regulation principles

7 - KRIs not always need to have targets

Some informative KRIs are important to empower users but do not

need to have minimum targets defined
Examples:
Coverage
Average installation and repairing time
Average waiting time in the call center
Average network speed
Average performance of using some popular applications
Proposed quality regulation principles
8 - Granularity and accuracy of KRIs empower end-
users

The more granular the measures, the more the KRIs will
be useful to help end-user make informed choices
Statistic validation should be an obsession, since it
avoids litigation on quality assessment results
Proposed quality regulation principles
9 - Technology neutral KRIs keep quality regulation
standing
Avoiding defining in regulation KRIs only applied to
specific technology (ex. LTE, ADSL, GPON, etc.)
KRIs should be more general and easily understandable
by end-users
Examples:
Refers to site instead of e-NodeB
 Proposed quality regulation principles
10 End-users satisfaction and service quality
perceived complement QoS assessment
Satisfactory results in technical KRIs may hinder poor quality
perception by the end-users
Surveying among end-users is the unique way to really measure
users satisfaction and quality perceived
Surveys results should ponder KRIs assessment
Example:
An user satisfaction or quality perceived positive variation may lower the pressure from regulator towards
improvements in network quality
 Proposed quality regulation principles
11 - Measurement tools and sampling plans are
better defined hearing all stakeholders
Important to establish (after regulation issuance) a quality
management group gathering regulator, operators and
measurement solution providers identified
Focus on defining detailed operational guidelines (ex. tools,
sampling plans, requirements, databases, sources, criteria,
collecting procedures, etc.)
 Proposed quality regulation principles
12 - Raising customer rights awareness pushes
quality improvement
Well informed end-users help to monitor quality and
request improvements
Empowering end-users by, for example:
Making clear in retail contracts the quality level, based on KRIs
targets defined (contract breaking with fees exemption in case of
poor quality results)
End-users compensation in case of poor quality levels
 Proposed quality regulation principles
13 - Massive publication of quality results impacts
operators marketing and investment strategies
Publication of quality results, specially ranking operators,
makes shareholders and board of directors worried about
In some operators, directors bonuses are paid based on
quality results and ranking positioning
Marketing strategies and investment planning should be
impacted by quality results
34th Meeting of the Quality of Service Development Group
QSDG
Johannesburg, South Africa, 26-27 July 2017

Thank You!
Tiago Sousa Prado
ANATEL/Brazil
tiago.prado@anatel.gov.br