Quality Assurance in Statistics: Producer Perspectives in the Light of International Practices

S.S. Colombage
50th Open Forum: Ensuring Quality of Survey Data Centre for Poverty Analysis
April 29, 2014

Outline Rationale for Quality Assurance International standards for statistics Sri Lankas Case Concluding Remarks

Rationale for Quality Assurance

Main statistical agencies
Census and Statistics Department (CSD)
Household income & expenditure surveys, labour force surveys Macro-level data: National accounts, prices & wages

Central Bank of Sri Lanka (CBSL)

Macro-level data: Balance of payments, financial sector, fiscal sector Consumer Finances Surveys - abandoned?

Decision makers in public & private sectors heavily depend on statistics Doubts about GDP, inflation, poverty & unemployment data.

Sri Lanka: Dramatic poverty reduction vs. moderate economic growth?

Sources: Department of Census & Statistics Central Bank of Sri Lanka

Sri Lanka: Dramatic poverty reduction vs. stagnant income share of the poorest?

Sources: Department of Census & Statistics Central Bank of Sri Lanka

What is statistical quality?

Old notion: quality = accuracy

Current version: quality = accuracy +

Relevance Timeliness Accessibility Clarity Coherence Comparability Efficiency etc.

Improving statistical quality means enhancing the fitness of data to meet the expectations of the users.

International Codes of Practices

At international level, increased attention on statistical quality management since the mid-1980s Common quality management tools developed by international agencies & national statistics agencies
UN Statistical Commission Expert Group: Generic Quality Assurance Framework (NQAF), 2010 Quality Assurance Framework of the European Statistical System IMF: Data Quality Assessment Framework (DQAF) Statistics Canada: Quality Assurance Framework Code of Good Practice in Statistics for Latin America & the Caribbean

Generic National Quality Assurance Framework (NQAF) template

NQAF 4: Assuring professional independence

Statistical agencies should develop, produce and disseminate statistics without any political or other interference from government agencies, private sector entities or persons. The French law on statistics states that "the conception, production and dissemination of official statistics are carried out in full professional independence". In Sri Lanka, no mention about autonomy of CSD, which comes under the purview of Ministry of Finance & Planning

Statistics Canada

NQAF 16: Assuring timeliness & punctuality

Statistical agencies should minimize delays in making data available

Timeliness: How fast the data are released Punctuality: Whether the data are released on the dates promised.

Sri Lanka CSD: Advance Data Release Calendar (ADRC) available on the website Delays e.g. Final report of Population Census (2011) still not released

NQAF 19: Managing metadata

Statistical agencies should provide information covering the underlying concepts, variables and classifications used, the methodology of data collection and processing, and indications of the quality of the statistical information - to enable the user to understand all attributes of the statistics. Metadata = Data about data SL CSD: Provision of metadata are inadequate. Statistics Canada
Integrated Meta Data Base (IMDB): repository of information on each of nearly 400 active surveys IMDB is available to outside users of SCs website. Each data release on the site includes hyperlinks to metadata from IMDB

Australian National Health Information Knowledgebase: An electronic storage site for health metadata United States Health Health Information Knowledgebase

Quality Assessment & Reporting

Purpose: To evaluate & prevent problems that may arise during statistical processes. Assess product quality in terms of relevance, accuracy & reliability, timeliness & punctuality, coherence & compatibility, accessibility & clarity Each of the above dimensions is assessed by using quality indicators. E.g. Accuracy: How complete is the sample frame? Quality assessments could be conducted by internal or external teams Statistical assessment systems are well established in several countries including Canada, Netherlands & South Korea

Sri Lanka: Positive initiatives

LankaDatta maintained by CSD
National data repository of Sri Lanka, which holds of over 350 data files from different surveys LankaDatta adheres to international standards in archiving microdata such as the Dublin Core Metadata Initiative

General Data Dissemination System (GDDS) of IMF

Both CSD & CBSL have been participants since 2000 Information/data are submitted to IMF under Article IV

Sri Lanka: Shortcomings

A National Quality Assurance Framework (NQAF) for statistics lacking in Sri Lanka Census & Statistics Department (CSD) & Central Bank (CBSL) are silent on quality assurance QA is explicit in websites of statistical agencies of some countries e.g. Australia, Canada CSD & CBSL are not very receptive to user feedbacks

Concluding Remarks
Quality assurance is not prominent in the Census & Statistics Dept., Central Bank & other statistical agencies in Sri Lanka Hence, credibility of data are at stake Recommendations:
National quality assurance framework for statistics following UN guidelines be adopted Quality culture needs to be inculcated User feedbacks need to be accommodated in statistical processes Professional independence of statistical agencies vital