PB 6-5 Customer Satisfaction Survey and Analysis

Performance Benchmarking and Quality of Supply and Service
Distribution Reform, Upgrades and Management (DRUM) Training Program
Customer Satisfaction Survey and Analysis

Dr. S. N. Nandi, CORE International, Inc.
Neena Singh, ERM India Private Limited
Introduction
Customer satisfaction measurement is a systematic process for collecting customer

data (e.g. surveys, audits, etc.), analyzing these data to make it into actionable
information, driving the results throughout an organization, and implementing
satisfaction improvement plan. Thus, customer satisfaction measurement (CSM) is a
management information system that continuously captures the voice of the customer
through the assessment of performance from the customer’s point of view
Such measurement could me made by using any of the following methods:

 Surveys
 Formal and informal feed back from customers
 Use of customer account data
 Complaints
However, survey is the most common method.
CSM will enable you to
 Understand how customers perceive your organization and whether your

performance meets their expectation.
 Identify PFI’s (priorities for improvement) areas where improvement in
performance will produce the greatest gain in customer satisfaction
 Set goals for service improvement and monitor progress against a customer
satisfaction index
 Benchmark your performance against other organization
 Increase profits through improved customer loyalty and goodwill
The organization’s process for requesting, measuring and monitoring feedback of

customer satisfaction and dissatisfaction should provide information on continual basis.
It should address conformance to requirements, meeting needs and expectations of
customers, as well as price and delivery of product.
. A typical customer satisfactions survey process includes following steps
Space for
Power Finance Corporation Ltd.
Institution Logo (A Govt. of India Undertaking)
1
Obtain Top Management Buy – In
Determine Survey Objectives
Exploratory Research
Plan Sampling and Data collection
Design of Questionnaire
Actual Survey
Data Refining and Analysis
Recommendation and Implementation Plan
Plan timing of next Survey
Fig. 1: Survey Methodology
One must begin with essential preliminaries, such as obtaining top management buy –in
and conducting exploratory research. From there, the sampling and data collection
details should be worked out and the key business processes identified.
Essential Preliminaries
The first this to figure out is why the customer satisfaction survey is being conducted in
the first place. Answers such as “to listen to the customer” or “because everybody has
Space for
2
one” indicate a lack of clear motivation. Good reasons for doing a customer satisfaction
survey include:
1. To find out where to focus process improvement efforts;

2. To determine whether previous improvement efforts have worked; or
3. To see whether strategic advantages or disadvantages exist.
It is imperative that the reason for doing the survey be clearly stated and understood
ahead of time.
Little can be done with customer satisfaction survey results without the approval of top
management. Thus, it is essential that the survey project have to management buy-in.
top management should agree with the purpose of the research and understand what
actions will result from the survey. One way of ensuring management buy-in is to
involve management in the design and construction of the survey.
It is also worth remembering that in any service, it is the front-line employees who have
the bulk of the customer contact and thus the greatest opportunity to implement process
improvements that directly affect the customer. Involving both management and front-
line employees is crucial, because it lays the groundwork for effective implementation of
the survey’s findings.
During the exploratory research, it is essential that customer expectations are captured
in terms of factors/dimensions. These factors are preferred to be linked with different
processes through which services are delivered. This approach helps in identification of
processes requiring improvements.
Sampling
The importance of sampling
The purpose of sampling is to select a small number of ‘units’ from the target
‘population’ in such a way that the sample is truly representative of the total population
being surveyed.
Sampling Reliability
The reliability of a sample is judged on repeatability. If you did exactly the same survey
again but with a different randomly selected sample of customers, would you get the
same results? This is primarily determined by sample size. Three specific aspects of
sample reliability determine the latter: precision, confidence level and variance.
Precision
Space for
3
The precision of a survey result can be defined in general terms as its accuracy. That
means that if you conducted the same survey with a different sample the result for the
‘yes’ vote could be anywhere between 33 and 43 per cent. With a much larger sample
e.g. 1000 you would achieve much better precision, say +/- 1 per cent.
Confidence level
You can calculate the precision of your sample at various confidence levels. Absolutely
critical research such as medical research is typically conducted at 99 per cent
confidence level. In case of distribution companies, it may be sufficient that sampling
the based on 95 per cent confidence levels, meaning that you can be confident that the
result would fall within your stated precision at least 19 out of every 20 times that the
survey was conducted.
Variance
The precision that you can expect from a specific sample size at a given confidence
level can still differ considerably depending on the variance of views held by the
respondents. So the lower the variance (the range of views expressed), the more
reliable any sample size will be at any confidence level.
Determining the sample size
The table shows required sample size against various levels of standard deviation and
precision
Table 1: Sample size determination
Required precision
(+/-%) Standard deviation assumed
0.5 0.75 0.1 1.25 1.5 1.75 2.0
5 4 9 15 24 35 47 61
4.5 5 11 19 30 43 58 76
4 6 14 24 38 54 74 96
3.5 8 18 31 49 71 96 125
3 11 24 43 67 96 131 171
2.5 15 35 61 96 138 188 246
2 24 54 96 150 216 294 384
1.5 43 96 171 267 384 523 683
1 96 216 384 600 864 1176 1537
0.75 171 384 683 1067 1537 2092 2732
0.5 384 864 1537 2401 3457 4706 6147
0.25 1537 3457 6147 9604 13830 18824 24586
Space for
4
. For example: if variation in the sample is small (low S.D. of 0.5) and required precision
is within 1%, a sample of 96 will suffice. If variation in the sample is large (high S.D. of
2.0) and required precision is within 1%, a sample of 1537 will be needed.
Most samples for customer satisfaction surveys would fall in the range of 200-500
respondents. Based on our experience this will yield precision of around +/- 1 per cent
at the 95 per cent confidence level for the vast majority of CSM surveys. Increasing the
sample size would improve the accuracy but at a very high cost. Typically, to double the
precision (for example from +/-2 per cent to +/- 1 per cent) you would have to increase
the sample size by four times, quadrupling the cost of your survey.
Sampling options
There are several types of sampling methods that could be adopted for the survey.
Some of those are outlined below:
Simple random sampling
To draw a simple random sample, the researcher starts with a complete list of the
customer groups to be surveyed. The size of the sample is determined and chosen from
the complete list on a random basis, which means that each individual has the same
likelihood of ending up in the sample.
Stratified random sampling
Random sampling can sometimes distort results in markets where some customers are
more important than others. In this case stratified random sampling is used. The method
involves weighting the sample on the basis of the importance of various segments
making up the market.
Cluster sampling
The third and less costly method of producing a random sample, is cluster sampling.
Random samples are therefore often drawn from a small number of tightly defined
locations (clusters) deemed to be typical of the target market in question. This method is
considered to be of sufficient statistical accuracy for most commercial market research.
Non- random samples:
Convenience samples
This would involve gathering data from any convenient group, for example, passers-by
in a street interview situation.
Judgmental samples
Space for
5
Here respondents would be selected on the basis of the researcher’s (or interviewer’s)
belief that they were representative of the target market being studied. The more
knowledgeable the person who selects the respondents the more accurate it is likely to
be.
Quota samples
Quota controlled samples are frequently used by commercial market research agencies
to minimize the cost of fieldwork. The research agency, using as a basis published
statistics, decides on controlled quotas (or groups) of respondents for each interviewer
in the field. For example, the interviewer might be told to question 20 females aged
between 20 and 35 years, 15 females aged between 36 and 50 years and 25 females
aged 51 years and over.
Design of Questionnaire
The basic principles of asking questions
When judging any question on a questionnaire you need to apply the following tests to
it:
 Will the respondent have the information / knowledge to answer it?

 Will the respondent understand the question?
 Is the respondent likely to give a true answer?
 Will the question bias the response?
It is very easy for a question to fail by any of these tests, so let’s examine each of them
carefully.
Necessary Information / Knowledge
As far as customer surveys are concerned, setting very clear objectives at the outset of
the project will minimize this pitfall., You may want to measure the satisfaction and
explore the attitudes only of knowledgeable people with direct experience of the
product.
Understanding the question
There are three problem areas in understanding.
Firstly, there is the loose or ambiguous meaning of many common words. A good
example is ‘regularly’. A regular habit to one person may seem a very occasional one to
Space for
6
someone else. ‘Do you eat out regularly?’ would therefore mean different things to
different people. Rather than try to define such words it is better to avoid them and
reword the question: ‘When did you last eat out?’, or, ‘How many times have you eaten
out in the last month?’
The second problem is the double question. ‘Does Ariel Future get clothes clean without
damaging the fabric?’ This one question should be split into two separate questions to
accommodate the full range of possible opinions. Sometimes questions use two very
similar adjectives: ‘Do you find the atmosphere in this restaurant relaxing and informal?’
If the researcher sees only one meaning to this question the closer of the two adjectives
should be used and the other eliminated.
The third problem is the long rambling question, or definition, Used in a self-completion
questionnaire, it results in many respondents not taking the trouble to figure it out, either
omitting it altogether or, even worse, ticking any available answer. In telephone surveys,
too, the long question is difficult for both respondent and interviewer.
Giving a true answer
First, respondents often find it difficult to articulate their views, especially where
attitudes are concerned. They should therefore be given plenty of time to formulate their
ides into words.
The Second potential pitfall is respondents’ memory. People may believe they are
giving a truthful answer but in reality, due to a defective memory, they are giving an
inaccurate response.
The third pitfall is more difficult to identify and negotiate. Some respondents may be
unwilling to answer truthfully to some questions. This is not malicious. There is no
evidence that respondents try to deliberately mislead researchers in customer surveys
or any other kind of research but there is evidence that respondents will give what they
consider to be an acceptable answer which does not really reflect their views.?
A fourth reason why you may generate inaccurate data n a customer survey is that the
question itself may suggest a form of answer to the respondent.
The most likely form of bias in a customer survey is to influence the customer towards
satisfaction rather than dissatisfaction. To avoid this it is better to use phrases like ‘Rate
our performance’ rather than ‘How satisfied are you with…’
Piloting
As well as studying the questionnaire an applying to it the tests of knowledge,

understanding, truthfulness and absence of bias, you must pilot it to test just how well it
works in practice.
Space for
7
Types of question
The main distinction between types of question is between the closed and the open
question. In the survey itself, where you may have time constraints and where you need
to produce quantitative answers for statistical analysis, you will use mainly closed
questions.
Closed questions
Closed questions give respondents a fixed selection of answers to choose from. They
are quick and easy to administer and analyse, offer the least scope for interviewer or
respondent error and produce quantifiable data. The interviewer (or the respondent in a
self-completion questionnaire) need tick only the relevant box.
Closed questions could be dichotomous with only two alternative answers possible.
They can be multiple choices, enabling respondents to choose from any number of
possible answers. Some questions demand that the respondent selects only one
answer (single response), The table shows required sample size against various levels
of standard deviation and precision other questions allow selection of all relevant
answers (multi-response).
Closed questions can also be linked with a rating scale to qualify the strength of the
response.
Open questions
Open questions are used when the researcher does not want to lead the respondent in
any way. These questions are more commonly used at the exploratory stage rather than
at the survey stage.
Purpose of questions
Whether the questions are open or closed, they can be used for different purposes,
usually to find out about respondents’ behavior, to understand their attitudes, or to
gather details about them for classification purposes.
Behavioral questions
Behavioral questions are about what the respondent does or has done in the past. They
are both factual and tangible which usually make them easier to answer so they tend to
be placed at the beginning of questionnaires.
Space for
8
Attitudinal questions
Although behavioral questions can ask the respondent to dig deeply into his or her
memory, attitudinal questions usually require more thought and more decision making
and should therefore follow rater than precede behavioral questions. This gives the
respondent some time to think about the issues involved. It also provides an opportunity
for the interviewer to build rapport – very useful later when the interviewer wants the
respondent to make the effort to give proper consideration to any difficult attitudinal
questions.
Customer satisfaction measurement is primarily concerned with attitudinal questions.

Attitudes towards the product and services and selecting the supplier, attitudes about
the performance of the product and supplier and, most difficult of all, attitudes about the
image of the supplier, will all come within this category.
Rating scales
Since customer satisfaction measurement is all about measuring attitudes, the method
you use to do the measuring is an extremely important aspect of the exercise. Market
researchers have developed several different types of ‘rating scale’ to measure the
strength of peoples’ attitudes and her ewe examine those scales with most relevance to
customer satisfaction measurement: Likert scales; verbal scales; semantic differential
scales; un-graded scales; numerical rating scales; ordinal scales; SIMALTO scales.
Likert scales
Likert scales are designed to measure degrees of agreement with a statement, the
wording shown in the example reflecting the Market Research Society’s recommended
approach. However, some people maintain that it is better to offer only four possible
responses, eliminating the middle ground, easy option.
Verbal scales
Similar in principle to Likert scales, verbal scales use words to describe degrees of the
attitude being measured. In the case of customer satisfaction measurement, of course,
the concepts involved are importance and satisfaction.
Semantic differential scales
Semantic differential scales display an attitude battery between two opposing words.
Typically you would allow more options than the four or five given in the Likert scales
but you would not label the options in any way. The respondent is left to define the
strength of his or her attitude by selecting the proximity to the appropriate adjective.
Space for
9
Numerical rating scales
A numerical rating scale requires respondents to ascribe a mark, out of 10 for example,
to indicate the strength of their attitude. There is no firm rule on the size of the scale.
Some companies use a scale of 1 to 5. Federal Express uses a 101-point scale (0 to
100) to measure ‘micro-shifts’ in customer satisfaction. In the UK, a ten-point scale
works very well. It is understood by respondents who have often grown up with marks
out of ten at school and it is wide enough to accommodate a good range of customer
opinion.
Ordinal scales
Ordinal scales require the respondent to indicate the relative strength of his or her
attitude to the different criteria by ranking them in order of importance, preference and
so on. No further qualification is demanded.
Actual Survey
There are two main methods of gathering information from customers: qualitative and
quantitative:
Quantitative Research Qualitative Research

(Hard issues and facts) (Soft issues and opinions)
Telephone surveys Focus groups /user groups / customer panels
Self-completion questionnaires Face-to-face interviews
(Comment cards)
Postal surveys
Third party surveys
Online surveys
Qualitative methods provide feedback from customers on ‘soft issues and opinions’ and
give the organization an opportunity to probe customers’ feelings and attitudes on an in-
depth basis. Typical qualitative techniques include user panels, customer groups, and
personal interviews.
Quantitative methods allow the organization to quantify customer opinions in a

numerical fashion. Typical quantitative techniques include telephone surveys, self –
completion questionnaires, the monitoring of compliments and complaints, and mystery
shopping.
Care is required in the choice of the best methodology. The best plan is to adopt a
variety of these techniques – an organization is wise not to restrict itself to just one
method. Forte Hotels, for example, uses mystery guest reports, quarterly telephone
surveys and biannual guest-satisfaction postal audits to measure customer retention. It
Space for
10
is normally possible to gain a greater degree and depth of information from qualitative
methods, although quantitative routes reach a wider audience and often prove more
cost-effective. Best practice shows that organizations need to use qualitative research
to first gain insight into customers’ attitudes and behavior before using quantitative
methods.
Data Refining and Analysis
Data collected through the questainnaire are first checked for incompleteness,
inconsistencies, and inaccuracies.
Structuring the questionnaire according to business processes, makes it possible to

structure the data analysis in a managerially meaningful way. The analysis determines
links between the following components:
Dimension (Subprocess)
Process
Overall Satisfaction
Fig. 2: Hierarchy of Dimensions
Customers’ satisfaction level with the dimensions (subprocesses) of a process affects

their satisfaction level with that process. Likewise, customers’ satisfaction level with the
various processes influences their satisfaction level with the firm overall.
Using the statistical models for predicting satisfaction, it is then possible to estimate the
relative importance of the various processes in predicting satisfaction with the firm, and
the relative importance of the dimensions in predicting process satisfaction. These
importance weights can then be related to performance, as measured by the percent
satisfied. Charting the importance and performance information can help provide
preliminary ideas about where resources should be allocated.
Given that a complete data set has been constructed, there are both simple (bivariate)
and more complex (multivariate) ways to relate the process satisfaction levels to the
overall satisfaction level. The easiest way is to use a procedure called “regression.”
Regression is a statistical method that allows you to quantify the relationship between
variables. Further, once that relationship has been established, the relationship can be
used to predict various outcomes.
Space for
11
Dependent Variable = OVSAT

Independent Beta Standard t p
Variable Coefficient Error
Billing .50 .15 3.33 .00
R2 = .25
Fig. 3: Predicting Overall Satisfaction - Example Bivariate Results
The t-statistic tells you how certain you can be that the beta coefficient is an accurate
predictor. The farther the t-statistic is from 0, either positively or negatively, the more
confident you can be that the beta coefficient is significant.
Predicting Process Satisfaction
You have seen how the process satisfaction scores can be related to the overall
satisfaction score. Ultimately, though, you will need to know what drives the overall
process satisfaction. Thus you must relate the process dimension scores to the process
satisfaction level. Again, you may do this through the simple (bivariate regression)
Importance – Performance Mapping
Another data presentation device that many managers find appealing is the importance-
performance map. The idea is derived from standard quadrant analysis, as has been
used for many years in business strategy. In general, this approach argues, we should
be most concerned about those issues for which importance is high and performance
(typically measured by average satisfaction) is poor. These yield the greatest potential
for gain.
Recommendation and implementation plan
Based on analysis of data regarding sub-processes (dimensions./factors),processes,

and overall, one can formulate important recommendations which could be considered
for implementation. For example, one could work out process wise satisfaction levels,
as shown in Fig.4. Take the case of metering process in a distribution company, say,
which has been found to have a low score in satisfaction, the concerned department
may look for possible causes and then draw up an implementation plan.
Another set of valuable recommendations could be drawn up from the analysis of

importance-performance matrix. As shown in Fig.5,the matrix has four divisions in
which values of different dimensions/factors will fall in. Depending upon which division a
factor is located, certain actions could be suggested. For example, if a factor/sub-
process/process is placed in a division-Low importance and High
performance/satisfaction level, cost reduction program may be undertaken for the same
factor/ process. As this step will not affect the satisfaction level, higher profitability may
result in.
Space for
12
Satisfaction Level (% age)
A B C D
System / Process / Factors

Fig 4: System / Process-wise Satisfaction Level
High
ACTION CONSOLIDATE
Importance
LOW PRIORITY COST CUTTING

Low
Low Performance High
Fig. 5: Areas for Actions

Conclusion
Monitoring of Customer Satisfaction is now very important for a distribution company,

as it will lead to better customer experience and thereby goodwill. A quality survey is
required to be undertaken periodically. The proper methodology for the survey as briefly
described here will go a long way.
Space for
13

PB 6-5 Customer Satisfaction Survey and Analysis

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

PB 6-5 Customer Satisfaction Survey and Analysis

Uploaded by

Copyright:

Available Formats

Performance Benchmarking and Quality of Supply and Service

Distribution Reform, Upgrades and Management (DRUM) Training Program

Customer Satisfaction Survey and Analysis

Customer satisfaction measurement is a systematic process for collecting customer

Such measurement could me made by using any of the following methods:

However, survey is the most common method.

CSM will enable you to

 Understand how customers perceive your organization and whether your

The organization’s process for requesting, measuring and monitoring feedback of

. A typical customer satisfactions survey process includes following steps

Obtain Top Management Buy – In

Determine Survey Objectives

Plan Sampling and Data collection

Data Refining and Analysis

Recommendation and Implementation Plan

Plan timing of next Survey

Fig. 1: Survey Methodology

1. To find out where to focus process improvement efforts;

The importance of sampling

Determining the sample size

Simple random sampling

Stratified random sampling

Non- random samples:

The basic principles of asking questions

 Will the respondent have the information / knowledge to answer it?

Necessary Information / Knowledge

Understanding the question

There are three problem areas in understanding.

Giving a true answer

As well as studying the questionnaire an applying to it the tests of knowledge,

Customer satisfaction measurement is primarily concerned with attitudinal questions.

Semantic differential scales

Numerical rating scales

Quantitative Research Qualitative Research

Quantitative methods allow the organization to quantify customer opinions in a

Data Refining and Analysis

Structuring the questionnaire according to business processes, makes it possible to

Fig. 2: Hierarchy of Dimensions

Customers’ satisfaction level with the dimensions (subprocesses) of a process affects

Dependent Variable = OVSAT

Fig. 3: Predicting Overall Satisfaction - Example Bivariate Results

Predicting Process Satisfaction

Importance – Performance Mapping

Recommendation and implementation plan

Based on analysis of data regarding sub-processes (dimensions./factors),processes,

Another set of valuable recommendations could be drawn up from the analysis of

Satisfaction Level (% age)

System / Process / Factors

LOW PRIORITY COST CUTTING

Fig. 5: Areas for Actions

Monitoring of Customer Satisfaction is now very important for a distribution company,

You might also like