Measuring Customer

Satisfaction

A workshop approach to
creating the right questions

PERSIGN associates
5 Whitlars Drive
Kings Langley
Herts WD4 8DG

T: 01923 269597
M: 07887 712902
E: Winston@persign.co.uk
Measuring Customer Satisfaction and then acting
Introduction
The following paper describes with a workshop that may be run as a single event over
one or two days or as a series of team sessions lasting about an hour each and
spread over several weeks. The team participants should be a sample population
from the company that regularly deal with the customer. However, as we shall see it
is not always easy to define the organisations immediate customers. The purpose of
any satisfaction survey is to identify areas where the company can make
improvements to its performance, and to achieve this the survey must consider not
only the end-user but also the intermediate 'customers' who are part of the
distribution chain. So we define a customer as,
'Someone who receives output from the organisation that enables them to complete
their tasks".
This also allows us to use the workshop to develop an internal customer satisfaction
survey between departments or functions. Our definition of an internal customer
would then be as
'Someone who receives output from the department that enables them to complete
their tasks".

Typically many surveys focus their attention on the end-user, the final customer of the
product or service, but by doing so many opportunities for real improvements can be
lost or hidden.

On completion of the workshop we move into Data Gathering then Data Analysis
and the last step of implementing Improvements. For details on How to Construct a
Questionnaire refer to the supporting paper on this disc.

Note that it is not necessary to create a questionnaire. The questions identified can
be asked via interviews, telephone calls etc.

The Workshop
Specific techniques that are used are printed in italics; details of these can be had
from Arundel Quality Services. If the workshop is being run to create an internal
survey then simply substitute department or function for the word organisation.

Step1. Outputs & Customers

Brainstorm the organisations outputs.
Identify the customers that receive the outputs.
(At this stage do not attempt to eliminate potential customers or outputs, if necessary,
this will take place later)

Step 2. The criteria

Brainstorm all the possible criteria that may be used to measure the quality of the
output or performance.
In brainstorming the possible criteria the team should try to take the perspective of
the customer and the routes to market. For example a company manufacturing
kitchen equipment has several routes to market.

1. The architect who designs the house may specify the kitchen fittings,
and will be concerned with the quality of the drawings
 2. The builder or installer will be concerned with ease of fitting and in the
case of replacement fittings may have a strong influence over the
end users choice.
3. The warehouse or showroom may be concerned with minimising
storage space and this can influence the products they are willing to
hold.
4. The domestic end user may be more concerned with the
appearance of the products in the showroom.
5. The commercial end user may be concerned with robustness and
reliability

This is not a definitive list and is not intended to show all the quality criteria that come
into play when a decision on choice is made. It is important however to address the
right questions to the right person in the distribution chain. This should be discussed
with the team

Step 3. Prioritise criteria

Through group discussion remove duplicated and unnecessary items from the list
generated in step 2. If the list is large, use Multi or Delphi Voting to reduce the
number of criteria and then use Paired Comparison to identify the top six criteria. If
the team has identified more than one 'customer' in the distribution chain then there
is likely to be more than one set of criteria. If this is the case then before prioritising
get the team to identify which criteria should be used with a specific customer, there
may be some overlap and it may be necessary to rewrite the lists under specific
customer headings.

Step 4. Create the first questions

This is the most difficult step. Split the team into smaller groups (2 or 3) and ask them
to combine specific outputs with specific criteria to create appropriate questions.
On completion write the results on a flipchart and then through group discussion
identify the merits or otherwise of the suggested questions. Issues to be addressed
are:-
¾ What will we do with the results of this question?
¾ Is there any ambiguity in the question that may devalue the results?
¾ Do the questions address the real issues?

This last question may result in a directional change; this is likely to occur when the
actual process of achieving the end result is visible at all times to the customer.
Consider a maintenance department within the organisation. Although the final
output is important other features such as agreeing the task, meeting the agreed
start and finish dates, minimising disruption to other employees during the work and
returning the work environment to a appropriate condition on completion are now
likely to have equal importance. The Customer in this case can 'see' the process.
Note how the 'client satisfaction' questionnaire for consultancy is focussed on the four
aspects of the work model and not purely on the direct outcomes of the consultancy
process.

Step 5. Refine the questions

Split the team into 'interest groups' if necessary. Through group discussion create the
set of questions that could be addressed to the customer.

Many questionnaires/surveys suffer from poorly constructed questions.

• It is important to ensure that you understand why you are asking the question and
what you wish to learn from the results.

• Avoid ambiguity – “Are you satisfied with the colour and finish of the product?”
will leave the customers confused over whether they are addressing the issue of
colour or finish. How do they answer if they are satisfied with colour but not finish?

• Be specific – “How do you rate the quality of reception?” Is the customer now
being asked about the décor, the approach of the receptionist on the way in or
way out or perhaps the nice fish tank in the corner?

• Don’t assume the question will be understood in the same way as you intended it.
“How well do we meet your expectations?” is better worded “With regard to ….
….how well do we meet your expectations? Then there can be no confusion over
what the response means.
• Be careful of words that have different meanings to different people. “How do
you rate the quality of our bicycles?” The young boy will consider the quality of
ownership high as the thirty gears impress his friends, but the trained engineer may
consider the whole thing over engineered and therefore rate its quality of design
low.

• Don’t second guess your customers and leave out questions where you think you
already have the answers. One supplier to the shipbuilding industry consistently
failed to meet agreed delivery dates, but was pleasantly surprised to find that the
customers did not see the situation this way. The goods were always ordered well
in advance of the real required date, they were never late for installation.

• Consider carefully whether the recipient is the best person to ask a specific
question. Can they give you a genuine answer? Asking the store man whether the
financial reports are appropriate and the managing director whether the pallet
size for delivery is correct may be the wrong way around! Where you can not
avoid sending the questions out to people who may not use a particular aspect
of your service then it may be worthwhile asking ‘How often do you use this
feature?”

• Can you identify areas of best practice that allows your customer to draw
comparisons?

• How can your customer propose improvements, how can they register the
problems they encounter even when things have gone well?

• Do not raise customer expectations unnecessarily by asking open-ended

questions, “What else can we give you?” may supply you with responses you
cannot fulfil. Questions like this are best used in follow up interviews.

• Test your questionnaire out on someone not involved in the design before sending
it out. Find out if it really is as user friendly as you hoped.

Create version 1 of the question set.

Step 6. How will we do it?

Through group discussion identify the method to be used, whether this is to be
questionnaires, interviews or a mix of both. Identify also who will be responsible for the
activity. Where the number of customers is large then the size of the sample
population can be identified from statistical sampling tables.
Step 7. When will we do it?
Timing is critical, being asking how well the forecourt staff handled the sales enquiry
three years after the customer bought her new car is unlikely to provide you with valid
data. At what stage of the life cycle and to whom do we address our questions
should now be identified. Note that in an internal survey when the transactions are
on a day-to-day basis then timing is not so critical. The frequency of the survey, if it is
to be repeated, should also be agreed.

Step 8 Test Run

Identify a small sample and test run the questionnaire or interview. The lessons learnt
form this activity should now be used to create the final version of the questions to be
used.

Data Gathering
The requirements for this stage have been identify during the workshop, and this
activity should be relatively straight forward, but the team will still have to consider
the logistics of the exercise.

Data Analysis
A variety of statistical techniques can be used from simple frequency, percentages
and Pareto analysis through to rank correlation and standard deviation. The results
should be assessed to identify:-

¾ Areas of weakness that require urgent attention.

¾ Areas of improvement previously thought satisfactory
¾ What problems do the customers encounter?
¾ How well what is done relates to the customers' needs?

Where it is possible check the validity of any conclusions by asking the customers if
they agree with your summaries.

Implement Improvements
¾ Implement any obvious changes immediately
¾ Prioritise the remaining issues and plan how to deal with them
¾ Identify the teams or individuals who will resolve these issues
¾ Are cross-functional teams needed?
¾ How will the support of the organisation in total be achieved?

In addition to tackling the issues the team must also decide how customers will be
kept informed and how regular feedback can be maintained from the customer
base.