Understanding Percentiles and Norms (Culture)

Overview
Denison Organizational Culture Survey results are presented in the form of percentiles. The purpose of
this document is to describe what percentiles are, why they are used, and how they are used by Denison
Consulting to help make culture survey results more understandable.
What are percentiles?
Percentiles show how an organizations culture ratings compare to ratings of other organizations. They
indicate the percent of organizations that scored below a certain value. For instance, if your
organizations Core Values index average is at the 57th percentile, it means that 57% of organizations
in the benchmark database had lower index averages than your company.
Why use percentiles?
The most difficult parts of conducting any type of evaluation are determining how well someone or
something actually performed and then communicating that information in a meaningful way. For
instance, suppose that Heather, a fourth-grader, knows 18 out of 30 words on a vocabulary test, or that
Steve, a high school senior, runs a mile in 7 minutes and 40 seconds. Most people would not find these
numbers to be very meaningful. Is knowing 18 out of 30 words an indication that Heather is falling
behind in vocabulary, or is she right where a fourth-grader should be? Is 7 minutes and 40 seconds a
poor time for a senior running a mile or should Steve consider running track in college? A logical
question to ask about the performance of Heather or Steve is Is that good or bad?
Both of these examples use what is known as a raw score. A raw score is the unadjusted value that has
been measured, whether it is a fourth-graders score on a vocabulary test, a high school seniors time in
the mile run, an infants weight, a retail outlets sales revenue, or an IT consulting firms average survey
score on a five-point scale on the Core Values index. By themselves, raw scores may not be very
useful or informative for anyone other than a knowledgeable specialist. In short, raw scores often
cannot answer the question, Is that good or bad?
Percentiles to the rescue!
One way of addressing the Is that score good or bad? problem is to compare the score for one
organization with the scores attained by other organizations. This is known as norm-referencing, and
it is best known for its use in educational testing. Most parents with school-age children have probably
seen achievement testing results reported in percentile terms, particularly if their children are old enough
to have taken the PSAT, SAT, ACT, or any of a number of different state-administered achievement tests
given in the U.S. For instance, if Angie scores at the 82nd percentile on the Math Concepts section of
the Iowa Tests of Basic Skills this means that she scored higher than 82% of same-grade students in a
normative sample that took the same Math test. For Angies parents, this is a useful thing to know.
Relative to other fourth-graders, Angies progress in mathematics is extremely good. Unless they were
highly knowledgeable about the Math Concepts scale of the Iowa Tests, however, being told that
Angie had a raw score of 19 would not tell them very much about her progress.

The basis for comparisonnorm groups

Reporting test scores or survey results in this manner requires that there be some group that sets the
standard. This group is known as the normative sample, benchmark group, or simply as the
norms and assessment specialists generally take great pains to make sure that this group is large and
representative enough to serve as an acceptable baseline. The Denison Organizational Culture Survey
database, for example, contains a normative sample of 707 companies to which an organizations survey
results are compared. Whenever an organization participates in a new culture survey project, its mean
ratings on the 60 items and 12 indexes are compared to the item and index ratings obtained by the
organizations in this normative group.
How did Denison Consulting develop this benchmark group?
The database was built over seven years of conducting survey projects within organizations. The
database contains raw mean scores (averages of individual employees ratings) of items and indexes for
each organization. Organizations mean values were listed from lowest to highest for each item and
index and a statistical procedure was used to assign percentiles based on where those values fell in the
distribution of means.
The 2004 DOCS norms are based on 707 organizations as rated by over 280,000 total respondents.
Every organization included is rated by at least 25 and as many as 16,000 respondents. On average,
organizations in the benchmark were rated by 405 employees. Regardless of the number of respondents,
however, each individual organization is weighted equally in the norms. In other words, an organization
that surveys 3,000 people does not have a larger influence on the benchmark than an organization that
surveys 300. Organizations in the database were all assessed between 1995 and 2004, with the majority
(60%) surveying after 2000.
How does Denison change survey responses into percentiles?
For simplicity, this example is limited to the Vision index of the Denison Organizational Culture Survey.
Note that the process described here is identical for the other 11 indexes as well as all individual survey
items. Between 1995 and 2004, Denison Consulting collected Vision index ratings from hundreds of
companies and organized them into a database. This database contains average Vision index ratings for
each organization, listed from lowest to highest. A statistical procedure was performed in order to
determine which percentiles were associated with which raw scores. This procedure normalizes the
database, which puts all of the values on a normal curve (see next page).

A Normal Distribution of Raw Score Means for the

Vision Culture Index
50th
20th

80th

Percentile

Percentile

30%

5th

30%

95th

Percentile

Percentile

15%

2.62

15%

2.84

3.06

3.31

Frequency of Means

Percentile

3.56

Raw Score Means

Raw Score Means refers to the original, unadjusted averaged values obtained from survey results. Raw score means are
arranged in this diagram in ascending order; as you move from the left to the right in this distribution, they become increasingly
large.
Frequency of Means refers to how common particular raw score means are in the distribution. The higher the curve at a
particular point, the larger the raw means are under that area. Means near the middle of the range, such as those between 3.30
and 3.60, tend to be most common. On the other hand, means in the two tails, or extreme ends, are much more unusual. Only
about 10% of all organizations have Vision indexes with raw means less than 2.98 or greater than 3.96.
A raw mean of 2.62 percentile is extremely low. It exceeds only about 5% of all raw means. In other words, it is smaller than
about 95% of means.
A raw mean of 2.84 is quite low. It exceeds 20% of all other raw means and is smaller than 80% of all other raw means.
A raw mean of 3.06 is at the 50th percentile, also known as the median. This value splits the distribution in halfit is larger
than about 50% and smaller than about 50%.of all raw means.
A raw mean of 3.31 is at the 80th percentile, which is higher than 80% and smaller than 20% of all raw means.
A raw mean of 3.56 is extremely high. It is higher than 95% and lower than just 5% of all raw means in the distribution.

After the normalization procedure was performed, each raw score mean was associated with a particular
percentile. Using this information, a new raw mean, such as an organizations average on the Vision
index, can be converted into a percentile. At this point, the normative database was ready for use with
new survey projects.

How does this affect my survey project?

Organizations raw means on the Vision index can be converted into percentiles so stakeholders can find
out how well their organization is doing relative to the companies in the normative database.
Suppose for example that two companies, Company A and Company B both participated in culture
survey projects. Each company had a few hundred employees complete the Denison Organizational
Culture Survey. For each company, employees ratings were averaged to create raw means for each item
and index. The raw mean of the ratings provided by Company A employees on the Vision index was
3.60. The raw mean of the ratings provided by Company B employees on this index was somewhat
lower at 3.27. These raw means are just a starting pointby themselves, they do not communicate
much information. Fortunately, because of the normative database it is possible to convert them into
something more useful. The raw means on the Vision index for Company A and Company B can be
compared to raw means on the Vision index obtained by the 707 organizations that contributed to the
normative database. The database includes not just raw means but also the percentiles associated with
those raw means. At this point, it is a simple matter of converting the raw means for these two
companies into percentiles:

Vision Normative Database

Company A

Company B

Raw Score Mean

Percentile

3.96
3.83
3.75
3.70
3.64
3.60
3.56
3.51
3.48
3.44
3.41
3.38
3.35
3.32
3.27
3.20
3.13
3.07
2.98

95
90
85
80
75
70
65
60
55
50
45
40
35
30
25
20
15
10
5

Note that the actual norms have percentiles ranging from 1 to 99.
To save space, this example table only shows every 5th percentile.
These values are meant to serve as an example only and do not actually
represent normative data for the Vision index.

As the table shows, Company As raw mean of 3.60 is at the 70th percentile. This indicates that it is
higher than 70% of raw means obtained by the other organizations in the normative database. Company

A seems to be doing a better job than 70% of the companies in the benchmark group when it comes to
creating a vision for the organization.
On the other hand, Company Bs raw mean of 3.27 is only at the 25th percentile. Company B is in the
bottom quartile when it comes to establishing a clear, shared, long-term vision for the companyits raw
mean is higher than just 25% of the raw means in the normative database.
Company A and Company B both receive survey reports with their percentiles on the Vision index.
Their survey responses started off as raw scores, were averaged into raw mean scores, and now appear
as percentiles. The point of this whole process was to make the culture survey results easier to interpret
to give people a better understanding of their organizations relative strengths and weaknesses.