Professional Documents
Culture Documents
The British Psychological Society 2007. All rights reserved. No part of this publication may be reproduced
or transmitted, in any form or by any means, without permission.
This test has been granted registration as a psychological test by the British Psychological Society,
Psychological Testing Centre.
Permission has been granted to the distributor / publisher named above to distribute copies of this review in
paper or PDF file format so long as such copies are not amended or changed in any way from the original
version published by the BPS.
S. Robertson
& G. Hodgkinson
I. Robertson
P. A. Lindley
P. A. Lindley
S. Waters
2007 BPS
Page 2 of 17
Administration is by means of a selfadministered booklet or, alternatively, via computer, using the
Thomas International Human Resources Software. The latter is a Windowsdriven computer program that
enables users to administer, score and interpret responses to the PPA and HJA.
Data gathered through the paperpencil mode of administration may be scored in one of three ways:
1. by keying responses from the answer sheet directly into a personal computer, via the Human Resources
Software;
2. by scoring the answer sheets manually;
3. via a consultancy service operated by Thomas International, the publisher and distributor (for which there
is an extra charge).
Extensive reportgenerating facilities are available, with a mixture of graphical and narrative information,
and with pointers for feedback/interview discussion across a wide range of issues. Fifteen different types of
profile reports and job comparisons can be generated through the Human Resources Software, when the
PPA is employed in conjunction with the HJA:
1. PPA profile
2. Management audit
3. Sales audit
4. Admin/tech audit
5. How to manage
6. Management interview questionnaire
7. Sales interview questionnaire
8. General questionnaire
9. Strengths and limitations
10. Career guidelines
11. Training needs
12. Executive summary
13. Human job analysis definition
14. Human job analysis comparison
15. Graphs and scores
Each of the above items are charged as separate reports, with the exception of item 15, for which there is
no charge.
Classification
Content Domains:
Personality Trait
Intended or main area(s) of Use:
Work and Occupational
Intended mode of use (conditions under which the instrument was standardised and validated):
Supervised and controlled administration. Test administration under the control of a qualified
administrator or proctor
Description of the populations for which the test is intended:
Working population, but new (experimental) version (J10) intended for school students, and an
experimental new form APNQ are not yet released.
Number of scales and brief description of the variable or variables measured by the test:
The four scales of the PPA are as follows:
1. Dominance
2. Influence
3. Steadiness
4. Compliance
Personal Profile Analysis
2007 BPS
Page 3 of 17
Items format:
Forced choice, mixed scale alternatives (ipsative)
Number of test items:
The Personal Profile Analysis (PPA) is a 24item, self completion, and multiscale instrument, designed for
the assessment of workrelated dispositions. American in origin, the PPA is an integral part of a system
used by line managers in work interviews. A companion instrument, for completion by employers, known as
the Human Job Analysis (HJA), designed to assess jobs along a parallel set of dimensions, is intended for
use in conjunction with the PPA, in order to enable the assessment of personjob fit. This review is
concerned only with the PPA.
Response mode:
Manual operations
Paper and pencil
Computerised
Time:
Preparation: Estimate 5 minutes or less
Administration: Approximately 1015 minutes
Scoring: Computer scoring would take only a few seconds, hand scoring a few minutes, while the bureau
service would depend on speed of communication
Analysis: Computer scoring would take a few seconds, hand scoring a few minutes
Feedback: Approximately 4045 minutes, using either the computer generated graphic report on the
computergenerated interpretative report as a basis
Different forms of the test:
An experimental alternative version is discussed in the manual but is not yet available
2007 BPS
Page 4 of 17
ComputerGenerated Reports
Are computer generated reports available with the instrument?:
Yes
Do distributors offer a service to correct and/or develop computer generated reports?:
Yes
Brief description of Reports..
Media:
Text only
Complexity:
Medium
Report Structure:
Construct based
Criterion based
Sensitivity to context:
Clinicalactuarial:
Modifiability:
Not modifiable
Degree of 'finish':
Publication quality
Transparency:
Directive
Intended recipients:
2007 BPS
Page 5 of 17
This amounts to around 32 + VAT per person (assuming that a questionnaire and report are required for
each respondent). Prices are all subject to volume discounts.
Prices for other bureau services:
These are provided by Thomas Consultants for an additional charge.
Testrelated qualifications required by the supplier of the test:
Test specific accreditation
Professional qualifications required for use of the test:
None
2007 BPS
Page 6 of 17
iii) Standardisation:
iv) Norms:
v) Reliability:
vi) Validity:
Quality of the Procedural instructions provided for the user:
i) For test administration:
ii) For test scoring, norming etc:
iii) For interpretation and reporting:
iv) For providing feedback and debriefing test takers and others:
v) For providing good practice issues on fairness and bias:
vi) Restrictions on use:
vii) References and supporting materials:
Quality of the materials:
i) General quality of test materials (test booklets, answer sheets, test
objects, software, etc):
ii) Test quality of the local adaptation (if the test has been translated and
adapted into the local language):
N/A
iii) Ease with which the test taker can understand the task:
iv) Ease with which responses or answers can be made by the test taker:
v) Quality of the items:
Reviewer's comments on the documentation (comment on rationale, design, test development and
acceptability):
The instructions associated with the questionnaire booklet are clear and its overall appearance is
satisfactory. The instructions associated with the "Human Resources Software " version of the instrument
and the reportgenerating facility are similarly clear and straight forward and the overall quality of the
graphics is satisfactory.
In general the printing quality is good. The basic training manual comes in a selfcontained binder. The
training pack is of reasonable quality. Previously essential technical information was distributed in several
disparate reports. Now there is a new technical resource manual which presumably replaces this.
Some of the background theory and discussion of technical issues regarding ipsativity are likely to be
beyond the grasp of the average user. The issues of ipsativity are dealt with in a way that glosses over the
complexities and implications for establishing the psychometric properties of the instruments and the
Personal Profile Analysis
2007 BPS
Page 7 of 17
Overall adequacy:
i) Appropriateness for local use, whether local or international norms:
N/A
N/A
N/A
No information is supplied
0 stars
2007 BPS
Page 8 of 17
Validity:
Overall adequacy:
Construct Validity (overall adequacy):
i) Designs used:
No information is supplied.
iv) Median and range of the correlations between the test and other
similar tests:
v) Quality of instruments as criteria or markers:
vi) Differential Item Functioning (DIF) analyses:
N/A
No information supplied.
iv) Median and range of the correlations between the test and criteria:
2007 BPS
Page 9 of 17
2007 BPS
Page 10 of 17
While some high correlations are reported between PPA and relevant criteria, or between PPA and other
instruments, they are rendered extremely difficult to interpret due to the strong ipsative character of the
questionnaire and the amount of overlap between the scales.
Intercorrelations, between the PPA scales average at 0.37, slightly higher than the theoretical value for a
fourscale fully ipsative device (0.33). The intercorrelations vary from 0.24 (steadiness and compliance) to
0.75 (dominance and steadiness). Given the strength of the correlation between Dominance and
Steadiness, they might be better thought of as two ends of a bipolar dimensions rather than separate
scales.
In the new Technical Resource book, there is more detailed explanation and comment on the various
studies. As far as predictive validity is concerned, the emphasis is on configural prediction deriving a
profile shape that will predict success in different jobs. Some of the profiles appear to be based on common
sense, a few have been test empirically
Reliability:
Overall adequacy:
i) Data provided about reliability:
No information supplied.
Internal consistency:
i) Sample size:
ii) Median of coefficients:
Test retest stability:
i) Sample size:
ii) Median of coefficients:
Equivalence reliability:
i) Sample size:
ii) Median of coefficients:
2007 BPS
Page 11 of 17
Internal Consistency: The studies referred to earlier were also used to derive some reliability evidence. One
sample of 291 were tested using paper and pencil, and a larger sample of 1512 were tested by computer.
Cronbachs, Alpha was used to compute internal consistency reliability for the 4 scales using the composite
scores. For both samples the results were similar to the earlier study with the Dominance scale being the
most homogeneous, .77, and the C scale being unsatisfactory, .62 and .58.
Alpha coefficients using the same two samples were also calculated for Experimental form, APNQ. The
scoring procedure for this version of the PPA is different from the old DISC. The reliabilities in both samples
ranged from .77 to .86, the P scale being the most internally consistent and the A scale the least.
Test Retest: A study whose purpose was to demonstrate the effectiveness of PPA for career choice in
schools, was reported separately in 2003 (not in the new manual). A sample of 151 students from 2 schools
were given both DISC and a revised version that used short descriptive phrases (this was the experimental
form J10 not the experimental
APNQ). The highest testretest reliabilities were achieved (interval 5 days), when the new version was
presented first. These results (.89, .89, .80 and .78 for DISC) are substantially the same as previous
research in the USA and Europe. The same results are also described as parallel form reliabilities.
Parallel Form reliabilities for PPA and APNQ intercorrelations are reported in the new Technical Resource
book. In my opinion, this new version seems to be more like a different measure than a parallel form. The
domain names are different, descriptive phrases are used as well as words, and the scoring procedure is
different. The scales of APNQ also seem to be more reliable than the original DISC.
The schools study, 2003, (N = 151) also presents parallel form reliability statistics for the standard form and
the J10 version, with correlations greater than .8 on three scales.
10
2007 BPS
Page 12 of 17
employer".
Scores are presented for the 4 scales, DISC, with no explanations.
There is no introduction, and no comment about accuracy or distortion. There are no disclaimers, at the
end it is recommended that the report be used in conjunction with the interview.
It appears to be addressed to the client organisation, being written in the third person, in a directive style
under the following headings:
Self image
Self motivation
Job emphasis
How others see you
Behaviour under pressure
General comments
It is very user friendly, reads well, and at first glance appears to be a useful quick pen portrait of the person
concerned. As it is not at all clear how the comments track back to the profile, it is very hard to judge how
reliable and how valid it is. It could however, be extremely useful if used as suggested as an interview
prompt, with hypotheses to be explored and confirmed or not. For example, 3 alternative suggestions are
given as to why the person concerned might be frustrated and under pressure.
Overall, it is short, succinct and could be a very useful tool in a development situation if time were taken to
explore the hypotheses. It might be very tempting for a manager under pressure to use the report as "fact"
in a recruitment situation.
FINAL EVALUATION
Evaluative report of the test:
The PPA is a relatively brief, easy to administer instrument, designed to assess the suitability of candidates
in a range of employment situations. Used in conjunction with the HJA, a companion instrument for
assessing job characteristics, the training documentation supplied by the publisher and distributor suggests
that PPA provides a basis for retention and selection, training and development, team bonding and
building, releasing people potential, stimulating employees, diagnosing and resolving people problems,
career guidance, empowerment and change and appraisal.
No requirement is placed upon those wishing to train in PPA to meet any of the standards of competence in
occupational testing associated with Level A and B. Little of the underpinning knowledge associated with
these standards is reflected in the PPA System manual. As a consequence those who do train may not
have the knowledge and skill base required to evaluate PPA properly (or even to read the manuals with
understanding), and not to use the instrument in an inappropriate manner. The fact that PPA is
quasiipsative adds an extra layer of complexity for the novice user, in that a part of the reason why, for
example, people who score high on Dominance often score low on Steadiness is because of the structure
of the questionnaire and the correlations its format imposes.
A section in the training documentation on fair recruitment and appraisal methods at work is
underdeveloped, in that the key concepts arising from the relevant legislation, such as direct and indirect
discrimination are not mentioned. Contrary to the assertions of the documentation, the fact that the
instrument is available in a number of different languages does not necessarily provide additional
safeguards in this respect, since fundamental differences in meaning may occur in the process of
translation.
The training manual does not cover in sufficient detail technical issues, such as how the profiles are
generated, limitations of computer narrative reports, or fundamental concepts of psychometrics. Research
Personal Profile Analysis
2007 BPS
Page 13 of 17
evidence concerning the psychometric properties of the instrument is available now in the new manual
which unfortunately still makes few concessions for the average user, who is likely to be a
nonpsychologist. Although this manual is a big improvement over the format of the series of technical
documents, unfortunately it is not particularly user friendly. It is written historically, so one has to jump
around looking for evidence of, say, construct validity, which could be in several different places. Moreover,
there is an abundance of statistics, not really reported in an easily digestible way. The style is rather
verbose. There is some confusion of reporting about DISC and APQN. If the latter is to be the version of
choice, maybe a user friendly technical manual outlining the rationale and supporting statistics for this new
measure alone, would be very welcome.
Questions as to the nature and significance of data derived through ipsative instruments, how such
instruments are best evaluated in terms of their psychometric properties, and whether or not they can
sensibly be used as a basis for selection decisions have been debated at length in recent years (see, for
example, Baron, 1996; Bartram, 1996; Closs, 1996; Johnson et al., 1988; Saville & Wilson, 1991; Cornwell
& Dunlap, 1994). Several of the points raised in the literature on ipsative tests are explicitly acknowledged
in various places throughout the PPA technical documentation. Indeed, the Johnson et al. (1988) reference
is cited in the Technical Handbook. Previous reviewers (Paltiel, 1986; Cook, 1992) have questioned both
the reliability and validity of PPA scales on the basis their ipsativity and the available research.
The validation studies report productmoment correlations between PPA scores and various jobrelated
performance criteria. Construct validity data, relating the PPA scales to a range of other personality
instruments, are also reported, again based on product moment correlations.
The question as to whether or not there are situations in which conventional psychometric analyses can be
performed on ipsative data in this way remains controversial at the present time. It is clear, however, that
the limitations are more likely to prove problematic in cases where the instrument concerned comprises
relatively few scales (Baron, 1996), as in the case of the PPA. Considerable caution, therefore, is required
when interpreting the evidence presented in the technical documentation concerning the psychometric
properties of the PPA. Another problematic area concerns the underlying theoretical rationale of the
instrument, as set out in the documentation. As noted earlier, the instrument has been derived on the basis
of an underlying interactionist theory, attributable ultimately to the work of William Marston. It is Marstons
work which provided the underlying rationale for the adoption of an ipsatised format. Candidates are
required to explicitly identify characteristics that they feel are most and least like them, in an effort to probe
beneath the surface of their personalities. However, there is considerable confusion within the
documentation that was supplied by the publishers for the purposes of this review as to the precise
meaning of the L pattern.
It is evident that there are two fundamentally opposing interpretations of L scores. In the copy of a
document entitled PPA Technical Manual, its authors state:
However, this is not to say that the L pattern measures unconscious aspects of personality or that it
cannot be distorted. It serves primarily when compared with the M pattern as an indicator of the probable
consistency of the individuals usual behaviour. (Hendrickson & Associates Inc., undated, p. 7)
The original technical manual, by contrast, stated: The L pattern measures unconscious aspects of
personality and as such is so extremely difficult to distort as to allow the claim that it cannot be distorted.
(Irvine, 1988, p. 26). Given that the L pattern is clearly of such fundamental significance to the scoring and
interpretation of the PPA, the failure to address this issue is a major concern.
Conclusions:
In summary, the PPA is a relatively brief, easy to administer instrument, designed to assess the suitability
of candidates in a range of employment situations and it is accompanied by userfriendly training materials.
Technically, the PPA appears to be a questionnaire that is sufficiently complex structurally (due to its
quasiipsative nature) to render straightforward interpretation of its internal and external correlations very
difficult. Taking into account the limitations identified in this review, it is strongly recommended that
Personal Profile Analysis
2007 BPS
Page 14 of 17
intending users should proceed with due caution. The combination of its psychometric properties, together
with the lack of conceptual clarity regarding the nature and significance of the L pattern, raise questions, for
both academic researchers and practitioners alike.
Because Irvine states that the PPA is not a "psychometric test of all relevant personality domains", but is
merely "a unique interview form" and have been "evaluated by the strictest of metric methods and scientific
protocols" this somehow suggests that this is acceptable even though there may be a few shortcomings in
technical information and evidence. Having said that, however, Thomas International are to be commended
for the work they have done in recent years on research and revalidation of this instrument.
Recommendations:
Suitable for use in the area(s) of application defined by the distributor, by test users who meet the
distributor's specific qualifications requirements
Notes, references and bibliography:
Anonymous (1996). The Thomas International PPA System. Thomas International Limited.
This is the training manual supplied to all course participants enrolled for training.
Baron, H. (1996). Strength and limitations of ipsative measurement. Journal of Occupational and
Organizational Psychology, 69, 4956.
Bartram, D. (1996). The relationship between ipsatized and normative measures of personality. Journal of
Occupational and Organizational Psychology, 61, 2539.
Closs, S. J. (1996). On the factoring and interpretation of ipsative data. Journal of Occupational and
Organizational Psychology, 61, 4147.
Cook, M. (1992). An evaluation of the DISC/Personal Profile Analysis. Selection and Development Review,
8(3), 36.
Cornwell, J. M. & Dunlap, W. P. (1994). On the questionable soundness of factoring ipsative data: A
response to Saville and Wilson (1991). Journal of Occupational and
Organizational Psychology, 67, 89100.
Hendrickson, T. M. & Associates Inc. PPA Technical Manual. Thomas International
Management Systems Europe Limited
One of the main sources of technical information supplied by the publishers.
Irvine, S. H. (1988). Personal Profile Analysis: Technical Handbook. Ormskirk: Thomas Lyster.
Key reference supplied by the publishers.
IRVINE, S.H., METTAM, D. and SYRAD, 1 (1994) Valid and more valid? Keys to understanding personal
appraisals at work. Current Psychology: Developmental, Learning, Personality, Social, 13 (1),2759.
Johnson, C. E., Wood, R. & Blinkhorn, S. E. (1988). Spuriouser and spuriouser: The use of ipsative
personality tests. Journal of Occupational and Organizational Psychology, 61, 153162.
Marston, W. M. (1920). Reactiontime symptoms of deception. Journal of Experimental Psychology, 3,
7287.
Marston, W. M. (1925). Negativetype reactiontime symptoms of deception.
Psychological Review, 32, 241247.
Marston, W. M. (1927). Motor consciousness as a basis for emotion. Journal of Abnormal and Social
Psychology, 22, 140150.
Personal Profile Analysis
2007 BPS
Page 15 of 17
Three references cited in the publishers documentation as the theoretical basis of the PPA.
Paltiel, L. (1986). Selfappraisal personality inventories. Guidance and Assessment Review, 2(3), 37.
Saville, P. & Wilson, E. (1991). The reliability and validity of normative and ipsative approaches in the
measurement of personality. Journal of Occupational and Organizational Psychology, 64, 193205.
2003
1998
Constructs Measured:
Administration Mode:
Response Mode:
Manual operations
Paper and pencil
Computerised
Instrument Evaluation:
Characteristics
Evaluation
Quality of Documentation
Quality of Materials
N/A
Not recorded
2007 BPS
Page 16 of 17
The British Psychological Society 2007. All rights reserved. No part of this publication may be reproduced
or transmitted, in any form or by any means, without permission.
2007 BPS
Page 17 of 17