Psychological assessment in executive coaching

Dr Elizabeth Allworth Allworth Juniper Organisational Psychologists & Barbara Griffin The University of Sydney

The University of Sydney

Overview

Types of tests and measures used in

coaching Job-person fit model Testing issues Benefits and limitations of testing

The University of Sydney

Case study

Steve, your new client, is the marketing director for a major Australian retail organisation A recent merger has resulted in a narrowing of Steves responsibilities His most recent performance review highlights leadership and planning as areas for development Having been in the role for five years, Steve is considering what he wants to do next He is not getting on well with his new CEO who doesnt seem to listen to him Can psychological assessment help Steve?

The University of Sydney

Types of psychological assessment measures

Psychometric tests Performance measures Job analysis

Psychometric tests Standardised measure of:

Scores compared with a norm group e.g.

ability/aptitude cognitive ability tests personality values interests etc.

other managers Used to profile an individuals preferences, work style, capabilities, needs etc.

The University of Sydney

Performance measures

Ratings of observable behaviours

360-degree feedback Multi-rater feedback Supervisor/manager ratings Upward feedback

Used to establish a base-line

performance level and to monitor/review progress
The University of Sydney

Job analysis
Importance of understanding the coachees
job context in terms of:

Assessment of the individual needs to be

matched with an assessment of the job
The University of Sydney

Job requirements Cultural environment Current and future change Future opportunities

A model for assessment in coaching:

Minnesota Theory of Work Adjustment (Dawis & Lofquist, 1984)
Satisfactory Performance

Requirements for knowledge, skills & abilities (KSAs) JOB Supply of rewards & reinforcers

Individual supplies of KSAs

PERSON Individual needs, values, goals, interests etc.

Tenure

Satisfaction

The University of Sydney

Individual KSAs

Individual supplies of KSAs

JOB

PERSON

The University of Sydney

Coachees KSAs

What do they do well? What skills & knowledge do they have?

work or non-work related; transferrable How do these relate to job requirements? What are the gaps?

Individual supplies of KSAs

The University of Sydney

Cognitive ability tests Measures of:

Provide an indication of:

problem-solving

General cognitive (intellectual) ability: Specific abilities - numerical, verbal and abstract learning potential capacity to manage complexity in problem solving,
decision making etc. speed of thinking and judgment

The University of Sydney

Examples of cognitive ability tests

SHL Management & Graduate Item

Bank (numerical and verbal) ACER Graduate and Managerial Assessment Ravens Standard and Advanced Progressive Matrices Wonderlic Personnel Test

The University of Sydney

Personality tests

Three types of measures:

Broad based measures of personality e.g.
OPQ, CPI, 16PF

Measures of Type e.g. MBTI Measures of the Big Five e.g. NEO-PI-R,
NEO-FFI, Hogan Personality Inventory

The University of Sydney

Broad-based personality measures

Assess a range of attributes e.g.

interpersonal confidence, team style, achievement drive, organisation and planning, resilience, energy, empathy etc

Results should be interpreted against

the requirements of the coachees job

The University of Sydney

Measures of the Big Five

Five factors that account for most of

personality

Conscientiousness Neuroticism (Emotional Stability) Agreeableness Extroversion Openness to Experience

The University of Sydney

The Big Five and job performance

Conscientiousness & Emotional Stability

predictive across most jobs

Extroversion & Agreeableness predict

management and sales

Openness to Experience predicts

training outcomes

The University of Sydney

Individual needs, values, interests

Individual supplies of KSAs

JOB

PERSON Individual needs, values, goals, interests etc.

The University of Sydney

Motivation/Values

What drives the person

Individual needs, values, goals, interests etc.

dynamic work environment, need for security or achievement etc

money, status, advancement etc

How they like to be rewarded Preferred management style

directive vs participative

Preferred work style

structured vs flexible

Match with the organisational culture or values

The University of Sydney

Examples of motivation/values assessments

Work Aspect Preference Scale (WAPS) ACER Occupational Motivation

Questionnaire (OMQ) SHL Motivation Questionnaire (MQ) Gordon Personal and Interpersonal Values

The University of Sydney

Career interests
work:

Individual needs, values, goals, interests etc.

What do you like doing? work and non Helping people Analysing problems/diagnosing Managing, promoting services Handling data/writing reports Using your hands Art

The University of Sydney

Examples of interest measures

Strong Interest Inventory (SII) Self-Directed Search (SDS) Occupational Card Sorts - Centre for

Work Life Counseling SHL Advanced Occupational and Management Interest Inventories Campbell Interest & Skill Survey

The University of Sydney

Linking individual KSAs with job requirements

Requirements for knowledge, skills and abilities (KSAs) JOB Individual supplies of KSAs

PERSON Individual needs, values, goals, interests etc.

The University of Sydney

Requirement for KSAs

What is done on the job? What are the skills, knowledge and

Requirements for knowledge, skills and abilities (KSAs)

attributes that are critical to success? Qualifications? Experience? What are the organisations goals and strategies? - competencies needed?

The University of Sydney

Models and tools for assessing jobs

Interview; observation; diaries; task lists O*Net (Occupational Information

Network) TMS Types of Work Profile SHL Work Profiling System

The University of Sydney

Linking individual needs with job rewards/reinforcers

Requirements for knowledge, skills and abilities (KSAs) JOB Supply of rewards and reinforcers Individual supplies of KSAs

PERSON Individual needs, values, goals, interests etc.

The University of Sydney

Rewards and reinforcement

Money Promotional opportunities Social interaction Being able to do what you like doing Opportunities for learning Opportunities to manage people Security.
The University of Sydney

Supply of rewards and reinforcers

Assessment of organisational/job reinforcers

Self-ratings against same dimensions in

which the individuals needs/reinforcers were assessed Culture surveys,e.g. Human Synergistics OCI Climate surveys e.g. Macquarie University VOICE.

Job performance
Satisfactory Performance Requirements for knowledge, skills & abilities (KSAs) JOB Supply of rewards & reinforcers

Individual supplies of KSAs

PERSON Individual needs, values, goals, interests etc.

The University of Sydney

Behavioural measures
Ratings of behaviour on-the-job, in
simulations e.g. assessment or development centres Self assessment, upward feedback, manager feedback, peer or customer feedback, multirater feedback including 360-degree Provide a baseline against which progress can be measured

The University of Sydney

Examples of behavioural measures Off-the-shelf

TMS Linking Skills Profile MLQ Multi-factor Leadership Questionnaire Human Synergistics Life Styles Inventory (2) and
Management Effectiveness Profile System

Customised

designed to measure specific job-relevant or

target behaviours enable within-person comparisons over time no normative data

The University of Sydney

Minnesota Theory of Work Adjustment

(Dawis & Lofquist, 1984)
Satisfactory Performance

Requirements for knowledge, skills & abilities (KSAs) JOB Supply of rewards & reinforcers

Individual supplies of KSAs

PERSON Individual needs, values, goals, interests etc.

Tenure

Satisfaction

The University of Sydney

Assessing job satisfaction

Subjective ratings along same

dimensions as assessment of individual needs/reinforcers Staff and climate surveys

Conclusion
Psychological tests represent the best, fairest, and most accurate technology available for making many important decisions about individuals National Academy of Science (1982)

The University of Sydney

Limitations of testing

Accessibility requirements for

professional accreditation Time requirements Costs of testing Some tests are culturally biased Validity of tests varies there are good tests and not so good tests

The University of Sydney

Testing traps

Tests that look good are not necessarily

good

Face validity is important for acceptance but does not ensure validity

A good test used inappropriately is no

longer a good test

The University of Sydney

How to select a good test

Based on sound scientific theory Measures relevant job competencies Adequate and appropriate norms

>200 in normative sample Norms are relevant to your candidate pool

The University of Sydney

How to select a good test

Satisfactory reliability

Stability of results over time (test-retest

reliability) Extent items measure the same thing (internal consistency)
> .90 .90 - .80 .70 - .80 < .70 Excellent Good Adequate Limited applicability

The University of Sydney

How to select a good test

Demonstrated validity

The extent to which a test measures what it says it measures (construct validity)
The extent to which the test items are representative of the relevant behaviours or knowledge (content validity) The extent to which performance on the test predicts performance on the job (criterion validity)
> .35 .21 - .35 .11 - .20 < .11 Very beneficial Likely to be useful Depends on circumstances Unlikely to be useful

The University of Sydney

Validity for future performance

Cognitive ability tests Structured behavioural interviews Assessment centres Personality tests Unstructured interviews Academic achievement Age
The University of Sydney

.51-.53 .51 .37 .13-.31 .14 .11 .01

Appropriate use of tests

Accreditation issues
Tests limited to registered psychologists Tests for registered users e.g. MBTI, SHL

Training Assessing factors that may impact on test

performance

Language competence, disability, culture, anxiety Test bias, adverse impact issues

The University of Sydney

Appropriate use of tests

Standardized administration Provide feedback Confidentiality issues

Privacy legislation Clarify who the client is Future use of test results Maintain security of test materials

Benefits of psychometric assessment in coaching

Information: Provide data not easily obtained by other methods Self-awareness: Profile individual across a wide range of personal attributes Comparison: An individuals abilities, preferences, values etc benchmarked against others in a particular norm group e.g. managers, health workers, general population Reliability and validity: Increases confidence in the data used to assist planning of coaching program

The University of Sydney

Benefits of behavioural assessment in coaching

Program planning:
performance

Establish baseline

Monitor and review progress: Periodic

self and other ratings across relevant behaviours

The University of Sydney

Thank you

The University of Sydney