Clinauditchap3 PDF

Examples of clinical audit projects
INTRODUCTION
Examples of clinical audits
This section was compiled from examples of clinical audit projects provided by members of the
FOCUS network and other CAMHS professionals. The examples are not intended as ideal
templates but as a source for learning about clinical audit in practice. The purpose of this chapter
is to promote ideas about topic areas and designs for clinical audit projects in CAMHS. It was
felt that it was important to provide examples of real clinical audits, rather than ideal recipes,
since this enables readers to gain insight into actual benefits experienced, as well as the practical
problems encountered, in conducting a clinical audit project. Ways of preventing/overcoming
these difficulties in future similar projects are also suggested by contributors.
The examples were chosen to reflect a range of topic areas, contexts and professional
disciplines. They have been grouped together under subject areas, such as responsiveness of
services and therapy process. For some of the examples, particular stages of the clinical audit
cycle were not completed. Where this is the case, the stages omitted are shown clearly by the
shaded clinical audit cycle diagram presented on the front page of the example.
Where possible, data collection tools (record forms, questionnaires, etc.) have also been
included. Once again, these are intended to generate ideas for a variety of collection methods.
For use in a different setting, these tools would undoubtedly need to be tailored to the specific
requirements of the service providers undertaking the clinical audit. Any other relevant tools
(check-lists, protocols, template letters, etc.) have been included under Additional resources.
References are cited for most examples, but it would be advisable for anyone considering
replicating the clinical audit project to conduct their own literature search in order to ensure that
more recent articles, guidelines, and research evidence are traced and used to inform the work.
We would like to thank all of those CAMHS professionals who have been prepared to share
their experiences of clinical audit projects with such honesty and enthusiasm in order that others
embarking on a similar process may benefit. Without the help of these individuals this chapter
could not have been written.
29
SECTION A ACCESS
TO SERVICES
EXAMPLE A1: CLINICAL AUDIT OF COMMUNITY SERVICES PROVIDED TO CHILDREN WITH AUTISM
10 Re -audit
1 Select topic
2 Review literature
9 Set/review standards
Number of times around

clinical audit cycle = 1
3 Set standards
8 Set/review
8 Changestandards
practice
= not completed
in this example
4 Design audit
7 Feed back findings
6 Analyse data
5 Collect data
CONTRIBUTORS
Dr V. Chandra Senior Registrar (Audit Project Lead)
Ms S. Smith Clinical Effectiveness and Audit Facilitator
Lifespan Health Care Children Services, Lifespan Health Care
Cambridge NHS Trust, Cambridge
Stage 1 Select topic
BACKGROUND RATIONALE FOR CLINICAL AUDIT

Autistic spectrum disorder consists of a triad of impaired social interaction, communication and
imagination associated with a rigid, repetitive pattern of behaviour. Onset is usually during the
first three years of life and problems can also begin in later childhood. It affects 34 per 10 000
children, 75% of whom require special education.
Early diagnosis and intervention is essential to ensure that families and carers have access
to appropriate services and professional support (Rapin, 1997). Children with autism need support
from professionals from a range of disciplines. It is, therefore, crucial for paediatricians to coordinate
their services to support these children and their families and to identify gaps in the services.
This clinical audit was conducted to examine services provided to pre-school and early school
age children (<7 years) with autism, in Cambridge. The main objective of the clinical audit project
was to ensure that children with autism and their families are being seen and assessed by the
appropriate professionals from health, education and social services.
30
Stage 2 Review literature
The most useful and relevant articles found were: Gillberg (1991), National Autistic Society
(1995), Rapin (1997), Webb et al (1997), Wing (1996), Woodhouse et al (1996).
Stage 3 Set standards
SOURCE OF STANDARDS
These standards were based on local guidelines.
STANDARDS SET
Prior to diagnosis
1. 100% of children with suspected autism will be referred to the Child

Development Centre by the general practitioner and be seen by a
paediatrician before the age of three years standard not applicable to
those cases referred which have no record of age in the case notes.
2. 100% of children with suspected autism will have their head measured
by the paediatrician at their first assessment appointment.
3. 100% of children with suspected autism will have a full developmental
assessment by the clinical psychologist.
4. 100% of children with suspected autism will be assessed by the speech
and language therapist.
After diagnosis
5. 100% of children with autism will be referred to the Communication

Disorder Clinic.
7. 100% of children with autism will be assessed by the pre-school learning

support teacher soon after diagnosis.
Stages 4 and 5 Design audit/collect data
SAMPLE
The sample comprised children referred to the Child Development Centre and Communication
Disorder Clinic between January 1993 and mid-June 1997 (n=30).
DATA COLLECTION
6. For 100% of children with autism, the health visitor or social worker will
visit their parents at home to share information (e.g. benefits, respite,
schooling etc.).
The case notes of these children (from the Child Development Centre, Communication Disorder
Clinic, Addenbrookes Hospital, the community child health service and any psychologists
consulted) were retrospectively audited by a senior registrar using a data collection form (see
Data collection tool).
Stage 6 Analyse data
The completed forms were analysed by the audit department. Totals and percentages were
calculated in order to establish overall achievement of each of the standards set.
KEY FINDINGS
Standards 3 and 4 were achieved fully.
The remaining standards were not met achievement was poorest for standards 1 and 2.
31
Stage 7 Feed back findings
The findings of the project were presented by the clinician responsible for the clinical audit
project at a multi-disciplinary meeting of staff providing services to children with autism. At this
meeting, suggestions for an action plan were proposed and discussed.
Stage 8 Change practice
The following changes were suggested and used to develop an action plan.
Development of a system which would enable easier identification of children with autism
soon after diagnosis.
Development of a system which would facilitate communication between the Child

Development Centre and the Communication Disorder Clinic, particularly regarding the
position on the waiting list of those children to be seen.
Identification of criteria for routine case conferences.
Examination of the feasibility of health visitors using the CHAT check-list (which examines
a number of aspects of the childs development e.g. social skills, imaginative play, handeye coordination etc.) on children with language delay before referring them to the speech
and language therapist and to the Hearing Assessment Clinic.
Stages 9 and 10 Review standards/re

-audit
standards/re-audit
A date for re-audit was not decided.
COMMENTS ON THE CLINICAL AUDIT PROCESS

Resources
Total staff time required to complete the audit was approximately

6075 hours.
Additional assistance was required from an audit facilitator.
Apart from staff time, there were no additional costs involved in this
clinical audit project.
Additional points
Notes were often difficult to obtain because they were being used for
clinics.
Poor communication among professionals made the audit difficult.
Data were often not documented in the notes
Looking at five sets of notes for each child at different locations was
time-consuming.
Hints from contributors
32
If you are attempting a similar clinical audit project, talk to a range of

other professionals to obtain the information you require, as some
data may not be documented in the notes.
DATA
COLLECTION TOOL
A1
AUTISM AUDIT CHECK-LIST
1. Name
2.
M/F
3. No.
4.
DoB
5. Referred by:
General practitioner
Health visitor
Speech therapist
Psychologist
Psychiatrist
Registrar/CMO/SCMO
Teacher
Other
6. Age at referral:
years
months
7. Date of referral:
8. Date first seen at CDC:
10. Date first seen by CROFT/Communication Disorder Clinic:
11. What information was sent from CDC to CROFT/Communication Disorder Clinic?
12. Letter from CROFT to CDC:
Y/N
9. Date of referral to CROFT/Communication Disorder Clinic:
13. Date of letter:
Comments
contd ...
33
DATA
COLLECTION TOOL
A1
AUTISM AUDIT CHECK-LIST (CONTD)
Examination
14. Head circumference:
15.
16. Neurocutaneous marker:
Centile:
Y/N
17. Does the child have any associated medical conditions?

Tuberous sclerosis
Y/N
Neurofibromatosis
Y/N
Epilepsy
Y/N
Hydrocephalus
Y/N
Downs syndrome
Y/N
Williams syndrome
Y/N
Retts syndrome
Y/N
Fragile X
Y/N
Other
Investigations
18. Were any of the following investigations performed?
Chromosomes
Y/N
Results filed in notes?
Y/N
Fragile X
Y/N
Y/N
Metabolic screen blood
Y/N
Y/N
Metabolic screen urine
Y/N
Y/N
CT brain
Y/N
Y/N
MRI brain
Y/N
Y/N
EEG
Y/N
Y/N
T4
Y/N
Y/N
TSH
Y/N
Y/N
Woods lamp examination
Y/N
Y/N
Comments
contd ...
34
DATA
COLLECTION TOOL
A1
AUTISM AUDIT CHECK-LIST (CONTD)
Assessment/therapy
19. Was the child seen by a psychologist?
Y/N
20. Was a full developmental assessment done by the clinical psychologist?
Y/N
21. Does the child attend therapy sessions with the psychologist?
Y/N
22. Was the child assessed by a speech and language therapist?
Y/N
23. Does the child attend speech and language therapy?
Y/N
24. Was the child referred to the Communication Disorder Clinic?
Y/N
25. Do the child and family have input from the health visitor?
Y/N
Comments
Social work
er input
worker
Y/N
27. Was the child seen by the childrens disability team?
Y/N
28. Does the family get Disability Living Allowance?
Y/N
29. Does the family get any respite?
Y/N
30. Have the parents been introduced to the Autistic Society?
Y/N
31. Have the parents been given booklets/information on autism?
Y/N
Comments
Education
32. Has the child been assessed by the pre-school learning support teacher?
Y/N
33. Has the child been seen by an educational psychologist?
Y/N
34. Has the child been statemented?
Y/N
35. Does the child attend mainstream school?
Y/N
36. Does the child attend special class in mainstream school?
Y/N
37. Does the child get support from learning support assistants?
Y/N
38. Does the child attend special school?
Y/N
39. Does the child attend assessment centre?
Y/N
40. Was a case conference held?
Y/N
26. Was the family seen by a social worker?
Comments
35
EXAMPLE A2: CLINICAL AUDIT OF ACCIDENT AND EMERGENCY PRESENTATION AND PAEDIATRIC
ADMISSION OF CHILDREN WHO MISUSE DRUGS AND ALCOHOL
10 Re -audit
1 Select topic
2 Review literature

3 Set standards
8 Set/review
8 Changestandards
practice
= not completed
in this example
4 Design audit
6 Analyse data
5 Collect data
CONTRIBUTORS
Dr F. Subotsky Consultant Psychiatrist
Dr A. Santhouse Registrar
Belgrave Department of Child and Adolescent Psychiatry,
Kings College Hospital, Maudsley NHS Trust, London

In the UK, drug and alcohol use and misuse has increased among young teenagers (Miller &
Plant, 1996; NHS Health Advisory Service, 1996; Coleman, 1997). In the USA substance misuse
(alcohol or drugs) has been reported as commonly associated in teenagers with presentations in
accident and emergency (A&E) departments or trauma admissions (Felter et al, 1987; Hicks et al,
1990; Loiselle et al, 1993; Bates et al, 1995; Mannenbach et al, 1997; Spain et al, 1997). However,
in younger age groups this has been under-evaluated. Recommendations have been made to
use screening more, and subsequently make referrals to appropriate services (Maio et al, 1994;
Buchfuhrer & Radecki, 1996). A recent British inner-city casualty audit (Robson, 1997) revealed that
about 200 people aged under 17 years were brought in under the influence of alcohol in one year.
Anecdotally, the inner-city casualty department of the hospital in which the child psychiatry
department is located reported an increase of children attending in intoxicated states. Very few,
however, had ever been referred to child psychiatry.
36
The aim of the audit was to attempt:
to establish the incidence of casualty presentations and admissions related to drug and
alcohol misuse; and
to examine whether young people misusing drugs and alcohol were being referred
appropriately by staff for further psychosocial intervention.
Potentially relevant material was identified and reviewed from Medline searches, the contributors
own files and discussions with addiction consultants. Twelve of the most useful references are:
Felter et al (1987), Hicks et al (1990), Loiselle et al (1993), Maio et al (1994), Bates et al (1995),
Buchfuhrer & Radecki (1996), Miller & Plant (1996), NHS Health Advisory Service (1996), Coleman
(1997), Mannenbach (1997), Robson (1997), Spain et al (1997).
SOURCES OF STANDARDS
Local discussions with A&E consultants and paediatricians, social services and child mental
health staff were used to inform areas for data collection, as described below.
STANDARDS SET
For this audit project no specific standards were set. Information was collected on:
whether reference to drugs or alcohol was present or absent in the casualty notes and
whether this was positive or negative
2.
whether in the admission case notes of children admitted with alcohol intoxication there
had been referral to social work, child psychiatry or another appropriate agency.
SAMPLE
Sample one Casualty notes of one weeks attendance at A&E, of 1115 year olds (n= 73).
Sample two Case notes of one years admissions of under-16s with alcohol intoxication
identifiable through the hospital PAS system (n=10).

DATA COLLECTION
1.
Data were collected retrospectively from casualty notes and case notes.
The number of young people presenting at A&E over a one-week period where there was
reference to drugs or alcohol in the notes was recorded.
The number of young people admitted for alcohol intoxication in one year was recorded.
The number of young people admitted for alcohol intoxication who were referred on to
social services and/or child psychiatry was recorded.
KEY FINDINGS
In the one-week sample of A&E presentations (n=73) there was no reference to drugs or
alcohol in any of the cases.
37
Of the cases (n=10) identified from the PAS, the notes were unavailable for one, there
were definite referrals in four, possible referrals in two and no referrals in three cases.
The findings were fed back to and discussed further with the alcohol audit group, the adult
addiction services, the main child mental health services, the paediatric department and the
hospital social work service.
For A&E staff, a protocol for dealing with possibly intoxicated children was agreed (see
opposite).
An information leaflet outlining local and national sources of help for young people with
drug and alcohol problems was developed.
It was recommended to paediatricians that all children admitted with intoxication should
be referred to the child psychiatric service.

-audit
standards/re-audit
It was decided that a re-audit should be undertaken annually.

Resources
The following time was required for data collection: sample one, 46 hours;
sample two, 3 hours.
Additional assistance was required from the casualty nurse and the hospital
information and audit departments.
Additional costs included the opportunity costs of prior planning and later
discussion time.
Additional points
The lack of identification suggested poor recording in other departments.

The team tried to help remedy this by discussing ethical issues, such as
confidentiality and consent, and by clarifying pathways for action.
HINTS FROM CONTRIBUTORS

Motivation of other departments is crucial. Hospital information systems vary enormously, as
do casualty systems, but should be utilised to identify the key population before looking at case
notes.
38
ADDITIONAL RESOURCE A2
DRUG
DRAFT PRO
TOCOL FOR ACCIDENT AND EMERGENCY
ROT
AND ALCOHOL MISUSE IN UNDER-16S AND CHILD PSY
CHIA
TR
Y
SYCHIA
CHIATR
TRY
Presentation of possibly intoxicated under

-16s
under-16s
Get history of injury.
Always ask about and record contribution of drink or drugs.
Note any signs of intoxication. Investigate as appropriate for medical management needs.
Attempt to get hold of parent: tell parent of situation and record response.
When should child psychiatry be contacted?
Child psychiatry should be contacted if there is a possibility of self-harm or serious

mental illness.
Phone 3219 during weekdays 9 a.m. to 5 p.m. Otherwise, air-call the duty child psychiatry
senior registrar/specialist registrar through the Maudsley switchboard.
When should a child be admitted?
Serious intoxication demands paediatric admission and subsequent psychiatric referral.
Encouraging request for help
Give parent appropriate leaflet regarding seeking help for problem (to be supplied).
If the parent is not contactable
Consider admission and/or referral to social services.
Afterwards
Ensure full record.
The GP should always be notified by letter, including a reference to the intoxication.
39
SECTION B RESPONSIVENESS
OF SERVICES
EXAMPLE B1: CLINICAL AUDIT OF RESPONSE TIMES IN A CHILD AND ADOLESCENT MENTAL HEALTH CLINIC
10 Re -audit
1 Select topic
2 Review literature

3 Set standards
8 Set/review
8 Changestandards
practice
= not completed
in this example
4 Design audit
6 Analyse data
5 Collect data
CONTRIBUTOR
Dr J. Roberts Consultant Psychiatrist
Hornsey Rise Child and Family Consultation Service,
Camden and Islington Community Health Services NHS Trust,
London

This was the fourth time that Hornsey Rise Child and Family Consultation Service audited the
length of time professionals take to make contact with both those who are referred and with the
referrers. The audit takes place annually over the first three months of the year.
Responsiveness to referrers was viewed as extremely important by team members. In
meetings held with referrers, the need to be kept updated about cases was emphasised and the
importance of responsiveness was reinforced.
No literature was reviewed at this stage, although a useful article by Mutale (1995) was found
after the study was undertaken.
40
SOURCE OF STANDARDS
The standards were developed by the team based on their clinical experience and views about
what would be manageable, realistic and acceptable.
STANDARDS SET
No specific targets were set for the following standards, but the assumption was that they
should be met 100% of the time.
1.
Responses to referrers will be made either by phone or by letter within two weeks of the
referral being received.
2.
Responses to those referred will be made by letter within two weeks of the referral being
received this standard did not apply to cases when families have moved out of the area
or have been transferred, or cases when the referrer requests a consultation, rather than
the family/child being seen directly.
3.
The client(s) or professional will be seen by the department within six weeks of the
referral being received this standard did not apply to cases when the families have
moved out of the area or have been transferred.
4.
A summary/letter will be sent to the referrer GP (whether or not the GP was the referrer)
within eight weeks of the referral being received this standard did not apply to cases
when permission to contact the GP was denied by the client.
SAMPLE
DATA COLLECTION
All professionals in the clinic, including trainees, completed forms (see Data collection tool B1)
in which were entered the names of those referred and the referrer (both name and agency).
Agreement was established with all staff involved in the audit that responses made by telephone
would be documented in the case notes. All of the information was therefore collected from the
case notes using a data collection form. It should be noted that psychiatrists do not formally
need to seek permission to contact the GPs of those who are referred. All other professionals
cannot contact the GP unless they have such permission.
All referrals to the service were audited over the three months of January, February and March
1997 (n=93).
The percentage of cases meeting each standard, not meeting each standard and not applicable
was calculated. For those not meeting a standard, percentages were calculated in weekly bands
(e.g. for standard 1: less than three weeks, less than four weeks, greater than four weeks). This
enabled staff to know how late the response time was, on average.
KEY FINDINGS (COMPARING RESULTS OF CLINICAL AUDIT 1997 WITH CLINICAL AUDIT 1996)
The responsiveness of the service to referrers had marginally increased since the previous
audit, with 84% being contacted within two weeks of the referral.
The responsiveness to those referred had decreased almost entirely accounted for by
the higher non-applicable figure.
The percentage of those referred being seen within the target time had decreased slightly, a
reflection of increased pressure on staff when fewer professionals were working at the clinic.
41
There was a significant decrease in the percentage of cases for which a summary/letter
was sent to the referrer/GP within eight weeks the 1997 figure was skewed by a larger
percentage of cases classified as non-applicable.
Non-health professionals (e.g. social workers) and new members of staff (e.g. registrars)
were found to fail the set standards the most often.
A report of the findings was written and circulated to all of the relevant people including members
of the team, managers, lead clinicians, the local education authority, and the medical director.
For standard 4, the non-applicable figure was found to be 33%. It was felt that this was
possibly artificially high, in that the family may not have attended, and a letter back to the
referrer saying that they had not attended was perhaps not counted as a summary.
Further discussion of this matter within the team was suggested.
The data collection forms were modified to make analysis easier (e.g. lines were inserted).
Ways of improving the number of new staff achieving the standards were incorporated
into practice, for example they were explicitly informed about the need to send
summaries to GPs.
Stage 9 Review standards
Standards have been modified so that they can be operationalised more easily and consistently
(e.g. how to count days (chronological or working) defining non-applicable cases).
Stage 10 Re -audit
It was decided that this clinical audit should be conducted annually. However, the Trust changed
its data collection system, which meant that the clinical audit could not be repeated using the
same design the following year.
Resources
The only costs entailed were staff time total staff time was approximately
23 hours.
The data collection forms had to be completed by the clinicians each month
and the data were analysed by the clinical audit lead.
Additional points
The audits of responsiveness have become more specific and focused over
time.
Collecting the forms from people can take some time.
The clinical audit project has led to improved practice.
The clinical audit project has proved to be politically very useful, for example
in providing evidence that the service is accessible (report given to local health
authority).
HINTS FROM CONTRIBUTOR

The contributor would endorse the usefulness and importance of auditing responsiveness to
referrers.
42
DATA
COLLECTION TOOL
RESPONSE
B1
TIME CHAR
T
CHART
Name of worker:
Month:
Referrer
(name or agency)
Target = 2 weeks
Target = 6 weeks
Target = 8 weeks
Time for contact

to be made with
referred
Time for contact

to be made with
referrer
Time before
appointment
offered to family
individual or
professional
Summary to
referrer or GP
43
Name of referred
Target = 2 weeks
EXAMPLE B2: CLINICAL AUDIT OF COMMUNICATION BETWEEN CHILD MENTAL

HEALTH PROFESSIONALS AND GENERAL PRACTITIONERS
10 Re -audit
1 Select topic
2 Review literature

3 Set standards
8 Set/review
8 Changestandards
practice
4 Design audit
6 Analyse data
5 Collect data
CONTRIBUTOR
Mrs R. Harris Consultant Clinical Psychologist
The Tavistock Clinic, Tavistock and Portman NHS Trust,
London

One of the most often cited requirements of GPs is the need for improvements in communication
following a referral to more specialist services. Effective communication is critical to good client
management (Kentish, 1987), yet most studies conclude that such correspondence is woefully
inadequate (McGlade, 1988; Penney, 1988; Westerman et al, 1990).
In a recent survey of GPs views of the institution, feedback suggested that staff were often
perceived as being poor communicators, in terms of the speed, frequency and content of
individuals responses to their referrals. Whether or not this assertion was justified, it required
attention. Not only would more effective communication improve patient care, but also GPs will
increasingly be the purchasers and gate-keepers of services. This audit project was designed to
explore and attempt to address this perceived problem.
44
A literature search and review was conducted when selecting the topic, and for setting the
standards. The following references were selected as being useful and relevant: Hoffman (1981),
Pullen & Yellowless (1985), Kentish (1987), McGlade (1988), Penney (1988), Westerman et al
(1990), Parry (1992), Cornwall (1993), Firth-Cozens (1993), Jones & Jordan (1993), Campbell et al
(1994), Cookson & Fuller (1995), Duff (1995).
SOURCE OF STANDARDS
Standards were developed from the following sources: review of the literature, discussions
with local GPs, and informal and formal discussions with staff within the organisation. A meeting
was also held with the head of contracting, to ensure congruence of standards with contracts.
STANDARDS SET
Standards were set for the frequency of written communications to GPs and the format/content
of these (see Data collection tool B2 for the list of standards). Of the standards agreed, those
outlined in the first section labelled frequency were considered to be particularly important.
These were written into the majority of peoples contracts as quality standards. A target of
100% was therefore set for these standards. The subsequent sections were seen as a model of
good practice; although ultimately a target of 100% may be hoped for, 80% was considered to
be an appropriate target in conducting this clinical audit project.
CLINICAL AUDIT A
Sample 1
All new cases from 1 April 1997 to 15 May 1997 were included in the
sample (n= 44).
Sample 2 (re-audit)
All new cases from 16 May 1997 to 30 June 1997 (n=43).
Following the initial process of consultation with GPs and staff, it was agreed that the clinical
audit would begin in the new financial year. At this point the communication criteria agreed
were in place in the department. Given that one of the standards for good practice was that
GPs/referrers would be responded to within two weeks of completing the assessment
(maximum of six sessions) or three months from referral (whichever was sooner), all new
cases from 1 April 1997 to 15 May 1997 were audited. A number of changes to practice
were made as a result of the first clinical audit, and following these changes the standards
were re-audited (Sample 2).
The design of this clinical audit project is presented diagramatically in Figure 3.1.
CLINICAL AUDIT B
Sample
20 randomly selected case notes (every other new referral received) of

new referrals from 1 April 1997.
Since the standards set were of communications to GPs/referrers up to and including closure of
cases, a longitudinal audit was also undertaken and audited against the standards at threemonthly intervals, that is six weeks after the standards were in place in the department (July and
October 1997).
DATA COLLECTION
The data were collected from the case notes, using the audit tool, which consisted of a checklist with met, not met and not-applicable (see Data collection tool B2).
45
SELECTING THE T
OPIC
TOPIC
R
eview literature on audit
Review
Assess current needs and concerns of the organisation
Discuss conclusions with staff
SETTING THE ST
AND
ARDS
STAND
ANDARDS
Literature search
Meeting with GP
s
GPs
Discuss within formal and informal structures within the
organisation (e.g
(e.g.. staff meetings)
Meeting with head of contracting to ensure congruence

of standards with contracts
CLINICAL A
UDIT A
AUDIT
CLINICAL A
UDIT B
AUDIT
All new cases from 1 April to

15 May
20 randomly chosen new

referrals received in April 1997
FFeed
eed findings back to staff group
agree to changes in practice
R
e -auditing of the same case
Re
notes in July 1997
Second clinical audit all new

cases from 16 May to end June
R
e -auditing of the same case
Re
notes in October 1997
FIG. 3.1 The clinical audit process for example B2
46
The proportion of times that the standards were met/not met was calculated, presented as
percentages and compared to the standards set. Some of the standards were not applicable at
the time the clinical audit project was conducted and the percentage of case notes for which
this was the situation was also recorded. A standard was considered not to be applicable to a
set of case notes when the data were not yet available, rather than a standard not being met.
The data were then factored up and percentages were also calculated on these case notes alone.
KEY FINDINGS
Clinical audit A first audit (sample 1)
GP details were found to be absent in a surprising number

of cases (see Table 3.1)
Routine information required by GPs was sometimes found
to be missing (see Table 3.1)
Clinical audit A re-audit (sample 2)
Overall performance on the standards improved considerably

in the second audit, which showed improvements in the
number of cases in which GP details were known and the
percentage of cases in which standards were met for criteria
listed under Frequency. The first four standards, focusing
on the format of communications with GPs after initial
assessment, were met completely.
Clinical audit B
Performance improved to within the targets on all items

regarding the format and content of initial letters.
Follow-up letters were also good, but there were some areas
of missing information (e.g. number of sessions seen and
name of referrer being recorded).
The results of the clinical audit projects were fed back to the teams formally and informally.
Each team was alerted to their particular set of case notes. A written report of the audits was
circulated to the relevant people.
Closure letters were also of a high standard with some room

for improvement, for example they should include some
indication of the outcome of the work, and about the process
of re-referral.
Changes implemented as a result of clinical audit A, sample 1 are shown in Table 3.1.
TABLE 3.1 Key findings from clinical audit A, sample 1, and changes in practice implemented
KEY FINDINGS
CHANGES IN PRACTICE
GP details were found to be absent in a surprising

number of cases
It was decided that no referrals would be accepted

by the department without details of the GP
GP,, even if
permission to contact the GP was denied
Routine infomation required by GP

GPss was sometimes
found to be missing
A template, which automatically incorporated certain

information into any written correspondence with the
GP in relation to a particular case, was devised for
use with each set of case notes
47

-audit
standards/re-audit
A standard that was poorly met was informing GPs of other agency involvement in a case.
It was decided that this should be the focus of the next round of internal consultations
that will take place in the department.
The time scale within which letters should be sent out, according to the standards, was
felt to be unrealistic for cases when the patients first appointment had been delayed. It
was agreed that this time limit should apply from the date of the first appointment attended
by the patient.
Services offered by clinicians at GP practices/health centres were difficult to audit since

the GP tended to know about the appointment prior to the clinician involved. It was agreed
that a more sensitive way of looking at these groups should be developed.
It was decided that some of the standards needed reassessing. Some were too vague or
could not be operationalised meaningfully, e.g. GPs informed of the action they should
take was difficult to audit, and expectations of GPs were difficult to define and assess.
It was decided that another clinical audit of this nature would be done in six months time.

Resources
To complete the two clinical audit projects took approximately 100 hours.
This figure includes the time required for initial meetings with GP and staff.
A clinical audit assistant collected the data.
Additional costs (excluding staff time) included approximately 30 for printing
the reports and template letters and putting the standards at the front of
each case file.
Additional points
The clinical audit process was useful in both examining and improving practice.
In carrying out this clinical audit project, unexpected and very important
findings emerged, for example the lack of information the department had
about who many of the GPs were for the children and families referred. What
might appear to be a fairly simple change in administrative practice, such as
not taking referrals without GP details, soon changed this.
Some of the other changes that took place were also superficially simple.
For example, a template for all written communication about any given case,
for example, was developed. The joint decision-making process between
staff, giving a mandate to those who implemented the changes, however,
had not been achieved easily over similar issues in the past. The cyclical
process of conducting a clinical audit project of this nature may have been
helpful in this respect.
One standard that was poorly met was informing GPs of other agencies
involved. This was a key point raised by GPs in discussions. They are the
primary providers of care to their patients and take their responsibility for
overall care very seriously. This is particularly true for children and young
people, where the GP may be the main or only person monitoring their wellbeing. Where there are concerns in relation to child protection, GPs need to
know. The department seemed to have failed them in one of the areas they
were keen to highlight. This will be the focus of the next round of internal
consultations that will take place in the department.
It would have been better to have had a longer gap between sample 1 and 2
(in clinical audit A) ideally about three months. It would also have been
useful to have piloted the definitions for the criteria to ensure that they could
be meaningfully operationalised.
HINT FROM CONTRIBUTOR

Make sure that you fully understand the purpose of clinical audit and define your standards clearly.
48
DATA COLLECTION TOOL B2

GP/REFERRER
COMMUNICA
TION ST
AND
ARDS CHECK-LIST
COMMUNICATION
STAND
ANDARDS
STAND
ARD
ANDARD
M ET
NOT
MET
N/A
1. Frequency (target 100%)
Once appointment is offered, GP is informed

(unless permission is denied).
An initial detailed response is made to the referrer

within two weeks of completing the assessment
(maximum six sessions) or three months from the
first appointment (whichever is sooner).
Follow-up letters are sent at termly intervals (or

whenever significant changes occur or new
information is obtained).
A letter containing a closing summary is sent to

the referrer within four weeks of case closure.
2. Format/content (target 80%)
The name of the family.
The name of the referred patient.
The date of birth of the referred patient.
The date of the original referral.
The name of the referrer.
The number of sessions seen to date.
An outline of the main problems and current issues

the GP should be aware of these could be done
as bullet points.
A professional opinion about the case and how this

was presented to the clients, including any advice
given.
An outline of expectations of or action GP has to

take (e.g. to monitor, provide information etc.)
The likely duration of treatment, if known.
The date of the next session, or any other

arrangement made.
The name of any other agency involved, where

possible, and request relevant information to be
sent.
(a) Communications after assessment or treatment

has begun all communications are written in
letter-form and will include:
contd ...
49
DATA COLLECTION TOOL B2

GP/REFERRER
COMMUNICA
TION ST
AND
ARDS CHECK-LIST (CONTD)
COMMUNICATION
STAND
ANDARDS
S TAND
ARD
ANDARD
2. Format/content (target 80%) (contd ...)
(b) Follow-up letter will include:
The date of the original referral
An outline of the main problems and current issues
the GP should be aware of these could be done as
bullet points.
A professional opinion about the case and how this
was presented to the clients, including advice given.
An outline of expectations of or action GP has to
take (e.g. to monitor, provide information etc.).
The likely duration of treatment, if known.
The date of the next session, or any other
arrangement made.
The name of any other agency involved, where
possible, and ask for relevant information to be sent.
An update of current issues, noting when the last
correspondence happened.
(c) Closure letters all communications are written
in letter-form and will include:
The date of the original referral.
A summary of the current positions and concerns.
An indication of the outcome of treatment.
A description of any ongoing or potential concerns
and, if relevant, possible action that could be taken.
A statement about the availability of re-referral and
the process for doing so.
The name of any other agency involved.
50
M ET
NOT
MET
N/A
SECTION C ATTRITION
RA
TES
RATES
EXAMPLE C1: CLINICAL AUDIT OF NON-ATTENDANCE AT INITIAL APPOINTMENTS IN A CHILD AND FAMILY
PSYCHIATRY OUT-PATIENT CLINIC
10 Re -audit
1 Select topic
2 Review literature

3 Set standards
8 Set/review
8 Changestandards
practice
= not completed
in this example
4 Design audit
5 Collect data
CONTRIBUTOR
Dr J. Green Senior Lecturer and Honorary Consultant
Department of Child and Family Psychiatry,
Booth Hall Childrens Hospital,
Manchester Childrens Hospitals NHS Trust, Manchester
6 Analyse data

The rate of initial clinic non-attendance in child psychiatry is probably not substantially different
from that in other specialities. However, the impact of non-attendance on clinic functioning and
professional time is considerable because of the time taken over initial assessment (1.52 hours).
This is important from the point of view of professional training, student teaching and efficient
use of resources.
Lloyd et al (1993) examined adult patients initial appointments at ear, nose and throat and
gastro-enterology clinics, and factors related to non-attendance. Their conclusions suggested
that patient attendance was more likely to occur if they had been able to discuss their problem
in detail prior to the referral.
Farid-Basem (1993) identified that with a non-attendance rate of 22%, the quality of referral
letters for non-attenders was significantly worse than for the attenders.
51
Initiatives had been made to reduce initial non-attendance rates at Booth Hall (e.g. social
worker home visit, confirmation of appointment request, etc.) but with uncertain success, since
there were no baseline data with which to work. The proposal for this study was to compare
initial non-attenders with attenders, with all the associated variables of personal and demographic
characteristics. The aim of the study was to indicate ways of improving attendance rates and to
assess the effectiveness of the initiative already implemented.
A literature search was conducted although, at the time when this clinical audit project was
undertaken, only four relevant articles were found: Thapar & Ghosh (1991), Baggaley (1993),
Farid-Basem (1993), Lloyd et al (1993).
STANDARDS SET
No standards were set for the initial clinical audit since the aim was to collect baseline information
to facilitate the setting of standards and to identify areas of practice which required changing.
SAMPLE
The sample consisted of all new patients offered a first appointment at the clinic from April
1991 to 1992 (n=430).
DATA COLLECTION
Information was collected retrospectively from:
the Patient Administration System (PAS) on:
day, date and time of appointment
patients gender, age, name and address
patients GP and district of residence
the new patient referral logbook (kept by the secretary) on:
name and profession of referrer
referring problem and provisional diagnosis; and
the appointment diary (kept by receptionists) on:
whether the patient had confirmed their appointment
whether a home visit by the social worker had been requested by the consultant.
The data were analysed using non-parametric statistics on SPSS.

KEY FINDINGS
Patients who were given appointments within one month of referral attended. The longer a
patient waited for an appointment the more likely he/she was not to attend. It was apparent that
the major reason for non-attendance was the length of time the patient had waited for an appointment.
52
Results of the project were fed back to the Trust Audit Committee, and were described at a
meeting for purchasers of mental health services in Manchester. A publication of the clinical
audit project was planned.
The two main recommendations were made as a result of the clinical audit project findings.
To hold party clinics whereby 10 patients are given a joint appointment to attend one
afternoon for a session with a brief meeting with members of the child and adolescent
psychiatry team. Patients are then given a proper appointment with the consultant and
team as a follow-up appointment. This should enable more patients to be seen earlier, and
help to allay fears and apprehensions about coming to the department, before their first
proper appointment.
Greater input from the senior registrar in new patient clinics.

-audit
standards/re-audit
Non-attendance was re-audited for new referrals from April 1992 to April 1993 using the original
methodology. Standards for waiting times were set in the clinics quality standards as a result of
the first audit, and used for the re-audit. The re-audit showed:
an increase in referrals, but a reduction in waiting time
a decrease in the non-attendance rate.
Resources
It is difficult to estimate the total staff time involved in this project. An audit
assistant was employed by the Trust and worked on the project half-time
over several months. Designing and writing-up the project took considerable
time (approximately 20 hours of the consultant psychiatrists time).
Administrative staff within the unit spent approximately 10 hours helping the
audit assistant to collect the clinical information.
Statistical assistance was gained from a local statistician.
There were no other costs involved in this project.
Additional points
This completed clinical audit cycle confirmed the importance of waiting time
as the prime variable influencing non-attendance and showed the
effectiveness of a number of changes in clinical practice. These changes
succeeded in decreasing waiting time (in spite of an increasing number of
referrals), using the same staff, and produced a corresponding and predicted
decrease in initial non-attendance rates with all the attendant savings in
efficiency.
The use of party clinics as brief initial contact clinics appeared to be one of
the major factors improving success. The social workers home visit did not
substitute for the brief assessment contact.
The design of the audit successfully addressed the objectives of the project.
The use of an audit assistant is extremely advantageous.
Statistical advice should be sought at the beginning of the design of a clinical audit
project.
53
EXAMPLE C2: RE-AUDIT OF ATTENDANCE AFTER THE INTRODUCTION OF A PARENTAL LETTER
10 Re -audit
1 Select topic
2 Review literature

clinical audit cycle > 2
3 Set standards
practice
8 Set/review
8 Changestandards
4 Design audit
6 Analyse data
5 Collect data
CONTRIBUTORS
Dr F. Subotsky Consultant Psychiatrist
Dr P. Misch Senior Registrar
Belgrave Department of Child and Adolescent Psychiatry,
Kings College Hospital, Maudsley NHS Trust, London

Appointment failures are common in child psychiatry (Cottrell et al, 1988; Potter & Darwish,
1996). They are especially wasteful of resources if more than one member of the multi-disciplinary
team has set aside time, as with a first appointment diagnostic. Studies in the UK and elsewhere
suggest that a worthwhile reduction in attendance failure rate can be obtained by making some
sort of pre-contact, whether by letter, telephone or meeting (Brockless, 1990; Joshi et al, 1986;
Kourany et al, 1990; Mathai & Markantonakis, 1990; Wenning et al, 1995). Delaying the offer of
an appointment, however, should be avoided, as a long wait in itself can increase non-attendance
(Jaffa & Griffin, 1990; Stern & Brown, 1994) and is perceived as unsatisfactory (Subotsky &
Berelowitz, 1990).
54
In a small team, which is part of a larger child psychiatric service in an inner city, there was
already a simple spreadsheet in place which was used and updated weekly for the clinical team
management and was also used for regular audit. The relevant data were: the date the referral
was first received; the date of the first appointment and the final administrative outcome. If
appointments are sent, the outcomes can be seen, failed or cancelled. There may be a
number of reasons for not sending appointments, such as unsuitability or withdrawal.
Routine clinical audit indicated that the failure to attend rate was rising, despite requests for
confirmation, so it was decided to introduce a parental letter to be sent to the parents of all
non-urgent cases prior to an appointment being scheduled or offered. This included a query
about parents main concerns for the child and their contact with other agencies as well as a
request for their home telephone number. Information about the clinic was also enclosed. This
had to be completed by the parents and returned to the department before an appointment was
offered. Appointments were sent as soon as possible on receipt of response. It was, therefore,
decided to re-audit attendance after the parental letter system had been introduced (see Additional
resource C2 for the parental letter template).
The following relevant articles were found from Medline searches and the contributors own
files: Joshi et al (1986), Cottrell et al (1988), Brockless (1990), Jaffa & Griffin (1990), Kourany et al
(1990), Mathai & Markantonakis (1990), Subotsky & Berelowitz (1990), Stern & Brown (1994),
Wenning et al (1995), Potter & Darwish (1996).
SOURCE OF STANDARDS
Local experience and national and hospital statistics about attendance failure
in child psychiatry.
Waiting time
A previous local survey had indicated that waits up to eight weeks were
acceptable, but that satisfaction diminished after this (Subotsky & Berelowitz,
1990).
STANDARDS SET
1.
Under 15% of new referrals sent appointment will have outcome failed, never seen,
without undue prolongation of waiting time.
2.
85% of cases will be offered their first appointment within eight weeks of receipt of
referral.
Attendance
SAMPLE
All cases referred for two years after the introduction of the parental letter (n=240).
(Note: An audit of attendance had already been conducted examining all new cases referred for
the two years preceding the introduction of the letter.)
DATA COLLECTION
Data were regularly recorded on a team spreadsheet backed up by PAS data and case notes.
Appointments were classified as either sent or not sent. If sent, the possible final
administrative outcomes were failed, seen or cancelled by parent. If not sent, the reasons
were: not accepted as suitable; withdrawn by referrer; withdrawn by parent; or no
response to letters.
55
The annual percentage of new referrals offered appointments who failed to attend any of their
appointments, and the annual percentage of new cases offered appointments in under eight
weeks, were calculated.
KEY FINDINGS
In the two years following the introduction of the parental letter, the standards were met.
Findings were fed back at clinical audit meetings to all teams within the service.
It was decided to maintain and, where suitable, extend the use of the parental letter to other
teams, since it had been found to produce a clear advantage of attendance, with the added
advantage of improved communication without unduly lengthening the waiting time.

-audit
standards/re-audit
It was decided that attendance rates would be re-audited annually.

Resources
At the start of the project, clinical and secretarial time was spent on devising,
implementing and fine-tuning the administrative system, but now it takes no
longer than the previous system. The data input process was only minimally
altered.
Additional points
Changing any long-established practice takes time and persistence.

This intervention was focused on getting a result rather than attempting a
research project, since potentially successful methods had been identified
by others.
Decisions had to be made about non-responders and whether to offer another
chance.
Referrers, who were mainly GPs, were kept informed.
As telephone numbers were now being obtained it was also easier to prompt
again before the appointment.
Not only was there less appointment failure, but more new cases were able
to be seen overall.
56
The need to improve or maximise attendance versus, say, access will vary in priority
according to the particular CAMHS setting. Reducing high failure rates is, however, feasible.
The monitoring system should be part of some general data collection system and ongoing, rather than a one-off study. Hospital PAS systems are usually set up so that they can
extract length of wait before a first appointment and whether the appointments were
attended, failed or cancelled.
ADDITIONAL RESOURCE C2
PARENT
AL L ETTER TEMPLA
TE
ARENTAL
EMPLATE
Consultant:
Dear
Your GP has writen to us about your concerns about
Please complete this form and return it within two days in the enclosed stamped addressed
envelope.
When we receive your reply we will send you an appointment.
We enclose an information leaflet about our service.
Yours sincerely
Clinic Secretary
What are your concerns about your child?

1.
2.
3.
Name of childs school:
Home telephone number:
Please tick boxes to let us know if these other agencies have been involved:
Paediatricians
Other child psychiatrists
Educational psychologist
Speech therapist
Social services
Parent(s) signature:
Date:
57
SECTION D THERAPY
PROCESS
EXAMPLE D1: CLINICAL AUDIT OF THE PRE-ASSESSMENT PROCESS UNDERTAKEN BY OCCUPATIONAL THERAPISTS
ON OUT-PATIENT TEAMS PRIOR TO OFFERING INDIVIDUAL PLAY THERAPY WITHIN CHILD AND FAMILY PSYCHIATRY
10 Re -audit
1 Select topic
2 Review literature

3 Set standards
8 Set/review
8 Changestandards
practice
= not completed
in this example
4 Design audit
6 Analyse data
5 Collect data
CONTRIBUTOR
Mr A. Evans Head Occupational Therapist
Child and Family Psychiatry Department,
Westcotes House, Leicestershire Mental Health Service NHS Trust,
Leicester

Within out-patient teams in Leicester it has been part of the role of the occupational therapist
(OT) to offer individual play therapy. This resource has been developed over the 10 years of the
departments evolution. The team was aware of different approaches existing between individual
OTs and felt that it would be useful to define the process of assessment in this area in order to
generate a check-list of good practice against which staff could audit their own performance.
This could then be passed on to new staff joining the department and used as a teaching tool.
It was hoped that this audit would help to achieve the following: more standardisation of
practice; fewer inappropriate assessments; and a greater emphasis on the pre-assessment
stage of initiating work.
58
The audit assistant conducted searches on a range of databases using a number of keywords
provided by the team. The following references were located: Jeffrey (1984), Levens (1986),
Copley et al (1987), Florey (1989), Le Roux (1993), Mawson & McCreadie (1993), Whalley-Hammell
(1994), Telford & Ainscough (1995), Sturgess & Ziviani (1996).
SOURCE OF STANDARDS
The standards were developed by:
reviewing current practice in the out-patient teams
reviewing literature
forwarding a letter to the National Association of Paediatric Occupational Therapists

requesting any known standards
contacting the Disability Information Services at the College of Occupational Therapy for
any relevant literature
requesting assistance from the clinical audit department (e.g. the audit assistant carried
out a literature search for the team)
reviewing related literature (e.g. art therapy, play therapy, drama therapy, child
psychotherapy).
The standards were, therefore, based on the literature and on a local consensus.
STANDARDS SET
(a) Before seeing the child
(b) Seeing the child
1.
For 100% of cases, a written referral to the OT should be received.
2.
For 100% of cases, additional contact with the referrer should

be made by the OT (e.g. phone or face-to-face meeting).
3.
For 100% of cases, the OT should obtain and read the case notes.
4.
For 100% of cases, the OT should be aware of other agency

involvement (internal and/or external to the child and family
psychiatry service).
5.
For 100% of cases, the family should be seen at least once for a
family interview by the OT.
6.
For 100% of cases where an individual assessment of the child

is considered there should be communication to the family of
what this entails (e.g. hand-outs, verbal discussions).
7.
For 100% of cases where an individual assessment of the child

does not occur, a written outcome of the family interview should
be given to the referrer.
8.
For 100% of cases where an individual assessment is offered,

an appointment should be made with the family.
9.
For 100% of cases the OT must ensure that the child, family and
referrer are clear and agreed about appointment arrangements
(number of sessions, transport, timings, etc.).
10.
For 100% of cases, the child should be seen by the same

therapist in the same room at the same time for each assessment
session.
Standards were set for assessing children for individual work. These were as follows:
59
11.
For 100% of cases a case manager (referrer) should be clearly

identified.
12.
For 100% of cases a feedback session by the OT should be given

to the family after the assessment period.
SAMPLE
All open play therapy cases from the case loads of three individual therapists were reviewed
(n=16).
DATA COLLECTION
Data were collected from the case notes using a check-list (see Data collection tool D1).
The data were analysed by all of the OTs in the department. Assistance with the data analysis
was also received from a research occupational therapist. The number and percentage of
cases meeting and not meeting each standard were calculated. Graphs were produced of the
results.
The findings of the clinical audit project were presented at a lunch-time in-service training meeting
for the child and adolescent mental health multi-disciplinary team, and at a meeting for the
mental health occupational therapist clinical audit group.
As a result of the clinical audit project, it was decided that there was a need for:
greater clarity in the referral (written), outlining expected aims of treatment
more consistency with the room bookings, in order to provide consistent accommodation
in out-patient settings
more consistency in meeting with families prior to the assessment beginning.
Stage 9 Review standards
It was found that for certain cases there were acceptable and logical reasons why some standards
were not met. Targets of 100% were consequently felt to be unrealistic for some of the standards.
These were, therefore, altered appropriately.
Stage 10 Re -audit
Because of the length of treatment for some cases, it would not be appropriate to re-audit for
18 months (i.e. some cases may still be ongoing if the audit were repeated sooner). It was,
therefore, decided that this audit should be repeated every 18 months.
60

Resources
This clinical audit project took approximately 36 hours (including training in

clinical audit) of qualified therapists time and approximately 10 hours of an
audit assistants time. Additional assistance required included:
a visit by a research OT from the Trent Region who explained the process
of clincial audit, and helped the team to plan and think about a topic (this
was funded via the district OT budget)
an audit assistant who conducted computerised database searches (e.g.
Medline) and helped with analysing and presenting the results
information from the National Association of Paediatric Occupational
Therapists and the Disability Information Services at the College of
Occupational Therapy.
Additional points
A relatively large time commitment was required from members of the team
in order to understand the clinical audit process and how it could be of help
to them.
One of the difficulties with this clinical audit project was that it was very
narrow. It examined just a small part of OTs clinical commitments and
impacted only on those who provided individual play therapy (about half of
the team) .
In retrospect, it would have been better if completion dates had been set by
the team for each part of the clinical audit process. The time scale, ideally,
should not have been allowed to slip.

Do not get tied up with too many details.
Make inquiries early on as to what resources may be available in your Trust which could
be of help to you (e.g. the clinical audit department). The help given to the team by the
audit assistant was invaluable.
61
DATA
COLLECTION TOOL
ASSESSMENT
D1
FOR INDIVIDU
AL
NDIVIDUAL
WORK
Part 1 Before seeing the child
Criterion
Standard
A referral to the OT should be received in writing
100% of the time
Additional contact with referrer should be made

(e.g. telephone, face-to-face meeting)
100% of the time
The OT should obtain and read the case notes
100% of the time
The OT should be aware of other agency

involvement (internal and/or external to CFPS)
100% of the time
Yes
No
Yes
No
The family should be seen at least once for a family 100% of the time
interview
If an individual assessent of the child is to be
considered, there should be a communication to
the family of what this entails (e.g. hand-out,
verbal discussion)
100% of the time
If an individual assessent of the child is not to occur, 100% of the time

a written outcome of the family interview should be
given to the referrer
If an individual assessment is offered, an
appointment should be made with the family
100% of the time
Part 2 Seeing the child
Criterion
62
Standard
The OT must ensure that the child, family and the

referrer are clear and agreed about appointment
arrangements (e.g. number of sessions, transport,
timings)
100% of the time
The child should be seen by the same therapist

in the same room at the same time for each
assessment session
100% of the time
A case manager (referrer) should be clearly

identified
100% of the time
A feedback session should be given to the family

by the OT, after the assessment period
100% of the time
EXAMPLE D2: CLINICAL AUDIT OF THERAPISTS INTERACTIONS WITH FAMILIES DURING THEIR FIRST ATTENDANCE
AT A CHILD PSYCHIATRY DEPARTMENT
10 Re -audit
1 Select topic
2 Review literature

3 Set standards
8 Set/review
8 Changestandards
practice
= not completed
in this example
4 Design audit
5 Collect data
CONTRIBUTORS
Dr A. Thompson Senior Registrar in Child Psychiatry
Dr F. Dunne Registrar in Psychiatry
Marion Family Centre, Darlington Memorial Hospital NHS Trust,
Darlington
6 Analyse data
BACKGROUND RATIONALE FOR CLINCAL AUDIT

Clinical audit of communication with patients is an important area because central government
and local managers are placing increasing emphasis on patient satisfaction as a measure of
quality (Secretaries of State for Health, Wales, Northern Ireland and Scotland, 1989), and because
patient satisfaction with communication correlates with satisfaction with other aspects of
clinicianpatient contact (Ley, 1988). Nicol (1990) acknowledged the need to audit patient
satisfaction as a part of clinical audit in child psychiatry. This audit examined families satisfaction
with communication during first sessions at a child psychiatry department.
This topic was selected for clinical audit for three key reasons. First, trainees at the centre
often felt anxious about seeing families for the first time, and wanted feedback about how they
were perceived by families. Second, no-one had examined the views of patients before in the
Marion Centre. Third, research shows that the quality of initial contact with patients can influence
subsequent patient care in three ways:
63
1.
First sessions with families provide important opportunities for establishing a therapeutic
alliance (Minuchin & Fishman, 1981).
2.
Parents continued attendance at a child psychiatry clinic depends on the extent to which
their expectations are met in initial sessions (Plunkett, 1984).
3.
Good communication with patients improves their compliance with treatment (Ley, 1988).
The aim of this clinical audit project was to set standards surrounding the first contact between
clinicians and families and to evaluate how well these standards were being met.
A Medline search was conducted, but was not very productive. A manual search was then
undertaken through issues of the Psychiatric Bulletin and Association for Child Psychology and
Psychiatry Review and Newsletter from recent years, and papers cited by others were then
retrieved. The following relevant articles were located: Mash & Terdal (1976), Wolf et al (1978),
Larsen et al (1979), Minuchin & Fishman (1981), Plunkett (1984), Bloch (1986), Cottrell et al (1988),
Ley (1988), Rashid et al (1989), Secretaries of State for Health, Wales, Northern Ireland and
Scotland (1989), Nicol (1990), Fitzpatrick (1991), Wilkinson et al (1994).
SOURCE OF STANDARDS
The standards set for clinicianpatient communication during first sessions were derived from
criteria felt necessary for successful psychotherapy. These were based on Bloch (1986), and on
the teams clinical experience regarding what constitutes a good first session. The standards
were first devised by the two authors, then discussed with the team, and a few small modifications
were made.
STANDARDS SET
All family members should:
feel relaxed during the session
have the chance to put their point of view across during the session
feel they have been listened to and understood during the session
feel hopeful that the department might help them as a result of the first session
feel able to come back to another session after the first one.
SAMPLE
The sample comprised all new cases attending their first appointment over a six-month
period (n=83). Cases excluded from the sample were re-referrals, ward consultations and
medico-legal cases. Data were obtained from 51 families (62%). Questionnaires were not
completed for the following reasons: 29 were not offered to families by the therapist;
two family members declined to complete the questionnaire; and one was returned
incomplete.
DATA COLLECTION
Existing instruments for measuring patient satisfaction did not seem suitable for the purposes
of this project as they are largely oriented towards treatment for physical health and ignore
64
the family dimension (Wolf et al, 1978; Larsen et al, 1979). The team therefore designed a
self-report questionnaire for adults and children aged over 11 years (see Data collection
tool D2) to assess satisfaction with communication after first sessions. This comprised five
questions addressing the set standards, each with a Likert scale of items allowing four or
five possible responses. Clinicians were also asked to rate the first session using the same
five questions. In an attempt to minimise responder bias the questionnaires were anonymous
and patients were asked to complete them after leaving the interview room, and return
them to secretarial staff in envelopes as they left the department.
The study was piloted to confirm that the method was workable before the audit was
conducted.
The data were analysed using a standardised statistical package. For the report, a series of
percentage values and bar charts were produced.
KEY FINDINGS
Family members generally viewed first sessions favourably three of the five questions
failed to generate any negative answers.
A minority of respondents indicated feeling tense/very tense during the first session, and
a minority left the initial meeting feeling that they had not had the opportunity to speak as
fully as they would have wished.
Therapists gave less favourable ratings of communication during the first session than
families themselves (this difference was not statistically significant).
In general, fathers and children were less satisfied than mothers with their first contact
with the department.
The results were presented to the multi-disciplinary team, where they were discussed.
Possible reasons for children and fathers being less satisfied than mothers with the first
session were discussed by the team. Ideas included:
gender issues (e.g. a predominately female work force, sociocultural expectations of men)
fact-finding nature of first appointments, which may exclude children to some extent.
The team considered the need to obtain the views of children aged 10 years and under, and
planned to do a future audit of childrens perceptions of sessions at the department. No other
ways of improving practice and implementing change were identified as a result of the clinical
audit project.

-audit
standards/re-audit
There was an intention to repeat the clinical audit after two years, but the Trust management
also began to encourage consumer feedback surveys, and the department was encouraged
to develop a more general satisfaction questionnaire. This included versions for parents,
adolescents and children, which were used to audit overall satisfaction with the
two departments managed by the Trust. It was intended to repeat this satisfaction audit
annually.
65

Resources
The existing resources of a computerised departmental audit system were

used to administer the questionnaires.
The data were collated, analysed and written up by two psychiatrists in
training.
The total time required for the clinical audit project was approximately 3040
hours.
Additional support was required from secretaries and advice through
supervision was also used.
There were no additional costs for this clinical audit project.
Additional points
Sample bias It is possible that families excluded from the survey, because
they were not given a questionnaire or because they declined to complete it,
experienced less satisfactory communication during first sessions. Some
therapists reported feeling unable to broach the subject of the clinical audit
after intense, difficult first sessions. It was hypothesised that even genuine
forgetting may have sometimes been influenced by subconscious resistance
to offering the questionnaire because of communication difficulties. In
retrospect, it was decided that it would have been useful to post
questionnaires (along with stamped addressed envelopes) to families who
were not given them after their first session.
Obtaining childrens views In retrospect it would have been relatively easy
to offer a questionnaire for younger children at the same time as doing this
clinical audit project, perhaps using a choice of smiling/frowning faces to
allow poorly literate children to indicate their responses.
Collecting data by self-report Auditing communication by behavioural
analysis of interactions in sessions was originally considered. Although
potentially more objective, this method was rejected as being too timeconsuming and intrusive for a clinical audit which needs to be incorporated
into everyday practice. The brief self-report questionnaire used in this clinical
audit project proved to be acceptable to parents and children (i.e. very few
declined to complete it or had difficulty doing so).
Feasibility in a busy clinical team With enthusiastic secretarial support it
was possible to run this form of clinical audit in a busy clinical department.
The main practical difficulty encountered was maintaining momentum for
the clinical audit among clinicians, some of whom joined the team midway
through the project.
Disseminating results In retrospect the results of the clinical audit project
should probably have been circulated more widely to Trust managers and
purchasers.
66
Be aware that the clinical audit process may only work when the team has accepted the
culture of clinical audit and has worked through the initial resentment and suspicion which
often seems to arise when clinical audit is first introduced to a service.
Auditing patients views may be particularly threatening to practitioners (although research

suggests that patients are generally likely to make favourable comments).
Staff at the Marion Centre were used to a relatively sophisticated clinical audit system and
were therefore well prepared to progress to a clinical audit involving patient views. It
would not be advisable for others to plan this type of clinical audit as a first venture in
auditing a teams work.
DATA
QUESTIONNAIRES/INSTRUCTIONS
COLLECTION TOOL
D2
FOR CLINICAL AUDIT OF THERAPISTS INTERA

CTIONS
INTERACTIONS
ANCE AT A CHILD PSY

CHIA
TR
Y DEP
AR
TMENT
WITH FAMILIES DURING THEIR FIRST ATTEND
TTENDANCE
PSYCHIA
CHIATR
TRY
DEPAR
ARTMENT
Questionnaire for parents and young people
We are asking people who come to the Marion Family Centre to help us understand more
about what it is like to come here for the first time.
We would like you to answer the following questions. Your name will not be known when
we look at the answers to the questions.
Please circle the word/phrase which best describes the way you feel.
Please try and answer all the questions.
1. How relaxed did you feel during the session you have just had?
very relaxed
quite relaxed
tense
very tense
2. Did the interviewer make it easier or more difficult for you to say what you wanted to
say today?
easier
not sure
more difficult
very difficult
3. How well do you think your point of view was understood today?
very well
quite well
not sure
poorly
very badly
4. Do you think the Marion Centre might have something to offer you?
yes, definitely
probably
not sure
probably not
definitely not
much easier
5. If you have decided to come back for another appointment, do you think coming
back will be?
much easier
than today
a bit easier
than today
the same
as today
harder
than today
much harder
than today
THANK YOU VERY MUCH FOR YOUR HELP

Please put the sheet in the envelope and give it to the receptionist as you go out.
67
DATA
COLLECTION TOOL
D2

CTIONS WITH
INTERACTIONS
ANCE AT A CHILD PSY

CHIA
TR
Y DEP
AR
TMENT (CONTD)
FAMILIES DURING THEIR FIRST ATTEND
TTENDANCE
PSYCHIA
CHIATR
TRY
DEPAR
ARTMENT
Instructions for therapists
Please try and remember this communications audit project!

Please give each adult and all children aged 11 and over a questionnaire each and one
envelope between them at the end of your first session.
Please mark the front sheet of each questionnaire with a letter to identify which member
of the family will be filling in the sheet:
a = father
b = mother
c = oldest child
d = second oldest child, etc.
Please explain to the family:

We are asking people who come to the Marion Centre to help us understand what it is
like to come here for the first time by filling in a short questionnaire before they leave.
We do not want to know anyones name (but we are using the letters so that we can see
whether adults and children see things differently).
Please could they fill in the questionnaire in the waiting room, put the sheet in the envelope
and give it into the reception office as they leave.
If anyone would like any help filling the sheet in, please tell the secretaries and they will
find someone to help.
Please fill in the interviewers sheets as soon as possible after the first session.
Please fill in a separate sheet for each family member who got a questionnaire, labelling
them with the corresponding letter.
Even if the questionnaires are not used in your session, please return them to X anyhow.
THANK YOU
68
DATA
COLLECTION TOOL
D2

CTIONS WITH
INTERACTIONS
ANCE AT A CHILD PSY

CHIA
TR
Y DEP
AR
TMENT (CONTD)
FAMILIES DURING THEIR FIRST ATTEND
TTENDANCE
PSYCHIA
CHIATR
TRY
DEPAR
ARTMENT
Questionnaire for therapists
1. How relaxed do you think the family felt during the session ?
very relaxed
quite relaxed
tense
very tense
2. Do you think the session made it easier or more difficult for the family to say what
they wanted to say today?
much easier
easier
not sure
more difficult
very difficult
3. How well do you think the family felt their point of view was understood today?
very well
quite well
not sure
poorly
very badly
4. Does the family think the Marion Centre may have something to offer them?
probably
not sure
probably not
definitely not
5. If the family have decided to come back for another appointment, will coming back
be?
much easier
than today
a bit easier
than today
the same
as today
harder
than today
much harder
than today
yes, definitely
THANK YOU FOR TAKING TIME TO DO THIS
69
EXAMPLE D3: CLINICAL AUDIT OF THE THERAPISTS INITIAL CONTACT WITH A FAMILY ATTENDING A FIRST
APPOINTMENT FOR FAMILY THERAPY
10 Re -audit
1 Select topic
2 Review literature

3 Set standards
8 Set/review
8 Changestandards
practice
= not completed
in this example
4 Design audit
6 Analyse data
5 Collect data
CONTRIBUTORS
Ms S. Pettle Consultant Clinical Psychologist
Dr P. Bruggen Consultant Psychiatrist
Hill End Adolescent Unit, St Albans,
West Herts Community Health NHS Trust, Herts

This project was conducted several years ago at a time when clinical audit was being promoted
in CAMHS. One of the main purposes of this project was to encourage members of staff within
the unit to undertake clinical audit. It was therefore decided to select a topic area that would be
of interest to staff, would be relatively quick to audit and would ultimately prove helpful.
At the in-patient adolescent unit the first meeting with the young person and their family was
perceived as very important and was thus selected as the focus of this particular clinical audit
project. It was decided that standards would be developed by the team about how this first
meeting should be run, and that these would then be audited.
In addition to the interactions between the therapist and the family during the first appointment,
several other aspects of the service, such as tidiness of waiting room, and promptness of
appointment, were also examined but will not be described in this example.
70
No literature was reviewed for this clinical audit project.
SOURCE OF STANDARDS
The standards were internally developed, based on a team consensus regarding what constitutes
good practice in this area.
STANDARDS SET
Although no explicit targets were set, the team felt that the standards should be met 100% of
the time, and were interested in identifying cases where they were not being met, reasons why,
and what could be done about it. The six standards developed were as follows:
1.
The therapist will extend a hand and offer eye contact with each member of the
family.
2.
The therapist will introduce her/himself to the family.
3.
The therapist will describe and explain equipment in use.
4.
The therapist will answer questions asked by the family relating to equipment.
5.
The therapist will make a clear statement about the structure and length of the
session.
6.
The therapist will describe the team and method of live supervision.
SAMPLE
The sample comprised all families coming for their first appointment over a one-month period
(n=20).
DATA COLLECTION
One member of the team designed a simple form which contained: a check-list for the six
standards above; the name of the therapist and the name of the team member completing the
form; the date; and the family surname (Bruggen & Pettle, 1993). It was felt important to include
clinicians names in order to be able to identify those individuals who needed more training in
engagement skills.
At each first appointment a team member sat behind a one-way screen and ticked indicators
of good practice observed. Another member of staff was given the task of checking (at the end
of each day) that the forms were being completed. These forms were collected and considered
by the team at the end of one month.
The data analysis involved calculating percentages for cases meeting or not meeting the standards
and, most importantly, identifying any possible reasons why standards were not being met for
particular cases.
KEY FINDINGS
The clinical audit project revealed that good practice was not occurring when the therapist was
a trainee or on the one day in the week when the team was very rushed.
71
The findings of the clinical audit project were continually fed back to clinicians at training and
fixed supervision sessions. The findings were openly discussed and ways of improving practice
were suggested by staff. The clinical audit project was presented at the Association for Family
Therapy Conference 1991, and a related paper was later published (Bruggen & Pettle, 1993).
As a result of the clinical audit project, the following changes in practice occurred.
The team reorganised the booking of the first appointment to allow them greater time.
Following a reading seminar on the initial interview (Fisch et al, 1982; Burnham, 1986) the
team developed a training exercise for less experienced staff to develop relevant joining
skills.
The team also discussed the role of those behind the screen and the relationship between
them and the therapist in the room with the family. One person mentioned a paper on the
comparable relationship when there is no screen (Kingston & Smith, 1983) and agreed to
disseminate copies to members of the team.

-audit
standards/re-audit
Three months after the first audit, the first five minutes of initial appointment sessions were reaudited, and a marked improvement was found in performance with regards to all of the standards.
Resources
Minimal staff time was required for this clinical audit project (35 hours in
total).
There were no additional costs involved.
Additional points
The team was so impressed with this clinical audit project that they talked
enthusiastically about embarking on another. They also commented on their
awareness of the audit and the systemic nature of the process. They
hypothesised that the establishment of indicators of good practice may
have influenced their behaviour prior to the audit commencing. They noted
that the process of observing theirs and others behaviour had made them
more aware of aspects of their clinical practice that they had ceased to give
so much attention to. They were struck by the fact that with audit, as with
systemic therapy, one change leads to another (Bruggen & Pettle, 1993).
72
It is important for team members to discuss what they consider to be good practice so
that the clinical audit project is not viewed as a way of judging them, but as a means of
collecting data about and improving the quality of clinical practice.
Clinical audit needs to be viewed as a useful tool, and it is therefore important, to begin
with, to choose topics which can be completed reasonably quickly and will be seen as
helpful by members of staff.
SECTION E DOCUMENT
ATION
OCUMENTA
EXAMPLE E1: ARE DISCHARGES FROM THE IN-PATIENT UNIT MEETING CARE PROGRAMME APPROACH
STANDARDS?
10 Re -audit
1 Select topic
2 Review literature

3 Set standards
8 Set/review
8 Changestandards
practice
4 Design audit
5 Collect data
CONTRIBUTORS
Dr G. Richardson Consultant Psychiatrist
Ms T. Foxton Acting In-Patient Team Leader
Mr M. Cross Acting In-Patient Team Leader
Lime Trees Child, Adolescent and Family Unit,
York Health Services NHS Trust, York
6 Analyse data

The aim of the clinical audit was to ensure consistency of care prior and subsequent to the
period of in-patient care. This was considered to be particularly important since patients were
often being transferred back to services some distance away from the in-patient service. The
purpose of the clinical audit was to:
decide which standards incorporated in the Care Programme Approach (CPA) should be
applied to discharges from Lime Trees Psychiatric Unit
find out whether these standards were being met.
Basic national and local CPA information and instructions were reviewed.
73
SOURCE OF STANDARDS
Standards were developed from the CPA (Department of Health, 1996).
STANDARDS SET
The following standards were established:
1.
For 100% of cases, the medical case notes will contain a pre-admission information form
behind the front sheet of the notes, provided by the admitting doctor.
2.
For 100% of cases, the keyworker will ensure that the CPA form is completed in the
patients nursing notes.
3.
For 100% of cases the keyworker will ensure that the decision as to whether the case is
complex or simple CPA is recorded in the significant events section of the nursing notes
within five days of admission.
4.
For 100% of cases the keyworker will ensure that the definitive CPA status (simple or
complex) is recorded in the nursing notes immediately after the assessment review.
5.
For 100% of cases the keyworker will ensure that: the standard invitation letter to
the discharge planning meeting is dated; that all participants invited to the meeting
are listed; and that a copy of this is in the correspondence section of the medical
notes.
6.
For 100% of cases the medical notes will contain typed notes of the discharge planning
meeting which will include the following information: the date of the meeting;
arrangements for ongoing psychiatric care; arrangements for domestic care and support;
arrangements for education; arrangements for any social work involvement; and the
names of all those who attended the meeting and to whom the conclusions of the
meeting are sent.
7.
For 100% of cases the named nurse will record their introduction to parents.
In addition it was felt that it would be useful to obtain information about:
the notice given to guests of the discharge planning meeting
the timing of the discharge planning meeting in relation to discharge.
SAMPLE
Clinical audit one
Twelve in-patients not yet discharged (April 1997).
Clinical audit two
Three discharged in-patients (June 1997).
Clinical audit three
Three discharged in-patients (October 1997).
DATA COLLECTION
Data were collected retrospectively from case notes using Data collection tool E1.
KEY FINDINGS
Clinical audit one
74
The results of the first clinical audit showed poor compliance with
the standards and no consistent way of recording compliance.
Clinical audits two and three
Compliance improved with all standards in the second two clinical

audits, but the target of 100% was not met for all of the standards.
In clinical audit three, standards 2, 5, 6 and 7 were met 100% of the
time.
The findings of the project were fed back through inter-disciplinary nursing-medical discussions,
team meetings and clinical audit meetings.
After the first clinical audit, a discharge check-list (see Additional resource E1) was
devised for the keyworker in an attempt to improve performance in this area.
A check-list for items covered at the discharge planning meeting was developed, and the
responsibility for each task was explicitly allocated.
A standard discharge planning meeting invitation letter was devised.
A minimum time period was set for invitations to the discharge planning meeting to be
sent.

-audit
standards/re-audit

Resources
Nursing/medical time was required to develop and complete the forms.

The total time taken by one doctor and one nurse to complete each audit
was approximately 2 hours.
There were no other costs involved in this clinical audit project.
Additional points
Difficulties were encountered finding information, and locating where the

forms were being collected.
Devise check-lists that are easy to complete.
Discuss the project widely with staff.
Allow staff to audit issues which interest them about the stated topic.
Use patient and family questionnaires to assess the quality of information given to them.
It was decided that compliance with the CPA standards should be re-audited on an annual basis,
beginning in October 1998.
75
DATA
COLLECTION TOOL
DATA COLLECTION FORM
FOR
E1
CP
A AUDIT
CPA
S TAND
ARD
ANDARD
Do the medical notes contain a completed, correctly
filed pre-admission form?
Is the Care Programme Approach form completed?
Was the initial Care Programme Approach status recorded
within 5 days?
Was the definitive Care Programme Approach status
recorded after the assessment?
Was the discharge planning meeting invitation dated and
did it list all invited participants?
Did the discharge planning meeting notes contain infomtion on:
ongoing psychiatric care?
domestic care?
education?
social work involvement?
name of attenders?
Notice given of discharge planning meeting?
Did the named nurse record the introduction to patients?
Contact name and number recorded at pre-admission?
Parent/child received information leaflets?
Evaluation form completed by parents?
Discharge planning meeting timing in relation
to discharge?
Discharge date?
76
A TT
AINMENT
TTAINMENT
ADDITIONAL RESOURCE E1
LIME TREES
ADOLESCENT IN-PATIENT SERVICE
DISCHARGE CHECK-LIST
1.
Name of patient
2.
Name of named nurse
3.
Is a complete pre-admission form behind the front sheet of the medical notes?
(admitting doctor responsibility)
4.
Is the date of the key workers introduction to the patient recorded in the significant
events section of the nursing notes? (keyworker responsibility)
5.
Is the date of the key workers introduction to the parents recorded in the significant
events section of the nursing notes? (keyworker responsibility)
6.
Is the CPA form completed in the patients nursing notes?
7.
Is the decision as to whether the case is complex CPA or simple CPA recorded in the
significant events section of the nursing notes within five working days of admission?
(keyworker responsibility)
8.
Is the definitive CPA status (simple or complex) recorded in the nursing notes
immediately after the assessment review? (keyworker responsibility)
9.
Is the standard invitation letter to the discharge planning meeting dated and does it
list all participants invited to the meeting? Is a copy in the correspondence section of
the medical notes? (keyworker responsibility)
10.
Are there typed notes of the discharge planning meeting in the medical notes
containing information about: (medical staff responsibility)
(keyworker responsibility)
arrangements for ongoing psychiatric care?

arrangements for domestic care and support?
arrangements for education?
arrangements for any social work involvement?
names of all those who attended the meeting and to whom the conclusions of the
meeting are sent?
11.
Have discharge planning meeting notes been sent to:
date of the meeting?
the referrer?
the patients general practitioner?
all relevant professionals?
the patient?
the parents?
all other attenders at the meeting?
York Monitoring System (for York patients)?
In addition it may be helpful to know:
What notice was given to guests of the discharge planning meeting?
What was the timing of the discharge planning meeting in relation to the discharge?
77
SECTION F ADVERSE
OCCURRENCES
EXAMPLE F1: CLINICAL AUDIT OF ADVERSE OCCURRENCES
10 Re -audit
1 Select topic
2 Review literature

3 Set standards
8 Set/review
8 Changestandards
practice
= not completed
in this example
4 Design audit
6 Analyse data
5 Collect data
CONTRIBUTORS
Ms V. Pirie Nurse Therapist
Dr I. Wilkinson Consultant Clinical Psychologist
Marion Family Centre
Darlington Memorial Hospital NHS Trust, Darlington

One method of clinical audit which is derived from negative outcomes is the adverse occurrence
screening method. Basically, this derives from surgery and medicine whereby custom and practice
has been that all deaths are subject to clinical inquiry to determine any lessons that can be learned
from the event. In the field of psychological work with children and families, occurrence screening
can be taken to look at such events as unexpected family breakdowns, drop-outs with no explanation
given to the clinician, or complaints about the nature of the service provided (Wilkinson et al, 1994).
As part of an integrated clinical audit system adopted within the Marion Family Centre, it was
decided that the adverse occurrences within this department would be defined and audited.
This kind of clinical audit had become a common activity within the general hospital but, up until
then, had not been undertaken within the child and family department. The aims of the clinical
audit project were to (a) monitor the incidence of a set of pre-determined adverse occurrences;
and (b) review all events classified as adverse occurrences in detail in order to try to identify
ways in which these could potentially have been prevented.
78
No specific literature was reviewed for this clinical audit project.
SOURCE OF STANDARDS
The adverse occurrences were defined through team discussions, as part of the process of
developing a quality template for the service.
STANDARDS SET
No specific standards were set for this clinical audit project. Ten adverse occurrences were,
however, defined. If any of adverse occurrences 310 occurred, then clinicians were encouraged
to bring the case up at group supervision. The following were defined as adverse occurrences..
Child/family waits for more than three calendar months for their first appointment.
2.
Child/family never attends the department, nor are seen at home.
3.
Child is abused, or is the subject of an investigation under the child protection procedures,
while waiting for the first appointment or during the period of attendance at the department.
4.
Child commits suicide while waiting for the first appointment, or during the period of
attendance at the department.
5.
Child unexpectedly, deliberately harmed him/herself while waiting for the first appointment
or during the period of assessment or therapy.
6.
An unexpected family/foster placement breakdown occurs during the period of

assessment or therapy.
7.
Child/family stops attending during assessment or therapy without notifying the department.
8.
Child or another family member is discharged from the department during the six months
preceding this referral.
9.
A complaint is made by the child/family regarding any aspect of the management of this
case.
10. There is significant inter-agency disagreement or failure of communication during

SAMPLE
1.
All new cases referred to the department over a 12-month period were included in the clinical audit
project (n=316).
DATA COLLECTION
A data collection sheet with a list of the 10 adverse occurrences identified (see Data collection
tool F1) was attached to every set of case notes and completed by the therapist on the closure
of a case. If any of adverse occurrences 310 occurred, then clinicians were encouraged to
bring the case up at group supervision. Unfortunately, this often failed to happen.
The total number and percentage of adverse occurrences which occurred for the 316 cases
were calculated. Further analyses were conducted to investigate any possible relationships
79
between adverse occurrences and other variables (e.g. source of referral, age, diagnosis, and
aspects of practice which would be open to change).
The findings from this project were presented and discussed at a team meeting, and were fed
back to managers and to the local health authority.
Some suggestions for areas which need addressing were made by one of the nurses on the
team as a result of the clinical audit project. Examples of these are listed below.
Patients should have some choice over the gender and profession of therapist, as this
may possibly influence the incidence of deliberate self-harm.
Problems regarding unclear policies about written and telephone contacts following
appointment failures need reviewing.

-audit
standards/re-audit
The clinical audit project continued for another year until 1993, but only the major adverse occurrences
(i.e. suicide, self-harm and abuse) were examined. No plans were made to re-audit this topic.
Resources
Secretarial time was required to type up the draft and modified data collection
sheet and report.
Therapists time was also required at team meetings, and to complete the form.
The development of the forms, analysis of the data, and writing of the report
were performed by a nurse therapist and a student nurse.
The estimated total staff time required for the clinical audit project was 50 hours.
Additional costs were involved in photocopying the sheets and printing the
report approximately 15.
Additional points
Some difficulties encountered during this clinical audit project involved staff
compliance (i.e. getting staff to remember to complete the forms and to
bring cases of adverse occurrences to group discussions was sometimes
hard) and putting into practice the adverse occurence definitions, some of
which were vague. For example, it was unclear what should be categorised
as an inter-agency conflict? These needed defining more carefully and tightly.
In retrospect it would have been helpful to have conducted a literature search
before undertaking the project in order to have been more informed.
Overall, this clinical audit project brought to question whether adverse
occurrence screening is appropriate for child and family departments, since
it is very hard to define adverse occurrences relevant to CAMHS.
80
It may be helpful to think more carefully about clinical adverse occurrences, that is what
could be defined as such, and how they could be screened.
If attempting a similar project, try to measure how often adverse occurrences are being
brought to group supervision. Also, it would be useful to set explicit standards regarding
what should take place if any of the adverse occurrences you have defined should occur.
DATA
ADVERSE
COLLECTION TOOL
OCCURRENCE SCREEN FOR
F1
MARION FAMIL
Y CENRE
AMILY
Child identification number:
A.
This form is to be completed when the child is discharged from the department.
B.
Please consult explanatory notes for guidance.
C.
Please tick any of the following which occurred any time after the referral was
received.
1. Child/family waited more than three calendar months for first appointment.
2. Child/family never attended nor seen at home.
3. Child was abused, or was the subject of an investigation under the child
protection procedures while awaiting first appointment or during
4. Child committed suicide while waiting for the first appointment, or during
6. Unexpected family foster/placement breakdown occurred during assessment

or therapy.
7. Child/family stopped attending during assessment or therapy without
notifying the department.
8. Child or another family member was discharged from the department during
the six months preceding this referral.
9. A complaint was made regarding an aspect of the management of this
case.
5. Child unexpectedly, deliberately harmed him/herself while waiting for the

first appointment or during assessment or therapy.
10. During assessment or therapy there was significant inter-agency

disagreement or failure of communication.
11. None of the above occurred.
Signature:
Date:
81
SECTION G OUT
COMES
UTCOMES
OF THERAPY
EXAMPLE G1: CLINICAL AUDIT OF THE EFFECTIVENESS OF ANGER MANAGEMENT GROUP TRAINING FOR
ADOLESCENTS
10 Re -audit
1 Select topic
2 Review literature
8 Set/review
8 Change standards
practice*

3 Set standards
= not completed
in this example
*N/A current practice
found to be acceptable
4 Design audit
6 Analyse data
5 Collect data
CONTRIBUTOR
Mr R. Down Clinical Psychologist
Swansea Child and Family Clinic Trehafod,
Glan-y-Mr NHS Trust, Swansea

This was the first time that the clinic had offered group intervention for adolescents experiencing
self-control/anger difficulties. The group was run for 13 sessions. Six adolescents aged 1215
years participated. The team wanted to assess whether the programme devised was effective
in reducing the external expression of anger (i.e. aggression to others or to objects). The clinical
audit project involved a psychometric assessment of this, as well as obtaining additional qualitative
data which are not included in this report.
A literature search was not specifically undertaken for this clinical audit project, although the
following articles/books were consulted: Tulloch (1991), Spence (1994).
82
Standards were not set for this clinical audit project.
SAMPLE
Five of the six group members participated in this clinical audit project. One member of the
group failed to attend the last session and did not complete the post-intervention measures.
Therefore, complete data (i.e. pre- and post-intervention) were obtained for only five of the six
members of the group.
DATA COLLECTION
During the initial session (i.e. pre-intervention) all six participants in the group completed the
StateTrait Anger Expression Inventory (Spielberger, 1991) and five participants also completed
the inventory during the last session (i.e. post-intervention).
KEY FINDINGS
Four of the five participants who completed pre- and post-intervention assessments
demonstrated a reduction in the anger expression out factor over the course of the
intervention (mean effect size = 1.8 standard deviations (s.d.), range = 1.43.3 s.d.).
One participant demonstrated a very small increase (0.2 s.d.) in his t scores on the anger
expression out factor.
The mean effect size for all participants was 1.4 s.d. indicating a reduction in anger
expression out scores.
Pre- and post-intervention scores on each of the factors of the StateTrait Anger Expression
Inventory were compared on a case-by-case basis. The magnitude of any shift in factor
scores was calculated in terms of standard deviation using scaled t scores. The reliability of
difference in scores was not calculated, and no account was taken of statistical regression
to the mean.
The anger expression out factor was considered to be most clearly related to the objectives
of the group, that is a reduction in angry/aggressive acts towards others and objects. Findings
for this factor were reported.
Results of the clinical audit were fed back to team members via a presentation at the team
clinical audit meeting.
A similar presentation was also made to the Trust mental health services audit committee.
Attendees were predominately psychiatrists.
No changes were suggested or implemented as a result of this clinical audit project, since staff
were satisfied that four out of the five participants demonstrated a reduction in scores on the
anger expression out factor over the course of the intervention.
83

-audit
standards/re-audit
It was decided that a similar clinical audit may be conducted on any future anger management
groups that are run at the clinic.
Resources
The psychometric assessment used in this study took approximately 30

minutes to administer as a group exercise, pre- and post-intervention. Scoring
and interpretation of the results took about 6 hours in total.
Apart from staff time, and test materials, there were no additional costs
involved in this clinical audit project.
Additional points
There were doubts about some of the participants abilities to reliably complete
the assessment used. It was suspected that some had not attained the
necessary level of literacy and did not fully understand the assessment. It
would, therefore, have been better to have used a more age-appropriate and
culturally sensitive appropriate measure of external expression of anger.
A more substantial design would have calculated the reliability of differences
between pre- and post-intervention scores. All of the participants indicated a
very high anger out score pre-intervention. Statistical regression to the mean
may, therefore, have been partially responsible for the apparent reduction in
anger out scores.
It would also have been useful to have had a structure for analysing and
interpreting the variety of qualitative data gathered about the outcomes of
the group, such as reports from carers, school and participants.
84
Ensure that you stick to a detailed plan for the group intervention, with clear and distinct
therapeutic processes, otherwise it is difficult to attribute change to specific elements of
the intervention.
Do not rely on postal returns.
EXAMPLE G2: CLINICAL AUDIT OF OUTCOMES IN OUT-PATIENT CHILD PSYCHIATRY USING PROBLEM
RATING SCALES
10 Re -audit
1 Select topic
2 Review literature

3 Set standards
8 Set/review
8 Changestandards
practice
= not completed
in this example
4 Design audit
5 Collect data
CONTRIBUTOR
Dr Q. Spender Senior Lecturer in Child and Adolescent Psychiatry
Department of General Psychiatry,
St Georges Hospital Medical School,
St Georges Healthcare NHS Trust, London
6 Analyse data

Outcome measures are difficult to apply in routine child psychiatry. Various approaches to this
problem have been tried, including the Association for Child Psychology and Psychiatry minimum
data set (Berger et al, 1993) and the child and adolescent version of the Health of the Nation
Outcome Scales (HoNOSCA) (not available at the time of this study). This clinical audit project
examined consumer satisfaction with two out-patient services by comparing problem ratings
before and after treatment. There have been several consumer satisfaction studies published in
child and adolescent mental health, but none has taken precisely this approach (Loff et al, 1987;
Thomas & Hardwick, 1989; Kopec-Schrader et al, 1993; Carr et al, 1994).
The main objective of the clinical audit project was to assess the outcome of routine child psychiatry
out-patient treatment using questions related to consumer satisfaction, and problem rating scales.
Although both of these measure outcome, one is qualitative and the other quantitative.
85
A literature search was conducted, which generated the following relevant references: Loff et al
(1987), Fitzpatrick et al (1990), Nicol (1990), Subotsky & Berelowitz (1990), Callias (1992), Berger
et al (1993), Carr et al (1994), Reimers & Treacher (1995), Spender & Cooper (1995).
No specific standards were set for this clinical audit project.
SAMPLE
The sample included all families attending for their first appointment in two child guidance
clinics over a one-year period (n=333).
DATA COLLECTION
All families attending for their first appointment were asked to fill in a questionnaire. The initial
questionnaire asked for a list of problems, and a rating of severity on a five-point scale (see
Data collection tool G2). A follow-up questionnaire was given to each family six months later,
asking for ratings on the same problems, and comments about whether the clinic (or something
else) helped or did not help. Follow-up questionnaires were sent by post.
Only 25% of follow-up questionnaires were returned and a further 4% following a reminder.
Other answers were obtained by telephone (39%) or home visit (20%). Energetic chasing yielded
a sample of 291 (77% of first attendances).
The improvement score was defined as the difference between the mean of the initial ratings
and the mean of the follow-up ratings. A number of t-tests were performed on the data as
described below.
KEY FINDINGS
The improvement score was not related to: site; gender; age; mode of follow-up; thinking the
referral was a bad idea; something else getting better besides the problems listed; or getting
help from somewhere else. Improvement scores were, however, related on t-tests to thinking
the clinic was a waste of time (P=0.02); coping better with the same problems (P=0.01); and
benefiting from other help (P=0.04 for the whole sample and P=0.005 for just those who obtained
help). The first two of these factors (thinking the clinic was a waste of time and coping better
with the same problems) were highly negatively correlated (r=0.71, P<0.001), so multiple
regression did not clarify this picture. Thus, the perception of help (from whatever source) as
useful was significantly associated with subjective improvement, so those parents who did not
find professional input of value tended to rate improvement less.
Parents who found the clinic helpful commented that talking helped because it gave support,
or changed perspective, or provided useful advice. Parents who made critical comments about
the clinic complained particularly about professionals focusing too much on parents and not
enough on the child. Some parents said that their children hated the experience of coming to
the clinic and others that they wanted more advice and less talking. Those who spontaneously
commented on talking as a component of what helped, improved more than the rest (P=0.005).
Children also sometimes commented on either the positive values of talking or on the lack of
anything helpful.
The average improvement score was 1.67, with a standard deviation of 1.34, giving an
effect size of 1.25. An effect size in treatment trials of 1.0 or greater is considered satisfactory.
86
The absence of a no-treatment control group, however, lessens the importance that can be
attached to this. The effect size of improvement over time with no professional help is unknown.
These results implied that staff in out-patient child and adolescent mental health settings
should pay more attention to explaining the nature of the help on offer, perhaps clarifying whether
parents have come for advice or discussion.
The clinical audit findings also indicated that staff should focus more on the child presented
as the problem, in a way which is acceptable to the child.
The results of the clinical audit project were presented to the two clinics.
Some changes in practice were noted as a result of this clinical audit project. For example, since
the project professionals at the clinics have tried to focus more on the child, and less on how
the parent is handling the child.

-audit
standards/re-audit
No plans were made to re-audit.

Resources
The total grant obtained for this project was about 2000 which covered the
costs of data collection, photocopying and postage. Data entry was paid for
separately.
Additional points
Obtaining a reasonable percentage of completed questionnaires was difficult.

As only 25% were returned by post, many telephone calls and home visits
were required to obtain a maximal response. These sometimes had to be
repeated several times (up to a maximum of three) and this was very timeconsuming.
If such a clinical audit project were done again there might be other questions
one would want to ask. In general, however, it was felt that this project
encompassed all of the important aspects of consumer satisfaction that the
team wished to explore at the time and that it still considers to be important.
For this clinical audit project an audit assistant collected the data and
conducted all of the home visits (approximately 500 hours), and a research
assistant entered the data.

The contributor would advise others not to attempt a clinical audit project similar to this because
of the immense difficulties experienced in obtaining sufficient replies.
87
DATA
CONSUMER
COLLECTION TOOL
G2
SURVEYS FOR CLINICAL AUDIT OF OUT

COMES IN OUT-PATIENT CHILD PSY
CHIA
TR
Y
OUTCOMES
PSYCHIA
CHIATR
TRY
Covering letter for initial survey

We would like to know what you think of the service in the Sutton Child and Family Clinic.
So, could you please use the fifteen minutes before your appointment to fill out the
registration form and this brief questionnaire.
Most of you will have come because of a problem or problems with one of your children.
Could you please list these, or any reasons you have for coming here. (You do not have to
list five problems: any number will do.)
We would like you to tell us how severe each problem is by circling a number on the scale.
Think of the effects at home, on the family, on friendships and at school:
1
only just
a problem
mild
moderate
severe
this has
completely changed
the childs (or the
familys) life
Put a 1 if the problem is only just enough of a problem to mention.
Put a 5 if the problem is bad enough to have changed the lives of either the child
or other family members.
If the problem is halfway between these, put 3.
If it is bad, but not as bad as it might be, put 4.
If it is a minor problem, but could be better and still be a problem, put 2.
In about six months time, we will ask you whether the problems you listed are better,
worse or the same, and what helped or did not help.
At the end of the questionnaire, there is a cartoon for the child or teenager to fill in. This is
for the child or teenager to express their views. Parents can write in what the child says if
that is easier. (They may also contribute to your list of problems.)
If you have any problems filling in this form, or need some help with it, please ask the
person who is seeing you during your first appointment.
Thank you very much for sparing the time to help with this.
To give you an example

John refused to go to school. He complained of headaches (which got better when he
stayed off school).
His parents listed three problems:
(1) not going to school,
(2) headaches,
(3) not having any friends.
88
They rated not going to school as 5 because it had completely changed his life.
They rated the headaches as 2, a mild problem, because they agreed with the GP that
there was no serious medical condition.
And they rated not having any friends as 4, a severe problem, because they were very
worried about this.
DATA
CONSUMER
COLLECTION TOOL
G2 (CONTD)

CHIA
TR
Y
OUTCOMES
PSYCHIA
CHIATR
TRY
Preliminary consumer survey
Date:
Name of child:
Childs date of birth:
Code:
Name of people filling in this form, and relationship to child:
Was the referral your idea?
Yes / No
If no, whose was it? (e.g. your GP, health visitor, head teacher, social worker ...)
Yes / No
Problem 1:
only just
a problem
mild
moderate
severe
this has
completely changed
the childs (or the
familys) life
only just
a problem
mild
moderate
severe
this has
completely changed
the childs (or the
familys) life
Did you agree it was a good idea?
Problem 2:
contd ...
89
DATA
CONSUMER
COLLECTION TOOL
G2 (CONTD)

CHIA
TR
Y
OUTCOMES
PSYCHIA
CHIATR
TRY
Preliminary consumer survey (contd)
Problem 3:
only just
a problem
mild
moderate
severe
this has
completely changed
the childs (or the
familys) life
only just
a problem
mild
moderate
severe
this has
completely changed
the childs (or the
familys) life
only just
a problem
mild
moderate
severe
this has
completely changed
the childs (or the
familys) life
Problem 4:
Problem 5:
Add more problems at the end if you wish.

Did all of the people listed as filling in the form agree what to put down?
If no, who decided what to put?
Thank you for your time.
90
Yes / No
DATA
CONSUMER
COLLECTION TOOL
G2 (CONTD)

CHIA
TR
Y
OUTCOMES
PSYCHIA
CHIATR
TRY
Covering letter for follow

-up survey
follow-up
About six months ago, we asked you to list the problems that brought you to the Merton
Child Guidance Clinic. We also asked you to give them a score of 1 to 5. And we asked
your child to say whether anything should change.
We would now like to see if anything has changed, and if going to the clinic helped.
We have listed the problems you put on the first list. Please circle a number from 0 to 5 to
show how mild or severe you think each problem is.
Think of the effects at home, on the family, on friendships and at school:
0
no longer
a problem
only just
a problem
mild
moderate
severe
this has
completely
changed the childs
(or the familys) life
To remind you, here is the same example as before.
His parents listed three problems:

(1) not going to school,
(2) headaches,
(3) not having any friends.
Six months later, they rated not going to school as 2, a mild problem, because he was
managing to get to school, but still needed help, especially after holidays or half terms.
His parents rated the headaches as 1, only just a problem, because he still had them
from time to time, but no-one worried about them.
And they rated not having any friends as 3, a moderate problem, since he was only just
beginning to make friends at school.
At the end of the questionnaire, there is a cartoon for the child or teenager to fill in, to
express their views on whether coming to the clinic helped or not. Parents can write in
what the child says if that is easier. (They may also contribute to your list of problems.)
John refused to go to school. He complained of headaches (which got better when he

stayed off school).
Neither of us works at Merton. We will use your answers to help improve the quality of the
clinic. Individual information will not be made available to people working at the clinic.
Thank you very much for sparing the time to help with this.
91
DATA
CONSUMER
COLLECTION TOOL
G2 (CONTD)

CHIA
TR
Y
OUTCOMES
PSYCHIA
CHIATR
TRY
Follow
-up consumer survey
ollow-up
Date:
Name of child:
Childs date of birth:
Code:
Name of people filling in this form, and relationship to child:
Problem 1 (copied from your first list):
no longer
a problem
only just
a problem
mild
moderate
severe
this has
completely
changed the childs
no longer
a problem
only just
a problem
mild
moderate
severe
this has
completely
changed the childs
contd ...
92
DATA
CONSUMER
COLLECTION TOOL
G2 (CONTD)

CHIA
TR
Y
OUTCOMES
PSYCHIA
CHIATR
TRY
Follow
-up consumer survey (contd)
ollow-up
no longer
a problem
only just
a problem
mild
moderate
severe
this has
completely
changed the childs
no longer
a problem
only just
a problem
mild
moderate
severe
this has
completely
changed the childs
no longer
a problem
only just
a problem
mild
moderate
severe
5
this has
completely
changed the childs
Add more problems here if you wish.
contd ...
93
DATA
CONSUMER
COLLECTION TOOL
G2 (CONTD)

CHIA
TR
Y
OUTCOMES
PSYCHIA
CHIATR
TRY
Follow
-up consumer survey (contd)
ollow-up
Did anything else get better that isnt on the list?
Yes / No
If yes, what was it?
If the problems overall didnt get very much better or got worse, did you:
1. feel that going to the clinic was a waste of time?
Yes / No
2. come to feel better about the same problems

or more able to cope with them?
Yes / No
If you felt that going to the clinic was a waste of time, could you tell us what would have
made it more helpful for you?
If the problems overall got better, what do you think helped them get better?
Since you were referred to the Child and Family Clinic,

have you got help from anywhere else?
Yes / No
If yes,
1. who provided the help?
2. what was it?
3. did it make the problems better?
Yes / No
Did all of the people listed as filling in the form agree what to put down?
If no, who decided what to put?
Thank you all for your time.
94
Yes / No
DATA
CONSUMER
COLLECTION TOOL
G2 (CONTD)

CHIA
TR
Y
OUTCOMES
PSYCHIA
CHIATR
TRY
Girls question cartoons
What have
you come
for?
What would
you lik
e to be
like
different?
How has
coming
helped?
The
you
you
first time
came,
said:
How has coming

not helped?
A similar cartoon was used for boys, but with male figures.
95
SECTION H USER/PRO
VIDER
PROVIDER
SA
TISF
ACTION
SATISF
TISFA
EXAMPLE H1: CLINICAL AUDIT OF CHILD, PARENT AND STAFF OPINIONS ABOUT THE
EFFECTIVENESS OF DAY UNIT ADMISSIONS
10 Re -audit
1 Select topic
2 Review literature

3 Set standards
8 Set/review
8 Changestandards
practice
= not completed
in this example
4 Design audit
6 Analyse data
5 Collect data
CONTRIBUTORS
Ms P. Harkness & Ms V. Lake Community Nurses
The Woodlands Unit,
Department of Child, Adolescent and Family Psychiatry,
North Tees General Hospital, North Tees Health NHS Trust,
Cleveland

This clinical audit project examined the effectiveness of being admitted to the day unit, and
included the perceptions of the child, parent(s) and staff. One of the main aims of the project
was to identify any problem areas in day unit admissions.
Stage 2 Review litarature
The suggested framework for outcomes in CAMHS from the Health Advisory Service thematic
review Together We Stand (NHS Health Advisory Service, 1995) was used in this clinical audit
project. It was decided not to include the referrers views at this stage.
96
STANDARDS SET
There were no existing standards, and none was set for this clinical audit project.
SAMPLE
The project started on 15 May 1995, and the next 25 admissions to the day unit formed the sample.
DATA COLLECTION
Questionnaires for children, parents and staff were developed by three members of the multidisciplinary team, taking lead roles in the clinical audit project (see Data collection tool H1).
Draft questionnaires were presented to and discussed with the rest of the multi-disciplinary
team and some amendments were made. A questionnaire was completed by all subject groups
at the time of admission to the day unit, and at discharge.
The results of the questionnaires were collected for analysis in September 1995 and analysed
by those taking a lead on the clinical audit project in the team. Percentages were calculated for
the answers given on each question, and these were presented as pie charts.
KEY FINDINGS
78% of the children thought that attending the day unit was helpful.
Staff felt that day unit admissions were helpful in 94% of cases.
78% of parents felt that the admission of their child to the day unit was helpful.
56% of parents felt they had a better understanding of their childs problem after discharge.
The results of the clinical audit project were presented to the rest of the multi-disciplinary team,
and discussed at a team meeting. A report was written and circulated to the relevant people.
CHILDREN
It was felt that the results of the childrens questionnaire were very positive and that there was
no need to implement change in practice relating to children at this point in time.
STAFF
Again the results of the questionnaires were generally positive and thus no changes in practice
were recommended as a result of the findings.
PARENTS
The team felt that parents understanding of their childs problem could be improved upon as
only 56% of parents had a better understanding of this after discharge. An action plan to address
this issue was therefore developed and implemented.
97
ACTION PLAN FOR IMPROVING PARENTS UNDERSTANDING OF THEIR CHILDS PROBLEM
Staff are to spend more time with parents, giving them the opportunity to discuss their
childs problem on an individual basis.
Staff are to actively seek out information on specific disorders to enhance the quality of
information given to parents.
Leaflets containing information about various child problems and disorders are to be
displayed on the day unit for parents to take home. (Note: These have been devised and
are currently being audited.)
Plans were made to review current research with regards to the effectiveness of methods
of providing information to parents.
Stage 9 Set/review standards
It was also decided that a measurable standard relating to the effectiveness of information
provided to parents regarding their childs problem should be set.
Stage 10 Re -audit
The above changes in practice were implemented. The information leaflets given to parents
about specific child problems and disorders are currently being audited.
Resources
The amount of time taken to complete this clinical audit project is unknown.
Apart from staff time there were no additional costs involved in this project.
Additional points
Difficulties encountered in conducting this clinical audit project included:

collecting the questionnaires; finding time in the day for working on this
project; and finding ways to present the data that were both clear and interesting.
In terms of improving the design of this clinical audit project it would have
been better to have had a larger sample, and to have asked respondents to
expand on what they found helpful about the day unit admission.
In general, the project was regarded as a very useful exercise for a number
of reasons:
Although comments from parents, in the past, had appeared to be

positive about their childs admission to the day unit, we had no
formal documentation of this.
The clinical audit project was felt to be a useful team-building exercise.
The fact that staff, parents and children worked in collaboration

throughout this clinical audit project was felt to be very positive.
The clinical audit project was a useful learning experience for team
members, especially staff new to clinical audit.
The fact that day admissions were found to be of benefit to families

enhanced staff morale and resulted in a decision by the team that
these admissions should continue.
The importance of including parents and childrens views in clinical audits

was highlighted in this project, and it was recommended that a standard in
relation to this topic should be developed and implemented.
Be prepared to put in time and effort Motivation makes it work!
98
DATA
QUESTIONNAIRES
COLLECTION TOOL
H1
FOR CLINICAL AUDIT OF CHILD, PARENT AND ST

AFF OPINIONS ON
STAFF
DAY UNIT ADMISSIONS
Child questionnaire on admission
Patient ID:
Do you know why you are coming to hospital?
Yes
No
Reason, if given:

How big a problem do you have?

Very big
Yes
Quite big
Yes
Not very big
Yes
Very small
Yes
No
No
No
No

No
No
No
No

Outcome as perceived by patient on discharge
The child is to be asked to make a judgement on the extent, if any, of benefit of the contact
with the service.
Is the problem to do with?

Home
Yes
School
Yes
Friends
Yes
Something else
Yes
Please tick the box that reflects the answer:

1. Very helpful
2. Helpful
3. Some help
4. Made no difference
5. Worse
6. Not applicable
(e.g. very young child)
99
DATA
QUESTIONNAIRES
COLLECTION TOOL
H1 (CONTD)

AFF OPINIONS ON
STAFF
DAY UNIT ADMISSIONS
Parent/carer questionnaire to be completed on admission
Patient ID:
Severity of problem
Please tick the box that most closely reflects the severity of the problem:
Ratings
1. Mild
2. Moderate
3. Severe

Coping
Please indicate by ticking the most appropriate box how you are coping with your childs
behaviour:
Ratings
100
1.
2.
3.
4.
Coping quite well

Some mild difficulties in coping
Moderate difficulties in coping
Not coping at all

DATA
QUESTIONNAIRES
COLLECTION TOOL
H1 (CONTD)
FOR AUDIT OF CHILD, PARENT AND ST

AFF OPINIONS ON
STAFF
DAY UNIT ADMISSIONS
Parent/carer questionnaire to be completed on discharge
Patient ID:
Severity of problem
Ratings
1. Mild
2. Moderate
3. Severe

Coping
As a result of your involvement with the unit, do you now feel more able to cope with
your childs behaviour?
Ratings
1.
2.
3.
4.
5.
6.
More able to cope

Coping quite well
Some mild difficulties in coping
Moderate difficulties in coping
More difficulty in coping
Not coping at all

Very helpful
Helpful
Some help
Made no difference
Worse

Outcome
Ratings
1.
2.
3.
4.
5.
Please indicate by ticking the most appropriate box on how you are coping:
As a result of your involvement with the unit, how do you feel your understanding of
the problem has been affected?
Ratings
1. Better understanding
2. Worse understanding
3. Made no difference

101
DATA
QUESTIONNAIRES
COLLECTION TOOL
H1 (CONTD)

AFF OPINIONS ON
STAFF
DAY UNIT ADMISSIONS
Staff questionnaire new referral/admission
Patient ID:
Problem characteristics
Ratings
1.
2.
3.
4.
5.
Not complex
Low degree of complexity
Moderate complexity
High degree of complexity
Complexity not rated

Severity of problem
Ratings
102
1.
2.
3.
4.
Mild
Moderate
Severe
Severity not rated

DATA
QUESTIONNAIRES
COLLECTION TOOL
H1 (CONTD)

AFF OPINIONS ON
STAFF
DAY UNIT ADMISSIONS
Staff questionnaire outcomes on discharge
Patient ID:
Controllability
Ratings
1.
2.
3.
4.
5.
6.
5.
Facilitative external factors

Some facilitation
Balanced
Some hindrance
Counter-productive external factors
Cannot judge
External factors not rated

Compliance
Yes
Yes
Yes
Yes
Yes

No
No
No
No
No

Outcome
Ratings
1.
2.
3.
4.
5.
Very helpful
Helpful
Some help
Made no difference
Worse
Child
Family
School
Social services
Significant others
please specify

103
EXAMPLE H2: CLINICAL AUDIT OF SERVICE USER SATISFACTION AT AN ADOLESCENT IN-PATIENT UNIT
10 Re -audit
1 Select topic
2 Review literature
8 Set/review
8 Changestandards
practice

3 Set standards*
= not completed
in this example
*Standards implicit in
questionnaires used
4 Design audit
6 Analyse data
5 Collect data
CONTRIBUTOR
Mr A. Saunders Senior Manager
Leigh House Hospital, Adolescent Psychiatric Unit,
Winchester and Eastleigh Healthcare NHS Trust, Winchester

As a part of a continuing drive to improve service quality, it was decided to audit users satisfaction
with the Leigh House service using a questionnaire. The questionnaire was introduced in June
1996, updated in August 1996 and again in January 1997. The aim of this clinical audit project
was to obtain the patients view of each stage of their contact with the Leigh House Service in
order to identify ways of improving the service provided. In this report the most recent clinical
audit of this topic area (1997) is described.
Literature previously found and reviewed as part of a Diploma in Management was used to
develop a framework for this clinical audit project and to inform the design of the questionnaires
104
used. Relevent references were: Department of Health (1979), Griffiths (1988), Maxwell (1983),
Fisher (1984), Klein (1989), Coolican (1990), Daley & Carr-Hill (1991), Gowers et al (1991), Joss et
al (1994), Pearce & Holmes (1994), Holliday (1995), National Health Service Health Advisory Service
(1995), Ovretveit (1995).
STANDARDS SET
Although no explicit standards were set, these were implicit in the questionnaires, for example
all young people should receive an information pack about the unit prior to admission .
SAMPLE
All patients discharged during the period of 1 January31 December 1997 were asked to
participate in the clinical audit project (n=50); 34 of the sample returned and completed the
questionnaires.
DATA COLLECTION
Questionnaires were developed by the team for this project. These contained a mixture of open
and closed questions examining a range of aspects of patient care, from receiving an introductory
information pack to the treatment given (see Data collection tool H2). The questionnaires were
handed to patients during their last week of admission and they were encouraged to complete
them by their primary nurses.
The data were analysed by the clinical audit lead (senior manager at the unit) in consultation
with a consultant psychiatrist, senior nurse and clinical psychologist.
For the closed questions, and those with Likert scales, percentages were calculated for each
response category (e.g. percentage of respondents who answered yes and percentage who
answered no). These were presented as pie charts. A content analysis was conducted on the
open questions, through which common themes were identified. For example, for question 6
(What was important for you to hear at the first meeting?) data analysis revealed that individual
responses fell into one of three main categories: reassurance (12); treatment plans (8); and
culture and types of young people at Leigh House (7).
KEY FINDINGS
81% of young people received an information pack about Leigh House and 69% of those
found it useful/very useful.
Generally, young people wanted to hear about how a typical day at Leigh House would be
structured.
The majority of young people felt that their problems were well understood by the staff
team at their first meeting. Seven young people, however, felt that their problem was not
understood at all.
There was an improvement from the previous clinical audit in the number of patients
feeling that the reason for their admission had been explained to them. This had increased
from 70% in 1996, to 90% in this clinical audit project.
Only 51% of young people felt they had received an explanation of consent to treatment.
When asked what they liked most about Leigh House the most common answers
included the caring, friendly and supportive environment.
105
Aspects of Leigh House they disliked the most included: the rigid routine and structured
day; the requirement to contribute; and the emphasis on school.
Several of the young people felt that they had not had enough contact with certain
professionals on the team.
A draft report including recommendations for changes was initially circulated internally to all
members of the senior management for comments. The final report was then sent to:
the Division General Manager, the Director of Nursing and Quality and the Clinical Director
within the Trust
the six purchasing health authorities
regular referrers (25 consultants)
FOCUS (for advice regarding the future development of the clinical audit).
The results of the clinical audit project were also discussed at the monthly team business
meeting and actions were agreed.
The following changes in practice were recommended as a result of the clinical audit project.
INFORMATION
An outline timetable should be included in the information pack given to parents.
FIRST MEETING
The whole team should continue to give patients clear and consistent information about their
treatment.
ON ADMISSION
It will not always be possible for the staff to get patients to understand that they have received
an explanation of consent to treatment. As their condition improves, however, efforts should be
made to clarify treatment programs.
Despite the improvement in perception of why young people are admitted, it would be helpful
to have a clear statement of the aims of admission explained to every patient.
PATIENTS PERCEPTION OF QUALITY
The problem of achieving consistency in explanations to disturbed patients is endemic in this
type of unit. Staff need to take care that they are not giving conflicting messages and are
constantly aware of the possibility of being manipulated.
The expectation of patient contact with different members of the team was not consistent.
Staff should try to manage expectations by giving clear explanations of the frequency of their
contact.

-audit
standards/re-audit
The following changes to the questionnaire were recommended as a result of the project.
ADDITIONAL QUESTIONS
Question 5 should be amended to include the supplementary question, If you did not feel that
your views were listened to, why not?
106
It was recommended that a future question be included to obtain the young persons
perception of their treatment outcome at discharge:
1.
No improvement
2.
Some improvement
3.
Considerable improvement
4.
Problems resolved
5.
Problems worse
6.
Assessment only
7.
Not known
ANONYMITY
A significant number of patients felt that their problem was not understood. In order to explore
this in more depth, while maintaining anonymity, it was recommended that a coding system
should be developed which would enable follow-up questionnaires to be sent to the relevant
individual patients requesting further information (e.g. a member of the administrative staff
could hold a list of patient names and numbers which would not be accessible to the clinical
staff).
RE-AUDIT
The clinical audit project is ongoing in that the amended questionnaires will continue to be
administered. The findings will be analysed and written-up on an annual basis.
Resources
To produce the reports cost about 3040.

Additional points
Difficulties encountered in conducting this clinical audit project included:

ensuring that staff remembered to administer the questionnaire and getting
staff to own the results.
Have determination.
Put time and energy into involving staff as much as possible in the project from the outset
(e.g. design of questionnaires).
Feed back to staff about the project (e.g. preliminary findings) on a regular basis.
Remember to remove the names of individual members of staff when quoting anecdotal
comments of the young people.
For this clinical audit project the data analysis and report-writing took the
greatest amount of time (approximately 80 hours). Administrative time was
also required for printing and binding the reports (approximately 14 hours).
107
DATA
COLLECTION TOOL
LEIGH HOUSE
H2
SERVICE QUESTIONNAIRE
This questionnaire has been designed to find out what you think about us and the way we
looked after you during your stay in Leigh House.
Could you please spend a few minutes to answer these questions on your own. Please
tick the appropriate box.
Information
1.
Did you receive an information pack about Leigh House?
Yes
2.
If yes, was the information:

Very useful

Useful

Adequate

Poor
3.
Was there anything missing which you wished we had included?
No
First meeting
4.
When you first met with Leigh House staff how well do you feel that staff
understood your problem?

Very well

Well

Adequately

Not at all
5.
Did you feel your views were listened to?
6.
What was important for you to hear at that first meeting?
Yes
No
contd ...
108
DATA
LEIGH HOUSE
COLLECTION TOOL
H2
SERVICE QUESTIONNAIRE (CONTD)
Admission
7.
How well was the reason for your admission explained to you?

Very well

Well

Not very well

Not at all
8a. Were you given an explanation of consent to treatment?
Yes
No
8b. If you answered YES to question 8a, how well was it explained to you?

Very well

Well

Not very well

Not at all
9.
What did you like most about Leigh House?
10. What did you dislike most about Leigh House?
11. Are there any other comments you would like to make about the treatment you received
or about Leigh House in general?
contd ...
109
DATA
LEIGH HOUSE
COLLECTION TOOL
H2
12. We would like to know whether you were given enough opportunity to meet with the
members of your Core Team. Can you please tick one box per staff member. Please
only tick against staff who were involved with you.
Too much
About right
Not enough
Not at all
*Your doctor
*Your nurse
*Your teacher
*Psychologist
*Occupational therapist
*Social worker
Any other staff
please specify
*Delete as necessary
contd ...
110
DATA
LEIGH HOUSE
COLLECTION TOOL
H2
13. We would like to know which aspects of our treatment programme you found most
helpful. Can you please tick one box for each topic. We realise that there will be some
activities/treatments which are not applicable to you.
Activity/treatment
Very
Helpful
Helpful
Little
Help
No help
Not
Applicable
School
Boys/girls group
Social skills training
Positive eating
Individual sessions
OT, nurse, psychologist
Family meetings
Family therapy
Discovery group
Post-16 group
6 oclock meeting
Reflections group
Expressive art
Relaxation
PSE group
Sports/games
Anxiety management
training
Community meeting
Assertion training
Relaxation training
Independent living
skills
Community skills
programme
Comments
Thank you for taking the time to complete this questionnaire.

111
SECTION I DET
AILED
ETAILED
SERVICE PRO
VISION CLINICAL AUDITS
PROVISION
EXAMPLE I1: CLINICAL AUDIT OF TIER 2 CHILD AND ADOLESCENT FAMILY SUPPORT TEAM
10 Re -audit*
1 Select topic
2 Review literature
practice
8 Set/review
8 Changestandards

3 Set standards
*Ongoing
4 Design audit
6 Analyse data
5 Collect data
CONTRIBUTORS
Ms D. Gregory Family Support Team Co-Ordinator
Norfolk Social Services Office (Southern District)
Mrs S. McGeorge Clinical Effectiveness Co-Ordinator
Hellesdon Hospital (Norwich)

Following the allocation of joint services funding, a pilot project involving social services, health
and education was set up for the patient population aged 018, of 11 general practices in Norwich
and south Norfolk. This involved the reorganisation of work to support children and adolescents
with mental health problems according to the model advocated in the Health Advisory Service
Report Together We Stand (National Health Service Health Advisory Service, 1995).
This clinical audit project examined the effectiveness of this method of service delivery and
involved the collection and analysis of information from all agencies involved, the children, families
and referrers into the service. The clinical audit was conducted as part of a wider service
evaluation. In addition, therefore, to auditing the standards outlined below, the evaluation involved:
collecting descriptive information about the number of referrals and the presenting problems,
etc.; analysing the cost of the work; and measuring the progress of the project in relation to its
112
aims. The evaluation was designed to serve both formative and summative purposes. The interim
clinical audit reports were intended to be used to improve the functioning and development of
the pilot project known as the Family Support Team (FST).
This clinical audit project was based on the Health Advisory Service publication Together We
Stand (National Health Service Health Advisory Service, 1995).
Stages 3, 4 and 5 Set standards/design audit/collect data
STANDARDS SET, SOURCE OF STANDARDS, AND DATA COLLECTION METHODS

The clincial audit project was agreed before the team commenced work, and therefore operating
systems for the team were designed to meet the standards (e.g. Standard 4: a requirement of
referral is that the child and family are made aware of the reasons why it was made), and
documentation including referral forms were designed to include essential information. The
inclusion of Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA)
ratings required a change in practice for all team members.
100% of referrals to Tier 3 (Bethel Child and Family Centre a specialist child and
adolescent mental health centre) will be made through Tier 2.
Exceptions
When presentation exceeds Tier 2 referral criteria and Bethel intervention is obviously
necessary.
Source
Together We Stand.
Focus
Health.
Method
All referrals to Tier 3 (within time period and from participating practices) were
compared with Tier 2 referral criteria.
The project coordinator was consulted where necessary for advice on cases.
Standard 2
In 70% of cases there will be beneficial and demonstrable change in the presenting
problem as a result of the FST intervention.
Exceptions
Where the assessor indicates that symptom/problem improvement is not an

outcome or goal.
Source
Together We Stand.
Focus
Education, health, social services.
Method
HoNOSCA: ratings by FST member at first contact and on closure of case.

Comparison of time 1 and time 2 mean scores and mean sub-scores.
Standard 1
Completion of HAS data sets on case closure by:
FST member
referrer (by post)
client and carer (family) explained and given by FST member. For return to
central address (prepaid envelope given).
Standard 3
For 100% of cases there will be an opportunity for consultation between staff of
Tier 1 and the FST at least every 14 days.
Source
Consensus.
Exceptions
None.
Focus
Method
Check FST records/diaries. Questionnaire for Tier 1.

113
Standard 4
100% of children and families accepted by the FST will be able to explain who they
have been referred to, when they will be seen, why the referral was made and how
to contact their case-worker.
Source
Together We Stand.
Exceptions
None.
Focus
Method
FST member to ascertain and document clients understanding on first contact.

Check FST referral forms.
Standard 5
For 100% of cases the FST will provide services more closely to the clients homes
than existing Tier 3 services.
Source
Together We Stand.
Exceptions
None.
Focus
Method
FST to note location of contacts compare distance from home to existing

comparable Tier 3 service.
Client questionnaire on satisfaction with accessibility.
Monitor non-attendance rates.
Standard 6
90% of clients/families will be given an appointment within 13 weeks from receipt

of referral, and 100% within 26 weeks.
Source
Patients Charter Standard.
Exceptions
None.
Focus
Method
FST records.
Standard 7
(a) 100% of clients/families will be allocated a case-worker from the FST.

(b) Appointment dates, times and locations will be agreed with 100% of
clients/families.
(c) 100% of service users will be seen within 30 minutes of appointment time.
Source
Patients Charter Standard.
Exceptions
None.
Focus
Method
Records kept by FST include time from referral to date appointment sent.
Standard 8
The FST will positively promote mental health within the communities in which
they work.
Source
Together We Stand.
Exceptions
None.
Focus
Method
FST to produce evidence of health promotion activity. Focus group feedback.
SAMPLE
All referrals in the first 12 months of the service were included in the sample (n=222).
The data were collated, and analysed by clinical audit staff and the project coordinator.
114
KEY FINDINGS
The project demonstrated the benefit of a Tier 2 Family Support Team. The continuation and
expansion of the service was recommended as a result of the project. It was also recommended
for the findings to be incorporated into the jointly produced Childrens Services Plans (Social
Services Department), and specialist CAMHS services planning. The benefits included:
a reduction in the waiting list for specialist CAMHS services
an effective filter to re-direct referrals to appropriate agencies
an effective source of information for potential referrers
the need for and benefit of professional consultation
positive user feedback regarding access to a local service
the development of interprofessional relationships and understanding of roles.
A report of the findings was produced quarterly and disseminated to all relevant parties. They
were also distributed to all child and family services throughout the region.
The following changes in practice have occurred as a result of this project:

The development and delivery of referral protocols in primary health care teams and the
Education Department.
The development and delivery of a professional consultation service across health,

education and social services.
The development and delivery of training and staff development in the recognition, referral
and treatment of CAMHS problems in Tier 1.
The crucial role of a coordinator has been developed to process all referrals and to
re-direct referrals efficiently and appropriately, when necessary.

-audit
standards/re-audit
The clinical audit project has been ongoing since the inception of the team.
Resources
The resources used for data collection included HoNOSCA, Health Advisory
Service data sets, service user questionnaire and referrer questionnaire (see
Data collection tools for examples of some of these).
The main time consumed was in establishing the audit design. Data collection
and analysis took approximately 4 hours per month.
A quality assistant was employed to collect, collate and analyse the data.
This person was supervised within the clinical effectiveness department
(Norfolk Mental Health Care NHS Trust).
Additional costs included postage, photocopying and travel but these costs
were minimal.
Additional points
The following problems were encountered when conducting this clincial audit
project:
a lack of comparative data, especially regarding clinical effectiveness
a lack of familiarity with HoNOSCA forms for non-medics

115
a poor response from the user feedback forms
a lack of understanding about the relevance of clinical audit within the

team, which reduced their commitment to completing the paperwork.
In retrospect it would have been useful to have measured the time from the point of referral to
being seen, and to have developed a specific coding system for the nature of the problem (e.g.
conduct disorder). It would, also have been helpful to have measured the DNA rates in order to
compare them with other services.
116
Talk to someone who has done this kind of clinical audit before and can help.
Pilot the clinical audit design and re-evaluate before starting properly.
Train staff in the importance of clinical audit.
Have specific stages in the project for collection of the data.
Keep the paperwork as simple as possible.
DATA
CLINICAL
AUDIT OF TIER
COLLECTION TOOLS
I1
CHILD AND ADOLESCENT FAMIL

Y SUPPOR
T TEAM
AMILY
SUPPORT
User satisfaction questionnaire
What did you think about the F

amily Support T
eam?
Family
Team?
The FST was set up to help families with problems that do not need complicated or specialist
services. Because these problems might previously have been dealt with by Social Services,
the Bethel or the Education Department we have tried to make it possible for you to get
similar help from the Family Support Team.
The Family Support Team aims to provide support to children and families in their own
neighbourhood and to provide support in a way which is easy to understand and helpful.
Please answer the following questions with your family and tell us what you think about
the help you received from the Family Support Team.
1. Where did you usually meet with your case-worker?
2. Would you have preferred to meet them somewhere else? If so, where?
4. What was the best thing about the help you received?
5. What was the worst thing about the help you received?
3. How easy was it to contact your case-worker?
6. Please rate the help your family received:

Very helpful
Helpful
Some help
Made no difference
Made the problem worse

Thank you for telling us what you think. If you have other comments please write them on
the back of this form. Please return this form in the envelope provided.
117
DATA
CLINCIAL
AUDIT OF TIER
COLLECTION TOOLS
I1 (CONTD)
CHILD AND ADOLESCENT FAMIL

Y SUPPOR
T TEAM
AMILY
SUPPORT
Referrer
s form
eferrers
Part of the evaluation and clincial audit of the Family Support Teams work is to demonstrate
the outcome of the service, including referrers, case-workers and families perceptions
of the benefits.
You recently referred
to the Family Support Team.
What was your main aim when making this referral?
How well was your main aim achieved?

Completely
Partly
Not at all

Please rate the help this family received from the Family Support Team.
Very helpful
Helpful
Some help
Made no difference
Made the problem worse

Please make any other comments on the benefits of the Family Support Team to this
family:
Thank you for taking the time to complete this form.

Please return it in the envelope provided.
118

Clinauditchap3 PDF

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Clinauditchap3 PDF

Uploaded by

Copyright:

Available Formats

Examples of clinical audit projects

Examples of clinical audits

Number of times around

7 Feed back findings

Stage 1 Select topic

BACKGROUND RATIONALE FOR CLINICAL AUDIT

Stage 2 Review literature

Stage 3 Set standards

1. 100% of children with suspected autism will be referred to the Child

5. 100% of children with autism will be referred to the Communication

7. 100% of children with autism will be assessed by the pre-school learning

Stages 4 and 5 Design audit/collect data

Examples of clinical audits

Stage 6 Analyse data

Standards 3 and 4 were achieved fully.

Stage 7 Feed back findings

Stage 8 Change practice

Development of a system which would facilitate communication between the Child

Identification of criteria for routine case conferences.

Stages 9 and 10 Review standards/re

A date for re-audit was not decided.

COMMENTS ON THE CLINICAL AUDIT PROCESS

Total staff time required to complete the audit was approximately

Hints from contributors

If you are attempting a similar clinical audit project, talk to a range of

AUTISM AUDIT CHECK-LIST

10. Date first seen by CROFT/Communication Disorder Clinic:

12. Letter from CROFT to CDC:

Examples of clinical audits

9. Date of referral to CROFT/Communication Disorder Clinic:

13. Date of letter:

AUTISM AUDIT CHECK-LIST (CONTD)

16. Neurocutaneous marker:

17. Does the child have any associated medical conditions?

Results filed in notes?

Results filed in notes?

Metabolic screen blood

Results filed in notes?

Metabolic screen urine

Results filed in notes?

Results filed in notes?

Results filed in notes?

Results filed in notes?

Results filed in notes?

Results filed in notes?

Woods lamp examination

Results filed in notes?

AUTISM AUDIT CHECK-LIST (CONTD)

20. Was a full developmental assessment done by the clinical psychologist?

22. Was the child assessed by a speech and language therapist?

23. Does the child attend speech and language therapy?

24. Was the child referred to the Communication Disorder Clinic?

27. Was the child seen by the childrens disability team?

28. Does the family get Disability Living Allowance?

29. Does the family get any respite?

30. Have the parents been introduced to the Autistic Society?

31. Have the parents been given booklets/information on autism?

33. Has the child been seen by an educational psychologist?

34. Has the child been statemented?

35. Does the child attend mainstream school?

36. Does the child attend special class in mainstream school?

38. Does the child attend special school?