Professional Documents
Culture Documents
2d
Workbook 3
Needs
Assessment
WHO
World Health Organization
UNDCP
United Nations International Drug Control Programme
EMCDDA
European Monitoring Center on Drugs and Drug Addiction
This document is not a formal publication of the World Health Organization (WHO) and all rights are reserved by the
Organization. The document may, however, be freely reviewed, abstracted, reproduced and translated, in part or in
whole but not for sale nor for use in conjunction with commercial purposes. The views expressed in documents by
named authors are solely the responsibility of those authors.
Acknowledgements
Table of contents
Question 1 10
Question 2 14
Question 3 16
Question 4 23
Overview of
workbook series
This workbook is part of a series in- and cost-effectiveness using the in-
tended to educate programme plan- formation that comes from these
ners, managers, staff and other deci- evaluation activities.
sion-makers about the evaluation of
services and systems for the treat- This workbook (Workbook 2) de-
ment of psychoactive substance use scribes step-by-step methods for
disorders. The objective of this se- implementing evaluations. These
ries is to enhance their capacity for steps span from starting the study, to
carrying out evaluation activities. collecting, analysing, and reporting
The broader goal of the workbooks the data, to putting the results into
is to enhance treatment efficiency action in your treatment programme.
Introductory Workbook
Framework Workbook
Foundation Workbooks
Workbook 1: Planning Evaluations
Workbook 2: Implementing Evaluations
Specialised Workbooks
Workbook 3: Needs Assessment Evaluations
Workbook 4: Process Evaluations
Workbook 5: Cost Evaluations
Workbook 6: Client Satisfaction Evaluations
Workbook 7: Outcome Evaluations
Workbook 8: Economic Evaluations
What is a needs
assesment?
Needs assessment is a tool
for program planning.
Needs assessments evaluate: verse needs associated with PSU
disorders
l The capacity of treatment services
in the community in relation to the l The co-ordination of services
prevalence and incidence of PSU within a system of care in order to
disorders facilitate entry into the system,
smooth transition across specific
l The appropriate mix of services components and appropriate fol-
required to respond to the di- low-up
Why do a needs
assesment?
Over the last two decades, the role needed. In other areas with avail-
of needs assessment in the planning able services, the focus is now to
of services and systems for PSU dis- ask about how existing services
orders has increased in importance. might be better co-ordinated and
Several factors have contributed to more efficient.
this development, including:
l The increasing diversity of com-
l Questions that arise about the rela- munity interventions that are avail-
tive priority of different commu- able. There is acceptance in most
nity needs. In some jurisdictions jurisdictions that a range of com-
with no services for PSU disor- munity services is needed and that
ders, the focus is now to ask about people coming into treatment
new services that might be should be appropriately assessed
How to do a needs
assesment?
In this Most experts in the field of PSU dis- (DeWit and Rush, 1996). The four
workbook, orders agree that a single "all-pur- questions addressed are:
various
pose" needs assessment technique
approaches
to needs does not exist. This is because needs 1 How many people in the region
assessment assessment planners have different or community need treatment for
are described by goals for conducting assessments PSU disorders?
showing how making it unlikely that a single
they can be used method would suffice for all pur- 2 What is the relative need for treat-
to address four
poses. ment services across different re-
questions
most commonly gions or communities?
asked in In this workbook, various ap-
a needs proaches to needs assessment are 3 What types of services are needed
assessment described by showing how they can and what is the necessary capac-
project. be used to address four questions ity?
most commonly asked in a needs
assessment project. More details re- 4 Are existing services co-ordinated
garding many of these approaches and what is needed to improve the
can be found in recent reviews overall level of system functioning?
Use this The two case examples at the end of tary to the general steps for evalu-
specialised this workbook present two very dif- ation outlined in Workbooks 1 and
workbook
ferent approaches to needs assess- 2. When doing a needs assessment,
together,
simultaneously ment. The first (from Spain) relies you should carry through each of
with the upon existing computerised data- the general steps for evaluation de-
foundation bases, whereas the second (from scribed in Workbooks 1 and 2. Use
workbooks to South Africa) uses interviews and this specialised workbook simul-
maximise the focus groups. Despite their differ- taneously with the foundation
information that
ences, both evaluations are appro- workbooks to maximise the infor-
is presented.
priate because they take into account mation that is presented.
the unique needs and resources of
their settings. Using Workbook 1 as a guide, de-
termine which one of the above four
Each of these questions, and the questions is most relevant for your
methods for answering them, are programme evaluation question.
addressed below. Keep in mind Review that section below.
that this information is supplemen-
Question 1
How many people in the
region or community
need treatment
for PSU disorders?
This workbook will briefly describe come with their unique advantages and
three approaches to answering this disadvantages. The selection will have
question. Unfortunately, there is no to depend on your unique circum-
easy answer to this question because the stances and the expertise, time and re-
various strategies available to you each sources that are available.
1. Mortality-based
prevalence models
This method is easy to use, if you have the O = the total number of deaths
necessary data. For alcohol, for example, from liver cirrhosis reported
the formula is: for a given year in the area
or region of interest
A = P*(D/K),
where K = the annual death rate from
A = the total number of alco liver cirrhosis among al
hol dependent persons in cohol dependent persons
an area or region with complications (e.g.,
rate of death from liver
P = the proportion of liver cirrho cirrhosis per 10,000 alcohol
sis deaths due to alcohol use dependent persons).
By collecting the necessary statistical in- to liver cirrhosis (or suicide and alco-
formation for a region or community, one hol use) occur infrequently
should be able to fill in the required infor-
mation and estimate the number of prob- l the need to supplement the resulting
lem alcohol users. This is used as the esti- estimates of the in-need population
mate of the number of people in need of with estimates based on PS other than
treatment. alcohol
3. Capture-recapture models
The term This method requires that you have access estimates of the total population of PS
capture - to computerised records and a certain level users.
recapture is of statistical expertise. Its advantage is that
derived from this it overcomes the difficulty of accessing The case example from Spain, located at
process in which hard-to-reach segments of the PSU popu- the end of this workbook, uses the cap-
individuals in the lation by relying on sources of informa- ture-recapture method for a portion of its
first sample or tion that contain “naturalistic” samples of analyses. Their data sources included
list are captured known PS users. These sources of infor- records for treatment admissions, emer-
and identified mation might include police records of gency visits, and jail entrances.
(tagged), and arrest for possession of narcotics or court
then a certain convictions for PSU-related crime, hospi- The logic of the capture-recapture model
portion are re- tal emergency room admissions involving for estimating hidden populations of PS
captured or re- cases of PS overdose or admissions to PSU users is best understood by way of an ex-
identified on the treatment centres. Used in isolation, these ample. Suppose that for a given area or
second list. data sources are not particularly helpful region, one has two separate listings or
for estimating prevalence. However, com- naturalistic samples of known opioid us-
bining data from two or more sources of ers. The first list, which we will call list X
information can yield reliable and valid (sample 1), consists of opioid-related ar-
rest cases and the second list called list Y With f22, an estimate of the total popula-
(sample 2), consists of opioid overdose tion of opioid users is given by:
cases presenting to hospital emergency
rooms. With two lists or samples, there
are four possible locations where any given
individual may appear: on list X and not
on list Y, on list Y and not on list X, on list There is no restriction on the number of
X and on list Y and finally on neither list lists (samples) that may be used in the cal-
X or list Y. Figure 1 presents the range of culation of the estimate. In fact, the greater
possible locations in the form of a contin- the number of independent listings or
gency table. samples of opioid users, the more accurate
the estimate becomes.
In the figure on the next page, the only
unknown is cell f22, the frequency count Case in list Y
of the number of cases appearing on ei- (sample 2)
ther list or sample. Once we obtain the
number of cases appearing in the first three Yes No
cells, it becomes possible to estimate cell
f22, and subsequently the total population Case in list X Yes f11 f12
of opioid users. (sample 1)
No f21 f22=?
Obtaining a value for the first cell (f11)
requires that researchers attach unique
identifiers to each case appearing on both Advantages include:
lists. Examples of unique identifiers in-
clude date of birth, gender, marital status l a low-cost approach for helping to esti-
or ethnicity. Once this procedure is com- mate the number of people in need of
plete, it becomes possible to match the treatment for PSU disorders in your re-
number of individuals or cases appearing gion or community.
on both lists. The term “capture-recap-
ture” is derived from this process in that Disadvantages include:
individuals in the first sample or list are
captured and identified (tagged), and then l potential violation of the assumptions
a certain portion are re-captured or re- underlying the model, for example, in-
identified on the second list. The larger dependence of the samples (i.e., being
the number of unique identifiers, the on one list doesn’t influence the prob-
greater the precision in matching cases. ability of being on the other)
Cells f12 and f21 are easily estimated us-
ing the same identifying procedures. With l contamination of the samples through at-
values for the first three cells determined, trition (e.g., death) or mis-classification
the following formula, known as the
Peterson estimator, may be used to esti- l the length of time required to clean the
mate cell f22: lists and match cases if the unique iden-
tifiers
With values for the first three cells deter-
mined, the following formula, known as l lack the required detail and specificity
the Peterson estimator, may be used to es-
l limited background information about the
timate cell f22:
PS users on the lists making it difficult to
determine the types of treatment services
that may be most appropriate for them
Question 2
What is the relative need
for treatment services
across different regions
or communities?
Questions One way to answer this question is to com- 100,000 population); poverty (e.g., per-
about the pare the prevalence of the in-need treat- cent owner-occupied units with water sup-
relative need ment population as established with one ply and/or electricity), and drunk driving
for services for of the three methods described in the above and traffic accidents (e.g., rate of drivers
PSU disorders section. However, other, more easily ob- involved in personal injury accidents by
can be tained statistical data may also be avail- 100,000 licensed drivers).
answered with able that are correlated with PSU disor-
indices that ders in the community. Geographic areas Once the individual indicators have been
combine can then be ranked on the various indica- selected, you have different options for
information on tors and then all the indicators combined combining them into an overall index. Fairly
several into one index that reflects PSU disorders. sophisticated statistical procedures such as
problems The index may then be used to compare cluster analysis and factor analysis have
related to the the relative level of these disorders across been used to create this index (Beshai,
nature and the regions. This method requires that you 1984; Tweed and Ciarlo, 1992; Tweed et
prevalence of have access to computerised records and al., 1992). Adrian (1983) presents two less
these that you have the resources and expertise complicated methods. The first approach
disorders. to perform computer-based statistical involves ranking each indicator across the
analyses. various geographic areas being compared.
A mean rank is then calculated for each
Examples of indicators include indices of indicator and the mean rank for the indica-
alcohol availability (e.g., number of liquor tor is then ranked across the areas into an
stores per 100,000 population); mortality overall rank. This approach weights each
(e.g., rate of alcohol-related deaths per indicator equally and has the advantage of
being easy to calculate and interpret. The tive need for services for PSU disorders is
disadvantage is that the approach is rela- the reliability and validity of each of the
tively insensitive to the magnitude of the individual indicators. For example, many
difference between ranks. social indicators (e.g., income level, hous-
ing) have only indirect relationships to PS.
The second approach used by Adrian Other indicators, such as drunk driving
(1983) first gives a value of 100 to the arrests and convictions, will be influenced
overall rate for each indicator, for all ar- by the level of policing and judicial discre-
eas combined. The small area rates are tion. While it can be argued that the dis-
then calculated as a fraction relative to advantages of one indicator can be offset
the overall rate. For each area, the mean by the advantages of another, indicators
of the various indices is then calculated should only be selected if they are reliable,
to create the composite PSU index. Un- valid and of comparable meaning across
like the ranking method, this index ap- the regions.
proach is sensitive to the degree of dif-
ference in the ranks between the areas In summary, questions about the relative
being compared. The main disadvantage need for services for PSU disorders can
is that the mean of the individual indices be answered with indices that combine in-
is sensitive to extremely high values. The formation on several problems related to
index method is more helpful in assess- the nature and prevalence of these disor-
ing relative need because it retains the ders. After one has compared a region or
degree of difference across the areas community to other areas a stronger ar-
being compared, and thus the relative gument for reallocating resources may be
importance of different indicators. A possible. However, neither the estimates
map of the different areas being com- of the in-need population, nor the relative
pared can also be developed showing the need for services compared to other ar-
variation in the level of PSU disorders eas, provide much direction in determin-
in relation to the average for the entire ing the type of services or the amount of
region. these services that are needed. Other need
assessment strategies are required to an-
The main limitation of all these approaches swer such questions and these are de-
to comparing different areas on the rela- scribed below.
Question 3
What types of services
are needed and what is
the necessary capacity?
Client-centered Community
Needs Assessment
Client-centred Community Needs Assess- l needs should be expressed as specific types
ment (CCCNA) is a prospective data col- of intervention (e.g., outpatient PSU disor-
lection procedure that assesses what cli- der counselling; life skills training) that can
ents or patients think about services that be established in the community
are needed. It has been applied in both
mental health (Cox et al., 1979) and sub- l relevant demographic and clinical in-
stance use treatment services (DiVillaer, formation on those individuals in need
1990 & 1996). It is easy to complete, and of the interventions should be collected
has the added advantage of assessing the
point-of-view of potential consumers of l there should be some assurance that those
programme services. There are four im- individuals in need of the interventions
portant assumptions underlying this ap- would actually use the interventions if
proach: established in the community
l community needs should be identified, This method asks about basic client infor-
at least in part, on the basis of compre- mation (e.g., gender, age), his/her PSU
hensive clinical assessment of a large behaviour, and information about the
and representative sample of individu- “ideal” intervention required by the client.
als in need The listed intervention is then coded as:
1 the intervention does not exist in the com- as well as the client’s own perception of
munity the suitability of different service options to
meet their needs. However, the CCCNA
2 the intervention exists but is not available method is limited in the following ways:
(i.e., agency admission criteria rule out this
client) or accessible (i.e., certain factors l needs of people presenting for treat-
such as transportation, hours of operation ment may not reflect the needs of all
rule out participation) people experiencing disorders in the
community
3 the intervention exists and is available and/
or accessible to the client l the lack of widely agreed upon criteria
for matching clients to treatment means
4 the intervention exists and is available that considerable judgement is in-
and/or accessible to the client, but the volved on the part of clinicians and cli-
client is unwilling to attend the agency ents in establishing the “ideal” treat-
that offers it in the community ment intervention
As the information accumulates about the sta- l depending on the number of agencies
tus of interventions needed for particular types involved, considerable time and re-
of clients, a profile emerges of important gaps sources may need to be dedicated to
in service in the community on region. training of personnel, monitoring the
quality of the data collection and
The main advantage of this needs assess- analysing and reporting the resulting
ment strategy is that it incorporates in- information
formation directly about the person in need,
Case identification
youth
Comprehensive assessment only
Case Management
Withdrawal/ Mgmt - (home)
Withdrawal/ Mgmt - (social)
Withdrawal/ Mgmt - (facility)
Methadone Maintenance
Brief Intervention
Outpatient Counseling
Day/ evening treatment female only
l allows for creative thinking about new l inability to quantify the required capac-
service options not previously adopted ity and resource complement (e.g.,
in the region(s) staff, beds) of the services considered
to be needed
Normative approach
Normative need assessment models are cluded death rates from liver cirrho-
essentially “demand-based”, that is pro- sis, alcohol dependence, alcohol poi-
jecting future needs on the basis of past soning, suicide, homicides, automobile
demand on, and performance of, the accidents and alcohol-related psycho-
treatment system. This approach is fairly sis. These indicators were factor-
complicated, and best for those with com- analysed and two separate indices of
puter and statistical resources. The most alcohol-related problems emerged.
sophisticated of these approaches also The first factor was called a Chronic
takes into account local variation in the Health Index and was used to estimate
profile of PSU disorders. the prevalence of chronic, long-term
alcohol-related problems. The second
The Alcohol Treatment Profile System factor, called the Alcohol Causality In-
(ATPS) developed in the U.S.A. is a dex, was used to estimate the preva-
good example of a normative needs lence of acute alcohol intoxication.
assessment model (Ryan, 1984/1985). The value of this index does not indi-
The ATPS has two main components. cate how many individuals suffer from
The first component, referred to as the acute intoxication or chronic long-term
“need” component, was developed problems, but rather indicates “relative”
based on seven mortality-based indi- prevalence ratings for individual coun-
cators reported as average annual ties. The mortality indicators are avail-
death rates per 100,000 population for able nationally at the county level. Con-
the age group 15 to 74, and for the sequently, an index value for each county
period 1975-1977. The indicators in- has been calculated and published.
The second component of the ATPS nor- ners use these tables to compare the ex-
mative model is the “demand” component pected treatment capacity of a county im-
and is based on treatment data collected plied by the normative model with the
at the national level by the National Drug county’s actual or observed capacity.
and Alcohol Treatment Survey
(NDATUS) (Harris & Colliver, 1989). The Advantages include:
survey data provide estimates of the levels
and patterns of existing service use and ser- l ease of use once the necessary informa-
vice capacities for each planning area across tion has been compiled
the country. Level of use is expressed as
the number of clients served. Service ca- l for each estimate, the model provides
pacity is expressed as the number of treat- a high and low range for a given plan-
ment slots. NDATUS classifies treatment ning area and this is helpful in applying
into seven different modalities: medical the results in the decision-making pro-
detoxification, social detoxification, rehabili- cess
tation, custodial, ambulatory, limited care
and outpatient. Service use and capacity are Disadvantages include:
estimated separately for each of these treat-
ment modalities. l the social and health indicators that
comprise the problem indices in the
In the ATPS model, the NDATUS data model are subject to a wide variety of
form the dependent variable. “Observed” biases
treatment service levels and capacities for
an area are therefore modelled as a func- l the data on past treatment service
tion of the two indices of alcohol-related utilisation may not be based on all
problems. Because the relationship be- existing treatment facilities since
tween need and demand varies substan- some may not have participated in
tially according to different population the survey or otherwise have been
sizes, population size is included as a third excluded (e.g., treatment in the pri-
independent variable in the model. Esti- vate sector)
mates of total expected clients and total
treatment capacities and estimates bro- l the assumption that current or past
ken down by treatment modality, are re- treatment service utilisation patterns
lated to an area’s Chronic Health Index, are an adequate reflection of current
its Alcohol Causality Index and its popu- client needs at the time services are
lation size. For planning purposes, esti- provided and in the near future. For
mates of expected clients and treatment example, the needs of the potential
capacities are presented in a series of population of service users may not
tables according to an area’s population be identical to the needs of the client
size, Chronic Health Index and Alcohol population who have sought treat-
Causality Index. Needs assessment plan- ment in the past
Prescriptive approach
Unlike the demand-based ATPS normative 3 Considering the rate of recidivism and to
model, which relies on what actually exists in keep even with this 10 percent rate of in-
the treatment system in terms of service crease, 30 percent of all alcohol depen-
utilisation patterns, prescriptive models dent persons should be treated in a given
Unlike demand- specify the level of treatment services that year.
based normative should or “ought” to be provided to the resi-
models, which dents of a given planning region. This ap- 4 This figure should be divided into two
rely on what proach can be seen as an extension of the because alcohol dependent persons
actually exists in “continuum of care” approach described constitute only half of the in-need popu-
the treatment above, but more complicated and requiring lation. Thus, 15 percent of the overall
system in terms more computer resources. in-need population should be treated in
of service a year.
utilisation Prescriptive models usually begin with a
patterns, prevalence estimate of the size of the popu- 5 Add a 5 percent buffer to do more than
prescriptive lation in need. It is realistic to assume that keep pace with the growth of the prob-
models specify not all of these individuals will voluntarily lem. Therefore, 20 percent of the prob-
the level of seek treatment and that there are only lim- lem drinkers per year are considered as
treatment ited resources available to treat those who the target population.
services that come to the attention of treatment special-
should or ists. An objective, then, is to determine what One of the most serious problems with pre-
ought to be proportion of the in-need population should scriptive models is that the assignment of
provided to the receive treatment in a given year. Many pre- assumptive values to the estimated popu-
residents of a scriptive models arrive at a figure of 20% lation “in-need” of services is a rather arbi-
given planning based on a series of “assumptive” values or trary procedure based on empirical data
region. proportions assigned to the population with which are questionable in terms of reliabil-
alcohol use disorders in a region or area (see ity and validity. For example, rates of re-
below). This final value, indicating the level cidivism are estimated from treatment data.
of “demand” for treatment services, is then The figure of 10% to represent the increase
apportioned throughout various components in the number of alcohol dependent persons
of the ideal treatment system (detoxification, from one year to the next is not likely to be
case management, etc.). a constant. These proportions can be con-
sidered at best as very rough guesses. More-
Ford (1985) describes a standard set of over, the values are likely to vary across
procedures to arrive at the 20% estimate different planning regions and over time.
of the proportion of the in-need popula- Another problem with the prescriptive
tion to be treated each year: model is that it can be value laden, espe-
cially in those aspects of the model where
1 Two-thirds of alcohol dependent per- little empirical data exist to guide the se-
sons drink again within one year of lection of various parameters. For example,
treatment. treatment practitioners will have different
opinions concerning how the demand popu-
2 The rate of increase in alcohol depen- lation should be apportioned throughout the
dence is around 10 percent per year. treatment syste.
Efforts have been made to minimise this Another significant problem with this prescrip-
subjective component. A comprehensive tive approach is similar to that identified for
forecasting model for estimating the ca- the more basic continuum of care approach.
pacity of alcohol treatment services in Specifically, the model will project needs only
Ontario, Canada (Rush, 1990) bases for services identified a priori as being key
these estimates on six different sources components of the ideal treatment system. This
of information: published research litera- approach may restrict innovation in the plan-
ture on patient characteristics; cost-effec- ning and delivery of services for PSU disor-
tiveness of treatment, and rates of comple- ders if an outdated, or otherwise inappropri-
tion of treatment; a preliminary client ately structured, treatment system is used as
monitoring system for assessment and re- the foundation for model development.
ferral services; a detoxification reporting sys-
tem; a triennial provincial survey of alcohol
and drug programmes; informed opinion
from clinical and research experts and an
American forecasting model.
Question 4
Are existing services
co-ordinated and what is
needed to improve the
overall level of system
functioning?
Workbook 4 provides information about l staff sharing or exchange - staff of dif-
process evaluation of treatment services ferent services are permanently or tem-
and systems for PSU disorders. It includes porarily shared or loaned
a brief discussion of the evaluation of sys-
tem co-ordination. The issues to be ad- l other resource exchanges - the extent
dressed, and the measures of co-ordina- to which services share funds, meeting
tion that may be used, are similar for rooms, materials or other resources
process evaluation and community need
assessment. System co-ordination is typi- l consultations and case conferences -
cally assessed using reports and ratings exchanges that concern the treatment
from directors or managers of agencies of specific clients
that are expected to work together in ser-
vice planning and delivery. Ratings are l overlapping boards - the number of
typically given on: members in common to community
boards of different services
l mutual awareness - the extent to which
staff know about each other and their l normalisation of agreements - the extent
respective programmes to which services have developed formal
agreements to co-ordinate activities
l frequency of interaction - how often key
staff meet to discuss work-related issues Specific measures of service co-ordination
that may be used in a community needs
l frequency of cross referrals - how often assessment are not well-developed in
or how many clients are referred to and terms of reliability and validity. One often
from different services in the network takes a more qualitative approach based
on key informant or focus group interviews.
l information exchange - the extent to Such qualitative data collection procedures
which services exchange information are described in Workbook 1.
Exercise 1
Think about your treatment programme. Example: What types of services are
List five general areas in which you want needed for cocaine users in the community?
to know more about the needs of the com- 1)
munity. 2)
3)
4)
5)
Exercise 2
Assess the availability of existing records l number of patients receiving treatment
for each of the areas that you listed above. within a certain area and/or treatment
system
Do you have access to:
Your answers to these questions will help
l morbidity data you to choose needs assessment that
maximise use of existing data.
l mortality data
Exercise 3
Using the information provided in this l Choose a sampling procedure for
workbook, make the following decisions: choosing specific clients/data to
survey
l Decide what method you will use
to collect the data (e.g., general l Decide the timing of the evaluation
population survey, mortality-based
l Develop a procedure for ensuring con-
prevalence model). Review the infor-
fidentiality and promoting honesty
mation in this workbook as needed
to help you decide. l Decide who will help you collect data
Exercise 4
You will need to prepare an introduc- age Ethical Issues, for more informa-
tory letter and consent form that explains tion about the important topic of par-
the purpose of your study. Review Sec- ticipants rights in evaluation research.
tion 1A of Workbook 2, entitled, Man-
In general, all participants should be asked If you agree to participate, please read and
permission ahead of time before being en- sign the consent form (attached) and re-
rolled in the study. When you do this, your turn it in the stamped envelope with the
should explain the purpose, nature, and completed questionnaire. Thank you for
time involved in their participation. No your time.
person should be forced or coerced to Sincerely,
participate in the study. Dr. X
We are asking your help in understanding I have read the information above and
the needs of the community by filling out agree to participate.
a 2 page questionnaire about your sub- Signature:
stance use patterns. The questions will ask Date:
about your substance use and any effects
that it might have on your life. They will Now it’s your turn. Using the example
take about 10 minutes to complete. All above, and the information provided in
information that you provide us will re- Workbook 2, section 1A, write your own
main strictly private and confidential. introductory letter and consent form.
Exercise 5
Run a pilot test of your evaluation mea- l Can the questions be administered prop-
surement and procedures on 10-15 sample erly? For example, is it too long or too
participants to ensure that everything runs complicated to be filled out properly?
smoothly. Review section 1c of Work-
book 2 entitled Conduct a Pilot Test for l Can the information be easily managed
specific information about how to do this. by people responsible for tallying the
In general, pilot tests assess these ques- data?
tions:
l Does other information need to be col-
l Do the questions provide useful infor- lected?
mation?
Conclusion and
a practical
recommendation
In this workbook, a wide range of meth- however, to explore what the results mean
ods have been described that address four for your programme. Do changes need to
questions that are commonly asked in a happen? If so, what is the best way to ac-
needs assessment concerning PSU disor- complish this?
ders. These questions were:
Return to the expected user(s) of the evalu-
l How many people in the region or com- ation with specific recommendations based
munity need treatment for PSU disor- on your results. List your recommenda-
ders? tions, link them logically to your results,
and suggest a period for implementation
l What is the relative need for treatment of changes. The examples below illustrate
services across different regions or this technique.
communities?
Based on the finding that over 1/4 of ran-
l What types of services are needed and dom sample community respondents
what is the necessary capacity? had used cocaine in the past 90 days,
and among those, 58% were interested
l Are existing services co-ordinated and in receiving treatment, we recommend
what is needed to improve the overall that the programme institute a new co-
level of system functioning? caine treatment service. The service
should begin in March, which is tradi-
For each type of question, there are tionally a low-census month for the
choices to be made in selecting the spe- programme, and would allow for extra
cific need assessment models or methods. start-up time.
You must take into account the nature of
the decisions to be made with the result- Remember, needs assessments are a criti-
ing information and the time, expertise, cal first step to better understanding the
and resources available. Each model or PSU treatment requirements of the com-
method also has advantages and limita- munity. It isimportant to use the informa-
tions that must be carefully considered. tion that needs assessments provide to re-
direct treatment services. Through careful
After completing your evaluation, you examination of your results, you can de-
want to ensure that your results are put to velop helpful recommendations for your
practical use. One way is to report your programme. In this way, you can take im-
results in written form (described in Work- portant steps to create a ‘healthy culture
book 2, Step 4). It is equally important, for evaluation’ within your organisation.
References
Adrian, M. Mapping the severity of alco- hol, Health and Research World, 1989,
hol and drug problems in Ontario. Cana- 13(2):178-182.
dian Journal of Public Health, 1983, 74,
(Sept-Oct):335-342. Robins, L.N., Wing, J., Wittchen, H-U. &
Helzer, J.E. The Composite International
Beshai, N. Assessing needs of alcohol-re- Diagnostic Interview: an epidemiologic
lated services: A social indicators ap- instrument suitable for use in conjunction
proach. American Journal of Drug and Al- with different diagnostic systems and in
cohol Abuse, 1984,10(3):417-427. different cultures. Archives of General
Psychiatry, 1988, 45:1069-1077.
Cottler, L.B., Robins, L.N., Grant, B.F.,
Blaine, J., Towle, L.H., Wittchen, H-U., Rush, B.R. Systems approach to estimat-
Sartorius, N., and Participants in the ing the required capacity of alcohol treat-
Multicentre WHO/ADAMHA Field Tri- ment services. British Journal of Addic-
als. The CID-core substance abuse and tions, 1990, 85(1):49-59.
dependence questions: cross-cultural and
nosological issues. British Journal of Psy- Ryan, K. Assessment of need for alcohol-
chiatry, 1991, 159:653-658. ism treatment services: Planning proce-
dures. Alcohol Health & Research World,
Cox, G.B., Carmichael, S.J., & Dightman, 1984/1985, 9(2):37-44.
C.R. The optimal treatment approach to
needs assessment. Evaluation and Program Tweed, D.L., & Ciarlo, J.A. Social-indi-
Planning, 1979, 2:269-275. cator models for indirectly assessing
mental health service needs. Epidemio-
DeVillaer, M. Client-centred community logic and statistical properties. Evalua-
needs assessment. Evaluation and Program tion and Program Planning, 1992,
Planning, 1990, 13:211-219. 15(2):165-179.
Comments about
case examples
The following case examples describe dif- The second case presents a needs assess-
ferent types of needs assessments. As noted ment that was conducted without the
earlier in the workbook, most experts agree availability of computerised data re-
that a single, all-purpose needs assessment sources. In this situation, evaluators
technique does not exist. This is because wanted to know the service needs for a
evaluation planners have different goals, and rural and underdeveloped area of South
have different data resources available. Africa. Official data were unavailable, so
evaluators decided to use key informant
The first case example describes an evalua- surveys and focus groups as their primary
tion of treatments for PSU dependence in mode of data collection. Through meet-
Barcelona, Spain. Several computerised da- ing and interviewing representatives from
tabases were already available, and were used government, police, commerce, and the
by evaluators to estimate PSU prevalence general community, evaluators were able
and treatment needs within Barcelona. In this to determine perceived PSU trends and
respect, this case is an excellent example of treatment needs.
how existing data can be used effectively to
conduct needs assessments. The overall Of note, neither case relied upon client
evaluation is complex, and includes aspects opinions to assess needs. Direct interview-
of needs assessment, cost analysis (Work- ing of PS users is another option for needs
book 5), and outcome evaluation (Workbook assessments, and can generate highly use-
7). The planners wanted to know trends in ful data. Of course, this type of data would
psychoactive substance use, characteristics be qualitatively distinct from computerised
of PSU users, costs of PSU treatment, and databases and community key informant
effectiveness of care. Other evaluators in- surveys. There is no single right or wrong
terested solely in needs assessment could use way to assess needs; each technique pro-
similar techniques in a narrower scope. To vides a unique and potentially useful type
use this technique, computerised data must of data.
be available.
Case example of a
needs assessment
b) to offer enough treatment services so The global cost of this information system
that access could be guaranteed to ev- was 156,456 ECU for 1995.
ery person asking for it;
Besides this information about the general necessity of assessing social priorities to
population, data from Barcelona’s prison be answered.
files have been collected since 1993.
There has also been an attempt to develop
All these data were processed in such a a unit of analysis for alternative produc-
way that its validity and consistency could tivity, based on product analysis and an
be assured: estimation of time assigned to profession-
als for different care activities. An inter-
a) the data collection was carried out by disciplinary group defined the intermedi-
specially trained health professionals; ate care products accomplished by the
municipal CFCs, and mean time needed
b) a protocol had been developed defin- for every basic product was then calcu-
ing concepts and criteria for inclusion; and lated.
this was a reference protocol for all people
working in the SIDB at any stage of the Coverage evaluation
process;
An estimation of the target population was
c) there was a validated entry of data into needed for evaluating the programmes’
the computer. coverage. Otherwise, it would not be pos-
sible to ascertain if the programme were
After examining the reliability and inter- reaching only a small proportion of the
nal coherence of data gained from differ- population in need. The SIDB provided
ent recorded episodes using the chosen us with the data required to estimate cov-
indicators, it was concluded that there erage using capture/recapture techniques
was a need for an identifying element that (Domingo-Salvany et. al.).
could be used to link different registers
together. The chosen element was the first This kind of information is most useful for
three letters of both surnames (from fa- estimating the need for already existing
ther and mother), birth date and gender. services and for quantifying the volume of
Afterwards, we were able to use an algo- users in need of other care services. Given
rithm for maximising the probability of the chronic and relapsing nature of addic-
unequivocal identification and matching tions and to evaluate coverage properly, it
every individual with episodes protago- was important to differentiate between first
nised by himself. Validity confirmation treatment starts and patients who started
was thus achieved in 97% of pairings, again after drop-out, both among admis-
with sensitivity and specificity both over sions and patients following treatment.
95%.
Assessment of Effectiveness
Measurement of Activity, Productivity
and Cost of Care Various indicators, based on scales
which match several variables in an ac-
For measuring the activity of treatment cumulative way, had been proposed to
centres, standardised measurement units measure the efficacy of care. One indi-
were used for three types of activities: cator could be the percentage of treated
first interview, follow-up visits and patients maintaining abstinence at twelve
methadone dispensation. The assessment months follow-up. Nevertheless, the
of patients in current treatment has evaluation of effectiveness needs to be
proven to be useful, taking into account indirect, using such indicators as reten-
the diversity of drugs involved and the tion in treatment programmes; improve-
Table 1: Intermediate products defined for classifying the activity of four CFCs belonging
to the City Council with concerted management. Estimated mean time and proposed
equivalence in drug dependence care units.
a) Therapeutical refers to any care offered within the recovery programme (mainly counselling and psychotherapeutical
approach)
CFC: care and follow-up centre; DCU: drug dependence care unit
Table III (on page 36) presents this infor- those on drug-free programmes. These
mation related to municipal CFSs between results should imply a substantial redefini-
1991-1994. The number of people enter- tion of goals and objectives for the cen-
ing treatment for the first time went down tres, as well as a review of inclusion crite-
during this period to about 25%, what ria for methadone programmes.
could be attributed to a growing number
of drug users getting in contact with the With respect to the general population,
system. there has been a reduction in the percent-
age of people identifying drugs as one of
Assessment of effectiveness the most important social problems (from
9.7% in 1991 to 2.7% in 1993). Coincid-
Clear differences could be observed among ing with a stabilisation in victimisation, citi-
centres concerning their retention rate: af- zens tended to consider drug addicts as
ter 2 year follow-up, retention was 77% patients, demanding more treatment and
for patients on methadone and 6% for care resources.
Table 2: Activity and costs of the CFCs belonging to the City Council with concerted
management. Barcelona 1994.
a) Therapeutical refers to any care offered within the recovery programme (mainly counselling and psychotherapeutical
approach)
CFC: care and follow-up centre; DCU: drug dependence care unit
Table 3: Total treatment starts in four CFCs* belonging to the City Council. Barcelona,
1991-1994.
The utilisation of hospital emergency rooms in non-IDU population. In 1993, 177 new
depends on several factors, including the cases of tuberculosis were declared in IDUs
kind of answer given by the patient. A ser- (see Table V). Nevertheless, 314 IDU pa-
vice prone to administer or prescribe cer- tients with tuberculosis were registered in
tain drugs will automatically increase its in- Barcelona concerning chemotherapy admin-
flow of drug users. Regardless of the istered to them during the year (part of them
attraction exerted by each centre and the were patients notified the year before and
annual oscillations, Barce- lona’s hospital currently following treatment; others were
emergencies have reduced to around 20% patients who had dropped out of treatment
between 1988 and 1993 (Table IV). and were lost for follow-up). Information
on tuberculosis in different population
Tuberculosis and AIDS are monitored in groups was gathered since 1987.
the surveillance system, both diseases be-
ing strongly related. After an increase from The spread of HIV infection among IDUs
1988, tuberculosis and AIDS had both de- has partly been responsible for the increase
creased. Tuberculosis in intravenous drug in tuberculosis rates. Another consequence
users (IDUs) increased 47% between 1988 of this infection is obviously the rise in
and 1992 (from 155 to 228), descending AIDS cases among IDUs declared in
again the year after. Prevalence of tubercu- Barcelona residents. Between 1988 and
losis remained stable, showing a decrease 1993, while the definition of case by the
Table 5: Tuberculosis incidence in IDUs (a) and main population. Barcelona, 1987-1995.
Centres for Disease Control (CDC) was in AIDS definition, which meant the in-
still in force, AIDS cases soared. In 1993, clusion of new TBC cases as AIDS. Af-
229 cases were declared in the city, 47% terwards, there was a drop in incidence
more than in 1988. In 1994, there was a with a trend to stabilisation around levels
top incidence coinciding with the change of 1990 (Table VI).
Table 6: Annual evolution of AIDS cases in drug addicts and of total AIDS cases.
Barcelona 1988 to 1995. Data by 30.06.1996
AIDS cases reflect infections received for IDUs in contact with AIDS preven-
several years before. Therefore, it seemed tion programmes in Alicante (Spain) be-
better to analyse infections among cared tween 1987 and 1992. However, there
patients. Recent estimations on HIV in- is a need for critical appraisal when
fection rates among drug dependents in comparing data; indeed, several data
contact with Barcelona’s treatment cen- sources suggest that, in every popula-
tres, provided an incidence of 4.8 in- tion, frequency of HIV infection goes
fection/100 people/year of follow-up. down after a period of high incidence,
There has been a trend towards reduc- even without preventive interventions.
tion: from an incidence rate of 6.24 in
1991 to a rate of 3.46 in 1995. These Deaths because of overdoses increased
are big figures, but similar to those between 1988-1994 and tended to de-
given by the USA in IDUs (4 people a crease afterwards. Compared to mortal-
year). Compared to rates calculated at ity in other cities, Barcelona presented a
an European level, ours are lower than higher frequency of overdose deaths; one
those of Italy (7.4 among IDUs in treat- possible explanation being our higher
ment and lower than the annual HIV in- prevalence in intravenous administration
fection incidence rate (11.7) estimated (see Figure 1).
Case example of a
needs assessment
A study to determine the welfare
service needs in the Eastern
Transvaal, Republic of South Africa
By
The authors alone M. K. Christian
are responsible for Director: Professional Services
the views expressed National Deputy Executive Director
in this case example.
SANCA National
Who was asking the a investigate the social problems which oc-
cur in its region and consider, plan and
question(s) and what propose measures for the solution thereof;
did they want to know?
b determine of its own accord or on request
the existing or future welfare needs of the
The Eastern Transvaal region (now named inhabitants of the region or any part
Mpumalanga - one of the nine Provinces thereof;
of the Republic of South Africa) is a very
big and largely underdeveloped area. Out- c plan and prepare a welfare programme
side of a few developed urban and indus- with a view to future development or
trial areas, there is a farming community provision of welfare services/facilities
and a tourist industry as this Province in- which would be likely to be necessary
cludes the famous Kruger National Park to satisfy such
and a number of other scenic areas. So- (i) identified needs,
cial Welfare Services and facilities were (ii) and to recommend the or
almost non-existent for the majority of the der of priority in which such ser
black population. vices should be accorded;
The responsibility for the area concerned d up to 1990, the local government, the
fell under the Regional Welfare Board Transvaal Provincial Administration
Eastern-Transvaal who according to the (TPA) was the main role-player render-
National Welfare Act (Act 100 of 1978) ing only social welfare services at grass
had to:
roots level. Specific Services in the 4 act as a link between communities and
fields of: specialist services;
1 the social workers appointed were to be Except for the specialist input, all social
employees of the TPA and would receive welfare staff were employed to become an
their salaries from the TPA; the Commu- integral part of the entire social welfare
nity profile and needs assessment would programme, beginning with the commu-
be undertaken as a priority and as part of nity profile and needs assessment.
official duties;
Further on in the programme and ac-
2 the TPA would supply offices and ve- cording to the needs/priorities identified
hicles; a number of social work posts were ac-
tually allocated to the agencies - who
3 the specialist agencies would be respon- then proceeded to provide special train-
sible for professional supervision and in- ing to enable the workers to address the
service training of the social workers. problems with the community. One so-
cial work post per 20 000 people was
decided upon.
the local shopkeeper, shebeen owner (indig- preparation of the standard community pro-
enous tavern), Induna (minor chieftain). file, needs assessment and substance abuse
questionnaire proved invaluable in being
Community group meetings were held and able to organise the final reports. Very of-
discussions initiated - not only did a valu- ten it was not possible to get statistics or
able community profile emerge, but facts concrete facts - only general perceptions
and opinions were sought on a number of and informed opinions. What was most im-
issues. Group meetings were popular - portant was that there was seldom any con-
providing an opportunity for Community tradictions - opinions were firmly held.
to get together and enjoy refreshments
(this was minor but a most important cost Sophisticated computer analysis was not
in the programme). available and in many cases would not
have been meaningful because of the na-
The questionnaire on alcohol and drug use ture of the data gathering. Individual com-
was very comprehensive, target groups of pleted profiles and reports were analysed
respondents came from clinics, health care as available by the core group and the rec-
workers, nurses and doctors and other so- ommendations of various community
cial workers, traffic departments, police, members and social worker concerned
magistrate courts, teachers, ministers of re- were taken into consideration for Phase
ligion and members of the community and 2 and prior to Phase 3.
youth. Sometimes the workers left a ques-
tionnaire to be completed - in most cases What did they find out?
because of language and literacy difficul-
ties, these were completed by the social The Community Profile and needs assess-
workers. Availability and willingness of re- ment was able to pinpoint very specifically:
spondents to participate were the only cri-
teria used. No resistance was experienced. 1 the number of people involved and the
requirements concerning the needs of
The social workers received regular super- the blind, deaf and physically disabled
vision and encouragement’s. Reports and and mentally handicapped. This varied
completed work was finally co-ordinated only according to the size of popula-
by the TPA officials had core group. tion.
Each worker was responsible for the fi- There were two main substances used -
nal profile and report back on each com- alcohol and dagga (cannabis satavia). Glue
munity. In this regard the value of pre- and petrol sniffing were very minor.
training and the joint effort made in the
Dagga is used more by youth as it is cheap was unemployed and that the population had
(free) and exciting but is often continued into more than doubled the 1985 figures.
adulthood. Youth however, did not see dagga
smoking as serious, starting fairly early between The political climate in communities and
10 and 20 years with most users between the townships was as uncertain as the politi-
ages of 20 and 30, mainly male. cal development in South Africa. Where the
traditional Induna systems were still in op-
Generally, people were unaware of services eration, there was strong willpower to
or programmes that could help reduce the organise themselves.
use of alcohol and dagga and prevent the
social and health problems occurring. When Schools were overpopulated and grossly
health and social functioning deteriorated, the under served - influencing future educa-
community managed this within their ranks. tion and employment opportunities. Pri-
In some of the communities, help was avail- mary schools outnumbered high schools
able and alcoholics/dagga addicts could be 6 to 1. In 4 areas there were no high
referred to Themba Centre or dealt with schools. School was also very basic offer-
through local health clinics where some ing no additional skills or training.
knowledge was beginning to filter through,
TPA social workers throughout the region Alcohol use was obviously an important
had a case load of less that 30. part of entertainment, used by youth and
adult members of the communities. This
is seen in the extra-ordinary high number
Community problems evaluated
of shebeens (340), taverns (32) and
beerhalls (6) around compared to shops
The most serious problems identified were:
(22) and churches (46). The community
leaders, however, did view the drinking as
1 Unemployment
a serious set back to development and re-
2 Poverty quested awareness and education
programmes as urgent.
3 Lack of Infrastructure
4 Alcohol and Dagga Abuse Very limited sports (4) and recreation fa-
cilities (2) were found - usually only in the
family and community being negatively mining villages. Cinema and TV almost
influence by these. non-existent due to weak power supply
and poverty.
In communities, a greater percentage of
the children were in the care of grand- Religion played a big role in keeping the
parents who could not always provide community together and was very accept-
control or for their financial needs. Child ing of all conditions of life. Religious lead-
neglect and abandoned children were a ers still had the respect of the communi-
further indication of poverty as parents ties and even \political bodies and they
left to go to the cities to look for work. were usually the backbone of those in-
volved in problem solving.
The lifting of the influx control legislation
a few years earlier had a tremendous effect The needs for adequate shelter, water,
on the population in these communities, roads, electricity were expressed more
placing tremendous strain on the few exist- urgently that the needs for services for
ing resources, as people tried to get nearer physically disabled, etc. There was how-
to work opportunities. By 1994, it was esti- ever, an expectation that there should be
mated that 60% of the working population provision for these.
The youth, not affected by alcohol and dagga trained to assist at various levels in the
use, demonstrate a willingness to get involved community.
with community issues as well as to organise
their own entertainment. They appear to be 3 awareness campaigns were planned for the
impatient with older members of the commu- youth between the ages of 12 and 25 to
nity who demonstrate apathy to get involved capture their interest. In a very short while,
with health and welfare issues, especially when they formed into SANCA youth groups
there is no financial gain. In spite of this com- where attention was given to a comprehen-
mon trend, there are community members who sive life skills programme aimed at their own
do involve themselves, but require motivation stated needs.
and financial and practical support.
The youth quickly cottoned onto the
How did they use the fact that a life of alcohol and drug use
would only continue the misery for
information? many other youth. Strategies now in-
cluded training selected youth as peer
For the first time in South Africa, the role counsellors who could work among the
of alcohol and dagga in keeping people young people themselves, who could
and communities underdeveloped was promote a different lifestyle, give talks
demonstrated, both contributing to cause and workshops at schools, churches but
and effect of poverty, unemployment, etc. more importantly in places where
Large scale community development was young people congregated.
required before the development of spe-
cialist services. However, community Further training provided helping skills
work intervention was urgently required and early identification of substance
and could be implemented. TPA social abuse and a referral system of resources
workers already involved in the needs as- available elsewhere. Positive minded
sessment were allocated to specialist NGO youth were targeted and the peer coun-
agencies. SANCA was given 5 posts and selling movement had its origin in the
one supervisor. These were now given Eastern Transvaal.
specific training in substance abuse, prod-
uct knowledge and prevention models and 4 Where existing infrastructures such as clin-
public speaking. The Community profiles ics, hospitals were identified, the social
and their own involvement with the com- workers visited to created awareness and
munities concerned, already indicated suit- offer a training package suited to their
able target groups. needs, or those of their clients, a com-
mon example was the pre-natal clinics
It was felt that the most effective strate- visited by mothers-to-be.
gies to combat alcohol and drug abuse
would be: 5 Conditions were most often very
simple and lacking any refinements,
1 to establish an action committee of con- the social workers ‘ having to go
cerned people. Such a committee would well prepared to get the message
be informed and made aware of the over to a target group that largely
findings of the community project and lacked previous formal education
would be motivated to become part of and could not read.
the solution.
6 Social workers had to be careful not to
2 training in helping skills and early identifi- create unrealistic expectations in the
cation of users and people in need of help community — but to work with what was
would follow. A core group would be possible with maximum utilisation of com-
munity members, but at the same time ar- In the final analysis, it was the community
ranging meetings and putting them in touch members themselves who outlined and un-
with prospective or available resources. derlined and named their problems. All re-
source persons gave their names willingly and
7 The youth to youth movement had opened only a few respondents asked not to be
up many more opportunities to combat named in person - this was respected. So-
alcohol and drugs and had assisted the cial workers were well received as persons
social workers beyond the initial planned who were trying to help make a difference.
intervention. No problems regarding the need assessment
were encountered, only those of distance,
a) on-going training and motivation long hours and the continual evidence of many
was required and the development needs to be met. Monitoring and support for
of a training curriculum; the social workers were never neglected
throughout the years.
b) when funds were available, identi-
The Eastern Transvaal Region has now
fying badges, caps and T-shirts were
become an official Province of the Repub-
provided;
lic of South Africa with its own Depart-
ments of Health and Welfare. All strate-
c) the slogan “say YES to life and NO gies and welfare programmes mentioned
to drugs” was adopted with the have been adopted and programmes con-
SANCA lo o. tinued in co-operation with the agencies
concerned.
At the present time and because of the flex-
ible time-frame mentioned previously, all The model used for the community
of the above steps have been taken and profiles and needs assessment will be car-
are being met at various levels in seven of ried over to other communities in
the 15 communities. Mpumalanga.
Given time, the social workers will give 1 primary and secondary prevention strate-
less time to the established programmes gies were recognised as a priority;
and move into the next areas. There is now
a waiting list of sorts, as more and more 2 institutional treatment/rehabilitation re-
requests for similar programs to be estab- garding substance abuse already existed
lished are being received in respect of Al- in the province. Awareness of the need
cohol and Drug strategies already in op- for treatment and accessibility to the fa-
eration. Other information generated by cilities however formed an important part
the community profile has led to Social of the strategy:
Welfare programmes being developed to
move, into Phase 3 in some of the com- 3 Reconstruction and Development
munities concerned. Programme (the RDP) of the new Gov-
ernment structures has been assisting
There was sensitivity to the fact that all in the upliftment of these communities,
the Community/Townships surveyed were but still has far to go.
underdeveloped, plagued by poverty and
unemployment. Care had to be exercised,
to avoid labelling the community in any
manner.