JOURNAL OF MULTI-CRITERIA DECISION ANALYSIS

J. Multi-Crit. Decis. Anal. 16: 39–53 (2009)

Published online in Wiley InterScience
(www.interscience.wiley.com). DOI: 10.1002/mcda.437

Application of Multiple Criteria Decision Analysis

in the New Zealand Agricultural Industry
A. E. DOOLEY, D. C. SMEATON, G.W. SHEATH and S. F. LEDGARD
AgResearch Ltd, Hamilton, New Zealand

ABSTRACT
Agricultural decisions typically involve multiple criteria, some of which are subjective. Business, environmental, and
lifestyle criteria are all important criteria in these decisions. These criteria can be difficult to trade-off using traditional
methods. Multiple criteria decision analysis (MCDA) provides a formal, quantitative means of evaluating agricultural
decisions taking all these factors into account. MCDA for agricultural decisions was evaluated using three applied
case studies (use of new processing technology, selection of beef policies, and selection of farm systems in an
environmentally sensitive catchment). The case studies all differed in their problem types and decision-maker
requirements. A multi-attribute value theory approach was used for all the cases. This approach was selected using a
descriptive framework which took the method limitations, problem attributes, and decision-maker requirements into
account. The differences between the case studies, the application and the implementation of MCDA, the overall
success of the process, and the potential use of MCDA in agriculture going forward are discussed. While MCDA was
used to help identify the best decision, the main benefits that the decision makers identified included: learning about
the decision, a better understanding of their own and others’ perspectives, a structured way to work through the
decision, a means to explain the decision, and stimulation of discussion and sharing of ideas. These benefits were
particularly important for the group decisions. Participants were not overly concerned with the ranking accuracy.
Problems in implementation included an initial lack of commitment to the process, understanding of the process and
decision, and ownership of the decision. The limited time decision makers had available contributed to this. The
majority of the decision makers liked the MCDA process for these strategic decisions. The quantitative approach and
the graphic presentation appealed to them. Hence, simple MCDA approaches may be as effective as more complex
ones, and can deliver many of the benefits particularly where the time is limited. There has been interest in the use of
MCDA going forward. Copyright r 2010 John Wiley & Sons, Ltd.

KEY WORDS: multiple criteria decision analysis; agriculture; environment; New Zealand; decision making

1. INTRODUCTION future. Agricultural businesses will have to remain

Agriculture, forestry, and horticulture remain
significant industries in the New Zealand’s econ-
omy. These industries operate without subsidies.
The majority of the products are exported, earning
60% of the country’s export revenue and con-
tributing 18% to GDP. New Zealand is known for
its’ ‘clean, green’ pasture-based farm production
systems and high-quality products: attributes that
are important to our customers. The increasing
interest in sustainability, climate change and
environmental impacts, both internationally and
nationally will impact on agriculture more in the
*Correspondence to: AgResearch Ltd, Ruakura
Research Centre, Private Bag 3123, Hamilton,
New Zealand. E-mail: liz.dooley@agresearch.co.nz
viable both environmentally and economically,
while continuing to operate in a business environ-
ment that is likely to require increasingly higher
standards. As farming is also often a lifestyle
choice, lifestyle factors also need to be considered
in many agricultural business decisions.
Multiple criteria decision analysis (MCDA) has
the potential to be useful in agricultural and
environmental decision making to help trade-off
the economic, environmental, and social aspects that
need to be considered in making strategic decisions.
MCDA is a formal quantitative approach for
exploring decisions and assisting with decision
making where there are multiple, conflicting goals
measured in incommensurate units (RAC, 1992;
Beinat, 2001; Belton and Stewart, 2002). Other
advantages can include: making a decision more
transparent to others, providing a focus for

Copyright r 2010 John Wiley & Sons, Ltd.

40 A. E. DOOLEY ET AL.
discussion, providing a means of problem
structuring and working through the information,
and breaking the decision down so that people
better understand the decision both from their
own and from others’ perspectives (Belton and
Stewart, 2002).
Traditionally, multiple objectives in agriculture
have been traded-off formally using cost-benefit
analysis, but valuation of environmental factors
can prove difficult and controversial (RAC, 1992;
Roy, 1999). Consideration of these environmental
issues in agricultural policy decisions has become
increasingly important over the last decade
(Beinat, 2001) and ways to incorporate them are
needed. Farm business level strategic decisions
often combine a formal economic analysis, with
less formal consideration of other factors
(Shadbolt and Martin, 2005). While MCDA has
been used at all levels of agricultural decision
making, from farm-level to agricultural policy
decisions, it has rarely been used for agricultural
and environmental decision making in Australia
and New Zealand (RAC, 1992).
There is considerable literature on MCDA
techniques, particularly the various aggregation
procedures, and reports on results obtained
using hypothetical data or non-typical decision
makers. However, there is limited literature on
applications of MCDA, how to choose between
approaches or techniques, or why a particular
approach was chosen (Belton & Stewart, 2002;
Belton, 2001; French, 2000, 1998). The scarcity
of literature on real MCDA applications is
suggested by Roy (1999) to be because of the
considerable work involved in describing a real
decision process with all its complications, and
results that lack the characteristics required for
many scientific journals. Eden (1995) and
Montibeller (2007) argue the need for more
action research to capture the often complex
reality of decisions and the context in which they
are being made. Eden (1995) noted that real life
decisions differ from those in which students or
others act as decision makers: in reality the
decision makers have to invest time and money
into the decision, accept the risk and live with the
consequences, and work together to implement the
decision.
In this paper we will describe an action research
approach to evaluate the usefulness of MCDA in
agricultural decision making. Learnings from the
implementation of MCDA in an agricultural
decision-making context will be identified.
Copyright r 2010 John Wiley & Sons, Ltd.
Three diverse agricultural case studies involving
strategic decisions at a business level were evaluated
(Table I). While all the three case studies were
from the agricultural sector, they were very
different in their problem types and decision-
maker requirements. A descriptive framework
which took into account the problem attributes,
and method and decision-maker requirements was
developed to help select the most appropriate
MCDA methods to use in the case studies. The
three case studies were:
 An agribusiness decision: use of new technology
in an agricultural processing company.
 Farm business decisions: farmers selecting the
most suitable beef cattle policy.
 Farm business decisions: farmers in an envir-
onmentally sensitive catchment evaluating farm
systems that could be implemented in order to
reduce environmental impacts.
This paper will first describe the three case
studies, followed by the MCDA method selection
and evaluation processes. The case study
implementation and process evaluation will then
be discussed followed by a section on the potential
for MCDA in future agricultural decision making
in New Zealand.
2. DESCRIPTION OF THE THREE CASE
STUDIES
MCDA applications reported in the literature are
often policy decisions with large numbers of
stakeholders or stakeholder groups. In contrast,
our case study decisions were strategic decisions
made at a business level, although one decision
was looking at the responses of individual farmers
to environmental policy changes. The case studies
selected largely reflect the nature of farming
businesses in New Zealand.
The participants in the case studies were already
working with AgResearch on other projects which
required a decision to be made (Note: AgResearch
Ltd. is New Zealand’s largest government-owned
research organization, delivering agricultural
research from ‘pasture to plate’). The participating
businesses agreed to the inclusion of an MCDA
process to assist with the decisions they were
considering. Using real cases ensured that our
results reflected experiences and feedback from
people in genuine decision-making situations
J. Multi-Crit. Decis. Anal. 16: 39–53 (2009)
DOI: 10.1002/mcda
 APPLICATION OF MULTIPLE CRITERIA DECISION ANALYSIS 41

Table I. Description of the case studies

Agribusiness (AB) Farm business—Beef (FB) Farm business—Taupo (AE)

Problem Identify best strategy for objective
carcass measurement technology use
People Operations managers (OM) for
involved procurement, processing, and mar-
keting. Senior managers (SM) had
input at an initial and final group
meetings
Context Part of another project. OM were
told by SM to participate and come
up with a strategy. OM were new in
their positions
Time Limited. Asked OM to put some
time in outside meetings with lim-
ited success. Two group meetings.
2 h individual meetings with OM.
Half-hour evaluation
Understanding Problem and technology not clearly
understood (not realised initially)
What was Identify the best strategy. OMs to
specified as take ownership of the problem and
wanted? decision outcome (SM requirement)
Identify alter- None pre-defined. Report with
natives economic evaluation done. Some
group discussion. Identified by OM.
Identify Identified by OM. Value tree ap-
criteria proach using revenue, cost and risk
branches.
Score Subjective scores
performance
Weight criteria Various (swing weightings, trade-
offs, point allocation, ranking)
Process Survey. Interview OM. Observa-
Evaluation tions.
Identify the best beef policy (beef
cow farm and a finishing cattle farm)
Mentor group assisted in identify-
ing alternatives (2 meetings). Man-
ager (cows) and farm owners
(finishing) completed the analysis
Agreed to include MCDA in an-
other project with farmers wanting
to change beef policy. Industry
monitor farms
Less restricted—mentor group as-
pects part of another project. Re-
stricted to 2 h for individual
sessions, plus 1 h for evaluation
Problem and alternatives readily
understood
Select a policy or reinforce current
choice
Explore decision and criteria.
Evaluate quantitative decision-
making process
None pre-defined. Brainstorm.
Some identified by mentor group,
some by farmer.
Identified by farmers. Some mentor
group suggestions. Brainstorming.
Some subjective scores. Gross
margin measure provided.
Swing weightings
Interview or written response from
managers. Observations.
Identify the best farm system for
Lake Taupo catchment properties
4 Maori groups (trustee, manager
and/or consultant). 3 individual
farmers (couples or individuals). 4
researchers
2 Environment Waikato staff
Part of the research with farmers in
the catchment evaluating alterna-
tives they may need to adopt in
future
Limited. Half day group meeting to
identify criteria and alternatives.
Restricted to 2 h for individual
sessions, including evaluation
Problem and alternatives readily
understood
Identify and evaluate alternative
land uses. Understand what criteria
are important. Understand MCDA
process. Help out.
14 alternatives, most pre-defined by
Taupo researchers. Group sugges-
tions added. Worked in groups.
Defined at group meeting (14 cri-
teria). Worked in groups, then
larger group.
Some subjective scores. Gross
margin and nitrogen leaching mea-
sures provided.
Swing weightings
Survey. Observations during pro-
cess.

(Eden, 1995; Montibeller, 2007). However, this also
meant the decision makers were in control—
something that Eden and Ackermann (1998) note
as being important in dealing with clients. Hence,
effectively assisting the clients with their decision
making (i.e. their requirement) had to be as high a
priority, if not higher, than the research objective of
evaluating MCDA. We also faced the same
limitations as those planning and facilitating
MCDA for commercial clients, particularly with
regard to participants’ time availability and
willingness to cooperate (in part dependent on
their perceived personal gain from the process).
Barcus and Montibeller (2008) and Montibeller
(2007) discuss methods for the evaluation of
MCDA using action research. Our research was
action-led by necessity, but the incorporation of
some more formal analyses was possible allowing
some general theory building. The participation of
the researchers in the process and the inclusion of

Copyright r 2010 John Wiley & Sons, Ltd. J. Multi-Crit. Decis. Anal. 16: 39–53 (2009)
DOI: 10.1002/mcda
42 A. E. DOOLEY ET AL.
method selection by the researchers as part of the
case studies contributed to this being action
research (Massey and Hurley, 1999; Montibeller
2007).
Evaluation methods were largely dictated by
the clients’ time availability and differed between
the case studies (Table I). Note that the number of
people was too small for the quantitative data
(surveys) to be statistically significant. The case
studies are described next.
2.1. Agribusiness case study (AB case study)
A large meat processing company wanted to
identify the strategy to best utilize a new objective
carcass measurement technology developed in con-
junction with AgResearch. Operations managers for
procurement, processing, and marketing were dele-
gated the task of selecting a strategy by senior
management. They were to each consider the same
decision, but focus on their own area, although
the final strategy selected would be a senior
management decision. The operations managers
were reluctant participants in that they wanted
nothing out of the process other than a decision to
present to senior management. Senior management
wanted to include operations managers in the
process to get them to take some ownership and
responsibility for the decisions regarding implemen-
tation of the technology, and ensure their commit-
ment to the decision.
The operations managers had been at
the company for varying periods of time
(6 months–10 years), and had different base
levels of knowledge and skills. They worked
together effectively and were accustomed to
making frequent decisions about operations, but
had not been involved previously in strategic
decision making. The technology was new and
unfamiliar to most of the managers, although
some had been involved with the project to
develop the technology. Despite being provided
with a report on the technology use and the
economic benefits of different strategies, it became
apparent during the process that they had not
understood the technology.
There was an initial group meeting with
operational and senior managers to discuss
different strategies and criteria. The researchers
then worked individually with the operations
managers to evaluate the strategies for use in
their area. They identified the criteria important to
them in their area. Scoring was subjective.
Individual meetings were followed by a group
Copyright r 2010 John Wiley & Sons, Ltd.
meeting (operational and senior management) to
evaluate technology use across all areas and come
up with a combined overall strategy. As the
agribusiness case study was conducted with a
commercial company, the decision itself cannot be
discussed in any detail because of confidentiality
issues. However, learnings from the application of
MCDA can be discussed.
2.2. Beef case study (FB case study, i.e. farm
business)
Meat and Wool New Zealand use farmer levies to
fund research, extension and marketing activities
in the sheep and beef farming sectors. This
includes funding monitor farms which are com-
mercial farms whose owners have agreed to work
with researchers to investigate some aspect of their
farming system. These farms act as showpieces and
focal points for discussion on what can be done or
how systems can be improved, predominantly for
local farmers, although articles are also written in
the NZ farming press and included on their
website and publications.
AgResearch scientists were working with eight
monitor farms throughout New Zealand selecting
the most suitable beef cattle policies. Each farm
had a mentor group of 10–15 local farmers and
agribusiness people who provided ideas and
opinions, e.g. alternatives. The ultimate decision
on what to implement remained with the farm
owners. For this research, two of these monitor
farms (one beef cow policy decision, one fattening
cattle policy decision) agreed to incorporate an
MCDA component to assist them in making their
decision.
MCDA integrated well into this project.
Discussion of MCDA, and identification of
alternatives and some initial criteria were
incorporated into a mentor group meeting. The
researchers then worked with the two individual
case study farmers (one a farm manager, the other
a couple farming in partnership) to quantitatively
evaluate the alternatives. While the mentor group
and researchers had made some suggestions, the
owners identified and used criteria (including
subjective criteria) that were important to them.
Most of the scoring against the criteria was also
subjective and reflected their opinions. This
MCDA approach was liked by the mentor
groups and participating farmers, and was
subsequently extended to seven of the eight
monitor farms (Smeaton and Dooley, 2006;
Meat, 2007).
J. Multi-Crit. Decis. Anal. 16: 39–53 (2009)
DOI: 10.1002/mcda
 APPLICATION OF MULTIPLE CRITERIA DECISION ANALYSIS
2.3. Taupo case study (AE case study, i.e.
agricultural–environmental)
Lake Taupo is New Zealand’s largest lake. There
is concern that nitrogen contamination of Lake
Taupo is leading to slow deterioration in the lake’s
water quality. One of the main, manageable
sources of nitrogen impacts to the lake is nitrogen
leaching from agricultural land. Consequently,
farmers in the Lake Taupo catchment are faced
with specific legislation from the local council
(Environment Waikato) to stop any further
increase in nitrogen leaching, with nitrogen leach-
ing outputs for farms in the catchment around the
lake to be capped. The farms affected are
predominantly sheep and beef farms. Some are
privately owned owner-operator properties, while
others are tribal land owned by Maori trusts and
incorporations (most of which cannot be sold).
AgResearch has been working with these
farmer groups to evaluate environmental impacts
(potential nitrogen leaching) from different
farming systems. However, if systems are to be
adopted they have to be acceptable to farmers.
Criteria such as profit, risk, and lifestyle are also
important to farmers in selecting between farm
enterprises. Therefore, it is important to
understand what farmers consider when
developing alternative systems or new
technologies to reduce, or maintain nitrogen
outputs. MCDA was used to identify and
compare what some Lake Taupo stakeholders
(including environmental council members and
scientists) considered important in selecting
between farming systems and technologies to
reduce nitrogen pollution in a capped nitrogen
environment. While this was not a ‘real’ decision in
the true sense of the word, the researchers and
farmers were working together on developing and
evaluating different technologies and systems,
which the farmers in this area will have to
consider adopting within the next few years to
comply with the capped nitrogen leaching
environment. This work continues.
Comparisons were made between individuals
and groups (privately owned, Maori Trusts,
researchers, council staff) in criteria, criteria
importance, scores, and farm system rankings.
A comparative group approach was taken (Belton &
Pictet, 1997) allowing diversity between decision
makers in criteria, weights, and scores to be
identified. A common set of agreed alternative
farm systems and criteria were used, but weights,
and most scores or measures were individually
Copyright r 2010 John Wiley & Sons, Ltd.
43
assessed by the participants. A group meeting of
stakeholders was initially held to identify criteria,
followed by individual meetings with participants to
score a set of farm systems against the criteria and
then weight the criteria to identify the best systems.
There were 14 alternatives and 14 criteria. An
attempt was made to develop a constructed scale
for some criteria but participants found this difficult
to agree on and expressed a preference for subjective
scoring. Time available was limited, so participants
could reject outright an alternative stating a reason,
or ignore criteria (zero weight) that were not
important to them in order to reduce the number
of assessments required. Subjective weighting and
scoring meant that direct comparisons between
people in their assessments could not be made.
However, the alternative rankings and reasons they
rated well or otherwise could be compared. This
study is discussed in depth in Dooley et al. (2005).
3. MCDA METHOD SELECTION AND
EVALUATION
3.1. MCDA method selection
In this paper we adopt the definition of MCDA as
a process (Belton and Stewart, 2002), rather than
just the aggregation procedure used. MCDA
approaches differ in the way the information on
alternatives and their performance, and criteria
and their relative significance, are elicited, speci-
fied, and analysed (DTLR, 2001; Belton &
Stewart, 2002). However, the generic process
described below, is similar for many approaches.
1. Identifying the decision problem, context, and
those involved.
2. Identifying alternatives.
3. Identifying criteria (what is important in mak-
ing the decision).
4. Quantifying the importance of the criteria
(weights).
5. Quantifying the performance of the alternatives
on the criteria (measures or subjective scores).
6. Ranking the alternatives or identifying the best
alternative.
The most appropriate approach or methods
for a problem is dependent on decision-maker
requirements, method requirements, and pro-
blem characteristics (Tecle, 1992; Hanne, 1999).
A comprehensive literature review of MCDA was
undertaken to identify the strengths, weaknesses,
J. Multi-Crit. Decis. Anal. 16: 39–53 (2009)
DOI: 10.1002/mcda
44 A. E. DOOLEY ET AL.
potential uses, and restrictions associated with
each of the predominant MCDA methods
(Dooley, 2003, unpublished). This spanned the
MCDA, social sciences (including decision
sciences), operations research and mathematics,
and agriculture and environment literature. Key
factors that needed to be considered in matching
MCDA methods to the decision under
consideration were identified and incorporated
into a descriptive framework to assist in selecting
the most appropriate MCDA approach and
methods for a given problem. The most
appropriate methods to use at different stages
can become increasingly evident as the MCDA
process progresses, making this an on-going
process.
While all the three case studies were from the
agricultural sector, they had very different
combinations of problem types and decision-
maker requirements, individual and group
decisions, decision makers with similar or
different objectives, and familiar versus new
technologies. The case study attributes are
described in Table I.
The following characteristics were similar for all
the case studies.
 alternatives in all case studies were discrete;
 subjective judgements were required;
 there were limitations on time available;
 it is relatively easy to use and understand;
 alternatives are ranked, identifying the best
alternative;
 it is transparent, which was an advantage
particularly for the AB and AE case studies,
although this was not specified as a requirement
beforehand;
 software was available (AB) or readily devel-
oped (FB and AE).
However, the three case studies differed in other
respects, e.g. group versus individual decision
making and interactions, understanding of the
decision, willingness to be involved, decision-
maker objectives, and alternative and criteria
identification (Table I).
The multi-attribute value theory (MAVT)
approach described in Belton and Stewart (2002)
was used for all the three case studies. The
similarities between the case studies, particularly
the top three factors, contributed to the selection
of this method for all cases. MAVT is one of the
more widely used MCDA approaches in practice.
Copyright r 2010 John Wiley & Sons, Ltd.
It is recognized as being suitable for groups and
useful for facilitating learning (Belton and Stewart,
2002). It is relatively quick, requiring fewer
comparisons than similar methods such as AHP.
Its popularity may partly be the result of many
decisions having characteristics which lend
themselves to this method. The quantification
and analysis stages of the process were similar
for the case studies, particularly the FB and AE
cases (Table I). An additive value function model
was used to combine scores and weights. Weights
(in most cases) were derived using the swing
weighting method developed by von Winterfeldt
and Edwards (1986) in their ‘simple multi-attribute
rating technique with swings’.
Logical Decisionss software was used for the
AB case study. A spreadsheet model was
developed in Microsoft Excels for the FB and
AE case study analyses. This included output
graphs to display results from the model sensitivity
analyses. The use of Microsoft Excels software
allowed these models to be left with the farmers
for further exploration.
The farm systems scientists and social scientists
involved in the projects already had a good
relationship and a sense of rapport with the
farmers and business managers, which was an
advantage in working with them to include an
MCDA component in their projects. They were
skilled in facilitation and working with farmer
groups, and assisted in working with groups and
individuals to identify alternatives and criteria
(identifying alternatives was already a component
of their own projects). These researchers used
techniques they were familiar with, such as
brainstorming, breakout groups (AE), and
ranking in this process.
3.2. Evaluation of MCDA
An effective decision support tool or process
delivers close to what it has been designed to do
(Eden, 1995). Decision makers evaluated the
MCDA process used to assist with their decision
making against their expectations and require-
ments, and any other observations they had
regarding the usefulness (or otherwise) of the
process. Key issues that need to be considered
when implementing MCDA were identified.
Evaluation methods and questions asked were
similar to those used in other MCDA and decision
support comparisons and evaluations (Evans and
Riha, 1989; Hobbs et al., 1992; Gunderson et al.,
1994; Zapatero et al., 1997; Qureshi et al., 1999;
J. Multi-Crit. Decis. Anal. 16: 39–53 (2009)
DOI: 10.1002/mcda
 APPLICATION OF MULTIPLE CRITERIA DECISION ANALYSIS
Bell et al., 2001; Lai et al., 2002). Participants were
asked for their opinions on: the suitability of
MCDA for the decision; the importance of some
of the benefits of MCDA methods; how well the
process met what they considered important in
evaluating their decision and any other benefits or
drawbacks they identified; their perceptions of
each stage of the process (e.g. effectiveness,
understandability, timeliness, and ease of use);
and whether the decision arrived at was what was
intuitively expected.
Evaluation data were collected using written
questionnaires (especially where time was limited),
questioning and observation of the participants
during the process, and interviews (Table I).
Written questionnaires required about 15 min to
complete, and primarily consisted of questions
requiring answers on a Likert scale to enable
collection of the maximum amount of data in a
limited time. A few open-ended questions
requiring written replies were asked (e.g. least-
liked aspect, best-liked aspect, what was wanted).
Interviews of half an hour to one hour were
conducted with some AB and FB decision
participants who were prepared to commit time
to do this. Numbers within the participant groups
were small limiting the use of statistical
evaluation, but trends between and within the
groups will be reported. The participants were
forthcoming in making spontaneous comments on
the process and its potential use, particularly
during and after the quantitative evaluation stage
where the facilitator/researcher worked with one
to two participants at a time. This added rich
information to the results. Triangulation of
methods (survey, interviews, observation) and
researchers assisted in comparing and validating
data (Patton, 1990).
4. EVALUATION OF THE MCDA
PROCESS
4.1. Case study contrasts and similarities
Differences in planning and implementation, en-
vironment or circumstances, and people factors
contributed to the way participants viewed the
process. Some differences and similarities between
the case studies which affected the success of the
decision processes used are described in Figure 1.
The case study types are shown in brackets with the
case where this situation was most advantageous
(or least disadvantaged) presented on the left.
Copyright r 2010 John Wiley & Sons, Ltd.
45
Environment or context
• time limitations were stipulated in advance for all case studies, although
some were less restrictive than others (FB>AE>AB);
• the case studies were able to be incorporated within or alongside another
project (FB>AE>AB);
• the decision was an “individual” decision rather than a group decision
requiring less co-operation between decision makers (FB=AE>AB);
• greater flexibility was possible e.g. the decision maker was able to decide
on some aspects of the process to best suit their way of thinking
(FB>AE=AB);
• the alternatives were easy for the participants to arrive at (AE>FB>AB);
• the criteria were easy for the participants to decide on (AE>FB>AB);
People
• participants were willing to be involved, approached the process in a
positive manner and wanted to make a change (FB>AE>AB);
• the participants were easy for the researcher-facilitators to relate to
(especially initially), making it easier to determine how best to
implement the process (FB=AE>AB);
• the decision makers were quantitative people making it easier for them to
relate to the MCDA approach (AB>=FB=AE);
• decision makers were accustomed to making decisions similar to the one
they were evaluating (FB=AE>AB);
Planning and implementation (may also relate to the context)
• there was good understanding between those facilitating, which
contributed to a more confident and effective delivery overall
(FB=AE>AB);
• The decision makers knew most of those involved in facilitating the
MCDA process (same for all);
• the decision required was understood by all those involved e.g. decision
makers, facilitators (FB=AE>AB);
• the technologies or alternatives were understood by all those involved
(FB=AE>AB);
• the process was understood by all those involved (FB=AE>AB);
• the analysis process (and software) was interactive, and relatively
straightforward to understand and use to help facilitate the process (same
for all);
• the outputs were designed to be easy for the decision makers to
understand and included graphics to display the results (same for all).
Figure 1. Differences between the case studies.
4.2. Expectations and benefits
The required outcomes wanted by the decision
makers varied, and differed between, and within,
case studies (Table I). These outcomes were
achieved in some cases, and overall most partici-
pants were relatively satisfied with the results. The
AB case study decision had still not been finalized.
Some AE participants had hoped to learn more
about possible alternatives.
The benefits identified in this study have also
been identified in previous MCDA literature. The
key benefits identified in these studies were
learning and understanding, a structured process,
and transparency. Participants identified benefits
from MCDA other than those they initially
specified as being important (Figure 2). Many of
the benefits were identified after the process, rather
than prior to the process. While the first three
unspecified benefits (understanding, structured
process) were objectives for some participants,
they were identified by others as specified benefits.
Where a consensus group decision was required
J. Multi-Crit. Decis. Anal. 16: 39–53 (2009)
DOI: 10.1002/mcda
46 A. E. DOOLEY ET AL.
Unspecified benefits from working through the process which participants
identified included:
• a greater understanding of their own decision making e.g. what was
important to them in decision making;
• a better understanding of others’ perspectives;
• a structured, quantitative means to work through a decision;.
• quantification which contributed to keeping the decision process
objective;
• a means of documenting the decision;
• an objective decision process which provides a more transparent means
to explain the decision or stimulate discussion of alternatives (e.g. to
trustees, mentor group, senior managers);
• changing the way they thought about their decision making;
• validation that it was acceptable to consider non-profit criteria;
• a better understanding of where their area fits in, in the overall picture;
• getting together to discuss the decision, share ideas and come to an
agreement;
• a better understanding of the decision or alternatives, sometimes as a
result of identifying that more information was required and seeking
that information.
Figure 2. Benefits participants identified from using the
MCDA process.
(AB case study), ranking, understanding others’
perspectives, understanding which criteria have the
biggest impact and giving participants a sense of
ownership were more important than they were for
the more individual decisions (AE case study)
where participants were more interested in
exploring the decision.
MCDA is often regarded as a process or
quantitative tool to assist with decision making
by identifying the best alternative(s). Ranking
alternatives or identifying the best alternative
was specified as an objective by the AB
participants. This was less important in the FB
and AE case studies, particularly the AE case
where the decision was more futuristic. Despite
this, no-one appeared to perceive the MCDA
process as necessarily being expected to identify
the best alternative. They were not concerned if the
alternatives ranked differently from their intuitive
expectations, although their intuitive preference
was usually in their top two alternatives. Instead,
an unexpected ranking led them to question their
thinking on the decision, and to explore further to
ensure whether their inputs really reflected their
views. That is, is the model requisite? (Phillips
1984). Their final decision was not necessarily the
top-ranked alternative. This suggests that concern
over accuracy of MCDA results, which is often
raised in the literature, is not as important as some
other factors in implementing MCDA, e.g. time
available, ease of use.
The participants in these case studies were
more interested in exploring the decision to
understanding which criteria (scores and weights)
Copyright r 2010 John Wiley & Sons, Ltd.
have the biggest impact on rankings, than in
challenging the ranking itself. In some cases
decision makers in the FB case study accepted
that the top-ranked alternative was not their
intuitive preference, and even agreed after further
exploration that the rankings and the analysis were
correct. This led to one FB participant re-
evaluating his current decision-making process
(i.e. questioning whether it was too emotional
and determining to be more objective in his
decision making going forward).
While learning per se is not generally seen to be
the purpose of MCDA, it has been identified in the
literature as a key benefit of MCDA (Belton and
Stewart 2002). Almost all the case study
participants indicated that they had benefited
from some aspect(s) of learning, whether they
specified this as an objective or not (i.e. the
decision and technologies, criteria and their
importance, their own and others’ perspectives).
The impact on participants’ learning was also
reflected in the terminology they used to discuss
MCDA, i.e. words such as learn, explore, and
understand were all used, often synonymously.
Hence, these case study results strongly support
learning as the key benefit of an MCDA analysis.
The participatory nature of the process was
beneficial for both group and individual decision
making (although it should be recognized that
even the ‘individual’ decision making often
included more than one person). The greater
understanding of the decision, and their own and
each others’ perspectives gained through working
together during the process was particularly
important in the AB case studies where a group
consensus was required. However, even in the
other case studies, participants who worked in
groups for some aspects commented that their
increased understanding of what was important to
others stimulated their own thinking. They
enjoyed this aspect of the process.
A means of structuring the problem, allowing
participants to make more thorough decisions and
identify what was important in their decision
making, was regarded as desirable, and was
specified as a required outcome by some. The
case study participants liked the structured
approach, finding it helped them to think through
and understand the decision, confirming what is
recognized in the literature, i.e. that the structured
MCDA process provides greater understanding
(own and group), objectivity, and transparency
(Belton and Stewart, 2002). This structured process
J. Multi-Crit. Decis. Anal. 16: 39–53 (2009)
DOI: 10.1002/mcda
 APPLICATION OF MULTIPLE CRITERIA DECISION ANALYSIS
contributed to identifying gaps in peoples’
knowledge. For example, it was realized that the
technology was not understood when working
through the process in the AB case study
prompting further learning about the technology:
it could be argued that this recognition and
learning about the technology was the greatest
benefit from MCDA in this case study.
Most participants were accustomed to working
with numeric data and preferred structured or
semi-structured decision-making processes. This
may have meant the structured, quantitative
MCDA process was easier for them to use than
some people would have, and may have influenced
their opinion of MCDA, which was largely
positive. Hence, MCDA may be more useful
with similar types of people. However, many
people in the farming community would be
similar to our case study participants, suggesting
MCDA could be applicable for agricultural
business decisions.
Some decision makers observed that the
objectivity of the structured process had the
potential to reduce the opportunity for the decision
to be swayed by some individuals. While most
considered this to be important, three participants
commented that sometimes it is necessary for
an individual to step in and make the decision:
while it is important in a group decision to have a
decision-making process where all can contribute,
leadership is also required. The literature also
supports this.
The objective, quantified and structured MCDA
process provides a potentially transparent way to
explain the decision to others (Belton and Stewart,
2002). This was also recognized in hindsight by the
case study participants who were accountable to
others or were involved in group decisions (e.g.
couples; managers or consultants answerable to
senior management or boards of trustees, corporate
farm trustees).
4.3. Challenges in implementing the MCDA process
Possibly the greatest learning from this study was
in relation to the implementation of MCDA,
particularly with the AB case study which was
more difficult than the FB and AE case studies
(Figure 1). The decision context, people involved,
the process used and its implementation all affect
how smooth a decision-making process will be.
The main difficulties experienced in these case
studies are related to time, understanding, and
ownership. Our inexperience in implementing
Copyright r 2010 John Wiley & Sons, Ltd.
47
MCDA no doubt contributed to some of the
difficulties experienced.
The decisions reported here were all strategic
business decisions (agribusiness, farm business)
where ‘time is money’. Time limitations were
specified for all three case studies, and were a
key factor in MCDA method selection and
implementation. These time limitations affected
what was able to be achieved. However, more
thorough, accurate but time-consuming methods
are unlikely to have increased client satisfaction—
and may in fact have had the opposite effect. In
contrast, reported MCDA case studies are often at
the policy level, where processes are considered
more, with greater emphasis on ensuring the
involvement and understanding of a wide range
of participants. Time may be less of a limitation
relative to other requirements in these studies.
Those working through the decision in the FB
and AE case studies agreed to commit time to this
process. However, in the AB case study senior
managers committed operations managers to
working through the process. This created some
problems. The operation managers commented
that senior managers should have recognized that
they needed more time available to work on the
decision and stated this would have resulted in
them being more willing to participate and
achieved better results. This demonstrates that
company situations require a commitment
(including time) to the decision and the process
at higher levels within the company, as well as at
the level the decision is being made. Schein (1999)
made similar observations and recommends not
getting involved in facilitating a decision unless the
decision maker or their company is prepared to
make a commitment up front, preferably during a
paid meeting to ensure they are serious.
Participants in the AB case study had
difficulties in understanding the decision, and the
new and unfamiliar technology, demonstrating
that time needs to be allowed for early on in the
process, including during planning, to ensure there
is a common understanding (by facilitators and
planners as well as participants) of the decision
and technologies. While some prior information
was available, not all participants had read or
understood this. This affected their ability to
understand and assess its impact. These problems
in understanding were exacerbated by lack of time
for explanations early on, and the assumption that
people already understood the technology. When
this lack of understanding was realized part way
J. Multi-Crit. Decis. Anal. 16: 39–53 (2009)
DOI: 10.1002/mcda
48 A. E. DOOLEY ET AL.
through the process, further information was
required. Thus, time ‘saved’ early on in the
process, did not save time in the long run! More
time is required where proposed alternatives or
technologies are new. In contrast, the FB and AE
case studies were more successful than the AB case
study. Factors contributing to this included: the
decision process fitted more closely with on-going
work; the process was explained more clearly;
more information and time was available for the
researchers to learn about the context and get to
know those involved; and the participants were
already familiar with many of the alternatives.
Differences between people need to be
understood and allowed for when planning and
implementing the process as these can contribute
to differences in their understanding. Even while
working with a company decision such as in the
AB decision where those involved had similar
overall objectives, there were differences in their
understanding, objectives and perspectives because
of their differing backgrounds, experiences,
responsibility levels, and personalities.
The decision process and the need to be
accountable acted as a catalyst in the AB case
study resulting in most managers having a greater
sense of ownership of the decision at the end of the
process. While senior management in these case
studies wanted the operations managers to take
some ownership and responsibility for the
consensus group decision, the operations
managers were reluctant to be involved. They
were busy, were not committed but had to
participate, were unaccustomed to this level of
decision making, or saw no reason not to do things
the old way (senior management made decisions at
this level). This created some difficulties getting
them to actively participate in the process initially.
However, the later realization that they did not
understand the technology sufficiently to make a
decision to present to their managers resulted in
these operations managers taking initiative
between formal meetings to discuss the problem
between themselves. In this respect the MCDA
process was a success: one of the earliest signs of
success in getting commited to a change occurs
when external consultants find there has been
increased management of the decision by a group
through informal debate and discussion between
convened meetings (Mayon-White, 1990).
However, it should be noted that any structured
and interactive decision-making process could
have had the same effect as MCDA.
Copyright r 2010 John Wiley & Sons, Ltd.
The need for trust can affect method selection
and facilitation. This requirement for trust may be
more important with sensitive decisions (AE case
study) or agribusiness decisions (AB case study)
where information may be less readily available,
e.g. internal company information, people issues
within the company. Case study identification, an
understanding of the problem context, and gaining
the decision makers’ trust was achieved by our
working with AgResearch staff who already had
established relationships with the company or
farmer decision makers. AgResearch facilitators
probably had the advantages of both internal
facilitators (understanding of company and people
involved) and external facilitators (neutrality)
(Mayon-White 1990). While the methods used
had to suit all those involved, including the
AgResearch staff, the case studies would have
taken considerably longer had these AgResearch
staff not been involved.
4.4. Understanding and using the MCDA process
Participants found the qualitative aspects of the
process (understanding the problem, identifying
criteria and alternatives) more difficult than the
quantification stages of the process. The problem,
technology, and process were not clearly under-
stood by the AB participants. However, the
process used contributed to identifying that there
were misunderstandings about the technology.
These aspects will be discussed further later. The
AB participants had to identify both the criteria
and alternatives, and they disagreed or were
neutral as to whether this was easy to do. The
technology was new, and they had less time and
help available to assist with this than the other case
study participants. Managers who had put extra
time into this found it easier.
In contrast, most FB and AE participants
found the information on the problem, process,
and alternatives (AE) to be adequate. FB decision
makers had the assistance of a mentor group to
assist with alternative and criteria identification,
and more familiar technologies to consider. Even
so, it still took considerable time and input to
arrive at a set of alternatives and criteria, although
one FB participant noted that it was still quicker
than if they had done this without mentor group
assistance.
The majority of the time spent on the MCDA
process was in understanding the decision and
identifying criteria and alternatives. The time
required for this was underestimated, particularly
J. Multi-Crit. Decis. Anal. 16: 39–53 (2009)
DOI: 10.1002/mcda
 APPLICATION OF MULTIPLE CRITERIA DECISION ANALYSIS
in the AB case studies. In the FB and AE case
studies, these stages were better integrated into
other ongoing activities and the technologies were
more familiar.
Once the process was explained, most
participants had little hesitation about allocating
weightings and scores and accepting the results.
The most difficult aspect of the quantitative
analysis identified was understanding the swing
weighting method. Once explained, most people
agreed it was easy to use. Where people worked
together on weighting and subjective scoring (e.g.
farming couple, two managers, manager and
trustee) they largely agreed on weights and
scores, and did not appear to have any difficulty
coming to a compromise when there was initial
disagreement.
Questions were not raised about the methods
themselves, although there was considerable
discussion about the results and further analysis
in most cases, e.g. revision of the weights and
scores. It was interesting to observe that they spent
as much time reviewing and revising their inputs
and assessing the impact these had as they had on
setting up the model. This was despite the fact that
this was optional and they had specified that there
were time constraints! Previous findings report that
the visualization of results is important in assisting
peoples’ learning (Belton and Stewart, 2002) and
can support sensitivity analysis. This was supported
in this study where participants found the visual
presentation of the analysis results (graphs) and the
interactive approach helpful, contributing to their
learning and interest in the results.
Most case study participants would be pre-
pared to use the process again, with 5 out of
14 AE decision makers preferring it to their current
decision-making process (eight were neutral, one
disagreed). Not surprisingly, participants who were
more willing to be involved were more positive.
Some of the AB participants (operations managers)
who had no choice about participating, and AE
participants who were only ‘doing this to help’
perceived the process as being less useful or would
not want to use this process to assist in future
decision making. Participants considered the
process could be useful for future decision making
for themselves and for other agricultural decision
makers, with one participant commenting that he
thought the process was likely to be considered
useful for farm decision making by about 50% of
farmers, i.e. those who were ‘strategic managers
looking at farming as a business rather than a
Copyright r 2010 John Wiley & Sons, Ltd.
49
lifestyle’. They noted that it would be necessary to
have some guidance the first time it was used,
particularly in weighting criteria.
5. THE POTENTIAL FOR MCDA IN FUTURE
AGRICULTURAL DECISION MAKING
IN NEW ZEALAND
The findings in this study suggest there is potential
for MCDA to be used in agricultural decision
making in New Zealand to help balance economic,
environmental, and social or lifestyle goals in
strategic decision making. Agricultural and envir-
onmental organizations, including AgResearch,
are well aware of the increasing need to trade-off
agricultural and environmental goals. There is on-
going work looking at trade-offs, ways to bring
about change, likely farmer responses to initiatives,
farmer decision making, and new alternatives,
which may need to meet a range of criteria to be
adopted. Currently much of this work is at the
policy level and there is interest in MCDA methods
to assist. There has also been other farmer-targeted
work as a result of this MCDA study.
 The selection framework developed was used to
help determine the best MCDA method for
further work undertaken at AgResearch. This
included an outranking approach for use in a
decision support tool to help farmers select
forages (Snow et al., 2005) and a compromise
programming approach for water allocation.
 As previously mentioned, the scientist in charge
of the beef monitor farm project used MCDA
for five other beef monitor farms because it
integrated well with what they were doing, and
he was impressed with the process noting that
farmers liked it too. This work has been
reported at a farmer conference (Smeaton and
Dooley 2006) and written up in a Meat and
Wool R&D brief (Meat, 2007, Figure 3).
While the case study participants agreed (some
reluctantly) to use a MCDA process for their
decision making as part of their involvement in
another project, it is unlikely they would have
agreed to use this otherwise. Yet most found the
MCDA process to be more useful or interesting
than they expected with some key benefits
(learning, increased understanding, a structured
and transparent process) generally not recognized
until after the process had been used. Hence, the
J. Multi-Crit. Decis. Anal. 16: 39–53 (2009)
DOI: 10.1002/mcda
50 A. E. DOOLEY ET AL.
Figure 3. Meat and Wool R&D brief reporting the use
of MCDA to help farmers make beef policy decisions.
challenge is to identify how to best promote or
implement MCDA methods to encourage people
to consider using them when applicable. Given the
time commitment required and the need to step
outside their comfort zone, it is likely to be
difficult to get people to consider using these
methods unprompted. Reporting of MCDA
applications and associated benefits in industry
forums and publications could alert people to the
existence of these structured, quantitative methods
to help assist decision making.
Once people have participated in MCDA and
recognize the benefits they may be more likely to
use this again. However, the resources (tools,
skills, expertise) need to be available. Farm and
business consultants may be well placed to offer
MCDA as part of their toolkit, e.g. to assist clients
with exploring alternatives and helping with the
decision once alternatives have been identified and
some measures calculated. A concern with this
approach is that lack of expertise may result in
results that are less accurate than they could be.
However, it is uncertain how important this is, i.e.
Copyright r 2010 John Wiley & Sons, Ltd.
our results suggest that the structured process and
learning associated with MCDA is more important
than the rankings, and that the decision makers
were not overly concerned if their preferred
alternative was not top.
The quantitative nature of MCDA results
means the outcome of the decision-making
process is explicit. This may be of concern to
people who feel there is potential for results to be
misinterpreted and misused by others. In New
Zealand, there is a perception by some that
farmers do not care for their environment.
Consequently, farmers around Lake Taupo were
sensitive to the possibility that their results could
be misinterpreted to reinforce the views of these
people. These types of concerns need to be
considered in sharing the results of an analysis
with people other than the participants. It also
highlights the importance of making the following
factors clear in presenting and discussing results.
 Some criteria are a requirement rather than
being traded-off. For example, in the Taupo AE
study, the weighting on environment was low
for some farmers because a minimum level of
environmental performance was expected be-
fore an alternative was even considered, rather
than trading environmental criteria off against
other criteria. Above this acceptable level,
improvements in environmental performance
were small and not high priority, with some
participants noting that they already contribu-
ted to improving environmental impacts. How-
ever, this did not mean that the environment
per se was of low priority.
 There can be differences between peoples’
weights because of their different perceptions
or alternative sets, e.g. the Taupo AE case
study. Care needs to be taken in comparing
MCDA results between people, particularly
where evaluations are subjective rather than
measured. Ranges between the best and worst
possible performing alternatives on the criteria
will affect peoples’ weights and scores. Simi-
larly, results only apply to the set of alternatives
being evaluated, or alternatives that perform in
the same range as these systems: another
decision or set of alternatives may well require
different criteria and weightings.
The MCDA approach used in this study could
be useful to help identify systems or technologies
J. Multi-Crit. Decis. Anal. 16: 39–53 (2009)
DOI: 10.1002/mcda
 APPLICATION OF MULTIPLE CRITERIA DECISION ANALYSIS
farmers might adopt, and factors influencing their
decision. Although the number of participants in
the AE case study was small and results cannot
necessarily be considered applicable to all farmers
within the catchment, the wide variation in results
(Dooley et al., 2005) demonstrated the diversity in
farmer perspectives suggesting a range of different
farm systems will be needed. Our results also
found that some of those involved in the
development and selection of agricultural systems
and policy do not understand farmers’ perspectives
as well as they could. If policy people and
researchers want to facilitate change in farming
practices then it is essential that they understand
what is important to those who have to make the
change. MCDA could contribute to this.
6. CONCLUSIONS
The range of criteria involved in agricultural
decisions and the benefits identified in these case
studies suggest that MCDA can be helpful for
assisting with agricultural and agribusiness deci-
sion making. Most case study participants believed
MCDA would be useful for agricultural decision
making. The process was judged to be useful by
most participants but could have been better
implemented. Most would be willing to use it
again suggesting that the MCDA decision process
was appropriate: Eden (1995) noted that a decision
process that is liked by users and achieves what it
is supposed to demonstrates credibility and ade-
quacy. Participants recognized many of the bene-
fits only after being involved in the process. They
identified learning about the decision and technol-
ogies; what was important to them; and others’
perspectives as the key benefits. The structured
decision process provided an objective means to
work through the problem, and a potentially
transparent way for them to explain their decision
to others. The MCDA process contributed to AB
managers being more committed to the decision.
The key problems experienced in these case
studies included: insufficient time to complete the
decision process properly; difficulty in understanding
the problem, decision process, and technology (AB);
and decision makers unwilling to take responsibility
for the decision (AB) or not committed because they
felt they had little to gain from participating (AB,
AE). These problems were associated with
implementation and constraints on what was
achievable, and were inter-related. Most of these
Copyright r 2010 John Wiley & Sons, Ltd.
51
problems were encountered in the initial stages of the
decision process and could apply to any formal
decision process, particularly those involving groups.
While these issues are not discussed in much of the
MCDA literature, they have been identified as being
important in the wider decision making and
consultancy literature. Personal preferences and the
decision makers’ willingness to be involved
accounted for some of the variation in their
opinion of the decision process, with those who
were more willing to participate being more positive.
Planning and implementation could have been
improved particularly for the AB case studies. The
time, understanding, and ownership problems
largely occurred in the initial stages of the process,
and the opinion was expressed that these problems
were likely to have occurred with any formal or
group decision-making process. This concurs with
Schein (1999), who notes that most mistakes in
consultation tend to occur at the beginning.
Similarly, many of the benefits identified could
have resulted from any formal, structured or semi-
structured decision-making process.
Key points identified by this study, which need
to be considered in planning and implementing
MCDA in future include the following:
 Ensuring that decision makers or their company
are prepared to invest time in a decision. Those
involved should be informed of the extent of
their involvement and have adequate informa-
tion before the decision process commences.
Extra time needs to be allowed if the technology
is unfamiliar.
 Understanding the problem context before the
decision process is planned. This may be more
difficult with company agribusiness decisions
where access to information may be more
limited for facilitators or analysts.
 Understanding as much as possible about the
decision makers (attitudes, experience, skills,
reason for participating) in the planning phase.
 Ensuring that the problem, decision process,
technologies, and any pre-defined decision alter-
natives are understood by all decision makers,
and to some extent by facilitators and analysts at
the start. Do not assume these are understood.
 Recognizing the importance of establishing the
trust of the decision makers (and company if
applicable), and working with contacts to ensure
this if need be. This may require trade-offs between
working with those trusted by the decision makers,
methods used and facilitator skills.
J. Multi-Crit. Decis. Anal. 16: 39–53 (2009)
DOI: 10.1002/mcda
52 A. E. DOOLEY ET AL.
 Appreciating that considerable time and com-
mitment may be required to identify criteria and
alternatives, especially where these are not pre-
defined or when dealing with unfamiliar tech-
nologies.
 Being flexible in implementation and prepared
to revise the plan if required or if it is likely to be
advantageous. Some factors may not be under-
stood or may not arise until the process is
underway.
 Realising that not everyone will necessarily be
happy with the decision process and possibly
the outcome because of differences in person-
ality, background, objectives, and expectations.
Research by Barcus and Montibeller (2008) also
identified some of the above: problem structuring
was critical; the need for commitment and trust;
time required and the need to commit to this for best
results; learning through interaction with the model.
MCDA has the potential to be useful for future
agricultural decision making. While MCDA is
often seen as a process or tool to help identify the
best decision, it has the potential to do far more
than this. While there has been considerable
research on MCDA methods, it is unclear how
to best promote or implement MCDA methods to
encourage people to use them. Benefits such as
learning need to be promoted in advocating the
use of MCDA. Promotion of MCDA may best be
achieved as part of other extension or consultation
activities by someone who has used the process.
The time and commitment required may deter
people from considering a structured decision-
making process such as MCDA. Simpler MCDA
approaches are likely to be as effective in achieving
many of the benefits (e.g. learning, stimulating
thinking, objectivity) as more complex ones, and
are more likely to be acceptable to decision makers
given time constraints. The opportunity exists for
MCDA research in these areas.
ACKNOWLEDGEMENTS
The AgMardt post-doctoral fellowship which
allowed this research to be undertaken is gratefully
acknowledged. Thanks to AgResearch for hosting
the fellowship. Thank you to Rex Webby, Ian
Knowles, Terry Parminter, Warwick Waters, and
Alan McDermott of AgResearch who helped with
the case studies. The contribution of the farmers
Copyright r 2010 John Wiley & Sons, Ltd.
and business people who participated in the case
studies, and without whom the research could
not have been undertaken, is very much appre-
ciated. This post-doctoral research was funded by
AgMardt (Agricultural and Marketing Research
and Development Trust) Grant number: 20392.
REFERENCES
Barcus A, Montibeller G. 2008. Supporting the alloca-
tion of software development work in distributed
teams with multi-criteria decision analysis. Omega 36:
464–475.
Bell ML, Hobbs BJ, Elliott EM, Ellis H, Robinson Z.
2001. An evaluation of multi-criteria methods in
integrated assessment of climate policy. Journal of
Multi-Criteria Decision Analysis 10: 229–256.
Beinat E. 2001. Multi-criteria analysis for environmental
management. Journal of Multi-Criteria Decision
Analysis 10: 51.
Belton V. 2001. The need for interaction and integra-
tion. Journal of Multi-Criteria Decision Analysis 10:
127–128.
Belton V, Pictet J. 1997. A framework for group
decision using a MCDA model: sharing, aggregating
or comparing individual information. Revue des
Systemes de Decision 6(3): 283–303.
Belton V, Stewart T. 2002. Multiple Criteria Decision
Analysis: An Integrated Approach. Kluwer Academic
Publishers: Boston.
Dooley AE, Smeaton D, Ledgard SF. 2005. Identifica-
tion of Important Criteria in Farm Systems Decisions
Around Lake Taupo. New Zealand Agricultural and
Resource Economics Society, AERU No. 152: 27–35.
DTLR. 2001. Multi-criteria Analysis: A Manual.
Department for transport, local government and the
region. Available at: http://www.dtlr.gov.uk/about/
multicritreria [Accessed on 12 August 2002].
Eden C. 1995. On evaluating the performance of ‘wide-
band’ GDSS’s. European Journal of Operational
Research 81: 302–311.
Eden C, Ackermann F. 1998. Making Strategy: The
Journey of Strategic Management. Sage Publications:
London, Great Britain.
Evans GE, Riha JR. 1989. Assessing DSS effectiveness
using evaluation research methods. Information and
Management 16: 197–206.
French S. 1998. Decision making not decision theory.
Journal of Multi-Criteria Decision Analysis 7: 303.
French S. 2000. It’s good to talky. Journal of Multi-
Criteria Decision Analysis 9: 125–126.
Gunderson DE, Davis DL, Davis DF. 1994. Can DSS
technology improve group decision performance for
end users? An experimental study. Journal of End User
Computing 72: 3–10.
J. Multi-Crit. Decis. Anal. 16: 39–53 (2009)
DOI: 10.1002/mcda
 APPLICATION OF MULTIPLE CRITERIA DECISION ANALYSIS
Hanne T. 1999. Meta decision problems in multiple
criteria decision making. In Multicriteria Decision
Making: Advances in MCDM Models, Algorithms,
Theory, and Applications, Gal T, Stewart T, Hanne T
(eds). Kluwer Academic Press: Boston, USA; 6.1–6.25.
Hobbs BF, Chankong V, Hamadeh W, Stakhiv EZ.
1992. Does choice of multicriteria method matter? An
experiment in water resources planning. Water
Resources Research 28(7): 1767–1779.
Lai VS, Wong BK, Cheung W. 2002. Group decision
making in a multiple criteria environment: a case
using the AHP in software selection. European Journal
of Operational Research 137: 134–144.
Massey C, Hurley E. 1999. Action research as a
mechanism for client-driven development. Proceed-
ings of the 43rd Australian Agriculture and Resource
Economics Society Conference and the 6th New
Zealand Agricultural and Resource Economics Society
(CD-ROM), Christchurch.
Mayon-White B. 1990. Problem solving in small groups:
team members as agents of change. In Tackling
Strategic Problems: the Role of Group Decision
Support, Eden C, Radford J (eds). Sage Publications:
London, Great Britain; 78–89.
Meat NZ. 2007. Farmer Decision Making—A Tool to Aid
the process. Available at: http://www.meatnz.co.nz/main.
cfm?id 5 44&spid 5 96 [Accessed on 1 January 2008].
Montibeller G. 2007. Action-researching MCDA inter-
ventions. In Key-note Papers, 49th British Operational
Research Conference, Shaw D (ed.). The OR Society:
Melbourne, Australia.
Patton MQ. 1990. Qualitative Evaluation and Research
Methods (2nd edn). Sage Publications: Newbury Park,
California, USA.
Phillips LD. 1984. A theory of requisite decision models.
Acta Psychologica 56: 29–48.
Qureshi ME, Harrison SR, Wegener MK. 1999.
Validation of multicriteria analysis models. Agricul-
tural Systems 62: 105–116.
Copyright r 2010 John Wiley & Sons, Ltd.
53
RAC. 1992. Multi-criteria analysis as a resource
assessment tool. RAC Research Paper No. 6, Resource
Assessment Commission, March 1992.
Roy B. 1999. Decision-aiding today. In Multicriteria
Decision Making: Advances in MCDA Models, Algo-
rithms, Theory, and Applications, Gal T, Stewart TJ,
Hanne T (eds). Kluwer Academic Press: Boston;
1.1–1.35.
Schein EH. 1999. Process Consultation Revisited: Build-
ing the Helping Relationship. Addison-Wesley: Mas-
sachusetts, USA.
Shadbolt N, Martin S. 2005. Farm Management in
New Zealand. Oxford University Press.
Smeaton DC, Dooley AE. 2006. Adoption of new
technologies or management systems on sheep and
beef farms. Proceedings of the New Zealand Grassland
Association 68: 229–235.
Snow VO, Knowles IM, Fraser TJ, McIvor S, Lovatt SJ,
Dooley AE, Finlayson JD. 2005. The M&WNZ
ForageMasters: Decision support for forage selec-
tion. In MODSIM 2005 International Congress on
Modelling and Simulation, Zerger A, Argent RM
(eds). Modelling and Simulation Society of Australia
and New Zealand, December 2005. University of
Melbourne, 259–265. ISBN: 0-9758400-2-9. Available
at: http://www.mssanz.org.au/modsim05/proceedings/
papers/snow.pdf. [Accessed on 23 June 2006]
(Modelling and Simulation Society of Australia and
New Zealand).
Tecle A. 1992. Selecting a multicriterion decision
making technique for watershed resources manage-
ment. Water Resources Bulletin 28(1): 129–140.
von Winterfeldt D, Edwards W. 1986. Decision Analysis
and Behavioural Research. Cambridge University
Press: Cambridge.
Zapatero EG, Smith CH, Weistroffer HR. 1997.
Evaluating multiple-attribute decision support sys-
tems. Journal of Multi-Criteria Decision Analysis
6: 201–214.
J. Multi-Crit. Decis. Anal. 16: 39–53 (2009)
DOI: 10.1002/mcda