The DeLone and McLean Model of Information Systems Success: A Ten-Year Update

Author(s): William H. DeLone and Ephraim R. McLean

Source: Journal of Management Information Systems, Vol. 19, No. 4, Information Systems
Success (Spring, 2003), pp. 9-30
Published by: M.E. Sharpe, Inc.
Stable URL: http://www.jstor.org/stable/40398604
Accessed: 09/12/2010 04:53
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=mes.
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

M.E. Sharpe, Inc. is collaborating with JSTOR to digitize, preserve and extend access to Journal of
Management Information Systems.

http://www.jstor.org

TheDeLoneand McLeanModelof
Information
SystemsSuccess:
A Ten-YearUpdate
WILLIAM H. DeLONE AND EPHRAIM R. McLEAN
ofInformation
William DeLone is anAssociateProfessor
SystemsandChairofthe
at theKogodSchool of BusinessatAmerican
Information
Department
Technology
areasofresearch
include
DeLone's primary
inWashington,
DC. Professor
University
of information
and value,theimplementation
theassessment
systemseffectiveness
in smalland medium-sized
and use of information
businesses,and the
technology
He has been publishedin various
of information
technology.
global management
JourDATABASE,
Research,MIS Quarterly,
Systems
Information
journalsincluding
and Journalof Information
nal of Global Information
Technology
Management,
fromVillanovaUniProfessor
DeLone earneda B.S. in mathematics
Management.
anda
fromCarnegieMellonUniversity,
administration
an M.S. in industrial
versity,
ofCalifornia,
Los
andInformation
Ph.D. inComputers
SystemsfromtheUniversity
Angeles.
Scholar's
andGeorgeE. SmithEminent
EphraimR. McLean is a Regents'Professor
of
Business
at
in
the
Robinson
Chairin Information
GeorgiaState
College
Systems
in
he was on
State
to
to
Prior
Atlanta.
1987,
University
coming Georgia
University,
Los Angeles(UCLA) for18 years.Dr.
of California,
thefacultyof theUniversity
ofinformation
focuseson themanagement
McLean'sresearch
services,thevalueof
He has publishedover 125
and careerissuesforIS professionals.
IS investments,
Research,JournalofManagement
Systems
papersin suchjournalsas Information
MIS Quarterly,
Science,Communications
ofthe
Management
Systems,
Information
andothers;
SloanManagement
HarvardBusinessReview,
Review,
ACM,DATABASE,
nowin itsthird
andhiscoauthored
book,Information
Technology
forManagement,
in theworld.Dr. McLean
thesecondlargestsellingIS textbook
edition,is currently
andhis S.M.
fromCornellUniversity
earnedhisB.M.E. in mechanicalengineering
InstiattheMassachusetts
andPh.D. degreesfromtheSloanSchoolofManagement
for
the
Association
Director
of
Executive
is
also
the
He
tuteofTechnology
(MIT).
the
of
AIS.
in
was
a
Fellow
made
and
1999
Information
Systems(AIS)
theDeLone andMcLean Information
Abstract: Tenyearsago,we presented
Systhecomplexand modelformeasuring
tems(IS) Success Model as a framework
IS
In thispaper,we discussmanyoftheimportant
variableinIS research.
dependent
of thelast decade,focusingespeciallyon research
successresearchcontributions
toouroriginal
model.
andproposeenhancements
efforts
thatapply,validate,
challenge,
tothe
we proposeminorrefinements
ofthosecontributions,
Based on ourevaluation
modelandproposean updatedDeLone andMcLean IS SuccessModel.We discuss
e-commerce
oftheupdatedmodelformeasuring
theutility
systemsuccess.Finally,
of
measurement
current
andfuture
we makea seriesofrecommendations
regarding
IS success.
/ Spring2003,Vol. 19,No. 4, pp.9-30.
JournalofManagement
Systems
Information
2003 M.E. Sharpe,Inc.
0742-1222/2003 $9.50 + 0.00.

10

DeLONE AND McLEAN

ofinformation
Key words and phrases: evaluation
impactofinformation
systems,
information
systemssuccess,servicequality,sysquality,information
technology,
usersatisfaction.
use ofinformation
temsquality,
systems,

is critiThe measurementof informationsystems(IS) successoreffectiveness

andIS
actions
of
ofthevalueandefficacy IS management
cal to ourunderstanding
tobringsome
In 1992,we publisheda paper[8] in whichwe attempted
investments.
- IS success- inIS research.
We
variable"
tothe"dependent
awarenessandstructure
"D&M
IS
as
the
to
referred
interactive
model
a
and
an
(hereafter
proposed taxonomy
success.
IS
andoperationalizing
forconceptualizing
SuccessModel")as frameworks
Sincethen,nearly300 articlesinrefereedjournalshavereferred
to,andmadeuse of,
of themodelis strongevidenceof the
thisIS SuccessModel.The widepopularity
IS researchfindings.
in orderto integrate
framework
needfora comprehensive
The D&M IS SuccessModel,thoughpublishedin 1992,was basedon theoretical
in the 1970s and
and empiricalIS researchconductedby a numberof researchers
last
decade.
the
1980s.The roleofIS has changedandprogressed
Similarly,
during
overthe
has progressed
of IS effectiveness
academicinquiryintothemeasurement
in
all thearticles Inforsameperiod.We reviewedmorethan100 articles,
including
and MIS
mationSystemsResearch,JournalofManagement
Systems,
Information
The
measurement.
success
review
of
IS
since
in
order
to
inform
this
1993
Quarterly
evaluate
Model
and
IS
is
the
D&M
Success
of
this
to
paper,therefore, update
purpose
inlightofthedramatic
itsusefulness
changesinIS practice,
especiallytheadventand
ofe-commerce.
explosivegrowth

The D&M IS SuccessModel

The primarypurposeof the original DeLone andMcLeanpaper[8] was to synthesizepreviousresearchinvolving
IS successintoa morecoherent
bodyofknowlBased on thecommunications
researchers.
edge and to provideguidanceto future
ofMaresearch
ofShannonandWeaver[43] andtheinformation
"influence"
theory
son[31],as wellas empirical
studresearch
information
(MIS)
systems
management
ies from1981-87, a comprehensive,
multidimensional
modelof IS success was
as
ShannonandWeaverdefinedthetechnicallevelof communications
postulated.
theaccuracyandefficiency
ofthecommunication
that
information.
system produces
Thesemantic
levelis thesuccessoftheinformation
meaninconveying
theintended
levelis theeffectof theinformation
on thereceiver.In the
ing.The effectiveness
D&M IS SuccessModel,"systems
success;"information
quality"measurestechnical
measures
semantic
and
user
individual
success;
satisfaction,
"use,
quality"
impacts,"
and "organizational
In
measure
effectiveness
success.
impacts"
spiteofthepassage
oftimesincetheShannonandWeaverframework
in 1949andMason'sextensions
in
1978,bothappearas validtodayas whenwe adoptedthema decadeago.

THE DeLONE AND McLEANMODEL OF INFORMATIONSYSTEM SUCCESS

11

of success
thesesix dimensions
Based on bothprocessandcausal considerations,
thanindependent.
Thishas important
rather
areproposedtobe interrelated
implicain
of
success
and
IS
tionsforthemeasurement,
empiricalstudies.
analysis, reporting
first
various
an
is
that
IS
model
A temporal,
created,containing
suggests
process
variousdegreesof systemand
as exhibiting
whichcan be characterized
features,
thesefeatures
users
and
information
Next,
byusingthe
experience
managers
quality.
or
its
information
the
or
with
dissatisfied
either
satisfied
and
are
prodsystem
system
thenimpactsor influences
ucts.The use ofthesystemanditsinformation
products
theindividualuserin theconductof his or herwork,and theseindividualimpacts
Theresultant
D&M IS SuccessModelis
resultinorganizational
impacts.
collectively
1
in
reproduced Figure [8, p. 87].
of
toa processmodel,a causal orvariancemodelstudiesthecovariance
Incontrast
ifthereexistsa causalrelationship
todetermine
thesuccessdimensions
amongthem.
and
user
satisfaction
to
to
lead
is
Forexample,highersystem
higher
quality expected
in
on
individual
to
resulting organizational
use,leading positiveimpacts
productivity,
with
thesuccesstaxonomy
The purposeof combining
improvements.
productivity
ofthepossiblecausalinterrelationthesuccessmodelwastoaidintheunderstanding
to
and
of
success
the
dimensions
exposiprovidea moreparsimonious
shipsamong
has
this
combination
critics
for
some
the
of
tion
proved
Unhappily,
relationships.
Thesewillbe disofreformulations.
troublesome,
leadingthemto suggesta number
cussedlaterinthispaper.
conclusionsoftheoriginalpaperwere:
The primary
natureofIS successrequirescareandinterdependent
1. The multidimensional
ofeach aspectofthisdepenandmeasurement
to thedefinition
fulattention
to measurethepossibleinteractions
dentvariable.It is important
amongthe
variof
various
effect
the
isolate
order
to
in
dimensions
success
independent
dimensions.
success
ableswithone ormoreofthesedependent
on the
and measuresshouldbe contingent
2. Selectionof successdimensions
tested
where
the
of
context
and
but,
possible,
investigation;
empirical
objectives
be
used.
should
measures
andproven
natureofIS success,an attempt
andcontingent
3. Despitethemultidimensional
measuresused
ofdifferent
thenumber
shouldbe madetoreducesignificantly
to measureIS successso thatresearchresultscan be comparedandfindings
validated.
and incorporate
4. Morefieldstudyresearchshouldinvestigate
organizational
measures.
impact
and valida5. Finally,"[t]hissuccessmodelclearlyneedsfurther
development
IS meaof
selection
for
the
a
basis
as
serve
it
could
tionbefore
appropriate
sures"[8, p. 88].

ModelAdoption
Research adoption of the D&M IS Success Model has exceededourexpectaof2002yielded285 refereed
searchinthesummer
tions.A citation
papersinjournals

12

DeLONE AND McLEAN

System
Quality

' i
j |'
j
^

Information

Quality

Use
'
/

j 1/

j F

II
II

II
>x

User

j k
Jj'
i
:

*
Organizational
1 Individual
J
Impact
ft Impact

' 1/

Satisfaction j J

Figure1. D&M IS SuccessModel.

W. DeLone andE. McLean,Information
Reprinted
bypermission,
SystemsSuccess:The
Variable.Information
Research,
1992,pp. 60-95.
QuestfortheDependent
3(1),
Systems
ofManagement
Sciences(nowINFORMS), 901 Elkridge
1992,The Institute
Copyright
MD 21090 USA.
LandingRoad,Suite400, Linthicum,

and proceedings
thathavereferenced
theD&M Model duringtheperiod1993 to
mid-2002.Manyofthesearticlespositioned
themeasurement
orthedevelopment
of
theirdependent
thecontext
oftheD&M IS Successframework.
variable(s)within
By
forreporting
andcomparing
researchwork
usingthemodelas a commonframework
IS successoreffectiveness,
we believeone oftheprimary
involving
purposesofthe
originalarticlehas beenachieved.
Although
manyofthecitedarticlestendedtojustifytheirempiricalmeasurement
ofIS successbycitingtheD&M IS SuccessModel,someofthemfailedtoheedour
cautions.Someresearchers
haveusedthemodeltosupport
theirchosensuccessvariable ratherthanto inform
thedevelopment
of a morecomprehensive
successcon- thatIS success is a
struct.Theyoverlookedthemainconclusionof thearticle
- andthatitis therefore
multidimensional
andinterdependent
construct
to
necessary
the
or
"Researchers
study interrelationships
among, tocontrolfor,thosedimensions.
shouldsystematically
combineindividual
measuresfromtheIS successcategories
to
createa comprehensive
measurement
instrument"
[8, pp. 87-88]. Althoughthese
authors
didnotchooseto measure(orcontrolfor)thevariousdimensions
ofIS suca
number
of
other
researchers
haveusedmultidimensional
cess,
measuresofIS success intheirempiricalstudiesandhaveanalyzedtheinterrelationships
amongthem.
Someofthesestudiesaresummarized
in thenextsection.

ModelValidation
Unlike a process model, whichmerelystatesthatB followsA, a causal model
thatA causesB; thatis,increasing
A willcauseB toincrease(ordecrease).
postulates

THE DeLONE AND McLEANMODEL OF INFORMATIONSYSTEM SUCCESS

13

inour
In the1992articlewe proposedsuchinterrelationships
amongthedimensions
Since 1992,a numberof studieshave
model;butwe did nottestthemempirically.
ofthemultidimensional
undertaken
investigations
relationships
empirical
amongthe
measuresofIS success.
Empiricaltestingand validationoftheD&M IS Success Model was theprimary
purposeoftworesearchstudies[38,41]. SeddonandKiew [41] surveyed104users
relaofa recently
university
accounting
systemandfoundsignificant
implemented,
with
"user
satisfaction"
and
"individual
between
impact,"
"systemquality"
tionships
and"individual
and
between"information
impact,"
quality"with"usersatisfaction"
a
and "individualimpact."Rai et al. [38] performed
between"usersatisfaction"
Model
based
on
the
D&M
IS
Success
test
on
entire
survey
responses
goodness-of-fit
student
IS. The studyfoundthatsomegoodness-of-fit
from274 usersofa university
butotherswerenot.However,all ofthepathcoefficients
indicators
weresignificant
oftheD&M IS SuccessModelwerefoundtobe signifiamongsuccessdimensions
cant.
testedtheassociationsamongthemeasuresidenstudiesexplicitly
Otherempirical
tifiedin theD&M IS SuccessModel [10, 12, 14, 16, 22, 50]. Yet otherempirical
successdimensions
testedthemodelbyinvestigating
studieshaveimplicitly
multiple
and theirinterrelationships
54,
57,
58,
17,
48,
52,
60].
[11,
Figure2 displaysthe
inthe16
not
confirmed
and
the
Model
IS
Success
D&M
(or confirmed)
relationships
studiescitedabove.Thesestudieswereselectedbasedon thefactthatthey
empirical
andtheymeasuredtheassociationamong
successconstructs
usedmultidimensional
theseempiricalresultsaresumIn thefollowing
thesuccessconstructs.
paragraphs,
withthestrongest
Links
were
tested.
links
that
success
marizedby the
empirical
arediscussedfirst.
support

SystemUse- IndividualImpacts
Sevenofthe16 studiesin Figure2 [12, 14, 16,48, 52, 54, 60] testedtheassociation
between"systemuse" and "individual
impacts"andtheassociationwas foundto be
andwasmeasured
usewastypically
the
studies.
in
of
each
voluntary
System
significant
anddependency.
ofaccesses,usagepattern,
ofuse,timeofuse,number
as frequency
anddecision-making
Individual
impactsweremeasuredin termsofjob performance
performance.

- IndividualImpacts
SystemQuality
All fivestudies[10, 12,41, 50,57] thattestedthedirectassociationbetween"system
signifiimpacts"foundthoseassociationstobe statistically
quality"and"individual
cant.Systemqualitywas measuredin termsofease-of-use,
reliability,
functionality,
Individualimpacts
and importance.
dataquality,portability,
integration,
flexibility,
andjob performance.
weremeasuredas qualityofworkenvironment

14

ii

r^
O

T3

S
^
o
X l
J ^ c

II

C
C

si1!
i

' '

NO-.

If

/ /

/ /

s| S|a

/ /
/ /

g2 ^

'' ' '

' '
'

'

g S "

Mwfvh'

/ i ^ >^

V'

L -J

Is

--

fe
|
-S
co

I l

a) /-^

^ 1^
^r

"3 rT

h ^ v L_l '

/ LjLT || ||

JL

Se ?

S?
s ^Q

'

Sil
!" fSSS

J_
5

111
H

'

' I I

SA"
ttC

1 /

!*
la

.S

li
i Sa!
'i -S-

c ^ 31
o S - o
o
'33 HM
s

3
c

D -o 2 - :
fi C 5
c
.S <tf13 ^

c U

n -8 -
SP^ g ^

k, C P C

THE DeLONE AND McLEANMODEL OF INFORMATIONSYSTEM SUCCESS

15

- IndividualImpacts
Information
Quality
between"information
The fourstudies[10, 41, 50, 57] thattestedtherelationship
Information
impacts"foundtheassociationtobe significant.
quality"and"individual
and
in
terms
of
measured
was
relevance,
timeliness,
accuracy,
completeness,
quality
in
terms
was
measured
Individual
decision-making
performance,
impact
consistency.
andqualityofwork.
job effectiveness,

OtherLinks
all theotherlinksorassociationsintheD&M IS SuccessModel
Withoneexception,
validated.The one empiricalstudythatfoundtheassociationsnot
wereempirically
was a surveyofDutchmanagers[11],wheretheassociationbetweensyssignificant
was notstatistically
revenuesandprofitability
temuse andorganizational
significant.
tested
36 ofthe38 successfactorassociationsthatwereempirically
In conclusion,
Taken
as a
found
to
be
2
were
in
summarized
in the 16 studies
significant.
Figure
fortheproposedassociations
studiesgivestrong
among
whole,theseempirical
support
in themodel.
thecausal structure
andhelpto confirm
theIS successdimensions

ModelIssues
In addition to the manypapersthathavetestedandvalidatedtheD&M IS Sucor extend
cess Model,severalarticleshavebeenpublishedthatchallenge,critique,
toa better
themodelitself.On balance,thesearticleshavecontributed
understanding
ThesearticlesandtheissuestheyraisetotheD&M IS
ofsuccessanditsdimensions.
below.
SuccessModelaresummarized

ProcessVersusCausal Models
arebasedon a process
anditssix successcategories
TheD&M IS successtaxonomy
resultareinterrelated,
we arguethatthesixdimensions
modelofIS [43].In addition,
direction
as the
in
the
same
flows
that
ingina successmodelthatindicates causality
information
process.However,citingan earlierpaperbyNewmanandRobey[35],
Seddonarguesthat"theboxesandarrowsin variance-andprocess-model
diagrams
in one
be
combined
cannot
and
different
meaningfully
concepts
quite
represent
is
models
and
variance process
model.. . . Unfortunately,
exactlywhat
combining
todo" [40].Seddonfurther
arguesthatDeLone
[DeLoneandMcLeanhave]attempted
ofIS
tocombinebothprocessandcausalexplanations
andMcLeanhave"attempted
withthismodelforsomeyears,ithas become
successintheirmodel.Afterworking
intheirmodel
thattheinclusionofbothvarianceandprocessinterpretations
apparent
leads to so manypotentially
meanings"[40, p. 240]. Seddongoes on to
confusing
of
IS success.
model
variance
a
propose respecified

16

DeLONE AND McLEAN

of processand variance
We agreewithSeddon's premisethatthecombination
of IS successin one modelcan be confusing.
However,we believe
interpretations
of theD&M Model intotwopartialvariancemodels
thatSeddon'sreformulation
oftheorigitheintent
thesuccessmodel,defeating
[40,p. 245] undulycomplicates
nal model.
Thecreation
oftheD&M IS SuccessModelwas drivenbya processunderstanding
of
thecreation
Thisprocessmodelhasjustthreecomponents:
ofIS andtheirimpacts.
a system,
theuse of thesystem,and theconsequencesof thissystemuse. Each of
fortheresultant
condition
butnotsufficient,
thesestepsis a necessary,
outcome(s).
orbenefits.
Forinstance,
without
However,
use,therecanbe noconsequences
system
there
or ill-informed,
withsystemuse, evenextensiveuse, whichis inappropriate
a
of
IS
the
dimensions
to
understand
be
no
benefits.
also
success,
Thus,
fully
may
the
out
himself
as
Seddon
variancemodelis also needed.Thus,
pointed [40, 42],
variance
applicationof ourmodelto empiricalresearchalso requiresa contextual
first
is
the
three
are
Here
too
there
the
model.
of
produccomponents:
specification
whether
is
The
is
net
benefits.
the
third
is
and
the
second
use,
tion,
onlyargument
intoonemodel.AlongwithSeddon,
canbe combined
dimensions
thesetwonecessary
aredifferent.
we believethattheycan; onlyourformulations

SystemUse as a SuccessMeasure
Seddon[40] further
arguesfortheremovalof "systemuse" as a successvariablein
forinclusionin
thecausalsuccessmodel,claimingthatuseis a behavior,
appropriate
a processmodelbutnotin a causalmodel.He arguesthatuse mustprecedeimpacts
andbenefits,
butitdoesnotcausethem.Wedisagree.Webelievethatsystem
usageis
an appropriate
measureofsuccessin mostcases.
ofthiscomplexvariable.
definition
The problemto datehas beena too simplistic
thenature
will
without
that
more
use
more
benefits,
considering
yield
Simplysaying
ofthisuse,is clearlyinsufficient.
mustalso considerthenature,
Researchers
extent,
could
be
of
use
of
use.
The
nature
system
quality,and appropriateness thesystem
ofa systemis beingusedfor
addressedbydetermining
whether
thefullfunctionality
theintended
purposes.
YoungandBenamati[59],forexample,suggestthatfullfunctionaluse ofan e-commerce
use,transactional
systemshouldincludeinformational
andBrancheau
and
customer
service
use.
With
to
the
extent
of
Lassila
use,
use,
regard
variousstatesofsystems
utilization
basedon theuse ornonuseofbasic
[27] identify
andadvancedsystemcapabilities.Simplymeasuring
theamountoftimea systemis
of
useddoes notproperly
the
between
usageandtherealization
capture relationship
be
an
results.
On
the
other
it
can
be
that
hand,
expected
argued decliningusagemay
indication
thattheanticipated
benefits
arenotbeingrealized.
important
The rejection
ofsystemuse as a successvariablewhensystemusageis mandatory
is also flawedforthereasonscitedabove.Even whenuse is required,
in
variability
on
the
realthequalityandintensity
ofthisuse is likelyto havea significant
impact
At
izationof thesystembenefits.
no systemuse is totallymandatory.
Furthermore,
someleveloftheorganization,
anexecutive
committee
haschosento
ormanagement

THE DeLONE AND McLEAN MODEL OF INFORMATION SYSTEM SUCCESS

17

a system
andrequireemployees
touse it.Thus,whereasusageofa system
implement
one
the
be
at
continued
level,
may mandatory
adoptionanduse of thesystemitself
based
ata higherlevel.Manbe
may whollyvoluntary, uponmanagement
judgment,
the
of
a
that
the
option discontinuingsystem is notproviding
agementalwayshas
desiredresultsandbenefits.
tobe usedas a dependent
variableina number
ofempirical
Systemusagecontinues
studiesandcontinues
tobe developedandtestedbyIS researchers
[11, 12,14,16, 17,
in e-commerce
38, 47, 48, 52, 54, 60]. Systemuse has takenon new importance
andessentialto desiredoutwherecustomer
use is voluntary
successmeasurements
comes[7, 29, 36]. "WhilemoststudiesthatfollowD&M replacetheUse box with
to maintainUse as in theoriginalwork.In e-commerce
. . . , we prefer
Usefulness
and
[33,p. 6]. We agreewiththeseIS researchers
systemsUse is largelyvoluntary"
and effective
believethatuse,especiallyinformed
use,willcontinuetobe an imporofIS successformanysystems.
tantindication

Role ofContext
ofapplyingtheD&M IS Sucon thedifficulty
havecommented
Severalresearchers
IS successin specificresearchconcess Modelinordertodefineandoperationalize
"Thissuccessmodelclearlyneedsfurther
Thiswasnotunexpected:
texts.
development
IS
serveas a basis fortheselectionof appropriate
it
could
and validationbefore
success
measures"[8, p. 88]. Jiangand Klein [20] foundthatuserspreferdifferent
that
et
found
al.
of
evaluated.
on
the
measures,
Whyte
system
being
type
depending
variaand
from
differences
"thereareimportant
user,
systems
deriving organizational,
theviewas towhichattributes
tionswhichcanmodify
(successmeasures)areimpora
contribution
tant"[55,p. 65]. Seddonet al. [42] makean important
byproposing
measuresbasedon thetype
IS effectiveness
matrixforclassifying
two-dimensional
theIS is beingevaluated.
in whoseinterest
ofsystemstudiedandon thestakeholder
in
the
1992
As
stated
In thisregard,
we completely
article,"no singlevariable
agree.
so thechoiceofsuccessvariablesis oftena functhananother,
better
is intrinsically
. . . etc."[8,p. 80,emphacontext
theorganizational
tionoftheobjectiveofthestudy,
sis added].

Variables
VersusDependent
Independent
to theD&M IS Success Model flowfroma
Manyof thesuggestedimprovements
variableandwhatis partofthedependent
betweenwhatis an independent
confusion
arebuttwo
and
involvement"
"User
success.
variable,IS
support"
"topmanagement
examplesof suggestedadditionsto theD&M Model; yettheseareclearlyvariables
in ERP" may
thanbeinga partofsuccess."Investing
thatmaycause successrather
of
IS
butthe
"information
to
lead
success),
(an
quality" aspect
(ormaynot)
improved
variable.
of
the
is
latter
whereas
the
variable
is an independent
former
dependent
part
controlvaribetweenthemanagement
It is essentialthatIS researchers
distinguish
andimpacts.
use satisfaction,
ablesandthedesiredresultsintermsofquality,

18

DeLONE AND McLEAN

ModelExtensions
ServiceQuality
The emergenceof end user computinginthemid-1980splacedIS organizations
inthedualroleofinformation
an information
provider(producing
product)andserviceprovider(providing
supportforend userdevelopers).Pittet al. observedthat
thanthe
rather
used measuresofIS effectiveness
focuson theproducts
"commonly
will
mismeasure
servicesoftheIS function.
there
is
a
that
IS
researchers
Thus,
danger
IS effectiveness
iftheydo notincludein theirassessment
packagea measureof IS
servicequality"[37,p. 173].Otherresearchers
haveagreedwiththis,citingtheneed
fora servicequalitymeasuretobe a partofIS success[25,28, 56].
Researchers
whohavearguedthatservicequalitybe addedto thesuccessmodel
frommarhaveappliedandtestedthe22-itemSERVQUAL measurement
instrument
of tangibles,
uses thedimensions
keting[25, 37] to an IS context.This instrument
tomeasureservicequality.Some
assurance,andempathy
reliability,
responsiveness,
instrument
items
include:
sampleSERVQUAL

"IS has up-to-date

hardware
andsoftware"
(tangible);
"IS is dependable"(reliability);
"IS employeesgiveprompt
serviceto users"(responsiveness);
"IS employeeshavetheknowledgeto do theirjob well" (assurance);and
"IS has users'bestinterests
at heart"(empathy).

Van Dykeet al. [53] challengedthisSERVQUAL metric,identifying

"problems
withthereliability,
discriminant
and
validity,
validity
convergent
validity, predictive
of themeasure.. . . [F]urther
workis neededin thedevelopment
of measuresfor
services."Recently,
assessingthequalityofinformation
Jianget al.'s [21] empirical
concludedthattheSERVQUAL
studyamong168 usersand 168 IS professionals
measureis a valuableanalytical
toolforIS managers.
The studyfoundhighconverfor
the
assurance,and empathyof the
gentvalidity
reliability,
responsiveness,
scales
and
found
levels
of
anddiscriminant
validSERVQUAL
acceptable
reliability
andempathy
scales.
ityamongthereliability,
responsiveness,
Whereaswe agreethattheSERVQUAL metricneedscontinued
and
development
we
nevertheless
believe
that
"service
deserves
validation,
measured,
quality,"
properly
tobe addedto "systemquality"and"information
ofIS sucquality"as components
cess.Although
a claimcouldbe madethat"servicequality"is merelya subsetofthe
model's"systemquality,"
thechangesintheroleofIS overthelastdecadearguefor
- the"servicequality"dimension.
a separatevariable
Of course,eachofthesequalitydimensions
willhavedifferent
weightsdepending
"information
uponthelevelof analysis.To measurethesuccessof a singlesystem,
be
the
most
Formeaquality"or"system
quality"may
important
qualitycomponent.
the
overall
success
of
the
IS
as
to
individual
suring
department, opposed
systems,
"servicequality"maybecomethemostimportant
variable.Onceagain,context
should
dictatetheappropriate
andapplication
oftheD&M IS SuccessModel.
specification

THE DeLONE AND McLEAN MODEL OF INFORMATION SYSTEM SUCCESS

19

NetBenefits
have
As the"impacts"of IS haveevolvedbeyondtheimmediate
user,researchers
suchas workgroupimpacts[18,34],interadditional
IS impactmeasures,
suggested
consumerimpacts[3, 15], and societal
and
[5,
6],
impacts
organizational industry
is
continuum
of
fromindividuthere
a
entities,
ever-increasing
impacts[40]. Clearly,
Thechoice
als tonationaleconomicaccounts,whichcouldbe affected
byIS activity.
orsystems
ofwheretheimpactsshouldbe measuredwilldependonthesystem
being
evaluatedand theirpurposes.Ratherthancomplicatethemodelwithmoresuccess
measures,we preferto movein theoppositedirectionand groupall the"impact"
called"netbenefits."
for
measuresintoa singleimpactorbenefit
category
Although,
we resistedsuchfurther
some studies,suchfinergranularity
maybe appropriate,
in theAnalysisand
Thisis discussedfurther
refinements
forthesakeofparsimony.
section.
Recommendations

Enhancements
Measurement
sucTables 1 through 6 in the 1992article [8, pp. 65-83] listedthenumerous
thathadbeenusedinpreviousempirical
eachsuccesscategory
cess measureswithin
variable
in thenumberof dependent
reduction
studies.We calledfor"a significant
Since
be
can
measuresso thatresearchresults
then,several
compared"[8, p. 80].
whichmeasureoneormoreof
studieshavedevelopedandtestedsurveyinstruments,
thesesix successconstructs.
literature
Based on a comprehensive
review,MiraniandLederer[32] developeda
Their
derivedfromIS projects.
benefits
tomeasureorganizational
instrument
33-item
straoforganizational
benefits:
ofthreecategories
consisted
framework
measurement
was
tested
instrument
The
transactional.
and
informational,
empirically
proposed
tegic,
in a surveyof 200 IS managersand systemsanalysts.The resultsshowedstrong
for
threesubdimensions
Further
evidenceofdiscriminant
analysisidentified
validity.
subdividedintocomwerefurther
benefits
each of thebenefit
categories.Strategic
Informational
benbenefits.
customer-relations
and
petitiveadvantage,alignment,
information
and
information
efitsincludedinformation
access,
flexibility
quality,
includedcommunication
benefits
transactional
andfinally,
subdimensions;
efficiency,
We believe
subdimensions.
andbusinessefficiency
efficiency,
development
systems
suchas this,thatmeasureIS benefitsare an important
thatvalidatedinstruments,
to IS successmeasurement.
contribution
oftheKaplanand
etal. [30] suggestanadaptation
In a conceptual
paper,Martinsons
oforganizaNorton"BalancedScorecard"(BSC) [23] approachforthemeasurement
thefinancial
The BSC consistsoffourperformance
tionalperformance.
perspectives:
and
businessprocessperspective,
theinternal
thecustomer
perspective,
perspective,
the
authors
to
an
IS
andgrowth
thelearning
context,
proposea
Applied
perspective.
a usermeasurement
balancedIS scorecardto includea business-value
dimension,
dimenfuture-readiness
and
a
an
orientation
dimension,
dimension, internal-process
thensuggestspecificmeasuresrelatedto each IS BSC dimension.
sion.The authors

20

DeLONE AND McLEAN

on inandreturn
Forexample,costcontrol,
revenuegeneration,
alignment,
strategic
dimension.
forthebusiness-value
vestment
areamongthemeasuressuggested
12andDoll [52] developeda fourBased on a literature
Torkzadeh
factor,
review,
users
A
of
409
end
of
IS.
iteminstrument
formeasuring
theindividual
survey
impact
The
instrument.
from18 different
was used to testthemeasurement
organizations
evidencealso supports
overallreliability
ofthe12-item
scalewas 0.92.Theempirical
individual
of
instrument.
The
theconvergent
the
anddiscriminant
resulting
validity
dimensions
are:
impact
Taskproductivity
- theextentto whichan application
theuser'soutimproves
putperunitoftime;
Taskinnovation
- theextentto whichan application
helpsuserscreateandtry
outnewideasin theirwork;
- theextentto whichan application
Customer
satisfaction
helpstheusercreate
or
and
for
firm's
internal
external
value the
customers;
- theextent
Management
work
towhichtheapplication
control
helpstoregulate
and
processes performance.
instrument,
Jiangand Klein [20] surveyed
Usinga 24-itemimpactmeasurement
113 managers
across
three
different
system
types:transacimpacts
regarding
systems
tionprocessingsystems(TPS), information
systems(1RS), and decision
reporting
decisionqualityimpactsin
Their
found
that
users
value
(DSS).
systems
study
support
valuesystems
DSS, butotherwise,
impactsforTPS and1RS.Thesefindperformance
fordifferent
that
different
measures
are
typesofsysingssuggest
impact
appropriate
tems.
- oftheUserInformation
Therecontinues
to be muchuse- anddiscussion
Satisfaction(UIS) instrument
[2, 19].Saarinen[39] hasdevelopedanexpandedusersatisfactioninstrument
to the
thatadds development
processand IS impactdimensions
use and productqualitydimensionsof thetraditional
UIS instrument
[2, 19]. His
four52-itemusersatisfaction
was developedandtestedbasedon
instrument
factor,
48 completedIS development
and validitytestsfoundthefour
projects.Reliability
satisfaction
constructs
hadacceptablereliability
andsatisfactory
Theresultvalidity.
instrument
as
serve
a
more
forIS sucing
may
comprehensive
perceptual
surrogate
cess. Li [28] also proposedan extendedinformation
satisfaction
instrument
butdid
nottesttheinstrument.
Otherauthorshaveidentified
problemswithUIS [51] andhaveproposedan alternatesatisfaction
measure[44]. In spiteof thesecriticisms,
UIS continuesto be the
mostcommonly
usedanddevelopedsuccessmeasure;but,whenusedalone,itcannotfullymeasureIS success.
As indicated
usecontinues
tobe a popularsuccessmeasure[ 17, 26,
earlier,
systems
et
Strub
al.
studied
458 usersof a voice mail systemand
47, 48]. However,
[46]
foundthatself-reported
systems
usageandcomputer-recorded
usagewerenotcorrelated.Theirfindings
suggestthatself-reported
systemusageandcomputer-recorded
usageshouldbothbe measuredinempiricalstudiesbecausethetwodo notnecessarwithone another.
ilycorrelate

THE DeLONE AND McLEAN MODEL OF INFORMATION SYSTEM SUCCESS

21

Based on a literature
Doll andTorkzadeh[9] developeda multidimensional
review,
measureof systemsusagebased on thenatureand purposeof a system.
A 30-item
instrument
was
tested
409
users
from
1
8
system
usage
using computer
organizations.
The 30 itemsmeasurethreeunderlying
use fordecision
systemsusage constructs:
anduse forcustomer
use forworkintegration,
service.Theempirical
results
support,
andgeneralapplicability.
reliability,
providedevidenceoftheinstrument's
validity,
Researchers
shouldconsideradoptingand applyingthismorecomprehensive
systemsusageinstrument.
offuture
AgarwalandPrasad[1] studiedbothinitialsystemusageandintentions
use andfoundthatdifferent
factors
affected
initialuse versusfuture
use oftheWorld
Karahannaet al. [24] founddifferent
factorswereassociated
WideWeb.Similarly,
withintention
tousewindowsbetweenpotential
and
users.These
adopters continuing
thatearlyuse andcontinued
use can differ.
twoempiricalstudiesdemonstrate
a
variable
in
use
is
IS
success;
but,toofrequently,
understanding
clearly
key
System
this
construct.
variables
are
used
to
measure
Moreresearch
complex
simpleusage
ofsystems
suchas citedaboveis neededto refinethemultidimensionality
usage.
associatedwithsystemuse and net
Information
qualityhas provento be strongly
benefitsin recentempiricalstudies[38, 54, 57] and especiallyin thecontextof
toMolla andLicker,"[Although
e-commerce
systems[7,29,33, 36,49]. According
as
an
asset to modernbusiness,
has long been considered
information
important
i.e. information
... to higherlevelsof signifihas elevatedcontent,
e-commerce
cance"[33,p. 7]. Information
qualitymeasuresthathavebeenusedinrecente-commercestudies[7,33,36] includeaccuracy,
relevance,
understandability,
completeness,
and variety.
Researchers
are strongly
encourcurrency,
personalization,
dynamism,
oftheirsuccess
qualitymeasuresas a criticaldimension
agedto includeinformation
construct.
measurement

OtherSuccessFrameworks
theD&M IS
Not all of the researchers have attemptedtocritiqueormodify
frameworks
formeaSuccessModel. Some havedevelopedandproposedalternate
basedperspecGroveretal. usedan alternative,
theoretically
suringIS effectiveness.
construct
"tobuilda theoretically-based
oforganizational
tive(theory
effectiveness)
&
and
extends
the
which
for
effectiveness
IS
[DeLone McLean]
complements
space
conandevaluation-type
IS SuccessModel" [13,p. 178].Based on unit-of-analysis
The sixeffectivecreatedsixIS effectiveness
theauthors
textdimensions,
categories.
ness classes are infusionmeasures(i.e., "organizational
impacts"in theD&M IS
measures(notcoveredintheD&M IS SuccessModel),ecoSuccessModel),market
nomicmeasures(i.e.,"organizational
usagemeasures(i.e.,"systemuse"),
impacts"),
measures(i.e.,"indiand
"user
measures
satisfaction"),
(i.e.,
productivity
perceptual
considers"systemquality"and"information
vidualimpact").Theirframework
qualwhereastheD&M IS SuccessModel
effectiveness
constructs,
ity"to be antecedent
theGrover
ofsuccessitself.In summary,
dimensions
considersthemtobe important
servestovalidatetheD&M IS SuccessModel
framework
etal. [13] IS effectiveness

22

DeLONE AND McLEAN

froma theoretical
and suggestsan area forextension,
namely,market
perspective
orindustry
impacts.We includemarket
impactsinourupdatedmodeldescribedlater
inthispaper.
forIS evaluation
Smithson
andHirschheim
[45] proposeda conceptualframework
totheevaluainpracticebyapplying
theframework
anddemonstrated
itsusefulness
varioustheoretical
bases
tionofan outsourcing
Theirframework
situation.
presents
forIS evaluation
intothree"zones"ofevaluation:
effectiveness,
efficiency,
organized
andunderstanding.
constructs
ormetrics
couldbe drawnfromtheliteraAppropriate
orturestreamassociatedwitheachconceptualbase; forexample,software
metrics,
and so on. This framework
behavior,
ganizational
sociology,cognitive
psychology,
hardincludesevaluation
areasthatoverlaptheD&M successdimensions,
including
wareandsoftware
costmetrics("systemquality"),systemusage,usersatisfaction,
benefitanalysis,and so on, butalso suggestsmanyothertheoretical
sourcesof IS
evaluation
measures.
Theauthors
thatis a sourceforidentifying
providea framework
and developingIS evaluationmeasuresratherthana singleframework
of success
Their
dimensions
andtheir
framethe
D&M
IS
Success
Model).
(i.e.,
interrelationships
workdoesnotspecifyactualsuccessconstructs
Thismakesthe
andrelatedmeasures.
an
framework
difficult
to applyin practice.However,it does offertheresearcher
alternative
theoretical
framework
fordeveloping
IS evaluationschemes.

AnalysisandRecommendations
As DISCUSSEDEARLIER,IT NOWSEEMSAPPROPRIATE
to add a thirddimension,"service

tothetwooriginal
characteristics,
quality,"
system
"systems
quality"and"information
as discussedearlier,it appearsmoreparsimonious
to combine
quality."Conversely,
"individual"
and"organizational
into
a
"net
benefits."
variable,
impacts"
single
Thisnewvariable,"netbenefits,"
raisesthreeissuesthatmustbe taken
immediately
intoaccount:whatqualifiesas a "benefit"?
forwhom?andatwhatlevelofanalysis?
In theoriginalformulation
oftheD&M Model,theterm"impact"was used.Seddon
inhischaracterization
oftheoutcomes.
[40] used"consequences"and"netbenefits"
We havecometo prefer
theterm"netbenefits"
ourselvesbecausetheoriginalterm
"impacts"maybe positiveor negative,thusleadingto a possibleconfusionas to
whether
theresultsaregood orbad.Also,theinclusionof "net"in "netbenefits"
is
because
no
is
outcome
without
important
whollypositive,
anynegative
consequences.
of thefinalsuccess
Thus,"netbenefits"is probablythemostaccuratedescriptor
variable.
The secondissueofconcernis: benefits
forwhom- thedesigner,
thesponsor,
the
orstakeholders
have
different
user,orothers?Different
actors,players,
may
opinions
as towhatconstitutes
a benefit
tothem[42].Thus,itis impossible
todefinethese"net
benefits"
without
firstdefining
thecontextor frameof reference.
The factthatthe
D&M Model does notdefinethiscontextis a matter
ofdetail,notofoversight.
The
focusof anyproposedstudymustbe defined.Our modelmaybe usefulto both
Microsoft
andtheusercommunity,
buteach mayhavea verydifferent
definition
of
whatconstitutes
netbenefits
andthusIS success.

THE DeLONE AND McLEAN MODEL OF INFORMATION SYSTEM SUCCESS

23

tobe meaFinally,thelevelofanalysismustbe addressed[4, 42]. Arethebenefits

or
orthatoftheindustry
hisorheremployer,
suredfromtheindividual's
perspective,
into
a
of thenation?Collapsing"individual"and "organizational
single
impacts"
the
does notmaketheproblemgo away.It merelytransfers
variable,"netbenefits,"
needto specifythefocusofanalysisto theresearcher.
D&M IS SuccessModel
we haveupdatedtheoriginal
Basedontheseconsiderations,
research.
IS empirical
future
forframing
as a foundation

The UpdatedD&M IS SuccessModel

Based on research contributionssinceouroriginalpaper,andbasedonchanges
we haveupdatedouroriginal
of information
in theroleand management
systems,
on Figure3.
successmodel.The updatedmodelis presented
As discussedearlier,qualityhas threemajordimensions:"information
quality,"
"systemsquality,"and "servicequality."Each shouldbe measured or controlled
"use" and
orjointly,theywillaffectsubsequent
becausesingularly
for- separately,
"usersatisfaction."
themultidimensional
in interpreting
Giventhedifficulties
aspectsof"use"- manversusineffective,
effective
versusuninformed,
informed
datoryversusvoluntary,
measurein
alternative
touse"maybe a worthwhile
andso on- we suggest"intention
whereas"use" is a behavior.Substito use" is an attitude,
"Intention
somecontexts.
forthelattermayresolvesomeof theprocessversuscausal contheformer
tuting
andtheirlinkswithbehavior,
attitudes,
cernsthatSeddon(1997) hasraised.However,
to measure;andmanyresearchers
difficult
arenotoriously
maychooseto staywith
of
informed
a
more
with
but
understandingit.
"use," hopefully
of
theD&M Model,"use" and"usersatisformulation
As was truein theoriginal
in a process
"Use" mustprecede"usersatisfaction"
faction"arecloselyinterrelated.
in a
"usersatisfaction"
with"use" willlead to greater
sense,butpositiveexperience
willleadtoincreased"intention
increased"usersatisfaction"
causal sense.Similarly,
to use,"andthus"use."
willoccur.If
certain"netbenefits"
As a resultofthis"use" and"usersatisfaction,"
it is assumedthatthe"netbenefits"fromthe
theIS or serviceis to be continued,
and
of
or
owner
of
the
sponsor thesystemarepositive,thusinfluencing
perspective
still
are
feedback
These
satisfaction."
"user
and
"use"
loops
subsequent
reinforcing
is
The lackofpositivebenefits
arenegative.
evenifthe"netbenefits"
valid,however,
ofthesystemoroftheIS
likelytolead todecreaseduse andpossiblediscontinuance
is to
The
wholesale
itself
outsourcing). challengefortheresearcher
department (e.g.,
are
"net
benefits"
in
which
context
and
thestakeholders
defineclearlyandcarefully
tobe measured.
The updatedD&M IS Success Model includesarrowsto demonstrate
proposed
in a processsense,butdoes notshowposiassociationsamongsuccessdimensions
tiveor negativesignsforthoseassociationsin a causal sense.The natureof these
withinthecontextof a particular
study.
causal associationsshouldbe hypothesized
more
with
associated
will
be
use,
Forexample,inone instancea high-quality
system

24

DeLONEAND McLEAN

INFORMATION L
QUALITY
^^
K
'
SYSTEMQUALITY

'
s.

INTENTION
TO USE

USE

^^'

/
/
SERVICE
QUALITY

'/

>*

y^

s'

USER
SATISFACTION
'

^^

^*s^*v>^

Z^
NET
BENEFITS

^^

'

3. Updated
D&M IS SuccessModel
Figure
The proposedassociationswould
moreusersatisfaction,
and positivenetbenefits.
would
moreuseofa poorqualitysystem
thenall be positive.In another
circumstance,
Theproposedassoandnegative
netbenefits.
be associatedwithmoredissatisfaction
ciationswouldthenbe negative.

E-Commerce
Success
in particular,
is havinga
Informationtechnology in general, and theInternet
in
dramatic
impacton businessoperations.
Companiesaremakinglargeinvestments
e-commerce
of
their
e-combut
are
to
evaluate
the
success
applications
hard-pressed
mercesystems.
and
IS researchers
haveturnedtheirattention
to developing,
testing,
successmeasures[7, 29, 33, 36, 49]. Molla and Licker[33]
applyinge-commerce
successmodelbasedon theD&M IS SuccessModel.This
proposedan e-commerce
sectiondemonstrates
howtheupdatedD&M IS SuccessModelcanbe adaptedtothe
measurement
world.
challengesofthenewe-commerce
As a powerful
communications
andcommerce
theInternet
is a communimedium,
cationand IS phenomenon
thatlendsitselfto a measurement
framework
(i.e., the
D&M IS SuccessModel) thatis builton communication
and
Shannon
theory(e.g.,
Weaver[43]). Withinthee-commerce
the
users
are
customcontext, primary
system
ersorsuppliers
rather
thaninternal
users.Customers
andsuppliers
use thesystemto
makebuyingorsellingdecisionsandexecutebusinesstransactions.
Theseelectronic
decisionsand transactions
will thenimpactindividualusers,organizations,
indusand
even
national
economies.
This
communications
and
commerce
tries,
processfits
nicelyintotheupdatedD&M IS SuccessModel anditssix successdimensions.
"Systemquality,"
intheInternet
measuresthedesiredcharacterisenvironment,
ticsof an e-commerce
system.Usability,availability,
reliability,
adaptability,

THE DeLONE AND McLEANMODEL OF INFORMATIONSYSTEM SUCCESS

25

andresponsetime(e.g.,downloadtime)areexamplesofqualitiesthatarevaluedbyusersofan e-commerce
system.
"Information
the
content
issue.Webcontent
should
quality"captures e-commerce
be personalized,
to
and
relevant,
complete,
easy understand, secureifwe expect
toinitiate
transactions
viatheInternet
andreturn
prospective
buyersorsuppliers
tooursiteon a regularbasis.
theoverallsupport
"Servicequality,"
deliveredbytheserviceprovider,
applies
of
whether
this
is delivered
a neworgaregardless
support
bytheIS department,
nizationalunit,or outsourced
to an Internet
serviceprovider(ISP). Its importhanpreviously
tanceis mostlikelygreater
sincetheusersarenowourcustomwilltranslate
intolostcustomers
ersandpoorusersupport
andlostsales.
from
a
within
the
"Usage"measureseverything a visitto Website,tonavigation
toexecution
ofa transaction.
retrieval,
site,to information
remainsan important
meansof measuring
ourcustomers'
"Usersatisfaction"
our
and
should
cover
the
entire
of
e-commerce
customer
system
expeopinions
retrieval
and
riencecyclefrominformation
through
payment,
purchase,
receipt,
service.
arethemostimportant
successmeasuresas theycapturethebal"Netbenefits"
on our customers,
ance of positiveand negativeimpactsof thee-commerce
markets,
industries,
economies,and even
organizations,
employees,
suppliers,
our societies.Have Internet
purchasessaved individualconsumerstimeand
suchas largermarkets,
and
supplychainefficiencies,
money?Havethebenefits
for
an
net
benefits
Have
customer
organization?
yieldedpositive
responsiveness
ine-commerce
infrastructure
investments
countries'
yieldeda netpositivegrowth
in e-commerce
infraHave societalinvestments
in thegrossnationalproduct?
"Netbenefits"
measuresmustbe destructure
andeducationreducedpoverty?
investment.
foreache-commerce
andobjectives
termined
Thus,there
bycontext
but
"netbenefits"
willbe a variety
ofe-commerce
measures, manywillbe the
in general.
sameonesthathavebeendevelopedandtestedforIS investments

buttheycannotbe analyzed
successmeasuresaremostimportant,
"Netbenefits"
without
and understood
quality"measurements.
"systemquality"and "information
theimpactofa Websitedesignon
thee-commerce
Forexample,within
environment,
oftheusability
without
an evaluation
cannotbe fullyunderstood
customer
purchases
thatis
decisionsoftheinformation
oftheWebsiteandtherelevanceforpurchasing
purchaser.
providedto theprospective
how thesix dimensionsof theupdatedD&M IS Success
Table 1 demonstrates
framework
to organizethevarioussuccess
Model can be used as a parsimonious
literature.
intheIS ande-commerce
identified
metrics

andConclusions
Summary
the
Ten years ago we publishedan IS success frameworkinordertointegrate
researchworkon IS successthathad been done up to thatpointand to provide

26

DeLONE AND McLEAN

Table 1. E-CommerceSuccessMetrics
Systemsquality
Adaptability
Availability
Reliability
Response time
Usability
Information
quality
Completeness
Ease ofunderstanding
Personalization
Relevance
Security
Servicequality
Assurance
Empathy
Responsiveness
Use
Natureofuse
Navigation
patterns
Numberofsitevisits
Numberoftransactions
executed
User satisfaction
Repeat purchases
Repeat visits
User surveys
Netbenefits
Cost savings
Expandedmarkets
Incremental
additionalsales
Reduced search costs
Timesavings

to
an IS successmodelthatattempted
directions
forfuture
research.We presented
natureofIS success.Thehundreds
andinterdependent
capturethemultidimensional
orvalidatedtheoriginal
ofresearchworksthathaveapplied,developed,challenged,
andtothe
modelspeaktotheneedfora commonapproachto successmeasurement
researchdesigns.The succeedingtenyearshaveseen
valueof a modelforframing
intermsoftheimpactsofIS on businessandsocietyas wellas
tremendous
progress
progressin IS research.In lightof thisprogressand change,we feltcompelledto
review,evaluate,andupdateoursuccessmodel.
ourmodelas
studiesthatbothvalidateandsupport
therecentresearch
Considering
wellas thosethatchallengeit,we concludethatouroriginalmodelandrelatedcon-

THE DeLONEAND McLEANMODEL OF INFORMATIONSYSTEM SUCCESS

27

evenin thee-comelusions[8] stillforma soundbasisforIS successmeasurement

We believethatourproposedchangesin theupdatedD&M IS
merceenvironment.
SuccessModel are largelychangesin degree,notin kind.The additionof "service
impact"into
impacts"and"organizational
quality"andthecollapsingof"individual
natureofthemodel.
theparsimonious
stillpreserve
"netbenefits"
Based on ourreviewof theresearchexperienceof thelastdecade,we drawthe
conclusions:
following
studieshavevalidatedtheoriginalmodelanditsinterrelation1. Manyempirical
totheoriginal
enhancements
studieshaverecommended
other
ships,whereas
we proposeanupdatedD&M IS Success
model.Basedonthesecontributions,
of IS emand comparing
forthepositioning
Model to serveas a foundation
and
to
tested
continue
be
should
The
model
research.
challenged.The
pirical
of
this
continued
in thispaperareexamples
growthand
changesintroduced
We encourageotherstojoin in thiseffort.
refinement.
com2. The updatedD&M IS SuccessModel is a usefulmodelfordeveloping
1
in
Table
.
as
demonstrated
measures
success
e-commerce
prehensive
of
dimension
that"servicequality"be addedas an important
3. We recommend
ofIS support,
IS successgiventheimportance
especiallyin thee-commerce
serviceis crucial.
wherecustomer
environment
natureof IS successreandinterdependent
4. The complex,multidimensional,
ofeachdimension
measurement
and
tothedefinition
quirescarefulattention
tomeasurethepossibleinteractions
variable.Itis important
ofthisdependent
ofvariousindetoisolatetheeffect
in
order
dimensions
success
these
among
dimensions.
success
of
these
more
or
with
one
variables
dependent
pendent
the
The updatedD&M IS SuccessModelinFigure3 presents interdependent
andtested.
thatshouldcontinuetobe considered
relationships
andmeatheselectionofIS successdimensions
5. Foreachresearchendeavor,
of
the
and
context
the
on
be
suresshould contingent
empiricalinobjectives
but,wherepossible,testedandprovenmeasuresshouldbe used.
vestigation,
forselectionof
The Seddonet al. [42] contextmatrixis a valuablereference
successmeasuresbasedon context.
natureofIS success,an attempt
andcontingent
6. Despitethemultidimensional
thenumberofmeasuresusedto meashouldbe madeto reducesignificantly
valisureIS successso thatresearchresultscan be comparedand findings
dated. Some good progresshas been made in this area as notedin the
sectionofthispaper.Wherepossible,we advoEnhancements
Measurement
validatedmeasuresratherthanthedevelopcate theapplicationof existing,
mentofnewmeasures.
7. Withthegrowthof management
supportsystemsand theadventand develsystemsuse is morecommontovoluntary
systems,
opmentofe-commerce
toadvocatetheinclusion
continue
daythanitwas a decadeago.We,therefore
Actual
of "SystemUse" as a criticaldimensionof IS successmeasurement.
use measures.Also,usage
to self-reported
use measuresshouldbe preferred

28

DeLONE AND McLEAN

includmeasuresshouldcapturetherichnessofuse as a systemphenomenon
not
ofuse,andshould simplymealevel,andappropriateness
ingthenature,
ofuse.
surethefrequency
"Net
and incorporate
researchshouldinvestigate
8. Finally,morefield-study
this
conclusion:
and
Yuthas Youngsupport
measures.
Benefits"
"[Examining
to measuring
andusagemeasuresis notan acceptablealternative
satisfaction
arecorvariables
three
the
Net
Benefits]
[i.e.,
Although
directly.
performance
warrant
to
not
are
them
between
the
strong
sufficiently
related, relationships
has been
forone another"[60,p. 121].Good progress
theiruse as substitutes
on
measures theindiof"NetBenefits"
andtesting
madein thedevelopment
andnationallevels.
vidual,group,firm,
industry,
References
and perceived
1. Agarwal,R., and Prasad,J. The role of innovationcharacteristics
DecisionSciences,28, 3 (1997),
in theacceptanceofinformation
voluntariness
technologies.
557-580.
and analyzing
of a tool formeasuring
2. Bailey,J.E.,and Pearson,S.W. Development
530-545.
5
user
satisfaction.
Science,
29,
(1983),
Management
computer
welfare.
toconsumer
ofinformation
E. Thecontribution
3. Brynjolfsson,
Infortechnology
mationSystems
Research,7, 3 (1996), 281-300.
andindividual
andquantitative
4. Chan,Y.E. IT value:Thegreatdividebetweenqualitative
4
measures.
Journal
andorganizational
16,
2000),
(Spring
Systems,
Information
ofManagement
225-261.
information
coordination
5. Clemons,E.K., and Row,M.C. Limitsto interfirm
through
Journalof
Resultsof a fieldstudyin consumergoods packagingdistribution.
technology:
10, 1 (Summer1993),73-95.
Systems,
Management
Information
onthe
6. Clemons,E.K.; Reddi,S.P.; andRow,M.C. Theimpactofinformation
technology
JournalofManageofeconomicactivity:
The "movetothemiddle"hypothesis.
organization
mentInformation
10, 2 (Fall 1993),9-35.
Systems,
oftheWorldWideWeb.
factors
inuserevaluation
7. D'Ambra,J.,andRice R.E. Emerging
& Management,
38, 6 (2001), 373-384.
Information
success:Thequestforthedepen8. DeLone,W.H.,andMcLean,E.R. Information
systems
dentvariable.Information
Research,3, 1 (1992), 60-95.
Systems
use in
measureofsystems
9. Doll,W.J.,andTorkzadeh,
G. Developinga multidimensional
an organizational
context.
& Management,
33, 4 (1998), 171-185.
Information
modelof end usercomputing
10. Etezadi-Amoli,J.,and Farhoomand,
A.F. A structural
& Management,
satisfaction
anduserperformance.
30, 2 (1996), 65-73.
Information
11. Gelderman,
M. The relationbetweenusersatisfaction,
systems,
usage of information
andperformance.
& Management,
Information
34, 1 (1998), 11-18.
fitand individualperformance.
12. Goodhue,D.L., and Thompson,
R.L. Task-technology
MIS Quarterly,
19, 2 (1995), 213-233.
The con13. Grover,G.; Jeong,S.R.; and Segars,A.H. Information
systemseffectiveness:
struct
ofapplication.
& Management,
31, 4 (1996), 177-191.
space andpatterns
Information
14. Guimares,
thehumanside.
T.,andIgbana,M. Client/server
systemsuccess:Exploring
DecisionSciences,28, 4 (1997), 851-875.
15. Hitt,L., andBrynjolfsson,
E. The threefacesofIT value:Theoryandevidence.In J.I.
on
DeGross,S.L. Huff,andM.C. Munro(eds.),ProceedingsoftheInternational
Conference
Atlanta,GA: AssociationforInformation
Information
Systems.
Systems,1994,pp. 263-278.
16.Igbaria,M., andTan,M. Theconsequencesoftheinformation
acceptanceon
technology
individual
& Management,
32, 3 (1997), 113-121.
subsequent
performance.
Information
17. Igbaria,M.; Zinatelli,N.; Cragg,P.; and Cavaye,A. Personalcomputing
acceptance
factors
on smallfirms:
A structural
21, 3 (1997), 279-302.
equationmodel.MIS Quarterly,

THE DeLONE AND McLEAN MODEL OF INFORMATION SYSTEM SUCCESS

29

18. Ishman,M. Measuringinformation

systemsuccessat theindividuallevelin cross-culIn EJ. Garrity
andG.L. Sanders(eds.),Information
turalenvironments.
SuccessMeaSystems
surement.
PA: Idea Group,1998,pp. 60-78.
Hershey,
ofuserinformation
19.Ives,B.; OlsenM.; andBaroudi,J.J.The measurement
satisfaction.
Communications
oftheACM,26, 10 (1983), 785-793.
andKlein,G. Userevaluation
of information
20. Jiang,J.J.,
systems:By systemtypology.
IEEE Transactions
on Systems,
Man,and Cybernetics,
29, 1 (1999), 111-116.
21. Jiang,J.J.;Klein,G.; and Carr,C.L. Measuringinformation
systemsservicequality:
26, 2 (2002), 145-166.
SERVQUAL fromtheotherside.MIS Quarterly,
A longitudinal
22. Jurison,
J.Thetemporal
natureofIS benefits:
& ManInformation
study.
30, 2 (1996), 75-79.
agement,
D.R Translating
IntoAction:TheBalancedScorecard.
23. Kaplan,R.S.,andNorton,
Strategy
Boston:HarvardBusinessSchoolPress,1996.
N.L. Information
24. Karahanna,
E.; Strub,
D.W.;andChervany,
technology
adoptionacross
ofpre-adoption
andpost-adoption
beliefs.MIS Quarterly,
time:A cross-sectional
comparison
23,2(1999), 183-213.
withthe
25. Kettinger,
W.J.,and Lee, C.C. Perceivedservicequalityand usersatisfaction
DecisionSciences,25, 5-6 (1995), 737-765.
information
servicesfunction.
ininformafieldstudyofthedifferences
J.C.An exploratory
26. Larsen,T.J.,andWetherbe,
middlemanagers.
& Manusebetweenmore-andless-innovative
tiontechnology
Information
36, 2 (1999), 93-108.
agement,
ofcommercial
software
J.C.Adoptionandutilization
27. Lassila,K.S., andBrancheau,
packandtracks.JournalofManagement
utilization
transitions,
triggers,
equilibria,
ages:Exploring
16, 2 (Fall 1999),63-90.
Information
Systems,
A metaanalysisof
ofinformation
28. Li, E.Y. Perceivedimportance
systemsuccessfactors:
& Management,
32, 1 (1997), 15-28.
Information
groupdifferences.
K.P. ExploringthefactorsassociatedwithWeb sitesuccessin the
29. Liu, C, andArnett,
and Management,
commerce.
contextofelectronic
Information
38, 1 (2000), 23-33.
forthe
A foundation
30. Martinsons,
M.; Davison,M.R.; andTse,D. Thebalancedscorecard:
DecisionSupportSystems,
ofinformation
25, 1 (1999),71-88.
systems.
management
strategic
A communication
information
3 1. Mason,R.O. Measuring
systems
approach.Inforoutput:
mation& Management,
1, 5 (1978), 219-234.
benefits
of
forassessingtheorganizational
A.L. An instrument
32. Mirani,R., andLederer,
IS projects.DecisionSciences,29, 4 (1998), 803-838.
to extendand
33. Molla, A., and Licker,P.S. E-commercesystemssuccess:An attempt
SucCommerce
theDeLone andMcLeanmodelofIS success.JournalofElectronic
respecify
cess,2, 4 (2001), 1-11.
modelforassessV.R.A comprehensive
34. Myers,B.L.; Kappelman,
L.A.; andPrybutok,
Towarda theoryfor
of theinformation
systemsfunction:
ing thequalityand productivity
andG.L. Sanders(eds.),Information
In EJ. Garrity
assessment.
information
Systems
systems
PA: Idea Group,1998,pp. 94-121.
SuccessMeasurement.
Hershey,
MIS
35. Newman,M., andRobey,D. A socialprocessmodelofuser-analyst
relationships.
16, 2 (1992), 249-266.
Quarterly,
metrics.
36. Palmer,J.W.Websiteusability,
Systems
Information
design,andperformance
Research,13, 2 (2002), 151-167.
37. Pitt,L.F.; Watson,R.T.; and Kavan,C.B. Servicequality:A measureor information
MIS Quarterly,
effectiveness.
19, 2 (1995), 173-188.
systems
38. Rai,A.; Lang,S.S.; andWelker,R.B. Assessingthevalidityof IS successmodels:An
testandtheoretical
Research,13, 1 (2002), 50-69.
Systems
analysis.Information
empirical
success.Inforinformation
forevaluating
39. Saarinen,T. An expandedinstrument
systems
mation& Management,
31, 2 (1996), 103-118.
and extensionof theDeLone and McLean modelot IS
40. Seddon,PB. A respecification
success.Information
Research,8, 3 (1997), 240-253.
Systems
ottrieDeLone andMcLean
4 1. Seddon,P.B.,andKiew,M.-Y.A partialtestanddevelopment
modelof IS success.In J.I.DeGross,S.L. Huff,and M.C. Munro(eds.),Proceedingsofthe
on Information
International
Atlanta,GA: AssociationforInformation
Systems.
Conference
Systems,1994,pp. 99-110.

30

DeLONE AND McLEAN

ofinfor42. Seddon,P.B.; Staples,D.S.; Patnayakuni,

R.; andBowtell,MJ. Thedimensions
mationsystemssuccess.Communications
2, 20
Systems,
forInformation
oftheAssociation
View=pdf&x=
(November1999) (available at cais.isworld.org/articles/2-20/default.asp?
12&y=13).
Urbana,
43. Shannon,C.E., andWeaver,W. TheMathematical
TheoryofCommunication.
IL: University
ofIllinoisPress,1949.
Data
satisfaction.
44. Shirani,A.; Aiken,M.; andReithel,B. A modelofuserinformation
Base. 25. 4 (1994) 17-23.
R. Analysinginformation
45. Smithson,
S., and Hirschheim,
systemevaluation:Another
lookat an old problem.EuropeanJournalofInformation
7, 3 (1998), 158-174.
Systems,
E. Measuringsystemusage:Impli46. Strub,D.; Limayem,M.; andKarahanna-Evaristo,
cationsforIS theory
Science,41, 8 (1995), 1328-1342.
testing.
Management
information
47. Taylor,S., andTodd,P. Understanding
usage:A testofcompettechnology
Research,6, 2 (1995), 144-176.
ingmodels.Information
Systems
and
of operational
as a facilitator
48. Teng,J.,andCalhoun,K. Organizational
computing
DecisionScistudyof managers'perceptions.
managerialdecisionmaking:An exploratory
ences,27, 4 (1996), 673-710.
torcompetitive
49. Teo,T.S.H.,andChoo,W.Y.AssessingtheimpactofusingtheInternet
67-83.
1
&
39,
(2001),
InformationManagement,
intelligence.
50. Teo,T.S.H.,andWong,P.K.An empirical
impactofcomputerstudyoftheperformance
JournalofManagement
izationintheretailindustry.
Science,26,5
Omega- TheInternational
U998. 611-621.
A usersatisfaction
51. Thong,J.Y.T.,andYap, C. Information
apsystemseffectiveness:
32, 5 (1996), 601-610.
Processing& Management,
proach.Information
theperceived
of a toolformeasuring
52. Torkzadeh,
G., and Doll, W.J.The development
JournalofManageon work.Omega- TheInternational
impactof information
technology
mentScience,27, 3 (1999), 327-339.
53. VanDyke,T.P.; Kappelman,L.A.; and Prybutok,
V.R. Measuringinformation
systems
servicequality:Concernson theuse oftheSERVQUAL questionnaire.
MIS Quarterly,
21, 2
(1997), 195-208.
54. Weill,P., andVitale,M. Assessingthehealthof an information
An
systemportfolio:
MIS Quarterly,
23, 4 (1999), 601-624.
examplefromprocessengineering.
55. Whyte,G.; Bytheway,
userperceptions
ofinformaA.; andEdwards,C. Understanding
tionsystemsuccess.JournalofStrategic
6, 1 (1997), 35-68.
Information
Systems,
56. Wilkin,C, and Hewitt,B. Qualityin a respecification
of DeLone and McLean's IS
successmodel.In MehdiKhozrowpour
Con(ed.), Proceedingsof1999 IRMAInternational
PA: Idea GroupPublishing,
ference.Hershey,
1999,pp. 663-672.
57. Wixom,B.H., andWatson,H.J.An empiricalinvestigation
ofthefactorsaffecting
data
success.MIS Quarterly,
25, 1 (2001), 17-41.
warehousing
58. Yoon,Y; Guimares,
A. Exploring
successfactors
for
T.; andClevenson,
expertsystems
businessprocessreengineering.
JournalofEngineering
and Technology
15,2/3
Management,
(1998), 179-199.
59. Young,D., andBenamati,J.Differences
in publicWebsites:The current
stateof large
U.S. firms.
Journal
Commerce
Research,1,3 (2000) (availableatwww.csulb.edu/
ofElectronic
ournal
.htm).
web/j
s/jecr/p_i
60. Yuthas,K., andYoung,S.T. Materialmatters:
of materials
Assessingtheeffectiveness
IS. Information
& Management,
33, 3 (1998), 115-124.
management