You are on page 1of 30

Government Information Quarterly 22 (2005) 187 216

E-government success factors: Mapping practical tools to


theoretical foundations
J. Ramon Gil-Garca, Theresa A. PardoT
Center for Technology in Government, University at Albany, Albany, NY 12205-1138, USA

Abstract
Strategies are systematic and long-term approaches to problems. Federal, state, and local
governments are investing in the development of strategies to further their e-government goals. These
strategies are based on their knowledge of the field and the relevant resources available to them.
Governments are communicating these strategies to practitioners through the use of practical guides.
The guides provide direction to practitioners as they consider, make a case for, and implement IT
initiatives. This article presents an analysis of a selected set of resources government practitioners use to
guide their e-government efforts. A selected review of current literature on the challenges to information
technology initiatives is used to create a framework for the analysis. A gap analysis examines the extent
to which IT-related research is reflected in the practical guides. The resulting analysis is used to identify
a set of commonalities across the practical guides and a set of recommendations for future development
of practitioner guides and future research into e-government initiatives.
D 2005 Elsevier Inc. All rights reserved.
Keywords: Information technology; Government; IT implementation; Success factors; Relevance

1. Introduction
E-government has been conceptualized as the intensive or generalized use of information
technologies in government for the provision of public services, the improvement of
T Corresponding author.
E-mail address: tpardo@ctg.albany.edu (T.A. Pardo).
0740-624X/$ - see front matter D 2005 Elsevier Inc. All rights reserved.
doi:10.1016/j.giq.2005.02.001

188

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

managerial effectiveness, and the promotion of democratic values and mechanisms.1


Information technology (IT) has the potential to transform government structures and to
improve the quality of government services. Technology provides two main opportunities for
government: (1) increased operational efficiency by reducing costs and increasing
productivity, and (2) better quality of services provided by government agencies. Realizing
the benefits of these technologies requires organizations to understand and overcome the
challenges to their efforts. Technological complexity and incompatibility are not the only, nor
the most difficult, challenges to overcome. Managerial, political, and legal factors have been
identified as important elements to take into consideration in the design and development of
IT initiatives as well.2 Politics, privacy concerns, turf, and other institutional arrangements
can also affect the results of an IT project.3
Information managers in government must be aware of the many problems they face in ITintensive projects. IT initiatives in general, and e-government projects in particular, face
multiple and complex challenges.4 Identifying and overcoming these challenges is not always
easy. Many national, state, and local governments are developing tools to help managers
make decisions about IT investments and implementation. The purpose of this article is to
examine the extent to which IS research informs the development of practitioner tools for
government IT decision makers. Four tools were selected as the focus of this examination
based on their visibility and central role in informing practitioners at the national level in the
United States and Canada and at the state level within the United States. The examination
produced a number of observations about the practical tools in terms of their treatment of the
challenges to e-government and strategies for successful IT initiatives as identified by current
research. It also produced a set of recommendations for future efforts in both research and
practice.
This article is divided in seven sections including these introductory comments. Section 2
frames the debate about e-government research and relevance issues. Section 3 describes the
method used in this comparative effort. Section 4 presents some challenges and IT success
factors identified in different disciplines. Section 5 introduces the four practical tools selected
for this study. Section 6 identifies how the challenges and factors drawn from the various
disciplinary literature have been incorporated into the selected set of tools. Finally, the article
offers some conclusions and recommendations.

2. E-government research and relevance issues


As the interest in and pressure for new and expanded e-government increases, public
managers find themselves making decisions about information and information technology
for which they are often unprepared or ill-equipped. Recognition of the complexity and risk
of IT decisions and of the broad range of public managers involved in making these types of
decisions has spurred the development of many structured and rigorous tools to support IT
business case analysis and risk assessment strategies. These strategies, recommended in some
government agencies, and required in others, provide guidance for IT decision making within
particular organizational contexts.

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

189

It is important for both practitioners and researchers that these tools be grounded in the
latest information systems research and practice. At this point however, there is no empirical
evidence that this goal has been achieved. The extent to which practitioners have found the
results of years of IS research relevant to their efforts to produce tools that limit the risk of IT
initiatives is unclear. A continuing high level of systems failures, however, encourages an
examination of the relationship between research and practice.
The latest debate about the relevance of information systems research was presented in the
March 2001 issue of the Communications of the Association for Information Systems
(CAIS). This issue presented the thoughts, concerns, and recommendations of a wide range of
researchers in the field. The 26 articles included discussions about (1) expanding the notion of
relevance;5 (2) the question of relevance to whom;6 and (3) the need for better matches
between academic research goals and goals of constituents.7 Paul Gray, the editor of CAIS, in
his introduction to the special issue describes the relevance issue as one that bconcerns the
importance of academic IS research to the practitioner community.Q8 According to Gray, the
authors of the 26 papers in the special issue seem to have consensus that published work is
not being read by practitioners. The explanations presented for this gap include abstractedness
of writing, lack of practical experience of faculty, latency of publication of academic research,
and a failure to focus on applications for non-business constituencies such as the public sector
and the community use of computing.9
Therefore, the summary concern stated from articles and surveys is that practitioners are
not reading researchthe inference being, therefore, it is not informing their practices. The
premise of this article is that an examination of the tools being used by practitioners for
evidence of empirically supported practices and strategies will inform this discussion. We
may find that although practitioners are not systematically reading research, they realize the
benefit of research that is incorporated into the practical guides they are reading and in some
cases required to use.

3. Method
A gap analysis between a selected set of practitioner tools and a set of key success factors of
IT initiatives has the potential to inform questions about the relationship between research and
practice. A gap analysis strategy represents an opportunity to do a component-by-component
analysis to determine the extent to which the design of each reflects awareness of relevant
research on information systems success.10 This strategy could also inform future refinement
of practical tools as well as suggest strategies for the future development of research-based
practical tools. The gap analysis is comprised of the four-step process outlined below.
First, a review of current literature in information systems research is used to identify
factors found to influence the success of IT initiatives. This review includes the scanning of
the last 5 years (19992003) of five top journals in public administration.11 Articles with a
focus on e-government success factors were selected. The literature review also includes
selected journal articles and book chapters that specifically address IT success factors in both
public and private organizations.

190

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

Second, the research identified and described a set of tools used for government IT
initiatives. These tools were selected based on their visibility and central role in informing
practitioners at the national level in the United States and Canada and at the state level within
the United States. The review and description of the tools was selective and based on sections
that explicitly deal with IT risks and success factors. Third, a comparison of the factors
against the selective descriptions was conducted. This comparison focused on four categories
of challenges and strategies found in the IS success literature. Fourth, an identification of the
gaps between the research and the practical tools is presented and discussed.

4. E-government challenges and success strategies: a review of the literature


A number of disciplines have invested in efforts to build understanding of the challenges in
information technology initiatives. Research focused on technology, management, policy,
information, and organizational issues have all contributed to knowledge about these
challenges.12
Simultaneously, government practitioners have worked to improve their chances for
success by developing and adopting multiple, and sometimes interrelated, strategies for
responding to challenges to their IT initiatives. The research community too, has continued to
invest in research into the success of IT initiatives. Practitioner and research efforts are
moving beyond a view of technology as the primary determinant of success and are seeking
more broadly based and sophisticated understanding of the interaction among technology,
organizations, and environments.
4.1. Challenges to e-government initiatives
Although there is no single list of challenges to e-government initiatives, notable
consistencies exist across the disciplines. These consistent challenges are organized here as
primary challenges to e-government as information systems in context (see Table 1). The
primary challenges are grouped into five categories according to their core aspect: (1)
information and data, (2) information technology, (3) organizational and managerial, (4) legal
and regulatory, and (5) institutional and environmental.
4.1.1. Information and data challenges
E-government initiatives are about the capture, management, use, dissemination, and
sharing of information. A number of the challenges relate to the information that is at the core
of e-government initiatives. Redman,13 Kaplan et al.,14 and Ballou and Tayi15 are among the
many scientists who focus their research on data quality and data accuracy issues. According to
Redman,16 data quality problems include inaccuracies, inconsistencies, and incompleteness of
data. Kaplan et al.17 emphasize that data quality is very important not only for intraorganizational usage, but also for reports to different stakeholders. In addition, Tayi and Ballou18
identify the lack of appropriate data as a further challenge to IT initiatives. In this regard, it is
important to understand the challenges of using bhardQ legacy data for decision support

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

191

Table 1
Challenges for e-government initiatives
Challenge category(s) Challenge

Authors

Information and
data

Dawes, 1996; Redman, 1998; Tayi and Ballou, 1998;


Ballou and Tayi, 1999; Brown, 2000; Ambite et al.,
2002; Burbridge, 2002
Brown and Brudney, 2003
Davis, 1989; DeLone and Mclean, 1992; Caffrey,
1998; Brown, 2000; DeLone and Mclean, 2003;
Garson, 2003; Mahler and Regan, 2003
Irvine, 2000; Milner, 2000; Joshi et al., 2002; Moon,
2002; Holden et al., 2003; Luna-Reyes and
Gil-Garcia, 2003; Roy, 2003
Dawes, 1996; Chengalur-Smith and Duchessi, 1999;
Brown, 2001; Landsberg and Wolken, 2001; Dawes
and Pardo, 2002; Burbridge, 2002; Holden et al., 2003
Barki et al., 1993; Dawes and Nelson, 1995; Caffrey,
1998; Chengalur-Smith and Duchessi, 1999; West and
Berman, 2001; Garson, 2003
Caffrey, 1998; Brown, 2001; Dawes and Pardo, 2002;
Ho, 2002; Moon, 2002; Holden et al., 2003
Barki et al., 1993; Dawes and Nelson, 1995;
Caffrey, 1998; Ho, 2002; Roy, 2003
McFarlan, 1981; Barki et al., 1993
Heintze and Bretschneider, 2000; Gagnon, 2001

Information
technology

Information and data quality

Dynamic information needs


Usability

Security issues

Technological incompatibility

Technology complexity

Technical skills and experience


Technology newness
Organizational
and managerial

Legal and
regulatory

Project size
Managers attitudes and
behavior
Users or organizational
diversity

McFarlan, 1981; Davis, 1982; Smith et al., 2001;


Dawes and Pardo, 2002; Brown and Brudney, 2003;
Roy, 2003
Dawes and Nelson, 1995

Lack of alignment of
organizational goals and project
Multiple or conflicting goals
Dawes and Pardo, 2002; Brown, 2003; Kim and Kim,
2003
Resistance to change
Dawes and Nelson, 1995; Best, 1997; Caffrey, 1998;
Burbridge, 2002; Ho, 2002; Edmiston, 2003
Turf and conflicts
Barki et al., 1993; Dawes, 1996; Caffrey, 1998;
Bellamy, 2000; Jiang and Kleing, 2000; Barret and
Green, 2001; Burbridge, 2002; Edmiston, 2003;
Rocheleau, 2003; Roy, 2003
Restrictive laws and
Dawes and Nelson, 1995; NGA, 1997; Landsbergen
regulations
and Wolken, 1998; Chengalur-Smith and
Duchessi, 1999; Harris, 2000; Dawes and Pardo, 2002;
Mahler and Regan, 2002
One year budgets
Dawes and Nelson; Fountain, 2001; Dawes and Pardo,
2002
Intergovernmental
Bellamy, 2000; Harris, 2000; Landsberg and Wolken,
relationships
2001; Burbridge, 2002; Dawes and Pardo, 2002;
Rocheleau, 2003
(continued on next page)

192

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

Table 1 (continued)
Challenge category(s) Challenge
Institutional and
environmental

Authors

Privacy concerns

Andersen and Dawes, 1991; Caffrey, 1998; Milner,


2000; Joshi et al., 2002; Moon, 2002; Duncan and
Roehrig, 2003; Edmiston, 2003; Holden et al., 2003
Autonomy of agencies
Dawes, 1996; Caffrey, 1998; Fountain, 2001;
Landsberg and Wolken, 2001; Dawes and Pardo, 2002
Policy and political pressures
Bajjaly, 1999; Heintze and Bretschneider, 2000;
Mahler and Regan, 2002; Brown and Brudney,
2003; Edmiston, 2003; Rocheleau, 2003; Roy, 2003
Environmental context
Heintze and Bretschneider, 2000; Ho, 2002; La Porte
(social, economic, demographic) et al., 2002; Brown and Brudney, 2003; Edmiston,
2003; Holden et al., 2003

Source: Adapted and expanded from Jiang, J. and Klein, G. (2000). Software development risks to project
effectiveness. The Journal of Systems and Software, 52: 310.

systems when the decision process calls for bsoftQ data. Dawes19 and Ambite et al.20 address
how poor results in projects emerge from problems with data structures and data definitions.
Overall, Brown21 cautions us against taking information quality problems for granted.
4.1.2. Information technology challenges
System usability and ease of use are important factors to consider.22 Technology
incompatibility has also been identified as one difficult challenge to IT-intensive projects.23
Systems that are very different and sometimes very old increase the complexity of IT projects,
especially information integration initiatives.24 Complexity and newness of technology are
also constraints that can potentially affect the results of IT projects.25 The lack of relevant
technical skills within the project team has been found to be an important factor26 as well as the
shortages of qualified technical personnel.27 Legacy systems present additional challenges.28
For example, Duchessi and Chengalur-Smith29 reported conversion of mainframe applications
as one of the problems associated with implementing client/server technology.
4.1.3. Organizational and managerial challenges
Undoubtedly, the size of the project and the diversity of the users and organizations
involved are two of the main challenges to IT initiatives.30 There are at least two other
problems related to the goals and objectives of initiatives. The first is the lack of alignment
between organizational goals and the IT project.31 In addition, Dawes and Pardo32 identified
the existence of multiple, and sometimes conflicting, goals in the public sector as an
additional interorganizational challenge. Finally, individual interests and associated behaviors
lead to resistance to change, internal conflicts, and turf issues.33
4.1.4. Legal and regulatory challenges
Most of the time government organizations are created and operate by virtue of a specific
formal rule or group of rules. In making any kind of decision, including those in IT projects,
public managers must take into account a large number of restrictive laws and regulations.34
For example, government agencies must often contend with one-year budget cycles. One-year

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

193

budgets are common in many national and state governments, and this type of budgeting
affects the potential results of long-term IT initiatives.35 Federal systems, as in the United
States, present additional challenges derived from the particularities of the relationships
between different levels of governments and the formal checks and balances among the
executive, legislative, and judicial branches.36
4.1.5. Institutional and environmental challenges
There are additional challenges related to a more general institutional framework and the
policy environment in which government organizations operate.37 In this context, institutions
are not only laws and regulations, but also norms, actions, or behaviors that people accept as
good or take for granted.38 Privacy and related security issues are challenges that must be
adequately addressed in government IT initiatives.39 The United States legal framework does
not mention bagency autonomy.Q However, government agencies and programs often act as
independent and autonomous units without taking into account what other public
organizations are doing (stove pipes). This situation can constrain efforts to use technology
to integrate or share information across multiple agencies.40 Finally, external pressures such
as policy agendas and politics may affect the results of IT initiatives.41
The above discussion highlights the range of highly complex and diverse challenges public
managers must face as they work in the e-government arena. Success is not only about
selecting the right technology, but also about managing organizational capabilities, regulatory
constraints, and environmental pressures. For e-government managers to be successful in
their initiatives they must be aware of these challenges and use appropriate strategies to
overcome them.
4.2. Success strategies for e-government initiatives
A set of strategies for achieving success in e-government initiatives, drawn from the
literature, can be mapped onto the five challenge categories. Mapping the strategies to the
challenge categories illustrates the degree of correspondence in the research itself between
challenges and possible strategies for meeting those challenges (Table 2).
4.2.1. Information and data strategies
Dealing with information and data challenges requires an overall plan for managing data
and information products.42 A quality and compliance assurance program is an effective
strategy for dealing with information and data challenges.43 Developing appropriate data
structures and definitions is critical to the success of IT initiatives, in particular in
interorganizational initiatives. The challenge in this area stems not only from gaining
agreement that these are necessary, but also from engaging the necessary partners in the
development and adoption of common structures and standards.44 Managers have attempted
to minimize data-related problems by sharing standards, definitions and meta-data, with their
potential partners. Getting continual feedback from users is also an important strategy to
maintain data quality.45 Overall, having good quality and homogenous information seems to
be an important success factor.46

194

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

Table 2
Key success strategies for government IT initiatives
Challenge category

Key success strategy

Information and data

Overall plan
Continual feedback from
partners users
Quality and compliance assurance
Training
Information technology Ease of use

Usefulness

Organizational and
managerial

Demonstrations and prototypes


Project team skills and expertise

Well-skilled and respected IT


leader (technical and social skills)
Clear and realistic goals
Identification of relevant
stakeholders
End-user involvement

Planning

Clear milestones and measurable


deliverables

Good communication
Previous business process
improvement
Adequate training
Adequate and innovative funding

Current or best practices review

Source
Wang, 1998
Orr, 1998; CTG, 2000
Keil, 1995; Brown, 2000
Burbridge, 2002
Davis, 1989; DeLone and Mclean, 1992;
Caffrey, 1998; Brown, 2000; DeLone and
McLean, 2003; Garson, 2003
Davis, 1989; DeLone and Mclean, 1992; Brown,
2003; DeLone and Mclean, 2003; Garson, 2003
Caffrey, 1998; Dawes and Pardo, 2002
Barki et al., 1993; Jiang et al., 1996; Brown,
2000; Jiang and Klein, 2000; Regan and
OConnor, 2001; Dawes and Pardo, 2002;
Garson, 2003; Mahler and Regan, 2003;
Melitski, 2003
Gagnon, 2001; West and Berman, 2001;
Dawes and Pardo, 2002; Kim and Kim, 2003;
Mahler and Regan, 2003; Rocheleau, 2003
Best, 1997; Brown, 2000; Dawes and Pardo,
2002; Garson, 2003
Barret and Green, 2001; West and Berman,
2001; Dawes and Pardo, 2002; Brown, 2003
Caffrey, 1998; Regan and OConnor;
West and Berman, 2001; Garson, 2003;
Mahler and Regan, 2003
Bajjaly, 1999; Brown, 2000; Barret and Green,
2001; Landsberg and Wolken, 2001; Smith et al.,
2001; Garson, 2003; Kim and Kim, 2003;
Melitski, 2003
Flowers, 1996; Caffrey, 1998; Bajjaly, 1999
Rocheleau, 2000; Landsberg and Wolken, 2001;
Garson, 2003; Kim and Kim, 2003; Melitski,
2003
Caffrey, 1998; Jiang and Klein, 2000; Brown,
2001; Dawes and Pardo, 2002
Dawes and Nelson, 1995; Best, 1997; NGA,
1997; Harris, 2000; Dawes and Pardo, 2002
Caffrey, 1998; Brown, 2000; Barret and Green,
2001; Garson, 2003
NGA, 1997; Caffrey, 1998; Harris, 2000;
Barret and Green, 2001; Landsberg and Wolken,
2001; West and Berman, 2001; Dawes and
Pardo, 2002; Ho, 2002; Moon, 2002; Edmiston,
2003; Holden et al., 2003
Rocheleau, 2000; Mahler and Regan, 2003

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

195

Table 2 (continued)
Challenge category

Key success strategy

Source

Legal and regulatory

Information technology
policies and standards

Environmental or
institutional

Executive leadership or
sponsorship

Andersen and Dawes, 1991; Dawes and


Nelson, 1995; Caffrey, 1998; Milner, 2000;
Barret and Green, 2001; Landsberg and Wolken,
2001; Garson, 2003; Kim and Kim, 2003
Barki et al., 1993; Jiang et al., 1996; Brown,
2000; Brown, 2001; Landsberg and Wolken,
2001; Edmiston, 2003; Garson, 2003; Mahler
and Regan, 2003; Roy, 2003
Caffrey, 1998
Brown and Brudney, 1998; Barret and Green,
2001; Chen and Perry, 2003; Edmiston, 2003;
Garson, 2003; Melitski, 2003; Roy, 2003

Legislative support
Strategic outsourcing and
publicprivate partnerships

4.2.2. Information technology strategies


Two technology-related factors that can promote the success of information systems are
system usefulness and ease of use.47 Due to the relative complexity and newness of some
technologies a strategy for responding to information technology-related challenges is to
organize presentations about the technologies to build awareness and to focus early efforts on
developing system and process prototypes.48 Strong technical skills and expertise in the
hands of the project leader and some team members is critical.49 It is also important to take
into consideration potential shortages of qualified technical staff and an incremental approach
can help in dealing with this problem.50
4.2.3. Organizational and managerial strategies
Establishing clear and realistic goals is an important factor in the success of IT initiatives.51
Identifying relevant stakeholders and getting them involved in the project development
process, specially end-users, has also been found to be an effective strategy in overcoming
organizational and managerial challenges.52 Strategic planning techniques can be seen as an
umbrella for more specific strategies such as clear milestones and measurable deliverables;53
good communication channels;54 and previous business process improvement.55 It is also
extremely important to take care of developers and end-users current skills and training
needs.56 Successful projects need a balanced combination of technical, managerial, and
political skills and expertise among their members.57 Finally, financial resources are not
always the most important factor, but are necessary. Often, managers need to develop
innovative financial schemes and partnerships to get e-government initiatives off the ground.58
4.2.4. Legal and regulatory strategies
Restrictive laws and regulations developed prior to or in ignorance of technologies relevant
to e-government can affect the success of projects. One strategy for responding to these
challenges is to invest in changes to the regulatory environment that allow for or enable
adoption of emerging technologies.59 Digital signature technologies, for example, required
statutory changes in most jurisdictions before they could be adopted for use. Developing

196

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

appropriate government-wide IT policies and standards can also provide and adequate
framework for e-government initiatives to be successful.60 In this regard, state governments
are developing IT policies and standards and making them available through their official
Web sites.61
4.2.5. Institutional and environmental strategies
Individual leaders or managers cannot change institutionalized rules or practices. However,
if a coalition is large and varied enough to capture the attention of legislators or other policy
makers, some formal institutions can be changed.62 There are at least two strategies to deal
with institutional and environmental factors: getting executive and legislative support;63 and
using outsourcing strategically.64

5. Providing guidance to practitioners: four practical tools


The four categories of challenges and strategies provide a framework for tracking the
impact of research on the practices of public managers through the use of practical guides. A
summary of each of the four selected guides is presented below followed by a brief
comparative analysis (Table 3). Each practitioner guide is then examined relative to the four
challenge and strategy categories. This analysis concludes with a set of observations and
recommendations about future investments in practical guides to support government IT
decision makers.
5.1. Value measuring methodology65
This set of guides was produced by the Federal Chief Information Officer Councils Best
Practices Committee to improve government IT decision making. The Best Practices
committee is bchartered to provide in-depth examples and practical guidance to successfully

Table 3
Selected practitioner tools
Source

Year

Primary audience

Tool

Federal CIO Council

2002

Value Measuring Methodology (VMM)

Treasury Board
of Canada

1998

Members of the federal


information technology
community
Canadian public managers

National Association
of State Chief
Information Offices
Center for Technology
in Government

2003

State Agencies in all 50 states

2003

National, State, and Local


governments

Creating and Using a Business


Case for Information Technology
Projects (CUBC)
Business Case Basics and Beyond:
A Primer on State Government IT
Business Cases (BCBB)
Making Smart IT Choices (MSIT)

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

197

formulate, manage and maintain the portfolio of initiatives to ensure that investments made in
IT yield the anticipated benefitQ to members of the federal information technology community.
The Value Measuring Methodology (VMM) How-To-Guide builds on the prior work of
two efforts in particular. In 2001 the Social Security Administration (SSA), in cooperation
with the General Services Administration (GSA), began to develop a methodology to asses
the value of electronic services. Their goal was to produce a tool that that would be
bcompliant with current Federal regulations and OMB guidance, applicable across the Federal
Government, and pragmatically focused on implementation.Q66 In addition, a team from Booz
Allen Hamilton and the Kennedy School of Government at Harvard was asked to conduct a
related study. That report, based on interviews with a variety of professionals in the field as
well as the private sector and the academic community, presented the first version of the
VMM, its supporting theories, and philosophy. Since the initial release VMM has been
applied and refined into its current form.
VMM, like the methodologies and frameworks presented in the other guides, calls for the
inclusion of a broad set of stakeholders affected by the initiative, including direct users and
government partners. The warrant for this guide was the gap between current tools and the
bneed for a more thorough and rigorous analytical approach to investment evaluation,
planning, and management.Q67 VMM is positioned as responding to this need by providing a
bcomprehensive and quantitative way to capture the impact that possible investment
alternatives would have on each of these parties.Q68 The methodology is designed in particular
to focus analysis on the value, cost, and risk baseline for any initiative, changes to those
baseline measures over time, and the implications of those changes.
The VMM How-To-Guide is organized into eight sections. The Essential Factors
Framework of value, cost, and risk, the foundation of VMM, is introduced in Section 3
together with a discussion of the value gained from using the VMM methodology to analyze
e-government and other initiatives. Section 4 presents an overview of the four steps of the
VMM. Section 5 provides a comprehensive, step-by-step presentation of the techniques and
tools of VMM as well as a discussion of the resources necessary to complete a VMM
analysis, key concepts and real-life lessons from past implementations, and some best
practices observations.
5.2. Creating and using a business case for information technology projects69
This guide was issued by the Project Management Office, Chief Information Officer
Branch of the Treasury Board of Canada. The production of the guide was organized through
the Project Management Office and was staffed with volunteer members of a working group
as well as many additional volunteers who wrote, reviewed, and contributed to the guide. A
member of the working group was acknowledged for directing the effort and coordinating the
participation of others.
According to its statement of purpose this guide, developed by public service managers for
their colleagues, boffers a blueprint that managers can use to build the business cases needed
to make informed investment decisions.Q70 The Canadian guide is organized around two
consistent themes. The first is that a bbusiness case is the key element of front-end planning

198

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

and sets the stage for the management of the project and for the achievement of the planned
benefit.Q71 It is considered an bindispensable first activity in the life cycle of an IT
investmentQ72 and when bcorrectly used can serve as a management framework for the
initiative.Q73 The second theme is that no one size fits all. Public managers are urged to use
the business case development process to put their decisions into a bstrategic context.Q
This guide exists within a larger set of guidance from the Treasury Board. The guide
consistently points the reader back to the larger context of governing IT policies as well as the
official Treasury Board framework for managing IT projects. Specifically, wherever
appropriate, the reader is directed to relevant text in the Treasury Boards Management of
Information Technology Policy and their An Enhanced Framework for the Management of
Information Technology Projects.
This tool is structured for use as both a source book and a road map through the IT
investment process for public managers. The introduction also positions it as a tool to
bintroduce other stakeholders to the framework that shapes the decision-making model.Q74
The five central chapters of the guide introduce the framework for examining the
environment and seeking insight about the specific barriers, risks, and benefits of each
solution alternative being examined. The next two chapters focus on customizing the case for
specific audiencesagain stressing the idea that no one size or focus of presentation fits all
audiences. Chapter 10 focuses on tips and techniques for designing and managing ongoing
project reviews. Finally, the appendix introduces Logical Framework Analysis, a dynamic
technique for planning, communicating, and controlling project elements.
5.3. Business case basics and beyond: a primer on state government IT business cases75
This guide was produced by the National Association of Chief Information Officers
(NASCIO). A primary author from the practitioner community worked together with the
NASCIO Executive Committee to write the guide. Feedback was also provided by NASCIOs
customer relationship management committee and a range of government practitioners and
private sector and academic partners. Thirty-eight people from state and federal governments
and fourteen individuals from associations, academic institutions, and the private sector were
acknowledged for their contributions.
The guide addresses the emerging trend of business case use being broadened beyond the
analysis of one project to identify the benefits of whole programs such as data center
consolidation and Y2K. This guide identifies a review of current practitioner literature on IT
business cases as the source of a framework for an enterprise business case. It also presents a
discussion of the current challenges state governments, in particular, are facing in their egovernment initiatives and meeting the policy and service goals of their Governors. The
NASCIO guide provides btools, concepts, and a framework for addressing a number of critical
challenges facing state Governors, chief information officers and enterprise information
technology organizations.Q76 It has three main purposes: (1) provide the basics on State IT
business cases, (2) push beyond the bBasics to Use the Business Case to Address the
Challenges of Fiscal Years 2004 and 2005,Q77 and (3) embrace a statewide enterprise IT
investment management infrastructure. It contains four different types of information for

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

199

public managers: (1) business case basics, (2) public sector approaches to business cases and
examples, (3) resources and contacts, and (4) suggested solutions to some of the challenges.
One of the unique contributions of this guide is its specific discussion and treatment of an
enterprise business case as separate and unique from a business case for a specific project. It
also speaks to the value of technology in the business of government.
5.4. Making smart IT choices78
This two-part guide, available in print and online, was produced by The Center for
Technology in Government, an applied research center at the University at Albany, SUNY. The
Center formulated the Smart IT methodology through its work with government agencies in
projects where information strategies are applied to the challenges of public service delivery.
According to its introduction, this guide was designed to help public managers and
government organizations bmake good decisions about when and how to invest in information
technology (IT).Q79 Put another way, it was designed to bhelp public managers avoid becoming
one of the statistics that dominate reports on information technology investments.Q80
Smart IT is organized around a short list of basic issues and principles. The problematic
and expensive nature of IT decisions and the high failure rates, which result from hasty,
unrealistic, or uninformed decisions provide the basic issues framework. The principles that
guide the analytical strategy of Smart IT are that public managers must identify and listen to
stakeholders; they must understand what constitutes success for their initiative and they must
pursue it; and form must follow function. The three phase process involves the use of
analytical tools and techniques to first understand the problem and its context; second,
identify and test solutions; and third, evaluate alternatives and make choices. A wide range of
tools is introduced as well as a way of understanding how each tool might best be employed
in the development of a business case within a particular economic, policy, organizational,
managerial, process, and technology context.
Part 1 has four chapters and begins by considering the special characteristics of the public
sector as an environment for making management decisions and IT choices. In the second
chapter, the analytical process that accounts for program goals, stakeholders, processes, costs,
and technology alternatives is presented. Mini case examples are provided throughout.
Chapters 3 and 4 focus on turning the analysis into a business case and presenting it to various
audiences. Part 2 presents 33 skills, techniques, and tools to use in the analytical process.

6. Mapping practical tools to theoretical foundations


The four selected guides are very different in their genesis and designyet they all present
analysis and planning frameworks that can apply both to specific IT initiatives and more
broadly to enterprise IT strategic planning. They all represent the best ideas of leading
practitioners and academics in this area. The NASCIO guide expressed the warrant that all
four guides seem to be responding tothat in IT business cases brisks, sensitivities, and
contingencies tend to be undeveloped and contribute to lowered credibility.Q81

200

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

6.1. A brief comparative analysis of the selected guides


All four guides were produced through collaborative efforts of the public, private, and
academic communities (Table 4). The NASCIO, VMM, and the CUBC guides were produced
with a governmental sponsor and a range of volunteer expert participants. In each situation
the need for the guide emerged from the ranks and was responded to by a coordinating group
or professional association. The Smart IT Guide was produced through a different strategy
by practice-oriented researchers drawing on their experiences working directly with
government agencies in developing IT initiatives. Nonetheless, there is notable consistency
in the messages they deliver.
All of the guides stress contingency thinking, heavily investing in upfront or what CTG
calls bbefore the beginningQ analysis, collaborative management models, inclusion of IT and
business stakeholders, and the value of iterative analytical processes. The impact that a dwelldoneT business case can have in the ongoing design, development, implementation and
review of an e-government initiative is also acknowledged in all four guides.
The guides vary considerably in their recommendations of specific tools and techniques to
carry out a business case analysis. The Canadian guide and the VMM link the analytical
framework very closely to a specific tool. Smart IT presents an analytical framework and
links steps in the analysis to a group of tools from which public managers can choose. The
guides vary in their links between recommended actions and the specific issue or challenge
likely to be overcome by that particular action. Smart IT in particular tends to link analysis to
the challenge addressed so that practitioners can anticipate which tools or techniques will
Table 4
Selected tools: development strategy and focus
Tool

Development strategy

Focus

Value Measuring
Methodology

Best Practices Committee of the


Federal CIO Council built on
previous related efforts of the
committee. Refined in use at
several federal agencies
Developed by public service
managers for their colleagues

The guides provide a particular


methodology for evaluating and
selecting initiatives based on ongoing
value, cost, and risk determinations

Creating and
Using a Business
Case for Information
Technology Projects
NASCIO Business
Case Primer

Making Smart
IT Choices

Single author, broadly based


review committee

Developed by an applied research


center based on experiences in
working with government agencies

The guide offers a blueprint that


managers can use to build the business
cases needed to make informed
investment decisions
The primer provides tools, concepts,
and a framework for addressing a
number of critical challenges facing
state Governors, CIOs, and enterprise
information technology organizations
This guide provides concepts, techniques,
and tools to help organizations define an
information technology project and make
a solid case for needed financial and
organizational investments

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

201

impact which challenges. The NASCIO guide presents a warrant for business case analysis;
its unique contribution is a well-developed strategy for assessing the effectiveness of a
business case and the compilation of current practices across the states. Readers of the
NASCIO guide have the opportunity to review many frameworks and tools sets for a model
that will support their environments and issues as well as an extensive reference list.
The NASCIO guide offers examples of business cases and IT investment analysis
processes from as many states as possible. The VMM guide devotes most of its space to a
comprehensive risk and cost alternative analysis. The Smart IT guide focuses its presentation
on a set of tools and techniquesa toolkit of sort, and on the introduction of an analytical
framework to which public managers can apply appropriate tools from the toolkit. The
Canadian guide provides the reader with a comprehensive overview, targeted and actionable
summaries of activity within each step, and unique among the guides, provides
recommendations for the design of a project evaluation process.
6.2. Building the research practice bridge
The selected guides were designed to build awareness of challenges to e-government
initiatives and to present useful strategies, tools, and techniques to overcome the challenges.
The following section highlights the links between the research literature and the selected
guides through an analysis of each guide in terms of the four categories of challenges and
strategies derived from the literature review. Several observations about the guides and
recommendations for future development of practitioner resources and for further research
into the relevance of research to practice are then provided (Tables 5 and 6).
6.2.1. Information and data factors
Research published in the public management literature related to the capture,
organization, management, use, and archiving of information and data appears to be limited.
Coverage of the information and data challenges and the presentation of strategies related to
overcoming them is also limited. The Canadian guide identifies initial data collection and
conversion of archival data as a cost category that must be taken into account when
considering the overall cost of an initiative. Smart IT urges public managers to consider the
information and data that a particular solution strategy depends on to be successful so that
cost and risk analysis can include any threats to access. Other resources addressing these
factors are beginning to emerge from recent experiences with government information
integration initiatives. These initiatives are providing new insights into the challenges posed
by information and data issues;82 insights that need to be reflected in future practice guides.
6.2.2. Information technology factors
In the case of information technology, Canadas guide addresses almost all the challenges.
The risks section of the guide is based on research about software development done by the
Software Engineering Institute at Carnegie Mellon University.83 All three tools discuss the
need for technical skills and expertise. Two address the complexity of the technology used in
the project. Two others address usefulness and ease of use as important elements.

202

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

Table 5
E-government challenges address by selected tools
VMM

CUBC

BCBB

MSIT

Security issues

Technological incompatibility

Technology complexity

Technical skills and/or expertise

Technology newness

Information and data challenges


Information and data quality

Information technology challenges

U
U

Organizational and managerial challenges


Project size and related complexity

Users or organizational diversity

Lack of alignment between organizational


goals and IT project

Multiple or conflicting goals

Resistance to change

Turf and conflicts


Legal and regulatory challenges
Restrictive laws and regulations

One-year budget restrictions


Potential intergovernmental relationships

U
U

Institutional and environmental challenges


Privacy concerns

Institutional arrangements (e.g., autonomy of agencies)

Competition or political pressures (e.g., timing)

Identification of partners and their contributions

Lessons from previous IT experiences

U
U

6.2.3. Organizational and managerial factors


All of the guides speak to the challenges that stem from organizational and managerial
realities. Each introduces the discussion of these challenges at a different level, based on
their target audience and style. The NASCIO guide speaks to organizational challenges
primarily at an enterprise level seeking to encourage CIOs and other leaders to consider the

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

203

Table 6
Recommended by selected guides
VMM

CUBC

BCBB

Quality and/or compliance assurance

Ease of use

MSIT

Information and data strategy

Usefulness as one of the main goals

Information technology strategy


Demonstrations and prototypes

Well-established information technology


policies and standards
Organizational and managerial strategy
Project team skills and expertise (development
and application)

Well-skilled and respected project leader (technical


and social skills)

Establishment of clear and realistic goals

Identification of relevant stakeholders

End-users involvement (design, development


and evaluation)

Planning as a powerful management tool

Clear milestones and/or measurable deliverables

Good communication (bottomtop and topbottom)

U
U

Previous business process improvement

Adequate training
Adequate and/or innovative funding

Strategic outsourcing or other sourcing options

U
U

Best practices review

Evaluation tools and processes

Legal and regulatory strategy


Legislative support
Environmental and institutional strategy
Executive leadership and/or sponsorship

204

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

strategic role of the enterprise. The CUBC focuses more directly on initiative level
managerial challenges such as skills of the project leader, planning as a management tool,
and adequate funding. VMM provides techniques for assessing the impact of organizational
and managerial risks such as a lack of alignment. Further it provides techniques for testing
underlying assumptions about initiatives and for assessing the impact of incorrect
assumptions on an initiative plan. Smart IT raises awareness about organizational and
managerial risks such as lack of attention to users, a lack of alignment between initiative and
organizational goals, and multiple and conflicting goals. It provides techniques such as
structured service objectives, visioning, and a strategic framework for minimizing these
challenges. All of these guides urge stakeholder involvement as a critical strategy for
overcoming organizational and managerial challenges.
6.2.4. Legal and regulatory factors
Legal and regulatory challenges are well addressed in the guides. The NASCIO guide
covers the topic of legal frameworks and provides guidance on reviewing governing policies
and regulations. It advises public managers to be aware of regulatory frameworks in the
development of the IT initiatives. Smart IT speaks to these challenges as well by providing
tools such as partisan analysis, stakeholder analysis, and news analysis. Each guide presents a
discussion of the possible enabling and constraining influences of the regulatory environment
on e-government initiatives. Managers are urged, for example, to carefully consider security
policies that restrict or facilitate infrastructure expansion efforts.
6.2.5. Institutional and environmental factors
Challenges stemming from environmental realities and institutional practices are identified
in each the four selected guides. Timing, in particular, is presented as a strategy for
overcoming some of the environmental and institutional challenges. The NASCIO Guide for
example, encourages public managers to understand the impact of election cycles on their
initiatives. Smart IT also speaks to these environmental challenges and adds in the one-year
budgetary cycles of government and its challenges to multi-year, multi-institutional IT
initiatives. The NASCIO guide, the Canadian Treasury Board guide, and the Smart IT guide
speak to the need to scan the economic and political environments and to carefully choose the
btimingQ of the IT initiative. Smart IT provides a number of tools and techniques for
increasing awareness environmental and institutional factors and their influence.

7. Final comments and recommendations


Understanding and reducing risk in e-government initiatives is a high priority for both
researchers and practitioners. One consequence of attention to risk is that organizations, both
public and private, are increasing their investments in standard tools for planning and managing
IT initiatives. This study has sought to examine whether the tools available to practitioners
benefit from research in IS. We believe the answer is bYes.Q Increasingly, practical guides are
urging practitioners to think beyond technology; to think about bsystems in context.Q84

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

205

Our comparison of selected research findings and practical guides has provided insight into
the extent to which research is reflected in guides to inform practice. The analysis highlighted
the particular characteristics of the guides and provided a general review of the extent to
which these guides reflect current research. Five observations about the key commonalties
among the guides (see Table 7) and four recommendations for future efforts in both research
and practice also emerged from the analysis.
The four recommendations derived from the analysis of gaps between current research and
the guides are provided to inform both future IS research and practical guide development
efforts. This work also informs future efforts to conduct further empirical studies of the
impact of research on the focus and content of e-government investment decision making
tools in international, national, state, and local governments.
7.1. Characterize risk in context
The guides introduce the concept of risk and speak to the need to identify and invest in
strategies to manage risk. Risk as a concept receives very thorough treatment, however,
challenges tend to be characterized in general terms. A more thorough characterization of the
specific risks to specific kinds of initiatives would complement the presentation of risk
identification and analysis strategies. Practitioners would be more aware of the potential risks
they face and could consider the analytic methodologies in terms of a set of likely risks given
the context of a particular initiative.
7.2. Build understanding of information and data challenges
The impact of information and data challenges such as inconsistent data structures, semantic
issues, and incomplete data, on the success of e-government initiatives need to be explored
further in research and presented more thoroughly in practitioner guides. The success of egovernment initiatives involving multi-agency information sharing and integration such as
homeland security and public health depend on greater understanding of these challenges.

Table 7
Key commonalities of selected E-government practitioner guides
1. Risks, sensitivities, and contingencies tend to be undeveloped and that threatens the credibility of IS
initiatives (NASCIO 2003)
2. The iterative process of information gathering, analysis, and decision making is central to the effort to
build understanding of problems, solutions, alternatives, costs, and risks
3. Investment analysis and business case documents are living documents and if maintained and updated over
time can provide guidance to teams throughout an initiative and beyond
4. Contextualizing technology solutions is critical to success. bNo one size fits allQ is the underlying theme
throughout each of the guides. Finding out what size does fit in which situation is the goal each guide sets
out to achieve
5. Contingency thinking is necessary for planning and preparation for unexpected consequences and changes
in the bvalue, cost, and riskQ determinations

206

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

7.3. Build a business case for business case analysis


Empirical support for the investment in business case and investment analysis is lacking. The
guides present common-sense rationales for using business case strategies. Some outline the
necessary elements. The NASCIO guide outlines a set of key value indicators for a sound business
case. The Smart IT Guide is illustrated throughout with excerpts from a business case. However, a
robust empirical base for particular business case strategies within particular environments and
contexts would provide public managers with a more informed roadmap for their efforts.
7.4. Explore the impact of contingency thinking in project planning and management
The guides generally focus on the how-to of investment analysis and business case
development. Future guides need to continue to enhance the capability of public managers to
identify, influence and act on the dynamic set of contingencies that influence information
technology initiatives. Project management training for IT managers, for example, can
contribute as well to the ability of government managers to understand and prepare for the
dynamic environment of e-government initiatives.
Governments at all levels have invested in the development of guides and handbooks to
raise awareness of the challenges facing IT initiatives. These practical guides are providing a
bridge between knowledge gained through research in information systems, public
administration, political science, organization theory, and management and the knowledge
gained by practitioners participating in and leading these technology initiatives. These
resources provide an opportunity, when developed with full knowledge of current research and
practice, to build awareness about these challenges and to provide tools and techniques that can
lead to success. This study supports the conclusion that although practitioners may not read IS
research directly, this research is finding its way, at least recently, onto their desktops and into
their briefcases, and presumably into their IT initiatives through the pages of research-aware
practical guides, reference tools, and handbooks. Further empirical work focusing on the extent
to which practitioners are using these and similar guides and the nature of that use is called for.

Acknowledgments
The authors want to thank Sharon Dawes, Terry Maxwell, Anthony Cresswell, and Luis
Luna-Reyes for their valuable comments in early versions of this paper. Any mistakes or
omissions are the sole responsibility of the authors.

Appendix A. Bibliography included in Tables 1 and 2


Ambite, J. L. et al. (2002). Data integration and access. In W. J. McIver & A. K.
Elmagarmid (Eds.). Advances in Digital Government, Technology, Human Factors, and
Policy. Norwell, MA: Kluwer Academic Publishers.

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

207

Andersen, D. F., & Dawes, S. S. (1991). Government Information Management. A Primer


and Casebook. Englewood Cliffs, NJ: Prentice Hall.
Bajjaly, S. T. (1999). Managing emerging information systems in the public sector. Pubic
Performance and Management Review, 23(1), 4047.
Ballou, D. P., & Tayi, G. K. (1999). Enhancing data quality in data warehouse
environments. Communications of the ACM, 42(1), 7379.
Barki, H., Rivard, S., & Talbot, J. (1993). Toward an assessment of software development
risk. Journal of Management Information Systems, 10, 203223.
Barret, K., & Green, R. (2001). Powering Up. How Public Managers Can Take Control of
Information Technology. Washington, DC: CQ Press.
Bellamy, C. (2000). The politics of public information systems. In G. D. Garson (Ed.).
Handbook of Public Information Systems. New York: Marcel Dekker.
Best, J. D. (1997). The Ditigal Organization. New York: Wiley.
Brown, M. M., & Brudney, J. L. (1998). A bsmarter, cheaper, and fasterQ government?
Contracting for geographic information systems. Public Administration Review, 58, 335345.
Brown, M. M. (2000). Mitigating the risk of information technology initiatives: Best
practices and points of failure for the public sector. In G. D. Garson (Ed.). Handbook of
Public Information Systems. New York: Marcel Dekker.
Brown, M. M. (2001). The benefits and costs of information technology innovations: An
empirical assessment of a local government agency. Pubic Performance and Management
Review, 24(4), 35166.
Brown, M. M. (2003). Technology diffusion and the bKnowledge BarrierQ: The dilemma of
stakeholder participation. Public Performance and Management Review, 26(4), 345359.
Brown, M. M., & Brudney, J. L. (2003). Learning organizations in the public sector? A
study of police agencies employing information and technology to advance knowledge.
Public Administration Review, 63(1), 3043.
Burbridge, L. (2002). Accountability and MIS. Public Performance and Management
Review, 25(4), 421423.
Caffrey, L. (1998). Information Sharing between and within Governments: A Study Group
Report. London: Commonwealth Secretariat.
Chen, Y. -C., & Perry, J. (2003). Outsourcing for e-government: Managing for success.
Public Performance and Management Review, 26(4), 404421.
Chengalur-Smith, I., & Duchessi, P. (1999). The initiation and adoption of clientserver
technology in organizations. Information and Management, 35, 7788.
Davis, G. B. (1982). Strategies for information requirements determination. IBM System
Journal, 21, 430.
Davis, F. D. (1989) Perceived usefulness, perceived ease of use and user acceptance of
information technology. MIS Quarterly, 13, 319330.
Dawes, S. S. (1996). Interagency information sharing: Expected benefits, manageable
risks. Journal of Policy Analysis and Management, 15(3), 377394.
Dawes, S. S., & Nelson, M. R. (1995). Pool the risks, share the benefits: Partnerships in IT
innovation. In J. Keyes (Ed.). Technology Trendlines. Technology Success Stories from
Todays Visionaries. New York: Van Nostrand Reinhold.

208

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

Dawes, S. S., & Pardo, T. (2002). Building collaborative digital government systems.
In W. J. McIver & A. K. Elmagarmid (Eds.). Advances in Digital Government.
Technology, Human Factors, and Policy. Norwell, MA: Kluwer Academic Publishers.
DeLone, W., & Mclean, E. (1992). Information systems success: The quest for the
dependent variable. Information Systems Research, 3(1), 6095.
DeLone, W., & Mclean, E. (2003). The DeLone and McLean Model of information
systems success: A ten year update. Journal of Management Information Systems, 19(4),
930.
Duncan, G. T., & Roehrig, S. T. (2003). Mediating the tension between information
privacy and information access: The role of digital government. In G. D. Garson (Ed.), Public
Information Technology: Policy and Management Issues. Hershey, PA: Idea Group
Publishing.
Edmiston, K. D. (2003). State and local e-government: Prospects and challenges.
American Review of Public Administration, 33(1), 2045.
Flowers, S. (1996). Software failure: Management failure: Amazing Stories and
Cautionary Tales. New York: Wiley.
Fountain, J. E. (2001). Building the Virtual State. Information Technology and Institutional
Change. Washington, DC: Brookings Institution Press.
Gagnon, Y. -C. (2001). The behavior of public managers in adopting new technologies.
Pubic Performance and Management Review, 24(4), 337350.
Garson, G. D. (2003). Toward an information technology research agenda for public
administration. In G. D. Garson (Ed.). Public Information Technology: Policy and
Management Issues. Hershey, PA: Idea Group Publishing.
Gray, P. (2001). Introduction to the special volume on relevance. Communications of the
Association of Information Systems, 6.
Harris, N. D. (2000). Intergovernmental cooperation in the development and use of
information systems. In G. D. Garson (Ed.). Handbook of Public Information Systems. New
York: Marcel Dekker, Inc.
Heintze, T., & Bretschneider, S. (2000). Information technology and restructuring in public
organizations: Does adoption of information technology affect organizational structures,
communications, and decision making? Journal of Public Administration Research and
Theory, 10(4), 801830.
Ho, A. T. -K. (2002). Reinventing local governments and the e-government initiative.
Public Administration Review, 62(4), 434444.
Holden, S. H., Norris, D. F., & Fletcher, P. D. (2003). Electronic government at the local
level: Progress to date and future issues. Public Performance and Management Review,
26(4), 325344.
Irvine, C. E. (2000). Security issues for automated information systems. In G. D. Garson
(Ed.). Handbook of Public Information Systems. New York: Marcel Dekker, Inc.
Jiang, J., & Klein, G. (2000). Software development risks to project effectiveness. The
Journal of Systems and Software, 52, 310.
Jiang, J., Klein G., & Balloun, J. (1996). Ranking of system implementation success
factors. Project Management Journal, 27, 5055.

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

209

Joshi, J. B. D., Ghafor, A., Aref, W. G., & Spafford, E. H. (2002). Security and privacy
challenges of a digital government. In W. J. McIver & A. K. Elmagarmid (Eds.). Advances in
Digital Government. Technology, Human Factors, and Policy. Norwell, MA: Kluwer
Academic Publishers.
Keil, M. (1995). Pulling the plug: Software project management and the problem of
escalation. MIS Quarterly, 19(4), 421447.
Kim, S., & Kim, D. (2003). South Korean public officials perceptions of values, failure,
and consequences of failure in e-government leadership. Public Performance and Management Review, 26(4), 360375.
La Porte, T. M., Demchak, C. C., & de Jong, M. (2002). Democracy and bureaucracy in
the age of the Web: Empirical findings and theoretical speculations. Administration and
Society, 34(4), 411446.
Landsbergen, D. J., & Wolken Jr., G. (1998, October). Eliminating legal and policy
barriers to interoperable government systems. Paper presented at the Annual Research
Conference of the Association for Public Policy Analysis and Management. New York.
Landsbergen, D. J., & Wolken Jr., G. (2001). Realizing the promise: Government
information systems and the fourth generation of information technology. Public Administration Review, 61(2), 206220.
Luna-Reyes, L., & Gil-Garcia, J. R. (2003, May 1821). eGovernment & Internet Security:
Some Technical and Policy Considerations. Paper presented at the National Conference on
Digital Government Research, organized by the National Science Foundation, Boston, MA,
USA.
Mahler, J., & Regan, P. M. (2002). Learning to govern online: Federal agency Internet use.
American Review of Public Administration, 32(3), 326349.
Mahler, J., & Regan, P. M. (2003). Developing intranets for agency management. Public
Performance and Management Review, 26(4), 422432.
McFarlan, F. W. (1981). Portfolio approach to information systems. Harvard Business
Review, 59, 142150.
Melitski, J. (2003). Capacity and e-government performance: An analysis based on early
adopters of Internet technologies in New Jersey. Public Performance and Management
Review, 26(4), 376390.
Milner, E. M. (2000). Managing Information and Knowledge in the Public Sector. New
York: Routledge.
Moon, M. J. (2002). The evolution of e-government among municipalities: Rhetoric or
reality? Public Administration Review, 62(4), 424433.
NGA. (1997). Barriers to Intergovernmental Enterprise. Washington, DC: National
Governors Association.
Orr, K. (1998). Data quality and systems theory. Communications of the ACM, 41(2),
6670.
Redman, T. C. (1998). The impact of poor data quality on the typical enterprise.
Communications of the ACM, 41(2), 7982.
Regan, E. A., & OConnor, B. N. (2001). End-user Information Systems: Implementing
Individual and Group Work Technology. Second Edition. New York: Prentice Hall.

210

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

Rocheleau, B. (2000). Prescriptions for public-sector information management: A review,


analysis, and critique. American Review of Public Administration, 30(4), 414435.
Rocheleau, B. (2003). Politics, accountability, and government information systems.
In G. D. Garson (Ed.). Public Information Technology: Policy and Management Issues.
Hershey, PA: Idea Group Publishing.
Roy, J. (2003). The relational dynamics of e-governance: A case study of the City of
Ottawa. Public Performance and Management Review, 26(4), 391403.
Smith, L. D., Campbell, J. F., Subramanian, A., Bird, D. A., & Nelson, A. C. (2001).
Strategic planning for municipal information systems: Some lessons from a large U.S. City.
American Review of Public Administration, 31(2), 139157.
Tayi, G. K., & Ballou, D. P. (1998). Examining data quality. Communications of the ACM,
41(2), 5456.
Wang, R. Y. (1998). A product perspective on total data quality management.
Communications of the ACM, 41(2), 5863.
West, J. P., & Berman, E. M. (2001). The impact of revitalized management practices
on the adoption of information technology: A national survey of local governments.
Pubic Performance and Management Review, 24(3), 233253.
Notes and References
1. Gil-Garca, J. Ramon, & Luna-Reyes, Luis F. (2003). Towards a definition of electronic
government: A comparative review. In Mendez-Vilas A., et al. (Eds.), Techno-legal
aspects of information society and new economy: An overview. Extremadura, Spain7
Formatex Information Society Series.
2. Best, J. D. (1997). The Digital Organization. New York7 Wiley;
Dawes, S. S., & Pardo, T. (2002). Building collaborative digital government systems. In
W. J. McIver, & A. K. Elmagarmid (Eds.), Advances in digital government. Technology,
human factors, and policy. Norwell, MA7 Kluwer Academic Publishers;
Landsbergen, D. J., & Wolken Jr., G., (1998, October). Eliminating Legal and
Policy Barriers to Interoperable Government Systems. Paper presented at the
annual research conference of the association for public policy analysis and
management, New York.
3. Bellamy, C. (2000). The politics of public information systems. In G. D. Garson (Ed.),
Handbook of Public Information Systems. New York7 Marcel Dekker, Inc.;
Duncan, G. T., & Roehrig, S. T. (2003). Mediating the tension between information
privacy and information access: The role of digital government. In G. D. Garson (Ed.),
Public Information Technology: Policy and Management Issues. Hershey, PA7 Idea Group
Publishing;
Milner, E. M. (2000). Managing Information and Knowledge in the Public Sector. New
York7 Routledge;
Rocheleau, B. (2003). Politics, accountability, and government information systems.
In G. D. Garson (Ed.), Public Information Technology: Policy and Management
Issues. Hershey, PA7 Idea Group Publishing.

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

211

4. Bozeman, B., & Bretschneider, S. (1986). Public management information systems:


Theory and prescriptions [Special issue]. Public administration review, Vol. 46
(pp. 475487);
Caffrey, L. (1998). Information Sharing between and within Governments: A Study Group
Report. London7 Commonwealth Secretariat;
Garson, G. D. (2003). Toward an information technology research agenda for public
administration. In G. D. Garson (Ed.), Public information technology: Policy and
management issues. Hershey, PA7 Idea Group Publishing;
Southon, G., Sauer, C., & Dampney, K. (1999). Lessons from a failed information
systems initiative: Issues for complex organizations. International Journal of Medical
Informatics, 55(1), 3346.
5. Khazanchi, D., & Munkvold, B. E. (2001). Expanding the notion of relevance in IS
research: A proposal and some recommendations. Communications of the Association for
Information Systems, 6.
6. Cresswell, A. M. (2001). Thoughts on relevance of IS research. Communications of the
Association for Information Systems, 6.
7. Mathieson, K., & Ryan, T. (2001). A broader view of relevance. Communications of the
Association of Information Systems, 6.
8. Gray, P. (2001). Introduction to the special volume on relevance. Communications of the
Association of Information Systems, 6.
9. Ibid., p. 8.
10. Ho, J., & Pardo, T. A. (2004, January 58). Toward the Success of eGovernment
Initiatives: Mapping Known Success Factors to the Design of Practical Tools. Paper
presented at the 37th Hawaii International Conference on System Sciences (HICSS),
organized by the College of Business, University of Hawaii at Manoa, USA.
11. The Public Administration journals reviewed were Public Administration Review,
Journal of Public Administration Research and Theory, American Review of Public
Administration, Administration and Society, and Public Performance and Management.
12. Chengalur-Smith, I., & Duchessi, P. (2000). Client-server implementation: Some
management pointers. Transactions on Engineering Management, 47(1), 127145;
Ibid., p. 2, Dawes;
DeLone, W., & Mclean, E. (2003). The DeLone and McLean model of information
systems success: A ten year update. Journal of Management Information Systems, 19(4),
930;
DeSanctis, G., & Poole, M. S. (1994). Capturing the complexity in advanced technology
use: Adaptive structuration theory. Organization Science, 5(2), 121147;
Ibid., p. 2, Landsbergen and Wolken;
Orlikowski, W. J. (2000). Using technology and constituting structures: A practice lens
for studying technology in organizations. Organization Science, 11(4), 404428.
13. Redman, T. C. (1998). The impact of poor data quality on the typical enterprise.
Communications of the ACM, 41(2), 7982.
14. Kaplan, D., Krishnan, R., Padman, R., & Peters, J. (1998). Assessing data quality in
accounting information systems. Communications of the ACM, 41(2), 7277.

212

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

15. Ballou, D. P., & Tayi, G. K. (1999). Enhancing data quality in data warehouse
environments. Communications of the ACM, 42(1), 7379.
16. Ibid., p. 13.
17. Ibid., p. 14.
18. Ibid., p. 15.
19. Dawes, S. S. (1996). Interagency information sharing: Expected benefits, manageable
risks. Journal of Policy Analysis and Management, 15(3), 377394.
20. Ambite, J. L., et al. (2002). Data integration and access. In W. J. McIver, & A. K.
Elmagarmid (Eds.), Advances in Digital Government. Technology, Human Factors, and
Policy. Norwell, MA7 Kluwer Academic Publishers.
21. Brown, M. M. (2000). Mitigating the risk of information technology initiatives: Best
practices and points of failure for the public sector. In G. D. Garson (Ed.), Handbook of
Public Information Systems. New York7 Marcel Dekker.
22. Davis, F. D. (1989). Perceived usefulness, perceived ease of use and user acceptance of
information technology. MIS Quarterly, 13, 319330;
DeLone, W., & Mclean, E. (1992). Information systems success: The quest for the
dependent variable. Information Systems Research, 3(1), 6095;
Ibid., p. 4, Caffrey.
23. Ibid., p. 19;
Ibid., p. 12, Chengalur-Smith and Duchessi;
Brown, M. M. (2001). The benefits and costs of information technology innovations: An
empirical assessment of a local government agency. Public Performance and Management Review, 24(4), 351366;
Ibid., p. 2, Dawes and Pardo.
24. Burbridge, L. (2002). Accountability and MIS. Public Performance and Management
Review, 25(4), 421423.
25. Barki, H., Rivard, S., & Talbot, J. (1993). Toward an assessment of software development
risk. Journal of Management Information Systems, 10, 203223;
Dawes, S. S., & Nelson, M. R. (1995). Pool the risks, share the benefits: Partnerships in
IT innovation. In J. Keyes (Ed.), Technology Trendlines. Technology Success Stories from
Todays Visionaries. New York7 Van Nostrand Reinhold;
Ibid., p. 4, Caffrey;
Ibid., p. 12, Chengalur-Smith and Duchessi;
Ibid., p. 4, Garson.
26. Ibid., p. 4, Caffrey;
Ibid., p. 2, Dawes and Pardo.
27. Ibid., p. 23, Brown.
28. Randall, D., Hughes, J., OBrien, J., Rodden, T., Rouncefield, M., Sommerville, I., et al.
(1999). Banking on the old technology: Understanding the organizational context of
dlegacyT issues. Communications of the Association of Information Systems, 2;
Kelly, S., Gibson, N., Holland, C. P., & Light, B. (1999). A business perspective of
legacy information systems. Communications of the Association of Information
Systems, 2.

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

213

29. Duchessi, P., & Chengalur-Smith, I. (1998). Client/server benefits, problems, best
practices. Communications of the ACM, 41(5), 8794.
30. McFarlan, F. W. (1981). Portfolio approach to information systems. Harvard Business
Review, 59, 142150;
Davis, G. B. (1982). Strategies for information requirements determination. IBM System
Journal, 21, 430;
Ibid., p. 25, Barki et al.;
Ibid., p. 2, Dawes and Pardo.
31. Ibid., p. 25, Dawes and Nelson.
32. Ibid., p. 2, Dawes and Pardo.
33. Ibid., p. 25, Barki et al.;
Ibid., p. 25, Dawes and Nelson;
Ibid., p. 19;
Ibid., p. 2, Best;
Ibid., p. 4, Caffrey;
Ibid., p. 3, Bellamy;
Jiang, J., & Klein, G. (2000). Software development risks to project effectiveness.
Journal of Systems and Software, 52, 310;
Barret, K., & Green, R. (2001). Powering up. How public managers can take control of
information technology. Washington, DC7 CQ Press;
Ibid., p. 24;
Edmiston, K. D. (2003). State and local e-government: Prospects and challenges.
American Review of Public Administration, 33(1), 2045;
Ibid., p. 3, Rocheleau.
34. Ibid., p. 25, Dawes and Nelson;
NGA. (1997). Barriers to intergovernmental enterprise. Washington, DC7 National
Governors Association;
Ibid., p. 2, Landsbergen and Wolken;
Harris, N. D. (2000). Intergovernmental cooperation in the development and use of
information systems. In G. D. Garson (Ed.), Handbook of public information systems.
New York7 Marcel Dekker.
35. Ibid., p. 25, Dawes and Nelson;
Fountain, J. E. (2001). Building the virtual state. Information technology and institutional
change. Washington, DC7 Brookings Institution Press;
Ibid., p. 2, Dawes and Pardo.
36. Ibid., p. 3, Bellamy;
Ibid., p. 34, Harris;
Ibid., p. 3, Rocheleau.
37. Bajjaly, S. T. (1999). Managing emerging information systems in the public sector. Public
Performance and Management Review, 23(1), 4047;
Brown, M. M., & Brudney, J. L. (2003). Learning organizations in the public sector? A
study of police agencies employing information and technology to advance knowledge.
Public Administration Review, 63(1), 3043.

214

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

38. Scott, W. Richard (2000). Institutions and organizations. Thousand Oaks, CA7 Sage
Publications.
39. Andersen, D. F., & Dawes, S. S. (1991). Government information management. A primer
and casebook. Englewood Cliffs, NJ7 Prentice Hall;
Ibid., p. 4, Caffrey;
Ibid., p. 3, Milner;
Joshi, J. B. D., Ghafor, A., Aref, W. G., & Spafford, E. H. (2002). Security and privacy
challenges of a digital government. In W. J. McIver, & A. K. Elmagarmid (Eds.),
Advances in digital government. Technology, human factors, and policy. Norwell, MA7
Kluwer Academic Publishers;
Duncan, G. T., & Roehrig, S. T. (2003). Mediating the tension between information
privacy and information access: The role of digital government. In G. D. Garson (Ed.),
Public information technology: Policy and management issues. Hershey, PA7 Idea Group
Publishing;
Ibid., p. 33, Edmiston.
40. Ibid., p. 19;
Ibid., p. 4, Caffrey;
Ibid., p. 35, Fountain;
Ibid., p. 2, Dawes and Pardo.
41. Ibid., p. 3, Bellamy;
Ibid., p. 3, Rocheleau.
42. Wang, R. Y. (1998). A product perspective on total data quality management.
Communications of the ACM, 41(2), 5863.
43. Keil, M. (1995). Pulling the plug: Software project management and the problem of
escalation. MIS Quarterly, 19(4), 421447;
Ibid., p. 42;
Ibid., p. 21.
44. CTG. (2000). The insiders guide to using information in government. Albany, NY7
Center for Technology in Government http://demo.ctg.albany.edu/static/usinginfo
45. Orr, K. (1998). Data quality and systems theory. Communications of the ACM, 41(2),
6670.
46. Ibid., p. 44.
47. Ibid., p. 22, Davis;
Ibid., p. 22, DeLone and Mclean;
Ibid., p. 21;
Ibid., p. 4, Garson.
48. Ibid., p. 4, Caffrey;
Ibid., p. 2, Dawes and Pardo.
49. Ibid., p. 25, Barki et al.;
Jiang, J., Klein, G., & Balloun, J. (1996). Ranking of system implementation success
factors. Project Management Journal, 27, 5055;
Ibid., p. 21;
Ibid., p. 33, Jiang, J. and Klein.

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

215

50. Ibid., p. 23, Brown.


51. Ibid., p. 2, Best;
Ibid., p. 21;
Ibid., p. 2, Dawes and Pardo;
Ibid., p. 4, Garson.
52. Ibid., p. 33, Barret and Green;
Regan, E. A., & OConnor, B. N. (2001). End-user information systems: Implementing
individual and group work technology. Second Edition. New York7 Prentice Hall;
Ibid., p. 4, Garson.
53. Flowers, S. (1996). Software failure: Management failure: Amazing stories and
cautionary tales. New York7 Wiley;
Ibid., p. 37, Bajjaly;
Ibid., p. 4, Garson.
54. Ibid., p. 4, Caffrey;
Ibid., p., 33, Jiang, J. and Klein;
Ibid., p. 2, Dawes and Pardo.
55. Ibid., p. 25, Dawes and Nelson;
Ibid., p. 2, Best;
Ibid., p. 34, NGA.
56. Ibid., p. 21;
Ibid., p. 33, Barret and Green;
Ibid., p. 4, Garson.
57. Ibid., p. 2, Dawes and Pardo;
Gagnon, Y. -C. (2001). The behavior of public managers in adopting new technologies.
Pubic Performance and Management Review, 24(4), 337350;
Ibid., p. 52, Regan and OConnor;
Ibid., p. 3, Rocheleau.
58. Ibid., p. 34, NGA;
Ibid., p. 34, Harris;
Ibid., p. 33, Barret and Green.
59. Ibid., p. 33, Barret and Green;
Ibid., p. 25, Dawes and Nelson;
Ibid., p. 4, Garson;
Ibid., p. 3, Milner.
60. Ibid., p. 39, Andersen and Dawes.
61. Gil-Garca, J. Ramon (2004). Information technology policies and standards: A
comparative review of the states. Journal of Government Information, 30(5), 548560.
62. North, D. C. (1991). Institutions, institutional change, and economic performance. New
York7 Cambridge University Press;
Ibid., p. 35, Fountain.
63. Ibid., p. 25, Barki et al.;
Ibid., p. 49, Jiang et al.;
Ibid., p. 21;

216

J.R. Gil-Garca, T.A. Pardo / Government Information Quarterly 22 (2005) 187216

Ibid., p. 23, Brown;


Ibid., p. 4, Garson.
64. Brown, M. M., & Brudney, J. L. (1998). A bsmarter, cheaper, and fasterQ government?.
Contracting for geographic information systems. Public Administration Review, 58,
335345;
Ibid., p. 33, Barret and Green;
Chen, Y. -C., & Perry, J. (2003). Outsourcing for e-government: Managing for success.
Public Performance and Management Review, 26(4), 404421.
65. CIO Council. (2002). The value measuring methodology: Highlights. Washington, DC7
CIO Council, Best Practices Committee;
CIO Council. (2002). The value measuring methodology: How-to-guide. Washington,
DC7 CIO Council, Best Practices Committee.
66. Ibid., p. 65.
67. Ibid., p. 65.
68. Ibid., p. 65.
69. Treasury Board of Canada. (1998). Creating and using a business case for information
technology projects. Ottawa, ON7 Treasury Board of Canada, Secretariat.
70. Ibid., p. 69.
71. Ibid., p. 69.
72. Ibid., p. 69.
73. Ibid., p. 69.
74. Ibid., p. 69.
75. NASCIO. (2003). Business case basics and beyond: A primer on state government IT
business cases. Lexington, KY7 National Association of State Chief Information Officers.
76. Ibid., p. 75.
77. Ibid., p. 75.
78. Dawes, S. S., Pardo, T. A., Simon, S., Cresswell, A. M., LaVigne, M., Andersen, D., &
Bloniartz, P. A. (2004). Making smart IT choices: Understanding value and risk in
government IT investments. Albany, NY7 Center for Technology in Government.
79. Ibid., p. 78.
80. Ibid., p. 78.
81. Ibid., p. 75.
82. Ibid., p. 44.
83. Ibid., p. 69.
84. The Directorate for Computer and Information Science and Engineering at the National
Science Foundation has created a bSystems in ContextQ cluster. This cluster bsupports
research and education on the interaction between information, computation and
communication systems and users, organizations, government agencies, the scientific
community and the external environment.Q More information is available at http://
www.cise.nsf.gov/div/cluster.cfm?div=iis&cluster_id=3947