RETHINKING E-PROCUREMENT IN THE PUBLIC SECTOR THROUGH AGILE AND

LEAN THINKING
Francesco Gardenal, MSc, Economics for Public Administration and International
Institutions, has been working since 2007 as an e-procurement specialist and
consultant for the Italian public sector. His research interests are in public
procurement, change and performance management, public policy.
With the contribution of Federico De Marco, MSc.
ABSTRACT
e-Procurement has been defined as the sourcing of goods or services via electronic
means (Schoenherr, 2007) but scholars have proposed several more elaborated
phrasings, also to incorporate in the definition the significant organizational benefits
that could derive from carrying out procurement activities over the internet (CIPS,
2013).
During the last 15 years e-procurement implementation projects in the Public sector
have spread worldwide. However, findings from the literature show that e-procurement
adoption has probably not yet generated the positive impacts that were highly
anticipated (McCue and Roman 2012).
Starting with an analysis of the difficulties that e-procurement encounter in the public
sector, this paper suggests two organizational changes that may facilitate the
effective adoption of this innovation.
The proposed changes have been widely adopted in business contexts, but are rather
new in the public sector:
1. using Agile approaches and methodologies to
developments and project management activities;

manage

new

software

2. employing Lean thinking to reduce the waste in the administrative processes

of the procurement cycle.

1.1.

INTRODUCTION

Public e-procurement is one of those innovations made possible by the advancements

in the field of information technology (IT) whose goal is to digitalize the processes of
the public administration, while improving and simplifying the delivery of public
services to the community (e-government).
In order to understand the main characteristics and complexities linked to the
implementation and management of e-procurement solutions, it is convenient to
consider first the more general context of IT projects in the public sector.
This kind of projects have generally gathered a bad reputation among the public
opinion and accurate reports have been dedicated over the years to the ever
mounting number of inititiatives that didnt produce the expected results, wasting
significative amounts of public money (such as (Parliamentary Office of Science and
Technology, UK, 2003).
Indeed, the effective management of IT projects in the public sector implies several
complexities. The following Table 11 contains a synthesis of the specific difficulties
faced by the public sector, which can affect the political, organizational or technical
level of the project management activities.
Complexities linked to IT projects in the public sector
Political
level

Insufficient knowledge of the potential of IT solutions

Frequent reconsideration of priorities due to the high dynamicity of
politics
Projects not supported by valid business cases

Organizatio
nal level

Number and heterogeneity of the users

High resistance to change due to procedures rooted in the
organizational culture
Integration with IT systems already in use

Technical
level

Fast obsolescence of IT solutions and competencies

Information asymmetry between the Public Authority and IT partners
Table 1 Complexities linked to IT projects in the public sector

Another way to shed light over the potential complexities that IT projects could meet
in the public sector is to compare their general characteristics with those of similar
projects executed in the private sector. The following Table 22 highlights why the public
administration certainly represents a more challenging environment for the
implementation of such innovations.
IT project characteristics

Private sector

Public sector

Goals

Financiary and of service,

measurable

Heterogeneous,
hardly measurable

Competition with similar

projects
Stakeholders visibility

High

Low

Low

High

Law obligations

Low

Very high

Elaboration of the analysis presented in a report commissioned by the Dutch government: Why Government ICT Projects Run Into
Problems (Leydesdorff & Wijsman, 2008)
2

Elaboration of the analysis presented in a report commissioned by the British government: Getting IT Right for Government (Intellect Computing Services and Software Agency, UK, 2000)

Organizational culture

Risk-taker

Risk-adverse

Flexibility

High, to limit financial

risks
Single user

Low, due to high

complexity
Many different users

Number of users

Table 2 IT project characteristics (comparison between private and public sector)

Notwithstanding this complexities, a significant number of nations have spent more on
IT over the last decades than on traditional capital investments such as roadways,
airports, water purification plants and the like (World Bank, 2004) (Amemba, Nyaboke,
Osoro, & Mburu, 2015).
Indeed, during the slow recovery from the 2008 economic crisis, it is still very audible
in the political discourse the belief that trough the digitalization of processes the public
sector may increase its operational efficiency, promote innovation and foster growth.
One area within government that has seen significant reform efforts, as well as
notable investments in IT, is procurement. Those attempting to reform the
procurement process argue that the current system is broken, and unsuitable for the
demands of modern governance (Thai, 2001) (McCue & Roman, 2012).
The potential benefits that e-procurement may generate for public entities have been
largely acknowledged by the literature (Ronchi, Brun, & al., 2010) (Ilhan & Rahim,
2011), they consist of improvements in organizational performance, regarding the
dimensions of efficiency, effectiveness, transparency, competitiveness and
dematerialization (Gardenal, 2013).
However, achieving the substantial organizational and policy changes that eprocurement adoption requires is even more difficult than considering most IT projects
carried out in the public sector, given the high complexity of public procurement
processes and the ever-changing normative framework (Leukel & Maniatopoulos,
2005) (Henriksen & Mahnke, 2005).
Indeed, it regularly happens that expectations regarding the benefits of e-procurement
remain unfulfilled, mainly because users tend to incorporate new technologies into
existing procedures and operating principles, thus reaffirming the status quo (Mota &
Filho, 2011). Several evidences seem to attest that this technological and
organizational innovation has not produced yet the important results anticipated, often
causing the implementation project to be unsuccessful. The following Table 3 lists the
most relevant failure causes.
Failure causes of e-procurement in the public sector
Local and fragmentary implementation

(Hardy & Williams, 2008)

Procurement professionals seen as passive users

(Mota & Filho, 2011)

E-procurement not perceived as an institutional

expected characteristic or strategic asset

Difficult integration with other systems

(Chang & Wong, 2010)

(Bof & Previtali, 2007)
(Varney, 2011)
(Vaidya, Callender,
Sajeev, & Gao, 2004)
(Gardenal, 2013)
(McCue & Roman, 2012)
(Bulut & Yen, 2013)
(McCue & Roman, 2012)

Dissatisfaction of the users

(McCue & Roman, 2012)

Pricey and technically inadequate

(McCue & Roman, 2012)

Inadequate performance measurement systems

Use limited to basic administrative needs

Table 3 Failure causes of e-procurement in the public sector

Considering the complexities herewith outlined, this research contribution analyzes the
advantages that may derive from the introduction of two organizational changes in the
way the public sector manages e-procurement. The authors suggest adopting agile
methodologies for project management activities and a lean-thinking inspired
approach to guide the public administration in the implementation journey. Such
changes aim to facilitate the full disclosure of the aforementioned organizational
benefits, and to limit the failure risk of the projects.
Several case studies describing successful experiences related to both agile
methodologies and lean-thinking approaches exist in the private sector, but research
efforts that focus on the public administration are still limited. It is the authors belief
that these organizational modifications will generate a strong positive impact in the
public administration, wishing that it would soon be possible to measure empirically
the outcomes.
1.2.

ADOPTING AGILE
ACTIVITIES

METHODOLOGIES

1.2.1.

The Agile philosophy

FOR

PROJECT

MANAGEMENT

In the software development jargon, the word Agile (or lightweight) identifies a set of
operational methodologies and organizational approaches, which originated in the
1990s. The Manifesto for Agile Software Development (Beck, Martin, Fowler, & al.,
2001) states the basic principles of these methods. Thus, Agile is an umbrella term
that comprises several different software development approaches. Nevertheless,
more than a set of different methodologies and techniques, Agile represents a strong
cultural stance about the creative processes related to IT solutions (Moran, 2014).
Traditional approaches to software development originated in the second postwar
period in the defense sector. Apparently, their main advantage is simplicity; indeed,
the developers task is only to translate in coding language the detailed functional
requirements, which are produced by the customer 3. On the other hand, these
methods do not let the final user test the product until the creative process is
completely over. Consequently, the risk that the developed software does not meet
the customers expectations is remarkably high, as is the cost of managing
modifications to the initial requirements (Munassar & Govardhan, 2010).
Moving to Agile requires a radical cultural change: the functional requirements have to
emerge during the project (emergent design), and they are refined progressively,
as the customer begins to test the first developed elements of the software (Gero &
Kannengiesser, 2007). Indeed, in an Agile project, the functionalities under
development are discussed and validated frequently, letting the users try out small
pieces of the application, as soon as the development process is over (continuous
delivery). It is therefore an iterative approach, in which the team starts from the
discussion of the underlying objectives of the project, and discovers how to realize
them during the project itself. The generally recognized advantages of this approach
include (Koch, 2011):

increased ability to meet the customers needs;

more flexibility in adapting functional requirements during the project;

more collaboration between the stakeholders in every project phase;

Agile is in stark contrast with traditional software development methodologies, in which functional requirements are consolidated
completely before that actual development may begin (McCauley, 2001). The most famous traditional methodology is called waterfall:
software development projects are compared to a waterfall, which originates on top of the mountain (with the functional requirements
definition), then continuously falls until the project completion, which normally happens a long time later

increased motivation of the developers team (because it actively participates in

the design process);

increased quality of the final product;

increased ability of respecting time & budget boundaries.

However, a single paradigm cannot successfully apply to any kind of software

development initiative, that is to say that project managers need to comprehend the
specific nature of the project and select the proper development techniques
accordingly (Hawrysh & Ruprecht, 2000). Consequently, there is a need for both
traditional and agile approaches and combinations of the two (Glass, 2001).
While Agile methodologies are only marginally used in the public sector, they can be
regarded as the de-facto standard in projects managed by IT companies, from Silicon
Valley giants (Peter, 2011) to little local software houses (Bustard, Wilkie, & Greer,
2013). In 2011, a state of agile development survey showed that more than 80% of
respondents companies were using agile methods to some extent (Rodrguez,
Markkula, Markku, & Turula, 2012). Thus, it makes sense to explore the benefits that
may derive from a more extensive use of these approaches also in the public sector.
1.2.2. Agile in the public sector
Using Agile methodologies
administration:

is

not

something

completely

new

in

the

public

the British government envisioned in 2011 the adoption of Agile methods to

manage at least 50% of its ICT-related projects, over the course of the following
three years;

the Danish government produced a comprehensive guide for the adoption of

Agile methodologies for software development in the public sector (National IT
and Telecom Agency, 2010);

the Victoria and Demos state government (Australia) published practical

recommendations for the transition to the Agile government, in which Agile
approaches are applied to the general administrative and political activity of
public institutions;

the Brazilian government promoted the study of new project management

techniques based on Agile methodologies. These techniques are to be used in
public IT-projects in order to improve their outcome, given the numerous failures
experienced in the previous years (Frota, Andrade, Filho, & Albuquerque);

the Norwegian government initiated a work-group to produce a new set of

standard contracts to be used by the public sector to regulate the use of Agile
techniques (Systemutviklingsavtalen, SSA-S).

Nevertheless, skepticism regarding Agile in the Public sector is still widespread. The
underlying motivation regards the bureaucratic and hierarchical nature of the public
sector, which is seen to be used to command-and-control type of dynamics. These
dynamics stand poles apart from the Agile philosophy, which is based on collaboration
and trust between stakeholders. In fact, Agile methods require activities to be
managed by self-organizing teams of independent professionals, whose specific
competencies are equally valued (State Government of Victoria and Demos, 2008).
However, the majority of interested researchers and consultants sees Agile
methodologies as a genuine opportunity for the Public Sector, in order to fully exploit
its talents, and reach new levels of effectiveness and efficiency in managing IT

projects (Akingbe, 2013). Thus, explicit criticism 4 towards using Agile methods in the
Public Sector seems to have little or no foundation 5.
1.2.3. Agile e-procurement
Specific case studies regarding the implementation of e-procurement in the public
sector using Agile methodologies have not been explored thoroughly by researchers
yet. Therefore, this paper discusses how to introduce these methodologies,
highlighting the organizational benefits that may accrue for the Public organization.
This discussion relies on the conceptual framework proposed by the Danish National IT
and Telecom Agency (National IT and Telecom Agency, 2010).
When evaluating how to build and manage a system to manage public procurement
online, three fundamental variables need to be taken into account, each of which may
be positively impacted by choosing Agile methods: required functionalities, time and
budget constraints.
1.2.3.1.

Required functionalities

Public organizations generally need to publish an open tendering procedure in order to

award in the market the development and management services for an e-procurement
solution. Generally, and in alignment with the traditional waterfall approach, the
contracting authority exhaustively list in the needs statement every specific
functionality that wants to be included in the platform. Functional characteristics,
economic values, delivery deadlines, sanctions for delays, and so on, need to be
detailed in the document.
This approach has the advantage of allowing the contracting authority to compare
immediately several alternative solutions (considering how many required
functionalities are available in each proposal). However, on the other hand, waterfall
limits the e-procurement project scope to the requirements listed explicitly in the
needs statement. Therefore, with this type of approach, it is very difficult and costly to
manage new requirements and modifications to existing requirements. Such new
requirements will almost certainly emerge during the project lifetime, considering the
high variability that characterize public procurement regulations, the advancements in
e-procurement technologies and other needs that become evident only after
experiencing with the new system.
Managing a fixed project scope and emergent needs is even more complicated when
the contracting authority does not detail the requirements sufficiently. Especially when
specific e-procurement expertise is unavailable, the contracting authority may even
resort to include in the needs statement descriptions of functionalities that refers to a
specific platform offered in the market (thus, limiting the competitiveness of the
tendering procedure).
Another potential limitation of the waterfall approach regards opportunistic behavior
by the economic operators. Given that functional coverage is usually critical to
determine the award of the tendering procedure, economic operators may be tempted
to include in their offers functionalities that are partly unavailable or even missing
completely from their actual systems. Indeed, e-procurement projects are generally
long and profitable and even to incur in penalties is sometimes a viable strategy. This
4

The main critic toward Agile in the Public sector refers to responsibility assignments within the project team, which seem unclear in
Agile, as is the definition of the project scope. This is considered to be hindering the possibility of controlling the project, and appears to
be contrasting even with normative principles that require Public Authorities to be non-discriminatory and to promote transparency
(Ballard, 2011).
5

Agile projects consider operating developed software as the only metric to measure project advancements. Nevertheless, according
to Agile principles, if the quality of the project documentation is high, then the value of the software increases (Beck, Martin, Fowler, &
al., 2001). Therefore, Agile projects may be actually very well documented, which allows to compare alternative bidders for the same
project. Moreover, whereas Waterfall appears to be more easily controllable, because of its complex procedures, in reality Agile is
more transparent (because it is simpler) and allows to verify the actual project status in a more accurate way (Lennon, 2013)

may lead to the projects failure or to the deterioration of the strategic relationship
that should be established between the public authority and its e-procurement partner.
On the contrary, by definition in Agile projects there is no clear statement of which will
be the output of the project itself. This happens because required functionalities and
their specific characteristics are explored during the project in a collaborative fashion,
during the frequent workshops that are organized by the public authority and its eprocurement partner.
Consequently, the needs statement should limit to the definition of the core processes
that the e-procurement solution should manage (i.e. types of tendering procedures
available in public procurement regulations; vendor management; Management of
the source-to-pay process, and so on). The contracting authority should not state
technical specifications beforehand, except if the specification is of paramount
importance for the project success (i.e. integration with an existing system).
Thus, in an Agile context, it is more important to formalize the goals of the project
rather than specific functionalities. This open definition of Agile projects should imply
that tendering procedures for such initiatives are more competitive than in the case of
waterfall.
Technical bids for such procedures should be evaluated on the basis of different
criteria from functional coverage, such as:

specific development method adopted by the bidder and the extent to which
this method involve the contracting authority users;

project management and control methods, time use estimation methods, and
activity coordination methods;

specific software and organizational solutions that the bidder proposes to the
contracting authority, in order to realize the projects strategic goals (such as:
optimizing public expenditure, reduce the risk of disputes with suppliers;
reduce the lead-time of tendering procedures, etc.);

competencies, expertise and composition of the project team designated by the

bidder;

person-days that the bidder commit to guarantee during the project lifetime for
activities such as functional analysis, software development, maintenance,
consultancy, training and support;

guaranteed service levels, such as resolution time of technical issues or other

types of service requests;

other added-value generating services that the bidder is able to provide, such
as change management and business process re-engineering consultancy,
coaching, and guidance over the interpretation of new regulations;

how the bidder will make use of its prior experience during the process, if the
procurement law allow this criterion to be used, under the specific
circumstances of the tender.

With this approach, it is possible to valorize the specific expertise of each competitor,
thus stimulating the proposal of innovative solutions, carefully tailored to help the
public authority to reach its actual goals, rather than just imposing compliance with a
static list of functionalities.
Moreover, the inherent flexibility associated with Agile projects seems to be another
critical advantage for public authorities. This way, e-procurement platforms will be
more easily adaptable to the constant evolution of the public procurement
environment. As anticipated, flexibility is certainly desirable in order to manage the
numerous factor that are subject to high variability in an e-procurement project:

local, national and supranational regulations regarding public procurement;

new operational guidelines, databases and other fulfillments asked from public
bodies that oversee the public procurement sector;

policy makers priorities (such as reducing the access barriers to the public
procurement market versus valorizing local SMEs);

turnover of technological solutions available in the market;

particular needs that imply the intervention of the IT partner but stands outside
the scope of the contract.

Considering such continuous, significant environmental changes, it is reasonable to

note that e-procurement platforms are not commodities yet (i.e. a type of good that is
normally available in the market, can be immediately used and replaced, without the
need for customization or evolutionary maintenance). On the contrary, it appears
more appropriate to consider e-procurement as a service.
1.2.3.2.

Time available for project completion

According to the traditional waterfall approach, typically employed in the public sector,
the contracting authority seeks to define ex ante the projects time constraints too. On
the contrary, in the Agile approach, the time factor is not usually predetermined in a
rigid way.
Thus, in an Agile e-procurement initiative, its not appropriate to force the ICT partner
to respect rigid time schedules. That is because in an Agile context, software
development activities are planned during the project itself, with continuous
adjustments and re-prioritizations. Potential Agile suppliers cannot include detailed
plans for every development process (or iteration) before starting the actual activities.
Nevertheless, it is certainly possible to ask to formalize estimations related to the time
schedule of the project, which may include the project start and end date, the number
of expected iterations and the completion dates for each iteration.
If the project necessarily needs to complete certain activities within a specific period,
it is possible to inflict penalties to the supplier, whenever it does not respect these
dates. However, the contracting authority should envisage such penalties only for
activities that are determinant to respect cogent normative deadlines; or technical
ones, like integration with pre-existing systems, which the supplier should establish
before actually starting a new e-procurement platform.
Although achieving a binding planning of activities in an Agile project may seem
difficult, in fact the opposite is true. Indeed, using correctly Agile methodologies imply
a continuous planning effort, which includes short, medium and long term goal setting
and frequent reconsiderations of the project road-map. In mature Agile environments,
a detailed planning of each iteration is certainly available, with exact allocation of the
team members person-days. Moreover, it is a transparent type of planning: the Public
Authority always have clear visibility of what its ICT partner is currently doing.
Therefore, it is possible to exert a better control activity, as well as adjustments of the
project scope.
On the contrary, with the waterfall approach, it is necessary to wait for the final
delivery date in order to make sure that the developed software fits the contracting
authoritys needs. Consequently, it is frequently the case that the final product does
not meet at all the initial expectations. In this scenario, although the delivery date
may have been respected, the public authority would achieve an ill-suited system,
which will likely require expensive new integrations. Moreover, it is usually very
difficult to determine with certainty if the supplier is responsible for the
inappropriateness of the system.

In other words, using Agile methodologies requires the Public sector to cope with a
certain degree of variability in delivery dates, but also provides better guarantees
regarding the quality of the IT platform, its fitness with the authoritys needs, and a
positive degree of flexibility in handling emerging specifications.
In order to provide incentives to the schedule performance (timeliness in deploying
programmed activities) in the project activities, it is convenient to employ the classic
EVM (earned value management) technique, adapted for use in the context of eprocurement. For example, the public authority may use a simple measuring
technique, such as the following (Baldwin, 2014).
1. Calculate how many story points6 are planned (Planned Value, PV) for all of the
functionalities included in the next release e-procurement platform, and for
each iteration required to complete the release. In Agile projects, software
development is an incremental activity, which proceeds through several
iterations. When the team completes each iteration, final users can test semifinished functionalities, in order to get early feedback, test cases and ultimate
validation.
2. Measure how many story points have been earned during the process (Earned
Value, EV), i.e. the number of story points related to completed and validated
functionalities.
3. Measure, whenever necessary, the Schedule Performance Index (SPI = EV/PV),
which is useful to assess the pace of the project. If the index value is greater
than zero, the project is proceeding ahead of the planned schedule.
4. As soon as the project earns about 50% of the planned story point of an
iteration, ask the supplier to provide a binding estimation regarding the iteration
final delivery.
5. As soon as any iteration is completed, verify if any gap exists between the
binding estimation and the iteration actual completion date. When the
completion date is successive to the binding estimation date, the Public
Authority could consider blocking incentives whose aim is promoting the
suppliers schedule performance, or inflicting penalties.
However, the ICT partner should be safeguarded whenever the Public Authority re-negotiates the scope of

the iteration (number and characteristics of the required functionalities), when the
development process already begun. In this case, the Planned Value should be
reconsidered accordingly, and the expected completion date as well.
1.2.3.3. Budget available for the project

Among the factors to consider when choosing an Agile methodology instead of

Waterfall, the most rigid is probably the budget available for the project.
The main opportunity that passing to Agile offers to the Public sector relates to the
individuation of innovative remuneration schemas, which should provide sufficient
incentive to keep the ICT partner performing at best. Given that in Agile projects there
is no certainty on the final set of functionalities that will be developed, emphasis
should be placed on how to reach strategic goals.

A story point is a metric used in agile project management and development to determine (or estimate) the difficulty of implementing a
given story. In this context, a story is a particular business need assigned to the software development team. Using estimations of
story points rather than time (person-days) allows development teams to be less precise. It may be difficult, for example, to estimate
how long a particular feature will take to develop but relatively easy to understand if it is more complex than others, in which case it
should be assigned more story points

Consequently, it is possible to tie a part of the contractors remuneration to the value

actually generated for the public authority. Using an appropriate strategic performance
measurement model (SPMS7) it would be possible to track how the project is
contributing to achieve the Public authority targets, in the relevant dimensions
(efficiency, effectiveness, transparency, governance, etc.). Whenever significant
performance increment are measured, the Public Authority may reward the contractor
with economic incentives.
One of the known complexities related to ICT projects in the public sector is
information asymmetry between the Public Authority and its ICT partner. Exception
made for some public authority in which specific competences in technology and
information systems exist, normally the ICT partner in charge of the e-procurement
project has a far superior technical knowledge of the specific domain.
In a waterfall project, this information asymmetry generates inefficiencies and
uncertainties, which normally disadvantage the Public Authority under several points
of views:

quantifying the number of person/days necessary for a certain activity;

determining the economic value of technological components;

choosing the most appropriate technology to meet the requirements;

identifying the best practices to be inspired from;

evaluating the technical feasibility of certain activities.

Although it is certainly desirable that ICT-specific competencies spread at a faster pace

also in the Public sector, a certain degree of such an information asymmetry will
always exist in e-procurement projects. However, switching to Agile methodologies
can limit the risk that these asymmetries generate inefficiencies.
In a contract explicitly studied to regulate an Agile project, the Public Sector focuses
on strategic goals and things to do are decided collaboratively with the ICT partner,
thus exploiting the competencies of both sides of the relationship. Indeed, if
remuneration is linked to achieving the Public Authoritys goals, the contractor has a
strong incentive to propose innovative solutions and exploiting its expertise at the
most.
This argument also supports the idea that e-procurement platforms are not a
commodity yet, but as an added-value services supply activity, which takes place in
the context of a strategic partnership between the Public Authority and the contractor.
Therefore, the most appropriate regulatory approach for such a partnership appears to
be performance-based contracting (PBC 8). Adopting such an approach could
potentially generate additional important results related to expenditure control and
reduction (World Health Organization, 2007) (World Bank, 2006):

better contract outcomes;

reduction of total cost of ownership (TCO);

better ability to foresee emerging costs during the contracts lifetime;

better accountability;

better comprehension of performance drivers;

A strategic performance measurement system (SPMS) is a set of causally linked nonfinancial and financial objectives, performance
measures, and goals designed to align managers' actions with an organization's strategy (Webb, 2004). A SPMS specifically studied for
public e-procurement was proposed by (Gardenal, 2013)
8

The performance-based contract (PBC) is a type of contract particularly suited for services, whose characteristics are to allow for the
contractor remuneration only when services are delivered, and to link a part of the compensation to the actual performance (Hughes &
Shabnam, 2013) (Wang & Jin, 2012)

promotion of strategic benefits for contracting authorities and contractors;

integration of all the contractual aspects in a single set of performance

indicators;

reduction of information asymmetry.

1.3.

USING A LEAN THINKING APPROACH TO PROCESS RE-ENGINEERING

1.3.1.

The Lean philosophy

The terms Lean production and Lean thinking define an approach for operations
management whose ultimate goal is reducing waste and increasing the efficiency of
the production (Womack, J.; Jones, D.; Ross, D., 1990). This model was applied firstly in
the Japanese factories of Toyota, in the early 1970s, where Taiichi Ohno introduced the
Toyota production system (Ohno, 1978, 1998). The systematic attack on waste, or
efficiency losses, is the cornerstone of this organizational philosophy.
Lean thinking relies on five main principles listed in the following Table 4 (Womack &
Jones, 1996):
Lean thinking principles

Definition

Identify customers and specify

value

Only a small fraction of the total time and effort in

any organization actually adds value for the end
customer
The Value stream (flow) represents the end-toend process that delivers the value to the
customer
The product or service needs to flow to the
customer without any interruption, detour or
waiting
An organization should produce only what the
customer wants when the customer wants it
Creating flow and pull starts with radically
reorganizing individual process steps, but the
gains become significant as all the steps link
together

Identify and map the value

stream
Create flow by eliminating
waste
Respond to customer pull
Pursue perfection

Table 4 Lean thinking principles

Thus, the Lean thinking principles imply the following key concepts.

Customer: represents the final user of the goods/services produced by an

operational process, other internal organizational units are considered
customers too.

Value: represents something that the customer is willing to pay for, or that has
an utility.

Waste, or efficiency loss (in Japanese, muda): represents what the customer is
not willing to pay for, because unnecessary. Waste usually depends on
organizational inefficiencies.

Flow: represents a series of interconnected operational processes, the term

stresses the importance of letting activities follow each other in a natural,
harmonious way.

Variability (in Japanese, mura): processes are subject to variability, that causes
wastes. It is thus necessary to stabilize the cycle-time by managing one process
at-a-time, in a standardized way.

Lean thinking originates from a simple observation, that the most of the activities in a
process intrinsically generate wastes, might they be losses of time, energy, resources.
Consequently, it is necessary to eliminate waste (muda) and keep only what is
essential, the value. The following Table 5 lists the seven muda categories individuated
by Ohno.
Types of waste
(muda)
Overproduction

Definition
To produce more (or earlier) than the market demand

Waiting

Goods or services are not being processed

Transporting

Excessive movement and handling can cause damage

Over Processing

Excess of manufacturing compared to the customers need

Unnecessary Motion

This waste is related to excess of motion by operators

Defects

Quality defects resulting in rework represent a cost

Unnecessary
Inventory

Excess inventory tends to hide problems

Table 5 Types of waste (muda)

1.3.2. Lean Thinking in the public sector

According to several observers, in the last decades the Public sector is transforming
significantly, and, after years of wild bureaucratization, to generate value for the
citizen is considered the main strategic priority (Perrella & Delli Rocili, 2009). In order
to maximize value generation, it is necessary to employ new organizational models,
rethink the public sector functions, attracting new competencies able to face the new
challenges that await the administrative machine (Hood, 1991). Lean thinking plays a
role in this organizational renewal process.
The managerial model called Lean government (Negro & Ozzello, 2010) is based on
the five principles by Ohno (see Table 5), plus a sixth one: Integrate Authorities in the
Public sector. The authors wanted to highlight how the Public Sector represents a
complex heterogeneous system, in which it is necessary to attack also the waste
that generates in the network relationship.
Public Authorities that decide to adopt a Lean approach need to start a complex
renewal process of the organizational culture, employing both hard operating
principles, such as just-in-time service delivery, and soft ones, like the creation of new
professional roles tasked to support the teams in the continuous effort of
improvement.
The impacts of such renewal initiatives can be strong and long lasting. The goal is to
activate a virtuous circle that would allow increasing: the pace and efficiency of
service delivery, the citizens satisfaction, the employees engagement in the change
process and the management availability for training and formation of the operational
teams (Biniam, Petter, Mark, & Becca, 2012).
Therefore, Lean thinking had been already applied in several ways in the Public sector,
leading to successes and failures, particularly in the healthcare sector, for example in
US, UK, Sweden and Italy, see for instance (Radnor & Walley, 2008).
1.3.3. Lean e-procurement

Transparency, efficiency and timeliness in the delivery of public goods and services
have a great importance in the public opinion. In order to increase the satisfaction of
the citizens, the optimization of public procurement is particularly relevant 9 (Krn,
2004). Also considering the spending review policies, which have been promoted to
contain the economic crisis, the strategic importance of the procurement function
have been constantly rising (Radnor & Walley, 2008).
The main difficulties that procurement officers in the public sector had been
individuated in the Italian public administration, as follows (Nicoletti, 2010):

to get better results with less available resources;

to manage budget constraints on the general costs;

to limit the high error risk;

to get along with inefficient processes.

The main causes of these difficulties lie in the context of public procurement itself,
where problems that exist elsewhere in the public sector too, assume critical
relevance:

excessive bureaucratization and complicate approval process flows;

information asymmetry between the public sector and economic operators in

the market;

lengthiness derived from using complex procedures;

inefficiencies due to the transmission of information in a traditional way

(paper based).

Implementing an e-procurement solution certainly has the potential to reduce these

problems, and to help fighting the underlying causes. Adopting a Lean thinking
approach in this process means actively concentrating organizational energies in
critical areas in which inefficiencies and waste may settle. Particularly, it is necessary
to systematically attack muda and mura, thorugh these activities (Nicoletti, 2010):

re-engineering of the supply chain processes;

optimization of the interoperability between Public Authorities in the eprocurement project;

collaboration with economic operators (i.e. Supply chain collaboration);

simplification and transparency increase.

Thus, successfully employing a Lean approach in e-procurement implementation

activities should realize the following benefits for the Public sector:

Reduced variability of the procurement process and better governance;

Reduced length of procurement processes and the effort required to manage

them; increased procurement quality;

Reduced consumption of material resources during the procurement processes.

The following chapters focus on exploring these benefits, trying to point out how to
achieve them.
1.3.3.1.
Reduction of the variability of procurement
processes and better governance
9

Public procurement is worth on average 19% of the EU GDP. An analysis conducted in Italy (Bandiera, Prat, & Valletti, 2009)
concluded that if the less virtuous Public Authorities would get the same awarding prices that are paid by the 10% of the best
performers, the savings value would range from 1.6 to 2.1 % of the GDP (about 35 bln)

As anticipated, one of the Lean thinking principles requires the configuration of a

unique uninterrupted flow, from raw materials to the final product, with the goal of
eliminating mura (variability), and consequently the muda (waste) that the
contemporaneous existence of several active processes may create.
In this approach, which is known also as one-piece flow, different phases and outputs
of the process need to be close to each other and directly linked together (Marton &
Paulova, 2011). Thus, the time required for moving to one phase of the process to
another needs to be minimum or null. There should be no downtimes in the flow,
waiting times or queues, the product or service under processing needs to move
immediately to the next process phase, one piece at a time.
The adoption of an e-procurement solution may help contracting authorities in unifying
the flow underlying the whole procurement cycle. Indeed, every tool that is required to
manage the different phases may reside in a single platform, thus eliminating the
downtimes that naturally emerge when activities take place in different physical and
virtual environments.
The main phases of the procurement flow which may be integrated in a single
platform are the following:

planning, scheduling and control of procurement;

gathering and analysis of procurement needs (demand management);

managing economic operators registration, qualification and evaluation (vendor

management);

publishing and awarding tendering procedures (e-sourcing, e-tendering, eauctions);

evaluating and comparing received bids (e-catalogs, e-evaluation);

managing formal communications, committee minutes, reports and notifications

online;

digital archiving and substitutive retention of data and files;

signing and managing online the contracts (e-contracting, performance based

contracting);

managing orders and delivery receipts online (e-marketplace, e-ordering);

managing invoices and payments online (e-invoicing, e-payment, supply chain

collaboration);

integrating relevant data from/to observatories and databases;

elaborating data to produce analysis and dashboards (business intelligence,

performance measurement systems);

monitoring consumption in real time (warehouse and inventory management).

Moreover, the efficiency benefits deriving from the definition of a continuous flow in a
single environment are amplified when significant numbers of public authorities share
a single e-procurement solution. In this case, it is certainly possible to realize
important synergies, because several users belonging to different organizations may
now share the same working environment and even work on the same piece
(bundled tendering procedures, such as framework agreements, market analysis or
technical criteria definition).
1.3.3.2.
effort required

Reduction of the process length and of the

Core processes underlying traditional procurement are usually subject to high

variability. Indeed, considering the Italian public sector, even within the same
Contracting authority, different administrative procedures may coexist; depending on
which office is in charge of it (consider for instance offices dealing with small, ordinary
purchasing and those managing the awarding of public works). Moreover, even if the
normative framework is the same, it is arduous to find two contracting authorities that
operate in the same way. Customs and practices, as well as documents and forms, are
usually different to a certain extent.
Unfortunately, the first e-procurement experiences usually consisted in the mere
replica of traditional processes in a digital environment. This replica-approach gives a
null contribution in resolving the already mentioned difficulties that procurement
officers face today. Moreover, this way of adopting an ICT solution may generate even
more muda: for instance printing a received file represents an activity (time and
effort) that was unnecessary in traditional procurement. Consequently, from the
perspective of the Lean thinker, the replica-approach may even deteriorate the initial
situation in which e-procurement did not exist.
On the contrary, management theories inspired by Lean thinking support the
importance of adopting Standard Operating Procedures. Indeed, making use of
standards allows avoiding the risks of using different solution to manage the same
process. If codified, shared procedures are in place, every stakeholder knows which
operation to expect at any given time, reducing the possibility of human errors and
that of subsequent disputes with economic operators.
As in public procurement in general, virtuous and sub-optimal ways to exploit an eprocurement system clearly exist. Investing in the establishment of excellent
operational standards represents the best approach to tackle the re-engineering
process, which is needed to bring the less virtuous contracting authority at the level of
the best performers.
Therefore, this standardization effort may operate at different levels, eliminating
different types of waste, linked to the high variability of the way in which contracting
authorities manage:

tendering documentation;

commodity categorizations;

criteria for qualification in the vendor lists;

bids structure (segregation in different envelopes, content organization);

technical criteria for the evaluation of the bids 10;

algorithms to calculate the score of the economic bids;

reserve prices11.

Such a standardization activity may significantly reduce the effort that procurement
officers need to commit to configuring and evaluating the tendering procedure,
allowing a general increase of the procedures quality as well (because standardized
data derive from best practices). Consequently, contracting authorities may dedicate
more organizational energy to added-value generating activities, such as the definition
of the tendering strategy (Oxford college of procurement and supply, 2014).

10

We refer here to the opportunity of standardizing the technical characteristics of goods, services and work, and the way they are
compared and evaluated. For instance, in an e-procurement system, a buyer may access a library of standard technical
characteristics, which may be associated to items included in the tendering procedure. Several companies offer this type of libraries in
the market, but they are usually tailored to suit the needs of particular industries, and not studied specifically for the use in public
procurement
11

Through an adequate database, an e-procurement platform may calculate the average awarding prices for each commodity
categorization, thus suggesting the buyer a possible reserve price to be used

1.3.3.3. Reducing the consumption of material resources in the procurement

cycle
Any traditional public procurement procedure, even the simpler, implies a significant
consumption of paper, energy, other material resources and costs, such as those for
shipping envelopes or archiving documents when the procedure is over. After
switching to e-procurement, all of these costs are direct examples of muda. Using a
Lean thinking perspective may help to focus also on these consumptions, reducing
costs and de-cluttering the contracting authority workspace.
The contracting authority should especially focus on these activities:

exchanging information and data in a digital way;

using video forms within the e-procurement solution instead of attached files;

empowering and promote the use of content management repositories within

the platform;

allowing substitutive retention of archived documents in digital form;

allowing economic operators to observe and participate in the tendering

procedure evaluation process without physically going to the contracting
authority.

As it happens with other type of e-procurement related benefits, it is not sufficient to

adopt a platform to trigger dematerialization automatically. On the contrary, it is
necessary to individuate the process phases in which consumption happens, then
systematically study which alternative solutions are to be institutionally encouraged to
substitute, for instance, the production of paper based documents with structured data
filled in video forms.
An analysis conducted among contracting authorities in a region in northern Italy
noted that with e-procurement it is possible to reduce the production or printing of A4
paper sheets by more than 60% per tendering procedure and reducing the archiving
costs by 44% (Gardenal, 2013). However, these figures may increase significantly, if
specific organizational policies are in place to encourage dematerialization.
1.4.

CONCLUSION

Effectively implementing Agile methodologies and Lean thinking principles requires

the public sector to innovate its organizational culture in a significant and permanent
way. Moreover, public authority cannot introduce these innovations with a specific
project having this goal. Instead, a continuous and deep renovation of the public
organization modus operandi is necessary, embracing a goal-oriented approach and
focusing on eliminating waste.
The required modifications have to be implemented gradually, through daily change
management challenges. In other words, using the Lean vocabulary, Public authorities
need to adopt a Kaizen approach (from the Japanese change for the better), which
requires a continuous engagement of every level of the organization. Human
resources need to contribute creatively to the change process, and have to appreciate
directly the produced outcomes.
The public sector may accrue significant benefits from these little but significant
cultural revolutions, the following Table 6 and Table 7 synthesize the most important of
these benefits.
Agile methodology benefits for public e-procurement
Increased focus to strategic goals, rather than functionalities of the platform
Increased timeliness in answering the high dynamicity of public procurement
Increased transparency, reduced information asymmetry (between authorities and

IT partners)
Increased focus on performance measurement
Increased engagement of public officers and empowerment of skills
Table 6 - Agile methodology benefits for public e-procurement
Lean thinking approach benefits for public e-procurement
Reduced waiting times during the procurement cycle (one-piece-flow)
Reduced effort related to manual and repetitive activities due to automation
Increased efficiency of the tendering procedures, and continuous
improvement
Reduced costs related to the consumption of material resources,
dematerialization
Increased standardiziation of the process, which helps to increase quality
Table 7 Lean thinking approach benefits for public e-procurement
This paper focused on the specific case of e-procurement, which is particularly
significant because of the critical role that it plays in the public governance. However,
the described modifications may apply to other contexts related to the renewal of the
public administration under a managerial perspective. These measures perfectly fit in
the current of policy reforms that have been inspired already by the so-called new
public management, or collaborative public management (Aucoin, 1990) (Hood, 1991)
(Politt, 1993), whose goal is to orient the public action more towards its outcome,
rather than just focus on complying with regulations.
After more than 20 years since the first experimentations of these ideas in several
OCSE countries, many public authorities still rely on strict command-and-control type
of culture. Using performance-based contracts (like those to be ideally employed in an
Agile project), embarking in public process re-engineering activities to reduce and
simplify bureaucracy (with a Lean approach) do not fit with this organizational culture.
However, according to the authors, these are specifically the types of reforms that
may provide consistency to the political discourses about the role of the public sector
to leverage economic growth, innovation and systemic competitiveness that
frequently characterizes the policy makers speeches.

REFERENCES

Akingbe, P. (2013, June 10). Can Agile project management work in public sector projects? Tratto da Web
Technology Group: http://www.webtechnologygroup.co.uk/agile-project-management-public-sector/
Amemba, C., Nyaboke, P., Osoro, A., & Mburu, N. (2015). Challenges Affecting Public Procurement
Performance Process in Kenya. European Journal of Business and Management, Vol.7, No.7.
ANAC. (2014). Tratto da
http://www.anticorruzione.it/portal/public/classic/AttivitaAutorita/ContrattiPubblici/BandiTipo
ARCA Lombardia. (2012). Elenco Fornitori Telematico. Tratto da
http://www.arca.regione.lombardia.it/cs/Satellite?c=Page&childpagename=DG_CRA
%2FCRALayoutDettaglio&cid=1213440023091&pagename=DG_CRAWrapper
Aucoin, P. (1990). Administrative Reform in Public Management: Paradigms, Principles, Paradoxes and
Pendulums. Governance, 115-137.

Baldwin, C. (2014, December 19). Agile vs. Waterfall (Part 4) - Performance & Earned Value Management.
Tratto da Linkedin pulse: https://www.linkedin.com/pulse/agile-vs-waterfall-part-4-calvin-baldwin
Ballard, M. (2011, April 26). Agile will fail GovIT, Says Corporate Lawyer. Tratto da Public Sector IT:
http://www.computerweekly.com/blogs/public-sector/2011/04/agile-will-fail-govit-says-cor.html
Bandiera, O., Prat, A., & Valletti, T. (2009). How much public money is wasted, and Why? Evidence from a
Change in Procurement Law. American Economic Review.
Beck, K., Martin, R., Fowler, M., & al. (2001). Manifesto for Agile Software Development. Tratto da
www.agilemanifesto.org/: www.agilemanifesto.org/
Bertini, L., & Sciandra, L. (2001, settembre ). Impolicazioni teoriche dell'eprocurement ed analisi del modello
adottato dalla PA.
Biniam, G., Petter, H., Mark, M., & Becca, O. (2012). Transforming government performance through lean
management. McKinsey Center for Governement.
Bof, F., & Previtali, P. (2007). Organisational Pre-Conditions for e-Procurement in Governments: The Italian
Experience in the Public Health Care. The Electronic Journal of E-Government, Vol. 5, Issue 1, 1-10.
Bulut, C., & Yen, B. (2013). E-Procurement in the Public Sector: A Global Overview. Electronic Government
an International Journal, 189-210.
Bustard, D., Wilkie, G., & Greer, D. (2013). The Diffusion of Agile Software Development: Insights from a
Regional Survey. In Information Systems Development (p. 219-230). Springer New York.
CERGAS. (2012). I processi di acquisto di beni e servizi nelle aziende sanitarie: modelli di accentramento e
impatti sul sistema.
Chang, H., & Wong, H. (2010). Adoption of E-procurement and Participation of E-Marketplace on Firm
Performance: Trust as a Moderator. Information & Management, Vol. 47, 262-270.
Croom, S., & Brandon-Jones, A. (2004). E-Procurement: Key issues in e-Procurement Implementation and
Operation in the Public Sector. 13th International Purchasing & Supply Education & Research
Association (IPSERA) Conference. Catania, Italy.
Formez. (2004). L'innovazione nei processi di acquisto nella P.A. regionale e locale.
Frota, O., Andrade, P., Filho, F., & Albuquerque, A. (s.d.). PM5: One approach to the management of IT
projects applied in the Brazilian public sector.
Gardenal. (2013). A Model To Measure E-Procurement Impacts On Organizational Performance. Journal of
Public Procurement, Vol. 13, Issue 2, 215-242.
Gardenal, F. (2010). Tratto da
http://www.arca.regione.lombardia.it/shared/ccurl/746/1004/Paper_italiano_Public_eProcurementdefinire_misurarare_ottimizzare_i_benefici.pdf
Gardenal, F. (2013). A Model To Measure E-Procurement Impacts On Organizational Performance. Journal
of Public Procurement, Vol. 13, Issue 2, 215-242.
Gatewit. (2012). E-Procurement Savings in Europe.
Gero, J., & Kannengiesser, U. (2007). An Ontological Model Of Emergent Design In Software Engineering.
International Conference on Engineering Design. Paris, France.
Glass, R. (2001). Agile Versus Traditional: Make Love not War! Cutter IT Journal, 14(12), 12-18.
Hardy, C., & Williams, S. (2008). E-Government Policy and Practice: a Theoretical and Empirical Exploration
of Public e-Procurement. Government Information Quarterly, 25, 155-180.

Hawrysh, S., & Ruprecht, J. (2000). Light Methodologies: It's Like Dj Vu All Over Again. Cutter IT Journal,
13, 4-12.
Henriksen, H., & Mahnke, V. (2005). E-Procurement Adoption in the Danish Public Sector: The Influence of
Economic and Political Rationality. Scandinavian Journal of Information Systems, Vol. 17, Issue 2,
85-106.
Hood, C. (1991). A Public Management for All Season? Public Administration, 3-19.
Hughes, W., & Shabnam, K. (2013). Performance-based Contracting in the Construction Sector. University of
Reading.
Ilhan, N., & Rahim, M. (2011). Predicting e-Procurement Systems Implementation Benefits: Development of
a Model and Implications for Management. In Innovation and Knowledge Management: A Global
Competitive Advantage (p. 744-756).
Intellect - Computing Services and Software Agency, UK. (2000). Getting IT Right for Government.
Krn, S. (2004). Analysing customer satisfaction and quality in construction: the case of public and private
customers. Nordic Journal of Surveying and Real Estate Research, Special Series, Vol. 2.
Koch, A. (2011). 12 Advantages of Agile Software. Expert Reference Series of White Papers, Global
Knowledge Training LLC.
Lennon, S. (2013, Febraury 06). Three common misconceptions about Agile in the public sector . Tratto da
Unboxed Consulting: https://www.unboxedconsulting.com/blog/three-common-misconceptionsabout-agile-in-the-public-sector
Leukel, J., & Maniatopoulos, G. (2005). A Comparative Analysis of Product Classification in Public vs. Private
e-Procurement. The Electronic Journal of e-Government, Vol. 3, Issue 4, 201-212.
Leydesdorff, E., & Wijsman, T. (2008). Why government ICT Projects Run Into Problems. Netherlands Court
of Audit.
Marton, M., & Paulova, I. (2011). One piece flow - Another view on production flow in the next continous
process improvement.
McCauley, R. (2001). Agile Development Methods Poised to Upset Status Quo. SIGCSE Bullettin 33(4).
McCue, C., & Roman, A. (2012). E-Procurement: Myth or Reality? Journal of Public Procurement, Vol. 12,
Issue 2, 212-238.
Moran, A. (2014). Agile Software Development. Springer International Publishing.
Mota, F., & Filho, R. (2011). Public e-Procurement and the Duality of Technology: A Comparative Study in the
Context of Brazil and the State of Paraiba. Journal of Information Systems and Technology
Management, Vol. 8, Issue 2, 315-330.
Munassar, N., & Govardhan, A. (2010). A Comparison Between Five Models Of Software Engineering. IJCSI
International Journal of Computer Science Issues, Vol. 7, Issue 5.
National IT and Telecom Agency. (2010). Agile Methods in IT-Based Business Projects - Guide to Agile
Development in the Public Sector. Copenhagen: Ministry of Science Technology and Innovation, DK.
Negro, G., & Ozzello, S. (2010). Le Soluzioni Snelle per la PA. Maggioli.
Nicoletti, B. (2010). Procurement e gestione costi: uno scambio di esperienze. Procurement Director forum
2010. Gubbio.
Ohno, T. (1978, 1998). Tovota Production System: Beyond Large-Scale Production. Toyota.

Oxford college of procurement and supply. (2014, Agosto). Tratto da

https://www.oxfordcollegeofprocurementandsupply.com/the-benefits-of-e-procurement/
Parliamentary Office of Science and Technology, UK. (2003). Government IT projects.
Perrella, G., & Delli Rocili, G. (2009). Riduzione degli Sprechi e Miglioramento dei Servizi nella Pubblica
Amministrazione. Franco Angeli.
Peter. (2011, February 3). Apple, Google, or Microsoft? Which Does Agile Better? Tratto da
http://agilescout.com/: http://agilescout.com/apple-google-or-microsoft-which-does-agile-better/
Politt, C. (1993). Managerialism and the Public Services: the Anglo-American Experience, 2nd Ed. Oxford:
Blackwell.
Radnor, Z., & Walley, P. (2008). Learning to Walk Before We Try to Run: Adapting Lean for the Public Sector.
Public Money and Management, 13-20.
Rodrguez, P., Markkula, J., Markku, O., & Turula, K. (2012). Survey on Agile and Lean Usage in Finnish
Software Industry. Proceedings of the ACM-IEEE international symposium on Empirical software
engineering and measurement, 139-148.
Ronchi, S., Brun, A., & al. (2010). What is the value of an IT e-Procurement System? Journal of Purchasing
and Supply Management, 16, 131-140.
State Government of Victoria and Demos. (2008). Towards Agile Government. Melbourne: State Government
of Victoria and Demos.
Thai, K. (2001). Public Procurement Re-Examined. Journal of Public Procurement, Vol. 1, Issue 1, 9-50.
United Nations. (2011). E-procurement: toward transparency and Efficiency in public service delivey.
Vaidya, K., Callender, G., Sajeev, A., & Gao, J. (2004). Towards a Model for Measuring the Performance of eProcurement Initiatives in the Australian Public Sector: A Balanced Scorecard Approach. Australian
e-Governance Conference. Melbourne.
Varney, M. (2011). E-Procurement - Current Law and Future Challenges. ERA-Forum, 12(2), (p. 185-204).
Vona, R., & Di Paola, N. (2014). Lean thinking in sanit: il caso della logistica degli approvvigionamenti e
della distribuzione ospedaliera dei farmaci. MECOSAN.
Wang, T., & Jin, P. (2012). Planning performance based contracts considering reliability and uncertain
system usage. Journal of the Operational Research Society, vol. 63, 14671478.
Webb, R. (2004). Managers' Commitment to the Goals Contained in a Strategic Performance Measurement
System. Contemporary Accounting Research, Vol. 21, N. 4, 925-958.
Womack, J., & Jones, D. (1996). Lean thinking: Banish Waste and Create Wealth in your Corporation.
Womack, J.; Jones, D.; Ross, D. (1990). The Machine that Changed the World. New York: Free Press.
World Bank. (2004). Strategic Electronic Government Procurement - Strategic Overview an Introduction for
Executives. New York: World Bank Institute.
World Bank. (2006). Introduction to Performance-based Contracting. Tratto da The World Bank Group:
http://www-esd.worldbank.org/pbc_resource_guide/trn.htm
World Health Organization. (2007). What Is Performance-Based Contracting? Tratto da World Health
Organization: http://www.who.int/management/resources/finances/Section2-3.pdf