Professional Documents
Culture Documents
But as noted at the outset, the fact that they are complex undertakings likely to
encounter such a variety of difficulties should not necessarily give rise to
failure - or an increased incidence of failure – as long as the level of difficulty
was met by an appropriate level of response (this said, the more complex the
project, the greater the potential for the response to be insufficient). The
challenge, therefore, for any IT project, is to understand the degree of
complexity entailed and then respond accordingly. In line with our ‘opposing
forces’ concept discussed previously, ultimately, for the project to succeed, the
strength of the response needs to be at least equivalent to the strength of the
complexity. If the complexity exceeds the response, the project will fail. If the
response exceeds the complexity, then the project will cost more, take longer,
and consume more resources than it needed to. Given the number and
proportion of projects that run into difficulties, however, it is apparent that
very often the response is not adequate. This raises the obvious question –
why is this?
Whilst as the previous chapter has shown, there are any number of specific
factors which can cause projects to have problems, it seems that at a more
fundamental level, there are a small number of deeper-seated and
interconnected underlying ‘tendencies’ which give rise to a mismatch between
the strength of the response relative to the level of complexity inherent in a
project. In particular, these include:
It will probably come as no surprise to the reader (assuming anyone has got
this far) to hear that in my view, much of this stems from the first item on this
list, the tendency on the part of those who commission IT projects to
underestimate or to fail fully to appreciate the complexity, and the nature of
the complexity, implicit in any particular project (see for example Royal
Academy of Engineering/British Computer Society 2004). 1 Without such an
appreciation, there is an inevitable likelihood of committing to projects which
are overly ambitious (see Collins, 1998) 2, to under-resource the project teams
required to deliver them, and to neglect many of the senior management
responsibilities that go with their delivery.
This may help explain why organisations often appear to enter into such
initiatives without undertaking the necessary preparation and analysis to put
them in a position where they have as comprehensive an appreciation as
possible of the difficulties they are likely to encounter, and know that the
project they are undertaking is the right one under the circumstances.
Whereas objectively speaking, where managers lack a solid appreciation of the
subject matter, the logical response would be to undertake the research
needed or seek out appropriately qualified advice to ensure that the
complexities and risks are made explicit and understood (cf ‘Time spent in
reconnaissance is seldom wasted’), in practice, organisations seem frequently
to enter into such initiatives quite lightly, even casually, without considering
profoundly the real questions, issues and challenges confronting them, the
different options that might be available to them, the implications of these
options and their respective advantages and disadvantages. They seem all too
ready and all too quick to leap straight to one solution, namely to launch a new
IT project, without really considering the alternatives, without being sure that
this really is the best way of addressing the perceived organisational
requirements (and in some cases in ignorance of whether it will actually meet
them at all), without any deep appreciation of what such an undertaking is
likely to entail, in terms of things like resources and capacity, cost and
timescale, and without understanding the risks and rewards of such an
undertaking.
To leap so quickly into action is to miss the biggest single point of leverage in
the entire project, which of course comes right at the very start – in fact, and
perhaps more accurately, before what might be regarded as the start of the
undertaking, when such crucial decisions are taken around whether to commit
to the project, and if so, what form it should take, and how to set it up to
maximise the chances of success. None of this is of course rocket science. Any
project management textbook will set out in detail the various stages of the
ideal project life cycle, and emphasise the vital importance of preparation and
planning. But such discipline is clearly frequently lacking in the case of IT
initiatives.
As noted, perhaps because they see them as largely technical activities where
they have little to contribute, and perhaps also because they see themselves as
having performed their key role in ensuring the project was kicked off in the
first place, it is not unusual for senior managers to distance themselves from
such projects once they have commenced. However, the Royal Academy of
Engineering/British Computer Society emphasised the importance of executive
sponsorship as a critical success factor in the successful delivery of complex IT
projects (a perspective which was reinforced in the 2012 Standish CHAOS
Manifesto)8, and saw the frequent absence of this as a major cause of failure.
There appear to be three key aspects of such leadership that are critical. The
first of these is to demonstrate support for and commitment to the project in
question, so as to encourage organisational engagement with it and alignment
behind it. Without this, it is easy for others to regard the undertaking as not
being of any great importance. This becomes increasingly significant if and
when the project runs into difficulties – as of course many projects do at some
point. Secondly, there is also a need to ensure that all parts of the organisation
work together in support of the project’s delivery, and that inter-departmental
tensions and disagreements are effectively resolved. Given their nature, most
IT projects will span internal boundaries within an organisation, and hence
disputes (eg between user departments and the IT department, or between
different user departments) are highly likely to occur. Appropriate resolution
of these will very often not be possible without senior intervention. Thirdly, as
we have seen previously, IT projects normally give rise to a complex set of
choices and dilemmas – many of which extend well beyond purely technical
matters - where there is not necessarily an obvious right answer for the
organisation. Given this, senior management involvement in determining the
best course of action under the circumstances is highly desirable. Without this,
there is an increased likelihood that any choices made will be organisationally
sub-optimal.
A further manifestation of the failure to grasp the full complexity of the project
being undertaken is a tendency to under-resource the initiative, notably in
terms of the level and skillsets of the people allocated to work on it, as well as
the budget made available to it. It follows that if those commissioning IT
projects lack a full appreciation of the challenges involved, and regard the
initiative as simpler than it really is, then it will be resourced at the level
perceived to be appropriate, rather than the one that is actually required. In
terms of our ‘opposing forces’ construct, this means that if a project of – say –
factor 6 complexity (on a scale of 1 to 10, with 10 being the NPfiT) is regarded
as having only factor 4 complexity, the resources made available to the project,
and the appointments made, will reflect the perception of a factor 4 project. As
a result, the individuals chosen may likewise lack the appreciation of the
additional complexities associated with a factor 6 project. Related to this, a
common practice is to appoint an established manager with a good
understanding of the organisation, its activities and processes – but negligible
previous exposure to IT projects – to lead the project, perhaps with some
support from an IT professional or an external consultant. Again, without an
established frame of reference for the activity, without a clear sense of ‘what
good looks like’, and lacking the appreciation of the complexities, risks and
pitfalls inherent in IT projects, there is often a propensity for the individual in
question to downplay or neglect these as the project progresses. Such
individuals are also less well-placed to deal with the unknowns (in Donald
Rumsfeld's parlance this would include both 'known unknowns' and 'unknown
unknowns', since it is depth of experience that best equips anyone to respond
to both of these), because they 'don't know what they don't know', and there is
a high risk of 'unconscious incompetence'.
At the same time, although the suppliers and consultants working with the
project may have the knowledge and experience to diagnose some of the
problems being encountered, the structure of their contracts may mean that it
is not necessarily in their interests to raise these with the client organisation.
Moreover, suppliers and consultants may also struggle to understand some of
the complexities encountered by a particular project, particularly where these
relate to the unique circumstances and the internal dynamics and culture of the
client organisation. As third party outsiders they may at times find it difficult
to get their voices heard – for example the customer may be suspicious of the
third party’s motives in raising the concerns, given that they are both
participants in a commercial arrangement; or there may be internal
organisational political considerations which mean that the messages being
conveyed by the supplier are not necessarily what the organisation wishes to
hear, and therefore chooses to turn a deaf ear to them.
This bias on the part of at least some senior managers to fail to recognise or to
underplay the complexity inherent in many IT projects may be reinforced by
the actions of IT departments, IT project professionals, and third party vendors
and consultants, all of whom have an interest in projects proceeding. This is
because such projects provide challenging and potentially rewarding work for
IT and project people – after all, for IT project people, this is what they do, and
if there are no projects, they’ll be out of a job; while for supplier organisations
and consultants, these projects provide the source of much of their revenues.
Hence, these groups may also have an incentive to overstate the case for
proceeding, and to underplay the difficulties to be encountered. These forces
again tend to encourage a sort of ‘Groupthink’9, where there is broad alignment
among a community of key interested parties in support of a particular project,
and where any voices of dissent tend to be ignored or overridden.
On one level, the continuing incidence of problems experienced by IT projects
is perhaps surprising, given that by now there ought to be enough evidence
available to demonstrate that these are generally difficult and risky
undertakings. For instance, if you asked the average man or woman in the
street for their perspective on IT projects, the reaction would probably be quite
similar to that of a plumber asked for a quote to fix a leaking pipe. Tricky, and
several times more expensive than the number you first thought of. But type
‘IT project failures’ into Google, and up pops a listing of a multiplicity of recent
articles and reports – many of which identify the same set of factors to account
for the high incidence of unsuccessful initiatives as are referenced in earlier
analyses. Other than perhaps some signs of slowly improving performance
emerging from the Standish data10, there is little to suggest that the reports of
high profile project problems and disasters have caused organisations
significantly to modify their approach to such initiatives if and when they
themselves embark upon them, or to do so with greater circumspection;
equally neither are there obvious signs of improved practice on the part of the
providers. This perhaps points to the deep-seated nature of some of the
factors referred to above - if those concerned have misplaced faith in their
ability, if they fail to appreciate the complexities, if they ‘don’t know what they
don’t know’, then there is again no incentive to look to fill a knowledge gap that
they would not recognise they had, and hence for them to seek to learn lessons
from others’ experiences; moreover organisations and practitioners are often
imbued with a sense of overconfidence and superiority that causes them to
believe that they will never make the same mistakes or run into the same
difficulties as experienced elsewhere (I recall visiting a reference site where
the company in question had recently rolled out the application we were
interested in, and being told that it was a great package, but had proved very
difficult to implement, and thinking, ‘That’s okay, we know how to do these
things, that won’t be a problem for us’...only to find out subsequently for
ourselves just how difficult it was to implement).
Finally, they also identify two other factors which contribute to the results
observed. The first of these is a failure to learn lessons from previous
experiences. From their analysis of over 100 major transport infrastructure
projects since 1920, Flyvbjerg, Bruzelius and Rothergatter show that there is
no trend in the degree of cost overruns among these projects over time.
Today’s project is every bit as likely to exceed its cost estimates as was
yesterday’s. Hence, there was no indication that there had been any collective
learning about project viability over the period.15 Secondly, they argue that
these results also point to major weaknesses in the analysis and management
of project risk16 (which as discussed previously is closely related to what I’ve
been calling ‘complexity’).
(In parentheses, it is perhaps worth noting that Flyvbjerg and his colleagues
were focused on transport infrastructure projects, a large proportion of which
tend to be in the public sector, and some authors – eg Myddelton (2007)17 -
have argued that project failures occur disproportionately in public sector
projects where there may be less pressure to demonstrate a definitive return
from the investment, and some of the controls and imperatives facing private
sector organisations undertaking projects are less prevalent. If this really is
the case it would presumably apply equally with regard to IT projects. It isn't
really possible to validate this theory, as private sector organisations are not
always keen to share information publicly about the projects they are
undertaking, especially if they run into difficulties, but it's clear that from the
information that is accessible regarding private sector projects, there are
plenty of examples where things have not gone as well as may have been
wished or intended - eg the Channel Tunnel, Wembley Stadium, or the Boeing
787 Dreamliner (see also the ‘Why Projects Fail website ‘Catalogue of
Catastrophe’ maintained by Calleam Consulting Ltd).18 It is also apparent that
similar human dynamics, such as the wish to leave a mark, the desire on the
part of practitioners to undertake ambitious and difficult projects because
these are interesting and challenging things to do, and the commercial
motivations of suppliers are frequently just as evident in the private sector. In
the case of IT projects specifically, Flyvbjerg and Budzier (2011) in the study
referred to earlier of 1471 projects concluded that there was little difference
between projects in different types of organisations (this said, 92 per cent of
the projects in their sample were in public agencies).19 My own experience is
exclusively in the private sector, and I can personally testify to involvement in
a number of IT initiatives best described as ‘problematic’ – but then perhaps
this was just me...).
Returning to the discussion of the human and psychological factors that may be
at play in this regard, military history is also replete with examples of
commanders who underestimated the opposition and overestimated their own
abilities and those of the resources they had available, to their considerable
detriment (see Dixon 1976). Dixon identifies a number of other characteristics
associated with what he refers to as ‘military incompetence’. These include:
an inability to profit from past experience, a failure to make adequate
reconnaissance, a tendency to reject information which is unpalatable or which
conflicts with preconceptions, a suppression or distortion of bad news, a
tendency to abdicate from the role of decision maker, and an undue readiness
to find scape-goats for setbacks.20
It may be that some of these tendencies are in fact more in evidence in relation
to IT projects (and perhaps also organisational transformation projects) than
engineering and construction projects, not least because the broad technical
challenges associated with the latter are better understood. As a result – and
perhaps because there are rules and regulations, such as building regulations,
in place in most countries that need to be complied with – the likelihood of an
organisation undertaking an engineering or construction project without
significant preparation and hence a solid appreciation of the difficulties likely
to be encountered, and without reasonably detailed plans in place, seems low.
In contrast, organisations seem much more likely to embark casually on an IT
project. Again, this all points to naivety and a lack of appreciation of the
difficulties and complexities inherent in such undertakings.
Ultimately, the conclusion that I – and hopefully also we – arrive at here seems
irresistible. Projects struggle and fail – and this goes for all projects – not really
because they are hard (even though they very often are), but because the
quality of the response to the difficulties they encounter is inadequate. This
must be the case, because by definition if the response was sufficient to
address the difficulties, the project wouldn’t fail. A number of factors
contribute to the inadequacy of the response, but at the source of these is a
widespread propensity on the part of those who commission IT projects to
underestimate and/or to fail fully to appreciate the complexity and
complexities inherent in the undertaking.