Improving ROI and Success Rate of Your Business Intelligence Project

Improving ROI and success rate
of your Business Intelligence

project using an AGILE iterative
and incremental approach
By Martin Proulx and Dominic Danis
Executive Summary
About the authors Business intelligence (BI) applications are increasingly popular and
Over the years and currently as Vice-
organizations of all sizes are seeing the benefits. As a
President, Agile Business Intelligence at
Pyxis Technologies, Martin Proulx has
consequence, BI applications are no longer limited to senior
successfully participated in several executive but are increasingly deployed throughout departments
projects with companies from the and levels. The timely completion and successful implementation
pharmaceutical, retail, financial, and of BI projects provides a competitive advantage to the adopting
manufacturing sectors. Martin holds a organizations. Unfortunately, there are more failures and
Master's degree in management disappointments than successes.
sciences and a Bachelor of Business
Administration from HEC Montreal. He It is generally accepted that other pre-requisites such as: ensuring
cumulates over 15 years of experience executive support, aligning business and IT strategies or selecting
in: software development project
the appropriate technology platform are critical to the success of
management, business process
optimization, and dashboards and key
BI projects. In addition to these requirements, we believe that an
performance indicators (KPIs). AGILE iterative and incremental development approach combined
with light weight project management principles, such as SCRUM,
Dominic Danis cumulates over 10 years greatly improve this track record. In addition, since the
of experience in software development development approach and project management techniques are
and BI project management. In an mostly under the control of the information technology (IT)
effort to use a better approach to executive, they can more easily be altered in order to deliver
software development Dominic has better results to the organization.
been using Agile and Scrum since 2005.
He is a Certified Scrum Master and he
Our approach relies on simple and straightforward principles.
successfully applies this approach to
Business Intelligence projects. Dominic
Through short cycles of definition, planning, building, presenting,
is currently Principal, Agile Business and retrospecting, the development team iteratively delivers
Intelligence at Pyxis Technologies. value and incrementally adds components to each other to
achieve a complete BI application. As long as the development
About Pyxis Technologies team focuses on highest value KPIs, building them from beginning
Pyxis is the name of the constellation to end incrementally and routinely demonstrating their work to
of the compass and as such Pyxis acts decision makers for feedback, they will remain focused.
as beacon for companies seeking to
improve their ways of doing software Our objective is to increase the ROI of the BI initiatives by:
development.
 Reducing the initial investment required to launch a BI
Pyxis was founded in 2000 as the result project by quickly delivering tangible components to the
of reflection of engineers in computer business users instead of heavily investing upfront in
science questioning the traditional architecture and modelling costs.
approaches to software development.  Accelerating the availability of the information by using an
Improving ROI and success rate of your business intelligence project through an iterative and incremental approach 2
Their questioning led them to adopt incremental approach to present high level indicators at
new approaches to software first and refining the requirements towards more granular
development project management and information.
to improve their processes and
 Delivering the right solutions to the business users by
practices.
working iteratively with them to fully understand the
requirements.
 Prioritizing the work so the requirements with the highest
value and ROI are addressed first, thereby continuously
delivering values to the organization.
Track record of BI projects

There was a running gag in an organization we were working with that every Business Intelligence (BI)1
project required 3 years to complete and would cost over $3 Million. This running gag demonstrated an
unfortunate situation – most BI projects are expensive, time and resource consuming and have a low
rate of success. Along these lines, very few people would debate that traditional BI projects require
heavy investments and result in long delays before providing value to the organization.
Much has already been said about the importance of BI applications in organizations. One of the most
obvious and frequently repeated benefits of BI applications is the ability to increase the quality and
timeliness of the business decisions being made. This is increasingly true as BI applications are being
democratized and are now embedded in the decision making process at every level of the organization
and across multiple departments.
In addition, decision makers constantly need an in-depth understanding of their business’ strengths and
weaknesses and this is especially true during difficult economic times. The availability of timely and
accurate information can help business leaders make the right decisions. Along these lines, the survey of
over 1,500 CIOs conducted by Gartner EXP2 shows that despite predicted flat IT budget growth in 2009,
BI projects remain their number one technology priority.
There isn’t much need to emphasize further the value of early diagnosis and the implementation of
timely solutions but in a changing business environment the business needs evolve too quickly for
sequential waterfall development methodologies such as the one presented in the following diagram.
1
Within the context of this white paper and for simplicity, we combine Data Warehouse (DW) projects with
Business Intelligence (BI) projects.
2
http://www.gartner.com/it/page.jsp?id=855612
Copyright © 2009 Pyxis Technologies

www.pyxis-tech.com
Case in point, it is estimated that 60%3 of BI projects end in abandonment or failure and market data
confirms that out of the 3 standard project dimensions – time, resources, and scope – timelines and
resources typically exceed the initial plan while scope consistently fall short of the original expectations
and requirements. As such, an approach that delivers value early in the project while remaining aligned
with the business priorities is recommended.
In addition, estimates show that no more than 20%4 of business users actually use their BI applications
proactively and that a staggering 64%5 of systems functionalities are rarely or never used. This
regrettable situation compelled us to look for an alternate and improved approach to ensure BI projects
have greater success rate. More specifically that BI projects deliver the features required by business
users (scope) in a timely fashion (time) while controlling expenses (budget).
Our intent is neither to rehash the unfortunate track record of BI projects nor to highlight the most
dramatic failures but to propose a different approach to the development of BI projects.
This is a tall order but with the increasing popularity of AGILE iterative and incremental approaches and
with repeated success in other specialities; we know using such an approach in the development of BI
projects leads to much better outcomes.
An iterative and incremental approach for BI projects

The unimpressive track record in the development of BI applications led us to conceive an improved
approach to reduce the initial investment to launch a BI project and to modify the requirement
3
Business Intelligence Roadmap, Moss and Atre, 2003
4
Business Intelligence Summit by Gartner, 2008
5
Standish Group Study Reported at XP2002 by Jim Johnson

www.pyxis-tech.com
gathering phase in order to deliver value much more quickly and at lower cost. In addition, since our
incremental development approach remains aligned with the business priorities, our method allows
business users to further develop the knowledge of their business and better define their expectations
so that the end result meets their needs.
Although our approach requires a substantial paradigm shift for most people having experienced a
traditional waterfall approach for BI projects, the notions we propose are simple, logical and
straightforward. The proposed approach relies on AGILE principles and on SCRUM practices that have
been demonstrated to be successful in other types of initiatives time and time again. Our objective is to
present them in a concise manner in this white paper.
The proposed approach to BI development can be summarized by the following principles:

 Segment the development of components into small iterations of 1 to 4 weeks and frequently
deliver completed objects (key performance indicators, metrics, reports, dashboards, cubes,
etc.) at the end of these iterations;
 Repeat the iterative process and use increments to add components to the BI architecture and
infrastructure;
 Prioritize the development of components that will deliver the highest value to the organization
first;
 Use a “slicing approach” to go from source systems to presentation layer;
 Avoid analysis-paralysis by using a non-superfluous approach in documentation, architecture,
modeling, project management and other overhead activities;
 Learn from each of the iteration and adapt the process accordingly to increase efficiency.
Our approach promotes a process that encourages frequent inspection and adaptation, a set of
engineering best practices that allow for rapid delivery of high-quality software, and a business
approach that aligns development with customer needs and company goals. With this approach, the
project delivers small increments with minimal planning, rather than long-term planning which impact
the organization’s ability to react to changing needs.
As opposed to the traditional approach to building a BI application which is to plan the project, define
the requirements, prepare the architecture, develop, test, move to production in a sequential manner,
we use an iterative approach during which we build layers of the entire data warehouse through
iterations. For example, instead of spending time on defining high level requirements, working with the
end users to understand their business and the metrics used to successfully manage it, our objective is
to use an iterative approach and work backward starting with the required performance indicators back
to the source systems.

www.pyxis-tech.com
As can be seen in this diagram, the intent is not to divide the project into phases such as: data extraction
from the source systems, data cleansing and standardisation, Extract-Transform-Load (ETL) to a
centralized repository and so forth. Instead of the traditional approach, we use a “slicing approach”
approach where each iteration produces complete components developed from source system to the
presentation layer.
Success requires a paradigm shift

As we mentioned, the successful implementation of an AGILE iterative and incremental approach
requires an important paradigm shift in the way the project team operates. Although many other
variables may be impacted, we focus our efforts on five key areas: simplify requirements gathering,
become more flexible, reduce overhead, focus on value, and deliver on expectations.
1. Simplify requirements gathering

Our approach focuses on eliminating the assumption that decision makers clearly know what
information they need to perform their role and can easily describe what information they require
up front at the beginning of the project. By spreading the requirements gathering throughout the
duration of the BI project, the development team reduces and spreads the amount of stress
imposed on business users and is more likely to obtain meaningful requirements.
Based on the principle that end users may only know a subset of all the metrics they need to
successfully manage their business, we need to build the knowledge and expertise based on small
and specific known components at the beginning and evolve to more complex metrics once the
users understand the process.

www.pyxis-tech.com
2. Become more flexible

In an ever changing business context, one’s ability to anticipate and clearly document their need for
the next 12, 18, or 24 months seems absurd. Very few people would have predicted the current
financial context 6 or 12 months ago. Unfortunately, traditional development approach still heavily
relies on such a process to launch and develop a BI project. By allowing the project team to react to
changing requirements due to an ever changing business environment throughout the life of the
project, they increase the probability that the resulting cubes, reports, and dashboards get used
while ensuring the project team delivers value throughout the duration of the project.
The emphasis is on obtaining the smallest workable piece of functionality to deliver business value
early, and continually improving it and adding further functionality throughout the life of the
project.
3. Reduce overhead
The cone of uncertainty presents the reduction in the uncertainty of a project as it progresses in
time. For instance, the variability of the outcome is much more difficult to predict at the early stages
of the project than it is toward the end. As such, much less effort should be invested in planning,
architecting, modelling, documenting and defining requirements in the early stages of the project as
they are likely to change. Consequently, the level of details and the quality of the knowledge should
increase as the project progresses.

www.pyxis-tech.com
4. Focus on value
In contrast to a traditional approach which sequentially progresses through phases (i.e.
requirements definition, analysis, design, coding, and testing) in a strict pre-planned sequence which
measures progress in terms of delivered artefacts (requirement specifications, design documents,
test plans, code reviews), our approach focuses on completed useful software components.
Although these artefacts have value for the project team, we feel too much time and effort is
invested in these intermediate deliveries instead of working software. The recommended approach
does not completely eliminate these artefacts but reduces them to a non-superfluous level in order
to remain focused on the key outcome which is to deliver value.
In addition, our proposed approach focuses on value by increasing the ROI of BI components being
developed and deliver higher value components earlier in the process and halt the project when the
cost of developing exceeds the expected value.
5. Deliver on expectations
By presenting completed deliverables frequently, such as KPIs, metrics, reports or cubes to the
business users at the end of each iteration, this allows any potential issues with the quality and
accuracy of the data to surface and be addressed.
Four levels of an Agile BI project

Our approach to the development of a BI project is divided into 4 levels: Program, Project, Release and
Sprint. Contrary to sequential development, these levels are not consecutive but encompassing in
nature where a program is composed of projects which are themselves composed of releases and each
release is divided into sprints.
BI projects are often seen as large and complex endeavours and in order to maintain an enterprise wide
view and ensure constant alignment, they are grouped under an overall program.
At the project level, activities such as project definition, staffing, and budget definition are executed.
Although important, our intent is not to focus on the initiation of the project but it is critical to
remember that activities during the project level remain at a high level due to the amount of uncertainty
still surrounding the project.
A release is composed of an agreed upon number of sprints during which components of the BI
application will be completed. For example a release could cover the need of a specific business area
such as sales, marketing, operations or a business process such as order fulfilment, shipping, or client
satisfaction.

www.pyxis-tech.com
Once the release has been defined at a high level, it is then divided into smaller components – called
sprints. Based on the size and experience of the development team, a sprint can be anywhere between
1 to 4 weeks.
With a traditional development approach, each task must be completed before the development team
can move to the next phase whereas with an iterative and incremental approach, the development
team delivers small components frequently.
Five steps repeated throughout the four levels

The process is iterative and incremental at the four levels of the project with a major distinction in the
amount of details being analyzed. While the project team will remain at a high level during the project
cycle, they will look for more details during the release cycle and very granular details at the sprint cycle.
For example, at the project cycle the business user is expected to explain that the project team will need
to provide performance indicators for his consulting department. Once the team cycles through the
release cycle, the business user would explain that he is looking for utilization rate for his consultants
and at the sprint level he would provide information as to how the utilisation rate is calculated.
While the traditional approach to a BI project calls for much of the efforts to be invested in planning and
analysis, an incremental and iterative approach prefers to deliver small components frequently in order
to obtain feedback and learn from the exercise.
Define
The objective of the first step of our approach is to get familiar with the project, the main players, and
the expected outcome, as well as, understand the priorities. During these sessions, the project team
interviews the key business users to gather their requirements and refine their understanding of the
business challenges. From a project perspective, a business representative is assigned to the initiative

www.pyxis-tech.com
with the objective of coordinating the business users involved with the project, due to the fact that BI
typically impacts various departments.
During the first meetings the conversation with the business users is around their business issues,
concerns and priorities. The objective is to have a good understanding of the various dimensions of their
business and key members that will support the main representative for the definition of performance
indicators for their department. The outcome of the definition stage is to draft a project charter that
includes the following elements: vision for the project, objectives, assumptions, success factors, success
criteria, roles and responsibilities, priority matrix (scope, timeline, and budget), risks, and mitigation. In
addition to a project charter, the planning stage allows the project team to start populating a product
backlog. During the sessions, the team’s objective is to derive stories that will later be prioritized.
Through a well defined process, the team executes a requirement definition phase through stories6. We
typically use the following structure to derive the various stories: “As [role], I need [metric or ratio] by
[geography, time period, etc.] in order to [business decision or business value]”. For example, the
business representative speaking on behalf of his director of operations would formulate a story such
as: “As director of operations, I need to measure my resource utilization by month and by business
practice in order to ensure that I maximize my revenues”.
This simple story already allows us to determine the various roles, subject area, metrics, and
dimensions. In addition, such a story allows the project team to ask business related questions allowing
small increments of knowledge to be transferred to the project team and better understand the
business dynamics.
Subsequent questions would possibly be to explain what is meant by measure, how is resource
utilization defined, what are the business practices, etc. The objective is not to drill deep into each
discussion this early in the process but simply to note the details behind the stories so they can then be
prioritized. It is critical to remember that the level of details will increase as we progress through the
various stages of the project.
During the definition phase, the project team meet with the business representative to determine and
agree on the priority of the team for the duration of the release. It is critical to note that the intent is not
to agree on the details during this stage but the high level direction the team will follow for the duration
of the release. For example, during a recent project done for software development organization the
general manager who was acting as the business representative decided to focus primarily on KPIs for
his consulting practice instead of his product development and training departments. He felt that the
competitive pressures were stronger in his consulting unit and as such, the information he would obtain
from his BI application would be more valuable in the current context. It is critical to remember that we
are not trying to segment the development process by business unit but simply to help with the
definition of the objectives and requirements of each business unit.
6
User Stories Applied: For Agile Software Development by Cohn, 2004

www.pyxis-tech.com
Plan
Planning consists of selecting the high level stories to be detailed further from the product backlog. The
objective is to select enough of the highest value stories to be investigated further within the current
project.
At the beginning of the sprint, the team selects items from the backlog. The items with the highest value
are typically considered highest priority. The objective is to work on the highest yielding story while
showing results as quickly as possible. It is critical to focus on the content of the current sprint and not
to spend effort on planning and defining further that current sprint as business priorities might change
the value.
During the planning phase, the development team comes together to assess the efforts associated with
the selected stories. The objective of this phase is to estimate the efforts associated with each story in a
relative measure.
Once the stories are defined, the product owner needs to quantify the value of the cards. The measure
is not as important as the outcome. Some organizations use points, play money or even poker chips to
quantify the value of the stories. Although the process of quantifying can be fun, we are mostly
interested in the outcome. The stories are then sorted in descending value.
As such, stories such as calculating the employee satisfaction rate is certainly more complex than
reporting on the company’s sales, especially when the team suspect that the underlying information
may not be available or accurate.
In addition to needing to define and express his own needs and requirements, the product owner has to
have the ability to prioritize his stories appropriately in order to help the development team achieve
success. Since the AGILE approach is strongly tied to delivering business value, it is important that the
business owner be able to properly understand and communicate the value of each of the stories.
Contrary to a traditional approach where the value of a BI project is estimated at the beginning when
funding is established but not revisited unless additional funds are required, the AGILE approach brings
up the value discussion throughout the development cycle. In addition to helping to determine the
priority of the various stories, establishing value for each story helps to ensure alignment across the
business while avoiding building components that are not necessary. During the planning of each sprint,
asking the question “why are we doing this?” will help bring to the surface the real requirements and
keep in mind the importance of the team goal.
In addition, instead of simply tracking ongoing cost of the project, keeping track of the value of the
components delivered allows the project team to measure the cost and benefits at any point in time of
the project. As such, instead of spending money to develop requirements that may have been
determined months or years before, the question of needing to invest in additional metrics helps the

www.pyxis-tech.com
organization ensure they spend wisely. Instead of the approach of determining the scope upfront and
defining the budget and timelines as a consequence, we define the budget first and play with the scope.
Hence, why the benefits of each story are so critical.
In addition to spending too much time on planning, analysis, modeling and architecture, the proposed
approach reduces this overhead by using a non-superfluous approach where the project team focuses
from beginning to end on the selected priority. Where the traditional approach recommends defining
the entire scope of the project upfront and spends up to 70%7 of the efforts on the Extract-Transform-
Load (ETL) before the business users actually see results, an AGILE BI approach focuses on delivering
incremental value.
Build
The building phase takes the project team to a more granular level of details and requires the team to
move to the next level of the project.
Much happens during the building phase where each member on the development team selects stories
on which he/she will work for the duration of the sprint. Typically stories can be divided along two lines
where the developers take responsibility for development in addition to architecture and modeling
tasks.
Another requirement for a successful implementation is to ensure continuous testing and integration of
the components built. It is recognized that data quality is a critical success factor for a BI project and as
such, it remains critical to ensure that additional components built do not break existing scripts and
negatively impact data quality. As opposed to testing at the end of the various releases which is often
the case in traditional projects, continuous integration and testing ensure that testing occurs every time
a new component needs to be integrated.
In a traditional project, much effort is spent fixing quality of the underlying data even if only 10% or 20%
of all the data will actually be used. The incremental approach deals with inaccurate data just when it is
required. As such, the development team spends time working on fixing issues only when it is required.
Timely design and architecture prevents the development team from over architecting the solution and
relies on a non-superfluous solution. Perfection is difficult to reach and the cost for trying to achieve it
hardly justify spending it.
It is expected that the team will encounter some un-anticipated issues during the project. As such, it is
always possible to integrate some of the fixes required in the next sprint planning session. Similarly
performance improvements or database refactoring will be integrated within the various sprints.
To ensure success you have to make sure refactoring is a key component of your approach as you will be
tweaking and tuning your solution throughout multiple sprints and since it is better to present
7
The Data Warehouse Lifecycle Toolkit, Kimball & all., 2008

www.pyxis-tech.com
something to the business users than nothing at all, the development team needs to factor in some time
to fix some of the underlying issues all of this while delivering on an ongoing basis.
Present and Deploy

Everyone will agree that the real test is when actual users get to use the system. It is therefore expected
that at the end of each sprint, the development team presents the various metrics that have been
developed during the sprint to the business users. This practice has a few clear benefits in that unless
metrics can be demonstrated in the application – not on paper or in theory – and shown to provide the
expected information, they are not completed. As opposed to asking developers if their work is done, it
is much more accurate and telling to see the performance indicators and determine if they do meet the
expectation or not.
In addition, having to demonstrate the Key Performance Indicators (KPIs) within the application requires
that the proper testing has taken place otherwise the development team will have to present a faulty
indicator or dashboard.
Finally, being able to present indicators early in the process helps the end user better understand the
development team’s capabilities and fuels better business discussion which will eventually lead to
additional stories being added to the backlog.
Many BI initiatives have been cancelled or lost much credibility once decision makers realized that the
data presented was inaccurate. Unfortunately these situations come to the surface late in the game in a
waterfall approach. When using an iterative approach, inaccurate data quickly come to the surface and
are addressed quickly for the project to continue.
To ensure the development team is on the right track, they really need working software in front of
business users at each stage. Once end users see how things work, they quickly move to a deeper
understanding of the process and are in a better position to properly define their expectations. Delivery
of completed KPIs gives business users the opportunity to see and experiment with the KPIs which then
allows them to adjust their expectations and eventually their requirements.
With the traditional approach a substantial amount of time is spent at the end of the project to test and
integrate all components. As in other development processes, the value of identifying and resolving
issues early in the process is always less expensive and more efficient as it usually highlights weaknesses
in knowledge or skills and are addressed sooner to avoid repetition throughout the project.
Finally, the transition to the production environment is usually very intense and as such reducing the risk
by releasing smaller components more frequently not only ensure a smoother transition but also helps
develop the right processes to increase efficiency.

www.pyxis-tech.com
Retrospect
The final step of the iteration is the retrospection which has a few objectives where the most important
one is to allow the team to reflect on the successes and determines which areas need to be improved
prior to entering the next sprint. As such, the team collectively assesses its own performance and
determine the best way to adapt in order to successfully achieve its next sprint.
The approach is also very useful from a learning and growth perspective where many organizations
undertake BI projects without having all the necessary experience and skills. Short and frequent
development cycles allow the development to build the required skills, learn from the process and
improve their technical and business expertise.
Conclusion
The AGILE approach heavily relies on business involvement within the project team and throughout the
duration of the BI project. Having direct contact with the business team, the development team will
more easily understand the expectations and will increase its knowledge of the business which will make
a more knowledgeable and productive team altogether.
In order to achieve our objectives to: reduce the initial investment, accelerate the availability of the
information, deliver the right solution to the business users and constantly deliver value to the
organization we explained that an AGILE approach is the right solution. Through iterations and
increments, we use a “slicing approach” to develop from source systems to presentation layer in order
to deliver on the business users’ requirements.
The benefits of an AGILE approach to a BI project are substantial. From our perspective, the most
important one for the organization is that it forces the project team to focus on ROI and deliver high
value by spending efforts on activities that will bring the highest value.
By reducing the duration of the development cycles the business and development team can modify
their release strategy and focus on new priorities that may arise as a consequence of a changing
business environment.

www.pyxis-tech.com

Improving ROI and Success Rate of Your Business Intelligence Project

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Improving ROI and Success Rate of Your Business Intelligence Project

Uploaded by

Copyright:

Available Formats

Improving ROI and success rate

of your Business Intelligence

Track record of BI projects

Copyright © 2009 Pyxis Technologies

An iterative and incremental approach for BI projects

Copyright © 2009 Pyxis Technologies

The proposed approach to BI development can be summarized by the following principles:

Copyright © 2009 Pyxis Technologies

Success requires a paradigm shift

1. Simplify requirements gathering

Copyright © 2009 Pyxis Technologies

2. Become more flexible

Copyright © 2009 Pyxis Technologies

Four levels of an Agile BI project

Copyright © 2009 Pyxis Technologies

Five steps repeated throughout the four levels

Copyright © 2009 Pyxis Technologies

Copyright © 2009 Pyxis Technologies

Copyright © 2009 Pyxis Technologies

Copyright © 2009 Pyxis Technologies

Present and Deploy

Copyright © 2009 Pyxis Technologies

Copyright © 2009 Pyxis Technologies

You might also like