Professional Documents
Culture Documents
Table of Contents
Introduction...............................................................................................................3
Executive Summary....................................................................................................3
Recommendation Summary.........................................................................................4
Evaluation of Alternatives.........................................................................................4
Evaluation Methodology..............................................................................................4
PPM Alternatives Descriptions......................................................................................5
Conclusion.................................................................................................................6
Appendix A................................................................................................................8
TOTAL COST OF OWNERSHIP....................................................................................8
Appendix B..............................................................................................................13
SURVEY OF FUNCTIONAL EVALUATION CRITERIA.......................................................13
Appendix C..............................................................................................................17
FUNCTIONAL EVALUATION MATRIX..........................................................................17
Appendix D..............................................................................................................21
City Services Auditor Project Management Software Solution Evaluation Spring 2008 ...21
Appendix E..............................................................................................................28
VENDOR QUOTE INFORMATION ..............................................................................28
Page 2
PPM Alternatives Results & Recommendation Report
6/24/2009
Introduction
This is an evaluation with recommendations report for the Office of the CIO, which had
indicated the need for a review of cost alternatives for a Project Portfolio Management (PPM)
software application. A PPM application assists DT Management to view, analyze and
collectively manage all current or proposed projects, or groups of projects, across the
organization. Our objective is to identify suitable applications with the optimal Total Cost of
Ownership (TCO), as well as the best functional fit with our business and technical
requirements.
Executive Summary
DT’s Project Management Office (PMO) currently is using PRODUCT W for DT’s PPM
application. It is the central repository of all projects for which DT is responsible, including
COIT projects. This is what we call our “Book of Business”. However, the high-cost of the
PRODUCT W solution prompted DT to evaluate possible alternatives. To this end, Blair
Adams, DT’s Chief Technology Officer, and Chair of the COIT Performance Subcommittee,
undertook a project to accomplish this.
1. PRODUCT Z
2. PRODUCT Y
3. PRODUCT W
4. PRODUCT X
Note that this report does not preclude a more formal RFI/RFP process in the future.
Page 3
PPM Alternatives Results & Recommendation Report
6/24/2009
Recommendation Summary
Based on the weighted scores from our evaluation matrix (Appendix C), our
recommendation is to test PRODUCT Z, on a 90-day trial basis.
Evaluation of Alternatives
Evaluation Methodology
We began by gathering the business needs of those in DT who are currently using PRODUCT
W (PMO, COIT Staff, and PMs), as well as those in DT who are seeking similar functionality
(Contracts Section). From this a list of functional requirements was developed, and a
voluntary web survey was sent out to IT Managers across the City (see Appendix B), which
asked respondents to assign relative weights to each requirement. From the survey
responses a weighted functional evaluation matrix score sheet was derived (see Appendix
C).
Since it was decided that cost should be the most heavily weighted important to consider all
aspects of cost, a method to calculate Total Cost of Ownership (TCO) was identified and
adopted as well (see Appendix A). We made every effort to obtain cost information that was
as accurate as possible from vendors, as well as from DT’s relevant known internal costs.
When direct vendor cost quotations could not be obtained in a timely manner, cost
estimates were extrapolated from similar vendor sources. Costs were examined for two
modes of service: 1) On-Demand/Software-As-A-Service (SaaS), which is remote hosting by
the vendor; and, 2) On Premises/In-House hosting by DT.
After preliminary discussions with the alternate vendors, each one was then asked to demo
their solution to PMO Director Joe Armenta and R&D Manager Jay Nath, and each
performance was rated against our functional evaluation matrix score sheet. Since we are
currently using PRODUCT W, we were able to evaluate this ourselves. Finally, the Criteria
Weights derived from the survey were applied, and the final weighted scores were
calculated (Appendix C).
Page 4
PPM Alternatives Results & Recommendation Report
6/24/2009
PRODUCT X
The project manager can communicate project plans and task assignments to team
members via home pages in Web Access. Team members can communicate status and
changes to keep the project manager up to date.
Resource workloads can be analyzed by project and by resource with the Resource Center,
allowing organizations to forecast future resource requirements and make more efficient use
of resources.
However, at this time the vendor seems to be relying upon third-party vendors to provide
true project portfolio capability (dashboards, aggregate views, etc.) Also, since Windows
SharePoint seems to be a technical prerequisite for implementing PRODUCT X’s vision in this
arena, the true cost of implementation has the potential to be much higher.
PRODUCT W
PRODUCT W supports core project and portfolio management functionality. This functionality
provides the ability to unify PPM information and processes into a single system of record.
For individual projects, and groups of projects (portfolios), PRODUCT W can track resources,
organizational structures, budget and financial information, issues, and individual resource-
task timekeeping. A Project Administrator can control security defining users and access
rights. Access to PRODUCT W is via web browser.
Because PRODUCT W has a limited innate project scheduling capability (the vendor offers its
own open-source product for this), it supports the importing of project schedules from
either PRODUCT X (with limitations), or its own open-source product (preferred).
PRODUCT W also supports the attachment of external documents at the project and
portfolio levels, through its collaboration feature.
In our experience with using PRODUCT W, there have been some difficulties with
configurability and ease-of-use.
Page 5
PPM Alternatives Results & Recommendation Report
6/24/2009
PRODUCT Y
PRODUCT Y is currently being piloted by the Controller’s Audit Division. The Controller
initially looked at PRODUCT W, and some other candidate software offerings (see APPENDIX
D), before selecting PRODUCT Y for pilot. PRODUCT Y is aimed at project-based businesses,
providing automated time and expense tracking, billing, reporting, and basic project
management and work breakdown structures.
While PRODUCT Y provides most of the desired project portfolio functionality, some of the
desired graphics capability; such as, Gantt charting at the portfolio level, etc. are still “in the
works”, and are not yet available. Nevertheless, PRODUCT Y provides strong resources
tracking and analysis tools; is easily customizable; and has a robust reporting suite.
PRODUCT Z
The application can prioritize and measure projects using scorecards, and can create
business graphs of portfolio information such as status, budget variance, and completion
time line. The graphics support project status color codes and arrows to indicate
improvement.
More attractively from the individual Project Manager’s standpoint, the application also
provides robust collaboration tools; such as, centralized discussion boards for all project
participants, the ability to attach documents and links to other project objects, and the
tracking of who has read each message. PRODUCT Z is the only one of the four candidate
products which could demonstrate this capability. The application can also import project
plans from other vendor products.
Conclusion
• Project collaboration tools for the PMs, and project team members
Page 6
PPM Alternatives Results & Recommendation Report
6/24/2009
Page 7
PPM Alternatives Results & Recommendation Report
6/24/2009
Appendix A
TOTAL COST OF OWNERSHIP
The following worksheets summarize the Total Cost of Ownership (TCO) for each of the PPM
solutions which were evaluated. cost should be the most heavily weighted Each vendor
submitted a TCO worksheet, except for the current vendor. (See APPENDIX E, for additional
information.) We made every effort to obtain cost information that was as accurate as
possible from vendors, as well as from DT’s relevant known internal costs. When direct
vendor cost quotations could not be obtained in a timely manner, cost estimates were
extrapolated from similar vendor sources.
Below is the summary table of TCO for the PPM applications evaluated, followed by the
detailed worksheets for each one.
Cost Summary
PPM TCO Per User Annual Cost Per TCO Per User Annual Cost Per
Application (Three Years) User (Three Years) User
Page 8
PPM Alternatives Results & Recommendation Report
6/24/2009
PRODUCT X
There were two additional factors here which proved difficult to estimate. The first was the
fact that this vendor has third-party business partners providing the possible solutions, and
the second was that most of them require MS Sharepoint to be in place. Both these factors
have the potential to boost these estimates considerably. Also, this pricing is for a 3 year EA
scenario. The per year pricing for a 5 year EA would be more economical per year.
Total Costs $ 118,406 $ 97,200 $ 97,200 $ 312,806 $ 156,800 $ 123,700 $ 123,700 $ 404,200
TCO/User (150 actual users) $ 789 $ 648 $ 648 $ 2,085 $ 1,045 $ 825 $ 825 $ 2,695
Annual Cost/User $ 695 $ 898
Page 9
PPM Alternatives Results & Recommendation Report
6/24/2009
PRODUCT W
DT is currently using PRODUCT W, and this pricing reflects revised vendor quotation based
on 150 users. Needed application customization costs are not shown, for comparison
purposes.
Total Costs $ 90,000 $ 90,000 $ 90,000 $ 270,000 $ 162,600 $ 112,800 $ 112,800 $ 388,200
TCO/User (150 actual users) $ 600 $ 600 $ 600 $ 1,800 $ 1,084 $ 752 $ 752 $ 2,588
Annual Cost/User $ 600 $ 863
Page 10
PPM Alternatives Results & Recommendation Report
6/24/2009
PRODUCT Y
This vendor does not offer their software to be hosted in-house by the customer.
Page 11
PPM Alternatives Results & Recommendation Report
6/24/2009
PRODUCT Z
Although this solution’s software source code is Commercial Open Source, there are still
costs involved.
Total Costs $ 58,300 $ 40,800 $ 40,800 $ 139,900 $ 96,650 $ 41,550 $ 41,550 $ 179,750
TCO/User (150 actual users) $ 389 $ 272 $ 272 $ 933 $ 644 $ 277 $ 277 $ 1,198
Annual Cost/User $ 311 $ 399
Page 12
PPM Alternatives Results & Recommendation Report
6/24/2009
Appendix B
SURVEY OF FUNCTIONAL EVALUATION CRITERIA
Other CCSF Departments were solicited for their input via Web Survey. Their responses
and/or participation, was strictly voluntary. The survey asked respondents to weigh the
relative importance of functional requirements for a PPM software application. The final
weight for each criterion is the average of the responses received.
Response 2 4 5 6 7 8 9 11 15 16 17 18 22 23 24 AVG
Page 13
PPM Alternatives Results & Recommendation Report
6/24/2009
Page 14
PPM Alternatives Results & Recommendation Report
6/24/2009
Page 15
PPM Alternatives Results & Recommendation Report
6/24/2009
years
Please add any comments, suggestions, or requirements we may have missed. Thanks!
The product(S) selected must add value for the PM... i.e., the PM has to want to use the tools in his/her daily
job. It can't be simply another vehicle to report information about projects that use other products to actually
manage and control.
Page 16
PPM Alternatives Results & Recommendation Report
6/24/2009
Appendix C
FUNCTIONAL EVALUATION MATRIX
The following matrix computes the weighted score for each of the PPM solutions which were
evaluated. Raw scores were determined by Joe Armenta and Jay Nath.
Criteria
Weight
CRITERIA (1-9) Raw Score 1 (poor) - 5 (excellent) Weighted Score
Total Cost of Ownership
Relatively Inexpensive
TCO 3.00 1 2 3 4 3.00 6.00 9.00 12.00
Management
Requirements
Ability to view a list of
projects for an
organizational unit with
basic information
(start/end, budget, etc)
and status indicator(s) 4.40 4 5 5 5 17.60 22.00 22.00 22.00
Page 17
PPM Alternatives Results & Recommendation Report
6/24/2009
Ability to reserve
resources (different than
assigning) 3.07 2 2 5 4 6.13 6.13 15.33 12.27
Ability to establish
milestones and phases 4.53 5 4 5 5 22.67 18.13 22.67 22.67
Page 18
PPM Alternatives Results & Recommendation Report
6/24/2009
Page 19
PPM Alternatives Results & Recommendation Report
6/24/2009
Page 20
PPM Alternatives Results & Recommendation Report
6/24/2009
Appendix D
City Services Auditor Project Management Software Solution
Evaluation Spring 2008
• Reporting
Page 21
PPM Alternatives Results & Recommendation Report
6/24/2009
• Nice ability to ‘group’ projects by priority • Cannot enter payroll categories for
for viewing. projects (dealbreaker??)--May require
integration with a separate timesheet
• Good gantt views of project progress,
program.
phase status.
• Timesheets are generated for any period
• Good high level view of projects and
designated by user-may lack sufficient
alignment with business need. controls for our purposes.
Individual Project Management - Reporting-
• Clunky interface, via web applet. It is • Customized reports are not intuitive –
functional however-similar to MS Project, difficult to use and lacks documentation.
though more difficult to use and not laid
out as well. • Out of box reports generally do not meet
our needs.
Page 22
PPM Alternatives Results & Recommendation Report
6/24/2009
Reporting-
Page 23
PPM Alternatives Results & Recommendation Report
6/24/2009
Page 24
PPM Alternatives Results & Recommendation Report
6/24/2009
• Web Resource • Web Timesheet has robust time entry • Web Resource does not allow for portfolio
and reporting capabilities—it is the planning and has very elementary project
• Web
ONLY timesheet software that will allow scheduling/assigning features- rule out
Timesheet for allocating of project hours to multiple for CSA
funding sources without
customization/work arounds. • This is primarily a payroll/time capture
oriented program and will require
• Reasonable cost ($6,000 for licenses integration with other software solution to
plus $2,000/year for support) meet CSA’s needs.
• IF we use, will need to integrate with a Portfolio Planning-
project/portfolio management solution
(i.e. Daptive or Microsoft Project). • Inadequate
Page 25
PPM Alternatives Results & Recommendation Report
6/24/2009
• TeamSchedule • Continued use would avoid resource • TeamSchedule does not allow for projects
costs associated with implementation of to be scheduled in more than one ‘plan’-
• TeamTec
a new system (i.e., need for must either create a ‘new’ project or roll
development, training, etc.). over the plan.
• If TeamSchedule is used properly, allows • Cannot assign staff to holiday hours since
for projects to be displayed as assigned. it will allow for charging to a code not
compatible with CSA payroll needs.
Reporting-
• Clumsy/inadequate assignment
• Flexible reporting with the use of Crystal functionality.
Reports-meets most user and
• Clumsy gantt views, slow functionality,
management needs.
requires constant refreshing of screen.
Calendar Views
Individual Project Management –
• Simple user views of assigned projects.
• Projects can be assigned to users,
however, cannot be broken down into
tasks – CSA needs to decide if there is a
need to track project hours and to make
assignments based on task.
Page 26
PPM Alternatives Results & Recommendation Report
6/24/2009
• Reasonably priced ($30/user, plus • Payroll report may not be adequate for
$18,000 for first year ‘quickstart’ CSA.
installation package) – includes hosting.
• Custom report creation through the use of
• Full-featured work-flow system – may pivot tables may be inadequate for CSA
allow for integration of CSA needs.
policies/procedures (e.g. training
• Tracking of staff availability for
approvals, leave requests…) into
software system. assignments is confusing/cumbersome
(grid method).
• Notification emails work with Lotus
• Heavy reliance on dashboard views.
Notes.
Reports are not as useful.
• Can track biweekly schedules (per sales
• Rude/aggressive salesmen
– need to verify)
Page 27
PPM Alternatives Results & Recommendation Report
6/24/2009
Appendix E
VENDOR QUOTE INFORMATION
The following is additional correspondence from vendors, which was submitted in addition to
the TCO Worksheets.
*******************************************
Jay,
Pricing =
…..etc., etc., etc.
Page 28