How to Audit the

System Development Life Cycle

William J. Papanikolas, CISA, CFSA
Western Michigan Chapter
ISACA
October 20, 2011
1

Todays Agenda
Defining SDLC

How to Impact the SDLC Process

What to Audit at Each Step

Defining SDLC
The System Development Life Cycle (SDLC) is
the entire systems process from identifying a
need through the final implementation of a
solution.
SDLC is one of the best places for an auditor
- and yet one of the least audited.

Defining SDLC
Successful SDLC projects are measured three
ways:

Creating a Quality Product

Completing at Budgeted Cost
Completing on Approved Timetable
NOTE: The majority of SDLC projects fail to
achieve even two of these goals!
4

Defining SDLC
Project
Initiation

Business
Requirements
Definition

Technical
Requirements
Definition

Software
Selection /
Coding

Testing

Data
Conversion

Training and
Documentation

Final
Implementation

How to Impact the SDLC Process

Creating a Quality Product

Provide assurance the final product is going to

deliver what has been promised
Ensure proper controls (automated and manual)
have been designed into the new process

Completing at Budgeted Cost

Ensure SDLC oversight includes controls for costs

Completing on Approved Timetable

Ensure SDLC oversight includes controls for

timeliness
6

Project Initiation
Project champion determined.
Project charter developed.
High level timelines and budgets determined.
Project team assigned; roles and
responsibilities established.
Project monitoring and accounting set up.

Project Initiation - Audit Ideas

Quality Product
Appropriate stakeholders are involved.
Project champion represents the key stakeholders.
Project is consistent with the organizations strategic
plans.

Project Initiation - Audit Ideas

On Time and On Budget
Budget was properly determined (watch out for
approval cutoffs!).
Timeline is realistic given project magnitude and past
organizational experience.
Appropriate metrics and reporting schemes are
developed.

Business Requirements Definition

Primary system functions defined.
Usability targets established (e.g., 24x7, 1
second response time).
Management reporting requirements
understood.
Regulatory and legal implications considered.
End user screen requirements determined.

10

Business Requirements Definition - Audit Ideas

Quality Product
Appropriate stakeholders are represented.
Security requirements are defined.
Automated and manual controls are considered.

11

Business Requirements Definition - Audit Ideas

On Time and On Budget

Project plan and budget remain realistic given
business requirements.
Business requirements do not overly rely on new
and/or unproven technologies (e.g., a requirement
that all transactions will process over the intranet).

12

Technical Requirements Definition

Processing platform(s) determined
Necessary hardware acquisitions outlined.
System capacity requirements understood
(both processing speed and data storage).
Network modifications defined.
Data structures created.

13

Technical Requirements Definition - Audit Ideas

Quality Product
Technical requirements support the business
requirements.
Members of all impacted technical units represented.
Technology assumptions are properly validated
through internal experience or external site visits.
Links to existing applications are defined and
controlled (e.g., control totals)

14

Technical Requirements Definition - Audit Ideas

On Time and On Budget

Project plan and budget remain realistic given
technical requirements.
Lead times for purchasing, receiving, installing and
testing new hardware have been properly reflected in
the timeline.

15

Software Selection/Coding
Request for Proposal created.
Vendor and software selection criteria
established.
Contract terms established.
Programming teams assigned for coding and
modification.
Software loaded in test environment.

16

Software Selection/Coding Audit Ideas

Quality Product
RFP and vendor assessments come directly from
business and technical requirements.
Selected vendor has experience in your industry, with
companies your size, and with similar setups.
Vendor is financially stable and will be around for
long term support (alternatively, the source code
could be owned by your organization).
Proper change management and security controls are
set up for the coding environment.

17

Software Selection/Coding Audit Ideas

On Time and On Budget
Vendor contract terms are favorable, and include
clauses on cost overruns.
Vendor contract includes rewards/penalties for
project timeliness.
Project plan appropriately reflects the resources and
time necessary to install, code and modify.

18

Testing
Unit testing completed for each system
element.
Integrated testing completed for each system
module.
System testing completed for overall system
and related interfaces.
Stress testing completed for online
performance and data storage/retrieval.
End user testing completed.
19

Testing - Audit Ideas

Quality Product
All testing is performed in an appropriate
environment with adequate security.
All issues noted during testing are communicated to
the proper owner within the project.
Test cases reasonably reflect the environment as it
will appear in production.
Change management controls are in place as system
elements progress through the testing cycle.

20

Testing - Audit Ideas

On Time and On Budget
Resolution of test issues is focused on items that are
necessary to achieve business or technical
requirements (not all issues must be solved prior to
going live!).
Project plans are properly updated to reflect issues
noted in testing that must be resolved.

21

Data Conversion
Data from the old system(s) is properly
cleansed prior to conversion.
Converted data is evaluated to ensure it is
accurate and complete.

22

Data Conversion - Audit Ideas

Quality Product
Data is accurately mapped from the old system to the
new.
Key data elements are screened using software (or
manually in some cases) to ensure anomalies are
removed.
After conversion, sample data reflects accurate
transfer.
Control totals of key data fields/tables show
consistency in the old and new data structure.
23

Data Conversion - Audit Ideas

On Time and On Budget
Project plans are properly updated to reflect issues
noted in data conversion that must be resolved.

24

Training and Documentation

Users, operators, database administrators,
management, etc. receive the training
required to operate and use the system.
Documentation is provided for all users and
operators of the system.

25

Training and Documentation Audit Ideas

Quality Product
Training addresses both system usage and business
process.
Training includes all affected parties.
Training is provided close enough to implementation
to allow participants best retention.
Documentation (online and paper) is organized in a
way to be useful to users and operators.

26

Training and Documentation Audit Ideas

On Time and On Budget

Training and documentation are properly included in
the project plan and budget.

27

Final Implementation
Final system running in the production
environment.
New hardware, networking, etc. comes
online.
Business processes change over to
accommodate new system.

28

Final Implementation - Audit Ideas

Quality Product
Promotion to production environment follows
established change management procedures.
Parallel processing with old system(s) commences.
Help desk and swat teams are in place.
System backout procedures are established.

29

Final Implementation - Audit Ideas

On Time and On Budget
Final costs are captured and summarized (watch out
for implementation problems being defined as ongoing maintenance).
Project teams are closed down as the implementation
continues.

30

Whats Next?
Post-Implementation Review
Lessons Learned
Final Reporting

31

Questions?