ISA: INFORMATION SYSTEMS AUDIT COURSE 2.

Module 5
System development acquisition and maintenance

1.0 Chapter 1: Introduction to SDLC

5. Module: Systems Development: Acquisition,
Maintenance and Implementation (12%)

• Provide assurance or consulting

services that the management
practices relating to systems
Objective: development: acquisition,
maintenance and implementation
are appropriate to meet enterprise
strategy and requirements.

2
 Salient features of ISA Course 2.0
Introduction

Concepts associated with of SDLC

Typical phases of SDLC

Changes in SDLC phases

Secure SDLC

Risk associated with SDLC and mitigation planning

Roles and responsibilities of SDLC

3
Introduction

4
 Learning Objectives
Understand basic concept of Systems development life cycle (SDLC)and how it has changed over a period of time due
to changes in technology and environment, that has prompted for adoption of additional phases in SDLC.

Traditional SDLC phases and main activities

Additional phases due to availability of outsourcing and generic customizable software

Steps added in different phases due to security requirements (Secure SDLC or SSDLC)

5
What is SDLC?

6
 System Development life cycle (SDLC)

An information system
undergoes following steps:
• Conceptualization
• Feasibility
• Design and development
• Testing
• Implementation
• Use
• Retirement(replacement)

7
 Organization of chapters
Chapter 1 provides the basic information about SDLC that is required for IS auditor and project
manager involved in SDLC activities. The chapter tries to answer why it is called life cycle? What
are typical activities or phases through which development happens? These activities have
undergone changes over a period of time and hence knowledge about these changes and security
requirements in SDLC are covered.

Chapter 2 focuses on actions organizations need to take when to initiating SDLC project. This
covers initial phases of SLDC covering feasibility study, requirement definition, expected benefits
to organization and developing business case for SDLC project. Decision about execution of next
phases is based on business case and hence it necessary to understand these activities. Phase 1
feasibility study and Phase 2 requirement definition of typical SDLC phases (discussed in chapter
1) are covered in this chapter (2).

8
 Organization of chapters
Chapter 3 provides the basic knowledge on project management with focus on SDLC. Once the
organization has decided to initiate project for automated solution using SDLC, IS auditor may be
involved in reviewing project. Hence IS auditor need to have knowledge about SDLC project
management activities so as to audit the project or part of team. The contents are focused on Phase 3
and 4 for typical SDLC discussed in chapter 1. This chapter does not cover Project management in
general but focuses on practices required for executing SDLC project.

Chapter 4 introduces models and methods used for System design and development that goes
together. To ensure that final system meet business requirements, project manager and system analyst
need to decide upon development method for new application. IS auditor need to know basic
characteristics of these methods. All these methods have common objectives of application
development, with many stapes being similar.

9
 Organization of chapters
Chapter 5 discusses software acquisition and outsourcing of development. In case organization has
decided to purchase the software that means phase 3, 4 and 5 of typical SDLC is not responsibility
of organization, but the accountability of actions cannot be outsourced. This chapter describes the
process for acquisition and/or outsourcing development, in case an organization has decided to
acquire software or outsourced development and provides inputs on controls to be adopted.

Chapter 6 provides information on implementation, testing, UAT and Support

and maintenance activities covered on phases 6 to 9 respectively. Once the
application is ready to implement either developed or acquired, organizations
must implement it across the organization to derive the benefits.

10
 Organization of chapters

Chapter 7 discusses what IS auditor should know about

the new trends in technology deployment and its
impact on SDLC.

Chapter 8 provide basic and high-level information on

auditing of SDLC project.

11
 When is SDLC initiated?
When one or more of following situations arise:

• New service delivery opportunity that relates to a new or existing business

process (e.g. e-commerce)
• Issues and problems with an existing systems/business process (Complaints
from customers/users)
• Strategic focus change for an opportunity that will provide benefits to the
organization (Mergers and acquisitions, or new service delivery channels like
ATM for banks)
• New opportunity due to advancement existing technology or availability of new
technology (e.g. use of mobile technology for banking services)
• Use of automation by competitors to enhance quality of services

12
 What is SDLC?
A process of examining a business situation with the intent of
improving it through better procedures and methods.

Need to change business processes due to requirements arising out

of stakeholders expectations and/or business strategy.

To automate service delivery using technology.

System development involves:

• developing or acquiring application

• maintaining application systems

13
Phases of typical SDLC

14
 Phases of traditional SDLC

Feasibility Study
Requirement Definition/Analysis
System analysis
System Design
Development and programming
Testing
User Acceptance Testing
Implementation
Support and maintenance

15
 Feasibility Study

Studying proposed solution for:

• Technical feasibility : Technology is available and can be deployed
• Economic Feasibility: Benefits derived from proposed
automation are more than costs involved
• Social Feasibility: The solution is acceptable by stakeholders
Feasibility study helps in building business case.

16
 Feasibility Study: Audit points
Review the documentation for the reasonableness
Review cost justification/benefits should be verified along with the schedule of
when the anticipated benefits will be realized.
Identify if the business need used to justify the system actually exists or not

Justification for going for a development or acquisition.

Review the alternate solutions for reasonableness.

Review the reasonableness of the chosen solution.

17
 Requirement definition
Requirements : expectations from new application or solution
Functional/Compliance
Service level requirements Quality requirements Technical requirements
requirements

Stakeholders must be involved in determining requirements

Analyst help stakeholders in granular definition of requirements.

18
 Requirement definition: Audit points

Identify the affected users and the key team members on the project to verify
that they are having an appropriate representation.

IS Auditor should review detailed requirements definition document and verify

that its accuracy and completeness through interviews with the affected and
requested user departments.

Review the existing data flow diagrams and other related specifications like
transforms, data descriptions, etc., to ensure that they cover the user needs

19
 System analysis
Understanding and analyzing requirements to define problem
Existing process Expectations and gaps

Identify gaps in requirements and collect them

Nature of proposed solution:

Data Oriented (traditional MIS) Service oriented (Customer focus)

20
 System Analysis: Audit points

Verify that the management has approved the initiation of the project and the
cost.

In case of acquisition, determine that an appropriate number of vendors have

been given proposals to cover the true scope of the project and requirements of
the users.

Determine whether the application is appropriate for the user of an embedded

audit routine & if so request may be made to incorporate the routine in
conceptual design of the system.
21
 System Design
Finalize the requirement and scope baseline

Multiple user interactions to confirm requirements

Prepare new system design covering:

• parts of the system
• how they interface
• how the system need to be implemented
• type of hardware, operating system and other software
• network facilities
• program and database specifications
• security considerations

22
 System Design: Audit points .. 1
Review the system flowcharts for adherence to the general design

Review the input , processing and output controls designed into the system are appropriate

Assess the adequacy of the audit trails which provide traceability and accountability

Verify the key calculations and processes for correctness and completeness

Interview the users to ascertain their level understanding of the system design, input to the
system , screen formats and output reports
23
 System Design: Audit points .. 2
Verify that the system can identify erroneous data correctly and handles it invalid transaction

Review the conceptual design to ensure the existence of appropriate controls

Review the Quality Assurance & Quality Control results of the programs developed during
the phase.

Verify that the design for its completeness and correctness and meets the requirements.

Verify that functional data created during requirement phase is complete and test plan has
been developed during this phase.

24
 Development and Coding
Prepare Work breakdown for coding by team to transform design into
code/program/function/object for coding depending upon required functionalities and
hosting platform

Decide the language

Work cohesively to ensure interface are considered

The phase is iterative with testing phase

Developers must follow organization’s coding standard

25
Development and Coding
Reliable

Robust

Accurate
A good program must
be:
Efficient

Usable

Readable

26
 Development: Audit points

IS Auditor should ensure that documentation is

complete

IS auditor should review the QA report on adopting

coding standards by developers

Review the testing and bugs found report sent for

rework to developers
27
 Testing
Quality assurance testing(QAT) generally consist of unit testing, interface testing, integration
testing, peer reviews etc.

User acceptance testing(UAT) also known as final acceptance testing (Is defined as separate phase
due to formalization of process

Intermittent testing:

• Unit testing
• Interface testing
• Integration testing
• System testing
• Regression testing
28
 Testing : Audit points … 1
Review the test plan for completeness and correctness

Review that user participation was there during testing phase

Review error reports for their precision in recognizing erroneous data & for resolution of errors.

Verify cyclical processes for correctness( example: year -end process, quarter-end process)

Interview the end-users of the system for their understanding of the new methods, procedures and
operating instructions.

Review the system and end-users documentation to determine its completeness and correctness.
29
 Testing : Audit points … 2
Reconciliation of control totals and converted data should be performed to check for the integrity of the
data after conversion

Review all parallel testing results.

Test the system randomly for correctness.

Review unit test plans and system test plans to determine that tests for internal control are addressed

Verify that the system security is functioning as designed by developing and executing access tests.

Ensure test plans and rest results are maintained for reference and audit
30
 UAT

User acceptance or Final testing

• Formal sign-off is expected hence it is one of the major milestone for
application development/acquisition project.
• UAT should be performed in a secure testing or staging environment
Organizations may consider certification and
accreditation of application as supporting for UAT.
31
 UAT

UAT supports the process of ensuring that the system is

production-ready and satisfies all documented
requirements. The methods include:

Utilization of
Definition of Design of test
Execution of the results to
test strategies cases and
the tests verify system
and procedures scenarios
readiness

Note: It is expected that the test plan, data and results are to be maintained
for subsequent audits. 32
 UAT : Audit points

Determine that the formal acceptance has been

signed by project sponsor or suitable authority
from user group.

Ensure test plans, test data and test results are

maintained for reference and audit
33
 Implementation
Process to roll out application and related processes in production.

Preparation start much earlier in SDLC phase, typically after

system design in complete.

Depending upon nature of organization, type of application

implementation strategy need to defined.
• Cut-off
• Phased
• Pilot
• Parallel

34
 Implementation : Audit points
Determine that the formal acceptance has been signed by the Project development
team, user management, Quality Assurance, Security or Audit officer.

Verify that the system has been installed according to the organization’s change
control procedures

Review the programmed procedure used for scheduling and running the system
along with the system parameters used in executing the production schedule.

35
 Implementation : Audit points

Review all system documentation to ensure its completeness

and all recent updates from the Testing phase have been
incorporated.

Verify all conversion of data to ensure that they are correct and
complete before implementing the system and user sign-off is
obtained in this regard.
36
 Support and Operations

A defined and formal organizational

operations process
Update project
Establish, record, Assesses the management process
Provide support and
review and adequacy of the based on lessons
assistance to users in
implement system and learned and
smooth operations
deficiencies and projected ROI recommendations
and end-user
future changes measurements as per for future projects
management
required business case regarding system
development.
37
 Post Implementation: Audit points
Determine that the systems objective requirements were achieved

Determine if the cost benefits identified in the feasibility study are being measured.

Review the program change requests performed to assess the type of changes required of the system

Review the controls built into the system to ensure that they are operating to design.

Review operator’s error logs to determine if there are any resource or operating problems inherent with the
system. Logs may indicate the inappropriate planning or testing of the system prior to implementation.

Review input and output control balances and reports to verify that the system is processing data accurately.

38
Support and maintenance : Audit points … 1
Evaluate the adequacy of the organization’s procedures for authorizing,
prioritizing and tracking system changes.

Identify system changes and verify that appropriate authorization was

given to make the change in accordance with organizational standards.

Review permanent program documentation to ensure that evidence

(audit trail) is retained regarding program changes.

Evaluate the adequacy of the organization’s procedures regarding

program change control.
39
Support and maintenance : Audit points … 2
Evaluate the adequacy of the security access restrictions over production source and
executable modules.

Evaluate the adequacy of the organization’s procedures for dealing “emergency” program
changes.

Evaluate the adequacy of the security access restrictions over the use of the “emergency”
logon-ids.

Verify the existence and adequacy of the records for system changes.

Evaluate the adequacy of the access protection of the maintenance records.

40
Changes in SDLC

41
 Revised SDLC phases
 Options of product availability and Outsourcing of
development has introduced changes in SDLC phases.
 Based on Decision after feasibility study, organization
selects one of the three path
SDLC Phase Outsourcing Product
Acquisition
Feasibility Study
Requirement Definition
Analysis Vendor selection Product selection
Design Requirement baseline RFP
Development Agreement and SLA Procurement and
contract
Testing Vendor follows SDLC Configuration
UAT
Implementation
Support and Maintenance 42
 Make or Buy
Feasibility study takes into consideration ‘what is being done
in industry?’ Based on this information organization may
choose one of three options:
• Develop software using in-house resources
• Hire a third party vendor to develop the software (i.e.
outsource software development)
• Purchase customizable software that is available in market.

43
 Make or Buy
Important Points:

• Although organization is not developing software it is better to follow SDLC

model since final acceptance and testing are critical for any application.
• In case organizations miss out any of the steps, impact will be higher.
• For example, if organization misses out some requirements, vendor may not
include them in application. Making changes after implementation shall be
costly.
• Testing, if not performed meticulously, software may be implemented with
bugs.

44
 Selection of Vendor / product

Product
Outsourcing
acquisition
(Phase 3B)
(Phase 3C)

45
 Outsourcing (Phase 3B)

Organization may hire a third party firm having necessary skills and
experience to develop software specifically for organization’s use.
The third party shall follow SDLC process for development

Organizations may follow normal process for vendor selection.

46
 Product acquisition (Phase 3C)
Identify various products

Prepare functional gap analysis table

Evaluate Support, learning, acceptance and implementation costs and efforts

• Required functionality is not available but can be configured
• Required functionality cannot be configured but work around is possible or do not have
impact
• Additional functionality are available that were not considered in requirements

Select product

47
 Requirement Finalization and RFP
Outsourcing (phase 4B)

Vendor may need to baseline the requirements to estimate the effort for arriving at commercials of proposal

Product acquisition (Phase 4C)

Based on outcome of earlier gap analysis request for proposal is sent to product vendors/suppliers

The proposal must contain requirements for Support and maintenance.

48
 Contract, SLA and Purchase
• Defining terms of contract for application development
Outsourcing • Defining terms for service levels for delivery, support
(Phase 5B) and maintenance
• Sign-off

Product
• Issuing purchase order based on proposal and accepted
acquisition terms of contract
(Phase 5C)
49
 Development and Configuration
Outsourcing (Phase 6B) Product acquisition (Phase 6C)

• Vendor follows the development • Purchased product need to be

phases: Design, development, testing configured as per requirements of
• It is necessary to monitor the organization
progress to avoid surprises. The delay
in development might delay the
entire project.

Outsourcing development or purchasing and configuring software

might reduce the software development efforts, but introduces vendor
management and monitoring efforts and also risks associated with
outsourcing.
50
Secure SDLC

51
 What is Secure SDLC?
Embedding security in application development is referred as SSDLC or Secure SDLC

While developing application primary focus is on business requirements or functional

requirements. It might result in ignoring non-functional requirements particularly required for
Information security.

Building security in Application is better than providing security as after thought.

Trends are showing that application vulnerabilities are targeted by attackers.

52
 Additional tasks in SDLC for security

S ecure SDLC Phase

Requirement Development Implementation

Design Phase Testing Phase
Definition Phase Phase

53
 Requirement Definition

Security Steps
• Identify security requirements including compliance
for privacy and data loss
• Determine risks associated with security and prepare
mitigation plan
• Train users on identification and fixing of security bugs
54
 Design Phase

Security Steps

• Ensure security requirements are considered

during design phase
• Identify possible attacks and design controls e.g.
implementing least privilege principle for sensitive
data, and apply layered principle for modules

55
 Development Phase

Security Steps
• Develop and implement security coding
practices(e.g. input data validation, simple
coding)
• Train developers on security coding practices
56
 Testing Phase

Security Steps

• Review code for compliance of secure coding practices

• Develop test cases for security requirement testing
• Ensure security requirements are tested during testing
• Test application for identified attacks

57
 Implementation Phase

Security Steps

• Analyze all functions and interfaces are

secured
• Perform security scan of application after
implementation
58
SDLC: Risk Management

59
 SDLC: Risk management

Identify risks based on business requirements

Discovering methods to respond to risks (accept, avoid, mitigate,

transfer)

Accepting residual risk and going ahead with the project

60
 SDLC: Risk management
Typical Risk

• Project risks: Cost, Time and quality overrun

• Inaccurate requirements definition
• Inappropriate selection of platform (hardware, operating system,
database etc.)
• Compromise on quality and testing resulting in bugs after
implementation
• Missing or incomplete documentation
• Absence of skilled resources
61
 SDLC: Risk management
Typical Risk

• Scope creep (changing requirements)

• Poor coding techniques
• Inadequate documentation
• Inadequate QC and QA (including testing inadequacies)
• Lack of proper change control and controls over promotion into
production
• Inappropriate processes for development
• Technical vulnerabilities, and how these could materialize and be
controlled/addressed

62
 Mitigation
Most risks can be mitigated by implementing best practices / standards for:
Suitable methodology
Project management Coding practices Automated tools And so on….
(Like Prototyping)

In order to mitigate risks in time, it is best to perform risk assessment during each
phase of SDLC.

In case of outsourcing and/or purchased software the risk associated with

outsourcing and vendor management need to be managed by the organization.

63
 Examples of mitigation .. 1
Risk Mitigation plan
Inaccurate  Select prototyping model to finalize the requirements
requirements  Meet various stake holders to confirm identified requirements
definition
Inappropriate  Understand technical and performance requirements (e.g. user
selection of platform response time, number of transactions per second etc.) and determine
(hardware, operating technical specification of proposed platform
system, database
 In case platform specifications are available (existing hardware); ensure
etc.)
that application development can address requirements.
 Understand organization baseline for infrastructure and incorporate in
design

64
 Examples of mitigation .. 2
Risk Mitigation plan
Compromise on  Ensure standard coding practices are adopted
quality and testing  Provide enough time for building test cases to cover all function,
resulting in bugs performance and security requirements
after implementation
 Build test cases along with design
Missing or  Ensure completion of documentation along with design and
incomplete development. Standardization of coding practice helps in this process
documentation  Ensure documentation experts and technical writers are part of team
Absence of skilled  Consider outsourcing or hiring skilled resources on contract
resources

65
 Examples of mitigation .. 3
Risk Mitigation plan
Scope creep (changing  Perform scope base lining
requirements)  Introduce change management process to evaluate and adopt changes in
requirements
Poor coding  Develop and implement standard coding practices
techniques
Inadequate  Ensure technical writers and document specialist are part of team
documentation
Inadequate QC and  Plan QA process along with project and development plan
QA (including testing  Implement standard coding practices
inadequacies)  Implement performance monitoring process

66
 Examples of mitigation .. 4
Risk Mitigation plan
Lack of proper change  Perform scope base lining
control and controls  Introduce change management process to evaluate and adopt changes in
over promotion into requirements
production
Inappropriate  Understand requirements and select appropriate development method
processes for suitable for identified requirements.
development
Technical  Ensure integration and system testing are performed on target platforms
vulnerabilities, and
how these could
materialize and be
controlled/addressed

67
SDLC: Roles and responsibilities

68
SDLC: Roles and Responsibilities … 1
Steering Committee

Program Manager

Project Manager

69
 Steering Committee

A steering committee is set up to approve,

supervise and direct IT projects, including
SDLC. This committee provides overall
direction and monitors progress of all IT
projects including SDLC projects.
70
 Program Manager

Responsible for all projects that are

associated with large IT related
program. E.g. Implementation of
ERP across many locations of an
organization. A program consists of
multiple projects.
71
 Project Manager

A project manager is normally responsible for

more than one project and liaisons with the
client or the affected functions. This is a
middle management function, Project
manager is responsible for delivery of the
project within the time and budget.
72
SDLC: Roles and Responsibilities … 2

Systems Analyst

Module Leader/Team
Leader

Programmers

Database Administrator
(DBA)

73
 Systems Analyst
The system analyst also has a
responsibility to understand existing
problem/system/data flow and new
requirements. System analysts convert
the user’s requirements in the system
requirements in order to design new
system.
74
 Module Leader/Team Leader

A project is divided into several

manageable modules, and the
development responsibility for each
module is assigned to module leaders.

75
 Programmers

Programmers convert design into programs

by coding using programming language.
They are also referred to as coders or
developers.

76
 Database Administrator (DBA)

The data in a database environment has to be

maintained by a specialist in database
administration so as to support the application
program. The database administrator handles
multiple projects; and ensures the integrity and
security of information stored in the database.
77
SDLC: Roles and Responsibilities … 3
Quality
Assurance Testers
Team

Domain
Specialist

78
 Quality Assurance Team

This team checks compliance with the

SDLC related standards (Documentation,
Coding standards, Security requirements
etc.) set by the organization, by project
teams on a periodic basis.
79
 Testers

Testers are junior level quality assurance personnel

attached to a project. They test programs and
subprograms as per the plan given by the module /
project leaders and prepare test reports.

80
 Domain Specialist

Whenever a project team has to

develop an application in a field that's
new to them, they take the help of a
domain specialist, who understands the
business requirements and helps system
analyst, designer and programmer in
understanding the requirements.
81
SDLC: Roles and Responsibilities … 4

Technology Specialist

Documentation Specialist

IS Auditor
82
 Technology Specialist

IT is developing so rapidly that even IT

professionals find it difficult to keep track of
all developments, let alone develop expertise.
This has resulted in experts in specific
technology areas, such as Microsoft tech-
nology, Web-enablement and the like.
83
Documentation Specialist

These professionals are

responsible for the creation
of user manuals and other
documentation.
84
IS Auditor

As a member of the
team, IS auditor can
ensure that controls
required for process for
which application is
being developed are
included in
requirements.

85
Summary

86
 What we learned?
What is SDLC?

Typical Phases is SDLC and changes

Development Outsourcing Acquisition

Secure SDLC

Risk management in SDLC

Roles and responsibilities in SDLC

87
Questions

88
1. System development life cycle (SDLC)
primarily refers to the process of:
A. Developing IT based solution to improve business service delivery.
B. Acquiring upgraded version of hardware for existing applications.
C. Redesigning network infrastructure as per service provider’s needs.
D. Understanding expectations of business managers from technology.

Answer: A
SDLC primarily focuses on identifying IT based solution to improve
business processes delivering services to customers. Other activities may
be part of SDLC however, these are IT projects not SDLC projects.

89
2. Organizations should adopt programming/coding
standards mainly because, it:
A. is a requirement for programming using high level languages.
B. helps in maintaining and updating system documentation.
C. is required for security and quality assurance function of SDLC.
D. has been globally accepted practice by large organizations.

Answer: C
Adopting coding standards helps organization in ensuring quality of coding
and in minimizing the errors. It also helps in reducing obvious errors which
may lead to vulnerabilities in application. A is not true since it is required for
all languages; B is partially true but is not the main reason. D is not main
reason.

90
3. Which of the following is main reason to
perform User acceptance testing (UAT)?
A. To train and educate users on features of new solution.
B. To confirm form users that solution meets requirements.
C. To complete formality of sign-off to mark end of project.
D. To finalize the implementation plan for new IT solution.

Answer: B
UAT is mainly conducted to confirm from the users and application owners
that application meets their requirements. Sign-off is a formality to be
completed only if requirements are met. Training and implementation
planning are different activities which are not dependent on UAT.

91
4. An organization decided to purchase a configurable application
product instead of developing in-house. The outcome of which of the
following SDLC phase helped organization in this decision?
A. Requirement definition
B. Feasibility Study
C. System analysis
D. Development phase

Answer: B
Make or buy decision is the outcome of feasibility study where
technical, economical and social feasibilities are considered.

92
5. In which of the following phases of SDLC,
controls for security must be considered FIRST?
A. Requirement definition
B. Feasibility study
C. System design
D. Implementation

Answer: C
Security requirements must be considered during requirement definition.
However, the nature of controls to be implemented for security must be
considered first during design phase. This will ensure that necessary security
controls are built while developing application. Security controls are
implemented and not designed during implementation phase.

93
6. IS auditor has been part of SDLC project team. Which of the
following situation does not prevent IS auditor from performing
post implementation review? The IS Auditor has:
A. designed the security controls.
B. implemented security controls.
C. selected security controls.
D. developed integrated test facility.

Answer: D
Active role of IS auditor in design and development of controls affects the
independence. Hence, IS auditor cannot perform review or audit of the
application system. However, developing integrated test facility within
application is not a control, but a facility to be used by auditors in future.
Hence, this does not impact independence of IS auditor

94
 Additional References
Please refer to the following documents for more information:
 Highlights of ISA Course 2.0
 Body of Knowledge of ISA course 2.0
 Frequently asked Questions on ISA Course 2.0
 ISA Course 2.0 DVD
 www.cit.icai.in
 www.icai.org
 www.isaca.org

95
 Thank you!
Questions?

Email: cit@icai.in

96