Gap Analysis

COBIT

Aris Tjahyanto. AUDIT TI. 1

IT Governance Implementation Guide (COBIT)

Feedback
Raise Analyse
Select
awareness values Post-
processes
& make and risks implement.
decision review

Identify needs
Define Define
Analyse
where you where you
gaps
are want to be

Envision the solution

Develop &
Define
Implementation projects
implement
change plan

Road Map Plan the solution

Integrate Integrate
into day-to- measures
day into ITBSC
practices

Implement the solution

Aris Tjahyanto. AUDIT TI. 2
Aris Tjahyanto. AUDIT TI. 3
 Identify Needs

Raise Analyse
awareness & Select
values and
make processes
decision risks

Identify needs

Aris Tjahyanto. AUDIT TI. 4

 Cobit Management Awareness and
Diagnostic Tools
● Two fundamental and useful tools for getting
management’s attention and raising
management’s awareness:
– IT Governance Self-Assessment
– Management’s IT Concerns Diagnostic

Raise
awareness
& make
decision

Aris Tjahyanto. AUDIT TI. 5

 IT Governance Self-Assessment
● Asks management to determine, for each of the
COBIT processes:
– how important the process is for their business objectives;
– whether the process is well performed (the combination of
importance and performance provide a strong indicator of
risk);
– who performs the process and who is accountable for the
process (and is accountability unequivocal and accepted);
– whether the process and its control is formalised, i.e., is there a
thorough contract for an outsourced activity or a clear set of
documented procedures for internal processes; and
– whether the process is audited.

Aris Tjahyanto. AUDIT TI. 6

 The first tool :
IT Governance Self-Assessment

Aris Tjahyanto. AUDIT TI. 7

 Analyse values and risks
● Governance over IT and its processes with the
business goal of adding value, while balancing
risk vs return.
● Ensuring that IT delivers the promised benefits
against the strategy, concentrating on optimising
costs and proving the intrinsic value of IT.

Analyse
values and
risks

Aris Tjahyanto. AUDIT TI. 8

 The second tool :
Management’s IT Concerns
Diagnostic
● Identifies for a number of recent and specific management
concerns in IT (e.g., interconnectivity, Client/Server,
groupware, etc.) which processes are important to be under
control to address the concerns raised.
● Technology Concerns to Management (Gartner Group) :
– Management
– Internet/Intranet
– Enterprise package solution
– Client/server architecture
– Workgroup and groupware
– Network management
Aris Tjahyanto. AUDIT TI. 9
 The second tool :
Management’s IT Concerns
Diagnostic

Aris Tjahyanto. AUDIT TI. 10

 Select Process
● Identify core process
● Identify support process
● Map it

Select
processes

Aris Tjahyanto. AUDIT TI. 11

 Exercise
● Draw the core process of your organization

Aris Tjahyanto. AUDIT TI. 12

 Exercise
● Draw the core process of your organization

Penerimaan Kuliah
PraKuliah Ujian Penyaluran
Promosi MHS (Kuri &
(Orientasi) Kelulusan Lulusan
baru Extra Kuri)

Aris Tjahyanto. AUDIT TI. 13

 How to perform gap analysis?
● Using maturity model

Select
processes

Define Define
where you where you Analyse
are want to be gaps
(AS-IS) (TO-BE)

Aris Tjahyanto. AUDIT TI. 14

 Envision the solution

Define where
Define where
you want to Analyse gaps
you are
be

Aris Tjahyanto. AUDIT TI. 15

 Envision the solution
● Gap analysis and improvement planning
● Attributes for each process :
– Awareness and communication
– Policies, plans and procedures
– Tools and automation
– Skills and expertise
– Responsibility and accountability
– Goal setting and measurement

Aris Tjahyanto. AUDIT TI. 16

Envision the solution

Aris Tjahyanto. AUDIT TI. 17

 Generic Maturity Model
Level Characteristic
0 The enterprise has not even recognized that there is an issue
Non existent to be addressed.
1 There are ad hoc approaches that tend to be applied on an
Initial/Ad hoc individual or case-by-case basis. The overall approach to
management is disorganized.
2 Processes have developed to the stage where similar
Repeatable but procedures are followed by different people undertaking the
intuitive same task.
3 Procedures have been standardized and documented, and
Defined communicated through training.
4 Management monitors and measures compliance with
Managed procedures and takes action where processes appear not to
be working effectively
5 Processes have been refined to a level of good practice,
Optimized based on the results of continuous improvement and
maturity modeling with other enterprises.
Aris Tjahyanto. AUDIT TI. 18
 Maturity Attributes
● Awareness and Communication (AC)
● Policies, Standards and Procedures (PSP)
● Tools and Automation (TA)
● Skill and Expertise (SE)
● Responsibilities and Accountabilities (RA)
● Goal Setting and Measurement (GSM)

Aris Tjahyanto. AUDIT TI. 19

Who will be interviewed?
RACI Chart

Aris Tjahyanto. AUDIT TI. 20

 RACI
● Responsible - refers to the person who must ensure that
activities are completed successfully
● Accountable - refers to the person or group who has the
authority to approve or accept the execution of an
activity
● Consulted - refers to those people whose opinions are
sought on an activity (two-way communication)
● Informed - refers to those people who are kept up to
date on the progress of an activity (one-way
communication)

Aris Tjahyanto. AUDIT TI. 21

 Contoh Umum
● Salah satu Ketua Bidang HIMA mengajukan
proposal kegiatan Lomba Mewarnai
– R?
– A?
– C?
– I?

Aris Tjahyanto. AUDIT TI. 22

 Exercise
● Develop your own maturity model for your
organization
● Using the maturity model, perform gap analysis

Aris Tjahyanto. AUDIT TI. 23

 Exercise
● Mengidentifikasi core process sebuah sistem atau
sub-sistem
● Mengembangkan maturity model untuk sebuah
proses, meliputi enam atribute : awareness &
communication, ..., goal setting

Aris Tjahyanto. AUDIT TI. 24

 The Result
● Bla bla bla AA

5
BB GG
Parameter AS-IS TO-BE
AA 2 4
BB 3 4
AS-IS
CC 2 5 0 TO-BE
DD 2 3 CC FF
EE 3 4
FF 2 3
GG 3 4
DD EE

Aris Tjahyanto. AUDIT TI. 25

 Preprocessing
● Check Reliability
– ?
● Check Validity
– ?

Aris Tjahyanto. AUDIT TI. 26

 Methodology (example)
Identifikasi Penyebaran
Permasalahan Kuesioner ke seluruhstakeholder

Tujuan
Proses awal
penelitian

Studi Group Pengujian

Pendahuluan Discussion kuesioner

Penetapan Penentuan kriteria Analisis dan

Stakeholder dan subkriteria interpretasi
Penyebaran
Kuesioner awal Simpulan

Aris Tjahyanto. AUDIT TI. 27

 How can we use the result?
● Did you collect good quality data?
– First, we should perform reliability and validity test.
– Reliability = keterandalan
– Validity = kesahihan
● How can we perform these test?
– Tools for analysis: statistic descriptive
– Calculated numbers: mean, median, mode, SD, ...
– Graphs: histograms, dotplots, boxplot, ...

Aris Tjahyanto. AUDIT TI. 28

 Reliable

Reliability
● Reliability:
– Stability
– Consistency
– Ability to Repeat and Get Similar
Results
– “Apakah jika pengukuran dilakukan
dalam kondisi yang mirip, hasilnya
akan sama? “

Aris Tjahyanto. AUDIT TI. 29

 Validity
● Validity:
– Operational definition  concept will be measured?
– Apakah pengukuran yang dilakukan benar-benar mengukur
sesuatu yang akan diukur?
● Valid:

Aris Tjahyanto. AUDIT TI. 30

 Reliability & validity ?

?????

Valid & reliable Not valid

Not reliable

Aris Tjahyanto. AUDIT TI. 31

 Reliability & validity

New rifle
Old rifle New rifle sun glare

????? Aris Tjahyanto. AUDIT TI. 32

 Why need realibility & validity test?
● The error!!
● Observed value = true value + systematic error +
random error
● Systematic error: ??
– Example:
● IQ test written in English for Indonesian.
● UTS/UAS for Biology but the material taken from Geology
– They bias measurements in a particular direction,
underestimating or overestimating the true value.
– Which affects their validity. Do not adversely affect
reliability.
Aris Tjahyanto. AUDIT TI. 33
 Why need realibility & validity test?
● Random measurement error.
– It is the result of temporary or chance factors.
– Mood of subjects and respondents.
– Momentary surveyor fatigue.
● Unsystematic
● Tend to cancel each other

Aris Tjahyanto. AUDIT TI. 34

 An important note when collecting data
● Faking good-bad
– Question for IDT's citizen, “Is
Indonesian government good?”
● Acquiescence response set
– Fake response
– Pertanyaan bagi orang yang sedang
lapar, muram dan males ngomong,
“Jatah makanmu saya ambil ya?
Diam berarti boleh saya makan.”

Aris Tjahyanto. AUDIT TI. 35

 Test-retest reliability
● Two measurement for the same object but ...
● Tools: correlation
● Rule of thumb: y 1.2

– Correlation < 0.8 bad. 0.8

– Correlation >= 0.8 good. 0.6

0.4

20
0.2
15

10

5
0
0 0.2 0.4 0.6 0.8 x
1

r = .00
0 5 10

CORREL({3,2,4,5,6},{9,7,12,15,17})
equals 0.997054 Aris Tjahyanto. AUDIT TI. 36
 Test-retest reliability
y
1.2

● Rule of thumb: 1

0.8

– Correlation < 0.8 bad. 0.6

Correlation >= 0.8 good.

0.4

–
0.2

y
0
0 0.2 0.4 0.6 0.8 x
1

x
1
0.8

0.6
r = .00
0.8 0.4

0.6 0.2

0.4 0
0 0.2 0.4 0.6 0.8 1 x
0.2

0
0 0.2 0.4 0.6 0.8 1 x r = .40
r = .81 Aris Tjahyanto. AUDIT TI. 37
 Test-retest validity
● Validity cannot be assessed
directly
● But these can help:
– Face & Content validity
● Not statistical—involves the
judgment of the researcher
● Availability of Data
● Quality of Data
● “Are the Inputs Good?”

Aris Tjahyanto. AUDIT TI. 38

 Persiapan sebelum analisis
1. Hitung rata-rata nilai ekspektasi setiap kriteria
2. Hitung prosentase setiap kriteria = bobot . Total
bobot adalah 100%
3. Kalikan bobot dengan nilai ekspektasi untuk
masing-masing kriteria. Misal nilai ekspektasi 4.22
(dari skala 1-6), bobot 15%, maka hasil 0,633
4. Jumlahkan hasil langkah (3) untuk seluruh kriteria
untuk mendapatkan indeks kepuasan stakeholder.

Aris Tjahyanto. AUDIT TI. 39

 Reliability Test
Using SPSS
● Reliable
– Apakah alat ukur (pertanyaan) bisa dipercaya?
– Apakah hasilnya relatif konsisten?
● Menu /Analyze/Scale/Reliability Analysis
– Result:
● Alpha = ?, miminum 0.7 (koefisien Cronbach)
● r table = ?
●

Aris Tjahyanto. AUDIT TI. 40

 Validity Test
Using SPSS
● Validity
– Bisakah mengukur apa yang ingin diukur?
– Apakah pertanyaan bisa dipahami oleh responden?
(Diindikasikan dengan kecilnya prosentase jawaban yang
menyimpang dari rata-rata jawaban).
● Menu
– r table <-?-> r hitung, degree of freedom n=2, alpha=5%
– valid?

Aris Tjahyanto. AUDIT TI. 41