11 05 Metricas PDF

Advanced Research Project on Software Metrics by the Ministry of Economy, Trade and Industry, Japan (METI)
A Set of Metrics for Information Systems/Software Product Quality in Japan
"A Set of Metrics for Information Systems/Software Product Quality in Japan" is a compilation of quality metrics
summarized based on the survey of quality metrics in the Japanese industiries under the Advanced Research Project on
Software Metrics by the Ministry of Economy, Trade and Industry, Japan (METI).
1. Usage
This is used in order to define and evaluate the quality of the information system/software products.
It is not necessary to use all metrics, but select and use those suitable for the contexts of the information
system/software products to be realized.
Quality Requirement
[1] Specify a stakeholder, clarify the needs, risks and issues required for the usage scenario together with the level
of importance and select quality are to be realized.
[2] Based on the list , select pertinent quality characteristics from the standard quality model and create a "quality
model for the target system".
[3] Select metrics pertinent to the "quality model for the target system" from a set of Metrics for Information
Systems/Software Product Quality in Japan and define the quality requirements concretely and quantitatively.
Quality Evaluation
Measure deliverables using the metrics determined in the quality definition and evaluate the degree of achievement.
2. Contents of a Set of Metrics
A Set of Metrics consists of a total of 9 sheets with 8 sheets of product quality characteristics and 1 sheet of quality
in use characteristics.
No. of metrics for product quality model and quality in use model(Total No. of metrics:173 metrics)
No. of metrics for product quality model (total 8 sheets) No. of metrics for Quality in use
model (1 sheet)
Functional suitability 11 items Reliability 31 items Effectiveness
Performance efficiency 22 items Security 22 items Efficiency
Compatibility 4 items Maintainability 19 items Satisfaction 27 items
Usability 22 items Portability 15 items Freedom from risk
Context coverage
*Quality characteristics in use is 1 sheet.
Quality Characteristics
Quality characteristics are quoted from the following international standard.
ISO/IEC 25010
Systems and software engineering - Systems and software Quality Requirements and
Evaluation (SQuaRE) - System and software quality
1/29
3. Items of Metrics Table
Items and meanings of Metrics Table

Items Meaning
Number allocated to each metrics (XX-xx)
(a) Metrics No. XX is the abbreviated name indicating the quality characteristics and xx is the number
in the pertinent quality characteristics
(b) Sub quality Sub characteristics classified into the quality characteristics
characteristics
Name of metrics
(c) Name of metrics
Indicating the meaning of metrics, measurement objectives and evaluation method.
(d) Explanation of metrics However, the measurement objectives and evaluation methods are not limited to those
indicated here.
Indicating the definition example of metrics. "Quote" indicates metrics from the source
(e) Reference definition reference. "Reference" indicates metrics that are partially corrected in the source
reference (*Note 1).
Indicating the degree to which "measures of internal quality up to the unit test" are
(f) Up to recommended for use, ++ is strongly recommended and + is recommended.
Recommende unit test
d degree to
(g) After Indicating the degree to which "measures of external quality after the combined test"
be used
combined are recommended for use, xx is strongly recommended and x is recommended.
test
(h) Ratio of "Currently in use" replies in "Quality metrics utilization status research (*Note
Currently in 2)" (%) X = A/B, A = Number of "Currently in use" replies , B = Number of replies (19
use organizations)
Usage result
(i) Usage is Ratio of replies of "Not currently in use but thinking better to use" in "Quality metrics
recommend utilization status research (*Note 2)" (%) Y=C/(B-A), C = Number of replies of "Not
ed currently in use but thinking better to use"
*Note 1 Source reference of the metrics set reference definition

TS X 0111-2: Quality of software products - Part 2: External measurement method by JIS X
ISO/IEC 9126-2 0129-1 Software engineering-Product quality-Part 2: External metrics, Japanese Standards
Association
TS X 0111-: Quality of software products - Part 3 Internal measurement method by JIS X
ISO/IEC 9126-3 0129-1 Software engineering-Product quality-Part 3: Internal metrics, Japanese Standards
Association
TS X 0111-4: Quality of software products - Part 4 Quality measurement method in use by
ISO/IEC 9126-4 JIS X 0129-1 Software engineering-Product quality-Part 4: Quality in use metrics, Japanese
Standards Association
Nonfunctional requirement grade List of items related to nonfunctional requirement of the
Non functional system infrastructure, Information-Technology Promotion Agency, Japan, Software
Engineering Center
Critical infrastructure information system reliability council report, Information-Technology
Critical Infrastructure Promotion Agency, Japan Software Engineering Center
Embedded system development management guide: ESQR, Information-Technology

ESQR Promotion Agency, Japan Software Engineering Center
User Vender Collaboration Research Project II Report "Nonfunctional Requirement

Specification Definition Guideline", Ministry of Economy, Trade and Industry - Information
JUAS Service Industry Div., NTT Data Institute of Management Consulting, Inca, Japan Users
Association of Information Systems
SLA Guideline for IT systems for the private sector 3rd edition, Japan Electronics and
JEITA Information Technology Industries Association, Solution Service Business Committee
2/29
*Note 2 Quality metrics utilization example research

This research was conducted targeting 19 organizations related to information system/software industry in order to
clarify the utilization status of each metric.
Example of Metrics Table
Recommend
ed degree to Usage result
(b) Quality
(a) be used
sub (c) Name of (e) Reference
Items Metric (d) Explanation of metrics (g) (i) Usage
characteristic metrics definition (f) Up
s No. After (h) In is
s to unit
combi use recomme
test
nation nded
Example Fu-1 Functional Completenes Degree to which the functions X = 1 - A/B
completeness s of functional stated in the requirement spec. are A = No. of
implementatio correctly implemented with no missing
n omissions functions
detected
Evaluating whether the during
implemented functions completely evaluation
comply with the requirement spec. B = No. of
++ ++ 42.1% 45.5%
functions
E.g.) Comparing the number of stated in the
missing functions detected during requirement
evaluation and the number of spec.
functions stated in the requirement
spec. [Quote]
ISO/IEC9126-
3
4. Terms
In this quality metrics set, the terms are used in the following manner.
Meaning of ratio / density / degree
[Ratio]
Indicating a division of measure with the same unit
E.g.:
Metrics Completeness of functional implementation
Explanation Degree of how the functions stated in the requirement spec. are completely implemented without missing
Reference X = 1 - A/B
definition A = No. of missing functions detected during evaluation, B = No. of functions stated in the requirement spec.
[Density]
Indicating a division of measure with different units
E.g.:
Metrics Review indication density
Explanation No. of defects detected during review against unit size of software (1KLOC, etc.)
[Degree]
Indicating the degree of general objects (including ordinal scale and nominal scale)
E.g.:
Metrics Degree of skill to switch to a backup machine
Explanation How much is learnt through daily training performed in order to switch to a backup machine or recover easily
Meaning of defect / fault

[Defect]
Indicating the cause of nonconformity occurring in the development process
[Fault]
Indicating the phenomenon of nonconformity occurring after release
Note: The word "bug" is not used.
3/29
Metrics relative to Functional suitability
Definition of functional suitability:

Degree to which a product or system provides functions that meet the stated and implied needs when used under specific conditions
Recommende Usage result

Usage
After
Metric Up to Curren is
Sub characteristics Metrics Explanation of metrics Reference definition combi
s No. unit tly in recom
ned
test use mende
test
d
Functional Degree to which the set of functions covers all the specific tasks and user objectives
completeness
Functional Completeness of Ratio of the number of functions correctly X=1-A/B
completeness functional implemented with no omissions against the A= Number of functions in which problems are
implementation number of functions stated in the requirement detected in evaluation
spec. B= Number of functions checked
[Quote] ISO/IEC9126-3
Evaluating the presence of omitted functions in
order to check whether the implemented
Fu-1 functions completely comply with the ++ ++ 42.1% 45.5%
requirement spec.
E.g.) Comparing the number of incomplete

functions (functions containing defects or
omitted functions) detected during evaluation
and the number of functions stated in the
requirement spec.
Functional Degree to which a product or system provides the correct results with the needed degree of precision
Functional Correctness Ratio of the number of items complying with the X=A/B
correctness specific correctness against the number of items A= Number of data items implemented with
required to have specific correctness (within the specific levels of precision, confirmed in
regulated allowable range) as the requirement evaluation
spec. B= Number of data items that require specific
levels of precision
Fu-2 Correctness is a criteria of reproducibility, and [Quote] ISO/IEC9126-3 ++ ++ 21.1% 33.3%
the size of the random error of the calculation
under the same condition is evaluated. There is
the degree of correctness as a related concept,
however, this indicates the closeness of
calculation value against the true value.
Functional Correctness of Ratio of the number of items for which correct X=A/B
correctness calculation calculation is obtained against the number of A= Number of functions in which specific
calculation items stated in the spec. accuracy requirements had been implemented,
as confirmed in evaluation.
Evaluating the items for which the calculation B= Number of functions for which specific
complying with the stated spec. including accuracy requirements need to be
rounding processing is obtained. implemented.
[Reference] ISO/IEC9126-3
E.g.) Comparing the number of accurate
calculation items that are checked during Or
Fu-3 + ++ 15.8% 31.3%
evaluation and the number of calculation items X = A/B
stated in the specification. A = No. of correct calculation result items that
are checked during evaluation
Target examples) B = No. of calculation items stated in the spec.
Financial systems for which all calculation
should be correct
Medical inspection systems that should detect
abnormal values exhaustively.
Functional Processing margin Ratio of the actual turn around time against the Margin rate = 1 - measured turn around time /
correctness rate required turn around time required turn around time
Evaluating how much margin for processing

there is at normal, peak or degenerate
operation.
Fu-4 ++ 15.8% 37.5%
When the margin rate is close to 1, excess
performance may be highly possible.
If the margin rate is small, improve the
performance including hardware.
Functional Functional Ratio of the number of test items for which Ratio of the number of data for which results are
correctness correctness towards rationally expected results are actually obtained consistent with the known actual value (expected
expectation against total number of test items. value), when proven data from the past (multiple
Accuracy and correctness of calculation are number) is input as sample data at
comprehensively evaluated. comprehensive evaluation such as the total test
E.g.) When a test is performed and the result
Fu-5 ++ 15.8% 31.3%
differs unacceptably from the appropriately
expected result, the number of results is
compared.
4/29
Recommende Usage result

Usage
After
Metric Up to Curren is
Sub characteristics Metrics Explanation of metrics Reference definition combi
s No. unit tly in recom
ned
test use mende
test
d
Functional Degree to which the functions facilitate the accomplishment of specific tasks and objectives
appropriateness
Functional Specification change Ratio of changed specifications against overall Total number of changed specifications / total
appropriateness rate specifications number of specifications
[Quote] JUAS
Evaluating how appropriate the specification or
determined at first is depending on the degree of Total number of changed specifications / total
consensus number of functions
E.g.)
Fu-6 (1) No. of changed specifications against overall ++ ++ 31.6% 53.8%
size, or specification change size against overall
size
(2) No. of changed specifications against total
number of functions (evaluating how many
specifications are changed per function.)
Functional Completeness of Ratio of the number of implemented functions X=1-A/B

appropriateness functional against the number of functions stated in the A = Number of missing functions detected
implementation requirement spec. in evaluation
B = Number of functions described in
Evaluating whether the functions necessary to requirement specifications
realize the required specification are [Quote] ISO/IEC9126-2
appropriately implemented with no excess or
Fu-7 shortage. ++ ++ 42.1% 45.5%
E.g.) Comparing the number of functions

implemented inaccurately or the number of
missing functions detected during evaluation
with the number of functions stated in the
requirement spec.
Functional Functional Ratio of functions implemented with no problem X=1-A/B

appropriateness appropriateness compared to the number of functions to be A= Number of functions in which problems are
checked detected in evaluation
B= Number of functions checked
Evaluating whether the functions complying with [Quote] ISO/IEC9126-2
the requirements are appropriately implemented
Fu-8 ++ ++ 21.1% 46.7%
E.g.) Comparing the number of functions from
which problems are detected during evaluation
(testing) against the total target functions for
evaluation
Functional Compliance rate to Ratio of the number of items in compliance X = A/B

appropriateness software design against the number of items required in the A = No. of items in compliance
policy software design policy B = No. of target items for compliance
requirement
Evaluating the appropriateness of design from
the viewpoint of the compliance rate to software Example of software design policy)
Fu-9 design policy Aged/disabled people consideration design ++ + 26.3% 35.7%
policy
E.g.) Comparing the target items for compliance
in the software design policy that must be
complied with against the number of items that
are actually in compliance
Functional Presence of Ratio of the number of standards actually Application rate of the required standards,
appropriateness acquisition of acquired against the number of required Retention rate of the required standards
standard (compliance standards such as Product Safety Standard
standard, product
safety standard) When it is required to apply for a public standard
Fu-10 or retain public qualification, evaluate whether ++ + 10.5% 41.2%
such application or retention is maintained.
E.g.) Comparing the number of standards and

the number of acquired standards
Functional Stability of functional Ratio of the number of functions that are X=1-A/B
appropriateness specification changed after development starts against the A=Number of functions changed during
number of functions in the requirement spec. development life cycle phases
B=Number of functions described in
Evaluating the appropriateness of the functional requirement specifications
spec. If stability is low, pay attention to the [Quote] ISO/IEC9126-3
influence on associated functions.
Fu-11 + + 21.1% 46.7%
E.g.) Comparing the number of functions that
are changed after starting development and the
number of functions stated in the requirement
spec.
5/29
Metrics relative to Performance efficiency
Definition of performance efficiency:

performance relative to the amount of resources used under stated conditions
Recommended Usage result

Metrics Usage
After
Sub characteristics Metrics Explanation of metrics Reference definition Up to Currentl is
No. combin
unit test y in use recomm
ed test
ended
Time behavior Degree to which the response and processing times and throughput rates of a product or system, when performing its functions, meet
requirements
Time behavior Response time Duration from giving an instruction to starting a X=time
batch of tasks till receiving the first response. (calculated or simulated)
Average time, max. time, etc. are included. At evaluation (testing), measure and evaluate
Pe-1 the time using test data in which the operational ++ ++ 73.7% 80.0%
conditions are anticipated.
Time behavior Turn around time Duration from giving an instruction to starting a X=time
batch of tasks till completion of tasks. (calculated or simulated)
Pe-2 Average time, max. time, etc. are included. ++ ++ 73.7% 60.0%
Time behavior Throughput The number of tasks (work) that can be No. of tasks per unit of time
processed per unit of time [Quote] ISO/IEC9126-3
Pe-3 The operation types are normal, peak and ++ + 73.7% 60.0%
degenerate.
Time behavior Response Ratio of the number of processes that can be Response compliance rate = No. of processes
compliance rate carried out within the regulated processing time carried out within the regulated response time /
against the number of processes of the online total number of processes
system stated in the requirement spec.
Pe-4 It is anticipated that the operational processing is + ++ 63.2% 42.9%
There are normal, peak and degenerate limited at degenerate operation, so it is
operation types. preferable to adjust the denominator.
Time behavior Processing margin Ratio of actual turn around time against the Margin rate = 1 - measured turn around time /
rate required turn around time required turn around time
There are normal, peak and degenerate

operation types.
Evaluating how much margin for processing
Pe-5 + + 26.3% 35.7%
there is in each case
If the margin rate is close to 1, the performance

may exceed requirements. If the margin rate is
small, improve the performance including
hardware.
6/29

Metrics Usage
After
No. combin
ed test
ended
Resource utilization Degree to which the amounts and types of resources used by a product or system, when performing its functions, meet requirements
Resource utilization Data volume Volume of data retained by the system (Grade)
All data volume handled by the system
If only main data volume is determined, there is Processing data volume per function
a risk that disc space will need to be added due Processing data volume per unit of time
to data for which investigation was missed in the [Reference] Nonfunctional
post process.
Pe-6 ++ ++ 84.2% #####
E.g.) Temporary stored portion of the master

type table or main transaction data
Resource utilization Memory capacity Capacity of the primary storage of the computer Element of the size of the memory capacity
to be necessary when taking a specific step. (GB, MB, KB)
[Quote] JUAS
For the reuse at the time of an allotment and the
Pe-7 unnecessary to a demand, it is divided into 1 ++ ++ 73.7% 80.0%
user, 1 session, 1DB connection hit and may be
managed.
Resource utilization Hard disc capacity of Hard disc capacity of server, etc. that is required Factor of the size of memory capacity (TB, GB,
server, etc. when carrying out specific processing MB)
Pe-8 [Quote] JUAS ++ + 84.2% 66.7%
Resource utilization No. of I/O devices The number of I/O devices required when No. of I/O devices
carrying out specific processing [Reference] JUAS
Evaluating whether the operation is carried out

Pe-9 correctly with the min. equipment configuration ++ + 57.9% 62.5%
or standard equipment configuration under the
system operation conditions regulated in the
requirement.
Resource utilization CPU utilization rate Ratio of program CPU usage per unit of time CPU utilization rate
while in operation [Quote] Nonfunctional
Pe-10 ++ 68.4% 50.0%
Resource utilization Storage period of log Necessary storage period for data used by Storage period, No. of storage generations
data system infrastructure including OS and [Reference] Nonfunctional
middleware log among data to which the system
refers
Pe-11 + 73.7% 60.0%
It is determined for the type of data as
necessary.
Resource utilization Target storage range Necessary storage range for data used by (Grade: from narrower ones)
of log data system infrastructure including OS and Online referable range
middleware log among data to which the system Online referable range + archive
refers [Quote] Nonfunctional
Pe-12 ++ ++ 52.6% 44.4%
It is determined together with the archive and
referenceable range.
7/29

Metrics Usage
After
No. combin
ed test
ended
Capacity Degree to which the maximum limits of a product or system parameter meet requirements
Capacity No. of bases The number of bases where system operates No. of bases
[Quote] Nonfunctional
When data center or system is on multiple
Pe-13 bases, it is necessary to standardize the ++ ++ 73.7% 20.0%
operation rate of each base or to control
dynamic increase/decrease of the capacity.
Capacity No. of functions The number of functions used to realize the No. of functions
target tasks for systemization [Quote] Nonfunctional
Pe-14 ++ ++ 68.4% 33.3%
Capacity No. of batch The number of batch processing cases per unit No. of batch processing cases
processing cases of time [Reference] Nonfunctional
Pe-15 ++ ++ 57.9% 25.0%
Capacity Increase rate of data Increase or decrease rate of data volume that Increase/decrease rate of data volume
volume can be handled in a system according to compared with the standard data volume value
increase or decrease of business operation from to be handled
the start of system operation [Reference] Nonfunctional
It can be compared with the average value of

Pe-16 starting day or static state after starting as ++ ++ 63.2% 71.4%
necessary.
E.g.) Comparing data volume per unit of time

from the past and current data volume per unit
of time
Capacity No. of online The number of requests received per unit of time No. of requests
requests [Quote] Nonfunctional
Clarifying the assumed number of requests for
normal and peak operation as a requirement,
Pe-17 ++ + 73.7% 40.0%
setting the pertinent conditions and carrying out
evaluation.
Capacity No. of users increase Ratio of increase or decrease in the number of Increase/decrease ratio in the number of users
rate users due to increase or decrease in the number from the start of operation
of user registrations/deletions from the start of [Reference] Nonfunctional
the system operation
Clarifying the assumed number of users at

Pe-18 + ++ 73.7% 80.0%
normal and peak operation as a requirement,
setting the pertinent conditions and carrying out
evaluation. It can be compared with the number
of users at the time of start or in steady state
after starting in some cases.
Capacity (Max.) No. of Max. number of people accessing the system No. of accesses
simultaneous simultaneously (Max. No. of simultaneous [Quote] Nonfunctional
accesses accesses)
Pe-19 Or the number of people accessing the system + ++ 52.6% 11.1%
simultaneously at a certain time.
Capacity Number of users The number of users (end users) using the No. of users
system [Quote] Nonfunctional
Pe-20 + + 78.9% 50.0%
Capacity (Max.) usage rate of Ratio of usage amount of device against the limit X = Amax / Rmax
I/O device usage amount that I/O device has
Amax = MAX(Ai), (for i = 1 to N)
Rmax = required maximum I/O messages
Pe-21 MAX(Ai) = Maximum number of I/O + 47.4% 50.0%
messages from 1st to i-th evaluation.
N= number of evaluations.
[Quote] ISO/IEC 9126-2
Capacity (Max.) utilization rate Ratio of used amount of transmission against X = Amax/Rmax
of transmission the limit amount of the transmission system MAX(Ai) = Max. value of amount of transmission
system in the valuation from No.1 to No.i
If utilization rate becomes higher, it becomes Rmax = Limit amount of transmission system
difficult for communication to be established due N = No. of evaluation times
Pe-22 to excessive communication requests, so that a [Reference] ISO/IEC 9126-2 + 52.6% 44.4%
congested state occurs where transmission
cannot be carried out immediately for a
transmission request.
8/29
Metrics relative to compatibility
Definition of compatibility:
Degree to which a product, system or component can exchange information with other products, systems or
components, and/or perform its required functions, while sharing the same hardware or software environment

Metrics Usage
After
No. combin
ed test
ended
Degree to which a product can perform its required functions efficiently while sharing a common environment and resources with other
Co-existence products, without detrimental impact on any other product
Co-existence Usable co-existence Degree to which a target software can be used X=A/B
with no restrictions or faults when it is used with A= Number of entities with which product can
other software at the same time. co-exist as specified
B= Number of entities in production
Co-1 Target example) distributable products such as environment that require co-existence + + 21.1% 20.0%
COTS and package [Quote] ISO/IEC9126-3
Degree to which two or more systems, products or components can exchange information and use the information that has been exchanged
Interoperability
Interoperability Connection with The number of cases where information is No. of external systems
external system exchanged with external system or component [Quote] Nonfunctional
This is used to evaluate the degree of impact

Co-2 from usage of a system according to the type + + 57.9% 37.5%
and number of external systems to be linked
(outside of div., outside of a company, social
infrastructure system, etc.)
Interoperability Interface consistency Ratio of the number of correct interface protocol X = A/B
(protocol) implementations against the number of interface A = No. of interface protocols confirmed at the
protocol implementations defined in the spec. review to check that they are implemented as
specification
Co-3 Ensuring consistency between the operation and B = No. of interface protocols that should be + 26.3% 21.4%
action in order to minimize the procedures that a implemented as stated in the spec.
user must remember.
Interoperability Data exModifiability Ratio of the number of data exchange formats X = A/B
that are correctly implemented against the A = No. of data formats that are regarded as
number of data exchange formats determined being exchanged with other software or systems
between the linked systems without any problems during data exchange
testing period
Co-4 This is used when it is required to exchange a B = Total number of data formats to be + 21.1% 33.3%
message (data) in a transaction, etc. between exchanged
different organizations, between systems or [Reference] ISO/IEC9126-2
computers via communication lines under
regulated data exchange format.
9/29
Metrics relative to usability
Definition of usability:
Degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use

Metrics Usage
After
No. combin
ed test
ended
Appropriateness Degree to which users can recognize whether a product or system is appropriate for their needs
recognizability
Appropriateness Description Ratio of the functions (or types of functions) X= A/B
recognizability completeness stated in the manual against all usable functions A= Number of functions (or types of functions)
described in the product description
Evaluating the completeness of the product B= Total number of functions (or types of
functions stated in a document or manual. functions)
Evaluating how well functions can be understood [Quote] ISO/IEC9126-3
Us-1 after reading a manual. + + 10.5% 17.6%
Appropriateness Completeness of Ratio of the functions stated in manual and/or a X= A/B

recognizability user's documentation built-in help facility against all usable functions A= Number of functions described
and/or help function B= Total of number of functions provided
Evaluating the effectiveness of help facility or [Quote] ISO/IEC9126-3
documentation in a document or manual used
Us-2 + + 10.5% 23.5%
by a user to understand the usage method of the
product functions. Evaluating how well functions
can actually be operated accurately.
Appropriateness Understandable I/O Ratio of the number of items that a user can X= A / B
recognizability understand against the number of items of I/O A= Number of interface functions whose
data purpose is correctly described by the user
B= Number of functions available from the
Us-3 Clarifying what is required as input data or what interface + + 5.3% 22.2%
should be output so that a user can understand. [Quote] ISO/IEC9126-2
Appropriateness Degree of Ratio of the number of functions that a user can X= A / B

recognizability understanding of understand against the number of usable A= Number of input and output data items
function functions which user success to understand during
enough short time within a few trial use
Making the contents or purpose of functions B= Number of input and output data items
Us-4 understandable for a user. Holding a user test with which user attempts to understand + 5.3% 33.3%
using a question tag. during observation period
Learnability Degree to which a product or system can be used by specified users to achieve specified goals of learning to use the product or system
effectively, efficiently, free from risk and with satisfaction in a specified context of use
Learnability Processing time Time required to carry out business operation Business operation processing time including
(learning time) processing referring to manual, guide, etc. system usage
Time for correction of errors in processing is

Us-5 + 21.1% 53.3%
included.
Evaluating the time required to learn a usage
method referring to manual, guide, etc.
Learnability Ease of help access Ratio of the number of tasks where help items X = A/B
can be found correctly against the number of A = No. of tasks where correct online help can
Us-6 tasks requiring help function be found + 0.0% 42.1%
B= Total No. of tasks requiring help
access[Reference] ISO/IEC9126-2
Learnability Learning ease (for Time required from starting learning the T= Sum of user operation time until user
execution of work in operation method for performance of task to achieved to perform the specified task
use) starting the operation efficiently. within a short time.
Us-7 + 5.3% 33.3%
10/29

Metrics Usage
After
No. combin
ed test
ended
Operability Degree to which a product or system has attributes that make it easy to operate and control
Operability Monitoring The number of monitoring items and the No. of monitoring information
information contents of monitoring performed for overall [Quote] Nonfunctional
systems or hardware/software comprising a
system (including business applications) in order
Us-8 to maintain quality of a system + + 42.1% 54.5%
E.g.) Alive monitoring, error monitoring, resource

monitoring and performance monitoring are
included.
Operability Monitoring of system The number of monitoring items and contents for No. of monitoring items
level the state of overall system or multiple servers, [Quote] Nonfunctional
etc. making up hardware/software (including
business operation applications) that make up
the system
Us-9 + + 42.1% 45.5%
Judging whether a system is in a state where it
functions with no problem.
E.g.) Monitoring of backup or job is included.
Operability No. of conditions No. of operation conditions including start, No. of operation start conditions
including operation interruption or finish of operation
start conditions
Us-10 This is required as a trouble prevention measure + + 31.6% 46.2%
in order to promote automatization of operation
and decrease intervention by an operator.
Operability Operational Ratio of the operation that can be performed X=1 - A/B
consistency using consistent operation against all operations A=Number of instances of operations with
required to realize the functions of the system. inconsistent behaviour
Us-11 + + 15.8% 31.3%
B=Total number of operations
Operability Message clarity Ratio of the number of messages that a user X=A/B
(ratio) can understand against total number of A=Number of implemented messages with
implemented messages clear explanations. .
Us-12 B=Number of messages implemented + + 15.8% 37.5%
It is particularly important to clarify a messages [Quote] ISO/IEC9126-3
at error in order to recover.
Operability Customizing Ratio of the number of functions that can be X=A/B

possibility (ratio) customized by a user against the number of A=Number of functions which can be
functions that it may be necessary to customize customised during operation
Us-13 B=Number of functions requiring the + + 5.3% 33.3%
Evaluating the adaptability towards individual customization capability
requests [Quote] ISO/IEC9126-3
Operability Monitoring possibility Ratio of the number of functions that can actually X=A/B
(ratio) be monitored against the number of functions A=Number of functions having status monitoring
that must be monitored for usage or operational capability
Us-14 status B=Number of functions that are required to + 21.1% 26.7%
have monitoring capability.
Evaluating the ease of usage [Quote] ISO/IEC9126-3
11/29

Metrics Usage
After
No. combin
ed test
ended
User error Degree to which a system protects users against making errors
protection
User error protection Mis-operation ratio Ratio of mis-operation occurring during usage X = A/B
A = No. of incorrect operations
Measure the number of occurrences of user B = Total No. of operations
errors in the simulation using the system at [Reference] JUAS
acceptance inspection of the prototype system
or before releasing
Us-15 + + 15.8% 62.5%
It's possible to evaluate the effect of the
preventive function by comparing the number of
incorrect operations occurring with a function to
prevent incorrect operations furnished and the
number of incorrect operations that occur
without it.
User error protection Error correctionability Ratio of the number of errors that are corrected X = A/B
in use or recovered easily against total number of A = No. of system errors that are recovered
errors occurring in use B= No. of system errors that occurred
Evaluating the contents of user handling of
Us-16 errors + + 5.3% 44.4%
E.g.) Measure the number of operations carried
out to recover or time required from occurrence
to recovery.
User error protection Avoidance of Ratio of the number of functions actually X=A/B
incorrect operation implemented against the number of functions A=Number of functions implemented with
that should be furnished with the incorrect user error tolerance
operation avoidance capability B=Total number of functions requiring the
tolerance capability
Evaluating the implementation degree of [Quote] ISO/IEC9126-3
functions for prevention of user errors
Us-17 + + 5.3% 33.3%
E.g.) Comparing the number of functions
implemented to avoid critical or serious
malfunctions being caused by incorrect
operation and the number of incorrect operation
patterns based on it.
User error protection Error message Ratio of the number of cases where a message X=A/B
obviousness stating that it's possible to recover can be A=Number of error conditions for which
correctly presented against the number of cases the user proposes the correct recovery
in error status action
B=Number of error conditions tested
Evaluating the quality of an error message
Us-18 + + 10.5% 35.3%
NOTE: This metric is generally used as one
E.g.) Comparing the number of corrected errors of experienced and justified.
through error messages and the total number of [Quote] ISO/IEC9126-2
corrected errors
User interface Degree to which a user interface enables pleasing and satisfying interaction for the user
aesthetics
User interface Appearance Ratio of the number of user interface factors that X=A/B
aesthetics customizability of can be customized as the user desires against A=Number of types of interface elements that
user interface total number of user interface factors can be customised.
B=Total number of types of interface elements.
Us-19 E.g.) Number of functions that can be [Quote] ISO/IEC9126-3 + + 0.0% 31.6%
customized, time or the number of operations
required for customization
Accessibility Degree to which a product or system can be used by people with the widest range of characteristics and
capabilities to achieve a specified goal in a specified context of use
Accessibility Physical accessibility Ratio of the functions that can be accessed by X = A/B
anybody regardless of disability or capability A = No. of functions that a person with certain
against all functions characteristics can access
B = Total number of functions
Us-20 E.g.) Carrying out evaluation using a check list [Reference] ISO/IEC9126-3 + + 0.0% 42.1%
where the functions that should be accessible Measure for each characteristics
are determined
Accessibility Number of languages The number of languages supported Supporting multiple languages
It should be considered to avoid cases where a
user of a language is not supported.
Us-21 + + 31.6% 38.5%
E.g.) Counting the number of translatable
languages
Accessibility Interface factor Ratio of the number of interface factors that a X=A/B
clearness user can understand against total number of A=Number of interface elements which are
interface factors self-explanatory.
B=Total number of interface elements
Us-22 It is necessary to clarify interfaces (hardware [Quote] ISO/IEC9126-3 + 10.5% 17.6%
interface, software interface, user interface) that
mediate in the exchange of information with
others (system, software, user).
12/29
Metrics relative to Reliability
Definition of reliability:
Degree to which a system, product or component performs specified functions under specified conditions for a
specified period of time

Metrics After Usage is
Sub characteristics Metrics Explanation of metrics Reference definition Up to Currentl
No. combin recomme
unit test y in use
ed test nded
Degree to which a system, product or component meets needs for reliability under normal operation
Maturity
Maturity Test plan Ratio of the number of actually planned tests X=A/B
appropriateness against the number of tests that should be A=Absolute number of faults detected in review
performed in order to meet the requirements B=Number of estimated faults to be detected in
review (using past history or reference model)
Re-1 Evaluating appropriateness of the test plan [Quote] ISO/IEC9126-3 ++ 47.4% 20.0%
Maturity Test appropriateness Ratio of the number of tests actually performed X= A / B

(or reviewed) against the number of tests that A= Number of actually performed test
should be performed in order to meet the cases representing operation scenario
requirements during testing
B= Number of test cases to be performed
Re-2 Evaluating appropriateness of the performance + ++ 73.7% 40.0%
for the test plan [Quote] ISO/IEC9126-2
Maturity Test density The number of tests per unit size of software No. of est cases / Size of software (1KLOC,
(1KLOC, etc.) or unit of effort. etc.)
[Quote] JUAS
Evaluating the number of tests per unit size as a
ermination criteria of the test. If the test density
Re-3 is small, increase the number of test cases. If it ++ ++ 89.5% 100.0%
is big, there is a possibility that unnecessary
tests are performed.
Maturity Test coverage Passing ratio of the test in units of statement C0: Statement coverage rate: Executing all
code, or the ratio of passes for which a test is statements in the code more than once
performed against the total passes C1: Branch coverage rate: Executing all
branches in the code more than once
Re-4 Evaluating the coverage of the test for a C2: Condition coverage rate: Executing all ++ ++ 73.7% 100.0%
program conditions in the code more than once
Maturity Defect detection rate Ratio of the number of defect cases detected X = A/B
during review against the number of defect A = No. of defects detected at review
cases that are anticipated to be detected during B = No. of defects that are anticipated to be
development . detected at review (according to experience in
Re-5 the past or using the reference model) + 78.9% 25.0%
Reference *ISO/IEC9126-3
Maturity Defect density Density of the number of defects detected per Density of defects extracted at the test per unit
unit size of software (1KLOC, etc.) quantity
[Reference] Important infrastructure
Re-6 + ++ 94.7% 0.0%
Maturity No. of cases pointed The number of defects detected during review No. of defects extracted at review
out at review [Reference] Important infrastructure
Re-7 + + 89.5% 100.0%
Maturity Review indication The number of defects detected during review Density of defects extracted at review per unit
density per unit size of software (1KLOC, etc.) scale
[Quote] Important infrastructure
13 Re-8 + + 78.9% 75.0%
Maturity Defect elimination The number of defects (causes) eliminated X=A

A = No. of defects corrected at design/coding
Counting the number of defects eliminated Y = A/B
during design/coding and comparing with the A = No. of defects corrected at design/coding
number of defects detected at review during B = No. of defects detected at review
design/coding period (at testing)
a) X = A1/A2
Re-9 A1 = No. of corrected defect cases ++ ++ 73.7% 40.0%
A2 = Total No. of defect cases actually detected
b) Y = A1/A3
A3 = Total No. of potential defects forecast on
software products
13/29

No. combin recomme
unit test y in use
ed test nded
Maturity Defect elimination Ratio of the number of corrected defects against No. of corrected defects / No. of detected
rate the number of detected defects (causes) nonconformities
[Quote] Important infrastructure, ESQR
Re-10 + + 89.5% 100.0%
Maturity Defect convergence Ratio of the number of defects detected at the Defect convergence rate = No. of
rate final step of the test against the number of nonconformities at the final step of the test
defects (causes) detected at the initial step of period / No. of defects at the initial step of the
the test. test period
Re-11 + + 78.9% 75.0%
[Reference] Important infrastructure, ESQR
Maturity Missing rate Ratio of the defects that are not extracted at the Missing rate of process n = No. of missing errors
current process and are clarified at the post of process n / (No. of indicated errors at process
process n / No. of missed errors of process n)
[Reference] Important infrastructure
Re-12 + + 63.2% 100.0%
Maturity Test maturity Ratio of the number of tests that have passed X= A / B
against the number of tests that are set in order A= Number of passed test cases during
to meet the requirements testing or operation
B= Number of test cases to be performed.
Re-13 + 63.2% 57.1%
Availability degree to which a system, product or component is operational and accessible when required for use
Availability Working ratio Ratio of time over which service is actually Working rate = Value shown in the working
provided against the service time regulated in proportion after measuring operation time per
the operation schedule or objective recovery month excluding the planned stop time
standard [Quote] JEITA
Re-14 Calculating for each system target (online ++ 89.5% 100.0%

system, network service, etc.,) or target time
(operation duration, response time, reception
time, etc.).
Availability Operation time Time operating the system

(normal, special day)
Special day indicates a day where a different
schedule from normal operation schedule is
defined, such as holidays, public holidays or
Re-15 beginning or end of month. + + 78.9% 75.0%
Availability RTO (Recovery time Objective that is set at recovering when a failure Recovery time
objective), (Objective causing cessation of business occurs [Quote] Nonfunctional
recovery standard)
E.g.) RTO: Recovery time objective, RLO*
Recovery level objective, RPO: Recovery point
Re-16 objective + + 68.4% 83.3%
Availability Service switching Time required to restart business after taking

time measures in the event of trouble
The case where business is disconnected

temporarily due to trouble with hardware, etc. is
Re-17 anticipated as case of trouble. + + 63.2% 42.9%
A possible measure would be switching a server
in a cluster composition, etc.
Availability Batch processing Ratio of the number of processes completed Finish rate within the time of batch processing
normal finish rate within the defined processing time against the [Quote] Important infrastructure
number of batch processes actually carried out
Re-18 + + 36.8% 75.0%
Availability Reception time Service time slot in which the operation receives Reception time = Service time slot in which the
support operation receives support
[Quote] JEITA
Re-19 + 84.2% 33.3%
14/29

No. combin recomme
unit test y in use
ed test nded
Availability (trouble) handling Operation time slot in which trouble is detected Handling time = Operation time slot in which
time during operation of a system and it is handled trouble is detected during operation of a system
and it is handled
[Quote] JEITA
Re-20 + 63.2% 42.9%
Availability Handling time for job Operation time slot in which job operation Handling time = Operation time slot in which job
operation (processing deemed as an aggregation from the operation is carried out
viewpoint of a user) is carried out [Quote] JEITA
Re-21 + 52.6% 22.2%
Availability Service available Ratio of time during which service can be used Service time rate = Ratio of time during which
time of operation against the defined usage time facilities (system/software products) can be used
against the defined time
[Quote] JEITA
Re-22 + 73.7% 40.0%
Availability Working quality rate Degree of negative impact on a user due to the Checking that users are suffered from no
occurrence of trouble inconvenience due to occurrence of trouble
Ratio of the number of times users are suffered
E.g.) Ratio of the number of times when there is from by a user due to trouble against total asset
Re-23 ++ 31.6% 38.5%
an impact on the user due to trouble against the size
total asset size [Quote] JUAS
Availability Support and Ratio of the number of times when action is No. of times when action is executed following
execution rate executed following the standard against the the standard / Total No. of trouble occurrences
against hardware number of malfunction alarms of hardware of hardware
alarm [Quote] JUAS
Re-24 ++ 15.8% 62.5%
15/29

No. combin recomme
unit test y in use
ed test nded
Degree to which a system, product or component operates as intended despite the presence of hardware or software faults
Fault tolerance
Fault tolerance Functional shut down Ratio of faults causing shut down against total X= 1-(A / B)
avoidance faults A= Number of breakdowns
B= Number of failures
Re-25 ++ 47.4% 70.0%
Fault tolerance Redundancy Contents and the number of preventive or Handling contents and the number along with
(machines) avoidance measures retained in order to keep redundancy
required service levels against faults occurring [Reference] Nonfunctional
on networking devices including routers and
Re-26 + + 63.2% 42.9%
switches, servers, terminals, external memory
devices or lines
Fault tolerance Transaction Appearance of extremely large load in a short No. of transactions
protection period compared to the load at normal times, [Quote] Nonfunctional
presence of protection or countermeasures for
the state exceeding the anticipated peak of
Re-27 + ++ 78.9% 50.0%
business operation amount
Fault tolerance Segment division Contents and the number of segment divisions (Grade: From those with lower fault tolerance)
Not dividing
Dividing into units of sub system
Dividing according to the application
(application is shown in each business operation
such as online or batch from the control type
Re-28 application such as monitoring or backup. It is + + 47.4% 20.0%
anticipated that segments are divided according
to the application after dividing into units of sub
system)
[Reference] Nonfunctional
Fault tolerance Fault notification time Time from detecting an error (error of network Fault notification time = time from detecting an
service, fault in routine job operation, etc.) till error till reporting the fault state
report of fault state is sent [Quote] JEITA
Re-29 + 68.4% 66.7%
degree to which, in the event of an interruption or a failure, a product or system can recover the data directly affected and re-establish the
Recoverability desired state of the system
Recoverability Backup utilization Degree of necessity for utilization (acquiring) of (Grade: From those with low necessity)
range backup data used by the system Not acquiring backup
Prevention of data loss at occurrence of trouble
Recovery from a user error (in the case of
recovering from a user error, it is necessary to
return the process that is normally completed as
a system to the original state, so it is considered
Re-30 that the functions including backup control of + + 63.2% 71.4%
multiple generations or time specification
recovery (Point in Time Recovery) may become
necessary.
Long term storage of data (archive)
[Reference] JEITA
Recoverability Fault recovery time Time from fault detection till the service is Fault recovery time = Time from fault detection
recovered till the service is recovered
[Quote] JEITA
Re-31 + 57.9% 62.5%
16/29
Metrics relative to Security
Definition of security:
Degree to which a product or system protects information and data so that persons or other products or systems have the degree of data access appropriate to their types and levels of
authorization

Metrics Usage
After
No. combin
ed test
ended
Degree to which a product or system ensures that data is accessible only to those authorized to have access
Confidentiality
Confidentiality Operation limiting Contents of operational limitations including Locking a door or storage cabinet, limiting I/O
degree in a measure installation limitation of software and usage devices including USB, CD-RW and keyboard,
for a system limitation command execution limitation, minimum
necessary execution of program, operation of
Se-1 Clarifying the contents of usage limitation command, permitting access only for files, etc. ++ ++ 47.4% 30.0%
according to the authority of a user. [Reference]JEITA
Clarifying the contents of limitation of usage of
assets, etc. using software or hardware towards
the certified entity (user, equipment, etc.).
Confidentiality Illegal access control Ratio of the number of implemented functions X= A / B

performance against the number of functions used to control A= Number of detected illegal operations
illegal access B= Number of illegal operations anticipated
Or, in specification
Ratio of the number of controlled accesses [Reference]ISO/IEC 9126-2
against total number of illegal accesses
Se-2 ++ ++ 31.6% 38.5%
Evaluating the controlling capability of illegal
access
Checking whether a function is furnished to
prevent an unauthorized user from creating,
deleting or correcting programs or information.
Confidentiality Illegal access Ratio of the number of traces actually recorded X= A / B
surveillance against the number of monitoring traces required A= Number of user accesses to the system
performance for access to system and data and data recorded in the access history
database
Evaluating the monitoring capability of illegal B= Number of user accesses to the system
Se-3 access and data done during evaluation ++ ++ 26.3% 28.6%
Checking whether it is recorded in access history [Quote] ISO/IEC 9126-2
database by trying to access simulating the
invasion
Confidentiality Illegal monitoring Monitoring contents or monitoring range of illegal (Grade: from those of less important degree)
target (device) access to system and data or illegal packets on None
network Range of handling assets with a high degree of
importance or circumscribing part
Se-4 Clarifying the target to track, monitor or detect Overall system + + 63.2% 28.6%
illegality. [Reference] Nonfunctional
Determining the target for which a log should be
acquired under monitoring, recording volume or
period of log.
Confidentiality Detection range of Range of detecting illegal action to system on (Grade: from those of less important degree)
illegal communication network or illegal communication None
Range of handling assets with a high degree of
Se-5 Clarifying the area that should be tracked, importance or circumscribing part + + 52.6% 33.3%
monitored or detected for illegality. Overall system
Confidentiality Communication Contents of control items to shut down illegal No. of communication control items
control communication [Quote] Nonfunctional
Se-6 Evaluating the contents of network measures + + 47.4% 30.0%
Confidentiality Network convergence Contents of measures for convergence (non No. of convergence measures
measures available state due to intensified access) due to [Quote] Nonfunctional
attack on a network
Se-7 Evaluating the contents to avoid attacks that + + 47.4% 50.0%
cause disconnection of service
Confidentiality Presence of Presence of encryption and its content for Presence of encryption
encryption of keeping confidential data secret at transmission [Quote] Nonfunctional
11 Se-8 transmission data or storage + + 57.9% 25.0%
Confidentiality Data encryption Ratio of the number of encryption/decryption of swm

data items implemented correctly against those
numbers stated in the requirement spec.
Se-9 E.g.) Comparing the number of data items that + + 31.6% 30.8%
are implemented as spec. and can be
encrypted/decrypted and the number of data
items of required data encryption/decryption
functions.
17/29

Metrics Usage
After
No. combin
ed test
ended
Degree to which a system, product or component prevents unauthorized access to, or modification of, computer programs or data
Integrity
Integrity Preventive property Ratio or degree that serious (or minor) data X= 1 A / N
of data damage damage phenomenon is actually prevented A= Number of times that a major data
against the number of test cases per unit corruption event occurred
operation time where an attempt is made to N= Number of test cases tried to occur data
cause data damage phenomenon corruption event
or
Evaluating to what extent the phenomenon of Y= 1- B / N
Se-10 serious data damage or minor data damage can B= Number of times that a minor data ++ ++ 10.5% 47.1%
be prevented. corruption event occurred
[Quote] ISO/IEC 9126-2
E.g.) Comparing the number of data damage
instances actually occurring and the number of
operation and access times where data damage
or breakage is expected to occur.
Integrity Enhancement of Contents of measures related to unique threats No. of measures related to threats and
measures by secure and vulnerability of Web application vulnerability
Se-11 coding, web server [Quote] Nonfunctional + + 57.9% 37.5%
setting, etc.
Integrity Presence of Contents of security diagnostics for web server No. of execution times of web diagnostics
execution of web and web application performed for the web site [Quote] Nonfunctional
diagnostics
Se-12 + + 63.2% 28.6%
Integrity Risk handling range Handling range of anticipated threats (Grade: from those with narrow range)
after starting Not handling
operation Determining as the policy of measures for Handling threats related to assets with high
threats after starting operation. degree of importance or threats of
Se-13 circumscribing part + 52.6% 22.2%
Handling overall clarified threats.
Integrity Security risk revising Analysis range of new threats discovered after (Grade: from those with low frequency)
frequency starting operation of the target system and their None
influence Executing (occasionally) when the event
Se-14 related to security occurs + 36.8% 50.0%
Objective is to review security risks. Executing (occasionally) when the event
related to security occurs + executing on a
regular basis
Integrity Security patch Application range, policy and the contents of [Reference]
(Grade: fromNonfunctional
those with narrow range)
application range application timing regarding application of Not applying security batch
security batch for handling vulnerability, etc. of a Range handling assets with high degree of
target system. importance or circumscribing part
Se-15 Overall system + 63.2% 28.6%
When considering application of security batch, [Reference] Nonfunctional
it is necessary to check the influence on the
overall system and to judge the suitability of
application of batch.
Non-repudiation Degree to which actions or events can be proven to have taken place, so that the events or actions cannot be repudiated later
Non-repudiation Key management Contents of key management of encryption for (Grade: from those with less important control
keeping confidential data secret at transmission contents)
or accumulation. None
Key management using software
Se-16 Checking whether the contents of key Key management by withstand tamper resistant + + 57.9% 25.0%
management are clarified when encrypting data. device
Non-repudiation Presence of Presence of deployment of digital signature that Presence of usage of signature
utilization of digital enables proof that information is properly [Quote] Nonfunctional
signature processed and stored and to detect falsification
of information.
Se-17 + + 57.9% 37.5%
Checking the presence of deployment of
technology that verifies the sender of a
document and assures that the document is not
falsified.
Accountability Degree to which the actions of an entity can be traced uniquely to the entity
Accountability Acquisition of log Presence of acquisition of log, contents of log Monitoring record storage amount, period of log
Log is acquired to check "when", "who", "from
Se-18 where", "what is carried out" and "what happens ++ ++ 84.2% 66.7%
as a result" and to use measures promptly.
Accountability Log storage time Period of storing a log Log storage time
Se-19 Determining the period to retain the processed + + 78.9% 50.0%
trail
18/29

Metrics Usage
After
No. combin
ed test
ended
Authenticity Degree to which the identity of a subject or resource can be proved to be the one claimed
Authenticity Authentication of an Presence and contents of authentication Presence of authentication of control entity
entity with control function used to identify an entity with control [Quote] Nonfunctional
authority authority (system administrator or business
administrator) or an entity who uses the assets
(user, machine, etc.)
Se-20 Preventing acquisition or abuse of control ++ + 73.7% 60.0%

authority by an attacker
E.g.) Authentication using ID/password or using

IC card are included as methods of
authentication.
Authenticity Establishment of Contents and the number of rules for information Presence of control rules
control rules necessary for authentication [Quote] Nonfunctional
Se-21 E.g.) Necessary information used to specify an + + 68.4% 66.7%

entity uniformly for authentication includes
ID/password, fingerprints, iris and vein pattern.
Authenticity Presence of in- Presence of measures and their contents No. of rules, laws and guidelines
company regulations, including organization regulations, rules, laws E.g.)
rules, laws and and guidelines regarding information security Information security policy
guidelines that should that should be observed Anti-unauthorized access law
be obeyed Personal information protection law
Electronic signature law
Provider liability law
Act on regulation of transmission of specified
electronic mail
Se-22 Sarbanes-Oxley Act + + 63.2% 42.9%
Basic IT law
ISO/IEC27000 system
Uniform standard for information security
measures of government organization
FISMA
FISC
PCI DSS
19/29
Metrics relative to Maintainability
Definition of maintainability:
Degree of effectiveness and efficiency with which a product or system can be modified by the intended maintainers

Metrics Usage
After
No. combin
ed test
ended
Degree to which a system or computer program is composed of discrete components such that a change to one component has minimal
Modularity
impact on other components
Modularity Condensability Degree of the relational strength between the LCOM* (Lack of Cohesion in Methods) is an
function factors and information factors in a index to measure condensability of source code
class or package of the object-oriented language.
LCOM* = (aveA-m)/(1-m)
aveA = sumA/a
sumA = A1+A2++Aa
Ma-1 AjMember variable of No. j of the focused class + 15.8% 43.8%
a: No. of member variables of the focused class
m: No. of methods of the focused class
It is said that the smaller the value of LCOM*, the

higher the condensability.
Reusability Degree to which an asset can be used in more than one system, or in building other assets
Reusability Execution of Ratio of target assets that can be reused against No. of reused libraries / No. of target assets for
reusability total number of assets reuse that are controlled by a reused library
[Quote] JUAS
Ma-2 Ratio of reuse modules against all modules, + + 10.5% 41.2%

reuse rate at the project level, etc.
Degree of effectiveness and efficiency with which it is possible to assess the impact on a product or system of an intended change to one or
Analyzability more of its parts, or to diagnose a product for deficiencies or causes of failures, or to identify parts to be modified
Analyzability Conformance rate to Ratio of the number of applied standard items No. of conforming items / No. of items required
coding code against all items related to the development to conform
standard determined as the coding rule standard [Quote] JUAS
and organization Or, ratio of the complying items against the
Ma-3 number of applied standard items 42.1% 45.5%
Analyzability Maintenance Ratio of the number of documents against the No. of documents that should be prepared / No.
document sufficiency number of maintenance documents that should of documents that are prepared
lead to improvement of analyzability [Quote] JUAS
It is necessary to determine the necessary or X = A/B

maintenance documents beforehand, including A = Types (No.) of documents actually stated in
Ma-4 ++ 21.1% 40.0%
function spec., DB cross reference, data item B
cross reference, transaction reference, change B = Types (No.) of maintenance documents that
procedure (organizational change, institutional lead to improved analyzability
change, change of limit amount, etc.).
Analyzability Execution record Ratio of the number of logs actually recorded X=A/B
against the number of required execution logs A=Number of implemented data login items as
specified confirmed in review
Checking the furnishing status of the execution B=Number of data items to be logged defined in
record (log) in order to find out the defect or the specifications
trouble causes in a product. [Quote] ISO/IEC9126-3
Ma-5 + + 21.1% 26.7%
Comparing the number of items of which
execution log is recorded and the number of
items for which a log is required as spec.
Analyzability Diagnosis function Ratio of the number of implemented diagnosis X=A/B

sufficiency functions against the number of required A=Number of implemented diagnostic functions
diagnosis functions. as specified confirmed in review
B=Number of diagnostic functions required
Checking the furnish status of diagnosis [Quote] ISO/IEC9126-3
Ma-6 + + 10.5% 29.4%
functions in order to discover defects or troubles
of a product.
Comparing the number of implemented
diagnosis functions and the number of required
diagnosis functions.
Analyzability Trail audit ability Ratio of the number of operation logs actually X= A / B
recorded against the number of required A= Number of data actually recorded
operation logs. during operation
B= Number of data planned to be recorded
E.g.) Counting the number for which the cause enough to monitor status of software during
Ma-7 + + 21.1% 20.0%
can be identified operation
Checking the furnishing status of functions in [Quote] ISO/IEC9126-2
order to find out the cause of trouble from the
operation record.
20/29

Metrics Usage
After
No. combin
ed test
ended
Degree to which a product or system can be effectively and efficiently modified without introducing defects or degrading existing product
Modifiability quality
Modifiability Scale up / Scale out Scale up/scale out methods and the number of (Grade: from those with smaller ability)
servers that can be increased to prepare for the Not reinforcing by scale up or scale out
increase in the amount of business operation in Setting only a certain server as the target
the future. Setting multiple servers as the targets
Clarifying the server processing ability
Ma-8 reinforcement method. + + 73.7% 60.0%
Scale up: Reinforcing the processing ability by
switching to a server with larger processing
ability. Scale out: Reinforcing the processing
ability by preparing multiple equivalent servers
and increasing the number of servers.
Modifiability Localization degree The number of variables affected by a change X=A/B

of correction impact A=Number of affected variable data by
(appearance degree Evaluating the impact on software products due modification, confirmed in review
of trouble after a to change or correction. B=Total number of variables
change) [Quote] ISO/IEC9126-3
or
Ma-9 X = A/B + + 10.5% 35.3%
A = No. of modules affected by correction as
confirmed at review
B = Total No. of modules
Modifiability Maintainability Ratio of conformity against required observance X = A/B

standard functional to standard, agreement or rule regarding A = No. of conforming items regarding
completeness ratio maintainability implemented maintainability
B = Total No. of conformity items regarding
Ma-10 maintainability + + 5.3% 44.4%
Modifiability Change history Ratio of the number of actually recorded cases Change history recording ratio = No. of actual
recording ratio against the number of target cases for recording, records of change history of software / No. of
which is required as a change history to trace cases where a change history of software should
changes in software. be recorded
Ma-11 [Quote] JUAS + + 5.3% 44.4%
Evaluating the ability to control change.
Modifiability Influence rate of Ratio of the cases where bad influence has X = 1A/B
change occurred after correction against all corrected A = No. of corrections that have caused bad
cases. influence
B = No of executed corrections
Ma-12 Evaluating frequency of correction that causes [Reference] ISO/IEC9126-3 + 15.8% 56.3%
unpredictable incident including nonconformity of
data, etc.
Modifiability Change execution Average working time required to resolve trouble Average time: = Tav = (Tm)/N
elapsed time Tm = Tout = Tin: Elapsed time from
Evaluating the ease of change to resolve occurrence to removal of trouble
trouble. Tout = Time when cause of trouble is removed
by changing software (or time when a status
report is replied to a user)
Ma-13 Tin = Time when cause of trouble is discovered + 5.3% 50.0%
N = No. of registered troubles that are removed
(Possible to use man-hours instead of time)
Modifiability Change success rate Ratio of the number of trouble instances within a X= Na / Ta
certain period after a change against the number Fluctuated frequency of encountering
of trouble instances within a certain period failures before/after change
before the change Y = {(Na / Ta) / (Nb / Tb) }
Evaluating the effectiveness of a change. Na = Number of turns which user

encounters failures during operation after
software was changed
Nb = Number of turns which user
Ma-14 encounters failures during operation before + 5.3% 50.0%
software is changed
Ta = Operation time during specified
observation period after software is
changed
Tb = Operation time during specified
observation period before software is
changed
21/29

Metrics Usage
After
No. combin
ed test
ended
Testability Degree of effectiveness and efficiency with which test criteria can be established for a system, product or component and tests can be
performed to determine whether those criteria have been met
Testability Functional Ratio of the number of implemented test X=A/B
completeness of functions against the number of required test A=Number of implemented built-in test function
embedded test functions (No. of facilities) as specified confirmed in review
functions B=Number of built-in test function required
Ma-15 E.g.) Comparing the number of test functions [Quote] ISO/IEC9126-3 + 15.8% 37.5%
implemented as spec. and the number of
required test functions.
Testability Autonomous Ratio of the number of tests that can be carried X=A/B
testability out independently against all tests that have no A=Number of dependencies on other systems
choice but to depend on other system for testing that have been simulated with stubs
B= Total number of test dependencies on other
Ma-16 Tests that depend on other system include systems + 5.3% 27.8%
cases that can be simulated by stub (program [Reference] ISO/IEC9126-3
that intermediates when a certain program calls
another program).
Testability Readiness of Ratio of the number of tests that can be X= A / B

embedded test executed without preparing the functions for test A= Number of turns which maintainer can
functions (simulation function, test tool, etc.) separately use suitably built-in test function
against total number of operation tests. B= Number of turns of test opportunities
Ma-17 [Quote] ISO/IEC9126-2 + 0.0% 21.1%
Evaluating usability of the test environment.
Testability Retesting efficiency Average test time after resolving trouble Average X= Sum(T) / N
T = Time spent to test to make sure whether
Evaluating promptness of judgment to release reported failure which was resolved or not.
N= Number of resolved failures
Ma-18 [Quote] ISO/IEC9126-2 + 0.0% 26.3%
Testability Test restartability Ratio of the number of test cases where a test X=A/B
can be stopped or restarted freely against the A = Number of turns which maintainer can
number of various test cases pause and restart executing test run at
desired points to check step by step
Ma-19 Evaluating convenience of the execution points B= Number of turns of pause of executing + 0.0% 21.1%
of a test test run
22/29
Metrics relative to Portability
Definition of portability:
Degree of effectiveness and efficiency with which a system, product or component can be transferred from one hardware, software or other operational or usage environment to another

Metrics Usage
After
No. combin
ed test
ended
Degree to which a product or system can effectively and efficiently be adapted for different or evolving hardware, software or other operational
Adaptability or usage environments
Adaptability Presence of adoption Presence of specification of adopting open (Grade: from those without specification)
of specific product source product or third party product (ISV/IHV) No specification of specific product
specified by a user. Specific products are specified partially
Product for which support is difficult is specified
When a specific product is adopted or specified, [Quote] Nonfunctional
Po-1 it is necessary to consider the difficulty degree of ++ ++ 36.8% 25.0%
support.
Adaptability No. of supported No. of supported languages No. of languages

languages [Quote] Nonfunctional
When using in a country for which language is
not supported, it may become impossible to
access or exchange information at all.
Preparing a check list containing necessary
Po-2 technologies or notice points to support multiple + + 21.1% 40.0%
languages beforehand and evaluating the
supporting status.
E.g.) Counting the number of translatable

languages.
Accessibility Degree of effectiveness and efficiency with which a product or system can be successfully installed and/or uninstalled in a specified
Installability environment
Accessibility System transfer time Time required from the transfer work plan to full Transfer period
Installability operation [Quote] Nonfunctional
E.g.) system transfer time, system stoppable

day/time, presence of parallel operation
Po-3 ++ + 47.4% 40.0%
Accessibility Ease of Ratio of correct implementation of a system X = A/B

Installability implementation A = No. of operations performed by changing the
performance It is preferable to implement a system evenly procedures at implementation
and correctly by following a certain procedure. B = A series of total number of operations
required for implementation
Po-4 + ++ 21.1% 20.0%
Accessibility Rehearsal range Range that the setting rehearsal should be (Grade: from those with low attention degree)
Installability carried out No rehearsal
Main normal cases only
Clarifying a setting rehearsal (including a All normal cases
rehearsal where trouble during setting is Normal case + Abnormal case of cutting back
Po-5 + + 57.9% 25.0%
anticipated). to the status before transfer
Normal case + Abnormal case of recovering
from a system trouble
Accessibility Presence of external Presence of necessity for rehearsal regarding (Grade: from those with no necessity)
Installability cooperative rehearsal connection specification at cooperating None
externally Present (no change of external connection
spec.)
Present (there is a change of external
Po-6 + + 57.9% 12.5%
connection spec.)
23/29

Metrics Usage
After
No. combin
ed test
ended
Replaceability Degree to which a product can replace another specified software product for the same purpose in the same environment
Replaceability Complexity of transfer No. of conversion rules for transfer tool (Grade: from those with low complexity)
tool (number of Transfer tool is not necessary or existing
conversion rules) Evaluating complexity of transfer work. It is transfer tool can support it
considered that the more conversion rules there No. of conversion rules is less than 10
Po-7 are the higher the complexity of the transfer tool No. of conversion rules is less than 50 ++ ++ 26.3% 14.3%
is. It is necessary to pay attention at transfer if No. of conversion rules is less than 100
the complexity is higher. No. of conversion rules is 100 or more
Replaceability No. of base Development contents and the number of steps (Grade: from those with low difficulty level)
development steps required at transferring a system and at new No regulation because of a single base
(system development development Simultaneous development
method) Less than 5 steps
Evaluating difficulty level of system transfer work Less than 10 steps
Po-8 related to multiple bases. It is considered that Less than 20 steps + ++ 47.4% 20.0%
the larger the number of steps there are the 20 steps or more
higher the level of difficulty. Difficulty level No. of steps level is determined considering the
becomes higher even at simultaneous risks at development for each base.
development in some cases according to risk at [Quote] Nonfunctional
development of base.
Replaceability Work division of Degree of user transferability in the transfer (Grade: from those with low difficulty level)
user/vendor of work. All users
Po-9 transferring Executing in cooperation between users and ++ 36.8% 16.7%
Clarifying roles in the transfer work. vendors
All vendors
Replaceability Functional Ratio of how long the similar functions to X=A/B

comprehensibility software before transfer can be used. A = number of functions which produce as
enough similar results as used to be
Evaluating the replaceability of functions. produced and of which changes have not to
be required;
Po-10 E.g.) Comparing the number of functions that B = number of tested functions which are ++ 26.3% 28.6%
bring the same results and are not changed and similar to functions provided by other
the number of functions that have been the software to be replaced.
target of continuation from software before [Quote] ISO/IEC9126-2
transfer.
Replaceability User support function Ratio of consistency of user interface of new T=Sum of user operating time to be spared
consistency system against existing user interface to complete adaptation of the software to
users environment, when user attempt to
Evaluating replaceability of user interface. In the install or change setup.
case of user interface of dialog type, if the (Person-hour may be used instead of time.)
Po-11 + + 10.5% 29.4%
behavior of the user interface of a system after [Quote] ISO/IEC9126-2
replacement is unacceptably different from the
expectations of users in the past, it is considered
that replaceability is low for a user.
Replaceability Transferring contents Contents and the number of the target facilities (Grade: from those with low difficulty level)
of facilities/equipment for transferring that are used in the system No transferring targets
before transferring are to be replaced with new Replacing hardware
facilities in the new system. Replacing hardware, OS, middleware
Po-12 Replacing the whole system + + 47.4% 10.0%
Evaluating the contents and the number of the Replacing the whole system and integrating
targets for transferring. It is considered that the [Quote] Nonfunctional
larger the contents for transferring the higher the
difficulty level.
Replaceability Continuous usage of Ratio of continuous usage of same data for data X = A / B
data of software before transfer A = number of data which are used in other
software to be replaced and are confirmed
Evaluating replaceability of data. that they are able to be continuously used.
B = number of data which are used in other
Po-13 E.g.) Comparing the number of data confirmed software to be replaced and planned to be ++ 31.6% 23.1%
as continuously usable from software before continuously reusable.
transfer and data planned to be able to use [Quote] ISO/IEC9126-2
continuously from software before transfer.
Replaceability Transfer data amount Amount of business data that needs to be (Grade: from those with less volume data)
transferred on the system before transferring No targets for transfer
(including program). Less than 1TB
Less than 1PB
Po-14 E.g.) No targets for transfer, less than 1TB, less 1PB or more ++ ++ 68.4% 33.3%
than 1PB, 1PB or more [Quote] Nonfunctional
Evaluating volume of target data for
replacement.
Replaceability Transfer media The amount of target media for transfer and the Transfer media amount
amount number of media types required at transfer. [Quote] Nonfunctional
Evaluating the number and types of target media

Po-15 + 47.4% 20.0%
for replacement.
24/29
Metrics relative to Quality in use characteristics
Definition of quality in use characteristics

Degree to which a product or system can be used by specific users to meet their needs to achieve specific goals effectively, efficiently, free from risk and with satisfaction in
specific contexts of use.
Explanation of metrics
*Work means a series of activities performed by a Recomme
Usage is
Metric Quality in use Sub user till accomplishment of the objectives using the nded Currently
Metrics Reference definition recommen
s No. characteristics characteristics target system. (e.g. withdrawing money using an Degree to in use
ded
ATM, obtaining air tickets using a ticketing system, be used
etc.)
accuracy and completeness with which users achieves specified goals

Effectiveness
Effectiveness - Work Ratio of the number of work instances in X = A/B

effectiveness which target work output is correctly achieved A = No. of work instances where
against the number of individual work target work output is not
instances performed by a user achieved
B = No. of work instances to
Evaluating the effectiveness of part of work for obtain target work output
Efe-1 usage. Evaluating the effectiveness of results ++ 31.6% 38.5%
of individual work (operation) when a user
uses a system.
Effectiveness - Work Ratio of the number of completed work X = A/B

completeness instances against the number of work
instances executed by a user A = number of tasks completed
B = total number of tasks
Evaluating effectiveness of a series of work attempted
for usage. Evaluating effectiveness of overall [Quote] ISO/IEC9126-4
work when a user attempts to achieve an
Efe-2 + 21.1% 53.3%
objective using a system.
Effectiveness - Failure Ratio of the number of failures caused by a X = A/T

frequency user occurring during user's work against total
number of work instances A = number of errors made by
the user
Efe-3 Evaluating effectiveness of user interface for T= time or number of tasks ++ 36.8% 25.0%
usage. [Quote] ISO/IEC9126-4
Efficiency resources expended in relation to the accuracy and completeness with which users achieve goals
Efficiency - Work duration Duration necessary from starting work till X = Ta

completion of work Ta = task time
Efi-1 Evaluating efficiency of work duration ++ 57.9% 37.5%
Efficiency - Auto Ratio of the number of items of which actual No. of qualitative effects for
measurement of qualitative effects can be measured against which a system to count and
qualitative the number of evaluation items of qualitative report the effects is prepared /
effects effects Total number of qualitative
effects
Evaluating efficiency of qualitative evaluation. [Quote] JUAS
Efi-2 When the qualitative evaluation of 5 steps, + 5.3% 11.1%
etc. is required according to KPI (key
performance indicator), it is preferable to
prepare the system to measure the effects
automatically.
Efficiency - Work efficiency Ratio of achieved work objectives per unit of X = M1 / T

time (work effectiveness)
M1 = task effectiveness
Evaluating efficiency of work duration. T = task time
It increases by increase in work effectiveness [Quote] ISO/IEC9126-4
or reduction of work duration. (it's possible to
compare and evaluate an interface that makes
Efi-3 work duration longer though its work + 15.8% 43.8%
effectiveness is high or an interface for which
work duration is short though work
effectiveness is low)
25/29
Usage is
ded
etc.)
Efficiency - Economic Ratio of work objective achieved per unit of X = M1 / C

productivity cost (work effectiveness)
M1 = task effectiveness
Evaluating efficiency of work cost. C = total cost of the task
It increases by increase in work effectiveness [Quote] ISO/IEC9126-4
or reduction of work cost. (it's possible to
compare and evaluate interface that makes
Efi-4 + 15.8% 37.5%
work cost higher though its work effectiveness
is high or interface of which work cost is low
though work effectiveness is low)
Efficiency - Productive ratio Ratio of time when a user carries out X = Ta / Tb

productive activity against total work time
Ta = productive time =
Evaluating efficiency of work time. When task time - help time - error time
Efi-5 using this index, it is necessary to monitor and - search + 5.3% 11.1%
analyze a series of work. time
Tb = task time
Efficiency - Relative user Ratio of work efficiency of a regular user X=A/B

efficiency against work efficiency of skilled operator
A = ordinary users task
Skilled operator means a person who is able efficiency
Efi-6 + 10.5% 41.2%
to utilize the target system. B = expert users task efficiency
Efficiency - Overall lead time Ratio of actual overall lead time against the Overall lead time/ target overall
ratio target overall lead time lead time
[Quote] JUAS
Efi-7 + 26.3% 7.1%
Lead time is time required from start to end of
work.
Satisfaction Degree to which user needs are satisfied when a product or system is used in a specified context of use
Satisfaction Usefulness Degree to which a user is satisfied with their perceived achievement of pragmatic goals, including the results
of use and the consequences of use
Satisfaction Usefulness Satisfaction Ratio of the number of users who are satisfied X = A/B
scale with using a system against total number of A = Result of questionnaire
users (question table) using
psychometrics scale
Sa-1 ++ 47.4% 70.0%
Measuring using a questionnaire (question B = Average of parent
table). population
Satisfaction Usefulness Discretionary Ratio of the number of functions that are X = A/B
utilization actually used against the number of functions A= number of times that specific
that can be used freely software
It is a usable index it is an available index functions/applications/systems
when it's possible to select a function to use are used
Sa-2 freely. B = number of times they are ++ 21.1% 53.3%
intended to be used
It cannot be used for a function that should [Quote] ISO/IEC9126-4
always be used.
Satisfaction Usefulness No. of customer The number of claims made by a customer No. of claims made by a
Sa-3 claims customer ++ 63.2% 42.9%
[Quote] JUAS
Satisfaction Trust Degree to which a user or other stakeholder has confidence that a product or system will behave as intended
Metrics for this quality sub characteristic do not exist.

Satisfaction Pleasure Degree to which a user obtains pleasure from fulfilling their personal needs
Satisfaction Comfort Degree to which the user is satisfied with physical comfort
26/29
Usage is
ded
etc.)
Freedom from Degree to which a product or system mitigates the potential risk to economic status, human life, health, or the environment
risk
Freedom from Economic risk Degree to which a product or system mitigates the potential risk to financial status, efficient operation,
risk mitigation commercial property, reputation or other resources in the intended contexts of use
Freedom from Economic risk Return of Ratio of effective amount against invested ROI = Effect amount / Invested
risk mitigation investment amount amount
(ROI), IT asset IT asset investment = Expected
investment For the ratio of effective amount related to IT effect amount / IT investment
compared to IT invested amount, the amount
utilization degree of IT assets can be [Quote] JUAS
evaluated as IT asset investment.
This is used to monitor the effect that can be
converted into a monetary amount including
reduction in personnel expenses, shrinkage of
inventory assets, reduction of stock, reduction
Fr-1 of material cost through concentrated ++ 47.4% 60.0%
purchase.
Payout time (inverse number to ROI) is

normally from 2 to 10 years. In the case of
reconstruction, payout time is prolonged
compared with new development.
Freedom from Economic risk Comparison with The status of own company against the status ((IT investment amount / sales)
risk mitigation other companies of other top class companies in the industry or of our company) / ((IT
(benchmark) in same business investment amount / sales) of
target company for comparison),
E.g.) (IT investment amount/sales) of own etc.
Fr-2 company / (IT investment amount / sales) of [Quote] JUAS ++ 36.8% 33.3%
industry, etc.
Freedom from Economic risk Balance score Quantitative evaluation result on IT investment Evaluation from 4 viewpoints;
risk mitigation card (BSC) effects from 4 viewpoints; finance, customer, unique finance to BSC,
business operation processes and HR customer, business operation
development processes and HR development
[Quote] JUAS
Viewpoints regarding IT investment evaluation
Fr-3 are set uniquely, expected effects are listed up ++ 26.3% 35.7%
and the numeric objective is set as the
quantitative index to use as material for
investment evaluation.
Freedom from Economic risk Ratio of revenue Ratio of actual revenue against revenue Actual revenue from a customer
risk mitigation for each new objective from a customer, a usage target of / Objective of revenue from a
customer the system customer
[Quote] JUAS
Evaluating the status of opportunity loss for
Fr-4 + 26.3% 28.6%
provision of new function.
There are several attributes of customers
such as existing and new.
Freedom from Economic risk Opportunity loss Opportunity loss amount when investment is Opportunity loss amount
risk mitigation not carried out. [Quote] JUAS
It is used to forecast the effect of a system

Fr-5 including future possibilities. + 26.3% 28.6%
Freedom from Economic risk Economic Ratio of the number of occurrences of X = 1A/B
risk mitigation damage economic damage against the total number of A = No. of occurrences of
usage times economic loss
B = Total No. of usage status
Fr-6 Economic damage indicates monetary [Quote] ISO/IEC9126-4 + 15.8% 37.5%
damage caused by a system.
27/29
Usage is
ded
etc.)
Freedom from Economic risk No. of delayed The number of delayed cases from delivery No. of delayed cases from
risk mitigation cases from time delivery time
delivery time No. of actual delayed cases from
delivery time. Monitoring lead
time for a customer from
ordering to delivery and
Fr-7 + 36.8% 33.3%
checking the handling status of
the desired delivery time of a
customer.
[Quote] JUAS
Freedom from Economic risk No. of missing The number of missing item cases No. of missing item cases
risk mitigation item cases It is used for improvement of
customer service.
Fr-8 [Quote] JUAS + 26.3% 35.7%
Freedom from Economic risk Software Ratio of the number of cases where damage X = 1-A/ B
risk mitigation damage occurs against the number of usage cases of
software program, file, data, etc. within a A = number of occurrences of
certain period. conomic damage
B = total number of usage
Damage indicates the status where file is not ituations
Fr-9 stored or data is broken because software [Quote] ISO/IEC9126-4 + 5.3% 16.7%
program freezes due to unpredictable request,
etc.
28/29
Usage is
ded
etc.)
Freedom from Health and Degree to which a product or system mitigates the potential risk to people in the intended contexts of use
risk safety risk
mitigation
Freedom from Health and Degree of Degree of bad influence suffered by Degree of nuisance to
risk safety risk nuisance to customers customers = No. of affected
mitigation customers people x time x degree of
significance
Fr-10 Significance degree is ++ 36.8% 25.0%
determined according to the
degree of nuisance to customers
[Quote] Important infrastructure
Freedom from Environmental Degree to which a product or system mitigates the potential risk to property or the environment in the intended
risk risk mitigation contexts of use
Freedom from Environmental Degree of Degree of support required for environmental (Grade: from those with low
risk risk mitigation compliance with burden supporting degree)
Law on Support is not necessary
Promoting Green Support includes usage of Partial usage of products
Purchasing equipment/consumable items with low complying with the standard of
environmental burden through purchasing Law on Promoting Green
products complying with Law on Promoting Purchasing
Fr-11 ++ 26.3% 50.0%
Green Purchasing and investigation into Usage of only products
minimizing disposal of items throughout the complying with the standard of
life cycle. the Law on Promoting Green
Purchasing
Freedom from Environmental Objective value Degree that objective value of CO2 emission (Grade: from those with low
risk risk mitigation of CO2 emission amount is required degree)
Not necessary to set the
CO2 emission amount at operation is basically objective value
linked to power consumption. This amount Objective value is presented
plus the CO2 emission amount from Objective value is presented
Fr-12 production/disposal becomes the emission and a request for additional + 15.8% 62.5%
amount of overall life cycle. reduction is made.
Context Degree to which a product or system can be used effectively, efficiently, free from risk and with satisfaction in both specified contexts of
coverage use and in contexts beyond those initially explicitly identified
Context Context Degree to which a product or system can be used effectively, efficiently, free from risk and
coverage completeness with satisfaction in all the specified contexts of use
Metrics of this quality sub characteristic do not exist.
Context Flexibility Degree to which a product or system can be used with effectively, efficiently, free from risk and
coverage with satisfaction in contexts beyond those initially specified in the requirements
Context Flexibility Recovery ratio Ratio of the number of days actually required No. of days till operating
coverage from disaster for recovery against the number of planned normally / Defined No. of days
days for recovery from disaster [Quote] JUAS
Determining the number of days required till

normal operation or operation in fail soft state
from non operational sate of system due to
occurrence of disaster.
Co-1 ++ 15.8% 62.5%
Possible to set by disaster type, such as
widespread disaster or local disaster.
Context Flexibility Degree of skill to Degree of learning through daily training in (Grade: from those with weaker
coverage switch to a order to switch to a backup machine or degree of skill)
backup machine recovery easily Feeling anxious about switching
Co-2 No problem because it is ++ 21.1% 40.0%
performed on a daily basis
[Quote] JUAS
29/29

11 05 Metricas PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

11 05 Metricas PDF

Uploaded by

Copyright:

Available Formats

Advanced Research Project on Software Metrics by the Ministry of Economy, Trade and Industry, Japan (METI)

A Set of Metrics for Information Systems/Software Product Quality in Japan

2. Contents of a Set of Metrics

3. Items of Metrics Table

Items and meanings of Metrics Table

*Note 1 Source reference of the metrics set reference definition

Embedded system development management guide: ESQR, Information-Technology

User Vender Collaboration Research Project II Report "Nonfunctional Requirement

*Note 2 Quality metrics utilization example research

Example of Metrics Table

Meaning of ratio / density / degree

Meaning of defect / fault

A Set of Metrics for Information Systems/Software Product Quality in Japan

Metrics relative to Functional suitability

Definition of functional suitability:

Recommende Usage result

E.g.) Comparing the number of incomplete

Evaluating how much margin for processing

Recommende Usage result

Functional Completeness of Ratio of the number of implemented functions X=1-A/B

E.g.) Comparing the number of functions

Functional Functional Ratio of functions implemented with no problem X=1-A/B

Functional Compliance rate to Ratio of the number of items in compliance X = A/B

E.g.) Comparing the number of standards and

A Set of Metrics for Information Systems/Software Product Quality in Japan

Metrics relative to Performance efficiency

Definition of performance efficiency:

Recommended Usage result

There are normal, peak and degenerate

If the margin rate is close to 1, the performance

Recommended Usage result

E.g.) Temporary stored portion of the master

Evaluating whether the operation is carried out

Recommended Usage result

It can be compared with the average value of

E.g.) Comparing data volume per unit of time

Clarifying the assumed number of users at

A Set of Metrics for Information Systems/Software Product Quality in Japan

Metrics relative to compatibility

Recommended Usage result

This is used to evaluate the degree of impact

A Set of Metrics for Information Systems/Software Product Quality in Japan

Metrics relative to usability

Recommended Usage result

Appropriateness Completeness of Ratio of the functions stated in manual and/or a X= A/B

Appropriateness Degree of Ratio of the number of functions that a user can X= A / B

Time for correction of errors in processing is

Recommended Usage result

E.g.) Alive monitoring, error monitoring, resource

E.g.) Monitoring of backup or job is included.

Operability Customizing Ratio of the number of functions that can be X=A/B

Recommended Usage result

A Set of Metrics for Information Systems/Software Product Quality in Japan

Metrics relative to Reliability

Recommended Usage result

Maturity Test appropriateness Ratio of the number of tests actually performed X= A / B

Re-7 + + 89.5% 100.0%

Maturity Defect elimination The number of defects (causes) eliminated X=A

Recommended Usage result

Re-14 Calculating for each system target (online ++ 89.5% 100.0%

Availability Operation time Time operating the system

Availability Service switching Time required to restart business after taking

The case where business is disconnected