Elicitation of standards for Non-Functional requirements of a

software system
1.

Introduction:

The quality of a software system is defined not only by its functional but also
from the Non-functional requirements of usability, performance,
interoperability, security and flexibility [1]. The functionality of the system is
not useful without these necessary non functional entities. Software
companies need to understand the quality requirements of the customers
thoroughly to sustain the high competition in the market [2]. Non functional
requirements/attributes are henceforth the main differentiator for a company
from its competitors of a software system. The software quality should attain
maximum customer satisfactory levels without excessively draining the
resources and increasing the production cost [3]. To keep a balance between
the benefits of the quality attributes and the cost of the software system
being developed it is necessary maintain the non functional quality attributes
within optimum levels. Elicitation of these levels of quality requirements to
develop an efficient software system without increasing the cost is an
interesting research aspect which would be addressed in the following
sections of the paper through literature review.
2.

Background:

Non Functional requirement is a term which limns those requirements that

generally focus on quantifying the quality of the software than those
requirements which are essential entities for its operation. The quality refers
to the degree of satisfaction of the stated expectations of the software
system by the customer [4]. Non Functional requirements of a software
system are described not based on what the software will do but on how the
software would do the assigned task. The Non Functional requirements of a
software system may constitute of several attributes, which are as follows:
Reliability, Modifiability, Understandability, Usability, Portability, Inter
Operability, Maintainability, Reconfigureability, Customability, Adaptability,
variability, Volatility, Traceability, Ubiquity, Nomadicity, Simplicity, Security,
Clarity, Integrity,
Modularity, Robustness, Correctness, Cohesiveness,
Completeness, Responsiveness, User Friendliness, Timeliness, Correctness,
Conciseness, Efficiency, Precision, Accuracy, Performance, Cost Development
time, Low coupling.

The International Standards Organization (ISO) developed the ISO/IEC 9126

standards for these non functional requirements in 1991 and restructured it
in 2001 [5]. This standard represents the software quality model which
distinguishes 4 types of quality levels:

1. Quality in Use:
This level delineates the perceptions of the quality requirements in specific
context by specific users in the real environment. This level measures the
effectiveness of the software system and end user productivity in 4 other sub
categorized of effectiveness, productivity, safety and satisfaction.
2 & 3. External and Internal Quality
External and Internal quality are the two different quality levels which share
the same 6 characteristics and 26 sub characteristics. The internal view is
generally associated with static properties of the software such as code
elements, design and complexity which can be quantified early during the
development phase. The external quality is associated to reliability which can
be quantified as the mean time of failures incurred by the real time execution
of software on computer hardware. The six characteristics that these levels
are categorized are Efficiency, Portability, Maintainability, Functionality,
Usability and Reliability.
4. Process Quality:
This level is associated with the process of product development. As all the
other quality attributes can directly relate to the process of development,
this level can broadly influence all other attributes.
These 4 levels mentioned in the ISO/IEC 9126 standard can quantify the
quality attributes of any software system. The non functional requirements
largely depends over the Architectural (Physical structure of the system) and
functional characteristics (Tasks performed by the system and the user) of a
software system [4]. There are several other classification schemes to
categorize the quality attributes which are inconsistent with each other. All
these classifications are based on a software quality tree depicted in figure 1
[6], which elaborate the key non functional requirements and their various
possible correlations.

Certain non functional requirements are classified as soft requirements

based on the level of satisfaction which indicates the right levels of
requirement to be identified during requirement engineering [7]. The quality
requirements have also been elicited by adopting the goal model which
identifies the needs for improving the level of quality characteristics of a
software system. The goal model identifies the quality attribute which are
relevant to the stake holders and connects them to the desired end results of
the software system but fail to quantify the quality levels [8]. The ISO/IEC
9126 standard of classification scheme is considered as a benchmark
amongst all the other classification schemes. Various elicitation method such
as workshops, storyboards, questionnaires, interviews and prototyping have
been considered useful for requirement elicitation. These elicitation methods
due to their generality cannot quantify levels of the quality attributes.

Figure 1: Software Quality Tree [6]

3.

Literature review:

B. Paech and D. Kerkow [4] reviewed several literatures to understand the

true definition of the term quality in the perspective of software engineering.
The basic idea behind all the existing approaches for dealing with aspects of
NFR is identical. They identified the criteria to evaluate the requirements of a
Non Functional requirement approach and proposed an ideal method called
Non-Functional Requirement Engineering (NFRE). A Task oriented
Requirement Engineering (TORE) method designed for making decisions of
functional requirements in an IT based system was used to identify the
requirements of a NFRE. The features of a NFRE are defined by task, domain
and interaction level decisions which are parts of TORE. On the basis of
literature survey they grouped the features of NFRE into 5 distinguished
steps:
1.
2.
3.
4.
5.

Identification of NFRE
Prioritizing the NFRE based on their dependencies
Documentation of NFR
Evaluation of means to satisfy NFR and take decisions
Project Management

The identification of NFR is done to elicit the requirements through various

techniques such as workshops and interviews from different stakeholders.
Quality is termed to be relative to the person expecting it and to the
possibilities of implementing it. It is essential to understand the context of a
specific quality concept whilst considering its diversity. Prioritizing the NFRE
is quite a difficult task owing to their subjective nature. The Definition of a
NFR would change when confronted with other NFR. Hence the NFR should
include the derived dependencies to make the conflicts explicit. The NFR
algorithm describes the dependencies with the help of soft goal
interdependence graphs. Certain conceptualize models are being added into
the algorithm to detect dependencies which helps to collect knowledge of
typical influences between quality attributes.
Documentation of NFR is quite essential due to its complexity and longevity.
The NFR should be stated and integrated precisely in to the requirements
documents of the industry. The use of templates as per specified rules can be
used to integrate the NFR into the industry documents. Use of specific
notations and tools compatible with RE tools should be done to make the
approach suitable for industry application. Standardized vocabulary, precise
and distinctive quality levels, specific documentation styles for different
quality attributes may also be included NFR method.

Evaluation of means to satisfy the NFR involves three major issues of

identification, evaluation of NFR compared to the means and deciding tradeoffs among other means. Identification of NFR is done with the help of
creativity and experience. As there is no standard specified methodology for
identification of means, it can be done with different approaches as
mentioned in the literature such as with the use of agent oriented goal
graphs, Case maps, CBSP approach etc. Evaluation of means against the NFR
can be done through the Architecture Tradeoff Analysis Method (ATAM) which
can be further refined with the Cost Benefit Analysis Method (CBAM). The
positive and negative influences of different means are captured in a goal
graph along with the argument and is used for deciding trade-offs in a
machine based evaluation. General matrices of questions, options and
criteria can be used for human based evaluations. For every major change in
the system would trigger the changes in the identified set of NFR, Thus these
changes in the set of requirements should explicitly specified. There is no
standardized approach for project and change management of NFR, Hence it
can be usually done through capturing the changes with special labels in the
complicated goal graphs.
F. Fotrousi et al. [9] evaluated the levels of good enough quality attributes in
the field of telecommunications. In a telecommunication system the quality
of experience (QOE) by a user is affected by the quality of services (QOS)
provided by the supplier which is measured on the basis of the service level
agreements (SLA) between the system supplier and the customer. A method
of Quality Impact Inquiry was adopted to address the issues related to
determine the adequate levels of quality. This method is based on the
principles of generic relationship between QOS and QOE provided by M.
Fiedler et al. [10] translated into an inquiry based analysis process integrated
with various other methods of collecting data for quality measurement and
stakeholder opinions such as prototyping, questionnaires and workshops. The
methodology comprised of a 4 step inquiry cycle process which can be
iteratively applied until the stakeholder satisfaction is attained for the quality
attribute under investigation. The quality inquiry method adopted is depicted
in figure 2.

Figure 2: Quality Impact Enquiry Method

The first step of the inquiry cycle is generalized by the term Preparation, in
which the software prototype is prepared and tested among certain
stakeholders. These stakeholders provide their feedbacks about the quality
attributes of the software prototype by answering a questionnaire. Based on
these responses appropriate modifications can be done on the quality
attributes of the software prototype. Measurement comprises the second
step of the inquiry cycle wherein the stakeholders experience is quantified
based on the predefined ratings/scores provided by the users in the response
of the questionnaire which can be directly translates into the quality impact
of an attribute. Analysis being the third step of the inquiry cycle which
statistically measures the quality and the quality impact collected in the
previous step using a regression model. The regression function is calculated
from the measured quality and the quality impact and compared with
different regression functions of linear, logarithmic, power and exponential.
An estimation of quality value of a quality impact is done by the inverse
function of the regression model. If these values do not provide enough data
for analysis, modification in the software prototype is done which changes
the
quality
values
artificially.
The final step of the inquiry cycle is Decision-Making in which the desired
levels of quality attributes are identified using the analysis result. This
decision is further used to provide modifications to the software requirement
specifications. This step helps us to understand the maximum applicable
quality impact based on technical feasibility, product strategies and resource
limitation.
Method tailoring of the quality impact inquiry process can be done based on
the specific requirements of the software system. F. Fotrousi et al. [9] has
demonstrated one such inquiry process in the real world scenario for a

Diabetes Smartphone Application used by diabetes patients for measuring

the blood glucose and send the collected data to a diabetes specialist to plan
the schedule for insulin injections. An inquiry workshop was conducted by
the requirement engineers, product managers for the end user diabetic
patients in a laboratory with the prototype application. A quality of
experience questionnaire was prepared by the requirement engineer based
on the software requirement specification (SRS) document and instrumented
the software with a time-stamp logger. A set of instructions for the use of the
software was provided to the end users and the response time for the use of
software system was measured. The end user answered the prepared set of
quality of experience questionnaire with quantitative scales. On analysis of
the data collected from the questionnaire and the time-stamp logs, it was
identified that the quality impact values did not deviate from the expected
levels of quality attributes of the software system. Decision making step in
the development of a software system of a telecommunication area such as
the case of diabetes smartphone application involves of setting up of a
threshold value for the quality attribute such as response time to the
minimum extent possible closer to the value defined in the SRS document.
The method adopted here in this quality inquiry process in the workshop got
completed in few hours and hence can be termed as an efficient means
which can be further scaled up by working with multiple users in parallel.
B. Regnell, et al. [3] also proposed a new model for identifying the levels of
quality attribute and determining the good enough quality of a software
system. The model proposed by them in the literature has been termed as
QUPER (Quality-Performance) which provides reasoning concepts for
quality which relates cost and values. It is proposed that instead of viewing
the quality levels in binary properties of Good and Bad, it should be
viewed with a more continuous and non linear approach depicted through
different shades on a sliding scale. Keeping this approach in view QUPER
model is designed with three basic strategies; robust to uncertainties, easy
to use, domain relevant.
The QUPER model quantifies the levels of quality attributes in three general
views based on two basic concepts of breakpoint and barrier. The breakpoint
relates quality with benefits whereas the barrier provides a relationship
between quality and cost. The benefit view is the first general view which is
sub- divided into three breakpoints which limn the shift of benefit levels with
respect to its market value and user experience. The appropriateness of the
product in the market with respect to its usability is marked by the utility
breakpoint, such as that below this mark the product is termed to be useless.

After crossing the utility breakpoint the product attains a market value and
moves towards the differentiation breakpoint, which resembles the
competitive quality of the product in the market. The saturation breakpoint is
the third breakpoint in the benefit view, crossing which the product is said to
attain an excessive quality level which does not significantly effects the
benefits in specific usage contexts.
The cost view indicates the second general view which is further divided by
barriers that represent the non linear nature of the relationship between
quality and cost. The quality cost relationship is seen to have different
steepness ranges which are indicated by barriers as the costs shifts from a
plateau like situation to sharp rise indicating the cost penalty incurred by the
increase in quality. This shift to steep rise in cost is due to additional cost
incurred for reconstruction of the product architecture to attain a certain
quality attribute. The third general view is termed as the roadmap view
which combines the breakpoints and barriers including the future targets on
a same scale to visualize the relation of current product quality with that of
its competitors.
QUPER model suggested by B. Regnell, et al. [3] is composed of six steps
which start with defining the quality attributes. Secondly, breakpoints and
barriers are estimated for each quality attribute. Estimation of the current
quality attributes with respect to that of the competitors is done in the third
step to set targets for new quality attributes for future in the fourth step.
Approving, communicating and revising of the roadmaps after several
iterations are done in the final two steps. A smart phone manufacturing
company Sony Ericsson introduced this approach to evaluate the desired
level of quality for a specific quality attribute of one the model of the smart
phone. The performance quality attribute tested was defined as the response
time to play music after pressing the play button of one of their smart phone.
The break points of utility, differentiation and saturation was set in the
benefit view and then compared with that of the competitors in the market.
Further the targets were identified and set keeping consideration the barriers
in the cost view. The QUPER model was also tested in several other
companies and was found to be successful in identifying the barriers and
breakpoints. The number of quality attributes that can be tested with QUPER
model needs to be identified based on the domain of the product and its
utility. QUPER Model provides a better picture of quality targets by allowing
the compare the functional and non functional requirements with the market
and calibrates them accordingly.

J. Doerr et al. [11] explained a systematic method of elicitation of NFR with

the help of three different cases. They developed an experience based
systematic framework to elicit, analyze and document NFR in such a way
that the practical problems emerging during the software development are
taken into consideration. According to them the key features of this NFR
method involves common treatment of different quality attributes (QA),
experience based quality models, distinction between different types of QA,
detailed guidelines in terms of checklist and questionnaire, guidelines for
documentation, justification of each NFR, Treating NFR equally to functional
requirements and system architecture and requirement management
support with dependency analysis.
Quality Attributes (QA) are referred to the non functional characteristics of a
system, tasks or specific aspects of the development process. These QA are
specified by a certain set of values describes as NFR that are required to be
achieved by the specific project. Goal Graphs based on notations are
deployed evaluations of dependencies and refinement of a QA with reference
to their respective models. In the generalized process, elicitation and
documentation to prioritize a QA is done with the aid of a questionnaire. The
selected Quality Model (QM) based on the rankings of their respective QA is
then tailored in a workshop by experts of the company as per the
requirements of the project. Checklists and templates are further tailored
based on the QM, which is further used in a second workshop to elicit the
actual NFR. The QM is updated based on the experience from different
projects. Three such project cases were explained by J. Doerr [11] which
explained in the further sections.
Wireless Plant Control Case Study: This case study was done along with a
company that manufactures embedded system. An embedded system for a
wireless control and monitoring of a manufacturing plant with the aid of
handheld computers was studied to identify the QA of reliability, efficiency
and maintainability of the system. A prioritized questionnaire was filled in the
workshop conducted in the company based on which the QA of reliability,
efficiency and maintainability and the respective NFR were selected based
on quality models. Prioritizing of QA was found to be very useful and helped
in finding new and important NFR. The quality models used were updated
due to the tailoring performed on them. More than 90% of the measurable
NFR were achieved in this case study. It was found that with the use of this
method of NFR the development cost could be reduced by 50%. It was also
found beneficial for dependency analysis for identifying conflicting
requirements in an early phase of the product development.

Multifunctional Printer System Case Study: Multifunctional printer (MFP)

system is equipment used in offices which perform multiple tasks of printing,
scanning and other functionalities. They are a form of embedded system
which the required performance is achieved through highly interconnected
hardware and software system. The requirement analysis of the MFP system
is highly complicated owing to its functionalities and architectural constrains.
The NFR method was applied in this case to evaluate the efficiency
requirements of a middle ranged color MFP product with automatic document
feeder mechanism and network interface. The NFR method discussed in the
previous case study was used to elicit the NFR. It was found in this
experience that a granularity is required to be present between the
requirement engineer and the developer to manage the NFR systematically.
The criteria for the requirements granularity was attained through checklists,
combined reference QM and analysis through workshops. The requirement
management for every iterations while developing a highly integrated
hardware and software system.
Geographical Information Case Study: This project was developed to enable
farmers to access data and maps of their respective areas with the help of
internet. The information in this system is subjected to privacy policy
security requirements hence security was selected as prioritized QA. An
initial Quality Model was selected based on the standard specified by the
quality experts and was tailored as per the requirements of the project. This
QM was further refined to prepare a checklist and conduct workshops. The
questions in the checklist pointed out several aspects of security other than
the one that were already specified and made the workshop straight forward.
It was observed that a slightest change in any of the NFR would make it
essential to newly assess all the possible alternatives.
J. Boegh [1] elaborately explains the new standards for quality requirements
by providing details of the recently published ISO/IEC 25030 standard for
software quality requirement and indicating the limitations of the earlier
version of the standard ISO/IEC 9126 explained in the above background
section. He relates software quality requirements with the system quality
requirements, since software being a part of the system it is to be viewed as
the system requirement itself. It was proposed by the committee members of
the standard developing agency to include a system model into the
standards for software quality requirements. The system model includes the
computer system, mechanical parts and the human processes which
satisfactorily describes software quality requirements in the system
perspectives. The software quality model of the ISO/IEC 9126 was adopted in

the drafting of the new standard ISO/IEC 25030. The ISO/IEC 9126 software
quality model lags in interpreting the software functional abilities and
handling the size of the data of software quality requirements by the model
was complex.
The newly introduced ISO/IEC 25030 standard does not provide a specific
elicitation method but instead provide a general applicable guideline to be
used in specific approaches. The system view approach has been considered
in the new standard which emphasizes on documentation of the stake holder
needs, expectations, desires which may even be completely unrealistic. The
systems intended purpose needs to be described as requirement for the new
standard process. This standard is subjected to both acquirers and suppliers
to provide requirements and recommendations for specifying quality
requirements of the software and the system.

4.

Conclusion:

B. Paech and D. Kerkow [4] eventually concluded that instead of developing a

systematic approach for NFR, One should concentrate upon understanding
the notion of quality. It can be achieved by performing more empirical based
research on the impact analysis of NFR on the performance of the project
which should also include experience and cost based information. Subjective
and objective quality attributes should be identified in a more distinctive
manner. Ethnographical and graphical design methods should be utilized in
NFRE.
The 4 step Quality-Impact enquiry method discussed above provides
essential guidelines for the requirement engineers to evaluate the good
enough software quality by considering the viewpoints of the stake holders.
The stakeholders experience of the software prototype is quantified in the
form of a subjective feedback to evaluate the quality impacts. This method
can be flexibly tailored according to the specific software system
requirement. This method was tested in real world scenario to quantify the
quality attributes. The quantified quality attributes can be mentioned as
minimal, maximal and expected quality in the service level agreement (SLA)
and software requirement specification (SRS) documents.
The QUPER model discussed above also terms out to be the most appropriate
approach for setting the limits for quality attributes by relating quality
attributes with the cost of the software product. The QUPER model quantifies

the levels of attributes based on the views of the benefits and cost
implication of a specific quality attribute of the software product by
categorizing them on the basis of breakpoint and barriers and comparing
them with the market and evaluates targets to outline a roadmap for the
future. This model based on its six steps approach clearly defines the quality
attributes and studies its implication on both quality benefits and cost
implications. It has been tested and adopted in several industries to evaluate
and is being recognized as a simple and easy to learn model.
J. Doerr et al. concluded from the case studies that high degree of customer
satisfaction was achieved. Definition of common ground and enhancement of
communication are considered to be the major benefits of the proposed NFR
methodology. The method is capable to elicit and document a sufficient and
minimal set of traceable and measurable NFR.
The ISO/IEC 9126 standard was also studied to understand the international
standard procedure for evaluating the quality level attributes which are
related only to the software quality attributes. Inclusion of the system quality
attributes which also involve system hardware and human processes has
been recently done in the new version of the quality requirement standard
ISO/IEC 25030. This new standard dictates to acquire quality requirement
suggestions from both acquirers and the suppliers for the software and the
system.
Even though all the four papers discussed above followed a similar approach
for the elicitation of non functional requirements, the models reported in
each of the paper were different with another in several aspects. The model
proposed by B. Paech and D. Kerkow [4], focus mainly upon prioritizing NFR
based on their interdependency evaluated through goal graphs. F. Fotrousi et
al. [9] quantifies NFR from customer satisfaction which is evaluated based on
questionnaires in an iterative process. B. Regnell [2] elicit the NFR by
comparing the market value of the quality attributes among the competitors
with the help of barriers and breakpoint in his QUPER model. The experience
based systematic framework of the model proposed by J. Doerr et al. [11]
evaluated the QA by conducting two consecutive workshops. The ISO/IEC
9126 proposes to consider system requirements along with the software
requirements in the process to elicit the NFR.
Scope of future work:

The future work should be aimed at validating the available different

models for elicitation of levels of quality requirements of a software
system in large scale requirement engineering situations.
Several different combinations of the software quality attributes should
be studied to evaluate the quality impact and understand the interrelationships among the attributes.
The easy to use model like QUPER should be considered in the
standards which directly relate the benefits of the system attributes to
the cost of the software system.
The further studies are required to understand the effects of system
quality attributes to the software quality attributes.

References:

1. J. Boegh, "A New Standard for Quality Requirements," IEEE Software, vol. 25,
pp. 57-63, 2008.
2. B. Regnell and J. Brinkkemper, Market-Driven Requirements Engineering for
Software Products, Engineering and Managing Software Requirements, A.
Aurum and C. Wohlin, eds., Springer, pp. 287308, 2005.
3. B. Regnell, R. Berntsson Svensson, and S. Olsson, "Supporting Roadmapping
of Quality Requirements," IEEE Software, vol. 25, pp. 42-47, 2008.
4. B. Paech, D. Kerkow, Non- Functional requirements engineering- quality is
essential, Proceedings of the 10th Anniversary International Workshop on
Requirements Engineering: Foundation of Software Quality (REFSQ'04), pp.
237-250, 2004.
5. ISO/IEC 9126-1:2001(E), Software Engineering - Product Quality - Part 1:
Quality Model, 2001.
6. B. W. Boehm, J. R. Brown, H. Kaspar, M. Lipow, G. J. MacLeod, M. J. Merritt,
Characteristics of Software Quality. North-Holland, Amsterdam (1978)
7. C. Irvine and T. Levin, "Quality of Security Service," presented at the 2000
Workshop on New Security Paradigms (NSPW'00), New York, NY, USA, 2000.
8. A. I. Antn and C. Potts, "The use of goals to surface requirements for
evolving systems," in International Conference on Software Engineering,
Kyoto, Japan, pp.157-166, 1998.
9. F. Fotrousi, S. A. Fricker, M. Fiedler, Quality Requirements Elicitation Based on
Inquiry of Quality-Impact Relationships, 22 nd international Requirements
engineering conference, IEEE, Karlskrona, pp. 303-312, 2014.
10.
M. Fiedler, T. Hossfeld, and P. Tran-Gia, "A generic quantitative
relationship between quality of experience and quality of service," Network,
IEEE, vol. 24, pp. 36-41, 2010.

11.
J. Doerr, D. Kerkow, T. Koeing, T. Olsson, T. Suzuki,NonFunctional Requirements in Industry Three Case Studies Adopting an
Experience-based NFR Method Proceedings of the 2005 13th IEEE
International Conference on Requirements Engineering (RE05), (2005)