Professional Documents
Culture Documents
to the
Certificate
Z10 13 06 67052 012
Software Tools for Safety Related Development
Distribution, copying or any other use of information in this report in part is strictly prohibited.
Revision Log
Rev. Date
Name
1.0
2009-07-21
1.1
2009-08-19
F. Rauch
1.2
2009-12-15
F. Rauch
1.3
2010-05-17
F. Rauch
1.4
2011-01-24
F. Rauch
1.5
2011-06-22
2.0
2011-12-19
S. Waldhausen
2.1
2012-06-26
S. Waldhausen
2.1.1
2.2
3.0
3.1
3.2
3.3
3.4
2012-08-28
2012-12-18
2013-06-25
2013-12-18
2014-06-13
2014-11-28
2015-05-29
S. Waldhausen, M. Braun
S. Waldhausen, M. Braun
S. Waldhausen, M. Braun
S. Waldhausen, M. Braun
S. Waldhausen, M. Braun
S. Waldhausen, M. Braun
S. Waldhausen, M. Braun
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933
Changes/History
Content
Page
IDENTIFICATION ................................................................................................................. 6
CERTIFICATION .................................................................................................................. 9
4.1
Standards ...................................................................................................................... 9
4.2
RESULTS .......................................................................................................................... 10
5.1
5.2
5.3
Requirements on software tools in IEC 61508, ISO 26262, and EN 50128.................. 11
5.3.1
General ................................................................................................................ 11
5.3.2
Static Analysis ...................................................................................................... 11
5.3.3
Polyspace tools .................................................................................................. 12
5.4
5.5
EN 50128 .................................................................................................................... 13
5.6
Tool classification and qualification according to ISO 26262 ....................................... 13
5.6.1
General ................................................................................................................ 13
5.6.2
Evaluation of the tool development process ......................................................... 15
5.6.3
Validation of the software tool............................................................................... 15
5.6.4
Summary .............................................................................................................. 15
5.7
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933
Polyspace Code ProverTM and Polyspace Bug FinderTM for releases R2013b and newer
Polyspace Client for C/C++ and Polyspace Server for C/C++ for releases R2013a and
before.
The divisions of The MathWorks, Inc. development organization responsible for the Polyspace
products for C/C++ have been audited to assess their development and quality assurance procedures. Recurring evaluations focus on the applied processes to implement enhancements and modifications, as well as quality engineering, and customer bug reporting processes.
The aim of the assessment was to determine the suitability for use in development processes which
need to comply with IEC 61508, ISO 26262, or EN 50128. The assessment covered tool classification and tool validation according to IEC 61508, as well as tool classification and tool qualification
methods according to ISO 26262.
The basic assessment is documented in the Technical Report MN72051T, recent modifications are
reported in Modification Reports according to the table below.
No.
Title
Report Number
Date
Revision
[R1]
MM82299T
10.09.2008
1.0
[R2]
MN72051T
28.06.2012
2.1
[R3]
MN84722T
18.12.2012
1.0
[R4]
MN85071T
24.06.2013
1.0
[R5]
MN85413T
18.12.2013
1.0
[R6]
MN85861T
13.06.2014
1.0
[R7]
MN86207T
28.11.2014
1.0
[R8]
MN86834T
29.05.2015
1.0
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933
2 Product overview
The Polyspace products for C/C++, are verification
tools used to increase the robustness of C and C++
code.
The Polyspace products for C/C++ provide code-based
verification that proves the absence of overflows, divides
by zero, out-of-bounds array access, and other run-time
errors in source code without requiring program execution, code instrumentation, or test cases. Polyspace
products use formal techniques such as Symbolic Execution and Abstract Interpretation techniques to verify
code. They can be used to verify handwritten code, generated code, or a combination of the two, before testing
the code.
The Polyspace products for C/C++ perform a static code analysis, but with a wider scope compared to what is often denoted with static code analysis: static analysis usually focuses on compliance with coding standards and coding rules, for instance MISRA. Polyspace products for C/C++
comprise the verification against a selectable coding standard (MISRA-C, MISRA-C++ or JSF++) as
a prerequisite for Abstract Interpretation, and produce code metrics such as cyclomatic complexity.
With release R2013b, the architecture of the Polyspace products for C/C++ has been reworked.
Compared with release R2013a, the overall functionalities and scope remain available and have
been extended with bug detection functionalities.
Polyspace Client for C/C++ and Polyspace Server for C/C++:
Until release R2013a the Polyspace products for C/C++ have been realized in a client-server architecture of the co-operating tools Polyspace Client for C/C++ and Polyspace Server for C/C++.
Polyspace Bug FinderTM and Polyspace Code ProverTM:
Starting with release R2013b, the Client-Server architecture has been replaced by the two products
Polyspace Bug FinderTM and Polyspace Code ProverTM .
Polyspace Bug FinderTM is intended for fast analysis and rapid feedback during development. Its
purpose is to
Find bugs:
numerical,
static / dynamic memory,
programming errors,
dataflow
coding rules check
determine code complexity metrics
o
o
o
o
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933
Polyspace Code ProverTM proves the existence and absence of certain categories of run-time
errors with static analysis that is based on formal methods.
Errors detected include for example:
Like with the predecessor products, the results of Polyspace Code ProverTM are presented by
highlighting the respective code with red / green / orange color according to its criticality.
For a detailed description of the kinds of bugs covered by each of the tools refer to the Tool Qualification package document according to the tables in section 3.
3 Identification
The following table lists the certified versions of the Polyspace products for C/C++ until R2013a.
Release
Date
Name
Ver.
Report
Documentation
IEC Certification Kit
[R1]
[R2]
[R2]
R2009a+
[R2]
R2009b
Sep.
2009
R2010a
R2007a+
August
R2008a
R2008b
4.2
R2010b
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933
[R2]
[R2]
[R2]
Release
R2011a
Date
April
2011
Name
Ver.
R2011b
Sept.
2011
R2012a
R2012b
Sept.
2012
R2013a
Report
Documentation
IEC Certification Kit
Verification of C and C++ Code Using
[R2]
[R2]
[R2]
[R2]
[R3]
The table below lists the certified versions of the Polyspace products for C/C++ of release R2013b
and newer:
Release
Date
R2013b
Ver.
Name
TM
1.0
[R4]
9.0
[R4]
Sept.
2013
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933
TM
Report
Documentation
Release
Date
R2014a
Ver.
Name
TM
R2014b
[R5]
9.1
[R5]
1.2
[R6]
9.2
[R6]
1.3
[R7]
9.3
[R7]
2.0
[R8]
9.4
[R8]
Oct.
2014
TM
TM
March
2015
R2015b
TM
TM
R2015a
1.1
March
2014
Polyspace Code Prover
TM
TM
Sept.
2015
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933
TM
Report
Documentation
4 Certification
4.1
Standards
Standard
Description
IEC 61508-3:2010
IEC 61508-7:20101
Functional Safety of electrical/electronic/programmable electronic safetyrelated systems Part 7: Overview of techniques and measures
EN 50128:2011
ISO 26262-8:2011
4.2
Basis of certification
1 Informative
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933
5 Results
5.1
The software development and quality engineering processes applied for Polyspace tools have
been audited, no objections were found.
To ensure adherence to the software development and quality engineering processes, as well as to
keep track of quality improvements, the processes to implement enhancements and modifications
are audited once a year by TV SD.
Product versions that are released in between two consecutive audits are subject to a defined approval procedure by TV SD. The procedure includes the following elements:
5.2
The MathWorks, Inc. documents new customer visible features for each release in the corresponding release notes.
The MathWorks, Inc. documents enhancements and new features of each Polyspace tools
version in an internal delta report.
Test procedures for enhancements and new features are referenced in the delta report to
document MathWorks internal validation activities for newly developed features.
MathWorks reports known critical bugs brought to its attention on its bug report system at
http://www.mathworks.com/support/bugreports/. The bug reports are an integral part of the documentation for each release.
The bug report system provides an interface for customers to view and submit bug reports. Customers can track the status of open bugs. Customers can choose to receive notifications for new or updated bug reports. The bug reports on this web site include internally as well as externally nominated bugs. If applicable, bug reports include provisions for known workarounds or file replacements.
Customers can use the bug report mechanism to nominate bugs. These nominations are processed
and evaluated by The MathWorks, Inc. development organization.
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933
5.3
5.3.1 General
ISO 26262, IEC 61508 and EN 50128 in their current versions contain explicit requirements on software tools.
They strongly recommend the application of development tools in software development. At the
same time, they demand to perform an analysis of the tools used, and an analysis on how they are
embedded in the development process:
analysis of tool usage (IEC 61508)
analysis of tool use cases (ISO 26262)
analysis on the effect of possible malfunctions of the applied tool(s).
Depending on the outcome of the above analysis, the standards referred to above demand
a) fault mitigation measures (process)
b) the qualification, respectively validation of tools.
These activities should complement each other, and the combination of both shall reduce the number of faults impacting the final product to a minimum.
Consistency analysis of the data flow (such as checking if a data object is interpreted everywhere as the same value);
Interface analysis (such as investigation of variable transfer between various software modules);
Dataflow analysis to detect suspicious sequences of creating, referencing and deleting variables; and
The Polyspace tools help supporting the above listed activities as outlined in the release-specific
Verification Workflow documentation according to the tables in Section 3.
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933
5.4
Polyspace tools can be integrated with other tools from The MathWorks, Inc. (cf. IEC
61508-3, 7.4.4.2, Note 3). A representative combination of tools is tested at the manufacturers site (cf. IEC 61508-3, 7.4.4.9, 7.4.4.18 a).
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933
5.5
The tool documentation for Polyspace tools (cf. IEC 6158-3, 7.4.4.4) is provided with the
products.
Each release of the tools is identifiable (cf. IEC 61508-3, 7.4.4.15 a).
MathWorks reports critical known bugs brought to its attention on its bug report system at
http://www.mathworks.com/support/bugreports/ (cf. IEC 61508-3, 7.4.4.6, Note 1).
The Release Notes provide the version history for Polyspace tools. Tool users can assess
available bug reports for different tool versions via the bug reports system (cf. IEC 61508-3,
7.4.4.6, Note 1).
The MathWorks, Inc., as well as 3rd party vendors, offer training courses for MathWorks tools
(cf. IEC 61508-3, 7.4.4.2, Note 6).
The MathWorks, Inc. developed a validation suite for Polyspace tools. The application of the
validation suite helps to uncover potential bugs in these products. Passing the validation
suite provides a high degree of confidence that potential bugs in Polyspace tools can be detected.
Test procedures for enhancements/new features are referenced in the delta report to document the Mathworks internal validation activities for newly developed features. The
MathWorks, Inc. validated Polyspace tools and provided documentation of this validation to
TV SD for review and approval (cf. IEC 61508-3, 7.4.4.6, 7.4.4.7).
EN 50128
EN 50128:2011 is an application standard derived from IEC 61508. The requirements for software
tools are explicitly derived from the requirements on software tools according to IEC 61508-3:2010.
Due to the equivalences between the two standards no separate testing has been performed with
respect to EN 50128.
The Polyspace tools are suitable to be used in the development of safety-related software according to EN 50128:2011 up to SIL 3/4. Tool certification for the Polyspace products for C/C++ versions listed in the above table can be claimed by referencing this certification report and the corresponding certificate.
5.6
5.6.1 General
The tool classification of the Polyspace tools according to ISO 26262 can be derived from the usecases described in 5.3.3.
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933
Based on these use cases, the maximum tool impact for the Polyspace tools is TI2, because
Polyspace products for C/C++ can fail to detect existing errors in the source code to be analyzed.
They may not introduce errors into an application.
TI2 requires an estimation of the tool error detection TD.
Conceptually, potential errors or malfunctions in verification tools such as the Polyspace tools can
be divided into three categories:
1) Non interference: The verification tool contains an error, but the software to be analyzed does
not invoke the erroneous portion of the tool.
2) False alarm (false positive): A verification tool error marks software to be analyzed as faulty,
although the software to be analyzed is correct
3) Masked error (false negative): A verification tool error leaves an error in the software to be analyzed undetected.
Errors belonging to the first two categories are considered as non-critical. Non interference errors do
not influence the correctness of the verification result. False alarm errors result in unnecessary verification efforts.
Only masked errors are considered as critical with respect to functional safety. However, Polyspace
tools are verification tools that support the static analysis of software for safety-related applications.
One goal of this static analysis is to reduce the number of errors in the software prior to dynamic
verification (i.e. dynamic testing).
A full list of the errors and malfunctions that were taken into account for the tool classification are
listed in the version-specific Tool Qualification package document according to the tables in section 3.
Existing runtime errors in the source code which are not detected by Polyspace products for C/C++
because of malfunctions or erroneous output of Polyspace may be uncovered by preceding or subsequent dynamic analysis. The detection rate depends on the application of preceding or subsequent dynamic analysis activities as a primary means to detect the presence of runtime errors.
If dynamic verification activities are applied with this objective, then the resulting error detection rate
is considered to be medium, resulting in TCL2 for this case.
If dynamic verification activities are not applied with the objective of detecting runtime errors, then
the resulting detection rate is considered to be low, resulting in TCL3 for this case.
A conservative estimate is given by the worse scenario where dynamic tests do not aim to detect
runtime errors. Therefore, the resulting overall error detection is considered to be low, i.e. the minimum tool error detection is TD3. TD3 results in a maximum tool confidence level of 3 (TCL3).
The spectrum of tool error detection levels and tool confidence levels resulting from the tool classification analysis of all use cases is documented in the version-specific Tool Qualification package
document according to the tables in section 3. Required tool confidence levels for the various use
cases range from TCL1 to TCL3.
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933
To address the identified range of tool confidence levels, the Polyspace tools versions listed in section 5.6.4 have been qualified up to a maximum tool confidence level of TCL3 by using a combination of following methods for tool qualification:
The MathWorks, Inc. documents customer visible features of each release in the corresponding release notes. The release notes were submitted to TV SD.
Since R2008a, The MathWorks, Inc. documents enhancements and new features of each
Polyspace tools version to be qualified in a comprehensive delta report. The delta reports were submitted to TV SD.
Since R2008b, The MathWorks, Inc. provides a tool validation suite for the Polyspace
tools. A successful validation using this validation suite is considered as a means of endto-end validation of the Polyspace tools. The validation reports were submitted to TV
SD.
Since R2008a, test procedures for enhancements and new features are referenced in the
delta report to document The MathWorks, Inc. internal validation activities for newly developed features.
5.6.4 Summary
All Polyspace tools versions listed in the subsequent table are qualified for all ASILs according to
ISO 26262 up to a maximum tool confidence level of TCL3. The review of the tool classification and
the assessment of the results of the measures applied to qualify the software tools were carried out
by TV SD.
If Polyspace products for C/C++ are used in accordance with the use cases mentioned in section
5.3.3, tool qualification for the Polyspace tools can be claimed for a maximum tool confidence level
of TCL3 by referencing this certification report and corresponding certificate.
When developing safety-related software, it is recommended to follow the guidance on how to use
Polyspace tools laid out in the release-specific Verification Workflow documentation according to
the tables in Section 3. Following this guidance helps to prevent errors caused by improper usage of
Polyspace tools and increases confidence that the results of Polyspace tools are interpreted correctly.
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933
Qualification Methods
Release
Maximum
TCL
Evaluation of the
development process
ASIL
Validation of the
software tool
Surveillance
Audit
Release Notes
Delta
Report
Validation
Suite
R2008a
A, B, C, D
R2008b
A, B, C, D
R2009a+
A, B, C, D
R2009b
A, B, C, D
R2010a
A, B, C, D
R2010b
A, B, C, D
R2011a
A, B, C, D
R2011b
A, B, C, D
R2012a
A, B, C, D
R2012b
A, B, C, D
R2013a
A, B, C, D
R2013b
A, B, C, D
R2014a
A, B, C, D
R2014b
A, B, C, D
R2015a
A, B, C, D
R2015b
A, B, C, D
5.7
IEC 62304
IEC 62304:2006 provides a framework of life cycle processes for the safe design and maintenance
of medical device software.
IEC 62304 does not place specific requirements on software tools, or on the qualification of tools,
but IEC 62304 advises that IEC 61508 can be looked to as a source of methods, tools and techniques that can be used to implement the requirements in IEC 62304 (IEC 62304:2006, C.1).
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933
It is highly recommended to apply the dynamic verification techniques according to the functional safety standard used and the risk level claimed. Due to the tool qualification and the
formal approach of Polyspace tools the amount of test coverage may be reduced.
The certificate Z10 13 06 67052 012 replaces the previous certificates Z10 08 08 67779 001, Z10 09
07 67052 003, Z10 09 07 67052 006, Z10 11 06 67052 011, and Z10 11 12 67052 015.
Munich, 2015-05-29
Technical Certifier
Peter Wei
TV SD Rail GmbH
Embedded Systems
Barthstr. 16
80339 Munich
Phone: +49 89 5791-4378; Fax: -2933