Professional Documents
Culture Documents
www.mathworks.com
www.mathworks.com/sales_and_services
User community:
www.mathworks.com/matlabcentral
Technical support:
www.mathworks.com/support/contact_us
Phone:
508-647-7000 (Phone)
IEC Certification Kit: Simulink Test ISO 26262 Tool Qualification Package
Revision History
September 2015
New for IEC Certification Kit Version 3.6 (Applies to Release 2015b)
Contents
1 Introduction ...................................................................................................................................... 1-1
1.1 Application Identification........................................................................................................ 1-2
1.2 Tool Overview and Identification ........................................................................................... 1-3
1.3 Tool Qualification Artifacts Summary .................................................................................... 1-4
2 Software Tool Criteria Evaluation Report........................................................................................ 2-1
2.1 Tool Environment ................................................................................................................... 2-2
2.2 Tool Configuration .................................................................................................................. 2-3
2.3 Reference Workflow ............................................................................................................... 2-4
2.4 Tool Use Cases ........................................................................................................................ 2-5
[SLTEST_UC1] Development and execution of tests for Simulink models ................................ 2-5
[SLTEST_UC2] Development and execution of tests for back-to-back testing between model and
code .............................................................................................................................................. 2-5
[SLTEST_UC3] Assessment of test results ................................................................................. 2-5
[SLTEST_UC4] Generation of test reports .................................................................................. 2-5
[SLTEST_UC5] Identification of traceability between requirements and tests cases.................. 2-6
2.5 Generic Tool Classification ..................................................................................................... 2-7
2.5.1
Potential Malfunctions and Erroneous Output ............................................................... 2-7
[SLTEST_E1] Incorrect behavior of test harness ........................................................................ 2-7
[SLTEST_E2] Incorrect run of test procedure ............................................................................. 2-7
[SLTEST_E3] Erroneous assessment of test results false negative .......................................... 2-7
[SLTEST_E4] Erroneous assessment of test results false positive ........................................... 2-7
[SLTEST_E5] Generation of erroneous test report ...................................................................... 2-7
[SLTEST_E6] Usage of incorrect input data ............................................................................... 2-7
[SLTEST_E7] Incorrect Tool Usage............................................................................................ 2-8
[SLTEST_E8] Incorrect or Modified or Incompatible with Environment Tool Installation........ 2-8
2.5.2
Error prevention and Detection Measures ..................................................................... 2-8
[SLTEST_M1] Requirements-based testing ................................................................................ 2-8
[SLTEST_M2] Tool installation integrity checks ........................................................................ 2-8
[SLTEST_M3] Configuration management ................................................................................. 2-8
[SLTEST_M4] Input data integrity checks .................................................................................. 2-8
[SLTEST_M5] Competency of project team ............................................................................... 2-8
[SLTEST_M7] Manual comparison of test results to expected results ........................................ 2-9
[SLTEST_M8] Manual review of test report content .................................................................. 2-9
2.6 Tool Classification Summary ................................................................................................ 2-10
3 Software Tool Qualification Report ................................................................................................. 3-1
3.1 Requirement for Tool Qualification ........................................................................................ 3-2
3.2 Tool Qualification Documentation .......................................................................................... 3-3
4 Confirmation Review of Tool Classification and Qualification ....................................................... 4-1
4.1 Requirement for Confirmation Review ................................................................................... 4-2
4.2 Validity of Generic Tool Classification .................................................................................. 4-3
4.3
4.4
vi
vii
1 Introduction
This document constitutes the ISO 26262 Tool Qualification Package for the Simulink Test
product. This document is intended for use in the ISO 26262 tool classification and qualification
process for software tools. It contains templates for the ISO 26262 tool qualification work
products (see ISO 26262-8, Clause 11).
The applicant shall review this template for applicability to the application under consideration,
and tailor and complete the information.
See also:
ISO 26262-8, Clause 11 provides provisions for software tools that are used to tailor activities or
tasks required by ISO 26262. The standard outlines a two-step approach to establish the
required confidence in the tools:
Tool classification determines the required level of confidence in the software tool.
Depending on the result of the tool classification, you might need to carry out a formal
tool qualification.
The following work products need to be created when applying this approach to a software tool
(see ISO 26262-8, 11.5):
Note The applicant needs to review this template for applicability to the project under
consideration and insert missing information.
<Insert information>
1-2
Version (Release)
Tool Vendor
Simulink
Test
1-3
for the Simulink Test product. The table also maps these tool qualification artifacts to sections in
this document and artifacts found elsewhere.
Tool Certification Artifact
Safety plan
<Insert ASIL>
1-4
Software tool criteria evaluation Customized and completed section Software Tool Criteria Evaluation
report
Report in the Simulink Test ISO 26262 Tool Qualification Package
(this document)
certkitiec_sltest_tqp.docx
Simulink Test Reference Workflow
R2015b
certkitiec_sltest_workflow.pdf
Certificate Z10 15 06 67052 016
June 2015
certkitiec_sltest_certificate.pdf
Report to the Certificate Z10 15 06 67052 016
June 2015
certkitiec_sltest_certreport.pdf
Software tool qualification
report
Confirmation review of
qualification of a software tool
1-5
1-6
2-2
Setting
Yes
2-3
2-4
2-5
2-6
2-7
2-8
2-9
Use cases
Prevention /
detection
measures
TD
[SLTEST_E1]
[SLTEST _UC1] TI2 Incorrect behavior of test
Incorrect behaviour [SLTEST _UC2]
harness could prevent
of test harness
errors in an object under
test from being detected.
[SLTEST _M1]
Requirements-based
testing
[SLTEST_E2]
[SLTEST _UC1] TI2 Incorrect run of test
Incorrect run of test [SLTEST _UC2]
procedure could prevent
procedure
errors in an object under
test from being detected.
[SLTEST_E3]
[SLTEST _UC3] TI1 Nuisance only, failed tests
Erroneous
have to be manually
assessment of test
reviewed and explained by
results passed test
user
indicated as failed
[SLTEST_E4]
[SLTEST _UC3] TI2 Incorrect assessment of
Erroneous
test results could prevent
assessment of test
errors in an object under
results failed test
test from being detected.
indicated as passed
[SLTEST _M1]
Requirements-based
testing
TCL1
None
TD3 -
TCL3
[SLTEST_E5]
Simulink Test
produces erroneous
test report which
doesnt correspond
to the actual test
data
[SLTEST_E6]
Usage of incorrect
input data
None
Justification for TI
Justification for TD
TCL
[SLTEST _M7]
TD1 Manual comparison of test results to TCL1
Manual comparison of
expected results can verify that results
test results to expected
have been correctly assessed by the
results
tool.
TD3 -
TCL3
[SLTEST _M8]
TD1 Manual review of test report content
Manual review of test
can verify that report has been
report content
correctly generated by the tool.
[SLTEST_E7]
All
Incorrect tool usage
[SLTEST _E8]
Incorrect or
Modified or
TI
All
TCL1
[SLTEST _M3]
TD1 Revision control and configuration
TCL1
Configuration
management facilitate integrity of the
management
input data. Using checksums allows
[SLTEST _M4]
the unique identification the input
Input data integrity
data.
checks
TI2 Incorrect tool usage could [SLTEST _M5]
TD1 Training of tool users can prevent
TCL1
prevent errors in an object Competency of project
these issues.
under test from being
team
detected.
TI2 Incorrect tool installation [SLTEST _M2]
TD1 Verification of the installed tool
TCL1
may lead to incorrect test Tool installation
version will detect invalid tool
run could prevent errors in integrity checks
installation.
2-10
Potential
malfunction or
erroneous
output
Incompatible with
Environment Tool
Installation
Use cases
TI
Justification for TI
Prevention /
detection
measures
TD
Justification for TD
TCL
Based on the preceding analysis, the maximum tool impact of the Simulink Test use cases taken
into account is TI2.
Subsequent use of error detection measures [SLTEST _M7] and [SLTEST _M8] provides high
degree of confidence that tool malfunctions SLTEST_E4 and SLTEST_E5 will be detected.
Therefore the tool confidence level for the capabilities implementing the corresponding use
cases SLTEST_UC3, SLTEST_UC4 and SLTEST _UC5 is TCL1. If no measures are applied
the tool confidence level is TCL3.
For the capabilities implementing use cases SLTEST_UC1 and SLTEST_UC2 the tool
confidence level is TCL1 provided the prevention/detection measures identified in the table
above are taken.
TV SD reviewed the generic tool classification and confirmed the results in Report to the
Certificate Z10 15 06 67052 016.
2-11
3-2
Evaluation of the tool development process (ISO 26262-8, Table 4, Method 1b).
Validation of the software tool (ISO 26262-8, Table 4, Method 1c).
According to ISO 26262-8, table 4, these two methods are permissible for all ASILs. Method 1b
is highly recommended for ASILs A, and B. Methods 1c and 1d are highly recommended for
ASIL D.
Tool qualification for the corresponding capabilities of the Simulink Test product can be
claimed for TCL1 and TCL3 by referencing the certification report and corresponding
certificate.
TV SD carried out an independent tool qualification assessment. MathWorks submitted the
results of the methods applied to pre-qualify Simulink Test to TV SD.
TV SD reviewed the generic tool qualification artifacts for Simulink Test and confirmed the
results in Report to the Certificate Z10 15 06 67052 016.
3-3
3-4
4-2
4-3
4-4
4-5