italy.ni.

com

The NI approach in the verification and validation of embedded systems: Real-Time Test
Javier Gutierrez

Business Development Manager, Europe

ni.com

Verification & Validation

Verification Validation
Requirement s
Plant Modeling

Are we building the product right. Are we building the right product.
Validation (reuse plant model)

Field Test

Model-in-theLoop

Verification (real plant) Lab Test Rigs Vehicles

Physical Testing

Rapid Control Prototyping

module test (model)

Hardware in the Loop (HIL)

Software-inthe-Loop

italy.ni.com

What is embedded software validation?

Check that a product meets initial requirements, specifications and regulations Often categorized into

Performance requirements Did it meet specifications? Functional requirements Did it work without error? Safety requirements Does it handle error conditions gracefully?

Many engineers inappropriately relegate this process to manufacturing test

Instead, perform validation during software development- far before hardware or manufacturing involvement
4

italy.ni.com

Why is software verification so important?

Required for certain markets and certifications

DO-178C ISO 26262

Increasing software content Your customers increasingly make buying decisions based on software features Integration of multiple technologies Margin enhancement through innovation Complexity

italy.ni.com

The holy grail

Oscilloscope for software Automated test system that traceably validates product requirements that is available at both the software and hardware phases of product development.

Automated We can run it overnight and after every build Traceable We have records and notifications of changes Validates product requirements We can technically test that all of our products requirements are covered Test at both the software and hardware phases of development Test is an integral part of software development before and after hardware targeting

italy.ni.com

Product selection

Automated test system that traceably validates product requirements that is available at both the software and hardware phases of product development.

Automated
o

NI TestStand NI Requirements Gateway NI VeriStand, NI LabVIEW, NI DIAdem NI VeriStand, NI Hardware Platform

Traceable
o

Validates product requirements

o

Test at both the software and hardware phases of development

o

italy.ni.com

Software Architecture

italy.ni.com

Traditional View of the Development Process

Physical Testing

Design

Prototyping

HIL Validation

Deployment

italy.ni.com

Testing Throughout the Process

Model-inthe-Loop Testing Physical Testing

Lab or Field Evaluation

HIL Validation

Software-inthe-Loop Testing

italy.ni.com

10

Test Continuity

Simulated Real
Stimulus, Data Logging, User Interface, Analysis
italy.ni.com
11

Test & Design Componentsonce, right, smarter

Requirements

Test Profiles & Automation

User Interfaces

Models
italy.ni.com

Analysis
12

Reports

LearSystemsValidationResults
Reusability of Test Cases
Innovation,NPDandComponentIntegration UnitModel Testing Integration Testing ProductionlineandIPC Vehicle Testing

ECUModule Testing(HIL)

80%oftestcases canbereused

100%oftestcases canbereused

60%oftestcases canbereused Morevehicle leveltestcases areaddedto reusedtestcases

Newtestcases arecreatedfrom requirements

Moretestcases areaddedto reusedtestcases

Moretestcases areaddedto reusedtestcases

Increasedreuseleadstohigherqualityandlowercost
UsedbypermissionofLearCorporation

LearSystemsValidationResults
High Quality Deliveries

UnitModel Testing

Integration Testing

ECUModule Testing(HIL)

Vehicle Testing

Field Issues

25%of issuesfound

25%of issuesfound

45%of issuesfound

5%of issuesfound

0%of issuesfound

Numerousproductshavehad0warrantyrelatedtofunctionalissues. Informationandoutcomesfromsoftwaretestingsustainandsupports Learsproductioncycle.

UsedbypermissionofLearCorporation

Embedded Software Development Process

Field Test
Requirements Gathering

Calibration

Model Simulation
+ -

Controller

Plant
+-

Controller

Plant

Physical Testing Test Cells

Rapid Prototypin g Code Generation and ECU Flashing

italy.ni.com
15

Hardware in the Loop Testing (HIL)

PUMP ECU
Pressure valve values and power voltage

[AI]

Unit Enabled

[DI]
Low and critical pressure feedback

[CAN] [AO]
Unit alive
italy.ni.com
16

Model In the Loop (MIL) Manual Testing

Requirements

italy.ni.com

17

Requirements

Requirement s

The test system will be used to validate an engine controller model with different environmental parameter values.

REQ_PR002_Rapid_PressureChange

REQ_PR001_Slow_PressureChange

Exercise the Pressure Unit with the rapid pressure change profile.

Exercise the Pressure Unit with the slow pressure change profile.

REQ_SG002_Rapid_PressureChangesDet ected

REQ_SG001_Slow_PressureChangesDetec ted
o

Check if all signals fit into range for rapid pressure change.

Check if all signals fit into range for slow pressure change.

REQ_PR003_Normal_PressureChange
o

Exercise the Pressure Unit with the normal pressure change profile.

REQ_SG003_Normal_PressureChangesD etected

Check if all signals fit into range for normal pressure change.

italy.ni.com

18

Model In the Loop (MIL)

Requirements

Models

italy.ni.com

19

Supported Modeling Environments

Supported

The MathWorks, Inc. Simulink software LabVIEW LabVIEW Control Design and Simulation Module NI MATRIXx SystemBuild Esterel SCADE Suite C/C++/FORTRAN/Ada MapleSim models from Maplesoft SimulationX from ITI GT-POWER engine models from Gamma Technologies Inc. Tesis DYNAWare models
italy.ni.com
20

Dymola models from Dynasim AMESim models from LMS

In Work CarSim from Mechanical Simulation AVL BOOST/CRUISE WaveRT Models from VI-grade Simulink is a registered trademark of The MathWorks, Inc. All Easy5 other trademarks are the property of their respective owners.

Model In the Loop (MIL)

Requirements

Models

Stimulus Profile

italy.ni.com

21

Model In the Loop (MIL)

Requirements

Models

Stimulus Profile

Analysis

italy.ni.com

22

Model In the Loop (MIL)

Requirements

Models

Stimulus Profile

Analysis

italy.ni.com

23

Model In the Loop (MIL) Automated Testing

Automation
Models Stimulus Profiles Analysis

Requirements

italy.ni.com

24

What is ATML?

ATML = Automatic Test Markup Language Driven by the Department of Defense, Industry Standardized as IEEE 1671, IEEE 1636.1, IEEE 1232 Common exchange format for all test and maintenance information

italy.ni.com

25

The Goals of ATML

Interoperability of Test System Components Standardize documentation and reporting format Require common component interfaces Documentation of Test Systems Define schemas describing components Require documentation of system characteristics

italy.ni.com

26

ATML Will

Enable integrated multi-vendor COTS-based solutions Support improvement of diagnostic accuracy Reduce repair time and cost Facilitate ATE technology insertion and TPS rehost

italy.ni.com

27

Scripting Real-Time Tests

Scripting Automation
Requirement s

Models

Stimulus Profiles

Analysis

italy.ni.com

28

Test Process Flow by use of ATML

Test Station

Requirements

Test Case

Test Station

Test Results

Test Description
UUT

Test Result

Process can be inherently automated through ATML Schemas

UUT Description

italy.ni.com

29

Embedded Software Development Process

MIL Testing
Requirements Gathering

HIL Testing

Field Test Calibration

Model Simulation
+ -

Controller

Plant

Controller

Plant

+-

Physical Testing Test Cells

Rapid Prototyping

Hardware in the Loop Testing (HIL)

Code Generation and ECU Flashing

italy.ni.com

30

Test Component Re-use Benefits

Re-use provides correlation of test procedures and results across all development phases Continuity of test components reduces test application development and maintenance costs Common test components and interfaces create operational efficiency throughout the process

italy.ni.com

31

Certifying Safety Critical Systems

IBM Rational and National Instruments are teaming together to provide an end-to-end quality management solution.

Traceability all the way to test and back Collaboration between ALL teams, including test Quality and test considered from the outset, not an afterthought, reducing cost and risk of identifying and correcting defects Test component re-use throughout project phases and between projects providing operational efficiency and accuracy

italy.ni.com

32

Why do customers choose NI?

Integrated hardware & software platform Increased efficiency from testing throughout the design process Easily reconfigurable modular test systems Greatest variety, value, & availability of commercial off-the-shelf (COTS) products

italy.ni.com

33

Questions?

Web Resources ni.com/veristand ni.com/hil ni.com/realtimetest javier.gutierrez@ni.com

italy.ni.com

34

italy.ni.com