Verifiaction Planing

Testbench Architecture & Implementation with SV-OVM Module #1: Verification Planning
Michael A. Warner
Worldwide Consulting Manager
Agenda
Verification Planning Testbench Architecture Testbench Implementation
SystemVerilog Basics OOP with SystemVerilog OVM Introduction
Confidential Information of Mentor Graphics
03/04/2010
Copyright 2010 Mentor Graphics Corp
Agenda
Why Plan?
It takes 50-80% of the total development effort to insure the intent of the specification is preserved in its implementationis that reasonable?
Area Power HDL Debug DFT Timing PD
(Automated) Design Path
Specification
Requirements Verification Path VAD Coverage Closure VID (Manual) HDL Debug Review Triage HVL Regression HVL Mgmt Debug
Tape-Out or Tap-Out
03/04/2010
Setting Yourself Up For Success

Thoroughly Understand Your Design Requirements Architect Before Coding Design for Reuse, Maintainability and Debugability
Area Planning Power HDL Debug DFT Timing PD
(Automated) Design Path
Specification
DRs VAD VID Review HVL
5
Verification Path (Manual)

Regression Mgmt Triage
Tape-Out or Tap-Out
HVL Debug
Coverage Closure HDL Debug
Key Concepts
Misaligned expectations between mgmt and engineering is the leading cause of pain and suffering in verification Aligning expectations with mgmt is especially difficult with verification given its nondeterministic and open-ended nature Managing expectations is a little easier when you have a proven planning process based on experience, history and science
03/04/2010
Scheduling: Bottoms-Up Planning

Bottoms up is the typical way to do scheduling Each person is assigned a chunk of code, does a I can write that code in , then doubles it to take into account debugging that code once written Various forces (e.g. culture, experience, personality) can create extreme estimates identify and normalize Some are better at estimating than others expect variability in quality of estimates
Scheduling Saboteurs
Convolutes schedule with details Wont commit without all the facts Backs up estimated with lots of data
All tasks are simple and easy Schedules are vague at best No basis for estimates
He tells you what he thinks you want to hear He is afraid of not pleasing others He is a people pleaser
Uses FUD to achieve goal Tremendous vector generator Little appreciation for the engineering process
03/04/2010
Scheduling: Need For Confirmation

Validate bottoms-up plan with tops-down estimates
Tops Down Estimates
External Influences
Strategy
Bottoms-up Estimates Bottoms-
Gap Reconciliation
Schedule Reviews
Plan Of Record
Staffing Plan
Tops Down Scheduling: Key Observations

Every verification campaign IS a large scale software development treat it as such Theres empirical data that shows a strong correlation between the number of bugs in your design and the number of gates plan accordingly Theres a relationship between your verification strategy and the total effort of your verification campaign choose wisely
03/04/2010
What is a Verification Plan?

A document describing the methodology, environment, resources, requirements, priorities and success criteria of intended verification activities.
Specifications
Strategy
Extraction Prioritization Plans Plans Plans Metrics Processes Implement Closure
Requirements
Start
Time
End
Strategy
This is how we go about doing verification in general These are our goals, tools, technologies and processes Specifications
Strategy
Extraction Prioritisation Plans Plans Plans Measure Processes Implement Closure
Requirements
Start
Time
End
03/04/2010
Strategy: Tools
Custom Other Tools Matlab Linters Static Tools Simulink Excel
Manual Review Property Checkers
Simulation Based Tools
Emulators Simulators Accelerators
Strategy: Methodology
Testbench
OVM
Home Grown
Assertions Golden model Checking Stimuli
Directed System models
Distributed checkers Random Directed
03/04/2010
Strategy: Language
SystemC C VHDL
Perl Assembler Verilog
TCL
SV (assertions) SV (design) SV (verification)
Requirements
This is what we need to verify in a particular design These are the features we need to hit and how important they are Specifications
Strategy
Requirements
Start
Time
End
03/04/2010
Requirements: Extraction
Identify your sources of information
Decompose your design into manageable blocks
Extract requirements
Requirements: Sources of Information

Where should you go to get your requirements? Functional specification is the obvious choice
although be aware that the required information for one feature might come from several places
In theory, it contains all you need
Functional Specification
03/04/2010

How many functional specifications though?
Conversations Referenced documents Compliance Specification Design specification Delta Functional Specification Base Functional Specification

What about implementation details?
the functional specification might not mention the architecture used in a block, but it still has to be verified to check corner case handling
FIFO full flags Counter wraparound Local shared buffer arbiters
Code review Implementation Specification
03/04/2010
10
Requirements: Block Level

Possible subsections
Major and Minor Modes of operation Exception handling Interfaces Registers Interrupts Implementation
Useful places to look

The specification headers, module key features, register fields, configuration parameters Interrupts, status bits, illegal inputs, what if questions Pin list, module key features, timing diagrams The register map The register map, pin list, the module key features, status bits The implementation specification, the RTL, the designer
Requirements: Implementation
Finite State Machines
Registers

encoding (one hot or only legal values) enters reset state on reset
Read-only bits can never be written reset to defaults updated correctly from bus
Interfaces

Arithmetic under- or over-flow Signals remain mutually exclusive Clock synchronisation blocks
usage assumptions not violated outputs become disabled during reset
FIFOs and stacks
full and empty flags used correctly
03/04/2010
11
Requirements: Read Between the Lines

The specification will not have all the details

learn to read between the lines what wasnt mentioned that should have been? what feasible error conditions should be dealt with?
When the transfer is completed, the DMA engine will assert an interrupt (if enabled) or go to the idle state. If the auto restart bit (ARS) is set, it immediately restarts the operation What if interrupts are enabled and ARS is set?
actually, the specification doesnt mention if the interrupt needs to be cleared to start a channel experience says that this might be a reasonable thing to do
Plans
This is exactly how we will verify a particular design The plan(s) contain the specific steps Specifications
Strategy
Requirements
Start
Time
End
03/04/2010
12
Plans: Verification Requirements

Requirements
Behavioural Analysis Derive required behaviour from the requirements
DUT
Mapping
Testbench
Plans: What is a verification requirement?

A verification requirement is the systematic set of checks, coverage points and/or stimuli corresponding to a specific design requirement Its something about the design that you want to stimulate, check and/or cover before youd be happy saying that the design has been verified Extracted before verification planning or implementation begins Requirements typically have a one-to-many relationship
03/04/2010
13
Plans: Example
The serial receive block has four buffers. The block checks for the parity and validity of the data frame on the RXD input and then writes correct data into its buffers. Check that RXD data is being properly written into buffers taking into consideration the parity and validity of the data. Using Data = 5 bits, Data = 6 bits, Data = 7 bits, Data = 8 bits Using Parity = OFF, Parity = EVEN, Parity = ODD Using Stop Bits = 1, Stop Bits = 2 Requirement
Check
Functional Coverage
Data[1:8] Parity[1:3] Stop Bits[1:2] Stall[0:10] Error[0:1]
Constraints
Plans: Behavioural Analysis

To map each requirement to a checker or an assertion
and to understand what is required from the testbench to do so
To map each requirement to functional coverage
and to understand what is required from the testbench to do so
To identify extra testbenches, actors or components that are required To cross link everything to provide traceability from the functional specification to the verification code
03/04/2010
14
Plans: Behavioural Analysis

For each requirement
Refine until the level of detail is correct
Analyse to determine testbench requirements
Decide on implementation approach
Plans: Refining your Requirements

Get the level right

too high and you dont really know what youre checking too low and youre checking things that the specification didnt specify, or maintaining too many requirements
Reword to make explicit

Check that the arbiter selects the correct channel instead of Check that the arbiter works or channel arbitration
Accuracy

avoid cycle accurate checks avoid verifying what you dont have to
03/04/2010
15
Plans: What level?

Requirement
Check that the DMA works Check that data is transferred correctly by the DMA Check that data is fetched from the correct source address Check that data is sent to the correct destination address Check that the sent data is equal to the fetched data Check that data is fetched from the specified source address (for the first transfer in a block) and then from incremented addresses for subsequent transfers in a block when INC_SRC is 1
3/4/2010 Page 31
Plans: Verification Requirements to Testbench Architecture

To determine a general structure for a testbench that can exercise this DUT To build a list of verification components and actors that you will need
Scoreboard
Test Controller
Coverage
Monitor
Monitor
Stimulus
Driver
TLM DUT RTL DUT
Responder
Slave
RTL
Abstraction Conversion
Transaction Level
03/04/2010
16
Plans: Finding Actors

Start with a block diagram of the DUT in a system
the DUT could be the entire design, a cluster or a single block
You will need an External Actor to interact with each DUT interface If you plan to just do black box verification, then these are probably all the actors you need If you are doing white or grey box testing, then you will need actors for interfaces within the DUT
Plans: External Actors

Configuration Interface Interrupt Handler Register Handler
Wishbone Slave Wishbone Master
Interrupts
Wishbone Master Wishbone Slave
Wishbone Master Wishbone Slave
03/04/2010
17
Plans: Internal Actors
Wishbone Wishbone Slave Slave
Interrupts Interrupts Arbiter
Wishbone Wishbone Master1 Master
Wishbone Wishbone Master Master 2
Arbiter
Plans: TLM Data Structures

For each actor, think about the transactions that will pass between

it and the DUT it and other actors its internal components Register Transaction Address[] Data[] Configuration Transaction DMA mode Channel Number Priority Src Address Dst Address Size Monitor Transaction Status Channel Number Priority #Bytes so far #Bytes to go Data[]
Bus Transaction Address Data Burst Type Size
03/04/2010
18
Plans: Mapping Deliverables

You should now have a good idea of the Actors you need

and therefore the sub-components per actor and the transactions you need
and a rough idea about the topology of the testbench The detail is missing though, and for that, you need to map results of behavioural analysis onto actors
Plans: Implementation
For each requirement, ranked by importance
and any required monitors, etc
Write the check and stimuli for it

Once all checks in your important list are implemented
For each requirement, ranked by importance
Implement functional coverage for it

03/04/2010
19
Agenda
39 Lexmark, June 25, 2007
Testbench Architecture
Scoreboard
Test Controller
Coverage
Monitor
Monitor
Stimulus
Driver
DUT
Responder
Slave
RTL
Abstraction Coversion
Transaction Level
03/04/2010
20
Random Generation Engine

Config
class rand_word { ... constraint interesting_values; constraint address_0_is_directed; function new() { address_0_is_directed.constraint_mode(OFF); endfunction endclass class generator; rand_word data; ... endclass
Data Model
Generation Engine
FLOW MODEL:
CFG1
CFG2
CFG3
TRAFFIC
Captures the flow of the chip
Normal
Long
Short
ERR
Const/Code
Diagnostic
Random Generation Engine

module GenEng (); initial forever begin : loop randsequence (main) main : cfg, traffic, diagnostic; cfg : { randcase cfg1: ; cfg2: ; cfg3: ; endcase }; // cfg traffic : { fork join_none }; // traffic diagnostic : {}; endsequence end // forever end /init endmodule
class rand_word { ... constraint interesting_values; constraint address_0_is_directed; function new() { address_0_is_directed.constraint_mode(OFF); endfunction endclass class generator; rand_word data; ... endclass
Data Model
Generation Engine
Control Model
Const/Code
03/04/2010
21
Scoreboard
Generator
BFM
DUT
Transfer Function
Monitor
Transfer Function

Available or Build? Language (C, Hvl)? Who Maintains? Cycle Accurate? Typical? Interesting?
Ordering
Data Structure
Compare
Scoreboard
LOG
Scoreboard Architecture
One big scoreboard that is a model of the entire chip Divide up into smaller, easier scoreboards daisy-chained together
Generator
BFM
BLK
DUT
BLK
BLK
Monitor
TF
DS
CMP
TF
DS
CMP
TF
DS
CMP
SB
SB
SB
03/04/2010
22
lenn ahc_ a md _gc : puo rgdne ... ;t c el e s _t s d _ p c ,t c el e s _ c r s _ p c s s o r c : t c el e s _t s d Xt c el e s _ c r s _ s s o r c ... ; s s e r d d a _ c r s _ c ni. rt t ni o p r e v o c : s s e r d d a _ c r s _ c ni _ p c ; e k a h s d n a h _ w h . rt t ni o p r e v o c : eka hsdn ah_ w h_pc ;t r a t s e r _ o t u a. rt t ni o p r e v o c : tr at s er _ ot u a _ p c ; yti r oi r p _ h c . rt t n i o p r e v o c : yti r o i r p _ h c _ p c ;t r at s e r _ w h. r t t ni o p r e v o c : tr at s er _ w h _ p c ; R O R R E _ e l b a n e _t ni. r t t ni o p r e v o c : R O R R E _ el b a n e _t ni _ p c ; E N O D _ el b a n e _t ni . r t t ni o p r e v o c : E N O D _ el b a n e _t ni _ p c ; Z S _ K H C _ e l b a n e _t ni. r t t ni o p r e v o c : Z S _ K H C _ el b a n e _t ni _ p c ;l e n n a h c _ a m d _ g c p u or gr e v o c
Channel 30 Channel 0 Channel Priorities Priority Arbiter M U X DMA Engine
03/04/2010
46 45 Channel 30 Channel 0 Channel Priorities

Priority Arbiter M U X Covering Stimulus & Response Covering Stimulus & Response
Coverage
Coverage
Testbench Monitors
Testbench Monitors
DMA Engine
Confidential Information of Mentor Graphics 46
interrupt
interrupt
Bus Interface 0
Bus Interface 0
Bus Interface 1
Bus Interface 1
handshake
handshake
23
Coverage Model
47 Channel Priorities Priority Arbiter Bus Interface 0
SVA
DMA Engine Channel 0 M U X Channel 30
SVA
Bus Interface 1 interrupt handshake
CG
SVA
SVA
SVA
Testbench Monitors
CG
Covering Stimulus & Response
Tying it all together

Design Specification

32-bit general purpose scalar processor 5 stage pipeline can experience stalls in any stage 16 x 32-bit general purpose registers Register forwarding Op-codes
Register based addition: add src1, src2, dst Register based addition with saturation: sadd src1, src2, dst nop
What would be a reasonable data model? What would be a reasonable control model? What would be a reasonable correctness model? What would be a reasonable coverage model?
03/04/2010
24
10 Minute Break
03/04/2010
25

Verifiaction Planing

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Verifiaction Planing

Uploaded by

Copyright:

Available Formats

Testbench Architecture & Implementation with SV-OVM Module #1: Verification Planning

Confidential Information of Mentor Graphics

Copyright 2010 Mentor Graphics Corp

Confidential Information of Mentor Graphics

(Automated) Design Path

Confidential Information of Mentor Graphics

Copyright 2010 Mentor Graphics Corp

Setting Yourself Up For Success

(Automated) Design Path

Verification Path (Manual)

Coverage Closure HDL Debug

Confidential Information of Mentor Graphics

Confidential Information of Mentor Graphics

Copyright 2010 Mentor Graphics Corp

Scheduling: Bottoms-Up Planning

Confidential Information of Mentor Graphics

Copyright 2010 Mentor Graphics Corp

Scheduling: Need For Confirmation

Tops Down Estimates

Bottoms-up Estimates Bottoms-

Confidential Information of Mentor Graphics

Tops Down Scheduling: Key Observations

Confidential Information of Mentor Graphics

Copyright 2010 Mentor Graphics Corp

What is a Verification Plan?

Extraction Prioritization Plans Plans Plans Metrics Processes Implement Closure

Extraction Prioritisation Plans Plans Plans Measure Processes Implement Closure

Copyright 2010 Mentor Graphics Corp

Custom Other Tools Matlab Linters Static Tools Simulink Excel

Manual Review Property Checkers

Simulation Based Tools

Emulators Simulators Accelerators

Confidential Information of Mentor Graphics

Assertions Golden model Checking Stimuli

Directed System models

Distributed checkers Random Directed

Confidential Information of Mentor Graphics

Copyright 2010 Mentor Graphics Corp

Perl Assembler Verilog

SV (assertions) SV (design) SV (verification)

Confidential Information of Mentor Graphics

Extraction Prioritisation Plans Plans Plans Measure Processes Implement Closure

Copyright 2010 Mentor Graphics Corp

Identify your sources of information

Decompose your design into manageable blocks

Confidential Information of Mentor Graphics

Requirements: Sources of Information

In theory, it contains all you need

Confidential Information of Mentor Graphics

Copyright 2010 Mentor Graphics Corp

Requirements: Sources of Information

Confidential Information of Mentor Graphics

Requirements: Sources of Information

Code review Implementation Specification

Confidential Information of Mentor Graphics

Copyright 2010 Mentor Graphics Corp

Requirements: Block Level

Useful places to look

usage assumptions not violated outputs become disabled during reset

FIFOs and stacks

full and empty flags used correctly

Confidential Information of Mentor Graphics

Copyright 2010 Mentor Graphics Corp