SystemVerilog Verification

Lecture 11
Coverage

1/28/2013

Krispan Inc. Confidential

Coverage types
Coverage is a generic term used to measure the progress of verifying a
design. Two types of Coverage Code and Functional
Code coverage:
Here you are measuring how many lines of code have been
executed(line coverage).
Which paths through the code and expressions have been
executed(path coverage)
Which states and transitions have been executed(FSM)
Single bit variable transitions from 0 or 1 (toggle coverage)
Most Simulators include a code coverage tool.
Code Coverage checks how accurately your tests exercised the
implementation of the design specifiacation and not the verification plan.
100% code coverage does not mean your design is completely verified
1/28/2013

Krispan Inc. Confidential

Code Coverage
The code coverage tool will not catch mistakes you make
Suppose your design implementation is missing a feature from the
design specification. Code Coverage will not catch it.
Suppose you forget to implement the reset logic in your flop. Code
coverage will not catch it. It will report that every line in the code is
exercised.
module dff (

output logic q,
input logic clk, d, reset_low);

always @(posedge clk or negedge reset_low) begin

q <= d;
end
endmodule
The reset logic was not implemented. Code Coverage tool will not catch it.

1/28/2013

Krispan Inc. Confidential

TestBench with Functional Coverage

Environment
Generator
Scoreboard
Driver

checker

Assertions

DUT

1/28/2013

Krispan Inc. Confidential

monitor

Functional Coverage

test
Test

Functional Coverage
Functional Coverage tells you how much of the design is exercised.
If the design is exercised completely for all possible scenarios then
the coverage is 100%.
Create a test plan with the list of tests that are needed from the
design specification.
Look at the coverage results, then decide on what actions one
needs take in order to achieve 100% coverage
First action is to run the existing tests with more seeds to see if you
can achieve 100% coverage.
Next step is to write more constraints and see if that fulfils the
cause.
Write Directed tests only if absolutely necessary
1/28/2013

Krispan Inc. Confidential

Coverage Convergence
For e.g. if we are looking at coverage metrics for processors
Did we cover all instructions
Did we cover all addressing modes
Did we cover all valid combinations of instructions and
addressing modes.

Functional coverage identifies tested and untested areas of

your design and makes sure you test for all
Corner cases
All states in a state machine and transitions are verified

Use a combination of seeds, constraints and directed tests if

needed to increase your coverage to 100%.
1/28/2013

Krispan Inc. Confidential

Functional Coverage
Verification Plan

Design Specification

design

Coverage
database

tests
Coverage
Analysis

Debug
pass

1/28/2013

Krispan Inc. Confidential

Functional Coverage
You can see from the previous diagram you test for pass/fail

of a test.
Pass gives you the coverage analysis
Fail has no significance
Functional Coverage data is discarded if the test failed due to design
bug.
The Coverage data gives you the verification metrics and gives you a
measure of how many items in the verification plan is complete which
is based on the design specifications
If the design does not match the specifications then the coverage
values have no meaning.
Analyze coverage with multiple runs with different seeds, more
constraints, and directed tests if needed to achieve the coverage goal.
1/28/2013

Krispan Inc. Confidential

covergroup
As we have seen before functional coverage starts with
writing of the verification plan from design specification
Then in SystemVerilog we write an executable version using
covergroup and cover points
covergroup construct is a keyword that encapsulates the
specification of a coverage model. It may include a clocking
event that samples values of variables and expressions which
are called cover points.
Cover groups are defined using the keyword covergroup and
endgroup.
A covergroup instance can be created using the new operator.
1/28/2013

Krispan Inc. Confidential

covergroup
Coverage model definiton
Encapsulated in the covergroup construct defined by the user
Clocking events for sampling variables or expressions which are cover
points . This is done using the keyword coverpoint
Cross coverage between coverage points using keyword cross
Optional coverage options
SystemVerilog functional coverage enables
Variable and expression coverage as well as cross coverage
Automatic and/or user-defined coverage bins
Filtering conditions (illegal values, values to ignore, etc.)
Automatic coverage sample triggering by events and sequences
Dynamic query of coverage information via built-in methods
1/28/2013

Krispan Inc. Confidential

10

covergroup
Syntax is
covergroup buslogic;
..
..
endgroup
buslogic bl;
bl = new;

An instance of buslogic covergroup bl is created by assigning

bl to new

1/28/2013

Krispan Inc. Confidential

11

covergroup example
program test1(busifc.TB ifc)
class BusTransaction;
rand bit [31:0] data;
rand bit [3:0] port;

//properties variables (data portion)

//random variables
//16 possible ports

endclass
covergroup portdata
coverpoint bt.port;
// Measure coverage
endgroup
initial begin
BusTransaction bt;
portdata pd;
pd = new();
// Instantiate group
bt = new();
repeat (32) begin
// Run a few cycles
assert(bt.randomize);
// Create a transaction
ifc.cb.port <= bt.port;
// and transmit
ifc.cb.data <= bt.data;
// onto interface
pd.sample();
// Gather coverage
@ifc.cb;
// Wait a cycle
end end endprogram
1/28/2013

Krispan Inc. Confidential

12

covergroup example
In this example a random transaction is created and driven to
the interface.
Testbench samples the value of the port field using the
portdata covergroup.
16 possible values, 32random transactions. Did your
testbench have enough coverage
vcs will generate a coverage report which gives you a
summary of your coverage.
If coverage does not satisfy your requirements then you can
rerun with a new seed value, add more constraints etc.

1/28/2013

Krispan Inc. Confidential

13

Sample of vcs output

Coverpoint Coverage report

CoverageGroup: portdata
Coverpoint:
bt.port
Summary
Coverage: 50
Goal: 100
Number of Expected auto-bins: 16
Number of User Defined Bins: 0
Number of Automatically Generated Bins: 7
Number of User Defined Transitions: 0
Automatically Generated Bins
Bin
# hits
at least
================================
auto[1]
7
1
auto[2]
7
1
auto[3]
1
1
auto[4]
5
1
auto[5]
4
1
auto[6]
2
1
auto[7]
6
1
================================

1/28/2013

Krispan Inc. Confidential

14

Sample of vcs output

This is just how the vcs output would look like. It is not a real
output.
You can see testbench generated the values for ports
1,2,3,4,5,6,7 but nothing was generated for
0,8,9,10,11,12,13,14,15.
The at least option specifies how many hits are needed
before a bin is covered.
The easiest way to improve coverage is to try to generate
more random transactions (more simulation cycles) or to try
new seed values.

1/28/2013

Krispan Inc. Confidential

15

Coverage bins
Bins can be created implicitly or explicitly
When you specifiy coverpoints if you do not specify any bins then
implicit bins are created.
In the example vcs file we were trying the coverage on each port
and since there are 16 ports it will automatically create 16 bins
The number of bins to be created can be controlled by using the
auto_bin_max parameter.
covergroup portdata
coverpoint bt.port;
{ option.auto_bin_max = 8; }
endgroup

In this case total bins created will be 8 half of 16 and 1 bin for
2 ports
1/28/2013

Krispan Inc. Confidential

16

Coverage Bins
Explicit bin creation is the preferred way. Not all cover points
are of interest to the user so he can use explicit bins if he
knows exactly what he wants to cover. You can also name the
bins
covergroup portdata
coverpoint bt.port {
bins portzero = {0};
bins portone = {[1:5], 7};
bins porttwo = {6, [8:15]};
}
endgroup // portdata

// 1 bin for port0

// 1 bin for ports 1:5, and 7
// 1 bin for ports 6, 8:15

In this case we created 3 bins as shown above.

1/28/2013

Krispan Inc. Confidential

17

Conditional Coverage
You can add the iff keyword to add a condition to the
coverpoint
covergroup portdata
coverpoint bt.port iff(!bus.if.reset)
endgroup

This way you can turn off cover for ports during reset
assuming reset is active high.
you can also use start and stop functions to turn on and off
cover.
Here is an example of that

1/28/2013

Krispan Inc. Confidential

18

Conditional Coverage
initial begin

BusTransaction bt;
portdata pd;
pd = new();
// Instantiate group
bt = new();
#1 pd.stop();
//stop coverage
bus.if.reset = 1;
//start reset
#100 bus.ifi.reset = 0; //end reset
pd.start();
//start coverage
repeat (32) begin
// Run a few cycles
assert(bt.randomize); // Create a transaction
ifc.cb.port <= bt.port; // and transmit
ifc.cb.data <= bt.data; // onto interface
pd.sample();
// Gather coverage
@ifc.cb;
// Wait a cycle
end
end
1/28/2013

Krispan Inc. Confidential

19

Covergroup range
You can use the $ to specify the upper limit if it is used to
specify the right side of the range to specify the upper limit
You can use the $ to specify the lower limit if it is used on the
leftside of the range to specify the lower limit
covergroup portdata
coverpoint bt.port {
bins portzero = {[$:5]};
bins portone = {[6,7]};
bins porttwo = {[8:$]};
}
endgroup // portdata

1/28/2013

// 1 bin for port 0:5

// 1 bin for ports 6 and 7
// 1 bin for ports 8:15

Krispan Inc. Confidential

20

Creating bins for enumerated

types
For enumerated data types SystemVerilog creates a bin for
each value of the enumerated type
typedef enum { idle, decode, data_transfer} fsmstate;
fsmstate cstate, nstate;
covergroup cg_fsm;
coverpoint cstate;
endgroup

In this case it will create a bin each for idle, decode and
data_transfer.
If you want to group multiple values into a single bin then
you have to define your bins just as we did before

1/28/2013

Krispan Inc. Confidential

21

Transition Coverage

Individual State transitions can be specified for a cover point.

For eg you can check if port went from 0 to 1 2 or 3
covergroup portdata
coverpoint bt.port {
bins p1 = (0 => 1), (0 => 2), (0 => 3); }
// Measure coverage
endgroup
Wildcard states and transitions:
Wildcard keyword is used to specify state transitions. X,Z or ? are treated as a wildcard
for 0 or 1. Here is a way to create a coverpoint with bins for even and odd values
bit [2:0] port;
covergroup cp;
coverpoint port {
wildcard bins even = {3b??0};
wildcard bins odd = {3b??1};
}
endgroup

1/28/2013

Krispan Inc. Confidential

22

Coverage group with event trigger

In this case the coverage is started only when a trigger event
happens. For eg you can start the coverage when the
transaction ready signal is triggered from the testbench
e.g.
event transaction_ready
covergroup cp @(transaction_ready);
coverpoint ifc.cb.port;
//measure coverage for port
endgroup

In this case we start measuring coverage on the coverpoint

only after the event is triggerred.

1/28/2013

Krispan Inc. Confidential

23

Cover group in a class

Cover group can be defined in a module program or class.
In any case they have to be instantiated explicitly to start
measuring coverage.
For a class you dont need to create a separate name when
you instance it. You can use the original cover group name
Cover group is triggered whenever there are new values to
the variables or expressions
The trigger can be done by a sampling function or by using
some kind of events to trigger the cover group as we had seen
under coverage group with event trigger.

1/28/2013

Krispan Inc. Confidential

24

Functional coverage in a class

class BusTransactor;
BusTransaction bt;
mailbox tr_in;
covergroup portdata;
coverpoint bt.port;
endgroup
function new(mailbox tr_in);
portdata = new();
this.tr_in = tr_in;
endfunction

class BusTransaction
rand bit [31:0] data; //random variables
rand bit [3:0] port; //16 possible ports

endclass
// Instantiate covergroup

task main;
forever begin
bt = tr_in.get;
ifc.cb.port <= bt.port;
ifc.cb.data <= bt.data;
portdata.sample();

// Get next transaction

// Send into DUT
// Gather coverage

end
endtask
endclass

1/28/2013

Krispan Inc. Confidential

25