Category Archives: Intelligent Coverage

OSVVM™ Webinar + World Tour Dates

Webinar Thursday June 26, 2014
OSVVM provides functional coverage and randomization utilities that layer on top of your transaction level modeling (tlm) based VHDL testbench. Using these you can create either basic Constrained Random tests or more advanced Intelligent Coverage based Random tests.  This simplified approach allows you to utilize advanced randomization techniques when you need them and easily mix advanced randomization techniques with directed, algorithmic, and file-based test generation techniques.  Best of all, OSVVM is free and works in most VHDL simulators.

Europe Session 3-4 pm CEST 6-7 am PDT 9-10 am EDT Enroll with Aldec
US Session 11 am-12 Noon PDT 2-3 pm EDT 8-9 pm CEST Enroll with Aldec

OSVVM World Tour Dates
VHDL Testbenches and Verification – OSVVM+ Boot Camp
Learn the latest VHDL verification techniques including transaction level modeling (tlm), self-checking, scoreboards, memory modeling, functional coverage, directed, algorithmic, constrained random, and intelligent testbench test generation. Create a VHDL testbench environment that is competitive with other verification languages, such as SystemVerilog or ‘e’. Our techniques work on VHDL simulators without additional licenses and are accessible to RTL engineers.

July 14-18 Munich, Germany Enroll with eVision Systems
July 21-25 Bracknell, UK Enroll with FirstEDA
August 18-22 and September 2-5 online class Enroll with SynthWorks
August 25-29 Portland, OR (Tigard/Tualatin) Enroll with SynthWorks
September 15-19 Gothenburg, Sweden Enroll with FirstEDA
October 20-24 Bracknell, UK Enroll with FirstEDA
October 27-31 and November 10-14 online class Enroll with SynthWorks
November 17-21 Baltimore, MD (BWI Area) Enroll with SynthWorks
December 1-5 and December 17-21 online class Enroll with SynthWorks

Presented by:
Jim Lewis, SynthWorks VHDL Training Expert, IEEE 1076 Working Group Chair, and OSVVM Chief Architect

Functional Coverage Goals and Randomization Weights

This is a continuing series of posts on OSVVM and functional coverage. If you are just getting started, you may wish to start with the OSVVM page.

In a constrained random approach, different items can be selected more frequently by using randomization weights. Items with a higher randomization weight are selected more frequently.

In Intelligent Coverage, the same effect can be achieved by using coverage goals. A coverage goal specifies how many times a value must land in a bin before the bin is considered covered. Each bin within the coverage model can have a different coverage goal. By default, coverage goals are also used as a randomization weight. Bins with a higher goal/weight will be generated more frequently. When a bin reaches its goal, it is no longer selected by Intelligent Coverage randomization.

Coverage goals and randomization weights are an important part of the Intelligent Coverage methodology. They allow us to randomize sequences with a particular distribution. For example, if a test does packet generation, then the following code generates normal operation 70% of the time, error case 1 20% of the time, and error case 2 10% of the time.

Bin1.AddBins( 70, GenBin(0) ) ;  -- Normal Handling, 70%
Bin1.AddBins( 20, GenBin(1) ) ;  -- Error Case 1,    20%
Bin1.AddBins( 10, GenBin(2) ) ;  -- Error Case 2,    10%

StimGen : while not Bin1.IsCovered loop
  iSequence := Bin1.RandCovPoint ; 
  case iSequence is   

    when 0 =>  -- Normal Handling   -- 70%
       DoTransaction(Rec, …, NO_ERROR) ; 
       . . . 

    when 1 =>  -- Error Case 1      -- 20%
       DoTransaction(Rec, …, ERROR_CASE1) ; 
       . . . 

    when 2 =>  -- Error Case 2      -- 10%
       DoTransaction(Rec, …, ERROR_CASE2) ; 
       . . .

Each of these sequences will be selected with a weighted uniform distribution until it reaches its coverage goal.

SystemVerilog supports specifying coverage goals on the entire coverage model, but not on individual coverage bins. Since they do not use functional coverage when randomizing, they do not have a compelling need for individual coverage goals. However, this also makes them lacking in advanced features.

This post provided some basic information about coverage goals. Intelligent Coverage contains numerous advanced features that go well beyond the basic capability. For example, coverage goals can be scaled to make tests run longer using a feature called coverage targets. Convergence to a coverage goal can be smoothed out using either thresholding or different randomization weight modes.

Learn more about the advanced Intelligent Coverage features in our VHDL Testbenches and Verification class.

OSVVM’s Intelligent Coverage is 5X or More Faster than SystemVerilog’s Constrained Random

If the measure of test case generation was a large number of well randomized test cases, SystemVerilog and UVM would be on par with VHDL’s OSVVM.  However the true measure of test case generation is functional coverage closure – all test cases identified in the test plan are done.  Functional coverage closure is a big challenge for constrained random approaches to verification as used in SystemVerilog or ‘e’.  On the other hand, functional coverage closure is the focus of OSVVM’s Intelligent Coverage™.   This article takes a look at why Constrained Random has challenges and how Intelligent Coverage solves it.

1. Constrained Random Repeats Test Cases

In my post, Functional Coverage Made Easy with VHDL’s OSVVM, we used randomization with a uniform distribution (shown below) to select the register pairs for the ALU. Constrained random at its best produces a uniform distribution. As a result, this example is a best case model of constrained random tests.

Src1 := RV.RandInt(0, 7) ; -- Uniform Randomization
Src2 := RV.RandInt(0, 7) ;

The problem with constrained random testbenches is that they repeat some test cases before generating all test cases. In general to generate N cases, it takes “N * log N” randomizations. The “log N” represents repeated test cases and significantly adds to simulation run times. Ideally we would like to run only N test cases.

Running the previous ALU testbench, we get the following coverage matrix when the code completes. Note that some case were generated 10 time before all were done at least 1 time. It took 315 randomizations to generate all 64 unique pairs. This is slightly less than 5X more iterations than the 64 in the ideal case. This correlates well with theory as 315 is approximately 64 * log(64). By changing the seed value, the exact number of randomizations may increase or decrease but this would be a silly way to try to reduce the number of iterations a test runs.


2. Intelligent Coverage

“Intelligent Coverage” is a coverage driven randomization approach that randomly selects a hole in the functional coverage and passes it to the stimulus generation process. Using “Intelligent Coverage” allows the stimulus generation to focus on missing coverage and reduces the number of test cases generated to approach the ideal of N randomizations to generate N test cases.

Lets return to the ALU example. The Intelligent Coverage methodology starts by writing functional coverage. We did this in the previous example too. Next preliminary stimulus is generated by randomizing using the functional coverage model. In this example, we will replace the call to RandInt (uniform randomization) with a call to RandCovPoint (one of the Intelligent Coverage randomization methods). This is shown below. In this case, Src1 and Src2 are used directly in the test, so we are done.

architecture Test3 of tb is
  shared variable ACov : CovPType ;  -- Declare 
  TestProc : process 
    variable RV : RandomPType ;
    variable Src1, Src2 : integer ;
    -- create coverage model
    ACov.AddCross( GenBin(0,7), GenBin(0,7) );  -- Model

    while not ACov.IsCovered loop    -- Done?
      (Src1, Src2) := ACov.RandCovPoint ; -- Intelligent Coverage Randomization

      DoAluOp(TRec, Src1, Src2) ;    -- Transaction
      ACov.ICover( (Src1, Src2) ) ;  -- Accumulate
    end loop ;

    ACov.WriteBin ;  -- Report 
    EndStatus(. . . ) ;   
  end process ;

When randomizing across a cross coverage model, the output of RandCovPoint is an integer_vector. Instead of using the separate integers, Src1 and Src2, it is also possible to use an integer_vector as shown below.

variable Src : integer_vector(1 to 2) ;
. . . 
Src := ACov.RandCovPoint ;      -- Intelligent Coverage Randomization

The process is not always this easy. Sometimes the value out of RandCovPoint will need to be further shaped by the stimulus generation process.  We do this in our VHDL Testbenches and Verification class.

The Intelligent Coverage methodology works now and works with your current testbench approach. You can adopt this methodology incrementally. Add functional coverage today to make sure you are executing all of your test plan. For the tests that need help, use the Intelligent Coverage.

3. Intelligent Testbenches in SystemVerilog

With SystemVerilog you can certainly buy a simulator that implements Intelligent Testbenches. However, this is a signification upgrade, so it will cost. In addition using an intelligent testbench tool tends to require vendor specific coding – so you are locked into a particular vendor.

On the other hand, with VHDL’s OSVVM, the Intelligent Testbench capability is built into the functional coverage modeling.  It is free.  All of the customizations to the randomization are done by writing VHDL code and initiating transactions.

4. References

Here are a couple of articles on Intelligent Testbench approaches that also remove or reduce the repetition of test cases. However, these solutions are not free like OSVVM.

Wally Rhines. From Volume to Velocity. DVCon Keynote March 2011

Mark Olen. Intelligent Testbench Automation Delivers 10X to 100X Faster Functional Verification. Verification Horizons Blog June 2011

Brian Bailey. Enough of the sideshows – it’s time for some real advancement in functional verification! EDA DesignLine Blog May 2012