You are browsing the archive for Functional Coverage.

OSVVM Webinar (June 25th) and Classes

June 22, 2015 in Event, Functional Coverage, OS-VVM in general, Randomization

Webinar OSVVM for VHDL Testbenches. Thursday June 25, 2015
Open Source VHDL Verification Methodology (OSVVM) is a comprehensive, advanced VHDL verification methodology. Like UVM, OSVVM is a library of free, open-source code (packages). OSVVM uses this library to implement functional coverage, constrained random tests, and Intelligent Coverage random tests with a conciseness, simplicity and capability that rivals other verification languages.

In 2015, OSVVM added comprehensive error and message reporting (January, 2015.01) and memory modeling (June, 2015.06). With this expanded capability, this presentation
takes a look at the big picture methodology progressing transactions to randomization to functional coverage to intelligent coverage to alerts (error reporting) and logs (message reporting) to memory modeling.

Worried about keeping up with the latest trends in verification? With Intelligent Coverage, OSVVM has a portable, VHDL-based, intelligent testbench solution built into the library. While Accellera is still working on their Intelligent testbench based portable stimulus solution (in the Portable Stimulus Working Group -PSWG), for OSVVM it is already here. Best of all, OSVVM is free and works in any VHDL simulator that support a minimal amount of VHDL-2008.

Europe Session 3-4 pm CEST 6-7 am PDT 9-10 am EDT Enroll with Aldec
US Session 10-11 am PDT 1-2 pm EDT 7-8 pm CEST Enroll with Aldec

OSVVM World Tour Dates
VHDL Testbenches and Verification – OSVVM+ Boot Camp
Learn the latest VHDL verification techniques including transaction level modeling (tlm), self-checking, scoreboards, memory modeling, functional coverage, directed, algorithmic, constrained random, and intelligent testbench test generation. Create a VHDL testbench environment that is competitive with other verification languages, such as SystemVerilog or ‘e’. Our techniques work on VHDL simulators without additional licenses and are accessible to RTL engineers.

July 20-24 and August 3-7 online class Enroll with SynthWorks
September 14-18 Bracknell, UK Enroll with FirstEDA
September 21-25 and October 5-9 online class Enroll with SynthWorks
October 26-30 Portland, OR (Tigard/Tualatin) Enroll with SynthWorks
November 9-13 Copenhagen, Denmark Enroll with FirstEDA
November 16-20 and November 30 – December 4 online class Enroll with SynthWorks

Presented by:
Jim Lewis, SynthWorks VHDL Training Expert, IEEE 1076 Working Group Chair, and OSVVM Chief Architect

OSVVM™ 2014.07 Preview and Questions

July 22, 2014 in Announcement, Functional Coverage, OS-VVM in general

Over the summer I have been working on the next set of revisions for OSVVM.

One of the new features allows a name to be specified for each bin. The name is specified in calls to AddBins and AddCross. If a name is present, calls to WriteBin will print it.

One motivation for a bin name is to correlate a requirement with it being tested (PASSED) or not tested (FAILED). Hence the output of WriteBin is now configurable and supports the following format:
{Prefix} [BinName] [PASSED|FAILED] [BinInfo] [Count]

The following report enables all fields and uses the defaults for the prefix and the Pass/Fail message:
%% State0 PASSED Bin:(0) Count = 1 AtLeast = 1
%% State1 PASSED Bin:(1) Count = 1 AtLeast = 1
%% State2 FAILED Bin:(2) Count = 0 AtLeast = 1
%% State3 FAILED Bin:(3) Count = 0 AtLeast = 1

Each call to WriteBin supports enabling or disabling each of these fields with the parameters shown below.

procedure WriteBin (
    WritePassFail   : CovOptionsType := COV_OPT_DEFAULT ;
    WriteBinInfo    : CovOptionsType := COV_OPT_DEFAULT ;
    WriteCount      : CovOptionsType := COV_OPT_DEFAULT ;
    WriteAnyIllegal : CovOptionsType := COV_OPT_DEFAULT ;
    WritePrefix     : string := "" ;
    PassName        : string := "" ;
    FailName        : string := ""
) ;

Using COV_OPT_DEFAULT and string value “” as defaults allows the defaults to be set independently from WriteBin.

Currently what I have implemented is a set of defaults for these statically set in the package. These can then be changed using the SetReportOptions method (same parameters as WriteBin). The default parameters can be overridden on each call to WriteBin – allowing printing of intermediate results that do not get observed by a requirements tracking tool.

The limitation to SetReportOptions is that it only sets the values for a single Coverage Model. Each coverage object will need its own settings. To get a consistent settings across a project, we need global settings.

Here are some choices:

  • 1: Use SetReportOptions to set global variables rather than local variables. An elegant implementation would use a local generic package for the settings and shared variables in CoveragePkg that are external to CovPType. Unfortunately this is elegant, but on the bleeding edge of support – it works in the most current release of simulators from Aldec and Mentor.
  • 2: Use SetReportOptions to set global variables rather than local variables. Do a brute force implementation that uses a dedicated package for handling the settings. The down side to this is that it requires the reference of a separate package, CovConfigPkg, to be able to change the settings within SetReportOptions or WriteBin. Of course, this can be mitigated some by providing a VHDL-2008 context declaration that includes it. This leaves more options as in 1 to 2 years from now, it would be nice to change to using the generic package.
  • 3: Initialize the report options using an initialization file (such as osvvm.ini). If the ini file is not set, then static settings will be used. Using an ini file could be powerful as it support a common setting for all tests in a design and simplify changing them to support a particular tool. It also allows changing the settings without recompiling the design. The ini file can either be set by a particular tool or manually edited by a user (with an initial file provided in the library).

I don’t see these options as necessarily being exclusive. For example, if we adopt 3, then we could also adopt either 1 or 2, or alternately use the ini file to initialize local settings and have SetReportOptions only change local objects (or never use it).

Currently I am leaning toward creating an ini file. I am thinking that SetReportOptions can be used to change local options – mainly because SetReportOptions is already implemented, however, I really can’t think of a use model where I would want to use SetReportOptions rather than just using WriteBin.

What do you think? Do you see other possibilities?

OSVVM™ Webinar and World Tour Dates

June 20, 2014 in Event, Functional Coverage, OS-VVM in general

Webinar Thursday June 26, 2014
OSVVM provides functional coverage and randomization utilities that layer on top of your transaction level modeling (tlm) based VHDL testbench. Using these you can create either basic Constrained Random tests or more advanced Intelligent Coverage based Random tests. This simplified approach allows you to utilize advanced randomization techniques when you need them and easily mix advanced randomization techniques with directed, algorithmic, and file-based test generation techniques. Best of all, OSVVM is free, works in all of Aldec and some other VHDL simulators.

Europe Session 3-4 pm CEST 6-7 am PDT 9-10 am EDT Enroll with Aldec
US Session 11 am-12 Noon PDT 2-3 pm EDT 8-9 pm CEST Enroll with Aldec

OSVVM World Tour Dates
VHDL Testbenches and Verification – OSVVM+ Boot Camp
Learn the latest VHDL verification techniques including transaction level modeling (tlm), self-checking, scoreboards, memory modeling, functional coverage, directed, algorithmic, constrained random, and intelligent testbench test generation. Create a VHDL testbench environment that is competitive with other verification languages, such as SystemVerilog or ‘e’. Our techniques work on VHDL simulators without additional licenses and are accessible to RTL engineers.

July 14-18 Munich, Germany Enroll with eVision Systems
July 21-25 Bracknell, UK Enroll with FirstEDA
August 18-22 and September 2-5 online class Enroll with SynthWorks
August 25-29 Portland, OR (Tigard/Tualatin) Enroll with SynthWorks
September 15-19 Gothenburg, Sweden Enroll with FirstEDA
October 20-24 Bracknell, UK Enroll with FirstEDA
October 27-31 and November 10-14 online class Enroll with SynthWorks
November 17-21 Baltimore, MD (BWI Area) Enroll with SynthWorks
December 1-5 and December 17-21 online class Enroll with SynthWorks

Presented by:
Jim Lewis, SynthWorks VHDL Training Expert, IEEE 1076 Working Group Chair, and OSVVM Chief Architect

My First Example with OS-VVM CoveragePkg

May 16, 2014 in Functional Coverage, OS-VVM in general

A Guest Blog from Alex Grove of FirstEDA

Here in Europe, I recently had the opportunity to work with Jim Lewis, OS-VVM Chief Architect and IEEE 1076 Working Group Chair, on the first Advanced VHDL Testbenches & Verification training course. This training, held in Bracknell, UK, was attended by engineers from several major European system companies who design and verify programmable devices (FPGAs). VHDL is by far the dominate language used by Europe’s system companies for the design and verification of FPGAs, however it is unclear to many how to enhance their verification with VHDL. What I have found is that experienced FPGA design engineers (including myself) are not utilising the VHDL language for verification.

Verification with VHDL

Jim Lewis introduces VHDL’s verification capabilities, including new VHDL 2008 features and the Open Source VHDL Verification Methodology (OSVVM). OSVVM provides a methodology for testbench development and verification packages that provide functional coverage and random value generation.

To demonstrate the use of the OSVVM functional coverage package ‘CoveragePkg’, I took a familiar Finite-state Machine (FSM) the IEEE 1149.1 TAP (Test Access Port) controller. This is a clever little state machine that I am rather fond of, having spent a lot of time working with it as part of my degree dissertation on Built-in Self Test (BIST).

So let’s take a look at our IEEE 1149.1 TAP FSM.

Post continues at Aldec Company Blog

Functional Coverage Goals and Randomization Weights

August 3, 2013 in Functional Coverage, Randomization

In a constrained random approach, different items can be selected more frequently by using randomization weights. Items with a higher randomization weight are selected more frequently.

In Intelligent Coverage, the same effect can be achieved by using coverage goals. A coverage goal specifies how many times a value must land in a bin before the bin is considered covered. Each bin within the coverage model can have a different coverage goal. By default, coverage goals are also used as a randomization weight. Bins with a higher goal/weight will be generated more frequently. When a bin reaches its goal, it is no longer selected by Intelligent Coverage randomization.

Post continues at SynthWorks OSVVM Blog

VHDL Functional Coverage is more capable than SystemVerilog

July 1, 2013 in Functional Coverage, OS-VVM in general

When writing functional coverage it is important to be able to capture all the details of a model. With item (aka point) coverage, both VHDL and SystemVerilog do a good job. However with cross coverage, if the model requires more than a simple Cartesian product, SystemVerilog falls short. OSVVM, on the other hand, offers a rich cross coverage capability.

Post continues at SynthWorks OSVVM Blog

Using OSVVM for DVB-S2 IP Core Validation

May 2, 2013 in Functional Coverage, OS-VVM in general, Randomization

Hi,

My name is Matthias Alles, I’m CEO and co-founder of Creonic, a Germany-based IP core provider in the field of communications. In this blog post I will show how we are using intelligent coverage provided by OSVVM for validation of our IP cores. I will use the DVB-S2 standard as an example IP core. DVB-S2 is the de-facto standard for satellite communications, e.g., for broadcasting HDTV signals.

The Creonic DVB-S2 IP core performs forward error correction as defined within the standard, i.e. LDPC and BCH decoding. Forward error correction is a technique to correct errors that occur during storage or transmission of digital data.  Before data transmission one adds redundant parity information to the payload information. Payload information plus parity information is denoted as code block. During transmission multiple bits of the code block can flip as they are disturbed for instance by clouds or rain. The receiver exploits the parity information to decode the original payload information.

DVB-S2 defines about 50 configurations that have to be supported by such an IP core. These configurations define parameters like code block size and amount of parity information. In order to achieve functional coverage during validation of a DVB-S2 IP core one obviously has to go through all the configurations. For each configuration, multiple blocks must be tested.

A straight forward approach is to go through all cases in a linear order, i.e., first we test let’s say 1000 code blocks for configuration 1, then 1000 blocks for configuration 2, and so on until all configurations were covered with at least 1000 blocks.

for configuration in 1 to 52 loop
    — Configure IP core now
     for block in 0 to 999 loop
        –- Write block of configuration to the IP core now
    end loop;
end loop;

The drawback of this approach is that is doesn’t reflect how a DVB-S2 system works. In reality, the system is adaptive at run-time to the current signal-to-noise ratio one achieves while communicating with the satellite. These conditions can change quickly, for instance consider a cloud or even rain that suddenly disturbs your transmission. A more realistic scenario is that the single configurations switch randomly and we want our IP core test reflect this when changing the configuration from one block to another. We use the CoveragePkg with a two-dimensional coverage variable to fulfill this requirement:

constant NUM_BLOCKS : natural := 1000; — number of different blocks per configuration
constant AT_LEAST   : natural := 10;   — how often to test each block

shared variable v_cover_cfg_blk : CovPType; — the coverage variable

variable v_cov_current : integer_vector(0 to 1);
variable v_cov_current_cfg : natural;
variable v_cov_current_blk : natural;

-– Generate two-dimensional coverage matrix: (configuration x block)
v_cover_cfg_blk.AddCross(AT_LEAST, GenBin(1, 52), GenBin(0, NUM_BLOCKS – 1));

– Check whether we achieved functional coverage
while not v_cov_cfg_blk.IsCovered loop

    — Get the point in the coverage matrix we are supposed to test next
    v_cov_current := v_cover_cfg_blk.RandCovPoint;
    v_cov_current_cfg := v_cov_current(0); — index 0 is for the configuration
    v_cov_current_blk := v_cov_current(1); — index 1 is for the block

    –
    — Now configure IP core with configuration v_cov_current_cfg and
    — write block v_cov_current_blk.
    –

    — Tell the coverage model that this (configuration x block) combination was exercised
    v_cov_current.ICover(v_cov_current);
end loop;

With just a few lines of code we ensure that we have tested all NUM_BLOCKS blocks for all configurations AT_LEAST number of times.

For the real IP core validation things are a bit more complicated. Not all of the 52 configurations are valid, but we have holes that are not allowed. We deal with this fact during definition of the coverage matrix.

v_cover_cfg_blk.AddCross(AT_LEAST, GenBin(1, 28) & GenBin(33, 52), GenBin(0, NUM_BLOCKS – 1));

With this assignment we exclude the configurations 29 to 32 from the coverage matrix such that we will never exercise the IP core with these invalid configurations.

At Creonic we have used OSVVM for more than one year now and have made it an integral part of our verification plan. The intelligent coverage saves us a significant amount of validation time. The fact that it is open source allowed us to contribute to the code and we are happy to see that our contributions became part of the new 2013.04 release.

Happy coding!
Matthias

Why no constraint solver? Are you going to add one?

December 7, 2012 in Functional Coverage, OS-VVM in general, Randomization

Nope. No constraint solver.  Instead OS-VVM implements an innovative “Intelligent Testbench” feature that does a random walk across functional coverage holes.  We call this feature “Intelligent Coverage”.

Constraint solvers are yesterday’s verification technology.  Intelligent testbenches are the way forward.   In his 2011 DVCON address, Mentor Graphics CEO Wally Rhines noted that constrained random environments generate a significant number of redundant vectors on their way to functional coverage closure.  Randomization theory tells us that a good constraint solver will take O(N * log N) randomizations to generate N different test cases.   The log N factor correlates with Mentor’s observation of a 10X to 100X speedup when using an intelligent testbench tool.

With OS-VVM, intelligent coverage is built into CoveragePkg.  As a result, it is code feature rather than a tool feature – and it is free.

The focus of the next set of enhancements planned for the OS-VVM packages is to further enhance the “Intelligent Coverage” features.  These steps will move OS-VVM further ahead of other verification languages.

Furthermore, since OS-VVM randomizes across functional coverage holes, it provides a naturally balanced solution.  As a supplement to intelligent coverage, the sequential constrained random methodology provided by RandomPkg is sufficient for the time being.

For more information on OS-VVM’s coverage modeling and randomization methods, see CoveragePkg_user_guide.pdf and RandomPkg_user_guide.pdf  (both in the download zip file).  This material plus additional advanced techniques are covered in the class, VHDL Testbenches and Verification – OSVVM Bootcamp.

Functional Coverage: The Heart of OS-VVM

November 20, 2012 in Functional Coverage, OS-VVM in general

Simply stated, the goal of verification is to test everything identified in test plan.  The goal of OS-VVM is to accelerate this process.

Like other methodologies, OS-VVM uses functional coverage to observe conditions on interfaces and within the design to determine that all items in the test plan have been tested.   Unlike other methodologies, functional coverage is the prime directive – it is where we start our process.

Functional coverage is important to all methodologies because it indicates when testing is done.  However, what other methodologies overlook, is that functional coverage also tells us what conditions our testbench needs to generate to be done.

“Intelligent Coverage” says lets take the functional coverage information and randomly pick a condition we have not seen yet and pass that to our stimulus generation process.  If the functional coverage is on an input, the stimulus generation process simply applies an appropriate transaction with the values.  If the functional coverage is on outputs or internal states, then stimulus generation process takes the desired conditions and maps them back to one or more transactions done on design inputs.

For this process to work, we need a functional coverage model that is accurate for both point and cross coverage.  This is also another strong point of OS-VVM. OS-VVM creates the functional coverage data structure sequentially using subprogram method calls.  For simple symmetric cross coverage, the modeling can be done with a single concise call.  For more complex cross coverage, the modeling can be done one item at a time or using any of the sequential language constructs.

For more details of how coverage modeling works, see CoveragePkg_user_guide.pdf (in the download zip file). This material plus additional advanced techniques are covered in the SynthWorks class, VHDL Testbenches and Verification – OSVVM Bootcamp.

Webinar about OS-VVM coming soon. Any questions?

July 12, 2012 in Event, Functional Coverage, OS-VVM in general, Randomization

Hello Fellow OS-VVMers,

A webinar introducing OS-VVM is coming soon. It will be broadcast twice on July 19th, 2012 (Thursday):

If you plan to attend, please register using links above. There will be a chance to ask questions during the webinar, but if you already have questions, please ask them now in the replies (comments) to this post. The most interesting ones will be answered live during the webinar — your name will not be mentioned unless you explicitly give us permission in the reply (something like this: “You can call me John from Chicago while answering my question.”)

Hope to see you during the webinar!

Your Friendly Admin.