You are browsing the archive for Functional Coverage.

Announcing OSVVM™ 2018.04

April 10, 2018 in Announcement, Functional Coverage, OS-VVM in general

OSVVM™ 2018.04 does some minor extensions and fixes to TbUtilPkg, ScoreboardGenericPkg, CoveragePkg, ScoreboardGenericPkg, and MessagePkg.

In TbUtilPkg, the procedures Increment and WaitForToggle were added for handshaking with type integer. The overloaded procedures RequestTransaction, WaitForTransaction, Toggle, and WaitForToggle were added for handshaking with type bit.

In ScoreboardGenericPkg, the pop functions were made visible. In previous revisions the function declarations were commented out due to issues in some simulators.

In CoveragePkg, a minor fix to the calculation of PercentCov was made to account for AtLeast values of less than or equal to zero. This issue would only impact you if you used a coverage goal (AtLeast parameter) of 0 or less.

Also in CoveragePkg, added “string’” to one of the calls to write to be language compliant – this snuck through as many simulators allowed this – OTOH GHDL issued a compile error – so kudos to the GHDL community for finding and reporting this. Removed the deprecated Increment procedures – but see TbUtilPkg as Increment was added there for type integer.

In AlertLogPkg did a minor update to PathTail to remove any trailing “:” in the path. Note that PathTail is intended to be used on the path returned by applying ‘PATH_NAME to the entity name.

In AlertLogPkg and ScoreboardPkg preparations were made for changing AlertLogIDType from a subtype of integer to a type. This change will happen in a future release of OSVVM. Make sure you are using AlertLogIDType for AlertLogIDs and not type integer.

In MessagePkg, some minor changes were made to the Alert message.

OSVVM™ Boot Camp

SynthWorks’ primary business is training. Our class, Advanced VHDL Testbenches and Verification – OSVVM™ Boot Camp will bring your team up to speed on using OSVVM. Follow this link to our current class schedule.

Getting the release

OSVVM release 2018.04 is available as a zip at OSVVM Downloads.

OSVVM release 2018.04 is available via git at OSVVM GitHub.

OSVVM: Webinar and Training

October 2, 2017 in Alerts and Logs, Announcement, Event, Functional Coverage, OS-VVM in general, Randomization, Transaction Based Testbench

Webinar OSVVM: ASIC Level Verification, Simple Enough for FPGAs. Thursday October 5, 2017

Just because your design is complex does not mean your testbench needs to be. With Open Source VHDL Verification Methodology (OSVVM), we have found that with proper abstractions we can create simple, readable, and powerful testbenches – and we can even have some fun while doing it.

OSVVM provides an ASIC level VHDL verification methodology that is simple enough to use even on small FPGA projects. OSVVM offers the same capabilities as those based on other verification languages:
• Transaction-Based Modeling
• Constrained Random test generation
• Functional Coverage with hooks for UCIS coverage database integration
• Intelligent Coverage Random test generation
• Utilities for testbench process synchronization
• Transcript files
• Error logging and reporting – Alerts and Affirmations
• Message filtering – Logs
• Scoreboards and FIFOs (data structures for verification)
• Memory models

OSVVM is implemented as a library of free, open-source packages. It uses these packages to create features that rival language based implementations in both conciseness, simplicity, and capability.

Looking to improve your FPGA verification methodology? OSVVM provides a complete solution for VHDL ASIC or FPGA verification. There is no new language to learn. It is simple, powerful, and concise. Each piece is separate and can be used separately. Hence, you can learn and adopt pieces as you need them.

This presentation will be a broad overview of each of OSVVM’s capabilities. While Transaction Based Modeling and Scoreboards were released to OSVVM in 2016, they are mature capabilities that have been used for over 10+ years now in SynthWorks’ VHDL Training classes.

Europe Session 3-4 pm CEST 6-7 am PST 9-10 am EST Enroll with Aldec
US Session 11 am -12 noon PST 2-3 pm EST 8-9 pm CEST Enroll with Aldec

OSVVM Training Dates

Advanced VHDL Testbenches and Verification – OSVVM+ Boot Camp

Like the Webinar? Ready to make Open Source VHDL Verification Methodology (OSVVM) part of your VHDL testbench and verification methodology? Join me for Advanced VHDL Testbenches and Verification, AKA The OSVVM Boot Camp.

In this class you will gain hands on experience in the latest VHDL verification techniques using OSVVM. You will create a VHDL testbench environment that is competitive with other verification languages, such as SystemVerilog or ‘e’. These techniques and capabilities include:
• Transaction-Based Modeling
• Constrained Random test generation
• Functional Coverage with hooks for UCIS coverage database integration
• Intelligent Coverage Random test generation
• Utilities for testbench process synchronization
• Transcript files
• Error logging and reporting – Alerts and Affirmations
• Message filtering – Logs
• Scoreboards and FIFOs (data structures for verification)
• Memory models

Our techniques work on VHDL simulators without additional licenses and are intended to be readable by Verification and RTL design engineers, as well as, system and software engineers.

The intention of OSVVM goes beyond capability though – OSVVM intends to make verification environments easy, readable, and fun. OSVVM can accomplish this for either a large complex ASIC or a simple FPGA RTL block.

Join us – learn to work smarter and not harder. Expect to work hard in class though as this is a 5 day class and our typical class day is 8 hours (8:30 – 17:30+)

October 16 – 20 and October 30 – November 3 Web Class Enroll with SynthWorks
November 6 – 10 Bracknell, UK Enroll with FirstEDA
November 27 – December 1 and December 11 – 16 Web Class Enroll with SynthWorks

OSVVM Webinar (June 25th) and Classes

June 22, 2015 in Event, Functional Coverage, OS-VVM in general, Randomization

Webinar OSVVM for VHDL Testbenches. Thursday June 25, 2015
Open Source VHDL Verification Methodology (OSVVM) is a comprehensive, advanced VHDL verification methodology. Like UVM, OSVVM is a library of free, open-source code (packages). OSVVM uses this library to implement functional coverage, constrained random tests, and Intelligent Coverage random tests with a conciseness, simplicity and capability that rivals other verification languages.

In 2015, OSVVM added comprehensive error and message reporting (January, 2015.01) and memory modeling (June, 2015.06). With this expanded capability, this presentation
takes a look at the big picture methodology progressing transactions to randomization to functional coverage to intelligent coverage to alerts (error reporting) and logs (message reporting) to memory modeling.

Worried about keeping up with the latest trends in verification? With Intelligent Coverage, OSVVM has a portable, VHDL-based, intelligent testbench solution built into the library. While Accellera is still working on their Intelligent testbench based portable stimulus solution (in the Portable Stimulus Working Group -PSWG), for OSVVM it is already here. Best of all, OSVVM is free and works in any VHDL simulator that support a minimal amount of VHDL-2008.

Europe Session 3-4 pm CEST 6-7 am PDT 9-10 am EDT Enroll with Aldec
US Session 10-11 am PDT 1-2 pm EDT 7-8 pm CEST Enroll with Aldec

OSVVM World Tour Dates
VHDL Testbenches and Verification – OSVVM+ Boot Camp
Learn the latest VHDL verification techniques including transaction level modeling (tlm), self-checking, scoreboards, memory modeling, functional coverage, directed, algorithmic, constrained random, and intelligent testbench test generation. Create a VHDL testbench environment that is competitive with other verification languages, such as SystemVerilog or ‘e’. Our techniques work on VHDL simulators without additional licenses and are accessible to RTL engineers.

July 20-24 and August 3-7 online class Enroll with SynthWorks
September 14-18 Bracknell, UK Enroll with FirstEDA
September 21-25 and October 5-9 online class Enroll with SynthWorks
October 26-30 Portland, OR (Tigard/Tualatin) Enroll with SynthWorks
November 9-13 Copenhagen, Denmark Enroll with FirstEDA
November 16-20 and November 30 – December 4 online class Enroll with SynthWorks

Presented by:
Jim Lewis, SynthWorks VHDL Training Expert, IEEE 1076 Working Group Chair, and OSVVM Chief Architect

OSVVM™ 2014.07 Preview and Questions

July 22, 2014 in Announcement, Functional Coverage, OS-VVM in general

Over the summer I have been working on the next set of revisions for OSVVM.

One of the new features allows a name to be specified for each bin. The name is specified in calls to AddBins and AddCross. If a name is present, calls to WriteBin will print it.

One motivation for a bin name is to correlate a requirement with it being tested (PASSED) or not tested (FAILED). Hence the output of WriteBin is now configurable and supports the following format:
{Prefix} [BinName] [PASSED|FAILED] [BinInfo] [Count]

The following report enables all fields and uses the defaults for the prefix and the Pass/Fail message:
%% State0 PASSED Bin:(0) Count = 1 AtLeast = 1
%% State1 PASSED Bin:(1) Count = 1 AtLeast = 1
%% State2 FAILED Bin:(2) Count = 0 AtLeast = 1
%% State3 FAILED Bin:(3) Count = 0 AtLeast = 1

Each call to WriteBin supports enabling or disabling each of these fields with the parameters shown below.

procedure WriteBin (
    WritePassFail   : CovOptionsType := COV_OPT_DEFAULT ;
    WriteBinInfo    : CovOptionsType := COV_OPT_DEFAULT ;
    WriteCount      : CovOptionsType := COV_OPT_DEFAULT ;
    WriteAnyIllegal : CovOptionsType := COV_OPT_DEFAULT ;
    WritePrefix     : string := "" ;
    PassName        : string := "" ;
    FailName        : string := ""
) ;

Using COV_OPT_DEFAULT and string value “” as defaults allows the defaults to be set independently from WriteBin.

Currently what I have implemented is a set of defaults for these statically set in the package. These can then be changed using the SetReportOptions method (same parameters as WriteBin). The default parameters can be overridden on each call to WriteBin – allowing printing of intermediate results that do not get observed by a requirements tracking tool.

The limitation to SetReportOptions is that it only sets the values for a single Coverage Model. Each coverage object will need its own settings. To get a consistent settings across a project, we need global settings.

Here are some choices:

  • 1: Use SetReportOptions to set global variables rather than local variables. An elegant implementation would use a local generic package for the settings and shared variables in CoveragePkg that are external to CovPType. Unfortunately this is elegant, but on the bleeding edge of support – it works in the most current release of simulators from Aldec and Mentor.
  • 2: Use SetReportOptions to set global variables rather than local variables. Do a brute force implementation that uses a dedicated package for handling the settings. The down side to this is that it requires the reference of a separate package, CovConfigPkg, to be able to change the settings within SetReportOptions or WriteBin. Of course, this can be mitigated some by providing a VHDL-2008 context declaration that includes it. This leaves more options as in 1 to 2 years from now, it would be nice to change to using the generic package.
  • 3: Initialize the report options using an initialization file (such as osvvm.ini). If the ini file is not set, then static settings will be used. Using an ini file could be powerful as it support a common setting for all tests in a design and simplify changing them to support a particular tool. It also allows changing the settings without recompiling the design. The ini file can either be set by a particular tool or manually edited by a user (with an initial file provided in the library).

I don’t see these options as necessarily being exclusive. For example, if we adopt 3, then we could also adopt either 1 or 2, or alternately use the ini file to initialize local settings and have SetReportOptions only change local objects (or never use it).

Currently I am leaning toward creating an ini file. I am thinking that SetReportOptions can be used to change local options – mainly because SetReportOptions is already implemented, however, I really can’t think of a use model where I would want to use SetReportOptions rather than just using WriteBin.

What do you think? Do you see other possibilities?

OSVVM™ Webinar and World Tour Dates

June 20, 2014 in Event, Functional Coverage, OS-VVM in general

Webinar Thursday June 26, 2014
OSVVM provides functional coverage and randomization utilities that layer on top of your transaction level modeling (tlm) based VHDL testbench. Using these you can create either basic Constrained Random tests or more advanced Intelligent Coverage based Random tests. This simplified approach allows you to utilize advanced randomization techniques when you need them and easily mix advanced randomization techniques with directed, algorithmic, and file-based test generation techniques. Best of all, OSVVM is free, works in all of Aldec and some other VHDL simulators.

Europe Session 3-4 pm CEST 6-7 am PDT 9-10 am EDT Enroll with Aldec
US Session 11 am-12 Noon PDT 2-3 pm EDT 8-9 pm CEST Enroll with Aldec

OSVVM World Tour Dates
VHDL Testbenches and Verification – OSVVM+ Boot Camp
Learn the latest VHDL verification techniques including transaction level modeling (tlm), self-checking, scoreboards, memory modeling, functional coverage, directed, algorithmic, constrained random, and intelligent testbench test generation. Create a VHDL testbench environment that is competitive with other verification languages, such as SystemVerilog or ‘e’. Our techniques work on VHDL simulators without additional licenses and are accessible to RTL engineers.

July 14-18 Munich, Germany Enroll with eVision Systems
July 21-25 Bracknell, UK Enroll with FirstEDA
August 18-22 and September 2-5 online class Enroll with SynthWorks
August 25-29 Portland, OR (Tigard/Tualatin) Enroll with SynthWorks
September 15-19 Gothenburg, Sweden Enroll with FirstEDA
October 20-24 Bracknell, UK Enroll with FirstEDA
October 27-31 and November 10-14 online class Enroll with SynthWorks
November 17-21 Baltimore, MD (BWI Area) Enroll with SynthWorks
December 1-5 and December 17-21 online class Enroll with SynthWorks

Presented by:
Jim Lewis, SynthWorks VHDL Training Expert, IEEE 1076 Working Group Chair, and OSVVM Chief Architect

My First Example with OS-VVM CoveragePkg

May 16, 2014 in Functional Coverage, OS-VVM in general

A Guest Blog from Alex Grove of FirstEDA

Here in Europe, I recently had the opportunity to work with Jim Lewis, OS-VVM Chief Architect and IEEE 1076 Working Group Chair, on the first Advanced VHDL Testbenches & Verification training course. This training, held in Bracknell, UK, was attended by engineers from several major European system companies who design and verify programmable devices (FPGAs). VHDL is by far the dominate language used by Europe’s system companies for the design and verification of FPGAs, however it is unclear to many how to enhance their verification with VHDL. What I have found is that experienced FPGA design engineers (including myself) are not utilising the VHDL language for verification.

Verification with VHDL

Jim Lewis introduces VHDL’s verification capabilities, including new VHDL 2008 features and the Open Source VHDL Verification Methodology (OSVVM). OSVVM provides a methodology for testbench development and verification packages that provide functional coverage and random value generation.

To demonstrate the use of the OSVVM functional coverage package ‘CoveragePkg’, I took a familiar Finite-state Machine (FSM) the IEEE 1149.1 TAP (Test Access Port) controller. This is a clever little state machine that I am rather fond of, having spent a lot of time working with it as part of my degree dissertation on Built-in Self Test (BIST).

So let’s take a look at our IEEE 1149.1 TAP FSM.

Post continues at Aldec Company Blog

Functional Coverage Goals and Randomization Weights

August 3, 2013 in Functional Coverage, Randomization

In a constrained random approach, different items can be selected more frequently by using randomization weights. Items with a higher randomization weight are selected more frequently.

In Intelligent Coverage, the same effect can be achieved by using coverage goals. A coverage goal specifies how many times a value must land in a bin before the bin is considered covered. Each bin within the coverage model can have a different coverage goal. By default, coverage goals are also used as a randomization weight. Bins with a higher goal/weight will be generated more frequently. When a bin reaches its goal, it is no longer selected by Intelligent Coverage randomization.

Post continues at SynthWorks OSVVM Blog

VHDL Functional Coverage is more capable than SystemVerilog

July 1, 2013 in Functional Coverage, OS-VVM in general

When writing functional coverage it is important to be able to capture all the details of a model. With item (aka point) coverage, both VHDL and SystemVerilog do a good job. However with cross coverage, if the model requires more than a simple Cartesian product, SystemVerilog falls short. OSVVM, on the other hand, offers a rich cross coverage capability.

Post continues at SynthWorks OSVVM Blog

Using OSVVM for DVB-S2 IP Core Validation

May 2, 2013 in Functional Coverage, OS-VVM in general, Randomization

Hi,

My name is Matthias Alles, I’m CEO and co-founder of Creonic, a Germany-based IP core provider in the field of communications. In this blog post I will show how we are using intelligent coverage provided by OSVVM for validation of our IP cores. I will use the DVB-S2 standard as an example IP core. DVB-S2 is the de-facto standard for satellite communications, e.g., for broadcasting HDTV signals.

The Creonic DVB-S2 IP core performs forward error correction as defined within the standard, i.e. LDPC and BCH decoding. Forward error correction is a technique to correct errors that occur during storage or transmission of digital data.  Before data transmission one adds redundant parity information to the payload information. Payload information plus parity information is denoted as code block. During transmission multiple bits of the code block can flip as they are disturbed for instance by clouds or rain. The receiver exploits the parity information to decode the original payload information.

DVB-S2 defines about 50 configurations that have to be supported by such an IP core. These configurations define parameters like code block size and amount of parity information. In order to achieve functional coverage during validation of a DVB-S2 IP core one obviously has to go through all the configurations. For each configuration, multiple blocks must be tested.

A straight forward approach is to go through all cases in a linear order, i.e., first we test let’s say 1000 code blocks for configuration 1, then 1000 blocks for configuration 2, and so on until all configurations were covered with at least 1000 blocks.

for configuration in 1 to 52 loop
    — Configure IP core now
     for block in 0 to 999 loop
        –- Write block of configuration to the IP core now
    end loop;
end loop;

The drawback of this approach is that is doesn’t reflect how a DVB-S2 system works. In reality, the system is adaptive at run-time to the current signal-to-noise ratio one achieves while communicating with the satellite. These conditions can change quickly, for instance consider a cloud or even rain that suddenly disturbs your transmission. A more realistic scenario is that the single configurations switch randomly and we want our IP core test reflect this when changing the configuration from one block to another. We use the CoveragePkg with a two-dimensional coverage variable to fulfill this requirement:

constant NUM_BLOCKS : natural := 1000; — number of different blocks per configuration
constant AT_LEAST   : natural := 10;   — how often to test each block

shared variable v_cover_cfg_blk : CovPType; — the coverage variable

variable v_cov_current : integer_vector(0 to 1);
variable v_cov_current_cfg : natural;
variable v_cov_current_blk : natural;

-– Generate two-dimensional coverage matrix: (configuration x block)
v_cover_cfg_blk.AddCross(AT_LEAST, GenBin(1, 52), GenBin(0, NUM_BLOCKS – 1));

– Check whether we achieved functional coverage
while not v_cov_cfg_blk.IsCovered loop

    — Get the point in the coverage matrix we are supposed to test next
    v_cov_current := v_cover_cfg_blk.RandCovPoint;
    v_cov_current_cfg := v_cov_current(0); — index 0 is for the configuration
    v_cov_current_blk := v_cov_current(1); — index 1 is for the block

    –
    — Now configure IP core with configuration v_cov_current_cfg and
    — write block v_cov_current_blk.
    –

    — Tell the coverage model that this (configuration x block) combination was exercised
    v_cov_current.ICover(v_cov_current);
end loop;

With just a few lines of code we ensure that we have tested all NUM_BLOCKS blocks for all configurations AT_LEAST number of times.

For the real IP core validation things are a bit more complicated. Not all of the 52 configurations are valid, but we have holes that are not allowed. We deal with this fact during definition of the coverage matrix.

v_cover_cfg_blk.AddCross(AT_LEAST, GenBin(1, 28) & GenBin(33, 52), GenBin(0, NUM_BLOCKS – 1));

With this assignment we exclude the configurations 29 to 32 from the coverage matrix such that we will never exercise the IP core with these invalid configurations.

At Creonic we have used OSVVM for more than one year now and have made it an integral part of our verification plan. The intelligent coverage saves us a significant amount of validation time. The fact that it is open source allowed us to contribute to the code and we are happy to see that our contributions became part of the new 2013.04 release.

Happy coding!
Matthias

Why no constraint solver? Are you going to add one?

December 7, 2012 in Functional Coverage, OS-VVM in general, Randomization

Nope. No constraint solver.  Instead OS-VVM implements an innovative “Intelligent Testbench” feature that does a random walk across functional coverage holes.  We call this feature “Intelligent Coverage”.

Constraint solvers are yesterday’s verification technology.  Intelligent testbenches are the way forward.   In his 2011 DVCON address, Mentor Graphics CEO Wally Rhines noted that constrained random environments generate a significant number of redundant vectors on their way to functional coverage closure.  Randomization theory tells us that a good constraint solver will take O(N * log N) randomizations to generate N different test cases.   The log N factor correlates with Mentor’s observation of a 10X to 100X speedup when using an intelligent testbench tool.

With OS-VVM, intelligent coverage is built into CoveragePkg.  As a result, it is code feature rather than a tool feature – and it is free.

The focus of the next set of enhancements planned for the OS-VVM packages is to further enhance the “Intelligent Coverage” features.  These steps will move OS-VVM further ahead of other verification languages.

Furthermore, since OS-VVM randomizes across functional coverage holes, it provides a naturally balanced solution.  As a supplement to intelligent coverage, the sequential constrained random methodology provided by RandomPkg is sufficient for the time being.

For more information on OS-VVM’s coverage modeling and randomization methods, see CoveragePkg_user_guide.pdf and RandomPkg_user_guide.pdf  (both in the download zip file).  This material plus additional advanced techniques are covered in the class, VHDL Testbenches and Verification – OSVVM Bootcamp.