Merging coverage databases

Hi Lyle,

What is your application? 

There is a ReadCovDb and WriteCovDb.  I will be adding a merge option to ReadCovDb in the next revision, however, with Intelligent Coverage I do not expect merging to be a main stream methodology. 

Let me explain.  OSVVM’s Intelligent Coverage is a closed loop test.  It generates stimulus based on the current coverage holes.  As long as there is a good correlation between the stimulus generation and the coverage model (for inputs there is 100% correlation), the test will achieve coverage closure in the sum of the coverage goals for each coverage bin.   A functional coverage model that is too big to be run in a single simulation run may be partitioned into multiple independent models and run separately.  You can even run the separately for your daily fast simulations and combine them together for your longer regression tests.

Intelligent Coverage then is alot different from SystemVerilog Constrained Random.  Constrained Random is an open loop test – there is no guarantee that a given set of constraints. will achieve coverage closure and indeed to achieve coverage closure you often end up with multiple testbenches that use different constraints, different controls, and perhaps even different seeds (gag). 

There are situations where we use Intelligent Coverage to configure a test.  In this situation, the configuration is randomized at the beginning.  The steps we do is read the coverage database, randomize the configuration, and write out the coverage database (before running the test).  In this case, we can launch multiple simulations provided that each waits long enough for the prior one to read and then write out the coverage database.  

If you show me an application where with you need merging with OSVVM, I can make sure that it gets added sooner – adding it is a trivial addition (where ReadCovDb  currently replaces the current value of the coverage with the read value, simply replace that with the current value plus the read value) – the more interesting part is testing it.

Jim

Hi Lyle,

Case 1:

To be able to run many simulations at the same time, a test randomizes its configuration, saves the coverage database, and then runs its test.  The next test reads the previous tests database, randomizes, updates the database, then runs its test.  The limitation then is that tests must be started in a structured manner to ensure that each test works with an updated version of the database.   

Case 2

This case does not really apply to OSVVM.  The reason for this approach is all about constrained random.  With constrained random, there are no constraints derived from the coverage model.  Nothing in this approach drives a simulation toward coverage closure other than running many, many tests.  Hence, we create different test cases that use different seeds, or slightly different constraints or controls.  We run tests and each test collects functional coverage across all aspects we are interested in.  

With OSVVM, within each testcase, we create a targeted functional coverage model for what we want to see during that test.  Intelligent Coverage drives a test to coverage closure by only randomly selecting items among the coverage holes.  

None the less, it is simple enough to add merging, so I plan on adding it in the next revision.  I would not want someone not to try out OSVVM because merging was not there – even if after they try out OSVVM, they never end up using merging. 

Jim

Hi Lyle,

One use model is to write out the coverage database file from separate simulation runs into separate files.  And hence, at the end of test1 we do:

...
UartRx_RxCov.WriteCovDb("Test1_CovDb.txt") ;

Likewise for test2 and so on:

...
UartRx_RxCov.WriteCovDb("Test2_CovDb.txt") ;

Then accumulate the coverage using a separate VHDL program:

UartRx_RxCov.ReadCovDb("Test1_CovDb.txt") ;
for i in 2 to NUM_TESTS_TO_MERGE loop 
  UartRx_RxCov.ReadCovDb("Test" & to_string(i) & "_CovDb.txt", MERGE =>TRUE) ;
end loop ;
... -- do stuff with the merged database

With the current implementation, there is no practical usage of having multiple databases written to the same file.  The append_mode was added to WriteCovDb for symmetry with WriteBin (with very little thought about its application – which is unfortunate and in your case misleading). Your use model of allowing ReadCovDb to merge a set of databases from the same file is interesting – it just was not implemented that way in this version.  I just looked at ReadCovDb, I don’t think it would take too much to modify it to make your use model work. It may be:

    ------------------------------------------------------------
    procedure ReadCovDb (FileName : string; Merge : boolean := FALSE) is
    ------------------------------------------------------------
      file CovDbFile         : text open READ_MODE is FileName ;
    begin
      ReadCovDb(CovDbFile, Merge) ;
      if Merge then
        while not EndFile(CovDbFile) loop
          ReadCovDb(CovDbFile, Merge) ;
        end loop ;
      end if; 
    end procedure ReadCovDb ;

I have not tried this, so there may be subtle issues. Let me know if you try it.

Jim