CoveragePkg useful for transitions?

[Matthias Alles]

Hi,

I’m currently wondering whether it is possible for me to use the CoveragePkg for a certain type of verification. As I can read from the documentation, it is possible to do a cross coverage. That comes close to what I need, however, it doesn’t match exactly.

What I need is the following: Consider I have an ALU that offers different operations like add, sub, mul, div. Now what I want to check is the sequence of the operations. I want to verify that after one operation the other operations complete successfully. So it is more a transistion-based verification. Of course I could use the RandCovPoint function to generate the transistions. The problem with this is that I then have to verify first the, e.g., add and then the sub. However, consider when I have verified the sub, it should be possible to go from this operation to all other operations without the need to do another sub. So I kind of want to stick to the last operation I performed.

This is just an example that is comparable to my application. In reality the single operations are very time consuming verification tasks for me, so it would save quite a bit of time if I wouldn’t have to start over again for each single transition of operations I want to verify, but if I could rely instead of the current status, i.e., the last operation that has been performed.

Is this possible with the coverage package? From the documentation I currently can’t see how I can do it.

Thanks,

Matthias

[Jim Lewis]

Hi Mattias,

Right now no.  Collecting coverage on current operation and previous operation is easy.  For now, just randomize the operations and collect the coverage.  Based on theory, it will generate all the sequences in O(N*LogN) randomizations.  It will take roughly Log 16 (around 3X) extra cycles to generate this.

Looking into the future, there is a feature planned for the next major release that will address this.  I am just finishing a current minor release.  It will take a little of time before I start the next set of revisions (which is the major).

Best Regards,

Jim

[Matthias Alles]

Hi Jim,

thank you for your quick response. I think it is still faster to use the cross coverage feature than simple randomization of the operations.

What I would like to have is e.g.,

add -> sub -> mul -> div

and what I can check with cross coverage is

(add -> sub), (sub -> mul), (mul -> div)

So in the optimal case I would need to perform (#transitions + 1) operations. With cross coverage as it is currently implemented in the package it is (#transitions * 2) operations. So it’s not too bad.

Best Regards,

Matthias

[Jim Lewis]

Hi Matthias,

Perhaps I can do it.  Lets first talk about what you want.  I think you want to generate all  transitions from one operation to another.  Hence you need to generate the following conditions:

  (add -> sub), (add -> add), (add -> mul), (add-> div)

  (sub -> sub), (sub -> add), (sub -> mul), (sub -> div)

  (mul-> sub), (mul -> add), (mul -> mul), (mul  -> div)

  (div -> sub), (
div -> add), (
div -> mul), (div  -> div) 

Going further, I think you want a minimal sequence to generated this.  

> So in the optimal case I would need to perform (#transitions + 1) operations. 

I agree with this.

> With cross coverage as it is currently implemented in the package it is (#transitions * 2) operations. 

Are using the cross coverage model for randomization?  If so, how because it is going to return a pair and does not currently consider your current state.   Of course, if you just picked the current state value as the randomly generated value for your next state value, as items in the coverage model get covered, the probability that it generates a next state value that you need to cover probably will go up – interesting.  In this case, then may be it converges faster than logN.  OTOH, the value of 2*#transitions that you are currently getting is close enough to the log 16 = 2.8*#transitions to due to a “lucky”seed.  At the end of the day, one can not differentiate between luck and skill.   

With some more thought (and creative programming)  I can get it down to #transitions.   Create one coverage model for each current state (add, sub, mul, div).   When the current state is “add”, use the “add” coverage model to generate the next state and mark the new value covered.  It is not pretty, it does not scale well, but it does get the job done.   

What I am finding is that for some problems, I use one (or in this case several) coverage model(s) to generate sequences, and a separate coverage model (using cross coverage) to check my coverage (usually at an output interface rather than an input interface).   The stimulus generation coverage model may have extra stuff in it to help shape other parts of the problem – but are not necessarily even items that I track.  

Best,

Jim

[Matthias Alles]

Hi Jim,

I’m not using the coverage package yet. Currently I’m playing around with it to get the ideas behind it.

The 2 * #transitions operations are there because with cross coverage and intelligent randomization each transition is performed just once. Each transistion can be performed by two operations. So the intelligent randomization gives me a transition that is uncovered so far (the pair of operations), then I can perform operation 1 and operation 2 in a sequence. So one call of RandCovPoint is used to perform two operations afterwards. That’s my current idea.

I also get your idea of splitting the generation process and the coverage monitoring process.

The ALU is just an example. In reality I have dozens of different operations.

BTW: Why is there no RandCovPoint function for simple bins? When the documentation is talking about intelligent coverage it only discusses cross coverage. How can I get the same functionality without cross coverage?

Matthias

[Matthias Alles]

OK, I discovered the RandCovHole and the GetCovHole functions…

[Jim Lewis]

> BTW: Why is there no RandCovPoint function for simple bins?

See my blog post on this topic.  It does work.  You just have to deal with the return value being integer_vector rather than integer.  

[Matthias Alles]

Thanks for the hint, Jim!

[Author]

Posts