Handling expected failures

[Mikkie]

I have a test that create a single failure on a transition from one mode to the other. How can I tell OSVVM that this error is expected. The AffirmIF runs multiple times in the loops and in one loop iteration it fails. The iteration may be different depending on the timeout settings, so i cannot just check the iteration count. I would rather have a way to still let the test “pass” even when it has specifically 1 failure. I seem to recall having read this somewhere but cannot find it now. Pointers VERY welcome.

[Jim Lewis]

Hi Mikkie
Use a negative value in the ExternalErrors parameter to EndOfTestReports. For example if you have 3 expected errors:

EndOfTestReports(ExternalErrors => (FAILURE => 0, ERROR => -3, WARNING => 0)) ;

The value is summed up with its corresponding field, then the absolute value is taken, and finally the separate values are summed to determine the number of test errors. Hence, if you expect 3 errors and get 2 errors and 1 failure, the test will fail with 2 test case errors.

OSVVM also has a request for a mechanism that would be used for known failures that we will eventually fix. I am looking at how to address this in the reporting.

Cheers,
Jim

[Mikkie]

Fantastic, thanks for the quick reponse.

[Author]

Posts