Safer Check if Simulation passed or Not

Today the outcome of the simulation is solely dependent on the OSVVM report server and if you have triggered an error or not.
But if you have for example an external library cell/ip that uses assertions, it can still look as the simulation passed while it indeed failed.

I added “assert false” in a testcase to illustrate the problem. This is the output log:

# %%         3510 ns    Log    PASSED    in manager_1: ReadResp SB,      Received: 0   Item Number: 150
# %%         3510 ns    Log    PASSED    in Default,                     slvBurstVector = D001, D003, D005, D007, D009
# ** Error: Assertion violation.
#    Time: 3510 ns  Iteration: 2  Instance: /tbaxi4memory/TestCtrl_1
# ** Note: %%         3530 ns    DONE   PASSED   TbAxi4_DemoMemoryReadWrite1  Passed: 334  Affirmations Checked: 334
#    Time: 3530 ns  Iteration: 3  Instance: /tbaxi4memory/TestCtrl_1

And if you check the generated html file, it also states that the simulation passed. Which is clearly wrong!

The safe way is to check the simulator status, which can be done with a simple tcl procedure, when the simulation is done:
Here is how that would look:

# %%         3510 ns    Log    PASSED    in Default,                     slvBurstVector = D001, D003, D005, D007, D009
# ** Error: Assertion violation.
#    Time: 3510 ns  Iteration: 2  Instance: /tbaxi4memory/TestCtrl_1
# ** Note: %%         3530 ns    DONE   PASSED   TbAxi4_DemoMemoryReadWrite1  Passed: 334  Affirmations Checked: 334
#    Time: 3530 ns  Iteration: 3  Instance: /tbaxi4memory/TestCtrl_1
# Break in Process ControlProc at ../AXI4/Axi4/TestCases/TbAxi4_DemoMemoryReadWrite1.vhd line 83
# Stopped at ../AXI4/Axi4/TestCases/TbAxi4_DemoMemoryReadWrite1.vhd line 83
# process_teststatus
# ** Error:#########################################################################
# ** Error:UCDB Teststatus:FAILED  TESTSTATUS > 1 ACTUAL=2
# **       TSTAT_REASON=
# ** Error:#########################################################################

To make it easy to handle, you just want to check in one place if the simulation passed or failed. Therefor it is also a good idea to trigger the teststatus in the AlertLogPkg.vhd, procedure PrintTopAlerts

   if TestFailed then
        report buf.all severity error ;
      else
        report buf.all severity note ;
      end if ;

      --WriteLine(buf) ;
    end procedure PrintTopAlerts ;

So if we remove the assert and trigger an error by OSSVM, now it looks like this:

# %%      ReadBurstFifo    Failures: 0  Errors: 0  Warnings: 0  Passed: 0
# Break in Process ControlProc at ../AXI4/Axi4/TestCases/TbAxi4_DemoMemoryReadWrite1.vhd line 83
# Stopped at ../AXI4/Axi4/TestCases/TbAxi4_DemoMemoryReadWrite1.vhd line 83
# process_teststatus
# ** Error:#########################################################################
# ** Error:UCDB Teststatus:FAILED  TESTSTATUS > 1 ACTUAL=2
# **       TSTAT_REASON=%         3530 ns    DONE   FAILED   TbAxi4_DemoMemoryReadWrite1  Total Error(s) = 1  Failures: 0  Errors: 1  Warnings: 0  Passed: 333  Affirmations Checked: 334
# ** Error:#########################################################################

This is how Questa works, I’m sure that other simulators have the same mechanism.
BR
Mikael Andersson, Siemens EDA