Reply To: Is there a built-in way in OSVVM to detect build failure without parsing logs?

Why OSVVM™? Forums OSVVM Is there a built-in way in OSVVM to detect build failure without parsing logs? Reply To: Is there a built-in way in OSVVM to detect build failure without parsing logs?

#2861
Jim Lewis
Member

By failed, do you mean test case failed or the build process errored out for some reason? OSVVM has are controls for both of these.

From the OSVVM Settings User Guide, you will find that they are controlled by variables:

variable FailOnBuildErrors        "true"    ;# simulator command had errors
variable FailOnReportErrors       "false"   ;# yaml reports caused html failure
variable FailOnTestCaseErrors     "false"   ;# one or more test case(s) had errors

Change these settings in your OsvvmSettingsLocal.tcl. See the OSVVM Settings User Guide for details.

If you look at the proc build in OsvvmScriptsCore.tcl, you will find the following code at the end. This code shows how the variable settings work.

      #
      #  Wrap up with error handling via call backs
      #
      # Run Callbacks on Error after trying to produce all reports
      if {$BuildErrorCode != 0} {   
        CallbackOnError_Build $Path_Or_File $BuildErrMsg $LocalBuildErrorInfo 
      } 
      if {$AnalyzeErrorCount > 0 || $SimulateErrorCount > 0} {   
        CallbackOnError_Build $Path_Or_File "Failed with Analyze Errors: $AnalyzeErrorCount and/or Simulate Errors: $SimulateErrorCount" $LocalBuildErrorInfo 
      } 
      if {($ReportErrorCode != 0) || ($ScriptErrorCount != 0)} {  
        CallbackOnError_AfterBuildReports $LocalReportErrorInfo
      } 
      # Fail on Test Case Errors
      if {($::osvvm::BuildStatus ne "PASSED") && ($::osvvm::FailOnTestCaseErrors)} {
          error "Test finished with Test Case Errors"
      }
      # Fail on Report / Script Errors?
      if {($ReportYamlErrorCode != 0) || ($ReportErrorCode != 0) || ($Log2ErrorCode != 0) || ($ScriptErrorCount != 0)} {  
        # End Simulation with errors
        if {$::osvvm::FailOnReportErrors} {
          error "Test finished with either Report or Script (wave.do) errors."
        }
      } 

While these controls are enough to get the information to Tcl, it is sometimes not enough to get the errors signaled outside of your simulator – and signaled to your Continuous Integration tools. For example, if you are running Questa (or RivieraPRO) you need to run the simulation by doing:
vsim -c -do "exit -code [catch {source MyScript.tcl}]

I generally break out the steps as follows (I generally set OsvvmLibraries as an environment variable):
vsim -c -do "exit -code [catch {source $OsvvmLibraries/Scripts/StartUp.tcl ; LinkLibraryDirectory ; build RunTest.pro}]

If you are running another simulator and have to do something like this, please share what you did with the community.