All AffirmIf Pass, but OSVVM HTML Report Returns NOCHECKS

Why OSVVM™? Forums OSVVM All AffirmIf Pass, but OSVVM HTML Report Returns NOCHECKS

Viewing 7 posts - 1 through 7 (of 7 total)
  • Author
    Posts
  • #2741
    inan ulas
    Member

    Hello,
    I want to state that I am a beginner in Osvvm therefore I could be missing some concepts. I am using OSVVM with GHDL v5.1.1 on Windows 11, with a 2-bit comparator design and trying to generate logs and HTML test report. I can see the PASSED result of AffirmIF’s on the logs and can see the functional coverage result as intended however, the test overall is labeled as FAILED due to NOCHECKS status.
    I have a checker process which I use AffirmIf inside:

    Checker: process
        variable a_int, b_int : integer;
        variable expected_less, expected_equal, expected_greater : std_logic;
    begin
        loop
            wait on A, B, A_less_B, A_equal_B, A_greater_B;
            wait for 10 ns;  -- output'lar?n olu?mas?n? bekle
    
            a_int := to_integer(unsigned(A));
            b_int := to_integer(unsigned(B));
    
            -- Beklenen de?erler
            if a_int < b_int then
                expected_less := '1'; expected_equal := '0'; expected_greater := '0';
            elsif a_int = b_int then
                expected_less := '0'; expected_equal := '1'; expected_greater := '0';
            else
                expected_less := '0'; expected_equal := '0'; expected_greater := '1';
            end if;
    
            AffirmIf(A_less_B = expected_less, "A_less_B : " & to_string(A_less_B),"Expected : " & to_string(expected_less));
            AffirmIf(A_equal_B = expected_equal, "A_equal_B : " & to_string(A_equal_B),"Expected : " & to_string(expected_equal));
            AffirmIf(A_greater_B = expected_greater, "A_greater_B : " & to_string(A_greater_B),"Expected : " & to_string(expected_greater));
            ReportAlerts;    
        end loop;
    end process;

    I can see the results on Logs as:
    %% 870 ns DONE PASSED tb_osvvm_comparator_VHDL Passed: 234 Affirmations Checked: 234
    %% 880 ns Log PASSED A_less_B : 1
    %% 880 ns Log PASSED A_equal_B : 0
    %% 880 ns Log PASSED A_greater_B : 0
    %% 880 ns DONE PASSED tb_osvvm_comparator_VHDL Passed: 237 Affirmations Checked: 237
    and at the end:
    ../tb_osvvm_comparator_VHDL.vhd:145:5:@881ns:(report note): Coverage holes: 0
    %% 881 ns DONE PASSED tb_osvvm_comparator_VHDL Passed: 237 Affirmations Checked: 237
    Simulation Finish time 11:21:45, Elapsed time: 0:00:01

    Hoever I get build error at the end as:
    BuildError: 2bitcomparator_sim FAILED, Passed: 0, Failed: 1, Skipped: 0, Analyze Errors: 0, Simulate Errors: 0, Build Error Code: 0
    and when I look at the HTML file the STATUS is NOCHECKS.

    I used SetTestName, ReportAlerts, EndofTestReports, WriteAlertSummaryYaml but couldnt achieve a working end report.

    Also this is the .pro file I am building:

    library osvvm “C:/OsvvmLibraries/VHDL_LIBS/GHDL-5.1.1/osvvm/v08”
    library work .
    analyze comparator_src.vhd
    TestName tb_osvvm_comparator_VHDL
    analyze tb_osvvm_comparator_VHDL.vhd
    simulate tb_osvvm_comparator_VHDL

    I would be glad if someone can help me and I would appreciate all the warnings, advices and questions regarding my code.
    Best Regards

    Here is my full testbench code if you want to look: (design file named comparator_src.vhd and testbench named tb_osvvm_comparator_VHDL.vhd)

    library IEEE;
    use IEEE.Std_logic_1164.all;
    use IEEE.Numeric_Std.all;
    library osvvm;
    context osvvm.OsvvmContext ;
    --use osvvm.RandomPkg.all;
    --use osvvm.CoveragePkg.all;
    
    entity tb_osvvm_comparator_VHDL is
    end tb_osvvm_comparator_VHDL;
    
    ----------------------------------
    ----------------------------------
    architecture Behavioral of tb_osvvm_comparator_VHDL is
    
    ------------------------------------------------------
    --Signal Declaration
    signal A : std_logic_vector(1 downto 0) := (others => '0');
    signal B : std_logic_vector(1 downto 0) := (others => '0');
    signal A_less_B : std_logic := '0';   
    signal A_equal_B : std_logic := '0';  
    signal A_greater_B : std_logic := '0';
    signal STOP : boolean := false;
    
    signal ExternalErrors : AlertCountType := (others => 0);
    
    ------------------------------------------------------
    --Constant Declaration
    constant OP_DELAY : TIME := 10 ns;
    
    --constant TimeOut : integer := 0;
    
    ------------------------------------------------------
    --Coverage Variables
    shared variable cp_A : covPType;
    shared variable cp_B : covPType;
    shared variable cp_A_B : covPType;
    
    begin
    SetTestName("tb_osvvm_comparator_VHDL");    
    SetLogEnable(INFO, TRUE);
    SetLogEnable(PASSED, TRUE);
    
    ------------------------------------------------------
    -- DUT Instantiation
    UUT : entity work.comparator_src
        port map (
            A => A,
            B => B,
            A_less_B => A_less_B,
            A_equal_B => A_equal_B,
            A_greater_B => A_greater_B
        );
    
    ------------------------------------------------------
    -- Stimulus Generator
    Stim: process
        variable RandA : RandomPType;
        variable RandB : RandomPType;
        variable allDone : boolean := false;
        variable nCycles : natural := 0;
    begin
        RandA.InitSeed(RandA'instance_name);
        RandB.InitSeed(RandB'instance_name);
    
        while not allDone and (NOW < 1 ms) loop
            A <= RandA.Randslv(0, 3,2);
            B <= RandB.Randslv(0, 3,2);
            wait for OP_DELAY;
    
            allDone := cp_A_B.isCovered;
            nCycles := nCycles + 1;
        end loop;
    
        wait for 1 ns;
        report "Number of simulation cycles = " & to_string(nCycles);
        STOP <= true;
        wait;
    end process;
    
    ------------------------------------------------------
    -- Coverage Bin Setup
    InitCoverage: process
    begin
        cp_A.AddBins(GenBin(0, 3));
        cp_B.AddBins(GenBin(0, 3));
        cp_A_B.AddCross(GenBin(0, 3), GenBin(0, 3));
        wait;
    end process;
    
    ------------------------------------------------------
    -- Sampling Coverage
    Sample: process
    begin
        loop
            wait on A, B;
            wait for 1 ns;
            cp_A.ICover(to_integer(unsigned(A)));
            cp_B.ICover(to_integer(unsigned(B)));
            cp_A_B.ICover((to_integer(unsigned(A)), to_integer(unsigned(B))));
        end loop;
    end process;
    
    ------------------------------------------------------
    -- Functional Checker (Self-Checking Logic)
    Checker: process
        variable a_int, b_int : integer;
        variable expected_less, expected_equal, expected_greater : std_logic;
    begin
        loop
            wait on A, B, A_less_B, A_equal_B, A_greater_B;
            wait for 10 ns;  -- output'lar?n olu?mas?n? bekle
    
            a_int := to_integer(unsigned(A));
            b_int := to_integer(unsigned(B));
    
            -- Beklenen de?erler
            if a_int < b_int then
                expected_less := '1'; expected_equal := '0'; expected_greater := '0';
            elsif a_int = b_int then
                expected_less := '0'; expected_equal := '1'; expected_greater := '0';
            else
                expected_less := '0'; expected_equal := '0'; expected_greater := '1';
            end if;
    
            AffirmIf(A_less_B = expected_less, "A_less_B : " & to_string(A_less_B),"Expected : " & to_string(expected_less));
            AffirmIf(A_equal_B = expected_equal, "A_equal_B : " & to_string(A_equal_B),"Expected : " & to_string(expected_equal));
            AffirmIf(A_greater_B = expected_greater, "A_greater_B : " & to_string(A_greater_B),"Expected : " & to_string(expected_greater));
            ReportAlerts;    
        end loop;
    end process;
    
    ------------------------------------------------------
    -- Report Coverage
    CoverReport: process
    begin
        wait until STOP;
        report "A Coverage details";
        cp_A.WriteBin;
        report "B Coverage details";
        cp_B.WriteBin;
        report "AxB Coverage details";
        cp_A_B.WriteBin;
        report "Coverage holes: " & to_string(cp_A_B.CountCovHoles);
        ReportAlerts;
    end process;
    
    WriteAlertYaml(
      FileName => "tb_osvvm_comparator_VHDL",
      ExternalErrors => (0, 0, 0),
      Prefix => "",
      PrintSettings => TRUE,
      PrintChildren => TRUE,
      OpenKind => WRITE_MODE,
      TimeOut => FALSE
    );
    
    WriteAlertSummaryYaml("tb_osvvm_comparator_VHDL", (0, 0, 0));
    
    EndOfTestReports(
      ReportAll => TRUE,
      ExternalErrors => (0, 0, 0),
      Stop => FALSE,
      TimeOut => FALSE
    );
    
    end Behavioral;
    #2742
    Jim Lewis
    Member

    Thanks for posting your whole testbench as it makes the issue clear.

    You are calling EndOfTestReports concurrently. So it runs at time 0 ns. You do not want it to run until your test case has finished. Same goes for WriteAlertYaml and WriteAlertSummaryYaml. However, you do not need to call these or ReportAlerts as they are all called by EndOfTestReports.

    So a quick fix is to update your stimulus process as follows:

    
    -- Stimulus Generator
    Stim: process
        variable RandA : RandomPType;
        variable RandB : RandomPType;
        variable allDone : boolean := false;
        variable nCycles : natural := 0;
    begin
        RandA.InitSeed(RandA'instance_name);
        RandB.InitSeed(RandB'instance_name);
    
        while not allDone and (NOW < 1 ms) loop
            A <= RandA.Randslv(0, 3,2);
            B <= RandB.Randslv(0, 3,2);
            wait for OP_DELAY;
    
            allDone := cp_A_B.isCovered;
            nCycles := nCycles + 1;
        end loop;
    
        wait for 1 ns;
        report "Number of simulation cycles = " & to_string(nCycles);
        report "Coverage holes: " & to_string(cp_A_B.CountCovHoles);
    
        EndOfTestReports(
          ReportAll => TRUE,
          ExternalErrors => (0, 0, 0),
          Stop => FALSE,
          TimeOut => FALSE
        );
    
        STOP <= true;
        wait;
    end process;

    Try that, that should get you running.

    #2743
    inan ulas
    Member

    Hi Jim,
    Thanks a lot for taking the time to help out. Your advice fixed the issue — really appreciate your attention and effort.

    #2744
    Jim Lewis
    Member

    This step is recommended, but not required. Do it after you get the above running.

    While I SetTestName, SetLogEnable can be called concurrently, it is more normal to call them in the process that is controlling the overall test. Currently this is the “Stim” process but we will return to this:

    Stim: process
        variable RandA : RandomPType;
        variable RandB : RandomPType;
        variable allDone : boolean := false;
        variable nCycles : natural := 0;
    begin
        SetTestName("tb_osvvm_comparator_VHDL");    
        SetLogEnable(INFO, TRUE);
        SetLogEnable(PASSED, TRUE);
    
        RandA.InitSeed(RandA'instance_name);
        RandB.InitSeed(RandB'instance_name);
    
    #2745
    Jim Lewis
    Member

    This step is recommended, but not required. Do it after you get the above running.

    To get the HTML based Functional Coverage reports, you need to use the singleton rather than the older, deprecated shared variable approach.

    This requires updating your architecture declarations, “InitCoverage”, “Sample”, and “CoverageReport” processes as follows. I would also recommend that you put your “InitCoverage” in the same process as “Sample”.

    
    architecture Behavioral of tb_osvvm_comparator_VHDL is
    ------------------------------------------------------
    --Coverage IDs
    signal cp_A    : CoverageIDType;
    signal cp_B    : CoverageIDType;
    signal cp_A_B  : CoverageIDType;
    
    begin
    
    ------------------------------------------------------
    -- Coverage Bin Setup
    InitCoverage: process
    begin
        cp_A <= NewID("cp_A") ; 
        cp_B <= NewID("cp_B") ; 
        cp_A_B <= NewID("cp_A_B") ; 
        wait for 0 ns ; -- since cp_A, ... are signals 
        AddBins(cp_A, GenBin(0, 3));
        AddBins(cp_B, GenBin(0, 3));
        AddCross(cp_A_B, GenBin(0, 3), GenBin(0, 3));
        wait;
    end process;
    
    ------------------------------------------------------
    -- Sampling Coverage
    Sample: process
    begin
        loop
            wait on A, B;
            wait for 1 ns; -- not needed.  A and B are updated.  Output not necessarily stable, but not looking at it so OK.
            ICover(cp_A, to_integer(unsigned(A)));
            ICover(cp_B, to_integer(unsigned(B)));
            ICover(cp_A_B, (to_integer(unsigned(A)), to_integer(unsigned(B))));
        end loop;
    end process;
    
    ------------------------------------------------------
    -- Report Coverage
    CoverReport: process
    begin
        wait until STOP;
        report "A Coverage details";
        WriteBin(cp_A) ;
        report "B Coverage details";
        WriteBin(cp_B) ;
        report "AxB Coverage details";
        WriteBin(cp_A_B);
        report "Coverage holes: " & to_string(CountCovHoles(cp_A_B));
        ReportAlerts;
    end process;
    

    Note, I have not tested this code, so there may be some typos. Found one and fixed it WRT CountCovHoles.

    Note that the “CoverReport” process is not really needed since now you have HTML reports that are much better than the text reports produced by WriteBin. The only thing that may be interesting is the report at the end of the “CoverReport” process. In the first step, I moved this to the end of the “Stim” process – this one also needs updated to:

        report "Coverage holes: " & to_string(CountCovHoles(cp_A_B));
    
    #2746
    Jim Lewis
    Member

    One final thing to try. If you like the Alert/AffirmIf printing, you might also like the log printing better than the vhdl Report statement. In the “Stim” process you could change these to logs:

       log("Number of simulation cycles = " & to_string(nCycles));
        log("Coverage holes: " & to_string(CountCovHoles(cp_A_B)));
    

    Logs have levels. These are level ALWAYS, which like report always print. You could give them a level and make them print only when enabled. But here ALWAYS is probably appropriate.

    You could also make it a requirement that the Coverage holes is zero by doing the replacing the log for CountCovHoles with:

    AffirmIfEqual(CountCovHoles(cp_A_B), 0, "Coverage holes") ;
    

    AffirmIfEqual is a short hand for what you were doing with AffirmIf. Your AffirmIf could be updated as follows with AffirmIfEqual. This will add the word “Actual” before the first value, but otherwise, it is the same as your output.

    AffirmIfEqual(A_less_B, expected_less, "A_less_B");
    
    #2747
    Jim Lewis
    Member

    If the CoverReport process is removed, then the “Stim” process can end with std.env.stop – which is typically how OSVVM ends the test cases.

    -- Stimulus Generator
    Stim: process
        variable RandA : RandomPType;
        variable RandB : RandomPType;
        variable allDone : boolean := false;
        variable nCycles : natural := 0;
    begin
        SetTestName("tb_osvvm_comparator_VHDL");    
        SetLogEnable(INFO, TRUE);
        SetLogEnable(PASSED, TRUE);
    
        RandA.InitSeed(RandA'instance_name);
        RandB.InitSeed(RandB'instance_name);
    
        while not allDone and (NOW < 1 ms) loop
            A <= RandA.Randslv(0, 3,2);
            B <= RandB.Randslv(0, 3,2);
            wait for OP_DELAY;
    
            allDone := cp_A_B.isCovered;
            nCycles := nCycles + 1;
        end loop;
    
        wait for 1 ns;
        log("Number of simulation cycles = " & to_string(nCycles));
        AffirmIfEqual(CountCovHoles(cp_A_B), 0, "Coverage holes") ;
    
        EndOfTestReports(
          ReportAll => TRUE,
          ExternalErrors => (0, 0, 0),
          Stop => FALSE,
          TimeOut => FALSE
        );
    
        std.env.stop ;
        wait;
    end process;
    

    When the stimulus generation gets more complex than this, I will move the OSVVM runner stuff to separate process called “ControlProc”, but that is not necessary in a simple test case like this one.

Viewing 7 posts - 1 through 7 (of 7 total)
  • You must be logged in to reply to this topic.