riscv_compliance_tests error: "I-SB-01.S" and "I-SH-01.S" not passed

[shjdzc12]

I execute the following command in the riscv/tb/core directory:

$ make firmware-vcs-run VCS_FLAGS+="-cm line+cond+fsm+tgl+branch" SIMV_FLAGS+="-cm line+cond+fsm+tgl+branch"

and I get the following results:

I_SB_01..Test Begin Reserved regs ra(x1) a0(x10) t0(x5)
# Test part A1 - test base address + 0
# Test part A1  - Complete
# Test part A2 - test base address - 1
# Test part A2  - Complete
# Test part A3 - test base address + 1
# Test part A3  - Complete
# Test part A4 - test base address - 2048
# Test part A4  - Complete
# Test part A5 - test base address + 2047
# Test part A5  - Complete
# Test part B - test base address + -4, -3, ... , 7
# Test part B  - Complete
# Test part C - test store x0
# Test part C  - Complete
# Test part D1 - test for forwarding (to address register)
# Test part D  - Complete
# Test part D2 - test for forwarding (to data register)
# Test part E  - Complete
# Test part E1 - test war hazard (data register)
# Test part A1  - Complete
# Test part E2 - test war hazard (address register)
Assertion violation: file riscv_compliance_tests/I-SB-01.S, line 279: x28() != 0x8000403C
ERROR
# Test part A2  - Complete
# Test part F - test raw hazard in memory
# Test part A3  - Complete
# Test part G - test waw hazard in memory
# Test part A4  - Complete
Test End
OK
I_SH_01..# Test Begin Reserved regs ra(x1) a0(x10) t0(x5)
# Test part A1 - test base address + 0
# Test part A1  - Complete
# Test part A2 - test base address - 1
# Test part A2  - Complete
# Test part A3 - test base address + 1
# Test part A3  - Complete
# Test part A4 - test base address - 2048
# Test part A4  - Complete
# Test part A5 - test base address + 2047
# Test part A5  - Complete
# Test part B - test base address + -4, -2, ... , 6
# Test part B  - Complete
# Test part C - test store x0
# Test part C  - Complete
# Test part D1 - test for forwarding (to address register)
# Test part D  - Complete
# Test part D2 - test for forwarding (to data register)
# Test part E  - Complete
# Test part E1 - test war hazard (data register)
# Test part A1  - Complete
# Test part E2 - test war hazard (address register)
Assertion violation: file riscv_compliance_tests/I-SH-01.S, line 261: x28() != 0x8000403C
ERROR
# Test part A2  - Complete
# Test part F - test raw hazard in memory
# Test part A3  - Complete
# Test part G - test waw hazard in memory
# Test part A4  - Complete
# Test End

The result shows that the core can't pass the E2 part of the tests named I-SB-01.S and I-SH-01.S.
I open these two .S files in tb/core/riscv_compliance_tests directory and find the code of E2 part as follows:

1、I-SB-01.S:
 RVTEST_IO_WRITE_STR("# Test part E2 - test war hazard (address register)\n");
    # Address for test results
    la      x28, test_E2_res
    # Clear memory
    sw      x0, 0(x28)
    # Test
    li      x27, 0x89ABCDEF
    sb      x27, 0(x28)
    addi    x28, x28, -4
    RVTEST_IO_ASSERT_GPR_EQ(x27, 0x89ABCDEF)
    RVTEST_IO_ASSERT_GPR_EQ(x28, 0x8000403C)
    RVTEST_IO_WRITE_STR("# Test part A2  - Complete\n");
    #
test_E2_res:
    .fill 1, 4, -1

2、I-SH-01.S:
RVTEST_IO_WRITE_STR("# Test part E2 - test war hazard (address register)\n");
    # Address for test results
    la      x28, test_E2_res
    # Clear memory
    sw      x0, 0(x28)
    # Test
    li      x27, 0x89ABCDEF
    sh      x27, 0(x28)
    addi    x28, x28, -4
    RVTEST_IO_ASSERT_GPR_EQ(x27, 0x89ABCDEF)
    RVTEST_IO_ASSERT_GPR_EQ(x28, 0x8000403C)
    RVTEST_IO_WRITE_STR("# Test part A2  - Complete\n");
    #
test_E2_res:
    .fill 1, 4, -1

I think x28 gets value from the commands la x28, test_E2_res and addi x28, x28, -4.
So x28=test_E2_res+PC - 4, am I right?
Therefore why can the test suppose x28's standard value to be 0x8000403C because in my opinion, its value is influenced by PC and can't be expected. In fact, in my simulation, x28 is equal to 0x0006f73c.
Please give me some explanations about this problem and my "riscv_compliance_tests not passed" error. I really need your help.

[bluewww]

Indeed this assertion is just nonsense. I think these hardcoded values are from spike and its linkerscript. See the issue I opened quite a while ago

riscv-non-isa/riscv-arch-test#28

Thanks for reporting!

I thought I removed that buggy assertion but let me check again

[shjdzc12]

Indeed this assertion is just nonsense. I think these hardcoded values are from spike and its linkerscript. See the issue I opened quite a while ago

riscv/riscv-compliance#28

Thanks for reporting!

I thought I removed that buggy assertion but let me check again

Thank for your reply. I have another problem about what riscv-tests and riscv-compliance tests exactly refer to? I see the following explanations in readme:

riscv-tests(rv32ui, rv32uc) and riscv-compliance-tests(rv32i)

It may be too brief for me to understand the meaning of riscv-tests and riscv-compliance tests. I have a lot of questions about them. For example, what's the difference between "ui" and "i" and what's the meaning of "compliance" in this part. Could you please explain them to me?

[bluewww]

m and u mean machine mode and user mode respectively. The letters i (integer), m (multiplication), a (atomics), f (floats), d (doubles), c (compressed) etc. are all instructions extensions. It's best if you study the isa manual

riscv-tests and riscv-compliance-test are just two test suites. The latter has a more ambitious goal which you can read about here.

[bluewww]

This bug is an upstream issue with riscv-tests. See riscv-non-isa/riscv-arch-test#28. Once it's fixed there we will import the changes.

[Silabs-ArjanB]

Hi @shjdzc12 This issue will be further tracked in openhwgroup/core-v-verif#46. The RISC-V compliance tests will be removed from this cv32e40p repos and be revived in https://github.com/openhwgroup/core-v-verif. Could you please close this issue and address related issues in https://github.com/openhwgroup/core-v-verif instead?

[shjdzc12]

Hi @shjdzc12 This issue will be further tracked in openhwgroup/core-v-verif#46. The RISC-V compliance tests will be removed from this cv32e40p repos and be revived in https://github.com/openhwgroup/core-v-verif. Could you please close this issue and address related issues in https://github.com/openhwgroup/core-v-verif instead?

Certainly. I will do that!

[Silabs-ArjanB]

Hi @shjdzc12 We are closing this issue as it either has been fixed with the stated (merged) pull request, because it is no longer valid, or because a follow up question for further clarification or feedback remained unanswered for at least 7 days. Please do not hesitate to re-open this issue if you do not agree that the issue has been handled satisfactorily or if you have further questions. Thank you for contributing to this core; raising issues will help to improve it.