diff --git a/bin/ci_check b/bin/ci_check index 620e89841c..b139c51c23 100755 --- a/bin/ci_check +++ b/bin/ci_check @@ -2,13 +2,13 @@ ################################################################################ # # Copyright 2020 OpenHW Group -# +# # Licensed under the Solderpad Hardware Licence, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at -# +# # https://solderpad.org/licenses/ -# +# # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. @@ -16,7 +16,7 @@ # limitations under the License. # # SPDX-License-Identifier:Apache-2.0 WITH SHL-2.0 -# +# ################################################################################ # # ci_check: python script to run a sanity regression. Intended to be used @@ -31,14 +31,12 @@ # Written with Python 3.6.9 on Ubuntu 18.04. Your python mileage may vary. # # Restriction: -# - Blindly uses .metrics.json wih no ability for user over-ride. +# - Blindly uses .metrics.json with no ability for user over-ride. # # TODO: # 1. Check results using the "isPass" key. -# 2. Don't assume DSIM_WORK and DSIM_RESULTS are always at the end of the -# "cmd" key. -# 3. Handle Verilator a little more elegantly. -# 4. Terminate if compile fails. +# 2. Handle Verilator a little more elegantly. +# 3. Terminate if compile fails. ################################################################################ import json @@ -284,8 +282,9 @@ if (uvm): if (key['name'] != 'uvmt_{}'.format(args.core.lower())): continue build_cmd_list = (key['cmd']).split() - build_cmd = ' '.join(build_cmd_list[0:-1]) # See TODO #3 + build_cmd = ' '.join(build_cmd_list[0:-1]) build_cmd = build_cmd.replace(' DSIM_WORK=/mux-flow/build/repo/dsim_work', '') + build_cmd = build_cmd.replace(' DSIM_RESULTS=/mux-flow/build/results', '') if (build_cmd != ''): build_cmd = build_cmd.replace('dsim', svtool) if (args.repo): diff --git a/cv32e40p/regress/cv32e40p_ci_check.yaml b/cv32e40p/regress/cv32e40p_ci_check.yaml index eefeadd693..a523e84de4 100644 --- a/cv32e40p/regress/cv32e40p_ci_check.yaml +++ b/cv32e40p/regress/cv32e40p_ci_check.yaml @@ -54,11 +54,11 @@ tests: dir: cv32e40p/sim/uvmt cmd: make test COREV=YES TEST=riscv_arithmetic_basic_test_0 - corev_arithmetic_base_test: + corev_rand_arithmetic_base_test: build: uvmt_cv32e40p description: Generated corev-dv random arithmetic test dir: cv32e40p/sim/uvmt - cmd: make gen_corev-dv test COREV=YES TEST=corev_arithmetic_base_test + cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_arithmetic_base_test num: 2 corev_rand_instr_test: @@ -68,10 +68,10 @@ tests: cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_instr_test num: 2 - corev_jump_stress_test: + corev_rand_jump_stress_test: build: uvmt_cv32e40p description: Generated corev-dv jump stress test dir: cv32e40p/sim/uvmt - cmd: make gen_corev-dv test COREV=YES TEST=corev_jump_stress_test + cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_jump_stress_test num: 2 diff --git a/cv32e40p/regress/cv32e40p_full.yaml b/cv32e40p/regress/cv32e40p_full.yaml index dd68e7f6ae..07505066bc 100644 --- a/cv32e40p/regress/cv32e40p_full.yaml +++ b/cv32e40p/regress/cv32e40p_full.yaml @@ -114,11 +114,11 @@ tests: cmd: make test COREV=YES TEST=riscv_arithmetic_basic_test_1 num: 1 - corev_arithmetic_base_test: + corev_rand_arithmetic_base_test: build: uvmt_cv32e40p description: Generated corev-dv arithmetic test dir: cv32e40p/sim/uvmt - cmd: make gen_corev-dv test COREV=YES TEST=corev_arithmetic_base_test + cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_arithmetic_base_test num: 2 corev_rand_instr_test: @@ -135,11 +135,11 @@ tests: cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_illegal_instr_test num: 2 - corev_jump_stress_test: + corev_rand_jump_stress_test: build: uvmt_cv32e40p description: Generated corev-dv jump stress test dir: cv32e40p/sim/uvmt - cmd: make gen_corev-dv test COREV=YES TEST=corev_jump_stress_test + cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_jump_stress_test num: 2 corev_rand_interrupt: diff --git a/cv32e40p/regress/cv32e40p_full_covg_no_pulp.yaml b/cv32e40p/regress/cv32e40p_full_covg_no_pulp.yaml index 914f962ac3..92e2b1d84e 100644 --- a/cv32e40p/regress/cv32e40p_full_covg_no_pulp.yaml +++ b/cv32e40p/regress/cv32e40p_full_covg_no_pulp.yaml @@ -132,11 +132,11 @@ tests: cmd: make test COREV=YES TEST=riscv_arithmetic_basic_test_1 num: 1 - corev_arithmetic_base_test: + corev_rand_arithmetic_base_test: build: uvmt_cv32e40p description: Generated corev-dv arithmetic test dir: cv32e40p/sim/uvmt - cmd: make gen_corev-dv test COREV=YES TEST=corev_arithmetic_base_test + cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_arithmetic_base_test num: 200 corev_rand_instr_test: @@ -160,11 +160,11 @@ tests: cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_illegal_instr_test num: 200 - corev_jump_stress_test: + corev_rand_jump_stress_test: build: uvmt_cv32e40p description: Generated corev-dv jump stress test dir: cv32e40p/sim/uvmt - cmd: make gen_corev-dv test COREV=YES TEST=corev_jump_stress_test + cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_jump_stress_test num: 200 corev_rand_interrupt: diff --git a/cv32e40p/tests/programs/corev-dv/corev_arithmetic_base_test/.gitignore b/cv32e40p/tests/programs/corev-dv/corev_rand_arithmetic_base_test/.gitignore similarity index 100% rename from cv32e40p/tests/programs/corev-dv/corev_arithmetic_base_test/.gitignore rename to cv32e40p/tests/programs/corev-dv/corev_rand_arithmetic_base_test/.gitignore diff --git a/cv32e40p/tests/programs/corev-dv/corev_arithmetic_base_test/corev-dv.yaml b/cv32e40p/tests/programs/corev-dv/corev_rand_arithmetic_base_test/corev-dv.yaml similarity index 90% rename from cv32e40p/tests/programs/corev-dv/corev_arithmetic_base_test/corev-dv.yaml rename to cv32e40p/tests/programs/corev-dv/corev_rand_arithmetic_base_test/corev-dv.yaml index cf2fb352e6..2223c1e173 100644 --- a/cv32e40p/tests/programs/corev-dv/corev_arithmetic_base_test/corev-dv.yaml +++ b/cv32e40p/tests/programs/corev-dv/corev_rand_arithmetic_base_test/corev-dv.yaml @@ -1,6 +1,6 @@ # Test definition YAML for corev-dv test generator -name: corev_arithmetic_base_test +name: corev_rand_arithmetic_base_test uvm_test: $(CV_CORE_LC)_instr_base_test description: > RISCV-DV generated arithmetic test diff --git a/cv32e40p/tests/programs/corev-dv/corev_arithmetic_base_test/test.yaml b/cv32e40p/tests/programs/corev-dv/corev_rand_arithmetic_base_test/test.yaml similarity index 78% rename from cv32e40p/tests/programs/corev-dv/corev_arithmetic_base_test/test.yaml rename to cv32e40p/tests/programs/corev-dv/corev_rand_arithmetic_base_test/test.yaml index 0190744f37..66bb01c51f 100644 --- a/cv32e40p/tests/programs/corev-dv/corev_arithmetic_base_test/test.yaml +++ b/cv32e40p/tests/programs/corev-dv/corev_rand_arithmetic_base_test/test.yaml @@ -1,7 +1,7 @@ # Test definition YAML for generated corev arithmetic base test -name: corev_arithmetic_base_test +name: corev_rand_arithmetic_base_test uvm_test: uvmt_$(CV_CORE_LC)_firmware_test_c description: > Math test generated by corev-dv - \ No newline at end of file + diff --git a/cv32e40p/tests/programs/corev-dv/corev_jump_stress_test/.gitignore b/cv32e40p/tests/programs/corev-dv/corev_rand_jump_stress_test/.gitignore similarity index 100% rename from cv32e40p/tests/programs/corev-dv/corev_jump_stress_test/.gitignore rename to cv32e40p/tests/programs/corev-dv/corev_rand_jump_stress_test/.gitignore diff --git a/cv32e40p/tests/programs/corev-dv/corev_jump_stress_test/corev-dv.yaml b/cv32e40p/tests/programs/corev-dv/corev_rand_jump_stress_test/corev-dv.yaml similarity index 87% rename from cv32e40p/tests/programs/corev-dv/corev_jump_stress_test/corev-dv.yaml rename to cv32e40p/tests/programs/corev-dv/corev_rand_jump_stress_test/corev-dv.yaml index a53c15761a..be4675955c 100644 --- a/cv32e40p/tests/programs/corev-dv/corev_jump_stress_test/corev-dv.yaml +++ b/cv32e40p/tests/programs/corev-dv/corev_rand_jump_stress_test/corev-dv.yaml @@ -1,6 +1,6 @@ # Test definition YAML for corev-dv test generator -name: corev_jump_stress_test +name: corev_rand_jump_stress_test uvm_test: $(CV_CORE_LC)_instr_base_test description: > RISCV-DV generated jump stress test diff --git a/cv32e40p/tests/programs/corev-dv/corev_jump_stress_test/test.yaml b/cv32e40p/tests/programs/corev-dv/corev_rand_jump_stress_test/test.yaml similarity index 83% rename from cv32e40p/tests/programs/corev-dv/corev_jump_stress_test/test.yaml rename to cv32e40p/tests/programs/corev-dv/corev_rand_jump_stress_test/test.yaml index 06a344790a..5c1f28ca2e 100644 --- a/cv32e40p/tests/programs/corev-dv/corev_jump_stress_test/test.yaml +++ b/cv32e40p/tests/programs/corev-dv/corev_rand_jump_stress_test/test.yaml @@ -1,6 +1,6 @@ # Test definition YAML for generated corev arithmetic base test -name: corev_jump_stress_test +name: corev_rand_jump_stress_test uvm_test: uvmt_$(CV_CORE_LC)_firmware_test_c description: > Jump stress test generated by corev-dv diff --git a/mk/uvmt/README.md b/mk/uvmt/README.md index ab9764bd3f..7084ebb8ca 100644 --- a/mk/uvmt/README.md +++ b/mk/uvmt/README.md @@ -5,7 +5,7 @@ The UVM testcases are at `CV_CORE/tests/uvmt`, and the test-programs can be found in `CV_CORE/tests/program`. See the README in those directories for more information.

Please refer to the [Verification Strategy](https://core-v-docs-verif-strat.readthedocs.io/en/latest/sim_tests.html#simulation-tests-in-the-uvm-environments) -for a discussion on the distinction between a _testcase_ (a SystemVerilog class extended from _uvm\_test_ that instantiates and controls the UVM environment) and a _test-program_ (a C or assembler program that runs on the core RTL model) in this environment. +for a discussion on the distinction between a _testcase_ (a SystemVerilog class extended from uvm_test that instantiates and controls the UVM environment) and a _test-program_ (a C or assembler program that runs on the core RTL model) in this environment.

To run the UVM environment you will need: - a run-time license for the Imperas OVPsim Instruction Set Simulator @@ -164,10 +164,11 @@ Before making changes to the code in your local branch, it is a good idea to run test to ensure you are starting from a stable code-base. The code (both RTL and verification) should _always_ pass sanity, so if it does not, please raise an issue and assign it to @mikeopenhwgroup. The definition of "sanity" -will change over time as the ability of the verification environment to +may change over time as the ability of the verification environment to stress the RTL improves. Running sanity is trivial: -

-**make sanity** +``` +make sanity +``` CI Mini-regression ------------------ @@ -176,14 +177,15 @@ located at the top-level of this repository. A pythin script `ci/ci_check` can "cv32 CI check regression" specified in the control script. Before issuing a pull-request for either the RTL or verification code, please run `ci_check`. Your pull-request will be rejected if `ci_check` does not compile and run successfully. Usage is simple: -
-**./ci__check -s xrun** -
-will run the CI sanity regression using Xcelium. +``` +./ci_check --core cv32e40p -s xrun +``` +will run the CI sanity regression on the cv32e40p using Xcelium.

Complete user information is obtained in the usual way: -
-**./ci__check -h** +``` +./ci_check -h +``` Available Test Programs ----------------------- @@ -201,9 +203,9 @@ Here are a few examples * **make test TEST=riscv_arithmetic_basic_test**:
run the riscv_arithmetic_basic_test program found at `/tests/programs/custom`.
There are also a few targets that do something other than run a test. The most popular is: -
+``` **make clean_all** -
+``` which deletes all SIMULATOR generated intermediates, waves and logs **plus** the cloned RTL code. COREV-DV Generated Tests @@ -218,14 +220,14 @@ up-to-date with the latest release of riscv-dv. Riscv-dv uses test templates such as "riscv_arithmetic_basic_test" and "riscv_rand_jump_test". Corev-dv has a set of templates for corev-dv generated test-programs at `/tests/programs/corev-dv`. Running these is a two-step process. The first step is to clone riscv-dv and compile corev-dv: -

-**make corev-dv** -

+``` +make corev-dv +``` Note that the `corev-dv` target need only be run once. The next step is to generate, compile and run a test. For example: -

-**make gen_corev-dv test TEST=corev_rand_jump_stress_test** -
+``` +make gen_corev-dv test TEST=corev_rand_jump_stress_test +``` RISC-V Compliance Test-suite and Regressions ---------------