Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

corev-dv rand tests issues #518

Merged
merged 3 commits into from
Mar 13, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 9 additions & 10 deletions bin/ci_check
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,21 @@
################################################################################
#
# Copyright 2020 OpenHW Group
#
#
# Licensed under the Solderpad Hardware Licence, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#
# https://solderpad.org/licenses/
#
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier:Apache-2.0 WITH SHL-2.0
#
#
################################################################################
#
# ci_check: python script to run a sanity regression. Intended to be used
Expand All @@ -31,14 +31,12 @@
# Written with Python 3.6.9 on Ubuntu 18.04. Your python mileage may vary.
#
# Restriction:
# - Blindly uses .metrics.json wih no ability for user over-ride.
# - Blindly uses .metrics.json with no ability for user over-ride.
#
# TODO:
# 1. Check results using the "isPass" key.
# 2. Don't assume DSIM_WORK and DSIM_RESULTS are always at the end of the
# "cmd" key.
# 3. Handle Verilator a little more elegantly.
# 4. Terminate if compile fails.
# 2. Handle Verilator a little more elegantly.
# 3. Terminate if compile fails.
################################################################################

import json
Expand Down Expand Up @@ -284,8 +282,9 @@ if (uvm):
if (key['name'] != 'uvmt_{}'.format(args.core.lower())):
continue
build_cmd_list = (key['cmd']).split()
build_cmd = ' '.join(build_cmd_list[0:-1]) # See TODO #3
build_cmd = ' '.join(build_cmd_list[0:-1])
build_cmd = build_cmd.replace(' DSIM_WORK=/mux-flow/build/repo/dsim_work', '')
build_cmd = build_cmd.replace(' DSIM_RESULTS=/mux-flow/build/results', '')
if (build_cmd != ''):
build_cmd = build_cmd.replace('dsim', svtool)
if (args.repo):
Expand Down
8 changes: 4 additions & 4 deletions cv32e40p/regress/cv32e40p_ci_check.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,11 +54,11 @@ tests:
dir: cv32e40p/sim/uvmt
cmd: make test COREV=YES TEST=riscv_arithmetic_basic_test_0

corev_arithmetic_base_test:
corev_rand_arithmetic_base_test:
build: uvmt_cv32e40p
description: Generated corev-dv random arithmetic test
dir: cv32e40p/sim/uvmt
cmd: make gen_corev-dv test COREV=YES TEST=corev_arithmetic_base_test
cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_arithmetic_base_test
num: 2

corev_rand_instr_test:
Expand All @@ -68,10 +68,10 @@ tests:
cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_instr_test
num: 2

corev_jump_stress_test:
corev_rand_jump_stress_test:
build: uvmt_cv32e40p
description: Generated corev-dv jump stress test
dir: cv32e40p/sim/uvmt
cmd: make gen_corev-dv test COREV=YES TEST=corev_jump_stress_test
cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_jump_stress_test
num: 2

8 changes: 4 additions & 4 deletions cv32e40p/regress/cv32e40p_full.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -114,11 +114,11 @@ tests:
cmd: make test COREV=YES TEST=riscv_arithmetic_basic_test_1
num: 1

corev_arithmetic_base_test:
corev_rand_arithmetic_base_test:
build: uvmt_cv32e40p
description: Generated corev-dv arithmetic test
dir: cv32e40p/sim/uvmt
cmd: make gen_corev-dv test COREV=YES TEST=corev_arithmetic_base_test
cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_arithmetic_base_test
num: 2

corev_rand_instr_test:
Expand All @@ -135,11 +135,11 @@ tests:
cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_illegal_instr_test
num: 2

corev_jump_stress_test:
corev_rand_jump_stress_test:
build: uvmt_cv32e40p
description: Generated corev-dv jump stress test
dir: cv32e40p/sim/uvmt
cmd: make gen_corev-dv test COREV=YES TEST=corev_jump_stress_test
cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_jump_stress_test
num: 2

corev_rand_interrupt:
Expand Down
8 changes: 4 additions & 4 deletions cv32e40p/regress/cv32e40p_full_covg_no_pulp.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -132,11 +132,11 @@ tests:
cmd: make test COREV=YES TEST=riscv_arithmetic_basic_test_1
num: 1

corev_arithmetic_base_test:
corev_rand_arithmetic_base_test:
build: uvmt_cv32e40p
description: Generated corev-dv arithmetic test
dir: cv32e40p/sim/uvmt
cmd: make gen_corev-dv test COREV=YES TEST=corev_arithmetic_base_test
cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_arithmetic_base_test
num: 200

corev_rand_instr_test:
Expand All @@ -160,11 +160,11 @@ tests:
cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_illegal_instr_test
num: 200

corev_jump_stress_test:
corev_rand_jump_stress_test:
build: uvmt_cv32e40p
description: Generated corev-dv jump stress test
dir: cv32e40p/sim/uvmt
cmd: make gen_corev-dv test COREV=YES TEST=corev_jump_stress_test
cmd: make gen_corev-dv test COREV=YES TEST=corev_rand_jump_stress_test
num: 200

corev_rand_interrupt:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Test definition YAML for corev-dv test generator

name: corev_arithmetic_base_test
name: corev_rand_arithmetic_base_test
uvm_test: $(CV_CORE_LC)_instr_base_test
description: >
RISCV-DV generated arithmetic test
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Test definition YAML for generated corev arithmetic base test

name: corev_arithmetic_base_test
name: corev_rand_arithmetic_base_test
uvm_test: uvmt_$(CV_CORE_LC)_firmware_test_c
description: >
Math test generated by corev-dv


Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Test definition YAML for corev-dv test generator

name: corev_jump_stress_test
name: corev_rand_jump_stress_test
uvm_test: $(CV_CORE_LC)_instr_base_test
description: >
RISCV-DV generated jump stress test
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Test definition YAML for generated corev arithmetic base test

name: corev_jump_stress_test
name: corev_rand_jump_stress_test
uvm_test: uvmt_$(CV_CORE_LC)_firmware_test_c
description: >
Jump stress test generated by corev-dv
38 changes: 20 additions & 18 deletions mk/uvmt/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ The UVM testcases are at `CV_CORE/tests/uvmt`, and the test-programs can be
found in `CV_CORE/tests/program`. See the README in those directories for more information.
<br><br>
Please refer to the [Verification Strategy](https://core-v-docs-verif-strat.readthedocs.io/en/latest/sim_tests.html#simulation-tests-in-the-uvm-environments)
for a discussion on the distinction between a _testcase_ (a SystemVerilog class extended from _uvm\_test_ that instantiates and controls the UVM environment) and a _test-program_ (a C or assembler program that runs on the core RTL model) in this environment.
for a discussion on the distinction between a _testcase_ (a SystemVerilog class extended from uvm_test that instantiates and controls the UVM environment) and a _test-program_ (a C or assembler program that runs on the core RTL model) in this environment.
<br><br>
To run the UVM environment you will need:
- a run-time license for the Imperas OVPsim Instruction Set Simulator
Expand Down Expand Up @@ -164,10 +164,11 @@ Before making changes to the code in your local branch, it is a good idea to run
test to ensure you are starting from a stable code-base. The code (both RTL
and verification) should _always_ pass sanity, so if it does not, please
raise an issue and assign it to @mikeopenhwgroup. The definition of "sanity"
will change over time as the ability of the verification environment to
may change over time as the ability of the verification environment to
stress the RTL improves. Running sanity is trivial:
<br><br>
**make sanity**
```
make sanity
```

CI Mini-regression
------------------
Expand All @@ -176,14 +177,15 @@ located at the top-level of this repository. A pythin script `ci/ci_check` can
"cv32 CI check regression" specified in the control script. Before issuing a pull-request for either
the RTL or verification code, please run `ci_check`. Your pull-request will be rejected if `ci_check`
does not compile and run successfully. Usage is simple:
<br>
**./ci__check -s xrun**
<br>
will run the CI sanity regression using Xcelium.
```
./ci_check --core cv32e40p -s xrun
```
will run the CI sanity regression on the cv32e40p using Xcelium.
<br><br>
Complete user information is obtained in the usual way:
<br>
**./ci__check -h**
```
./ci_check -h
```

Available Test Programs
-----------------------
Expand All @@ -201,9 +203,9 @@ Here are a few examples
* **make test TEST=riscv_arithmetic_basic_test**:<br>run the riscv_arithmetic_basic_test program found at `<CV_CORE>/tests/programs/custom`.
<br>
There are also a few targets that do something other than run a test. The most popular is:
<br>
```
**make clean_all**
<br>
```
which deletes all SIMULATOR generated intermediates, waves and logs **plus** the cloned RTL code.

COREV-DV Generated Tests
Expand All @@ -218,14 +220,14 @@ up-to-date with the latest release of riscv-dv.
Riscv-dv uses test templates such as "riscv_arithmetic_basic_test" and "riscv_rand_jump_test".
Corev-dv has a set of templates for corev-dv generated test-programs at `<CV_CORE>/tests/programs/corev-dv`.
Running these is a two-step process. The first step is to clone riscv-dv and compile corev-dv:
<br><br>
**make corev-dv**
<br><br>
```
make corev-dv
```
Note that the `corev-dv` target need only be run once. The next step is to generate, compile
and run a test. For example:
<br><br>
**make gen_corev-dv test TEST=corev_rand_jump_stress_test**
<br>
```
make gen_corev-dv test TEST=corev_rand_jump_stress_test
```

RISC-V Compliance Test-suite and Regressions
---------------
Expand Down