Verilator doesn't stdout Hello CVA6!

[davidmallasen]

I've been trying to set up the verilator workflow with the comments from the recent pull request #740 in a blank VM with Ubuntu 20.04 (until now I was using QuestaSim and the Genesys II). When executing work-ver/Variane_testharness $RISCV/riscv64-unknown-elf/bin/pk hello.elf after compiling with riscv64-unknown-elf-gcc hello.c -o hello.elf, it seems to run normally but in the end there is no "Hello CVA6!" output to stdout. I have tested the generated .elf on the Genesys II from my previous working flow and it seems to work fine. Any suggestions on what I could be missing? Thanks!

user@user-VirtualBox:~/cva6$ work-ver/Variane_testharness $RISCV/riscv64-unknown-elf/bin/pk hello.elf
This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.
Listening on port 45369
bbl loader
- /home/user/cva6/corev_apu/tb/rvfi_tracer.sv:55: Verilog $finish
- /home/user/cva6/corev_apu/tb/rvfi_tracer.sv:56: Verilog $finish
- /home/user/cva6/corev_apu/tb/rvfi_tracer.sv:56: Second verilog $finish, exiting
user@user-VirtualBox:~/cva6$ 

P.S: I've tried with Verilator 4.100, 4.014 and 4.002 with the same result.

[pcotret]

Same error there.
Nothing suspicious in the trace file, strange 😐

[zarubaf]

Strange that the rvfi_tracer finishes. @JeanRochCoulon do you have a suspicion why?

[pcotret]

Temporary solution:

• Commented the ecall loop: https://github.com/openhwgroup/cva6/blob/master/corev_apu/tb/rvfi_tracer.sv#L54
• Recompile the Verilator model (make clean && make verilator)

./work-ver/Variane_testharness /opt/riscv/riscv64-unknown-elf/bin/pk hello.elf
This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.
Listening on port 37879
bbl loader
Hello CVA6!
/opt/riscv/riscv64-unknown-elf/bin/pk completed after 1529936 cycles
CPU time used: 196366.65 ms
Wall clock time passed: 98656.32 ms

[davidmallasen]

I confirm that commenting lines 54-57 and line 61 of rvfi_tracer.sv temporarily solves the issue. Thanks @pcotret!

user@user-VirtualBox:~/cva6$ work-ver/Variane_testharness $RISCV/riscv64-unknown-elf/bin/pk hello.elf
This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.
Listening on port 39227
bbl loader
Hello CVA6!\n/home/user/riscv/riscv64-unknown-elf/bin/pk completed after 2258993 cycles
CPU time used: 351096.20 ms
Wall clock time passed: 175684.81 ms
user@user-VirtualBox:~/cva6$

[davidmallasen]

The steps to reproduce this issue in a fresh Ubuntu 20.04 installation:

0. Prerequisites

   sudo apt install -y git curl gcc g++ make autoconf
   

1. Clone cva6

   git clone https://github.com/openhwgroup/cva6.git
   cd cva6
   git submodule update --init --recursive
   

2. Set up environment variables:
   • Add the following to the end of the ~/.bashrc:

     export RISCV=/home/user/riscv
     export PATH=$PATH:$RISCV/bin
     

   • source ~/.bashrc
3. Install and setup:

   ./ci/setup.sh
   ./ci/install-riscvpk.sh
   

4. Run user-space application using verilator:

   echo '
   #include <stdio.h>
   
   int main(int argc, char const *argv[]) {
       printf("Hello CVA6!\n");
       return 0;
   }' > hello.c
   riscv64-unknown-elf-gcc hello.c -o hello.elf
   make verilate
   work-ver/Variane_testharness $RISCV/riscv64-unknown-elf/bin/pk hello.elf
   

   The output should be similar to:

   user@user-VirtualBox:~/cva6$ work-ver/Variane_testharness $RISCV/riscv64-unknown-elf/bin/pk hello.elf
   This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.
   Listening on port 45645
   bbl loader
   - /home/user/cva6/corev_apu/tb/rvfi_tracer.sv:61: Verilog $finish
   - /home/user/cva6/corev_apu/tb/rvfi_tracer.sv:61: Verilog $finish
   - /home/user/cva6/corev_apu/tb/rvfi_tracer.sv:61: Second verilog $finish, exiting
   user@user-VirtualBox:~/cva6$
   

[JeanRochCoulon]

Hello David, thank for reporting the point.

When simulation finishes on ecall, also if the cpu has finished to execute the program, the com interface has not finished to transmit the string "Hello CVA6!" to the port. That's why the string is not written out.
In testharness, the simulation is ended by ecall. If you remove ecall, the simulation finishes with the debug module help. And the string has time to be written out.

ecall was inserted to finish the simulation on other testbench than testharness, testbenches supported in core-v-verif projects (core only and uvm) for instance. This solution has also been implemented in riscv-compliance and riscv-tests regression suites.

To write out the string, an timing loop can be inserted before progam end.

Keep us in touch,
Jean-Roch

[davidmallasen]

So from what I understand this is the intended behaviour, as it was not ment to run user applications but to execute the verification tests. For now the solution proposed by Pascal is enough in my case but I'll take this into account if I need it in the future.

Thanks for your time and your insight!

[FranDuqueAyachi]

Temporary solution:

* Commented the `ecall` loop: https://github.com/openhwgroup/cva6/blob/master/corev_apu/tb/rvfi_tracer.sv#L54

* Recompile the Verilator model (`make clean && make verilator`)

./work-ver/Variane_testharness /opt/riscv/riscv64-unknown-elf/bin/pk hello.elf
This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.
Listening on port 37879
bbl loader
Hello CVA6!
/opt/riscv/riscv64-unknown-elf/bin/pk completed after 1529936 cycles
CPU time used: 196366.65 ms
Wall clock time passed: 98656.32 ms

Temporary solution:

* Commented the `ecall` loop: https://github.com/openhwgroup/cva6/blob/master/corev_apu/tb/rvfi_tracer.sv#L54

* Recompile the Verilator model (`make clean && make verilator`)

./work-ver/Variane_testharness /opt/riscv/riscv64-unknown-elf/bin/pk hello.elf
This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.
Listening on port 37879
bbl loader
Hello CVA6!
/opt/riscv/riscv64-unknown-elf/bin/pk completed after 1529936 cycles
CPU time used: 196366.65 ms
Wall clock time passed: 98656.32 ms

how would the file look once you have the comments, because it gives me syntax errors
thank you

[davidmallasen]

@FranDuqueAyachi just comment these lines:

cva6/corev_apu/tb/rvfi_tracer.sv

Lines 59 to 62 in 5c5c704

	if (rvfi_i[i].insn == 32'h00000073) begin
	$finish(1);
	$finish(1);
	end

And this line:

cva6/corev_apu/tb/rvfi_tracer.sv

Line 66 in 5c5c704

if (cycles > SIM_FINISH) $finish(1);

[FranDuqueAyachi]

@FranDuqueAyachi just comment these lines:

cva6/corev_apu/tb/rvfi_tracer.sv

Lines 59 to 61 in 5c5c704

	if (rvfi_i[i].insn == 32'h00000073) begin
	$finish(1);
	$finish(1);

And this line:

https://github.com/openhwgroup/cva6/blob/5c5c704d1d14190ec349fae12fe39fbe025456ae/corev_apu/tb/rvfi_tracer.svI

@FranDuqueAyachi just comment these lines:

cva6/corev_apu/tb/rvfi_tracer.sv

Lines 59 to 61 in 5c5c704

	if (rvfi_i[i].insn == 32'h00000073) begin
	$finish(1);
	$finish(1);

And this line:

cva6/corev_apu/tb/rvfi_tracer.sv

Line 66 in 5c5c704

if (cycles > SIM_FINISH) $finish(1);

OKey, I commented these lines, but when I execute make verilate:
%Error: /home/users/PFC/divsicad/fduque/tfg/cva6/corev_apu/tb/rvfi_tracer.sv:63:11: syntax error, unexpected else
63 | end else if (rvfi_i[i].trap)

[davidmallasen]

@FranDuqueAyachi Also comment line 62, I have updated my previous reply.

[FranDuqueAyachi]

@FranDuqueAyachi Also comment line 62, I have updated my previous reply.

Didnt see it, thank you!

[anjaliigedam]

Seeing the same issue with current CVA6 setup.
There is also no ecall in rvfi_tracer.sv in recent code.

[JeanRochCoulon]

As told previously, the simulation stops before printing out. Today the simulation do not stop anymore on ecall instruction execution but on a write to a special address (TOHOST_ADDR). I reopened this github issue, just to know if a delay could be added to wait for printf completion.

[Juan-Gg]

Any news on this?

[valentinThomazic]

@JeanRochCoulon I tried running the simulation for ~5mins but did not get any output.
However when running the hello world with the pk (now unsupported) as done by the first post in this issue I have no issue getting an output with the most recent version of the repo.
@zchamski maybe you know if the pk was indeed the only way to get an output with this hello world test ?

[zchamski]

@valentinThomazic yes, the Proxy Kernel (pk) is the layer in charge of generic host I/O. If I remember correctly, it uses the HTIF protocol (https://chipyard.readthedocs.io/en/stable/Advanced-Concepts/Chip-Communication.html) to communicate with the host via commands written to variable tohost.

[valentinThomazic]

Okay, that is what I thought ! I had some doubts there was some way to get an output even without the pk since thet test has been kept and getting an output is what you would expect from an hello world.
I think the issue can be closed again @JeanRochCoulon

[Juan-Gg]

So how would one go about running a simulation with an stdout output with the current setup?

[JeanRochCoulon]

Today there is no way to printout. CVA6 is an open source development. If you would like to contribute, we can support the effort.

[michael116688]

Hi, If there is no way to print out, how can I check simulation of hello.c is successful?

[JeanRochCoulon]

Today the printout is not supported by CVA6 project. The use of pk-kernel could be a solution to support it.