`make verilate` fails, and `make riscv_tests_simv` segfault #260

mablinov · 2023-11-08T07:56:55Z

Hi all, first time user:

I've checked out the repo and followed all the instructions on the front page README.md. Everything works up until I get to the make verilate step, this is what I get:

$ make verilate 2>&1 | tee -a out
rm -rf build/verilator; mkdir -p build/verilator
./bender script verilator -t rtl -t ara_test -t cva6_test -t verilator --define NR_LANES=4 --define VLEN=4096 --define RVV_ARIANE=1 > build/verilator/bender_script_default
/home/maxim/src/ara/install/verilator/bin/verilator -f build/verilator/bender_script_default           \
  -GNrLanes=4                                                         \
  -O3                                                                           \
  -Wno-BLKANDNBLK                                                               \
  -Wno-CASEINCOMPLETE                                                           \
  -Wno-CMPCONST                                                                 \
  -Wno-LATCH                                                                    \
  -Wno-LITENDIAN                                                                \
  -Wno-UNOPTFLAT                                                                \
  -Wno-UNPACKED                                                                 \
  -Wno-UNSIGNED                                                                 \
  -Wno-WIDTH                                                                    \
  -Wno-WIDTHCONCAT                                                              \
  -Wno-ENUMVALUE                                                                \
  -Wno-COMBDLY \
  --hierarchical                                                                \
  tb/verilator/waiver.vlt                                                       \
  --Mdir build/verilator                                                       \
  -Itb/dpi                                                                      \
  --compiler clang                                                              \
  -CFLAGS "-DTOPLEVEL_NAME=ara_tb_verilator"                                        \
  -CFLAGS "-DNR_LANES=4"                                              \
  -CFLAGS -I/home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp       \
  -CFLAGS -I/home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp \
  -CFLAGS -I/home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp \
  ""                                                             \
  -LDFLAGS "-lelf"                                                              \
  ""                                                              \
  --exe                                                                         \
  /home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/*.cc            \
  /home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp/*.cc      \
  /home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp/*.cc      \
  /home/maxim/src/ara/hardware/tb/verilator/ara_tb.cpp                                           \
  --cc                                                                          \
                                       \
  --top-module ara_tb_verilator &&                                                  \
cd build/verilator && OBJCACHE='' make -j4 -f Vara_tb_verilator.mk
make[1]: Entering directory '/home/maxim/src/ara/hardware/build/verilator'
make[2]: Entering directory '/home/maxim/src/ara/hardware'
/home/maxim/src/ara/install/verilator/bin/verilator -f build/verilator/Vlane_e_hierMkArgs.f
make[2]: Leaving directory '/home/maxim/src/ara/hardware'
make[1]: Leaving directory '/home/maxim/src/ara/hardware/build/verilator'
Makefile:80: "Specified QuestaSim version (questa-2021.2) not found in PATH /home/maxim/opt/node-v18.12.1-linux-x64/bin:/home/maxim/.local/bin:/home/maxim/opt/node-v18.12.1-linux-x64/bin:/home/maxim/.local/bin:/home/maxim/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
rm -rf build/verilator; mkdir -p build/verilator
./bender script verilator -t rtl -t ara_test -t cva6_test -t verilator --define NR_LANES=4 --define VLEN=4096 --define RVV_ARIANE=1 > build/verilator/bender_script_default
�[33;1mwarning:�[m Name issue with "cva6", `export_include_dirs` not handled
	Could relate to name mismatch, see `bender update`
/home/maxim/src/ara/install/verilator/bin/verilator -f build/verilator/bender_script_default           \
  -GNrLanes=4                                                         \
  -O3                                                                           \
  -Wno-BLKANDNBLK                                                               \
  -Wno-CASEINCOMPLETE                                                           \
  -Wno-CMPCONST                                                                 \
  -Wno-LATCH                                                                    \
  -Wno-LITENDIAN                                                                \
  -Wno-UNOPTFLAT                                                                \
  -Wno-UNPACKED                                                                 \
  -Wno-UNSIGNED                                                                 \
  -Wno-WIDTH                                                                    \
  -Wno-WIDTHCONCAT                                                              \
  -Wno-ENUMVALUE                                                                \
  -Wno-COMBDLY \
  --hierarchical                                                                \
  tb/verilator/waiver.vlt                                                       \
  --Mdir build/verilator                                                       \
  -Itb/dpi                                                                      \
  --compiler clang                                                              \
  -CFLAGS "-DTOPLEVEL_NAME=ara_tb_verilator"                                        \
  -CFLAGS "-DNR_LANES=4"                                              \
  -CFLAGS -I/home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp       \
  -CFLAGS -I/home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp \
  -CFLAGS -I/home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp \
  ""                                                             \
  -LDFLAGS "-lelf"                                                              \
  ""                                                              \
  --exe                                                                         \
  /home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/*.cc            \
  /home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp/*.cc      \
  /home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp/*.cc      \
  /home/maxim/src/ara/hardware/tb/verilator/ara_tb.cpp                                           \
  --cc                                                                          \
                                       \
  --top-module ara_tb_verilator &&                                                  \
cd build/verilator && OBJCACHE='' make -j4 -f Vara_tb_verilator.mk
make[1]: Entering directory '/home/maxim/src/ara/hardware/build/verilator'
make[2]: Entering directory '/home/maxim/src/ara/hardware'
/home/maxim/src/ara/install/verilator/bin/verilator -f build/verilator/Vlane_e_hierMkArgs.f
sh: 1: exec: /home/maxim/src/ara/install/verilator/share/verilator/verilator_bin: not found
%Error: Command Failed ulimit -s unlimited 2>/dev/null; exec /home/maxim/src/ara/install/verilator/share/verilator/verilator_bin -f build/verilator/Vlane_e_hierMkArgs.f
make[2]: *** [build/verilator/Vara_tb_verilator_hier.mk:361: hier_launch_verilator] Error 127
make[2]: Leaving directory '/home/maxim/src/ara/hardware'
make[1]: *** [Vara_tb_verilator_hier.mk:370: Vlane_e/lane_e.sv] Error 2
make[1]: Leaving directory '/home/maxim/src/ara/hardware/build/verilator'
%Error: make -C build/verilator -f Vara_tb_verilator_hier.mk  -j 1  hier_verilation exited with 2
%Error: Command Failed ulimit -s unlimited 2>/dev/null; exec /home/maxim/src/ara/install/verilator/bin/verilator_bin -f build/verilator/bender_script_default -GNrLanes=4 -O3 -Wno-BLKANDNBLK -Wno-CASEINCOMPLETE -Wno-CMPCONST -Wno-LATCH -Wno-LITENDIAN -Wno-UNOPTFLAT -Wno-UNPACKED -Wno-UNSIGNED -Wno-WIDTH -Wno-WIDTHCONCAT -Wno-ENUMVALUE -Wno-COMBDLY --hierarchical tb/verilator/waiver.vlt --Mdir build/verilator -Itb/dpi --compiler clang -CFLAGS -DTOPLEVEL_NAME=ara_tb_verilator -CFLAGS -DNR_LANES=4 -CFLAGS -I/home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp -CFLAGS -I/home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp -CFLAGS -I/home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp  -LDFLAGS -lelf  --exe /home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/dpi_memutil.cc /home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/sv_scoped.cc /home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp/verilator_memutil.cc /home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp/verilated_toplevel.cc /home/maxim/src/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp/verilator_sim_ctrl.cc /home/maxim/src/ara/hardware/tb/verilator/ara_tb.cpp --cc --top-module ara_tb_verilator
make: *** [Makefile:168: build/verilator/Vara_tb_verilator] Error 2

I can sort-of work around this by then typing the following commands manually:

$ make -C build/verilator -f Vara_tb_verilator_hier.mk  -j 1  hier_verilation
make: Entering directory '/home/maxim/src/ara/hardware/build/verilator'
make[1]: Entering directory '/home/maxim/src/ara/hardware'
/home/maxim/src/ara/install/verilator/bin/verilator -f build/verilator/Vlane_e_hierMkArgs.f
make[1]: Leaving directory '/home/maxim/src/ara/hardware'
make[1]: Entering directory '/home/maxim/src/ara/hardware'
/home/maxim/src/ara/install/verilator/bin/verilator -f build/verilator/Vara_tb_verilator_hierMkArgs.f
make[1]: Leaving directory '/home/maxim/src/ara/hardware'
make: Leaving directory '/home/maxim/src/ara/hardware/build/verilator'

This generates the build/verilator/Vara_tb_verilator.mk file, which then allows me to invoke

$ cd build/verilator && OBJCACHE='' make -j4 -f Vara_tb_verilator.mk                                   
<bunch of compile jobs>

And then I can run the tests with make riscv_tests_simv. However, what I'd like to do is generate trace dumps that I can then open with gtkwave, so I want to build with trace=1. Unfortunately my hack-around doesn't allow me to do this: If I invoke

$ make verilate trace=1
<Makefile errors out as above>
$ cd build/verilator && OBJCACHE='' make -j 1 -f Vara_tb_verilator.mk trace=1
<ok>
$ cd build/verilator && OBJCACHE='' make -j32 -f Vara_tb_verilator.mk trace=1
<ok>

and then try to run the tests, I get a segfault:

$ make riscv_tests_simv -j 1 trace=1
Makefile:80: "Specified QuestaSim version (questa-2021.2) not found in PATH /home/maxim/opt/node-v18.12.1-linux-x64/bin:/home/maxim/.local/bin:/home/maxim/opt/node-v18.12.1-linux-x64/bin:/home/maxim/.local/bin:/home/maxim/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
build/verilator/Vara_tb_verilator -t -l ram,/home/maxim/src/ara/apps/bin/rv64uv-ara-vaadd,elf &> build/rv64uv-ara-vaadd.trace
bash: line 1: 4040421 Segmentation fault      (core dumped) build/verilator/Vara_tb_verilator -t -l ram,/home/maxim/src/ara/apps/bin/rv64uv-ara-vaadd,elf &> build/rv64uv-ara-vaadd.trace
make: *** [Makefile:215: rv64uv-ara-vaadd] Error 139

The text was updated successfully, but these errors were encountered:

quswarabid · 2023-11-08T10:32:11Z

@mablinov can you try copying the content (or maybe just verilator_bin) from /home/quswarabid/ara/install/verilator/bin/ to /home/quswarabid/ara/install/verilator/share/verilator/bin/ and then try make verilate in ara/hardware

mablinov · 2023-11-08T11:48:46Z

@quswarabid That did the trick! I can build the normal verilator model (and run it) without having to run manual commands.

Unfortunately, the trace=1 variant is still segfaulting (probably an unrelated issue):

$ make simv app=hello_world trace=1
Makefile:80: "Specified QuestaSim version (questa-2021.2) not found in PATH /home/maxim/opt/node-v18.12.1-linux-x64/bin:/home/maxim/.local/bin:/home/maxim/opt/node-v18.12.1-linux-x64/bin:/home/maxim/.local/bin:/home/maxim/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
build/verilator/Vara_tb_verilator -t -l ram,/home/maxim/src/ara/apps/bin/hello_world,elf
Program header number 0 in `/home/maxim/src/ara/apps/bin/hello_world' low is 80000000
Program header number 0 in `/home/maxim/src/ara/apps/bin/hello_world' high is 80000fbd
Program header number 1 in `/home/maxim/src/ara/apps/bin/hello_world' high is 8000139f
Program header number 2 in `/home/maxim/src/ara/apps/bin/hello_world' high is 800013c7
Program header number 3 in `/home/maxim/src/ara/apps/bin/hello_world' is not of type PT_LOAD; ignoring.
Set `ram TOP.ara_tb_verilator.dut.i_ara_soc.i_dram 10 0x80000000 0x100000 write with offset: 0x0 write with size: 0x13c8
Simulation of Ara
=================

Tracing can be toggled by sending SIGUSR1 to this process:
$ kill -USR1 3dd0db
make: *** [Makefile:210: simv] Segmentation fault (core dumped)

Version:

$ ../install/verilator/bin/verilator_bin --version
Verilator 5.012 2023-06-13 rev v5.012

quswarabid · 2023-11-08T12:31:02Z

Let me take a look

mablinov · 2023-11-08T12:32:35Z

@quswarabid

I replaced the -O3 with -O0 in this line, and I can generate the traces now. Maybe the segfault is really just an assert or something?

quswarabid · 2023-11-08T13:58:25Z

@mablinov thanks for the tip. Though my simulation has just hunged up on me. xD

quswarabid · 2023-11-13T11:16:05Z

I have tried with different optimizations' flags (which probably shouldn't mess with final output, but it does) and I am only able to make verilate with -O0 but it hangs mid-simulation, and for -O>0 it doesn't even compile. I am told it is because of its memory footprint. I am currently on a VM with 8GB RAM allotted to Ubuntu 22.04.3 LTS. @mablinov you may consider closing the issue since your problem is addressed anyway.

llhe110 · 2023-12-18T07:21:28Z

Dear quswarabid,
I also encountered this problem, how do you deal with it now?

llhe@llhe-virtual-machine:~/rscratch/workspace/ara/ara/hardware$ app=hello_world make simv trace=1
Makefile:84: "Specified QuestaSim version (questa-2021.2) not found in PATH /home/llhe/rscratch/cadtools/QuestaSim2021.2.1Linux/install/questasim/RUVM_2021.2:/home/llhe/rscratch/cadtools/QuestaSim2021.2.1Linux/install/questasim/linux_x86_64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/llhe/rscratch/cadtools/interlijIDEA/idea-IC-232.10227.8/bin"
build/verilator/Vara_tb_verilator -t -l ram,/home/llhe/rscratch/workspace/ara/ara/apps/bin/hello_world,elf
Program header number 0 in `/home/llhe/rscratch/workspace/ara/ara/apps/bin/hello_world' low is 80000000
Program header number 0 in `/home/llhe/rscratch/workspace/ara/ara/apps/bin/hello_world' high is 80000fbd
Program header number 1 in `/home/llhe/rscratch/workspace/ara/ara/apps/bin/hello_world' high is 8000139f
Program header number 2 in `/home/llhe/rscratch/workspace/ara/ara/apps/bin/hello_world' high is 800013c7
Program header number 3 in `/home/llhe/rscratch/workspace/ara/ara/apps/bin/hello_world' is not of type PT_LOAD; ignoring.
Set `ram TOP.ara_tb_verilator.dut.i_ara_soc.i_dram 10 0x80000000 0x100000 write with offset: 0x0 write with size: 0x13c8
Simulation of Ara
=================

Tracing can be toggled by sending SIGUSR1 to this process:
$ kill -USR1 15d8
make: *** [Makefile:232: simv]

quswarabid · 2023-12-18T13:09:58Z

@llhe110 I bumped the verilator version back to v4.214 and it worked out for me. To do that, edit VERIL_VERSION in ara/Makefile. Then run make verilator in ara/ to build verilator, and make clean && make verilate trace=1 in ara/hardware again.

Tanishqgithub · 2024-07-04T05:11:28Z

@quswarabid @llhe110

For fst i tried the solution by changing -O3 to -O0 but the simulation sort of stopped midway and gave this o/p :

Set `ram TOP.ara_tb_verilator.dut.i_ara_soc.i_dram 10 0x80000000 0x100000 write with offset: 0x0 write with size: 0x81ff8

Simulation of Ara

=================



Tracing can be toggled by sending SIGUSR1 to this process:

$ kill -USR1 2515

Tracing enabled.

Writing simulation traces to sim.fst



Simulation running, end by pressing CTRL-c.



=============

  FMATMUL  =

============







------------------------------------------------------------

Calculating a (4 x 4) x (4 x 4) matrix multiplication...

-----------------------------------------------------------



alculating fmatmul...

The execution took 744 cycles.

he performance is 0.172043 FLOP/cycle (2.150538% utilization).



------------------------------------------------------------

Calculating a (8 x 8) x (8 x 8) matrix multiplication...

-----------------------------------------------------------



Calculating fmatmul...

The execution took 931 cycles.

The performance is 1.099893 FLOP/cycle (13.748657% utilization).



------------------------------------------------------------

Calculating a (16 x 16) x (16 x 16) matrix multiplication...

-----------------------------------------------------------



Calculating fmatmul...

The execution took 2593 cycles.

The performance is 3.159275 FLOP/cycle (39.490936% utilization).



------------------------------------------------------------

Calculating a (32 x 32) x (32 x 32) matrix multiplication...

-----------------------------------------------------------



Calculating fmatmul...

The execution took 11446 cycles.

[121474] %Warning: ara_tb_verilator.sv:45: TOP.ara_tb_verilator: Core Test *** FAILED *** (tohost = 2)

- /home/tanishq/Documents/ARA_new/ara/hardware/tb/ara_tb_verilator.sv:52: Verilog $finish

Received $finish() from Verilog, shutting down simulation.



Simulation statistics

=====================

Executed cycles:  ed41

Wallclock time:   147.126 s

Simulation speed: 412.823 cycles/s (0.412823 kHz)

Trace file size:  231eb5a B



You can view the simulation traces by calling

$ gtkwave sim.fst

make: *** [Makefile:232: simv] Error 2

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'

and while viewing sim.fst in terminal :

tanishq@Tanishq:~/Documents/ARA_new/ara$ gtkwave sim.fst

Gtk-Message: 08:07:46.919: Failed to load module "canberra-gtk-module"



GTKWave Analyzer v3.3.104 (w)1999-2020 BSI



GTKWAVE | Could not initialize 'sim.fst', exiting.

and by manually opening the sim.fst in the file location i could see only signals of CVA6

Along with this other query is how can i see the signals for the ARA [vector processor] only .
And in this same issue at last it is suggested to bump down verilator version to v4.214 , i did changes to make file and again ran make verilator &> verilator_install_4.log and the error occured is in :
verilator_install4.txt

mablinov changed the title ~~make verilate fails~~ make verilate fails, and make riscv_tests_simv segfault Nov 8, 2023

mablinov closed this as completed Nov 13, 2023

quswarabid mentioned this issue Dec 6, 2023

Failing to generate wavefrom #197

Open

quswarabid mentioned this issue Dec 18, 2023

Traces with verilate. Verilate the design is OK but Run the tests faild. #278

Closed

Tanishqgithub mentioned this issue Jul 3, 2024

Failed in RTL Simulation #314

Open

Tanishqgithub mentioned this issue Jul 5, 2024

VCD Dumping,where the vcd file is saved? #277

Open

Syedateeb24 mentioned this issue Jul 8, 2024

Similar issue as #320 #321

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`make verilate` fails, and `make riscv_tests_simv` segfault #260

`make verilate` fails, and `make riscv_tests_simv` segfault #260

mablinov commented Nov 8, 2023 •

edited

Loading

quswarabid commented Nov 8, 2023

mablinov commented Nov 8, 2023

quswarabid commented Nov 8, 2023

mablinov commented Nov 8, 2023

quswarabid commented Nov 8, 2023

quswarabid commented Nov 13, 2023

llhe110 commented Dec 18, 2023

quswarabid commented Dec 18, 2023

Tanishqgithub commented Jul 4, 2024

make verilate fails, and make riscv_tests_simv segfault #260

make verilate fails, and make riscv_tests_simv segfault #260

Comments

mablinov commented Nov 8, 2023 • edited Loading

quswarabid commented Nov 8, 2023

mablinov commented Nov 8, 2023

quswarabid commented Nov 8, 2023

mablinov commented Nov 8, 2023

quswarabid commented Nov 8, 2023

quswarabid commented Nov 13, 2023

llhe110 commented Dec 18, 2023

quswarabid commented Dec 18, 2023

Tanishqgithub commented Jul 4, 2024

`make verilate` fails, and `make riscv_tests_simv` segfault #260

`make verilate` fails, and `make riscv_tests_simv` segfault #260

mablinov commented Nov 8, 2023 •

edited

Loading