tcg/riscv: Utilitizing RISC-V 'V' extension in the backend #14

Swung0x48 · 2023-06-20T10:17:05Z

This patch utilizes RISC-V 'V' extension to support '_vec' suffixed TCG ops in RISC-V backend. This is a very preliminary implementation, as it suffers from the following defects:

Multiple vset{i}vl{i} instruction will be emitted even when there's no vector instruction in between
v0 is reserved specifically for vector masking, and explicitly excluded from normal vector operation register allocations
Only the bare minimal set of TCG ops are implemented. There should be more ops that could be done.

Add constraints for new vector registers and constants in order to support RISC-V 'V' extension

Declare vector register in order to support RISC-V 'V' extension

Detect V extension by reading 'vlenb' csr. Verify presence by catching SIGILL, like the mechanism detecting Zb* extensions. In case that V extension present, add vector registers to tcg_target_available_regs

This implementation is somewhat abusing vset{i}vl{i} instructions to emulate packed-simd-like ops emitted by tcg. Tested by translating SSE/AVX code using x86_64 variant of qemu-linux-user. Not a quite thorough test. Highly experimental. Only includes mandatory '_vec' ops. Ones that can be turned off by 'HAS_xxx' macro is not present in this commit and will be implemented in the future

include/tcg/tcg-opc.h will include tcg-target.opc.h if TCG_TARGET_MAYBE_vec is set to true, so we just place an empty file here.

TCG is using a bizzare pattern to present vector comparison result: storing -1 for true and 0 for false(Persumably from x86, but not quite sure). In order to accomdate for that, we deliberately reserved v0 reg for storing vector mask. Then we use the mask to broadcast -1 to required places. Such implementation can make 'cmpsel_vec' a bit tougher to implement, which need to restore the mask reegister (v0) to RISC-V native vector masking representation. But this implementation seems to be required by glibc (glibc-based program will blow up if not implemented that way) and works fine for now.

tcg/riscv/tcg-target-con-set.h

liweiwei90 · 2023-06-21T07:35:47Z

tcg/riscv/tcg-target.c.inc

+
+    if (have_v) {
+        if (vlen >= 64)
+            tcg_target_available_regs[TCG_TYPE_V64] = ALL_VECTOR_REGS;


这里是不是要看考虑vlen < 64/128/256时采用vector group?

后续可以添加这一部分

tcg/riscv/tcg-target.c.inc

liweiwei90 · 2023-06-21T07:42:04Z

tcg/riscv/tcg-target.h

+#define TCG_TARGET_HAS_sat_vec          0
+#define TCG_TARGET_HAS_minmax_vec       0
+#define TCG_TARGET_HAS_bitsel_vec       0
+#define TCG_TARGET_HAS_cmpsel_vec       0


这些目前都没有使能

liweiwei90 · 2023-06-21T07:43:24Z

tcg/riscv/tcg-target.opc.h

+ * Target-specific opcodes for host vector expansion.  These will be
+ * emitted by tcg_expand_vec_op.  For those familiar with GCC internals,
+ * consider these to be UNSPEC with names.
+ */


这个目前看起来是非必要的修改

这个注释需要写什么吗，我从别的后端抄来的这个（这个文件是必须要有一份空的放在这里的）

liweiwei90 · 2023-06-21T07:43:55Z

tcg/riscv/tcg-target-con-str.h

@@ -19,3 +27,4 @@ CONST('J', TCG_CT_CONST_J12)
 CONST('N', TCG_CT_CONST_N12)
 CONST('M', TCG_CT_CONST_M12)
 CONST('Z', TCG_CT_CONST_ZERO)
+CONST('U', TCG_CT_CONST_U4)


U 是用来表示 imm[4:0] 的，用在 OPIVI 格式指令中（见 Spec 5.）例如 vmv.v.i 指令（不过目前还未使用到）

liweiwei90 · 2023-06-21T07:44:34Z

tcg/riscv/tcg-target.c.inc

+#define VECTOR_REGS_WO_V0 MAKE_64BIT_MASK(33, 31)
+#define ALL_DVECTOR_REG_GROUPS 0x5555555500000000
+#define ALL_QVECTOR_REG_GROUPS 0x1111111100000000
+


这里看起来考虑了vector group,不过后面没用上

tcg/riscv/tcg-target.c.inc

in order to avoid requests being stuck in a BlockBackend's request queue during cleanup. Having such requests can lead to a deadlock [0] with a virtio-scsi-pci device using iothread that's busy with IO when initiating a shutdown with QMP 'quit'. There is a race where such a queued request can continue sometime (maybe after bdrv_child_free()?) during bdrv_root_unref_child() [1]. The completion will hold the AioContext lock and wait for the BQL during SCSI completion, but the main thread will hold the BQL and wait for the AioContext as part of bdrv_root_unref_child(), leading to the deadlock [0]. [0]: > Thread 3 (Thread 0x7f3bbd87b700 (LWP 135952) "qemu-system-x86"): > #0 __lll_lock_wait (futex=futex@entry=0x564183365f00 <qemu_global_mutex>, private=0) at lowlevellock.c:52 > #1 0x00007f3bc1c0d843 in __GI___pthread_mutex_lock (mutex=0x564183365f00 <qemu_global_mutex>) at ../nptl/pthread_mutex_lock.c:80 > #2 0x0000564182939f2e in qemu_mutex_lock_impl (mutex=0x564183365f00 <qemu_global_mutex>, file=0x564182b7f774 "../softmmu/physmem.c", line=2593) at ../util/qemu-thread-posix.c:94 > #3 0x000056418247cc2a in qemu_mutex_lock_iothread_impl (file=0x564182b7f774 "../softmmu/physmem.c", line=2593) at ../softmmu/cpus.c:504 > #4 0x00005641826d5325 in prepare_mmio_access (mr=0x5641856148a0) at ../softmmu/physmem.c:2593 > #5 0x00005641826d6fe7 in address_space_stl_internal (as=0x56418679b310, addr=4276113408, val=16418, attrs=..., result=0x0, endian=DEVICE_LITTLE_ENDIAN) at /home/febner/repos/qemu/memory_ldst.c.inc:318 > #6 0x00005641826d7154 in address_space_stl_le (as=0x56418679b310, addr=4276113408, val=16418, attrs=..., result=0x0) at /home/febner/repos/qemu/memory_ldst.c.inc:357 > #7 0x0000564182374b07 in pci_msi_trigger (dev=0x56418679b0d0, msg=...) at ../hw/pci/pci.c:359 > #8 0x000056418237118b in msi_send_message (dev=0x56418679b0d0, msg=...) at ../hw/pci/msi.c:379 > #9 0x0000564182372c10 in msix_notify (dev=0x56418679b0d0, vector=8) at ../hw/pci/msix.c:542 > #10 0x000056418243719c in virtio_pci_notify (d=0x56418679b0d0, vector=8) at ../hw/virtio/virtio-pci.c:77 > #11 0x00005641826933b0 in virtio_notify_vector (vdev=0x5641867a34a0, vector=8) at ../hw/virtio/virtio.c:1985 > #12 0x00005641826948d6 in virtio_irq (vq=0x5641867ac078) at ../hw/virtio/virtio.c:2461 > #13 0x0000564182694978 in virtio_notify (vdev=0x5641867a34a0, vq=0x5641867ac078) at ../hw/virtio/virtio.c:2473 > #14 0x0000564182665b83 in virtio_scsi_complete_req (req=0x7f3bb000e5d0) at ../hw/scsi/virtio-scsi.c:115 > qemu#15 0x00005641826670ce in virtio_scsi_complete_cmd_req (req=0x7f3bb000e5d0) at ../hw/scsi/virtio-scsi.c:641 > qemu#16 0x000056418266736b in virtio_scsi_command_complete (r=0x7f3bb0010560, resid=0) at ../hw/scsi/virtio-scsi.c:712 > qemu#17 0x000056418239aac6 in scsi_req_complete (req=0x7f3bb0010560, status=2) at ../hw/scsi/scsi-bus.c:1526 > qemu#18 0x000056418239e090 in scsi_handle_rw_error (r=0x7f3bb0010560, ret=-123, acct_failed=false) at ../hw/scsi/scsi-disk.c:242 > qemu#19 0x000056418239e13f in scsi_disk_req_check_error (r=0x7f3bb0010560, ret=-123, acct_failed=false) at ../hw/scsi/scsi-disk.c:265 > qemu#20 0x000056418239e482 in scsi_dma_complete_noio (r=0x7f3bb0010560, ret=-123) at ../hw/scsi/scsi-disk.c:340 > qemu#21 0x000056418239e5d9 in scsi_dma_complete (opaque=0x7f3bb0010560, ret=-123) at ../hw/scsi/scsi-disk.c:371 > qemu#22 0x00005641824809ad in dma_complete (dbs=0x7f3bb000d9d0, ret=-123) at ../softmmu/dma-helpers.c:107 > qemu#23 0x0000564182480a72 in dma_blk_cb (opaque=0x7f3bb000d9d0, ret=-123) at ../softmmu/dma-helpers.c:127 > qemu#24 0x00005641827bf78a in blk_aio_complete (acb=0x7f3bb00021a0) at ../block/block-backend.c:1563 > qemu#25 0x00005641827bfa5e in blk_aio_write_entry (opaque=0x7f3bb00021a0) at ../block/block-backend.c:1630 > qemu#26 0x000056418295638a in coroutine_trampoline (i0=-1342102448, i1=32571) at ../util/coroutine-ucontext.c:177 > qemu#27 0x00007f3bc0caed40 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > qemu#28 0x00007f3bbd8757f0 in ?? () > qemu#29 0x0000000000000000 in ?? () > > Thread 1 (Thread 0x7f3bbe3e9280 (LWP 135944) "qemu-system-x86"): > #0 __lll_lock_wait (futex=futex@entry=0x5641856f2a00, private=0) at lowlevellock.c:52 > #1 0x00007f3bc1c0d8d1 in __GI___pthread_mutex_lock (mutex=0x5641856f2a00) at ../nptl/pthread_mutex_lock.c:115 > #2 0x0000564182939f2e in qemu_mutex_lock_impl (mutex=0x5641856f2a00, file=0x564182c0e319 "../util/async.c", line=728) at ../util/qemu-thread-posix.c:94 > #3 0x000056418293a140 in qemu_rec_mutex_lock_impl (mutex=0x5641856f2a00, file=0x564182c0e319 "../util/async.c", line=728) at ../util/qemu-thread-posix.c:149 > #4 0x00005641829532d5 in aio_context_acquire (ctx=0x5641856f29a0) at ../util/async.c:728 > #5 0x000056418279d5df in bdrv_set_aio_context_commit (opaque=0x5641856e6e50) at ../block.c:7493 > #6 0x000056418294e288 in tran_commit (tran=0x56418630bfe0) at ../util/transactions.c:87 > #7 0x000056418279d880 in bdrv_try_change_aio_context (bs=0x5641856f7130, ctx=0x56418548f810, ignore_child=0x0, errp=0x0) at ../block.c:7626 > #8 0x0000564182793f39 in bdrv_root_unref_child (child=0x5641856f47d0) at ../block.c:3242 > #9 0x00005641827be137 in blk_remove_bs (blk=0x564185709880) at ../block/block-backend.c:914 > #10 0x00005641827bd689 in blk_remove_all_bs () at ../block/block-backend.c:583 > #11 0x0000564182798699 in bdrv_close_all () at ../block.c:5117 > #12 0x000056418248a5b2 in qemu_cleanup () at ../softmmu/runstate.c:821 > #13 0x0000564182738603 in qemu_default_main () at ../softmmu/main.c:38 > #14 0x0000564182738631 in main (argc=30, argv=0x7ffd675a8a48) at ../softmmu/main.c:48 > > (gdb) p *((QemuMutex*)0x5641856f2a00) > $1 = {lock = {__data = {__lock = 2, __count = 2, __owner = 135952, ... > (gdb) p *((QemuMutex*)0x564183365f00) > $2 = {lock = {__data = {__lock = 2, __count = 0, __owner = 135944, ... [1]: > Thread 1 "qemu-system-x86" hit Breakpoint 5, bdrv_drain_all_end () at ../block/io.c:551 > #0 bdrv_drain_all_end () at ../block/io.c:551 > #1 0x00005569810f0376 in bdrv_graph_wrlock (bs=0x0) at ../block/graph-lock.c:156 > #2 0x00005569810bd3e0 in bdrv_replace_child_noperm (child=0x556982e2d7d0, new_bs=0x0) at ../block.c:2897 > #3 0x00005569810bdef2 in bdrv_root_unref_child (child=0x556982e2d7d0) at ../block.c:3227 > #4 0x00005569810e8137 in blk_remove_bs (blk=0x556982e42880) at ../block/block-backend.c:914 > #5 0x00005569810e7689 in blk_remove_all_bs () at ../block/block-backend.c:583 > #6 0x00005569810c2699 in bdrv_close_all () at ../block.c:5117 > #7 0x0000556980db45b2 in qemu_cleanup () at ../softmmu/runstate.c:821 > #8 0x0000556981062603 in qemu_default_main () at ../softmmu/main.c:38 > #9 0x0000556981062631 in main (argc=30, argv=0x7ffd7a82a418) at ../softmmu/main.c:48 > [Switching to Thread 0x7fe76dab2700 (LWP 103649)] > > Thread 3 "qemu-system-x86" hit Breakpoint 4, blk_inc_in_flight (blk=0x556982e42880) at ../block/block-backend.c:1505 > #0 blk_inc_in_flight (blk=0x556982e42880) at ../block/block-backend.c:1505 > #1 0x00005569810e8f36 in blk_wait_while_drained (blk=0x556982e42880) at ../block/block-backend.c:1312 > #2 0x00005569810e9231 in blk_co_do_pwritev_part (blk=0x556982e42880, offset=3422961664, bytes=4096, qiov=0x556983028060, qiov_offset=0, flags=0) at ../block/block-backend.c:1402 > #3 0x00005569810e9a4b in blk_aio_write_entry (opaque=0x556982e2cfa0) at ../block/block-backend.c:1628 > #4 0x000055698128038a in coroutine_trampoline (i0=-2090057872, i1=21865) at ../util/coroutine-ucontext.c:177 > #5 0x00007fe770f50d40 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #6 0x00007ffd7a829570 in ?? () > #7 0x0000000000000000 in ?? () Signed-off-by: Fiona Ebner <[email protected]> Message-ID: <[email protected]> Signed-off-by: Paolo Bonzini <[email protected]>

Swung0x48 added 11 commits June 18, 2023 10:51

tcg/riscv: declaring support for basic '_vec' ops

008aa0e

tcg/riscv: const/reg constraints for 'V' extension

b05852d

Add constraints for new vector registers and constants in order to support RISC-V 'V' extension

tcg/riscv: declare vector registers

601f1e0

Declare vector register in order to support RISC-V 'V' extension

tcg/riscv: detect and init V extension

d886ed3

Detect V extension by reading 'vlenb' csr. Verify presence by catching SIGILL, like the mechanism detecting Zb* extensions. In case that V extension present, add vector registers to tcg_target_available_regs

tcg/riscv: use macro instead of magic number

ce08dd3

tcg/riscv: add '_vec' suffixed op definition in tcg_target_op_def

e70f92d

tcg/riscv: add unused tcg-target.opc.h

c5cebb9

include/tcg/tcg-opc.h will include tcg-target.opc.h if TCG_TARGET_MAYBE_vec is set to true, so we just place an empty file here.

tcg/riscv: remove duplicate sigaction init

4537300

tcg/riscv: fix a typo in tcg-target-con-set.h

4794f9e

liweiwei90 reviewed Jun 21, 2023

View reviewed changes

Swung0x48 added 3 commits June 21, 2023 16:24

tcg/riscv: formatting file

9730a7e

tcg/riscv: add brackets after single line if

422ba5d

tcg/riscv: fix argument indentation

dfcbbf2

liweiwei90 merged commit 6448287 into plctlab:plct-riscv-backend-rvv Jun 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tcg/riscv: Utilitizing RISC-V 'V' extension in the backend #14

tcg/riscv: Utilitizing RISC-V 'V' extension in the backend #14

Swung0x48 commented Jun 20, 2023

liweiwei90 Jun 21, 2023

liweiwei90 Jun 21, 2023

liweiwei90 Jun 21, 2023

liweiwei90 Jun 21, 2023

Swung0x48 Jun 21, 2023

liweiwei90 Jun 21, 2023

Swung0x48 Jun 21, 2023 •

edited

Loading

liweiwei90 Jun 21, 2023

tcg/riscv: Utilitizing RISC-V 'V' extension in the backend #14

tcg/riscv: Utilitizing RISC-V 'V' extension in the backend #14

Conversation

Swung0x48 commented Jun 20, 2023

liweiwei90 Jun 21, 2023

Choose a reason for hiding this comment

liweiwei90 Jun 21, 2023

Choose a reason for hiding this comment

liweiwei90 Jun 21, 2023

Choose a reason for hiding this comment

liweiwei90 Jun 21, 2023

Choose a reason for hiding this comment

Swung0x48 Jun 21, 2023

Choose a reason for hiding this comment

liweiwei90 Jun 21, 2023

Choose a reason for hiding this comment

Swung0x48 Jun 21, 2023 • edited Loading

Choose a reason for hiding this comment

liweiwei90 Jun 21, 2023

Choose a reason for hiding this comment

Swung0x48 Jun 21, 2023 •

edited

Loading