| Message ID | 20230425193146.2106111-1-richard.henderson@linaro.org |
|---|---|
| Headers | show |
| Series | tcg: Improve atomicity support | expand |
On Tue, 25 Apr 2023 at 20:32, Richard Henderson <richard.henderson@linaro.org> wrote: > > v1: https://lore.kernel.org/qemu-devel/20221118094754.242910-1-richard.henderson@linaro.org/ > v2: https://lore.kernel.org/qemu-devel/20230216025739.1211680-1-richard.henderson@linaro.org/ > > Based-on: 20230424054105.1579315-1-richard.henderson@linaro.org > ("[PATCH v3 00/57] tcg: Simplify calls to load/store helpers") > > The main objective here is to support Arm FEAT_LSE2, which says that any > single memory access that does not cross a 16-byte boundary is atomic. > This is the MO_ATOM_WITHIN16 control. > > While I'm touching all of this, a secondary objective is to handle the > atomicity of the IBM machines. Both Power and s390x treat misaligned > accesses as atomic on the lsb of the pointer. For instance, an 8-byte > access at ptr % 8 == 4 will appear as two atomic 4-byte accesses, and > ptr % 4 == 2 will appear as four 3-byte accesses. I hope you mean "four 2-byte accesses", although I'm never quite sure how odd s390x can get :-) -- PMM
On 5/2/23 17:11, Peter Maydell wrote: > On Tue, 25 Apr 2023 at 20:32, Richard Henderson > <richard.henderson@linaro.org> wrote: >> >> v1: https://lore.kernel.org/qemu-devel/20221118094754.242910-1-richard.henderson@linaro.org/ >> v2: https://lore.kernel.org/qemu-devel/20230216025739.1211680-1-richard.henderson@linaro.org/ >> >> Based-on: 20230424054105.1579315-1-richard.henderson@linaro.org >> ("[PATCH v3 00/57] tcg: Simplify calls to load/store helpers") >> >> The main objective here is to support Arm FEAT_LSE2, which says that any >> single memory access that does not cross a 16-byte boundary is atomic. >> This is the MO_ATOM_WITHIN16 control. >> >> While I'm touching all of this, a secondary objective is to handle the >> atomicity of the IBM machines. Both Power and s390x treat misaligned >> accesses as atomic on the lsb of the pointer. For instance, an 8-byte >> access at ptr % 8 == 4 will appear as two atomic 4-byte accesses, and >> ptr % 4 == 2 will appear as four 3-byte accesses. > > I hope you mean "four 2-byte accesses", although I'm never > quite sure how odd s390x can get :-) Heh. Yes, of course. r~
v1: https://lore.kernel.org/qemu-devel/20221118094754.242910-1-richard.henderson@linaro.org/ v2: https://lore.kernel.org/qemu-devel/20230216025739.1211680-1-richard.henderson@linaro.org/ Based-on: 20230424054105.1579315-1-richard.henderson@linaro.org ("[PATCH v3 00/57] tcg: Simplify calls to load/store helpers") The main objective here is to support Arm FEAT_LSE2, which says that any single memory access that does not cross a 16-byte boundary is atomic. This is the MO_ATOM_WITHIN16 control. While I'm touching all of this, a secondary objective is to handle the atomicity of the IBM machines. Both Power and s390x treat misaligned accesses as atomic on the lsb of the pointer. For instance, an 8-byte access at ptr % 8 == 4 will appear as two atomic 4-byte accesses, and ptr % 4 == 2 will appear as four 3-byte accesses. This is the MO_ATOM_SUBALIGN control. By default, acceses are atomic only if aligned, which is the current behaviour of the tcg code generator (mostly, anyway, there were bugs). This is the MO_ATOM_IFALIGN control. Further, one can say that a large memory access is really a set of contiguous smaller accesses, and we need not provide more atomicity than that (modulo MO_ATOM_WITHIN16). This is the MO_ATMAX_* control. Changes for v3: - More simplifications to load/store helper generation. - All tcg backends now query atom_and_align_for_opc to increase alignment when required for atomicity. - 128-bit atomic load/store for aarch64, ppc64, s390x. - Sparc64 updates. - Unaligned load/store for loongarch64, riscv. r~ Richard Henderson (57): include/exec/memop: Add bits describing atomicity accel/tcg: Add cpu_in_serial_context accel/tcg: Introduce tlb_read_idx accel/tcg: Reorg system mode load helpers accel/tcg: Reorg system mode store helpers accel/tcg: Honor atomicity of loads accel/tcg: Honor atomicity of stores target/loongarch: Do not include tcg-ldst.h tcg: Unify helper_{be,le}_{ld,st}* accel/tcg: Implement helper_{ld,st}*_mmu for user-only tcg/tci: Use helper_{ld,st}*_mmu for user-only tcg: Add 128-bit guest memory primitives meson: Detect atomic128 support with optimization tcg/i386: Add have_atomic16 accel/tcg: Use have_atomic16 in ldst_atomicity.c.inc accel/tcg: Add aarch64 specific support in ldst_atomicity tcg/aarch64: Detect have_lse, have_lse2 for linux tcg/aarch64: Detect have_lse, have_lse2 for darwin accel/tcg: Add have_lse2 support in ldst_atomicity tcg: Introduce TCG_OPF_TYPE_MASK tcg/i386: Use full load/store helpers in user-only mode tcg/aarch64: Use full load/store helpers in user-only mode tcg/ppc: Use full load/store helpers in user-only mode tcg/loongarch64: Use full load/store helpers in user-only mode tcg/riscv: Use full load/store helpers in user-only mode tcg/arm: Adjust constraints on qemu_ld/st tcg/arm: Use full load/store helpers in user-only mode tcg/mips: Use full load/store helpers in user-only mode tcg/s390x: Use full load/store helpers in user-only mode tcg/sparc64: Allocate %g2 as a third temporary tcg/sparc64: Rename tcg_out_movi_imm13 to tcg_out_movi_s13 tcg/sparc64: Rename tcg_out_movi_imm32 to tcg_out_movi_u32 tcg/sparc64: Split out tcg_out_movi_s32 tcg/sparc64: Use standard slow path for softmmu accel/tcg: Remove helper_unaligned_{ld,st} tcg/loongarch64: Assert the host supports unaligned accesses tcg/loongarch64: Support softmmu unaligned accesses tcg/riscv: Support softmmu unaligned accesses tcg: Introduce tcg_target_has_memory_bswap tcg: Add INDEX_op_qemu_{ld,st}_i128 tcg: Support TCG_TYPE_I128 in tcg_out_{ld,st}_helper_{args,ret} tcg: Introduce atom_and_align_for_opc tcg/i386: Use atom_and_align_for_opc tcg/aarch64: Use atom_and_align_for_opc tcg/arm: Use atom_and_align_for_opc tcg/loongarch64: Use atom_and_align_for_opc tcg/mips: Use atom_and_align_for_opc tcg/ppc: Use atom_and_align_for_opc tcg/riscv: Use atom_and_align_for_opc tcg/s390x: Use atom_and_align_for_opc tcg/sparc64: Use atom_and_align_for_opc tcg/i386: Honor 64-bit atomicity in 32-bit mode tcg/i386: Support 128-bit load/store with have_atomic16 tcg/aarch64: Rename temporaries tcg/aarch64: Support 128-bit load/store tcg/ppc: Support 128-bit load/store tcg/s390x: Support 128-bit load/store accel/tcg/internal.h | 5 + accel/tcg/tcg-runtime.h | 3 + include/exec/cpu-defs.h | 7 +- include/exec/cpu_ldst.h | 26 +- include/exec/memop.h | 36 + include/qemu/cpuid.h | 18 + include/tcg/tcg-ldst.h | 72 +- include/tcg/tcg-opc.h | 8 + include/tcg/tcg.h | 22 +- tcg/aarch64/tcg-target-con-set.h | 2 + tcg/aarch64/tcg-target.h | 6 +- tcg/arm/tcg-target-con-set.h | 16 +- tcg/arm/tcg-target-con-str.h | 5 +- tcg/arm/tcg-target.h | 3 +- tcg/i386/tcg-target.h | 7 +- tcg/loongarch64/tcg-target.h | 3 +- tcg/mips/tcg-target.h | 4 +- tcg/ppc/tcg-target-con-set.h | 2 + tcg/ppc/tcg-target-con-str.h | 1 + tcg/ppc/tcg-target.h | 4 +- tcg/riscv/tcg-target.h | 4 +- tcg/s390x/tcg-target-con-set.h | 2 + tcg/s390x/tcg-target.h | 4 +- tcg/sparc64/tcg-target-con-set.h | 2 - tcg/sparc64/tcg-target-con-str.h | 1 - tcg/sparc64/tcg-target.h | 4 +- tcg/tcg-internal.h | 2 + tcg/tci/tcg-target.h | 4 +- accel/tcg/cpu-exec-common.c | 3 + accel/tcg/cputlb.c | 1767 ++++++++++++++++++------------ accel/tcg/tb-maint.c | 2 +- accel/tcg/user-exec.c | 488 ++++++--- target/loongarch/csr_helper.c | 1 - target/loongarch/iocsr_helper.c | 1 - tcg/optimize.c | 15 +- tcg/tcg-op.c | 265 +++-- tcg/tcg.c | 270 ++++- tcg/tci.c | 150 +-- accel/tcg/ldst_atomicity.c.inc | 1373 +++++++++++++++++++++++ docs/devel/loads-stores.rst | 36 +- docs/devel/tcg-ops.rst | 11 +- meson.build | 52 +- tcg/aarch64/tcg-target.c.inc | 384 +++++-- tcg/arm/tcg-target.c.inc | 121 +- tcg/i386/tcg-target.c.inc | 362 ++++-- tcg/loongarch64/tcg-target.c.inc | 91 +- tcg/mips/tcg-target.c.inc | 104 +- tcg/ppc/tcg-target.c.inc | 261 +++-- tcg/riscv/tcg-target.c.inc | 132 +-- tcg/s390x/tcg-target.c.inc | 177 +-- tcg/sparc64/tcg-target.c.inc | 714 ++++-------- tcg/tci/tcg-target.c.inc | 8 +- 52 files changed, 4657 insertions(+), 2404 deletions(-) create mode 100644 accel/tcg/ldst_atomicity.c.inc