mbox series

[v5,00/54] tcg: Improve atomicity support

Message ID 20230515143313.734053-1-richard.henderson@linaro.org
Headers show
Series tcg: Improve atomicity support | expand

Message

Richard Henderson May 15, 2023, 2:32 p.m. UTC 
v4: https://lore.kernel.org/qemu-devel/20230503070656.1746170-1-richard.henderson@linaro.org/

The main objective here is to support Arm FEAT_LSE2, which says that any
single memory access that does not cross a 16-byte boundary is atomic.
This is the MO_ATOM_WITHIN16_* control.

While I'm touching all of this, a secondary objective is to handle the
atomicity of the IBM machines.  Power treats misaligned accesses as
atomic on the lsb of the pointer.  This is the MO_ATOM_SUBALIGN control.

By default, acceses are atomic only if aligned, which is the current
behaviour of the tcg code generator (mostly, anyway, there were bugs).
This is the MO_ATOM_IFALIGN control.

Changes for v5:
  - Drop MO_ATMAX_*.  The two cases we really care about become
    MO_ATOM_{IFALIGN,WITHIN16}_PAIR.  Some r-b dropped as a consequence.
  - Introduce struct TCGAtomAlign, and more consistently install
    it into struct HostAddress, to reduce the variance between
    the host backends.
  - All prerequisites merged!

Patches lacking review:
  01-include-exec-memop-Add-MO_ATOM_.patch
  02-accel-tcg-Honor-atomicity-of-loads.patch
  03-accel-tcg-Honor-atomicity-of-stores.patch
  07-tcg-Add-128-bit-guest-memory-primitives.patch
  09-tcg-i386-Add-have_atomic16.patch
  10-accel-tcg-Use-have_atomic16-in-ldst_atomicity.c.i.patch
  11-accel-tcg-Add-aarch64-specific-support-in-ldst_at.patch
  14-accel-tcg-Add-have_lse2-support-in-ldst_atomicity.patch
  20-tcg-arm-Adjust-constraints-on-qemu_ld-st.patch
  24-tcg-sparc64-Allocate-g2-as-a-third-temporary.patch
  25-tcg-sparc64-Rename-tcg_out_movi_imm13-to-tcg_out_.patch
  26-target-sparc64-Remove-tcg_out_movi_s13-case-from-.patch
  27-tcg-sparc64-Rename-tcg_out_movi_imm32-to-tcg_out_.patch
  29-tcg-sparc64-Use-standard-slow-path-for-softmmu.patch
  31-tcg-loongarch64-Check-the-host-supports-unaligned.patch
  36-tcg-Introduce-tcg_out_movext3.patch
  37-tcg-Merge-tcg_out_helper_load_regs-into-caller.patch
  39-tcg-Introduce-atom_and_align_for_opc.patch
  40-tcg-i386-Use-atom_and_align_for_opc.patch
  45-tcg-ppc-Use-atom_and_align_for_opc.patch
  46-tcg-riscv-Use-atom_and_align_for_opc.patch
  48-tcg-sparc64-Use-atom_and_align_for_opc.patch
  54-tcg-s390x-Support-128-bit-load-store.patch


r~


Richard Henderson (54):
  include/exec/memop: Add MO_ATOM_*
  accel/tcg: Honor atomicity of loads
  accel/tcg: Honor atomicity of stores
  tcg: Unify helper_{be,le}_{ld,st}*
  accel/tcg: Implement helper_{ld,st}*_mmu for user-only
  tcg/tci: Use helper_{ld,st}*_mmu for user-only
  tcg: Add 128-bit guest memory primitives
  meson: Detect atomic128 support with optimization
  tcg/i386: Add have_atomic16
  accel/tcg: Use have_atomic16 in ldst_atomicity.c.inc
  accel/tcg: Add aarch64 specific support in ldst_atomicity
  tcg/aarch64: Detect have_lse, have_lse2 for linux
  tcg/aarch64: Detect have_lse, have_lse2 for darwin
  accel/tcg: Add have_lse2 support in ldst_atomicity
  tcg/i386: Use full load/store helpers in user-only mode
  tcg/aarch64: Use full load/store helpers in user-only mode
  tcg/ppc: Use full load/store helpers in user-only mode
  tcg/loongarch64: Use full load/store helpers in user-only mode
  tcg/riscv: Use full load/store helpers in user-only mode
  tcg/arm: Adjust constraints on qemu_ld/st
  tcg/arm: Use full load/store helpers in user-only mode
  tcg/mips: Use full load/store helpers in user-only mode
  tcg/s390x: Use full load/store helpers in user-only mode
  tcg/sparc64: Allocate %g2 as a third temporary
  tcg/sparc64: Rename tcg_out_movi_imm13 to tcg_out_movi_s13
  target/sparc64: Remove tcg_out_movi_s13 case from tcg_out_movi_imm32
  tcg/sparc64: Rename tcg_out_movi_imm32 to tcg_out_movi_u32
  tcg/sparc64: Split out tcg_out_movi_s32
  tcg/sparc64: Use standard slow path for softmmu
  accel/tcg: Remove helper_unaligned_{ld,st}
  tcg/loongarch64: Check the host supports unaligned accesses
  tcg/loongarch64: Support softmmu unaligned accesses
  tcg/riscv: Support softmmu unaligned accesses
  tcg: Introduce tcg_target_has_memory_bswap
  tcg: Add INDEX_op_qemu_{ld,st}_i128
  tcg: Introduce tcg_out_movext3
  tcg: Merge tcg_out_helper_load_regs into caller
  tcg: Support TCG_TYPE_I128 in tcg_out_{ld,st}_helper_{args,ret}
  tcg: Introduce atom_and_align_for_opc
  tcg/i386: Use atom_and_align_for_opc
  tcg/aarch64: Use atom_and_align_for_opc
  tcg/arm: Use atom_and_align_for_opc
  tcg/loongarch64: Use atom_and_align_for_opc
  tcg/mips: Use atom_and_align_for_opc
  tcg/ppc: Use atom_and_align_for_opc
  tcg/riscv: Use atom_and_align_for_opc
  tcg/s390x: Use atom_and_align_for_opc
  tcg/sparc64: Use atom_and_align_for_opc
  tcg/i386: Honor 64-bit atomicity in 32-bit mode
  tcg/i386: Support 128-bit load/store with have_atomic16
  tcg/aarch64: Rename temporaries
  tcg/aarch64: Support 128-bit load/store
  tcg/ppc: Support 128-bit load/store
  tcg/s390x: Support 128-bit load/store

 docs/devel/loads-stores.rst      |   36 +-
 docs/devel/tcg-ops.rst           |   11 +-
 meson.build                      |   52 +-
 accel/tcg/tcg-runtime.h          |    3 +
 include/exec/memop.h             |   37 +
 include/qemu/cpuid.h             |   18 +
 include/tcg/tcg-ldst.h           |   72 +-
 include/tcg/tcg-opc.h            |    8 +
 tcg/aarch64/tcg-target-con-set.h |    2 +
 tcg/aarch64/tcg-target.h         |   15 +-
 tcg/arm/tcg-target-con-set.h     |   16 +-
 tcg/arm/tcg-target-con-str.h     |    5 +-
 tcg/arm/tcg-target.h             |    3 +-
 tcg/i386/tcg-target.h            |    7 +-
 tcg/loongarch64/tcg-target.h     |    3 +-
 tcg/mips/tcg-target.h            |    4 +-
 tcg/ppc/tcg-target-con-set.h     |    2 +
 tcg/ppc/tcg-target-con-str.h     |    1 +
 tcg/ppc/tcg-target.h             |    4 +-
 tcg/riscv/tcg-target.h           |    4 +-
 tcg/s390x/tcg-target-con-set.h   |    2 +
 tcg/s390x/tcg-target.h           |    4 +-
 tcg/sparc64/tcg-target-con-set.h |    2 -
 tcg/sparc64/tcg-target-con-str.h |    1 -
 tcg/sparc64/tcg-target.h         |    4 +-
 tcg/tcg-internal.h               |    2 +
 tcg/tci/tcg-target.h             |    4 +-
 accel/tcg/cputlb.c               |  840 +++++++++++-------
 accel/tcg/user-exec.c            |  489 +++++++----
 tcg/tcg-op.c                     |  254 ++++--
 tcg/tcg.c                        |  504 +++++++++--
 tcg/tci.c                        |  150 +---
 accel/tcg/ldst_atomicity.c.inc   | 1389 ++++++++++++++++++++++++++++++
 tcg/aarch64/tcg-target.c.inc     |  374 +++++---
 tcg/arm/tcg-target.c.inc         |  147 +---
 tcg/i386/tcg-target.c.inc        |  358 ++++++--
 tcg/loongarch64/tcg-target.c.inc |   92 +-
 tcg/mips/tcg-target.c.inc        |  108 +--
 tcg/ppc/tcg-target.c.inc         |  207 +++--
 tcg/riscv/tcg-target.c.inc       |  135 +--
 tcg/s390x/tcg-target.c.inc       |  175 ++--
 tcg/sparc64/tcg-target.c.inc     |  715 +++++----------
 tcg/tci/tcg-target.c.inc         |    5 +
 43 files changed, 4201 insertions(+), 2063 deletions(-)
 create mode 100644 accel/tcg/ldst_atomicity.c.inc