From patchwork Thu Aug 16 02:54:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 144306 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp1540827ljj; Wed, 15 Aug 2018 19:59:46 -0700 (PDT) X-Google-Smtp-Source: AA+uWPwCl1XJsiqx2V56b5aNRu+er9MC6vSciWmo2FPrWTbYS3w/gSUB+fjMGjBm9x+5sazpyoWX X-Received: by 2002:a37:7e05:: with SMTP id z5-v6mr789219qkc.239.1534388386196; Wed, 15 Aug 2018 19:59:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534388386; cv=none; d=google.com; s=arc-20160816; b=HPKpjNG1pbnGCzXvFcGl4qD3z6W2SfqiWSa0JEjcAnA7eI5vsXZpNJdtH4T3NnWRPr vCJLvvT9uID+HrRoK57Jot2Ag440VwdawdjLQ4spxMAmRtegi8c5E23cbdxoE/TdX2Yn nt7fHu99ERB/qChQ6FkHyNHrpqCF5SVNVcAX8vzs/oxj93G/t4FIf5+Brv8hOFjyWsR5 jPMpQ317Sz1mgE/VlteU1Yr5tglZ18dYRh26xmYGfA2yMi+VzZqKU+OK2O4RvFj3rza8 F6UqepWKkIipWH2gPEndDhDGPpODPiyd1W6Gc/gO/8wvNxjKaUyGC7Y6GFouh3AwF5go zR9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=utTT1tFGHMA9lVa/XS5scEPRMxbQt548UTW7BU+eN4w=; b=SxIt3LJQPFB5tXg4qYjIOzi0yV3muEND9M82XDVhmzDr9s9aUA8DbnJ7zT18klF795 NHmfra2otHKZjTul/R52Xe9vGnvtJtNw01eg5uOCoZPCx+owDnaIuGj4zOeHZ241mc6W c4SHnH6rFS+92HqXq6RPDz2IwTA62ihgA1a4GcVwWt2GxbmzeVtKdz6+ToNI4WUgrJvx 0CKffhNigKkiBjkbyvIkT2/FqNtPBokLdfLvp+JUP5/VY9AU4sDpbhqYX24BkYPqJckM EBSo1+Jlo1bS3cJRjqngXKdaV08PRniesznmOewLi4R1Enl7NsKTKbCFhhXFB4DoIPbS JeFQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=hvhc9e95; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id j3-v6si9141244qtc.98.2018.08.15.19.59.45 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 15 Aug 2018 19:59:46 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=hvhc9e95; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53245 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fq8Vl-0005sR-Mj for patch@linaro.org; Wed, 15 Aug 2018 22:59:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50451) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fq8RB-0002IW-5e for qemu-devel@nongnu.org; Wed, 15 Aug 2018 22:55:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fq8R7-0006i2-RW for qemu-devel@nongnu.org; Wed, 15 Aug 2018 22:55:01 -0400 Received: from mail-pf1-x443.google.com ([2607:f8b0:4864:20::443]:40105) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fq8R7-0006hX-HQ for qemu-devel@nongnu.org; Wed, 15 Aug 2018 22:54:57 -0400 Received: by mail-pf1-x443.google.com with SMTP id e13-v6so1330792pff.7 for ; Wed, 15 Aug 2018 19:54:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=utTT1tFGHMA9lVa/XS5scEPRMxbQt548UTW7BU+eN4w=; b=hvhc9e9538htRKrHfM1sVVkZotBhDBOlpcOurxUfOGmRHJUTb6IumejUpQrQUX7Bg9 6vXj1BJGcugSMDPpjHpdyEFjHv2DXSjeBrUoaaY6rhVDafL/sWxZgAmPp56yeDC/fBk4 I+4KJvlmN6S2/3je4S1/NHPcbzonB/N/BcoZc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=utTT1tFGHMA9lVa/XS5scEPRMxbQt548UTW7BU+eN4w=; b=fJG6txc1yCbvuGEIwLMkWeR8SVnZQCHtp8lN6CEE3mVyorM7+YGASYIJPne+c/TW4o tS7WImJbpGsUXZ2UbmBz/kD/bd1Oqfy2mibBKwDfRctkii6jQO37Lfeooj0cgMSa4Ll8 FMLIq1YpnNfB8q1xM+t+rDhm1C9lg3nHrclP0QmpwVwOo6P5dRUjHv9hdjCtXIUuTSp+ gdf5dankAs+7U4+LdniPpeoPp/t3anvqbdkTVeubM+6yG8PDJM6wn511W75SJkNRML7y A8/8Rh5YxtGJiB5yvHvhZugYlUwngGiFxHGE6E2smiXMKWnMgJTMqwLYczUBGUwfk7VG zWBg== X-Gm-Message-State: AOUpUlFP1wXjHeBO0vWnlThFcpxq5zamQRqTripV79GsaiUHmvRzwNCa 5IMpThUwSksQUBqC4vYMpwlBUf0abYY= X-Received: by 2002:a62:1a8f:: with SMTP id a137-v6mr30442676pfa.190.1534388096003; Wed, 15 Aug 2018 19:54:56 -0700 (PDT) Received: from cloudburst.twiddle.net (97-113-8-179.tukw.qwest.net. [97.113.8.179]) by smtp.gmail.com with ESMTPSA id k64-v6sm45231497pfc.160.2018.08.15.19.54.54 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 15 Aug 2018 19:54:54 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 15 Aug 2018 19:54:48 -0700 Message-Id: <20180816025452.21358-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180816025452.21358-1-richard.henderson@linaro.org> References: <20180816025452.21358-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::443 Subject: [Qemu-devel] [PATCH 1/5] tcg: Split CONFIG_ATOMIC128 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, cota@braap.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" GCC7+ will no longer advertise support for 16-byte __atomic operations if only cmpxchg is supported, as for x86_64. Fortunately, x86_64 still has support for __sync_compare_and_swap_16 and we can make use of that. AArch64 does not have, nor ever has had such support, so open-code it. Signed-off-by: Richard Henderson --- accel/tcg/atomic_template.h | 22 +++-- include/qemu/atomic128.h | 162 ++++++++++++++++++++++++++++++++++++ tcg/tcg.h | 29 ++++--- accel/tcg/cputlb.c | 3 +- accel/tcg/user-exec.c | 5 +- configure | 19 +++++ 6 files changed, 219 insertions(+), 21 deletions(-) create mode 100644 include/qemu/atomic128.h -- 2.17.1 diff --git a/accel/tcg/atomic_template.h b/accel/tcg/atomic_template.h index d751bcba48..a056b7b408 100644 --- a/accel/tcg/atomic_template.h +++ b/accel/tcg/atomic_template.h @@ -100,19 +100,24 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, DATA_TYPE ret; ATOMIC_TRACE_RMW; - ret = atomic_cmpxchg__nocheck(haddr, cmpv, newv); +#if DATA_SIZE == 16 + ret = atomic16_cmpxchg(haddr, cmpv, newv); +#else + ret = atomic_cmpxchg(haddr, cmpv, newv); +#endif ATOMIC_MMU_CLEANUP; return ret; } #if DATA_SIZE >= 16 +#if HAVE_ATOMIC128 ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr EXTRA_ARGS) { ATOMIC_MMU_DECLS; DATA_TYPE val, *haddr = ATOMIC_MMU_LOOKUP; ATOMIC_TRACE_LD; - __atomic_load(haddr, &val, __ATOMIC_RELAXED); + val = atomic16_read(haddr); ATOMIC_MMU_CLEANUP; return val; } @@ -124,9 +129,10 @@ void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP; ATOMIC_TRACE_ST; - __atomic_store(haddr, &val, __ATOMIC_RELAXED); + atomic16_set(haddr, val); ATOMIC_MMU_CLEANUP; } +#endif #else ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val EXTRA_ARGS) @@ -228,19 +234,24 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, DATA_TYPE ret; ATOMIC_TRACE_RMW; +#if DATA_SIZE == 16 + ret = atomic16_cmpxchg(haddr, BSWAP(cmpv), BSWAP(newv)); +#else ret = atomic_cmpxchg__nocheck(haddr, BSWAP(cmpv), BSWAP(newv)); +#endif ATOMIC_MMU_CLEANUP; return BSWAP(ret); } #if DATA_SIZE >= 16 +#if HAVE_ATOMIC128 ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr EXTRA_ARGS) { ATOMIC_MMU_DECLS; DATA_TYPE val, *haddr = ATOMIC_MMU_LOOKUP; ATOMIC_TRACE_LD; - __atomic_load(haddr, &val, __ATOMIC_RELAXED); + val = atomic16_read(haddr); ATOMIC_MMU_CLEANUP; return BSWAP(val); } @@ -253,9 +264,10 @@ void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, ATOMIC_TRACE_ST; val = BSWAP(val); - __atomic_store(haddr, &val, __ATOMIC_RELAXED); + atomic16_set(haddr, val); ATOMIC_MMU_CLEANUP; } +#endif #else ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val EXTRA_ARGS) diff --git a/include/qemu/atomic128.h b/include/qemu/atomic128.h new file mode 100644 index 0000000000..2613ebd352 --- /dev/null +++ b/include/qemu/atomic128.h @@ -0,0 +1,162 @@ +/* + * Simple interface for 128-bit atomic operations. + * + * Copyright (C) 2018 Linaro, Ltd. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + * See docs/devel/atomics.txt for discussion about the guarantees each + * atomic primitive is meant to provide. + */ + +#ifndef QEMU_ATOMIC128_H +#define QEMU_ATOMIC128_H + +/* + * GCC is a house divided about supporting large atomic operations. + * + * For hosts that only have large compare-and-swap, a legalistic reading + * of the C++ standard means that one cannot implement __atomic_read on + * read-only memory, and thus all atomic operations must synchronize + * through libatomic. + * + * See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80878 + * + * This interpretation is not especially helpful for QEMU. + * For softmmu, all RAM is always read/write from the hypervisor. + * For user-only, if the guest doesn't implement such an __atomic_read + * then the host need not worry about it either. + * + * Moreover, using libatomic is not an option, because its interface is + * built for std::atomic, and requires that *all* accesses to such an + * object go through the library. In our case we do not have an object + * in the C/C++ sense, but a view of memory as seen by the guest. + * The guest may issue a large atomic operation and then access those + * pieces using word-sized accesses. From the hypervisor, we have no + * way to connect those two actions. + * + * Therefore, special case each platform. + */ + +#if defined(CONFIG_ATOMIC128) +static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) +{ + return atomic_cmpxchg__nocheck(ptr, cmp, new); +} +# define HAVE_CMPXCHG128 1 +#elif defined(CONFIG_CMPXCHG128) +static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) +{ + return __sync_val_compare_and_swap_16(ptr, cmp, new); +} +# define HAVE_CMPXCHG128 1 +#elif defined(__aarch64__) +/* Through gcc 8, aarch64 has no support for 128-bit at all. */ +static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) +{ + uint64_t cmpl = cmp, cmph = cmp >> 64; + uint64_t newl = new, newh = new >> 64; + uint64_t oldl, oldh; + uint32_t tmp; + + asm("0: ldaxp %[oldl], %[oldh], %[mem]\n\t" + "cmp %[oldl], %[cmpl]\n\t" + "ccmp %[oldh], %[cmph], #0, eq\n\t" + "b.ne 1f\n\t" + "stlxp %w[tmp], %[newl], %[newh], %[mem]\n\t" + "cbz %w[tmp], 0b\n" + "1:" + : [mem] "+m"(*ptr), [tmp] "=&r"(tmp), + [oldl] "=&r"(oldl), [oldh] "=r"(oldh) + : [cmpl] "r"(cmpl), [cmph] "r"(cmph), + [newl] "r"(newl), [newh] "r"(newh) + : "memory", "cc"); + + return int128_make128(oldl, oldh); +} +# define HAVE_CMPXCHG128 1 +#endif /* Some definition for HAVE_CMPXCHG128 */ + + +#if defined(CONFIG_ATOMIC128) +static inline Int128 atomic16_read(Int128 *ptr) +{ + return atomic_read__nocheck(ptr); +} + +static inline void atomic16_set(Int128 *ptr, Int128 val) +{ + atomic_set__nocheck(ptr, val); +} + +# define HAVE_ATOMIC128 1 +#elif !defined(CONFIG_USER_ONLY) +# ifdef __aarch64__ +/* We can do better than cmpxchg for AArch64. */ +static inline Int128 atomic16_read(Int128 *ptr) +{ + uint64_t l, h; + uint32_t tmp; + + /* The load must be paired with the store to guarantee not tearing. */ + asm("0: ldxp %[l], %[h], %[mem]\n\t" + "stxp %w[tmp], %[l], %[h], %[mem]\n\t" + "cbz %w[tmp], 0b" + : [mem] "+m"(*ptr), [tmp] "=r"(tmp), [l] "=r"(l), [h] "=r"(h)); + + return int128_make128(l, h); +} + +static inline void atomic16_set(Int128 *ptr, Int128 val) +{ + uint64_t l = val, h = val >> 64, t1, t2; + + /* Load into temporaries to acquire the exclusive access lock. */ + asm("0: ldxp %[t1], %[t2], %[mem]\n\t" + "stxp %w[t1], %[l], %[h], %[mem]\n\t" + "cbz %w[t1], 0b" + : [mem] "+m"(*ptr), [t1] "=&r"(t1), [t2] "=&r"(t2) + : [l] "r"(l), [h] "r"(h)); +} + +# define HAVE_ATOMIC128 1 +# elif HAVE_CMPXCHG128 +static inline Int128 atomic16_read(Int128 *ptr) +{ + /* Maybe replace 0 with 0, returning the old value. */ + return atomic16_cmpxchg(ptr, 0, 0); +} + +static inline void atomic16_set(Int128 *ptr, Int128 val) +{ + Int128 old = *ptr, cmp; + do { + cmp = old; + old = atomic16_cmpxchg(ptr, cmp, val); + } while (old != cmp); +} + +# define HAVE_ATOMIC128 1 +# endif +#endif + +/* + * Fallback definitions that must be optimized away, or error. + */ + +#ifndef HAVE_CMPXCHG128 +Int128 __attribute__((error("unsupported cmpxchg"))) + atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new); +# define HAVE_CMPXCHG128 0 +#endif + +#ifndef HAVE_ATOMIC128 +Int128 __attribute__((error("unsupported atomic16_read"))) + atomic16_read(Int128 *ptr, Int128 cmp, Int128 new); +Int128 __attribute__((error("unsupported atomic16_set"))) + atomic16_set(Int128 *ptr, Int128 cmp, Int128 new); +# define HAVE_ATOMIC128 0 +#endif + +#endif /* QEMU_ATOMIC128_H */ diff --git a/tcg/tcg.h b/tcg/tcg.h index f9f12378e9..297b3f06ee 100644 --- a/tcg/tcg.h +++ b/tcg/tcg.h @@ -32,6 +32,8 @@ #include "qemu/queue.h" #include "tcg-mo.h" #include "tcg-target.h" +#include "qemu/atomic.h" +#include "qemu/int128.h" /* XXX: make safe guess about sizes */ #define MAX_OP_PER_INSTR 266 @@ -1454,27 +1456,28 @@ GEN_ATOMIC_HELPER_ALL(xchg) #undef GEN_ATOMIC_HELPER #endif /* CONFIG_SOFTMMU */ -#ifdef CONFIG_ATOMIC128 -#include "qemu/int128.h" - -/* These aren't really a "proper" helpers because TCG cannot manage Int128. - However, use the same format as the others, for use by the backends. */ +/* + * These aren't really a "proper" helpers because TCG cannot manage Int128. + * However, use the same format as the others, for use by the backends. + * + * The cmpxchg functions are only defined if HAVE_CMPXCHG128; + * the ld/st functions are only defined if HAVE_ATOMIC128, + * as defined by . + */ Int128 helper_atomic_cmpxchgo_le_mmu(CPUArchState *env, target_ulong addr, Int128 cmpv, Int128 newv, - TCGMemOpIdx oi, uintptr_t retaddr); + TCGMemOpIdx oi, uintptr_t ra); Int128 helper_atomic_cmpxchgo_be_mmu(CPUArchState *env, target_ulong addr, Int128 cmpv, Int128 newv, - TCGMemOpIdx oi, uintptr_t retaddr); + TCGMemOpIdx oi, uintptr_t ra); Int128 helper_atomic_ldo_le_mmu(CPUArchState *env, target_ulong addr, - TCGMemOpIdx oi, uintptr_t retaddr); + TCGMemOpIdx oi, uintptr_t ra); Int128 helper_atomic_ldo_be_mmu(CPUArchState *env, target_ulong addr, - TCGMemOpIdx oi, uintptr_t retaddr); + TCGMemOpIdx oi, uintptr_t ra); void helper_atomic_sto_le_mmu(CPUArchState *env, target_ulong addr, Int128 val, - TCGMemOpIdx oi, uintptr_t retaddr); + TCGMemOpIdx oi, uintptr_t ra); void helper_atomic_sto_be_mmu(CPUArchState *env, target_ulong addr, Int128 val, - TCGMemOpIdx oi, uintptr_t retaddr); - -#endif /* CONFIG_ATOMIC128 */ + TCGMemOpIdx oi, uintptr_t ra); #endif /* TCG_H */ diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 563fa30117..a7ea35a9e9 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -32,6 +32,7 @@ #include "exec/log.h" #include "exec/helper-proto.h" #include "qemu/atomic.h" +#include "qemu/atomic128.h" /* DEBUG defines, enable DEBUG_TLB_LOG to log to the CPU_LOG_MMU target */ /* #define DEBUG_TLB */ @@ -1188,7 +1189,7 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, #include "atomic_template.h" #endif -#ifdef CONFIG_ATOMIC128 +#if HAVE_CMPXCHG128 || HAVE_ATOMIC128 #define DATA_SIZE 16 #include "atomic_template.h" #endif diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index 26a3ffbba1..cd75829cf2 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -25,6 +25,7 @@ #include "exec/cpu_ldst.h" #include "translate-all.h" #include "exec/helper-proto.h" +#include "qemu/atomic128.h" #undef EAX #undef ECX @@ -615,7 +616,7 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, /* The following is only callable from other helpers, and matches up with the softmmu version. */ -#ifdef CONFIG_ATOMIC128 +#if HAVE_ATOMIC128 || HAVE_CMPXCHG128 #undef EXTRA_ARGS #undef ATOMIC_NAME @@ -628,4 +629,4 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, #define DATA_SIZE 16 #include "atomic_template.h" -#endif /* CONFIG_ATOMIC128 */ +#endif diff --git a/configure b/configure index 2a7796ea80..7e24d89843 100755 --- a/configure +++ b/configure @@ -5117,6 +5117,21 @@ EOF fi fi +cmpxchg128=no +if test "$int128" = yes -a "$atomic128" = no; then + cat > $TMPC << EOF +int main(void) +{ + unsigned __int128 x = 0, y = 0; + __sync_val_compare_and_swap_16(&x, y, x); + return 0; +} +EOF + if compile_prog "" "" ; then + cmpxchg128=yes + fi +fi + ######################################### # See if 64-bit atomic operations are supported. # Note that without __atomic builtins, we can only @@ -6610,6 +6625,10 @@ if test "$atomic128" = "yes" ; then echo "CONFIG_ATOMIC128=y" >> $config_host_mak fi +if test "$cmpxchg128" = "yes" ; then + echo "CONFIG_CMPXCHG128=y" >> $config_host_mak +fi + if test "$atomic64" = "yes" ; then echo "CONFIG_ATOMIC64=y" >> $config_host_mak fi From patchwork Thu Aug 16 02:54:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 144305 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp1539690ljj; Wed, 15 Aug 2018 19:57:42 -0700 (PDT) X-Google-Smtp-Source: AA+uWPxqvNHWr1ffV16u4FyepVnCDKWBVxs/Z+QtVgmIShBZEQK974GmAWQWd94rv5lD0USnrs9e X-Received: by 2002:aed:3c8c:: with SMTP id d12-v6mr28299743qtf.203.1534388262742; Wed, 15 Aug 2018 19:57:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534388262; cv=none; d=google.com; s=arc-20160816; b=yw6rknIbSD3U1zEywQYgIVJOaS6pEcBtXnL16yYa5Bo34gpjH4TGHAsKJ6oEF5NlsO Dxe/wsoTqLZ+bvPxlDiCmPWclD615pjLVdRKe7LHIN14JiDVQa3bCcSvYVfaoClptmZl d1peWn3X+SRpjTXu0l2Ri+NkybnNRjY/q6N4W87jRMEZtH9GwUXHWIBYH2+5pMK53OjS Dc1DtCu4Wq60qE72Obf57maoLE83uBx1PEHmp1xdHKnN/RhM41vcaPwvslHFXsT/tbnO 4d8S/JWzAGDQPENI5F6j83wrnnQSiaE6kwbwU/NZumNBfPYWnQzI/bG6vhUKDd4rAOvh bP5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=IhMON8SIkP+qjZyAFnHW2Wam9h9Vd5CT55tJoV/HJxY=; b=N5sHIugLtVElXaCaHbWhCrAxNZlFlo3YyOBgYlFXVwJF9hI+N4amayFbLaHD9q/jKs PnigHM8aThmnrXSy0+pNlOftSGMx+BkOBfNS15NyJAKiJCCchTCdFhAI6y0OVsRMJTTj G0ocBXY9bl4GUkELlbyfC94sFM4fejjmLQfi2AueDmY07xTv4Fe7CIItbNshAntJh1xt L/VvkLmEzRVW7dU0MUT9mx4rkFGzQsUKh3zxHWc6a+5dY576PPA4mBT0QyXLOuClCkdv b2Ls4+oJMv0mT76jC36oWHaoq/PRVbcfkHZf4maVnx4UJqeb4ik49FSq29xaC2zQXU6r AXLg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Xqp9x944; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id e54-v6si6694737qvd.129.2018.08.15.19.57.42 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 15 Aug 2018 19:57:42 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Xqp9x944; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53231 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fq8Tm-0003US-40 for patch@linaro.org; Wed, 15 Aug 2018 22:57:42 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50447) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fq8RB-0002IU-3z for qemu-devel@nongnu.org; Wed, 15 Aug 2018 22:55:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fq8R8-0006ix-Mk for qemu-devel@nongnu.org; Wed, 15 Aug 2018 22:55:01 -0400 Received: from mail-pg1-x533.google.com ([2607:f8b0:4864:20::533]:35049) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fq8R8-0006iK-GC for qemu-devel@nongnu.org; Wed, 15 Aug 2018 22:54:58 -0400 Received: by mail-pg1-x533.google.com with SMTP id w10-v6so1361295pgv.2 for ; Wed, 15 Aug 2018 19:54:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=IhMON8SIkP+qjZyAFnHW2Wam9h9Vd5CT55tJoV/HJxY=; b=Xqp9x9446bC+sg/TqiY72VYhJEEhIz1YHsVa28iFz6jR9FlPXrtE3S+OcHCEqyJt5Z m2v7kEFmjrhonx6i6KwLdjZzRqt8uSI/oxFNTEubBZYGx39AfTQ8rpzb6IASKBhyJ2MI wY45xQ6iL0n8cTwTMqKdtqdcuUnoYZ3bACme4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=IhMON8SIkP+qjZyAFnHW2Wam9h9Vd5CT55tJoV/HJxY=; b=OLLoLU83lvFwiDq1kdciZC8OgY+IZkqvpw91jXLQMaeiWzJ6lyxAMMBhmQkojjuBx8 2/vm8mQ3CRiFk6JiD5ED6GZPvlt9bqh10ximkP0bRzTjPfyOcoS2qLP0aFuMLs8j/+Ju q42eds/8l/Z21zcQUfBboZAjqt8Aic2YzDcJhpqdioDtEMRgZCE2qKMv+I75cWiHOJEf 8CbBI3A33H/yhwyqzecfCNZbcFqA5D63kFdHZKlkGpW1ougyZNhivoOo7qq64bnCMiZn Fs1rv5PVhL2oJQ/ohdNfRm6TtUEawc4925ZrzwvDjXC/QKASc6Akp1yGGXxTZR3V9PzA yXiA== X-Gm-Message-State: AOUpUlHd5/Ht24sEgCSa90PKTmpC5inAutkcd3pLAaL/+Ffk++5ODtWE NIQTJggACXvtk5bHo/iv4ym5OY9f9PI= X-Received: by 2002:a62:41d6:: with SMTP id g83-v6mr30104451pfd.219.1534388097231; Wed, 15 Aug 2018 19:54:57 -0700 (PDT) Received: from cloudburst.twiddle.net (97-113-8-179.tukw.qwest.net. [97.113.8.179]) by smtp.gmail.com with ESMTPSA id k64-v6sm45231497pfc.160.2018.08.15.19.54.56 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 15 Aug 2018 19:54:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 15 Aug 2018 19:54:49 -0700 Message-Id: <20180816025452.21358-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180816025452.21358-1-richard.henderson@linaro.org> References: <20180816025452.21358-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::533 Subject: [Qemu-devel] [PATCH 2/5] target/i386: Convert to HAVE_CMPXCHG128 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, cota@braap.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/i386/mem_helper.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) -- 2.17.1 Reviewed-by: Emilio G. Cota diff --git a/target/i386/mem_helper.c b/target/i386/mem_helper.c index 30c26b9d9c..6cc53bcb40 100644 --- a/target/i386/mem_helper.c +++ b/target/i386/mem_helper.c @@ -23,6 +23,7 @@ #include "exec/exec-all.h" #include "exec/cpu_ldst.h" #include "qemu/int128.h" +#include "qemu/atomic128.h" #include "tcg.h" void helper_cmpxchg8b_unlocked(CPUX86State *env, target_ulong a0) @@ -137,10 +138,7 @@ void helper_cmpxchg16b(CPUX86State *env, target_ulong a0) if ((a0 & 0xf) != 0) { raise_exception_ra(env, EXCP0D_GPF, ra); - } else { -#ifndef CONFIG_ATOMIC128 - cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); -#else + } else if (HAVE_CMPXCHG128) { int eflags = cpu_cc_compute_all(env, CC_OP); Int128 cmpv = int128_make128(env->regs[R_EAX], env->regs[R_EDX]); @@ -159,7 +157,8 @@ void helper_cmpxchg16b(CPUX86State *env, target_ulong a0) eflags &= ~CC_Z; } CC_SRC = eflags; -#endif + } else { + cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); } } #endif From patchwork Thu Aug 16 02:54:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 144303 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp1538352ljj; Wed, 15 Aug 2018 19:55:28 -0700 (PDT) X-Google-Smtp-Source: AA+uWPwqMx3ReXUHI+YamoeaPSu9+NZYv3WIeZSNS8oSnkKshR5U8lIWlzPNBKCjprAZYkk6PSki X-Received: by 2002:a37:5d02:: with SMTP id r2-v6mr26490095qkb.22.1534388128225; Wed, 15 Aug 2018 19:55:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534388128; cv=none; d=google.com; s=arc-20160816; b=MWvslEhkPTqUN4gWFpbfrdFCb1lh0o0k/ksFQkTEQiuJ62AVGD7WFO+sNjIlGIlAws 6n58Fr6yN/n2XQka+rOtQBtGEk+kUHp59x5BeaUX/GLXvii8A5+s/YD5+iAXSi7yD1OP BS8PIAB6j1cSJ4d71ztkB9Y+NG1lKXQOJ6LdBObMNuWvlRxsaZ9Tw1brMx8H71mmqK38 /5i09elsOKj75o5SrB7ajeY+gxWDpdtrr+iy0a+BAbvcjqZEfrAgP/Vh4hXgFgDlvUan wGXPW30UNUIjc8YN2VbnMtQ1GeFKcqcAm+GQmVk8CvgIRjnS/wQ13v6du8eTn6qqQeKl GEUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=6DVWL+KmnJpsTAnO1xJsQQaM7zZ9z/Qp4CCGaX7EdgU=; b=MtXjKZdY+4HNsji4Vwh0vHZqoybr4fh+WYncPAZB+qag6/48SEo7nqDBDMioYVe/hC hx9ed2ti0PV9Ze3nPVbQMb9zIL8esYP8MI5yu79rmvwrz/DSfX1ErtHWTVzuUdss6tXN LQb2WsfpIDcaXViz6PTqKGk6oH5s7LZdSi5PBPF4w62bk7gmVGqOqXvM9s0jj8CCuWQI c6OYwQmumCHiAvRT7M8yBSHiIcpqDNaa7LPg55V06nxhveCQlVsXJqpusZqw0B7vK0PH ecG8PAPQIb62AP1eux8VUXD7Yi3nKExac40JmSu89Q6bXWVLsMw2Qg3HLH/azw7FC3o4 lrLQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="dk5M/X/J"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id e28-v6si10707873qtf.175.2018.08.15.19.55.28 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 15 Aug 2018 19:55:28 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="dk5M/X/J"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53223 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fq8Rb-0002KJ-Iy for patch@linaro.org; Wed, 15 Aug 2018 22:55:27 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50455) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fq8RB-0002IX-Dv for qemu-devel@nongnu.org; Wed, 15 Aug 2018 22:55:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fq8RA-0006kW-BL for qemu-devel@nongnu.org; Wed, 15 Aug 2018 22:55:01 -0400 Received: from mail-pg1-x52a.google.com ([2607:f8b0:4864:20::52a]:36055) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fq8RA-0006jm-31 for qemu-devel@nongnu.org; Wed, 15 Aug 2018 22:55:00 -0400 Received: by mail-pg1-x52a.google.com with SMTP id h12-v6so1361009pgs.3 for ; Wed, 15 Aug 2018 19:54:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=6DVWL+KmnJpsTAnO1xJsQQaM7zZ9z/Qp4CCGaX7EdgU=; b=dk5M/X/Jh2tlckvNXiyow+cjLSvVAiwHTRFv9DSZRRTVpKXK+zGB4rDDB0BDpFv78a fGkMxvfMctgk0xG+lD2LI6apOna1HvDWGw0HkY+l3HchJ1jBTz8CZgtZ7b8/mb0zUWek cjXnE6FwnCmX73MubLib6OLxN/qKYIh6+oUqc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=6DVWL+KmnJpsTAnO1xJsQQaM7zZ9z/Qp4CCGaX7EdgU=; b=UncevT3c3c9Tacm2m0PV73YPaHC5jkcAUiGRdX/b40QoP7oTGH1jqg3AxDv2JxiC7+ 5o0lHH1j9UnNCBQdk1NzPuwdz3Cyn8hB7LGrvFXojTgqaRsR//AWPnRvK6GGbcB1MG9J q8EL0Xk3+7dC7cZe+3fCwbv72KKjBlqnHobefPfuVqKX4+NYNWzi9cbGZYE+zBUvGLBw t85i5v9gR/qrUYmb058Ytkm2QbXqx8XhzUJHLrJ5FqVx6NUBDpKKBezSyKVcAmgPq6Mm r+RbHIfRBiQVS+kd8JJOMCFazMBlOrBDvIqWwK5F33/BX2VlDHZ8eY8CiaihszwkYFRM sJnw== X-Gm-Message-State: AOUpUlGx6KNTdnsgbnF6ofuzVh/fuVAVD0c0ORsvR8dzo/QkToy5oAgh NmxQRvLX9VGZM1j06037ljM5R/bm/tQ= X-Received: by 2002:a62:f909:: with SMTP id o9-v6mr30617101pfh.141.1534388098772; Wed, 15 Aug 2018 19:54:58 -0700 (PDT) Received: from cloudburst.twiddle.net (97-113-8-179.tukw.qwest.net. [97.113.8.179]) by smtp.gmail.com with ESMTPSA id k64-v6sm45231497pfc.160.2018.08.15.19.54.57 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 15 Aug 2018 19:54:57 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 15 Aug 2018 19:54:50 -0700 Message-Id: <20180816025452.21358-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180816025452.21358-1-richard.henderson@linaro.org> References: <20180816025452.21358-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::52a Subject: [Qemu-devel] [PATCH 3/5] target/s390x: Convert to HAVE_CMPXCHG128 and HAVE_ATOMIC128 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, cota@braap.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/s390x/mem_helper.c | 87 +++++++++++++++++---------------------- 1 file changed, 37 insertions(+), 50 deletions(-) -- 2.17.1 diff --git a/target/s390x/mem_helper.c b/target/s390x/mem_helper.c index e21a47fb4d..908b1254c8 100644 --- a/target/s390x/mem_helper.c +++ b/target/s390x/mem_helper.c @@ -25,6 +25,7 @@ #include "exec/exec-all.h" #include "exec/cpu_ldst.h" #include "qemu/int128.h" +#include "qemu/atomic128.h" #if !defined(CONFIG_USER_ONLY) #include "hw/s390x/storage-keys.h" @@ -1375,7 +1376,7 @@ static void do_cdsg(CPUS390XState *env, uint64_t addr, bool fail; if (parallel) { -#ifndef CONFIG_ATOMIC128 +#if !HAVE_CMPXCHG128 cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); #else int mem_idx = cpu_mmu_index(env, false); @@ -1421,12 +1422,10 @@ void HELPER(cdsg_parallel)(CPUS390XState *env, uint64_t addr, static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, uint64_t a2, bool parallel) { -#if !defined(CONFIG_USER_ONLY) || defined(CONFIG_ATOMIC128) uint32_t mem_idx = cpu_mmu_index(env, false); -#endif uintptr_t ra = GETPC(); - uint32_t fc = extract32(env->regs[0], 0, 8); - uint32_t sc = extract32(env->regs[0], 8, 8); + uint32_t fc = extract32(env->regs[0], 0, 8), fsize = 4 << fc; + uint32_t sc = extract32(env->regs[0], 8, 8), ssize = 1 << sc; uint64_t pl = get_address(env, 1) & -16; uint64_t svh, svl; uint32_t cc; @@ -1442,7 +1441,7 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, } /* Sanity check the alignments. */ - if (extract32(a1, 0, 4 << fc) || extract32(a2, 0, 1 << sc)) { + if (extract32(a1, 0, fsize) || extract32(a2, 0, ssize)) { goto spec_exception; } @@ -1456,13 +1455,12 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, context in order to implement this. That said, restart early if we can't support either operation that is supposed to be atomic. */ if (parallel) { - int mask = 0; -#if !defined(CONFIG_ATOMIC64) - mask = -8; -#elif !defined(CONFIG_ATOMIC128) - mask = -16; + uint32_t max = 4; +#ifdef CONFIG_ATOMIC64 + max = 8; #endif - if (((4 << fc) | (1 << sc)) & mask) { + if ((HAVE_CMPXCHG128 ? 0 : fsize > max) || + (HAVE_ATOMIC128 ? 0 : ssize > max)) { cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); } } @@ -1532,16 +1530,7 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, Int128 cv = int128_make128(env->regs[r3 + 1], env->regs[r3]); Int128 ov; - if (parallel) { -#ifdef CONFIG_ATOMIC128 - TCGMemOpIdx oi = make_memop_idx(MO_TEQ | MO_ALIGN_16, mem_idx); - ov = helper_atomic_cmpxchgo_be_mmu(env, a1, cv, nv, oi, ra); - cc = !int128_eq(ov, cv); -#else - /* Note that we asserted !parallel above. */ - g_assert_not_reached(); -#endif - } else { + if (!parallel) { uint64_t oh = cpu_ldq_data_ra(env, a1 + 0, ra); uint64_t ol = cpu_ldq_data_ra(env, a1 + 8, ra); @@ -1553,6 +1542,13 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, cpu_stq_data_ra(env, a1 + 0, int128_gethi(nv), ra); cpu_stq_data_ra(env, a1 + 8, int128_getlo(nv), ra); + } else if (HAVE_CMPXCHG128) { + TCGMemOpIdx oi = make_memop_idx(MO_TEQ | MO_ALIGN_16, mem_idx); + ov = helper_atomic_cmpxchgo_be_mmu(env, a1, cv, nv, oi, ra); + cc = !int128_eq(ov, cv); + } else { + /* Note that we asserted !parallel above. */ + g_assert_not_reached(); } env->regs[r3 + 0] = int128_gethi(ov); @@ -1582,18 +1578,16 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, cpu_stq_data_ra(env, a2, svh, ra); break; case 4: - if (parallel) { -#ifdef CONFIG_ATOMIC128 + if (!parallel) { + cpu_stq_data_ra(env, a2 + 0, svh, ra); + cpu_stq_data_ra(env, a2 + 8, svl, ra); + } else if (HAVE_ATOMIC128) { TCGMemOpIdx oi = make_memop_idx(MO_TEQ | MO_ALIGN_16, mem_idx); Int128 sv = int128_make128(svl, svh); helper_atomic_sto_be_mmu(env, a2, sv, oi, ra); -#else + } else { /* Note that we asserted !parallel above. */ g_assert_not_reached(); -#endif - } else { - cpu_stq_data_ra(env, a2 + 0, svh, ra); - cpu_stq_data_ra(env, a2 + 8, svl, ra); } break; default: @@ -2091,21 +2085,18 @@ static uint64_t do_lpq(CPUS390XState *env, uint64_t addr, bool parallel) uintptr_t ra = GETPC(); uint64_t hi, lo; - if (parallel) { -#ifndef CONFIG_ATOMIC128 - cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); -#else + if (!parallel) { + check_alignment(env, addr, 16, ra); + hi = cpu_ldq_data_ra(env, addr + 0, ra); + lo = cpu_ldq_data_ra(env, addr + 8, ra); + } else if (HAVE_ATOMIC128) { int mem_idx = cpu_mmu_index(env, false); TCGMemOpIdx oi = make_memop_idx(MO_TEQ | MO_ALIGN_16, mem_idx); Int128 v = helper_atomic_ldo_be_mmu(env, addr, oi, ra); hi = int128_gethi(v); lo = int128_getlo(v); -#endif } else { - check_alignment(env, addr, 16, ra); - - hi = cpu_ldq_data_ra(env, addr + 0, ra); - lo = cpu_ldq_data_ra(env, addr + 8, ra); + cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); } env->retxl = lo; @@ -2128,21 +2119,17 @@ static void do_stpq(CPUS390XState *env, uint64_t addr, { uintptr_t ra = GETPC(); - if (parallel) { -#ifndef CONFIG_ATOMIC128 - cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); -#else - int mem_idx = cpu_mmu_index(env, false); - TCGMemOpIdx oi = make_memop_idx(MO_TEQ | MO_ALIGN_16, mem_idx); - - Int128 v = int128_make128(low, high); - helper_atomic_sto_be_mmu(env, addr, v, oi, ra); -#endif - } else { + if (!parallel) { check_alignment(env, addr, 16, ra); - cpu_stq_data_ra(env, addr + 0, high, ra); cpu_stq_data_ra(env, addr + 8, low, ra); + } else if (HAVE_ATOMIC128) { + int mem_idx = cpu_mmu_index(env, false); + TCGMemOpIdx oi = make_memop_idx(MO_TEQ | MO_ALIGN_16, mem_idx); + Int128 v = int128_make128(low, high); + helper_atomic_sto_be_mmu(env, addr, v, oi, ra); + } else { + cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); } } From patchwork Thu Aug 16 02:54:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 144304 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp1538372ljj; Wed, 15 Aug 2018 19:55:30 -0700 (PDT) X-Google-Smtp-Source: AA+uWPyDzBbECMUTEodT0T1nZoxhTnNesZt98krUkbFwxSGZzXkkDhbO4MEbgicYU6Dp8diRTpp8 X-Received: by 2002:a37:9642:: with SMTP id y63-v6mr26670816qkd.27.1534388130029; Wed, 15 Aug 2018 19:55:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534388130; cv=none; d=google.com; s=arc-20160816; b=KEEA3iV8wKke7yZckouJ0Z+Q1QeSRj1jHu/rUzceO0K0TWbPSUfdjA8UhNiaWOuv6E 6OyVuKoxaaV7Qu8G3UgAwGCHKtkCZdNkrsZkgbmnvIhN+Vbw+vpKgYpGhuEXPvmnUlME 58i+bkUNln0L2FySS5M04Nxc1/gaXh7tG/Y6cPYGmeOZYS7iGK9Ef5L5APdAfc9YaisC XtN+wfaIXpsXHfqNGGrK474FmhmnVlMU3TSrerHxXSD4wzH+kZx5toyBbzXdUtiWwuHZ 8XCX896tfIZfNOHzmEYGo8YVCrgAggPO6URisck+AQFfApu1S2t0GpC63tNZHYi3C0Rz j4uQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=4XT9TWmQ7Mm3nTEPbhj1uqZlWg1AdfWzugZWwXnH6SI=; b=imLJnqJ3J7FUa/Sj6Qv8ciHXgSKsHtWSjr1OgdUp+N3WEuio356vFBoI9VpcffjORg zJMeKFvRteYrdJrXJlOiQluBshAlUy+2o3yHjZ8KcAsJjTUVECF42QfEPpxDa8ihaw6Z /shd6UB0gHFN8I42Mnve1rQy1JjNP7EtFja/bwB40MhJeFmv9DjA5zXG6733VH6mDz8u xzYFHMpFPnk5fmVmOTwyV8HZ2Nd41JjKU4eUijhmdW2CRuQnJ+STJTR2plG3iXFRFiI1 sxZpQgNri/TPICFXYIiC6XV8crN2ivuXh1oNaMNpeLZJp6OB2UUahQU7E9q2umEBSEsF vcVA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=A5CrMWYT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id u81-v6si709858qki.141.2018.08.15.19.55.29 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 15 Aug 2018 19:55:30 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=A5CrMWYT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53224 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fq8Rd-0002Kr-Dk for patch@linaro.org; Wed, 15 Aug 2018 22:55:29 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50484) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fq8RD-0002J3-5z for qemu-devel@nongnu.org; Wed, 15 Aug 2018 22:55:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fq8RB-0006m3-Mn for qemu-devel@nongnu.org; Wed, 15 Aug 2018 22:55:03 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]:38194) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fq8RB-0006ks-D9 for qemu-devel@nongnu.org; Wed, 15 Aug 2018 22:55:01 -0400 Received: by mail-pf1-x42c.google.com with SMTP id x17-v6so1332571pfh.5 for ; Wed, 15 Aug 2018 19:55:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=4XT9TWmQ7Mm3nTEPbhj1uqZlWg1AdfWzugZWwXnH6SI=; b=A5CrMWYTURowKomeCNY/wHqATBD2yyZNSjsLPXRqvazU+/PepftMw5b88ZtXataJ6z 6k53/G29AS/5+ljgdj/eiWUAUFM0D+mbLfdgvlq0Q4p3OXTlZRSph4aLr3BnFYu/McrI IkSu83aotgsYCwKw9y9MConC2xL9tl0BDCZlk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=4XT9TWmQ7Mm3nTEPbhj1uqZlWg1AdfWzugZWwXnH6SI=; b=KgRK54LqcPEI2ch6ZYBMoK97DAvb+gMYHgWomAR6kH3doE5yhPFcEzDcG525QpWMtp vWGXImnzBIACjl+Um+ZTFtmSRESarUrCA3UlMnqL2kDHG3pYfb3hssTfDrkEEz7MB+jW cUNKn9AWK88a/u9TWcXjvV68ZdGABF9mBeF2JqlOY9yJEX9ngRBv9fSlIKA2uLdhar3x zSd5WLW2gPWiYA0qa/GbHVJlk/VD2ZJir0ckD6cZnydVzJXGOf7kWRG+TYvrqPG5eeo3 lxRZ2fjLzSEhvmO/IBM0mpoLpGx/ZXdvMlaSPhKw6jWjA9rg/NGEwR6pM+pAanoX7zKh S4Fg== X-Gm-Message-State: AOUpUlGTPCrf6tiqLZRSFTwXep++i7OVCUpSitRK2H2PFYPh2z3qDzhe u2uZK9DzZiUm/5+5+X6nVtrcXLS2D48= X-Received: by 2002:a63:5815:: with SMTP id m21-v6mr26679146pgb.78.1534388100111; Wed, 15 Aug 2018 19:55:00 -0700 (PDT) Received: from cloudburst.twiddle.net (97-113-8-179.tukw.qwest.net. [97.113.8.179]) by smtp.gmail.com with ESMTPSA id k64-v6sm45231497pfc.160.2018.08.15.19.54.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 15 Aug 2018 19:54:59 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 15 Aug 2018 19:54:51 -0700 Message-Id: <20180816025452.21358-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180816025452.21358-1-richard.henderson@linaro.org> References: <20180816025452.21358-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::42c Subject: [Qemu-devel] [PATCH 4/5] target/arm: Convert to HAVE_CMPXCHG128 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, cota@braap.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-a64.c | 259 +++++++++++++++++++++------------------- 1 file changed, 133 insertions(+), 126 deletions(-) -- 2.17.1 Reviewed-by: Emilio G. Cota diff --git a/target/arm/helper-a64.c b/target/arm/helper-a64.c index 7f6ad3000b..6e4e1b8a19 100644 --- a/target/arm/helper-a64.c +++ b/target/arm/helper-a64.c @@ -30,6 +30,7 @@ #include "exec/exec-all.h" #include "exec/cpu_ldst.h" #include "qemu/int128.h" +#include "qemu/atomic128.h" #include "tcg.h" #include "fpu/softfloat.h" #include /* For crc32 */ @@ -509,189 +510,195 @@ uint64_t HELPER(crc32c_64)(uint64_t acc, uint64_t val, uint32_t bytes) return crc32c(acc, buf, bytes) ^ 0xffffffff; } -/* Returns 0 on success; 1 otherwise. */ -static uint64_t do_paired_cmpxchg64_le(CPUARMState *env, uint64_t addr, - uint64_t new_lo, uint64_t new_hi, - bool parallel, uintptr_t ra) +uint64_t HELPER(paired_cmpxchg64_le)(CPUARMState *env, uint64_t addr, + uint64_t new_lo, uint64_t new_hi) { - Int128 oldv, cmpv, newv; + Int128 cmpv = int128_make128(env->exclusive_val, env->exclusive_high); + Int128 newv = int128_make128(new_lo, new_hi); + Int128 oldv; + uintptr_t ra = GETPC(); + uint64_t o0, o1; bool success; - cmpv = int128_make128(env->exclusive_val, env->exclusive_high); - newv = int128_make128(new_lo, new_hi); - - if (parallel) { -#ifndef CONFIG_ATOMIC128 - cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); -#else - int mem_idx = cpu_mmu_index(env, false); - TCGMemOpIdx oi = make_memop_idx(MO_LEQ | MO_ALIGN_16, mem_idx); - oldv = helper_atomic_cmpxchgo_le_mmu(env, addr, cmpv, newv, oi, ra); - success = int128_eq(oldv, cmpv); -#endif - } else { - uint64_t o0, o1; - #ifdef CONFIG_USER_ONLY - /* ??? Enforce alignment. */ - uint64_t *haddr = g2h(addr); + /* ??? Enforce alignment. */ + uint64_t *haddr = g2h(addr); - helper_retaddr = ra; - o0 = ldq_le_p(haddr + 0); - o1 = ldq_le_p(haddr + 1); - oldv = int128_make128(o0, o1); + helper_retaddr = ra; + o0 = ldq_le_p(haddr + 0); + o1 = ldq_le_p(haddr + 1); + oldv = int128_make128(o0, o1); - success = int128_eq(oldv, cmpv); - if (success) { - stq_le_p(haddr + 0, int128_getlo(newv)); - stq_le_p(haddr + 1, int128_gethi(newv)); - } - helper_retaddr = 0; -#else - int mem_idx = cpu_mmu_index(env, false); - TCGMemOpIdx oi0 = make_memop_idx(MO_LEQ | MO_ALIGN_16, mem_idx); - TCGMemOpIdx oi1 = make_memop_idx(MO_LEQ, mem_idx); - - o0 = helper_le_ldq_mmu(env, addr + 0, oi0, ra); - o1 = helper_le_ldq_mmu(env, addr + 8, oi1, ra); - oldv = int128_make128(o0, o1); - - success = int128_eq(oldv, cmpv); - if (success) { - helper_le_stq_mmu(env, addr + 0, int128_getlo(newv), oi1, ra); - helper_le_stq_mmu(env, addr + 8, int128_gethi(newv), oi1, ra); - } -#endif + success = int128_eq(oldv, cmpv); + if (success) { + stq_le_p(haddr + 0, int128_getlo(newv)); + stq_le_p(haddr + 1, int128_gethi(newv)); } + helper_retaddr = 0; +#else + int mem_idx = cpu_mmu_index(env, false); + TCGMemOpIdx oi0 = make_memop_idx(MO_LEQ | MO_ALIGN_16, mem_idx); + TCGMemOpIdx oi1 = make_memop_idx(MO_LEQ, mem_idx); + + o0 = helper_le_ldq_mmu(env, addr + 0, oi0, ra); + o1 = helper_le_ldq_mmu(env, addr + 8, oi1, ra); + oldv = int128_make128(o0, o1); + + success = int128_eq(oldv, cmpv); + if (success) { + helper_le_stq_mmu(env, addr + 0, int128_getlo(newv), oi1, ra); + helper_le_stq_mmu(env, addr + 8, int128_gethi(newv), oi1, ra); + } +#endif return !success; } -uint64_t HELPER(paired_cmpxchg64_le)(CPUARMState *env, uint64_t addr, - uint64_t new_lo, uint64_t new_hi) -{ - return do_paired_cmpxchg64_le(env, addr, new_lo, new_hi, false, GETPC()); -} - uint64_t HELPER(paired_cmpxchg64_le_parallel)(CPUARMState *env, uint64_t addr, uint64_t new_lo, uint64_t new_hi) -{ - return do_paired_cmpxchg64_le(env, addr, new_lo, new_hi, true, GETPC()); -} - -static uint64_t do_paired_cmpxchg64_be(CPUARMState *env, uint64_t addr, - uint64_t new_lo, uint64_t new_hi, - bool parallel, uintptr_t ra) { Int128 oldv, cmpv, newv; + uintptr_t ra = GETPC(); bool success; + int mem_idx; + TCGMemOpIdx oi; - /* high and low need to be switched here because this is not actually a - * 128bit store but two doublewords stored consecutively - */ - cmpv = int128_make128(env->exclusive_high, env->exclusive_val); - newv = int128_make128(new_hi, new_lo); - - if (parallel) { -#ifndef CONFIG_ATOMIC128 + if (!HAVE_CMPXCHG128) { cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); -#else - int mem_idx = cpu_mmu_index(env, false); - TCGMemOpIdx oi = make_memop_idx(MO_BEQ | MO_ALIGN_16, mem_idx); - oldv = helper_atomic_cmpxchgo_be_mmu(env, addr, cmpv, newv, oi, ra); - success = int128_eq(oldv, cmpv); -#endif - } else { - uint64_t o0, o1; - -#ifdef CONFIG_USER_ONLY - /* ??? Enforce alignment. */ - uint64_t *haddr = g2h(addr); - - helper_retaddr = ra; - o1 = ldq_be_p(haddr + 0); - o0 = ldq_be_p(haddr + 1); - oldv = int128_make128(o0, o1); - - success = int128_eq(oldv, cmpv); - if (success) { - stq_be_p(haddr + 0, int128_gethi(newv)); - stq_be_p(haddr + 1, int128_getlo(newv)); - } - helper_retaddr = 0; -#else - int mem_idx = cpu_mmu_index(env, false); - TCGMemOpIdx oi0 = make_memop_idx(MO_BEQ | MO_ALIGN_16, mem_idx); - TCGMemOpIdx oi1 = make_memop_idx(MO_BEQ, mem_idx); - - o1 = helper_be_ldq_mmu(env, addr + 0, oi0, ra); - o0 = helper_be_ldq_mmu(env, addr + 8, oi1, ra); - oldv = int128_make128(o0, o1); - - success = int128_eq(oldv, cmpv); - if (success) { - helper_be_stq_mmu(env, addr + 0, int128_gethi(newv), oi1, ra); - helper_be_stq_mmu(env, addr + 8, int128_getlo(newv), oi1, ra); - } -#endif } + mem_idx = cpu_mmu_index(env, false); + oi = make_memop_idx(MO_LEQ | MO_ALIGN_16, mem_idx); + + cmpv = int128_make128(env->exclusive_val, env->exclusive_high); + newv = int128_make128(new_lo, new_hi); + oldv = helper_atomic_cmpxchgo_le_mmu(env, addr, cmpv, newv, oi, ra); + + success = int128_eq(oldv, cmpv); return !success; } uint64_t HELPER(paired_cmpxchg64_be)(CPUARMState *env, uint64_t addr, uint64_t new_lo, uint64_t new_hi) { - return do_paired_cmpxchg64_be(env, addr, new_lo, new_hi, false, GETPC()); + /* + * High and low need to be switched here because this is not actually a + * 128bit store but two doublewords stored consecutively + */ + Int128 cmpv = int128_make128(env->exclusive_val, env->exclusive_high); + Int128 newv = int128_make128(new_lo, new_hi); + Int128 oldv; + uintptr_t ra = GETPC(); + uint64_t o0, o1; + bool success; + +#ifdef CONFIG_USER_ONLY + /* ??? Enforce alignment. */ + uint64_t *haddr = g2h(addr); + + helper_retaddr = ra; + o1 = ldq_be_p(haddr + 0); + o0 = ldq_be_p(haddr + 1); + oldv = int128_make128(o0, o1); + + success = int128_eq(oldv, cmpv); + if (success) { + stq_be_p(haddr + 0, int128_gethi(newv)); + stq_be_p(haddr + 1, int128_getlo(newv)); + } + helper_retaddr = 0; +#else + int mem_idx = cpu_mmu_index(env, false); + TCGMemOpIdx oi0 = make_memop_idx(MO_BEQ | MO_ALIGN_16, mem_idx); + TCGMemOpIdx oi1 = make_memop_idx(MO_BEQ, mem_idx); + + o1 = helper_be_ldq_mmu(env, addr + 0, oi0, ra); + o0 = helper_be_ldq_mmu(env, addr + 8, oi1, ra); + oldv = int128_make128(o0, o1); + + success = int128_eq(oldv, cmpv); + if (success) { + helper_be_stq_mmu(env, addr + 0, int128_gethi(newv), oi1, ra); + helper_be_stq_mmu(env, addr + 8, int128_getlo(newv), oi1, ra); + } +#endif + + return !success; } uint64_t HELPER(paired_cmpxchg64_be_parallel)(CPUARMState *env, uint64_t addr, - uint64_t new_lo, uint64_t new_hi) + uint64_t new_lo, uint64_t new_hi) { - return do_paired_cmpxchg64_be(env, addr, new_lo, new_hi, true, GETPC()); + Int128 oldv, cmpv, newv; + uintptr_t ra = GETPC(); + bool success; + int mem_idx; + TCGMemOpIdx oi; + + if (!HAVE_CMPXCHG128) { + cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); + } + + mem_idx = cpu_mmu_index(env, false); + oi = make_memop_idx(MO_BEQ | MO_ALIGN_16, mem_idx); + + /* + * High and low need to be switched here because this is not actually a + * 128bit store but two doublewords stored consecutively + */ + cmpv = int128_make128(env->exclusive_high, env->exclusive_val); + newv = int128_make128(new_hi, new_lo); + oldv = helper_atomic_cmpxchgo_be_mmu(env, addr, cmpv, newv, oi, ra); + + success = int128_eq(oldv, cmpv); + return !success; } /* Writes back the old data into Rs. */ void HELPER(casp_le_parallel)(CPUARMState *env, uint32_t rs, uint64_t addr, uint64_t new_lo, uint64_t new_hi) { - uintptr_t ra = GETPC(); -#ifndef CONFIG_ATOMIC128 - cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); -#else Int128 oldv, cmpv, newv; + uintptr_t ra = GETPC(); + int mem_idx; + TCGMemOpIdx oi; + + if (!HAVE_CMPXCHG128) { + cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); + } + + mem_idx = cpu_mmu_index(env, false); + oi = make_memop_idx(MO_LEQ | MO_ALIGN_16, mem_idx); cmpv = int128_make128(env->xregs[rs], env->xregs[rs + 1]); newv = int128_make128(new_lo, new_hi); - - int mem_idx = cpu_mmu_index(env, false); - TCGMemOpIdx oi = make_memop_idx(MO_LEQ | MO_ALIGN_16, mem_idx); oldv = helper_atomic_cmpxchgo_le_mmu(env, addr, cmpv, newv, oi, ra); env->xregs[rs] = int128_getlo(oldv); env->xregs[rs + 1] = int128_gethi(oldv); -#endif } void HELPER(casp_be_parallel)(CPUARMState *env, uint32_t rs, uint64_t addr, uint64_t new_hi, uint64_t new_lo) { - uintptr_t ra = GETPC(); -#ifndef CONFIG_ATOMIC128 - cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); -#else Int128 oldv, cmpv, newv; + uintptr_t ra = GETPC(); + int mem_idx; + TCGMemOpIdx oi; + + if (!HAVE_CMPXCHG128) { + cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); + } + + mem_idx = cpu_mmu_index(env, false); + oi = make_memop_idx(MO_LEQ | MO_ALIGN_16, mem_idx); cmpv = int128_make128(env->xregs[rs + 1], env->xregs[rs]); newv = int128_make128(new_lo, new_hi); - - int mem_idx = cpu_mmu_index(env, false); - TCGMemOpIdx oi = make_memop_idx(MO_LEQ | MO_ALIGN_16, mem_idx); oldv = helper_atomic_cmpxchgo_be_mmu(env, addr, cmpv, newv, oi, ra); env->xregs[rs + 1] = int128_getlo(oldv); env->xregs[rs] = int128_gethi(oldv); -#endif } /* From patchwork Thu Aug 16 02:54:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 144307 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp1540853ljj; Wed, 15 Aug 2018 19:59:48 -0700 (PDT) X-Google-Smtp-Source: AA+uWPzPcmOQTtcJnQXTb1Bgwv1PT5026+AvIanBrxv42L1vGfdJl5NPRoVAZw8idFSE2LA6kqZ/ X-Received: by 2002:ac8:2f28:: with SMTP id j37-v6mr28747646qta.103.1534388388456; Wed, 15 Aug 2018 19:59:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534388388; cv=none; d=google.com; s=arc-20160816; b=H5oDRlmg9F9INU820ylX2jxf04PJ77uN/iH5Mf7iu7apDpfm/MPtNCV3jhUoylEmui PcC3V3BYTzZQU5A5XqX0nLXD/jWfpdfQgLjVXuKIw1EtFpDI8i7u1dTVJqJWstu49JOo o5z6q+PMxgAvOenVXtBaG99EKgcNr49E+W6/3ks8JO0bFijQa2qhvRiVrF9vyHwL/ram s6NwM3opBbuXhkilWLQGunImjeyRaaIKLscvOlsr4QTd75LfWIeWM0nTi3ra/eZTzzKh uAEhi7nDE8zW8/AIPOSjt5dvOgOsdIC3AlxDY3LKbnQSBksJXttNN2eALeS5LG4W6/ie 7bMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=ktLMPDzhQZ0XWWdzaad2T/JmPXgvVwCZ5qRncug/9Ds=; b=0QeGiN3wAnbMikjS+2kjImHYtmiIJZPMmEUIUQZl9Y1l/xnOxEYm7IaqxvF7dJyEG9 tqENsP8YGZWmkW1G7PEpVaMzHR/uJARRVnATzZ2FUkBjw/Qs5zaP3VM3kgzlbsGsGhCn zXMApN8aObdlTm9fW1jJjTB82D25IlCCu6X7lXHPCBgNU1o41oa5+kawSpkzG7Q1ejP3 ouQKNqk5eykpnlX/U+tHqTLCZfaj48TeMpXlxD2EFtWVJNTc1d4SW45THWoibozb8J0Y y8+n0fB/dgvhvGv2+MFvh1fJAS6qKwwNbtWVG2h5XRZOTCbhiXr/qLYnnBnmZKpDibqE MJ2g== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=OJJDWXw5; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id v16-v6si10048828qvk.273.2018.08.15.19.59.48 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 15 Aug 2018 19:59:48 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=OJJDWXw5; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53246 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fq8Vn-00061L-VI for patch@linaro.org; Wed, 15 Aug 2018 22:59:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50495) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fq8RE-0002K8-Dd for qemu-devel@nongnu.org; Wed, 15 Aug 2018 22:55:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fq8RD-0006nA-2A for qemu-devel@nongnu.org; Wed, 15 Aug 2018 22:55:04 -0400 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]:35051) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fq8RC-0006mZ-O4 for qemu-devel@nongnu.org; Wed, 15 Aug 2018 22:55:02 -0400 Received: by mail-pg1-x535.google.com with SMTP id w10-v6so1361376pgv.2 for ; Wed, 15 Aug 2018 19:55:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ktLMPDzhQZ0XWWdzaad2T/JmPXgvVwCZ5qRncug/9Ds=; b=OJJDWXw5ROAGpKG7vcKvmPV+rDZ7g/V7QL5CiM+CU7saoxEBsV38zmqRfaTRVtPzPy Kih4VwW/so2bN8H9S+/RLdh9ixN/QtSm80sY2dlHHH4bqs0JrzylAEw3aiK797iMEZcl /gG36G+JAtOsEONa9YyR6AVFEA15J/+eHvnqA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ktLMPDzhQZ0XWWdzaad2T/JmPXgvVwCZ5qRncug/9Ds=; b=oKlCrlYgkW1YB3vZwYHvfA20VORLG13kTKcF75Z1UNZH71n8EDyFUT3v1T+x0P0wFe YCv4WNBs26xHnQsb3wANbAi31e50yGGOTgPDXxto8EiIhP1Hl8Hd79IGOJqOhm0u9lP0 tEI21zMi/F1RkvklmN5Zb3MCfO/qzd1HT9wAz1oTiMd2mNLRuKALRM05iab6KAZC8AYd iyjlyDyR+tWawTDgiKwjBILRrpS2tgzUZmmnuQUU3NWV9+xX0XbP9bticCBl++T5YWO1 Cftx3x64bDZvX2WSbefqVWFMUCY+rErvuQS28JqVFzeAy6jOlLZiVPh+wW5+Y0PP1ilv aMmg== X-Gm-Message-State: AOUpUlExtqPPlxKNlVRuF4xbtXGk4iWXGRItmtdxsNJ3K2QO5QbVG8WQ pMjyicLks0T8gCf1/GZdMBTG4qfHlC0= X-Received: by 2002:a63:bc0a:: with SMTP id q10-v6mr79754pge.60.1534388101443; Wed, 15 Aug 2018 19:55:01 -0700 (PDT) Received: from cloudburst.twiddle.net (97-113-8-179.tukw.qwest.net. [97.113.8.179]) by smtp.gmail.com with ESMTPSA id k64-v6sm45231497pfc.160.2018.08.15.19.55.00 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 15 Aug 2018 19:55:00 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 15 Aug 2018 19:54:52 -0700 Message-Id: <20180816025452.21358-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180816025452.21358-1-richard.henderson@linaro.org> References: <20180816025452.21358-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::535 Subject: [Qemu-devel] [PATCH 5/5] target/ppc: Convert to HAVE_CMPXCHG128 and HAVE_ATOMIC128 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, cota@braap.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/ppc/helper.h | 2 +- target/ppc/mem_helper.c | 33 ++++++++++-- target/ppc/translate.c | 115 +++++++++++++++++++++------------------- 3 files changed, 88 insertions(+), 62 deletions(-) -- 2.17.1 Reviewed-by: Emilio G. Cota diff --git a/target/ppc/helper.h b/target/ppc/helper.h index 5706c2497f..039f755eae 100644 --- a/target/ppc/helper.h +++ b/target/ppc/helper.h @@ -800,7 +800,7 @@ DEF_HELPER_4(dscliq, void, env, fprp, fprp, i32) DEF_HELPER_1(tbegin, void, env) DEF_HELPER_FLAGS_1(fixup_thrm, TCG_CALL_NO_RWG, void, env) -#if defined(TARGET_PPC64) && defined(CONFIG_ATOMIC128) +#ifdef TARGET_PPC64 DEF_HELPER_FLAGS_3(lq_le_parallel, TCG_CALL_NO_WG, i64, env, tl, i32) DEF_HELPER_FLAGS_3(lq_be_parallel, TCG_CALL_NO_WG, i64, env, tl, i32) DEF_HELPER_FLAGS_5(stq_le_parallel, TCG_CALL_NO_WG, diff --git a/target/ppc/mem_helper.c b/target/ppc/mem_helper.c index 8f0d86d104..a1485fad9b 100644 --- a/target/ppc/mem_helper.c +++ b/target/ppc/mem_helper.c @@ -25,6 +25,7 @@ #include "exec/cpu_ldst.h" #include "tcg.h" #include "internal.h" +#include "qemu/atomic128.h" //#define DEBUG_OP @@ -215,11 +216,15 @@ target_ulong helper_lscbx(CPUPPCState *env, target_ulong addr, uint32_t reg, return i; } -#if defined(TARGET_PPC64) && defined(CONFIG_ATOMIC128) +#ifdef TARGET_PPC64 uint64_t helper_lq_le_parallel(CPUPPCState *env, target_ulong addr, uint32_t opidx) { - Int128 ret = helper_atomic_ldo_le_mmu(env, addr, opidx, GETPC()); + Int128 ret; + + /* We will have raised EXCP_ATOMIC from the translator. */ + assert(HAVE_ATOMIC128); + ret = helper_atomic_ldo_le_mmu(env, addr, opidx, GETPC()); env->retxh = int128_gethi(ret); return int128_getlo(ret); } @@ -227,7 +232,11 @@ uint64_t helper_lq_le_parallel(CPUPPCState *env, target_ulong addr, uint64_t helper_lq_be_parallel(CPUPPCState *env, target_ulong addr, uint32_t opidx) { - Int128 ret = helper_atomic_ldo_be_mmu(env, addr, opidx, GETPC()); + Int128 ret; + + /* We will have raised EXCP_ATOMIC from the translator. */ + assert(HAVE_ATOMIC128); + ret = helper_atomic_ldo_be_mmu(env, addr, opidx, GETPC()); env->retxh = int128_gethi(ret); return int128_getlo(ret); } @@ -235,14 +244,22 @@ uint64_t helper_lq_be_parallel(CPUPPCState *env, target_ulong addr, void helper_stq_le_parallel(CPUPPCState *env, target_ulong addr, uint64_t lo, uint64_t hi, uint32_t opidx) { - Int128 val = int128_make128(lo, hi); + Int128 val; + + /* We will have raised EXCP_ATOMIC from the translator. */ + assert(HAVE_ATOMIC128); + val = int128_make128(lo, hi); helper_atomic_sto_le_mmu(env, addr, val, opidx, GETPC()); } void helper_stq_be_parallel(CPUPPCState *env, target_ulong addr, uint64_t lo, uint64_t hi, uint32_t opidx) { - Int128 val = int128_make128(lo, hi); + Int128 val; + + /* We will have raised EXCP_ATOMIC from the translator. */ + assert(HAVE_ATOMIC128); + val = int128_make128(lo, hi); helper_atomic_sto_be_mmu(env, addr, val, opidx, GETPC()); } @@ -252,6 +269,9 @@ uint32_t helper_stqcx_le_parallel(CPUPPCState *env, target_ulong addr, { bool success = false; + /* We will have raised EXCP_ATOMIC from the translator. */ + assert(HAVE_CMPXCHG128); + if (likely(addr == env->reserve_addr)) { Int128 oldv, cmpv, newv; @@ -271,6 +291,9 @@ uint32_t helper_stqcx_be_parallel(CPUPPCState *env, target_ulong addr, { bool success = false; + /* We will have raised EXCP_ATOMIC from the translator. */ + assert(HAVE_CMPXCHG128); + if (likely(addr == env->reserve_addr)) { Int128 oldv, cmpv, newv; diff --git a/target/ppc/translate.c b/target/ppc/translate.c index 9eaa10b421..c3c8b91729 100644 --- a/target/ppc/translate.c +++ b/target/ppc/translate.c @@ -33,6 +33,7 @@ #include "trace-tcg.h" #include "exec/translator.h" #include "exec/log.h" +#include "qemu/atomic128.h" #define CPU_SINGLE_STEP 0x1 @@ -2621,22 +2622,22 @@ static void gen_lq(DisasContext *ctx) hi = cpu_gpr[rd]; if (tb_cflags(ctx->base.tb) & CF_PARALLEL) { -#ifdef CONFIG_ATOMIC128 - TCGv_i32 oi = tcg_temp_new_i32(); - if (ctx->le_mode) { - tcg_gen_movi_i32(oi, make_memop_idx(MO_LEQ, ctx->mem_idx)); - gen_helper_lq_le_parallel(lo, cpu_env, EA, oi); + if (HAVE_ATOMIC128) { + TCGv_i32 oi = tcg_temp_new_i32(); + if (ctx->le_mode) { + tcg_gen_movi_i32(oi, make_memop_idx(MO_LEQ, ctx->mem_idx)); + gen_helper_lq_le_parallel(lo, cpu_env, EA, oi); + } else { + tcg_gen_movi_i32(oi, make_memop_idx(MO_BEQ, ctx->mem_idx)); + gen_helper_lq_be_parallel(lo, cpu_env, EA, oi); + } + tcg_temp_free_i32(oi); + tcg_gen_ld_i64(hi, cpu_env, offsetof(CPUPPCState, retxh)); } else { - tcg_gen_movi_i32(oi, make_memop_idx(MO_BEQ, ctx->mem_idx)); - gen_helper_lq_be_parallel(lo, cpu_env, EA, oi); + /* Restart with exclusive lock. */ + gen_helper_exit_atomic(cpu_env); + ctx->base.is_jmp = DISAS_NORETURN; } - tcg_temp_free_i32(oi); - tcg_gen_ld_i64(hi, cpu_env, offsetof(CPUPPCState, retxh)); -#else - /* Restart with exclusive lock. */ - gen_helper_exit_atomic(cpu_env); - ctx->base.is_jmp = DISAS_NORETURN; -#endif } else if (ctx->le_mode) { tcg_gen_qemu_ld_i64(lo, EA, ctx->mem_idx, MO_LEQ); gen_addr_add(ctx, EA, EA, 8); @@ -2772,21 +2773,21 @@ static void gen_std(DisasContext *ctx) hi = cpu_gpr[rs]; if (tb_cflags(ctx->base.tb) & CF_PARALLEL) { -#ifdef CONFIG_ATOMIC128 - TCGv_i32 oi = tcg_temp_new_i32(); - if (ctx->le_mode) { - tcg_gen_movi_i32(oi, make_memop_idx(MO_LEQ, ctx->mem_idx)); - gen_helper_stq_le_parallel(cpu_env, EA, lo, hi, oi); + if (HAVE_ATOMIC128) { + TCGv_i32 oi = tcg_temp_new_i32(); + if (ctx->le_mode) { + tcg_gen_movi_i32(oi, make_memop_idx(MO_LEQ, ctx->mem_idx)); + gen_helper_stq_le_parallel(cpu_env, EA, lo, hi, oi); + } else { + tcg_gen_movi_i32(oi, make_memop_idx(MO_BEQ, ctx->mem_idx)); + gen_helper_stq_be_parallel(cpu_env, EA, lo, hi, oi); + } + tcg_temp_free_i32(oi); } else { - tcg_gen_movi_i32(oi, make_memop_idx(MO_BEQ, ctx->mem_idx)); - gen_helper_stq_be_parallel(cpu_env, EA, lo, hi, oi); + /* Restart with exclusive lock. */ + gen_helper_exit_atomic(cpu_env); + ctx->base.is_jmp = DISAS_NORETURN; } - tcg_temp_free_i32(oi); -#else - /* Restart with exclusive lock. */ - gen_helper_exit_atomic(cpu_env); - ctx->base.is_jmp = DISAS_NORETURN; -#endif } else if (ctx->le_mode) { tcg_gen_qemu_st_i64(lo, EA, ctx->mem_idx, MO_LEQ); gen_addr_add(ctx, EA, EA, 8); @@ -3371,26 +3372,26 @@ static void gen_lqarx(DisasContext *ctx) hi = cpu_gpr[rd]; if (tb_cflags(ctx->base.tb) & CF_PARALLEL) { -#ifdef CONFIG_ATOMIC128 - TCGv_i32 oi = tcg_temp_new_i32(); - if (ctx->le_mode) { - tcg_gen_movi_i32(oi, make_memop_idx(MO_LEQ | MO_ALIGN_16, - ctx->mem_idx)); - gen_helper_lq_le_parallel(lo, cpu_env, EA, oi); + if (HAVE_ATOMIC128) { + TCGv_i32 oi = tcg_temp_new_i32(); + if (ctx->le_mode) { + tcg_gen_movi_i32(oi, make_memop_idx(MO_LEQ | MO_ALIGN_16, + ctx->mem_idx)); + gen_helper_lq_le_parallel(lo, cpu_env, EA, oi); + } else { + tcg_gen_movi_i32(oi, make_memop_idx(MO_BEQ | MO_ALIGN_16, + ctx->mem_idx)); + gen_helper_lq_be_parallel(lo, cpu_env, EA, oi); + } + tcg_temp_free_i32(oi); + tcg_gen_ld_i64(hi, cpu_env, offsetof(CPUPPCState, retxh)); } else { - tcg_gen_movi_i32(oi, make_memop_idx(MO_BEQ | MO_ALIGN_16, - ctx->mem_idx)); - gen_helper_lq_be_parallel(lo, cpu_env, EA, oi); + /* Restart with exclusive lock. */ + gen_helper_exit_atomic(cpu_env); + ctx->base.is_jmp = DISAS_NORETURN; + tcg_temp_free(EA); + return; } - tcg_temp_free_i32(oi); - tcg_gen_ld_i64(hi, cpu_env, offsetof(CPUPPCState, retxh)); -#else - /* Restart with exclusive lock. */ - gen_helper_exit_atomic(cpu_env); - ctx->base.is_jmp = DISAS_NORETURN; - tcg_temp_free(EA); - return; -#endif } else if (ctx->le_mode) { tcg_gen_qemu_ld_i64(lo, EA, ctx->mem_idx, MO_LEQ | MO_ALIGN_16); tcg_gen_mov_tl(cpu_reserve, EA); @@ -3428,20 +3429,22 @@ static void gen_stqcx_(DisasContext *ctx) hi = cpu_gpr[rs]; if (tb_cflags(ctx->base.tb) & CF_PARALLEL) { - TCGv_i32 oi = tcg_const_i32(DEF_MEMOP(MO_Q) | MO_ALIGN_16); -#ifdef CONFIG_ATOMIC128 - if (ctx->le_mode) { - gen_helper_stqcx_le_parallel(cpu_crf[0], cpu_env, EA, lo, hi, oi); + if (HAVE_CMPXCHG128) { + TCGv_i32 oi = tcg_const_i32(DEF_MEMOP(MO_Q) | MO_ALIGN_16); + if (ctx->le_mode) { + gen_helper_stqcx_le_parallel(cpu_crf[0], cpu_env, + EA, lo, hi, oi); + } else { + gen_helper_stqcx_be_parallel(cpu_crf[0], cpu_env, + EA, lo, hi, oi); + } + tcg_temp_free_i32(oi); } else { - gen_helper_stqcx_le_parallel(cpu_crf[0], cpu_env, EA, lo, hi, oi); + /* Restart with exclusive lock. */ + gen_helper_exit_atomic(cpu_env); + ctx->base.is_jmp = DISAS_NORETURN; } -#else - /* Restart with exclusive lock. */ - gen_helper_exit_atomic(cpu_env); - ctx->base.is_jmp = DISAS_NORETURN; -#endif tcg_temp_free(EA); - tcg_temp_free_i32(oi); } else { TCGLabel *lab_fail = gen_new_label(); TCGLabel *lab_over = gen_new_label();