From patchwork Wed Jan 7 20:56:31 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ola Liljedahl X-Patchwork-Id: 42855 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ee0-f71.google.com (mail-ee0-f71.google.com [74.125.83.71]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 1FA3120558 for ; Wed, 7 Jan 2015 20:56:37 +0000 (UTC) Received: by mail-ee0-f71.google.com with SMTP id c13sf3493642eek.2 for ; Wed, 07 Jan 2015 12:56:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:subject :precedence:list-id:list-unsubscribe:list-archive:list-post :list-help:list-subscribe:mime-version:content-type :content-transfer-encoding:errors-to:sender:x-original-sender :x-original-authentication-results:mailing-list; bh=kv4fRfWEDn/rOXJqW07yj2KbZkGhkCXW3P2TlVXrJm0=; b=lWHd2fLmSg9TBrLJ61L1BJ40qj5cP7q523TXpm3J20R4sggTJBEYeyrxvUW5sMmoog A0bctS612u/cCI1SWS7tEsTqJq1o6VUFlC55XwndJECUjReT6Aa9OApqxUPGJxmC0OtA 2m31g9cbgDCOOWJfXpX70yU5AQ+L6ugGxWuIx1m+x26w794BX/Mk8wXcCgMNY5F3h+3N iDvc2DzkedjzCv5Mma5df42B5KSnUHu7gwbxs31KhNOuWo+64+RJhAUM9/Y28/8lF/YK XiZXYiP1Foo7N8gj6vfnK89hX1y5w8n7ONjjUjCQKXljV4yjx3Tdy5KweOB7W1JNlT7A Vxfg== X-Gm-Message-State: ALoCoQlTZqeY3yFhkyuWx4TEff3APccPdsOnuhcZswoRWlW2oBVidIgkuir9OwqhsxSQC+wIQOSV X-Received: by 10.152.44.225 with SMTP id h1mr712557lam.2.1420664196325; Wed, 07 Jan 2015 12:56:36 -0800 (PST) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.5.9 with SMTP id o9ls235507lao.83.gmail; Wed, 07 Jan 2015 12:56:36 -0800 (PST) X-Received: by 10.112.84.225 with SMTP id c1mr8224749lbz.22.1420664196110; Wed, 07 Jan 2015 12:56:36 -0800 (PST) Received: from mail-la0-f51.google.com (mail-la0-f51.google.com. [209.85.215.51]) by mx.google.com with ESMTPS id k1si5015518laf.21.2015.01.07.12.56.36 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 07 Jan 2015 12:56:36 -0800 (PST) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.51 as permitted sender) client-ip=209.85.215.51; Received: by mail-la0-f51.google.com with SMTP id ms9so5805530lab.10 for ; Wed, 07 Jan 2015 12:56:36 -0800 (PST) X-Received: by 10.152.23.38 with SMTP id j6mr8217777laf.81.1420664196012; Wed, 07 Jan 2015 12:56:36 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.9.200 with SMTP id c8csp60430lbb; Wed, 7 Jan 2015 12:56:35 -0800 (PST) X-Received: by 10.224.131.4 with SMTP id v4mr7970734qas.99.1420664194669; Wed, 07 Jan 2015 12:56:34 -0800 (PST) Received: from ip-10-35-177-41.ec2.internal (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTPS id x10si3319775qat.90.2015.01.07.12.56.33 (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 07 Jan 2015 12:56:34 -0800 (PST) Received-SPF: none (google.com: lng-odp-bounces@lists.linaro.org does not designate permitted sender hosts) client-ip=54.225.227.206; Received: from localhost ([127.0.0.1] helo=ip-10-35-177-41.ec2.internal) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1Y8xeY-0007e7-Q6; Wed, 07 Jan 2015 20:56:30 +0000 Received: from mail-lb0-f173.google.com ([209.85.217.173]) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1Y8xeS-0007dk-Ry for lng-odp@lists.linaro.org; Wed, 07 Jan 2015 20:56:25 +0000 Received: by mail-lb0-f173.google.com with SMTP id z12so1754200lbi.4 for ; Wed, 07 Jan 2015 12:56:18 -0800 (PST) X-Received: by 10.152.45.4 with SMTP id i4mr8306692lam.74.1420664178775; Wed, 07 Jan 2015 12:56:18 -0800 (PST) Received: from macmini.lan (78-82-118-111.tn.glocalnet.net. [78.82.118.111]) by mx.google.com with ESMTPSA id s16sm687424lal.5.2015.01.07.12.56.17 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 07 Jan 2015 12:56:17 -0800 (PST) From: Ola Liljedahl To: lng-odp@lists.linaro.org Date: Wed, 7 Jan 2015 21:56:31 +0100 Message-Id: <1420664191-9848-1-git-send-email-ola.liljedahl@linaro.org> X-Mailer: git-send-email 1.9.1 X-Topics: patch Subject: [lng-odp] [PATCHv2] linux-generic: odp_atomic_internal.h: more atomic types X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: lng-odp-bounces@lists.linaro.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ola.liljedahl@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.51 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Limited support (swap, cas) for 128-bit atomic types and operations, conditionally enabled if and when the compiler supports it. Changed some return types from int to odp_bool_t to better match the semantics of the returned values. Signed-off-by: Ola Liljedahl --- (This document/code contribution attached is provided under the terms of agreement LES-LTM-21309) This is actually v2 of patch 1/2 from 2014-12-01. Patch 2/2 became the separate timer patch set. The 128-bit atomics support is optionally needed by the new lock-less timer implementation. .../linux-generic/include/odp_atomic_internal.h | 73 +++++++++++++++++++--- 1 file changed, 65 insertions(+), 8 deletions(-) diff --git a/platform/linux-generic/include/odp_atomic_internal.h b/platform/linux-generic/include/odp_atomic_internal.h index dafa222..91d2c3e 100644 --- a/platform/linux-generic/include/odp_atomic_internal.h +++ b/platform/linux-generic/include/odp_atomic_internal.h @@ -146,7 +146,7 @@ static inline uint32_t _odp_atomic_u32_xchg_mm(odp_atomic_u32_t *atom, * @retval 1 exchange successul * @retval 0 exchange failed and '*exp' updated with current value */ -static inline int _odp_atomic_u32_cmp_xchg_strong_mm( +static inline odp_bool_t _odp_atomic_u32_cmp_xchg_strong_mm( odp_atomic_u32_t *atom, uint32_t *exp, uint32_t val, @@ -304,7 +304,6 @@ static inline void _odp_atomic_u64_store_mm(odp_atomic_u64_t *atom, static inline uint64_t _odp_atomic_u64_xchg_mm(odp_atomic_u64_t *atom, uint64_t val, _odp_memmodel_t mmodel) - { #if !defined __GCC_ATOMIC_LLONG_LOCK_FREE || __GCC_ATOMIC_LLONG_LOCK_FREE < 2 return ATOMIC_OP_MM(atom, atom->v = val, mmodel); @@ -328,7 +327,8 @@ static inline uint64_t _odp_atomic_u64_xchg_mm(odp_atomic_u64_t *atom, * @retval 1 exchange successful * @retval 0 exchange failed and '*exp' updated with current value */ -static inline int _odp_atomic_u64_cmp_xchg_strong_mm(odp_atomic_u64_t *atom, +static inline +odp_bool_t _odp_atomic_u64_cmp_xchg_strong_mm(odp_atomic_u64_t *atom, uint64_t *exp, uint64_t val, _odp_memmodel_t success, @@ -389,7 +389,6 @@ static inline uint64_t _odp_atomic_u64_fetch_add_mm(odp_atomic_u64_t *atom, static inline void _odp_atomic_u64_add_mm(odp_atomic_u64_t *atom, uint64_t val, _odp_memmodel_t mmodel) - { #if !defined __GCC_ATOMIC_LLONG_LOCK_FREE || __GCC_ATOMIC_LLONG_LOCK_FREE < 2 (void)ATOMIC_OP_MM(atom, atom->v += val, mmodel); @@ -428,7 +427,6 @@ static inline uint64_t _odp_atomic_u64_fetch_sub_mm(odp_atomic_u64_t *atom, static inline void _odp_atomic_u64_sub_mm(odp_atomic_u64_t *atom, uint64_t val, _odp_memmodel_t mmodel) - { #if !defined __GCC_ATOMIC_LLONG_LOCK_FREE || __GCC_ATOMIC_LLONG_LOCK_FREE < 2 (void)ATOMIC_OP_MM(atom, atom->v -= val, mmodel); @@ -519,7 +517,7 @@ static inline void *_odp_atomic_ptr_xchg(_odp_atomic_ptr_t *atom, * @retval 1 exchange successful * @retval 0 exchange failed and '*exp' updated with current value */ -static inline int _odp_atomic_ptr_cmp_xchg_strong( +static inline odp_bool_t _odp_atomic_ptr_cmp_xchg_strong( _odp_atomic_ptr_t *atom, void **exp, void *val, @@ -542,7 +540,7 @@ static inline int _odp_atomic_ptr_cmp_xchg_strong( *****************************************************************************/ /** - * Initialize a flag atomic variable + * Initialize an atomic flag variable * * @param[out] flag Pointer to a flag atomic variable * @param val The initial value of the variable @@ -570,7 +568,8 @@ static inline int _odp_atomic_flag_load(_odp_atomic_flag_t *flag) /** * Test-and-set of atomic flag variable * @Note Operation has acquire semantics. It pairs with a later - * release operation. + * release operation in some thread. It does not provide release or + * acquire/release semantics. * * @param[in,out] flag Pointer to a flag atomic variable * @@ -595,6 +594,64 @@ static inline void _odp_atomic_flag_clear(_odp_atomic_flag_t *flag) __atomic_clear(flag, __ATOMIC_RELEASE); } +/* Check if target and compiler supports 128-bit scalars and corresponding + * exchange and CAS operations */ +/* GCC on x86-64 needs -mcx16 compiler option */ +#if defined __SIZEOF_INT128__ && defined __GCC_HAVE_SYNC_COMPARE_AND_SWAP_16 + +/** Preprocessor symbol that indicates support for 128-bit atomics */ +#define ODP_ATOMIC_U128 + +/** An unsigned 128-bit (16-byte) scalar type */ +typedef __int128 _uint128_t; + +/** Atomic 128-bit type */ +typedef struct { + _uint128_t v; /**< Actual storage for the atomic variable */ +} _odp_atomic_u128_t ODP_ALIGNED(16); + +/** + * 16-byte atomic exchange operation + * + * @param ptr Pointer to a 16-byte atomic variable + * @param val Pointer to new value to write + * @param old Pointer to location for old value + * @param mmodel Memory model associated with the exchange operation + */ +static inline void _odp_atomic_u128_xchg_mm(_odp_atomic_u128_t *ptr, + _uint128_t *val, + _uint128_t *old, + _odp_memmodel_t mm) +{ + __atomic_exchange(&ptr->v, val, old, mm); +} + +/** + * Atomic compare and exchange (swap) of 16-byte atomic variable + * "Strong" semantics, will not fail spuriously. + * + * @param ptr Pointer to a 16-byte atomic variable + * @param exp Pointer to expected value (updated on failure) + * @param val Pointer to new value to write + * @param succ Memory model associated with a successful compare-and-swap + * operation + * @param fail Memory model associated with a failed compare-and-swap + * operation + * + * @retval 1 exchange successul + * @retval 0 exchange failed and '*exp' updated with current value + */ +static inline odp_bool_t _odp_atomic_u128_cmp_xchg_mm(_odp_atomic_u128_t *ptr, + _uint128_t *exp, + _uint128_t *val, + _odp_memmodel_t succ, + _odp_memmodel_t fail) +{ + return __atomic_compare_exchange(&ptr->v, exp, val, + false/*strong*/, succ, fail); +} +#endif + /** * @} */