From patchwork Mon Apr 28 11:41:02 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taras Kondratiuk X-Patchwork-Id: 29212 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-oa0-f71.google.com (mail-oa0-f71.google.com [209.85.219.71]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 94BD320553 for ; Mon, 28 Apr 2014 11:41:58 +0000 (UTC) Received: by mail-oa0-f71.google.com with SMTP id j17sf40256901oag.10 for ; Mon, 28 Apr 2014 04:41:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:cc:subject:precedence:list-id:list-unsubscribe :list-archive:list-post:list-help:list-subscribe:mime-version :errors-to:sender:x-original-sender :x-original-authentication-results:mailing-list:content-type :content-transfer-encoding; bh=eP2QtD18GxlmEi6oZDf+wmLc688czMZ+5kAiXvZSCVA=; b=j8eLWJdzjBDFRhyYNrGSN2ehZCL+9GZE7AP/SqEWZuHRDKKcm59nVJEoxhKZF5K3RE gOZFMqNygxYbJA1DYZbZy41O2o0rVOKDiSt4oJfVbr10qr/ScB3wKUKthrGWFIIeRMqT +GEDYAKDPhMX2SToFNakTCIkuCxhtmU2la5xoyLKdnCUUXbyqtAXlexMo5+VDmUoWSat IV/4NTJ6ZEYaC7AJFj+IfsdP3HyK6Kuah6n94j7iIvey2r6SDbi9D5oMMRfzBJhOkxiv qYa1REFzJESUAJ1t/0lHTJMNL1kOzTdfSy1AQ/TK8WwVCge/icNPQenIcWA5aFJ1mhd5 BNRg== X-Gm-Message-State: ALoCoQlIS2kOk8L8VbQLH5p/rT+9N36T8CkUO2ynCtvv5GPVmYUWBkWdrY6KqCUSYpARkQoEmzpJ X-Received: by 10.43.126.2 with SMTP id gu2mr11784322icc.0.1398685318105; Mon, 28 Apr 2014 04:41:58 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.50.197 with SMTP id s63ls2392158qga.39.gmail; Mon, 28 Apr 2014 04:41:58 -0700 (PDT) X-Received: by 10.58.38.166 with SMTP id h6mr22786085vek.22.1398685317943; Mon, 28 Apr 2014 04:41:57 -0700 (PDT) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by mx.google.com with ESMTPS id sl10si725866vdc.165.2014.04.28.04.41.57 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 28 Apr 2014 04:41:57 -0700 (PDT) Received-SPF: none (google.com: patch+caf_=patchwork-forward=linaro.org@linaro.org does not designate permitted sender hosts) client-ip=209.85.220.182; Received: by mail-vc0-f182.google.com with SMTP id lf12so5563571vcb.41 for ; Mon, 28 Apr 2014 04:41:57 -0700 (PDT) X-Received: by 10.58.49.10 with SMTP id q10mr22817393ven.5.1398685317859; Mon, 28 Apr 2014 04:41:57 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.221.72 with SMTP id ib8csp110789vcb; Mon, 28 Apr 2014 04:41:57 -0700 (PDT) X-Received: by 10.224.167.132 with SMTP id q4mr30210471qay.77.1398685317287; Mon, 28 Apr 2014 04:41:57 -0700 (PDT) Received: from ip-10-141-164-156.ec2.internal (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTPS id s10si7596204qak.129.2014.04.28.04.41.56 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 28 Apr 2014 04:41:57 -0700 (PDT) Received-SPF: none (google.com: lng-odp-bounces@lists.linaro.org does not designate permitted sender hosts) client-ip=54.225.227.206; Received: from localhost ([127.0.0.1] helo=ip-10-141-164-156.ec2.internal) by ip-10-141-164-156.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1Wejw8-0002nZ-8s; Mon, 28 Apr 2014 11:41:28 +0000 Received: from mail-lb0-f177.google.com ([209.85.217.177]) by ip-10-141-164-156.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1Wejvo-0002lL-Ag for lng-odp@lists.linaro.org; Mon, 28 Apr 2014 11:41:08 +0000 Received: by mail-lb0-f177.google.com with SMTP id z11so4795878lbi.8 for ; Mon, 28 Apr 2014 04:41:30 -0700 (PDT) X-Received: by 10.112.61.199 with SMTP id s7mr18076671lbr.25.1398685290432; Mon, 28 Apr 2014 04:41:30 -0700 (PDT) Received: from uglx0153363.itg.ti.com ([195.238.92.128]) by mx.google.com with ESMTPSA id x10sm9873857lad.1.2014.04.28.04.41.28 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 28 Apr 2014 04:41:29 -0700 (PDT) From: Taras Kondratiuk To: lng-odp@lists.linaro.org Date: Mon, 28 Apr 2014 14:41:02 +0300 Message-Id: <1398685276-4610-4-git-send-email-taras.kondratiuk@linaro.org> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1398685276-4610-1-git-send-email-taras.kondratiuk@linaro.org> References: <1398685276-4610-1-git-send-email-taras.kondratiuk@linaro.org> X-Topics: patch Cc: linaro-networking@linaro.org Subject: [lng-odp] [RFC PATCH 03/17] Split out platform-specific part of odp_atomic.h X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: lng-odp-bounces@lists.linaro.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: taras.kondratiuk@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: patch+caf_=patchwork-forward=linaro.org@linaro.org does not designate permitted sender hosts) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 --- include/odp_atomic.h | 205 ++-------- platform/linux-generic/include/plat/odp_atomic.h | 457 ++++++++++++++++++++++ 2 files changed, 492 insertions(+), 170 deletions(-) create mode 100644 platform/linux-generic/include/plat/odp_atomic.h diff --git a/include/odp_atomic.h b/include/odp_atomic.h index 763436a..439e804 100644 --- a/include/odp_atomic.h +++ b/include/odp_atomic.h @@ -20,22 +20,22 @@ extern "C" { #include - +#include /** * Atomic integer */ -typedef volatile int32_t odp_atomic_int_t; +typedef plat_odp_atomic_int_t odp_atomic_int_t; /** * Atomic unsigned integer 64 bits */ -typedef volatile uint64_t odp_atomic_u64_t; +typedef plat_odp_atomic_u64_t odp_atomic_u64_t; /** * Atomic unsigned integer 32 bits */ -typedef volatile uint32_t odp_atomic_u32_t; +typedef plat_odp_atomic_u32_t odp_atomic_u32_t; /** @@ -45,10 +45,7 @@ typedef volatile uint32_t odp_atomic_u32_t; * * @note The operation is not synchoronized with other threads */ -static inline void odp_atomic_init_int(odp_atomic_int_t *ptr) -{ - *ptr = 0; -} +void odp_atomic_init_int(odp_atomic_int_t *ptr); /** * Load value of atomic integer @@ -59,10 +56,7 @@ static inline void odp_atomic_init_int(odp_atomic_int_t *ptr) * * @note The operation is not synchoronized with other threads */ -static inline int odp_atomic_load_int(odp_atomic_int_t *ptr) -{ - return *ptr; -} +int odp_atomic_load_int(odp_atomic_int_t *ptr); /** * Store value to atomic integer @@ -72,10 +66,7 @@ static inline int odp_atomic_load_int(odp_atomic_int_t *ptr) * * @note The operation is not synchoronized with other threads */ -static inline void odp_atomic_store_int(odp_atomic_int_t *ptr, int new_value) -{ - *ptr = new_value; -} +void odp_atomic_store_int(odp_atomic_int_t *ptr, int new_value); /** * Fetch and add atomic integer @@ -85,10 +76,7 @@ static inline void odp_atomic_store_int(odp_atomic_int_t *ptr, int new_value) * * @return Value of the variable before the operation */ -static inline int odp_atomic_fetch_add_int(odp_atomic_int_t *ptr, int value) -{ - return __sync_fetch_and_add(ptr, value); -} +int odp_atomic_fetch_add_int(odp_atomic_int_t *ptr, int value); /** * Fetch and substract atomic integer @@ -98,10 +86,7 @@ static inline int odp_atomic_fetch_add_int(odp_atomic_int_t *ptr, int value) * * @return Value of the variable before the operation */ -static inline int odp_atomic_fetch_sub_int(odp_atomic_int_t *ptr, int value) -{ - return __sync_fetch_and_sub(ptr, value); -} +int odp_atomic_fetch_sub_int(odp_atomic_int_t *ptr, int value); /** * Fetch and increment atomic integer by 1 @@ -110,10 +95,7 @@ static inline int odp_atomic_fetch_sub_int(odp_atomic_int_t *ptr, int value) * * @return Value of the variable before the operation */ -static inline int odp_atomic_fetch_inc_int(odp_atomic_int_t *ptr) -{ - return odp_atomic_fetch_add_int(ptr, 1); -} +int odp_atomic_fetch_inc_int(odp_atomic_int_t *ptr); /** * Increment atomic integer by 1 @@ -121,10 +103,7 @@ static inline int odp_atomic_fetch_inc_int(odp_atomic_int_t *ptr) * @param ptr An atomic variable * */ -static inline void odp_atomic_inc_int(odp_atomic_int_t *ptr) -{ - odp_atomic_fetch_add_int(ptr, 1); -} +void odp_atomic_inc_int(odp_atomic_int_t *ptr); /** * Fetch and decrement atomic integer by 1 @@ -133,10 +112,7 @@ static inline void odp_atomic_inc_int(odp_atomic_int_t *ptr) * * @return Value of the variable before the operation */ -static inline int odp_atomic_fetch_dec_int(odp_atomic_int_t *ptr) -{ - return odp_atomic_fetch_sub_int(ptr, 1); -} +int odp_atomic_fetch_dec_int(odp_atomic_int_t *ptr); /** * Decrement atomic integer by 1 @@ -144,10 +120,7 @@ static inline int odp_atomic_fetch_dec_int(odp_atomic_int_t *ptr) * @param ptr An atomic variable * */ -static inline void odp_atomic_dec_int(odp_atomic_int_t *ptr) -{ - odp_atomic_fetch_sub_int(ptr, 1); -} +void odp_atomic_dec_int(odp_atomic_int_t *ptr); /** * Initialize atomic uint32 @@ -156,10 +129,7 @@ static inline void odp_atomic_dec_int(odp_atomic_int_t *ptr) * * @note The operation is not synchoronized with other threads */ -static inline void odp_atomic_init_u32(odp_atomic_u32_t *ptr) -{ - *ptr = 0; -} +void odp_atomic_init_u32(odp_atomic_u32_t *ptr); /** * Load value of atomic uint32 @@ -170,10 +140,7 @@ static inline void odp_atomic_init_u32(odp_atomic_u32_t *ptr) * * @note The operation is not synchoronized with other threads */ -static inline uint32_t odp_atomic_load_u32(odp_atomic_u32_t *ptr) -{ - return *ptr; -} +uint32_t odp_atomic_load_u32(odp_atomic_u32_t *ptr); /** * Store value to atomic uint32 @@ -183,11 +150,7 @@ static inline uint32_t odp_atomic_load_u32(odp_atomic_u32_t *ptr) * * @note The operation is not synchoronized with other threads */ -static inline void odp_atomic_store_u32(odp_atomic_u32_t *ptr, - uint32_t new_value) -{ - *ptr = new_value; -} +void odp_atomic_store_u32(odp_atomic_u32_t *ptr, uint32_t new_value); /** * Fetch and add atomic uint32 @@ -197,11 +160,7 @@ static inline void odp_atomic_store_u32(odp_atomic_u32_t *ptr, * * @return Value of the variable before the operation */ -static inline uint32_t odp_atomic_fetch_add_u32(odp_atomic_u32_t *ptr, - uint32_t value) -{ - return __sync_fetch_and_add(ptr, value); -} +uint32_t odp_atomic_fetch_add_u32(odp_atomic_u32_t *ptr, uint32_t value); /** * Fetch and substract uint32 @@ -211,11 +170,7 @@ static inline uint32_t odp_atomic_fetch_add_u32(odp_atomic_u32_t *ptr, * * @return Value of the variable before the operation */ -static inline uint32_t odp_atomic_fetch_sub_u32(odp_atomic_u32_t *ptr, - uint32_t value) -{ - return __sync_fetch_and_sub(ptr, value); -} +uint32_t odp_atomic_fetch_sub_u32(odp_atomic_u32_t *ptr, uint32_t value); /** * Fetch and increment atomic uint32 by 1 @@ -224,26 +179,7 @@ static inline uint32_t odp_atomic_fetch_sub_u32(odp_atomic_u32_t *ptr, * * @return Value of the variable before the operation */ -#if defined __OCTEON__ - -static inline uint32_t odp_atomic_fetch_inc_u32(odp_atomic_u32_t *ptr) -{ - uint32_t ret; - - asm __volatile__ ("syncws"); - asm __volatile__ ("lai %0,(%2)" : "=r" (ret), "+m" (ptr) : "r" (ptr)); - - return ret; -} - -#else - -static inline uint32_t odp_atomic_fetch_inc_u32(odp_atomic_u32_t *ptr) -{ - return odp_atomic_fetch_add_u32(ptr, 1); -} - -#endif +uint32_t odp_atomic_fetch_inc_u32(odp_atomic_u32_t *ptr); /** * Increment atomic uint32 by 1 @@ -251,10 +187,7 @@ static inline uint32_t odp_atomic_fetch_inc_u32(odp_atomic_u32_t *ptr) * @param ptr An atomic variable * */ -static inline void odp_atomic_inc_u32(odp_atomic_u32_t *ptr) -{ - odp_atomic_fetch_add_u32(ptr, 1); -} +void odp_atomic_inc_u32(odp_atomic_u32_t *ptr); /** * Fetch and decrement uint32 by 1 @@ -263,10 +196,7 @@ static inline void odp_atomic_inc_u32(odp_atomic_u32_t *ptr) * * @return Value of the variable before the operation */ -static inline uint32_t odp_atomic_fetch_dec_u32(odp_atomic_u32_t *ptr) -{ - return odp_atomic_fetch_sub_u32(ptr, 1); -} +uint32_t odp_atomic_fetch_dec_u32(odp_atomic_u32_t *ptr); /** * Decrement atomic uint32 by 1 @@ -274,10 +204,7 @@ static inline uint32_t odp_atomic_fetch_dec_u32(odp_atomic_u32_t *ptr) * @param ptr An atomic variable * */ -static inline void odp_atomic_dec_u32(odp_atomic_u32_t *ptr) -{ - odp_atomic_fetch_sub_u32(ptr, 1); -} +void odp_atomic_dec_u32(odp_atomic_u32_t *ptr); /** * Atomic compare and set for 32bit @@ -287,11 +214,7 @@ static inline void odp_atomic_dec_u32(odp_atomic_u32_t *ptr) * @param src new value. * @return Non-zero on success; 0 on failure. */ -static inline int -odp_atomic_cmpset_u32(odp_atomic_u32_t *dst, uint32_t exp, uint32_t src) -{ - return __sync_bool_compare_and_swap(dst, exp, src); -} +int odp_atomic_cmpset_u32(odp_atomic_u32_t *dst, uint32_t exp, uint32_t src); /** * Initialize atomic uint64 @@ -300,10 +223,7 @@ odp_atomic_cmpset_u32(odp_atomic_u32_t *dst, uint32_t exp, uint32_t src) * * @note The operation is not synchoronized with other threads */ -static inline void odp_atomic_init_u64(odp_atomic_u64_t *ptr) -{ - *ptr = 0; -} +void odp_atomic_init_u64(odp_atomic_u64_t *ptr); /** * Load value of atomic uint64 @@ -314,10 +234,7 @@ static inline void odp_atomic_init_u64(odp_atomic_u64_t *ptr) * * @note The operation is not synchoronized with other threads */ -static inline uint64_t odp_atomic_load_u64(odp_atomic_u64_t *ptr) -{ - return *ptr; -} +uint64_t odp_atomic_load_u64(odp_atomic_u64_t *ptr); /** * Store value to atomic uint64 @@ -327,11 +244,7 @@ static inline uint64_t odp_atomic_load_u64(odp_atomic_u64_t *ptr) * * @note The operation is not synchoronized with other threads */ -static inline void odp_atomic_store_u64(odp_atomic_u64_t *ptr, - uint64_t new_value) -{ - *ptr = new_value; -} +void odp_atomic_store_u64(odp_atomic_u64_t *ptr, uint64_t new_value); /** * Add atomic uint64 @@ -340,10 +253,7 @@ static inline void odp_atomic_store_u64(odp_atomic_u64_t *ptr, * @param value A value to be added to the variable * */ -static inline void odp_atomic_add_u64(odp_atomic_u64_t *ptr, uint64_t value) -{ - __sync_fetch_and_add(ptr, value); -} +void odp_atomic_add_u64(odp_atomic_u64_t *ptr, uint64_t value); /** * Fetch and add atomic uint64 @@ -354,20 +264,7 @@ static inline void odp_atomic_add_u64(odp_atomic_u64_t *ptr, uint64_t value) * @return Value of the variable before the operation */ -#if defined __powerpc__ && !defined __powerpc64__ -static inline uint64_t odp_atomic_fetch_add_u64(odp_atomic_u64_t *ptr, - uint64_t value) -{ - return __sync_fetch_and_add((odp_atomic_u32_t *)ptr, - (uint32_t)value); -} -#else -static inline uint64_t odp_atomic_fetch_add_u64(odp_atomic_u64_t *ptr, - uint64_t value) -{ - return __sync_fetch_and_add(ptr, value); -} -#endif +uint64_t odp_atomic_fetch_add_u64(odp_atomic_u64_t *ptr, uint64_t value); /** * Subtract atomic uint64 * @@ -375,10 +272,7 @@ static inline uint64_t odp_atomic_fetch_add_u64(odp_atomic_u64_t *ptr, * @param value A value to be subtracted from the variable * */ -static inline void odp_atomic_sub_u64(odp_atomic_u64_t *ptr, uint64_t value) -{ - __sync_fetch_and_sub(ptr, value); -} +void odp_atomic_sub_u64(odp_atomic_u64_t *ptr, uint64_t value); /** * Fetch and subtract atomic uint64 @@ -388,20 +282,7 @@ static inline void odp_atomic_sub_u64(odp_atomic_u64_t *ptr, uint64_t value) * * @return Value of the variable before the operation */ -#if defined __powerpc__ && !defined __powerpc64__ -static inline uint64_t odp_atomic_fetch_sub_u64(odp_atomic_u64_t *ptr, - uint64_t value) -{ - return __sync_fetch_and_sub((odp_atomic_u32_t *)ptr, - (uint32_t)value); -} -#else -static inline uint64_t odp_atomic_fetch_sub_u64(odp_atomic_u64_t *ptr, - uint64_t value) -{ - return __sync_fetch_and_sub(ptr, value); -} -#endif +uint64_t odp_atomic_fetch_sub_u64(odp_atomic_u64_t *ptr, uint64_t value); /** * Fetch and increment atomic uint64 by 1 * @@ -409,10 +290,7 @@ static inline uint64_t odp_atomic_fetch_sub_u64(odp_atomic_u64_t *ptr, * * @return Value of the variable before the operation */ -static inline uint64_t odp_atomic_fetch_inc_u64(odp_atomic_u64_t *ptr) -{ - return odp_atomic_fetch_add_u64(ptr, 1); -} +uint64_t odp_atomic_fetch_inc_u64(odp_atomic_u64_t *ptr); /** * Increment atomic uint64 by 1 @@ -420,10 +298,7 @@ static inline uint64_t odp_atomic_fetch_inc_u64(odp_atomic_u64_t *ptr) * @param ptr An atomic variable * */ -static inline void odp_atomic_inc_u64(odp_atomic_u64_t *ptr) -{ - odp_atomic_fetch_add_u64(ptr, 1); -} +void odp_atomic_inc_u64(odp_atomic_u64_t *ptr); /** * Fetch and decement atomic uint64 by 1 @@ -432,10 +307,7 @@ static inline void odp_atomic_inc_u64(odp_atomic_u64_t *ptr) * * @return Value of the variable before the operation */ -static inline uint64_t odp_atomic_fetch_dec_u64(odp_atomic_u64_t *ptr) -{ - return odp_atomic_fetch_sub_u64(ptr, 1); -} +uint64_t odp_atomic_fetch_dec_u64(odp_atomic_u64_t *ptr); /** * Deccrement atomic uint64 by 1 @@ -443,10 +315,7 @@ static inline uint64_t odp_atomic_fetch_dec_u64(odp_atomic_u64_t *ptr) * @param ptr An atomic variable * */ -static inline void odp_atomic_dec_u64(odp_atomic_u64_t *ptr) -{ - odp_atomic_fetch_sub_u64(ptr, 1); -} +void odp_atomic_dec_u64(odp_atomic_u64_t *ptr); /** * Atomic compare and set for 64bit @@ -456,11 +325,7 @@ static inline void odp_atomic_dec_u64(odp_atomic_u64_t *ptr) * @param src new value. * @return Non-zero on success; 0 on failure. */ -static inline int -odp_atomic_cmpset_u64(odp_atomic_u64_t *dst, uint64_t exp, uint64_t src) -{ - return __sync_bool_compare_and_swap(dst, exp, src); -} +int odp_atomic_cmpset_u64(odp_atomic_u64_t *dst, uint64_t exp, uint64_t src); #ifdef __cplusplus } diff --git a/platform/linux-generic/include/plat/odp_atomic.h b/platform/linux-generic/include/plat/odp_atomic.h new file mode 100644 index 0000000..52a672a --- /dev/null +++ b/platform/linux-generic/include/plat/odp_atomic.h @@ -0,0 +1,457 @@ +/* Copyright (c) 2013, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + + +/** + * @file + * + * ODP atomic operations + */ + +#ifndef ODP_ATOMIC_H_ +#error This file should be included only into corresponding top level header +#else + +/** + * Atomic integer + */ +typedef volatile int32_t plat_odp_atomic_int_t; + +/** + * Atomic unsigned integer 64 bits + */ +typedef volatile uint64_t plat_odp_atomic_u64_t; + +/** + * Atomic unsigned integer 32 bits + */ +typedef volatile uint32_t plat_odp_atomic_u32_t; + +/* Define OPD types here to use them in inline functions below */ +typedef plat_odp_atomic_int_t odp_atomic_int_t; +typedef plat_odp_atomic_u64_t odp_atomic_u64_t; +typedef plat_odp_atomic_u32_t odp_atomic_u32_t; + + +static inline void odp_atomic_init_int(odp_atomic_int_t *ptr) +{ + *ptr = 0; +} + +/** + * Load value of atomic integer + * + * @param ptr An atomic variable + * + * @return atomic integer value + * + * @note The operation is not synchoronized with other threads + */ +static inline int odp_atomic_load_int(odp_atomic_int_t *ptr) +{ + return *ptr; +} + +/** + * Store value to atomic integer + * + * @param ptr An atomic variable + * @param new_value Store new_value to a variable + * + * @note The operation is not synchoronized with other threads + */ +static inline void odp_atomic_store_int(odp_atomic_int_t *ptr, int new_value) +{ + *ptr = new_value; +} + +/** + * Fetch and add atomic integer + * + * @param ptr An atomic variable + * @param value A value to be added to the variable + * + * @return Value of the variable before the operation + */ +static inline int odp_atomic_fetch_add_int(odp_atomic_int_t *ptr, int value) +{ + return __sync_fetch_and_add(ptr, value); +} + +/** + * Fetch and substract atomic integer + * + * @param ptr An atomic int variable + * @param value A value to be subtracted from the variable + * + * @return Value of the variable before the operation + */ +static inline int odp_atomic_fetch_sub_int(odp_atomic_int_t *ptr, int value) +{ + return __sync_fetch_and_sub(ptr, value); +} + +/** + * Fetch and increment atomic integer by 1 + * + * @param ptr An atomic variable + * + * @return Value of the variable before the operation + */ +static inline int odp_atomic_fetch_inc_int(odp_atomic_int_t *ptr) +{ + return odp_atomic_fetch_add_int(ptr, 1); +} + +/** + * Increment atomic integer by 1 + * + * @param ptr An atomic variable + * + */ +static inline void odp_atomic_inc_int(odp_atomic_int_t *ptr) +{ + odp_atomic_fetch_add_int(ptr, 1); +} + +/** + * Fetch and decrement atomic integer by 1 + * + * @param ptr An atomic int variable + * + * @return Value of the variable before the operation + */ +static inline int odp_atomic_fetch_dec_int(odp_atomic_int_t *ptr) +{ + return odp_atomic_fetch_sub_int(ptr, 1); +} + +/** + * Decrement atomic integer by 1 + * + * @param ptr An atomic variable + * + */ +static inline void odp_atomic_dec_int(odp_atomic_int_t *ptr) +{ + odp_atomic_fetch_sub_int(ptr, 1); +} + +/** + * Initialize atomic uint32 + * + * @param ptr An atomic variable + * + * @note The operation is not synchoronized with other threads + */ +static inline void odp_atomic_init_u32(odp_atomic_u32_t *ptr) +{ + *ptr = 0; +} + +/** + * Load value of atomic uint32 + * + * @param ptr An atomic variable + * + * @return atomic uint32 value + * + * @note The operation is not synchoronized with other threads + */ +static inline uint32_t odp_atomic_load_u32(odp_atomic_u32_t *ptr) +{ + return *ptr; +} + +/** + * Store value to atomic uint32 + * + * @param ptr An atomic variable + * @param new_value Store new_value to a variable + * + * @note The operation is not synchoronized with other threads + */ +static inline void odp_atomic_store_u32(odp_atomic_u32_t *ptr, + uint32_t new_value) +{ + *ptr = new_value; +} + +/** + * Fetch and add atomic uint32 + * + * @param ptr An atomic variable + * @param value A value to be added to the variable + * + * @return Value of the variable before the operation + */ +static inline uint32_t odp_atomic_fetch_add_u32(odp_atomic_u32_t *ptr, + uint32_t value) +{ + return __sync_fetch_and_add(ptr, value); +} + +/** + * Fetch and substract uint32 + * + * @param ptr An atomic variable + * @param value A value to be sub to the variable + * + * @return Value of the variable before the operation + */ +static inline uint32_t odp_atomic_fetch_sub_u32(odp_atomic_u32_t *ptr, + uint32_t value) +{ + return __sync_fetch_and_sub(ptr, value); +} + +/** + * Fetch and increment atomic uint32 by 1 + * + * @param ptr An atomic variable + * + * @return Value of the variable before the operation + */ +#if defined __OCTEON__ + +static inline uint32_t odp_atomic_fetch_inc_u32(odp_atomic_u32_t *ptr) +{ + uint32_t ret; + + asm __volatile__ ("syncws"); + asm __volatile__ ("lai %0,(%2)" : "=r" (ret), "+m" (ptr) : "r" (ptr)); + + return ret; +} + +#else + +static inline uint32_t odp_atomic_fetch_inc_u32(odp_atomic_u32_t *ptr) +{ + return odp_atomic_fetch_add_u32(ptr, 1); +} + +#endif + +/** + * Increment atomic uint32 by 1 + * + * @param ptr An atomic variable + * + */ +static inline void odp_atomic_inc_u32(odp_atomic_u32_t *ptr) +{ + odp_atomic_fetch_add_u32(ptr, 1); +} + +/** + * Fetch and decrement uint32 by 1 + * + * @param ptr An atomic variable + * + * @return Value of the variable before the operation + */ +static inline uint32_t odp_atomic_fetch_dec_u32(odp_atomic_u32_t *ptr) +{ + return odp_atomic_fetch_sub_u32(ptr, 1); +} + +/** + * Decrement atomic uint32 by 1 + * + * @param ptr An atomic variable + * + */ +static inline void odp_atomic_dec_u32(odp_atomic_u32_t *ptr) +{ + odp_atomic_fetch_sub_u32(ptr, 1); +} + +/** + * Atomic compare and set for 32bit + * + * @param dst destination location into which the value will be written. + * @param exp expected value. + * @param src new value. + * @return Non-zero on success; 0 on failure. + */ +static inline int +odp_atomic_cmpset_u32(odp_atomic_u32_t *dst, uint32_t exp, uint32_t src) +{ + return __sync_bool_compare_and_swap(dst, exp, src); +} + +/** + * Initialize atomic uint64 + * + * @param ptr An atomic variable + * + * @note The operation is not synchoronized with other threads + */ +static inline void odp_atomic_init_u64(odp_atomic_u64_t *ptr) +{ + *ptr = 0; +} + +/** + * Load value of atomic uint64 + * + * @param ptr An atomic variable + * + * @return atomic uint64 value + * + * @note The operation is not synchoronized with other threads + */ +static inline uint64_t odp_atomic_load_u64(odp_atomic_u64_t *ptr) +{ + return *ptr; +} + +/** + * Store value to atomic uint64 + * + * @param ptr An atomic variable + * @param new_value Store new_value to a variable + * + * @note The operation is not synchoronized with other threads + */ +static inline void odp_atomic_store_u64(odp_atomic_u64_t *ptr, + uint64_t new_value) +{ + *ptr = new_value; +} + +/** + * Add atomic uint64 + * + * @param ptr An atomic variable + * @param value A value to be added to the variable + * + */ +static inline void odp_atomic_add_u64(odp_atomic_u64_t *ptr, uint64_t value) +{ + __sync_fetch_and_add(ptr, value); +} + +/** + * Fetch and add atomic uint64 + * + * @param ptr An atomic variable + * @param value A value to be added to the variable + * + * @return Value of the variable before the operation + */ + +#if defined __powerpc__ && !defined __powerpc64__ +static inline uint64_t odp_atomic_fetch_add_u64(odp_atomic_u64_t *ptr, + uint64_t value) +{ + return __sync_fetch_and_add((odp_atomic_u32_t *)ptr, + (uint32_t)value); +} +#else +static inline uint64_t odp_atomic_fetch_add_u64(odp_atomic_u64_t *ptr, + uint64_t value) +{ + return __sync_fetch_and_add(ptr, value); +} +#endif +/** + * Subtract atomic uint64 + * + * @param ptr An atomic variable + * @param value A value to be subtracted from the variable + * + */ +static inline void odp_atomic_sub_u64(odp_atomic_u64_t *ptr, uint64_t value) +{ + __sync_fetch_and_sub(ptr, value); +} + +/** + * Fetch and subtract atomic uint64 + * + * @param ptr An atomic variable + * @param value A value to be subtracted from the variable + * + * @return Value of the variable before the operation + */ +#if defined __powerpc__ && !defined __powerpc64__ +static inline uint64_t odp_atomic_fetch_sub_u64(odp_atomic_u64_t *ptr, + uint64_t value) +{ + return __sync_fetch_and_sub((odp_atomic_u32_t *)ptr, + (uint32_t)value); +} +#else +static inline uint64_t odp_atomic_fetch_sub_u64(odp_atomic_u64_t *ptr, + uint64_t value) +{ + return __sync_fetch_and_sub(ptr, value); +} +#endif +/** + * Fetch and increment atomic uint64 by 1 + * + * @param ptr An atomic variable + * + * @return Value of the variable before the operation + */ +static inline uint64_t odp_atomic_fetch_inc_u64(odp_atomic_u64_t *ptr) +{ + return odp_atomic_fetch_add_u64(ptr, 1); +} + +/** + * Increment atomic uint64 by 1 + * + * @param ptr An atomic variable + * + */ +static inline void odp_atomic_inc_u64(odp_atomic_u64_t *ptr) +{ + odp_atomic_fetch_add_u64(ptr, 1); +} + +/** + * Fetch and decement atomic uint64 by 1 + * + * @param ptr An atomic variable + * + * @return Value of the variable before the operation + */ +static inline uint64_t odp_atomic_fetch_dec_u64(odp_atomic_u64_t *ptr) +{ + return odp_atomic_fetch_sub_u64(ptr, 1); +} + +/** + * Deccrement atomic uint64 by 1 + * + * @param ptr An atomic variable + * + */ +static inline void odp_atomic_dec_u64(odp_atomic_u64_t *ptr) +{ + odp_atomic_fetch_sub_u64(ptr, 1); +} + +/** + * Atomic compare and set for 64bit + * + * @param dst destination location into which the value will be written. + * @param exp expected value. + * @param src new value. + * @return Non-zero on success; 0 on failure. + */ +static inline int +odp_atomic_cmpset_u64(odp_atomic_u64_t *dst, uint64_t exp, uint64_t src) +{ + return __sync_bool_compare_and_swap(dst, exp, src); +} + +#endif +