From patchwork Fri Feb 22 07:04:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 158984 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp1395144jaa; Thu, 21 Feb 2019 23:05:06 -0800 (PST) X-Google-Smtp-Source: AHgI3IbnI43Wz2dgNWcvq7sOUooEBer6mRSH4zKoFoEqyq6oZ5J8RI0QYJ4YoEKDmRjgDbLLdx0o X-Received: by 2002:a05:6402:1495:: with SMTP id e21mr1994914edv.52.1550819106494; Thu, 21 Feb 2019 23:05:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550819106; cv=none; d=google.com; s=arc-20160816; b=LMsYNVHSLY8jOJxRfw4VKuAmLCHZL/Jq17k39cV/xkVZ85Nl0xYWOqIolv6W2uFWPY 3m6n8+YhTG71tDS+uif6RZW+sjgE5W/lzcm4cTuFKoEMHOs3zCTB7+NEeQeIOtw9NRoB x/ppgZF8sBhw20iT1Z7ecyXRz1rfAs5sMQdfRl47Dwr+8KcFOCTLlGYnqwnhCLQMwICi TDdKsaQtyDAt/yr4cX4cTr/CeauRJxFdPtZIb2tYKAHpGN7q1dPL2YxsagSS6KRAlCln sRnuc3KjnP8i6R7J9q6bCuzR0k5FkSbiBkFO+bRkokYCoTqO0gFrRKvuAq0WlVjOC9zS JpnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=AP5H+yt6NOoaufgJxYs57ZZwska9TvGeWsijmLbZ6m8=; b=XHvkohAbnUgndwvUXrsSEAoQfx0c9aREVqwZAjBpd3rhMVP9Wo92cXNOO39AuPgkp0 dSvSuZdwT2NSELwDYClkdFSRJmQBP/ztkHiwwhtKYZdEwRoX+v7ScQcPkYSlzRdthucr KvXKGBYrI9QQMkLTrtSboSk6A4ka+yNq3Cw37vOf4XJuAc/kIV4jPhv+YF83GBOhLk9E Wab6yHYKK60OsDN+ehNOD1ln1POVRSb4vNT7Akvc1AcuYU153vw8AOfP6lqeD4KZt4fP mbEl0xdbqNd/TBa6t3Z9nq5x1a/fJvfmsSnL7hHRKeRhCvqFZv3z87PBnwKm2Hi6pV+H tY3w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id i63si320795edc.199.2019.02.21.23.05.06; Thu, 21 Feb 2019 23:05:06 -0800 (PST) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1BB402B9C; Fri, 22 Feb 2019 08:05:06 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id E977D1D7 for ; Fri, 22 Feb 2019 08:05:03 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2787D80D; Thu, 21 Feb 2019 23:05:03 -0800 (PST) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.12.104]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A975D3F690; Thu, 21 Feb 2019 23:05:02 -0800 (PST) From: Honnappa Nagarahalli To: konstantin.ananyev@intel.com, stephen@networkplumber.org, paulmck@linux.ibm.com, dev@dpdk.org, honnappa.nagarahalli@arm.com Cc: gavin.hu@arm.com, dharmik.thakkar@arm.com, malvika.gupta@arm.com, nd@arm.com Date: Fri, 22 Feb 2019 01:04:23 -0600 Message-Id: <20190222070427.22866-2-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190222070427.22866-1-honnappa.nagarahalli@arm.com> References: <20181222021420.5114-1-honnappa.nagarahalli@arm.com> <20190222070427.22866-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [RFC v3 1/5] rcu: add RCU library supporting QSBR mechanism X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add RCU library supporting quiescent state based memory reclamation method. This library helps identify the quiescent state of the reader threads so that the writers can free the memory associated with the lock less data structures. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Steve Capper Reviewed-by: Gavin Hu --- config/common_base | 6 + lib/Makefile | 2 + lib/librte_rcu/Makefile | 23 ++ lib/librte_rcu/meson.build | 5 + lib/librte_rcu/rte_rcu_qsbr.c | 63 ++++++ lib/librte_rcu/rte_rcu_qsbr.h | 330 +++++++++++++++++++++++++++++ lib/librte_rcu/rte_rcu_version.map | 8 + lib/meson.build | 2 +- mk/rte.app.mk | 1 + 9 files changed, 439 insertions(+), 1 deletion(-) create mode 100644 lib/librte_rcu/Makefile create mode 100644 lib/librte_rcu/meson.build create mode 100644 lib/librte_rcu/rte_rcu_qsbr.c create mode 100644 lib/librte_rcu/rte_rcu_qsbr.h create mode 100644 lib/librte_rcu/rte_rcu_version.map -- 2.17.1 diff --git a/config/common_base b/config/common_base index 7c6da5165..af550e96a 100644 --- a/config/common_base +++ b/config/common_base @@ -805,6 +805,12 @@ CONFIG_RTE_LIBRTE_LATENCY_STATS=y # CONFIG_RTE_LIBRTE_TELEMETRY=n +# +# Compile librte_rcu +# +CONFIG_RTE_LIBRTE_RCU=y +CONFIG_RTE_LIBRTE_RCU_DEBUG=n + # # Compile librte_lpm # diff --git a/lib/Makefile b/lib/Makefile index d6239d27c..15b67e210 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -111,6 +111,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_IPSEC) += librte_ipsec DEPDIRS-librte_ipsec := librte_eal librte_mbuf librte_cryptodev librte_security DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry DEPDIRS-librte_telemetry := librte_eal librte_metrics librte_ethdev +DIRS-$(CONFIG_RTE_LIBRTE_RCU) += librte_rcu +DEPDIRS-librte_rcu := librte_eal ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y) DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni diff --git a/lib/librte_rcu/Makefile b/lib/librte_rcu/Makefile new file mode 100644 index 000000000..6aa677bd1 --- /dev/null +++ b/lib/librte_rcu/Makefile @@ -0,0 +1,23 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Arm Limited + +include $(RTE_SDK)/mk/rte.vars.mk + +# library name +LIB = librte_rcu.a + +CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 +LDLIBS += -lrte_eal + +EXPORT_MAP := rte_rcu_version.map + +LIBABIVER := 1 + +# all source are stored in SRCS-y +SRCS-$(CONFIG_RTE_LIBRTE_RCU) := rte_rcu_qsbr.c + +# install includes +SYMLINK-$(CONFIG_RTE_LIBRTE_RCU)-include := rte_rcu_qsbr.h + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_rcu/meson.build b/lib/librte_rcu/meson.build new file mode 100644 index 000000000..c009ae4b7 --- /dev/null +++ b/lib/librte_rcu/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Arm Limited + +sources = files('rte_rcu_qsbr.c') +headers = files('rte_rcu_qsbr.h') diff --git a/lib/librte_rcu/rte_rcu_qsbr.c b/lib/librte_rcu/rte_rcu_qsbr.c new file mode 100644 index 000000000..3c2577ee2 --- /dev/null +++ b/lib/librte_rcu/rte_rcu_qsbr.c @@ -0,0 +1,63 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * + * Copyright (c) 2018 Arm Limited + */ + +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "rte_rcu_qsbr.h" + +/* Initialize a quiescent state variable */ +void __rte_experimental +rte_rcu_qsbr_init(struct rte_rcu_qsbr *v) +{ + memset(v, 0, sizeof(struct rte_rcu_qsbr)); +} + +/* Dump the details of a single quiescent state variable to a file. */ +void __rte_experimental +rte_rcu_qsbr_dump(FILE *f, struct rte_rcu_qsbr *v) +{ + uint64_t bmap; + uint32_t i, t; + + RTE_ASSERT(v == NULL || f == NULL); + + fprintf(f, "\nQuiescent State Variable @%p\n", v); + + fprintf(f, " Registered thread ID mask = 0x"); + for (i = 0; i < RTE_QSBR_BIT_MAP_ELEMS; i++) + fprintf(f, "%lx", __atomic_load_n(&v->reg_thread_id[i], + __ATOMIC_ACQUIRE)); + fprintf(f, "\n"); + + fprintf(f, " Token = %lu\n", + __atomic_load_n(&v->token, __ATOMIC_ACQUIRE)); + + fprintf(f, "Quiescent State Counts for readers:\n"); + for (i = 0; i < RTE_QSBR_BIT_MAP_ELEMS; i++) { + bmap = __atomic_load_n(&v->reg_thread_id[i], + __ATOMIC_ACQUIRE); + while (bmap) { + t = __builtin_ctzl(bmap); + fprintf(f, "thread ID = %d, count = %lu\n", t, + __atomic_load_n(&v->w[i].cnt, + __ATOMIC_RELAXED)); + bmap &= ~(1UL << t); + } + } +} diff --git a/lib/librte_rcu/rte_rcu_qsbr.h b/lib/librte_rcu/rte_rcu_qsbr.h new file mode 100644 index 000000000..53e00488b --- /dev/null +++ b/lib/librte_rcu/rte_rcu_qsbr.h @@ -0,0 +1,330 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2018 Arm Limited + */ + +#ifndef _RTE_RCU_QSBR_H_ +#define _RTE_RCU_QSBR_H_ + +/** + * @file + * RTE Quiescent State Based Reclamation (QSBR) + * + * Quiescent State (QS) is any point in the thread execution + * where the thread does not hold a reference to shared memory. + * A critical section for a data structure can be a quiescent state for + * another data structure. + * + * This library provides the ability to identify quiescent state. + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include +#include +#include +#include +#include + +/**< Maximum number of reader threads supported. */ +#define RTE_RCU_MAX_THREADS 128 + +#if !RTE_IS_POWER_OF_2(RTE_RCU_MAX_THREADS) +#error RTE_RCU_MAX_THREADS must be a power of 2 +#endif + +/**< Number of array elements required for the bit-map */ +#define RTE_QSBR_BIT_MAP_ELEMS (RTE_RCU_MAX_THREADS/(sizeof(uint64_t) * 8)) + +/* Thread IDs are stored as a bitmap of 64b element array. Given thread id + * needs to be converted to index into the array and the id within + * the array element. + */ +#define RTE_QSBR_THR_INDEX_SHIFT 6 +#define RTE_QSBR_THR_ID_MASK 0x3f + +/* Worker thread counter */ +struct rte_rcu_qsbr_cnt { + uint64_t cnt; /**< Quiescent state counter. */ +} __rte_cache_aligned; + +/** + * RTE thread Quiescent State structure. + */ +struct rte_rcu_qsbr { + uint64_t reg_thread_id[RTE_QSBR_BIT_MAP_ELEMS] __rte_cache_aligned; + /**< Registered reader thread IDs - reader threads reporting + * on this QS variable represented in a bit map. + */ + + uint64_t token __rte_cache_aligned; + /**< Counter to allow for multiple simultaneous QS queries */ + + struct rte_rcu_qsbr_cnt w[RTE_RCU_MAX_THREADS] __rte_cache_aligned; + /**< QS counter for each reader thread, counts upto + * current value of token. + */ +} __rte_cache_aligned; + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Initialize a Quiescent State (QS) variable. + * + * @param v + * QS variable + * + */ +void __rte_experimental +rte_rcu_qsbr_init(struct rte_rcu_qsbr *v); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Add a reader thread, to the list of threads reporting their quiescent + * state on a QS variable. + * + * This is implemented as a lock-free function. It is multi-thread + * safe. + * Any reader thread that wants to report its quiescent state must + * call this API before calling rte_rcu_qsbr_update. This can be called + * during initialization or as part of the packet processing loop. + * Any ongoing QS queries may wait for the status from this registered + * thread. + * + * @param v + * QS variable + * @param thread_id + * Reader thread with this thread ID will report its quiescent state on + * the QS variable. + */ +static __rte_always_inline void __rte_experimental +rte_rcu_qsbr_register_thread(struct rte_rcu_qsbr *v, unsigned int thread_id) +{ + unsigned int i, id; + + RTE_ASSERT(v == NULL || thread_id >= RTE_RCU_MAX_THREADS); + + id = thread_id & RTE_QSBR_THR_ID_MASK; + i = thread_id >> RTE_QSBR_THR_INDEX_SHIFT; + + /* Worker thread has to count the quiescent states + * only from the current value of token. + * __atomic_store_n(cnt, __ATOMIC_RELAXED) is used to ensure + * 'cnt' (64b) is accessed atomically. + */ + __atomic_store_n(&v->w[thread_id].cnt, + __atomic_load_n(&v->token, __ATOMIC_ACQUIRE), + __ATOMIC_RELAXED); + + /* Release the store to initial TQS count so that readers + * can use it immediately after this function returns. + */ + __atomic_fetch_or(&v->reg_thread_id[i], 1UL << id, __ATOMIC_RELEASE); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Remove a reader thread, from the list of threads reporting their + * quiescent state on a QS variable. + * + * This is implemented as a lock-free function. It is multi-thread safe. + * This API can be called from the reader threads during shutdown. + * Ongoing QS queries will stop waiting for the status from this + * unregistered reader thread. + * + * @param v + * QS variable + * @param thread_id + * Reader thread with this thread ID will stop reporting its quiescent + * state on the QS variable. + */ +static __rte_always_inline void __rte_experimental +rte_rcu_qsbr_unregister_thread(struct rte_rcu_qsbr *v, unsigned int thread_id) +{ + unsigned int i, id; + + RTE_ASSERT(v == NULL || thread_id >= RTE_RCU_MAX_THREADS); + + id = thread_id & RTE_QSBR_THR_ID_MASK; + i = thread_id >> RTE_QSBR_THR_INDEX_SHIFT; + + /* Make sure the removal of the thread from the list of + * reporting threads is visible before the thread + * does anything else. + */ + __atomic_fetch_and(&v->reg_thread_id[i], + ~(1UL << id), __ATOMIC_RELEASE); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Trigger the reader threads to report the quiescent state + * status. + * + * This is implemented as a lock-free function. It is multi-thread + * safe and can be called from worker threads. + * + * @param v + * TQS variable + * @param n + * Expected number of times the quiescent state is entered + * @param t + * - If successful, this is the token for this call of the API. + * This should be passed to rte_rcu_qsbr_check API. + */ +static __rte_always_inline void __rte_experimental +rte_rcu_qsbr_start(struct rte_rcu_qsbr *v, unsigned int n, uint64_t *t) +{ + RTE_ASSERT(v == NULL || t == NULL); + + /* This store release will ensure that changes to any data + * structure are visible to the workers before the token + * update is visible. + */ + *t = __atomic_add_fetch(&v->token, n, __ATOMIC_RELEASE); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Update quiescent state for a reader thread. + * + * This is implemented as a lock-free function. It is multi-thread safe. + * All the reader threads registered to report their quiescent state + * on the QS variable must call this API. + * + * @param v + * QS variable + */ +static __rte_always_inline void __rte_experimental +rte_rcu_qsbr_update(struct rte_rcu_qsbr *v, unsigned int thread_id) +{ + uint64_t t; + + RTE_ASSERT(v == NULL || thread_id >= RTE_RCU_MAX_THREADS); + + /* Load the token before the reader thread loads any other + * (lock-free) data structure. This ensures that updates + * to the data structures are visible if the update + * to token is visible. + */ + t = __atomic_load_n(&v->token, __ATOMIC_ACQUIRE); + + /* Relaxed load/store on the counter is enough as we are + * reporting an already completed quiescent state. + * __atomic_load_n(cnt, __ATOMIC_RELAXED) is used as 'cnt' (64b) + * is accessed atomically. + * Copy the current token value. This will end grace period + * of multiple concurrent writers. + */ + if (__atomic_load_n(&v->w[thread_id].cnt, __ATOMIC_RELAXED) != t) + __atomic_store_n(&v->w[thread_id].cnt, t, __ATOMIC_RELAXED); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Checks if all the reader threads have entered the quiescent state + * 'n' number of times. 'n' is provided in rte_rcu_qsbr_start API. + * + * This is implemented as a lock-free function. It is multi-thread + * safe and can be called from the worker threads as well. + * + * @param v + * QS variable + * @param t + * Token returned by rte_rcu_qsbr_start API + * @param wait + * If true, block till all the reader threads have completed entering + * the quiescent state 'n' number of times + * @return + * - 0 if all reader threads have NOT passed through specified number + * of quiescent states. + * - 1 if all reader threads have passed through specified number + * of quiescent states. + */ +static __rte_always_inline int __rte_experimental +rte_rcu_qsbr_check(struct rte_rcu_qsbr *v, uint64_t t, bool wait) +{ + uint32_t i, j, id; + uint64_t bmap; + + RTE_ASSERT(v == NULL); + + i = 0; + do { + /* Load the current registered thread bit map before + * loading the reader thread quiescent state counters. + */ + bmap = __atomic_load_n(&v->reg_thread_id[i], __ATOMIC_ACQUIRE); + id = i << RTE_QSBR_THR_INDEX_SHIFT; + + while (bmap) { + j = __builtin_ctzl(bmap); + +/* printf ("Status check: token = %lu, wait = %d, Bit Map = 0x%x, Thread ID = %d\n", t, wait, bmap, id+j); */ + /* __atomic_load_n(cnt, __ATOMIC_RELAXED) + * is used to ensure 'cnt' (64b) is accessed + * atomically. + */ + if (unlikely(__atomic_load_n(&v->w[id + j].cnt, + __ATOMIC_RELAXED) < t)) { + +/* printf ("Status not in QS: token = %lu, Wait = %d, Thread QS cnt = %lu, Thread ID = %d\n", t, wait, RTE_QSBR_CNT_ARRAY_ELM(v, id + j)->cnt, id+j); */ + /* This thread is not in QS */ + if (!wait) + return 0; + + rte_pause(); + /* This thread might have unregistered. + * Re-read the bitmap. + */ + bmap = __atomic_load_n( + &v->reg_thread_id[i], + __ATOMIC_ACQUIRE); + + continue; + } + + bmap &= ~(1UL << j); + } + + i++; + } while (i < RTE_QSBR_BIT_MAP_ELEMS); + + return 1; +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Dump the details of a single QS variables to a file. + * + * It is NOT multi-thread safe. + * + * @param f + * A pointer to a file for output + * @param v + * QS variable + */ +void __rte_experimental +rte_rcu_qsbr_dump(FILE *f, struct rte_rcu_qsbr *v); + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_RCU_QSBR_H_ */ diff --git a/lib/librte_rcu/rte_rcu_version.map b/lib/librte_rcu/rte_rcu_version.map new file mode 100644 index 000000000..0df2071be --- /dev/null +++ b/lib/librte_rcu/rte_rcu_version.map @@ -0,0 +1,8 @@ +EXPERIMENTAL { + global: + + rte_rcu_qsbr_init; + rte_rcu_qsbr_dump; + + local: *; +}; diff --git a/lib/meson.build b/lib/meson.build index e8b40f546..a5ffc5dc6 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -21,7 +21,7 @@ libraries = [ 'compat', # just a header, used for versioning 'gro', 'gso', 'ip_frag', 'jobstats', 'kni', 'latencystats', 'lpm', 'member', 'power', 'pdump', 'rawdev', - 'reorder', 'sched', 'security', 'vhost', + 'reorder', 'sched', 'security', 'vhost', 'rcu', #ipsec lib depends on crypto and security 'ipsec', # add pkt framework libs which use other libs from above diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 8a4f0f4e5..a9944d2f5 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -96,6 +96,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrte_sched +_LDLIBS-$(CONFIG_RTE_LIBRTE_RCU) += -lrte_rcu ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_KNI) += -lrte_kni From patchwork Fri Feb 22 07:04:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 158985 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp1395254jaa; Thu, 21 Feb 2019 23:05:15 -0800 (PST) X-Google-Smtp-Source: AHgI3IZmUoZngVPFlrhkrJlFsxu3Q/kjk330deyGdaT3of4z5GuJwTemkvCl8EP43Zi02OwgkWD6 X-Received: by 2002:a17:906:1945:: with SMTP id b5mr1794053eje.156.1550819115876; Thu, 21 Feb 2019 23:05:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550819115; cv=none; d=google.com; s=arc-20160816; b=nKtU4SuCI+QRUotC0yhAjJoIx/7n0bJUopaWXQw7jIgt5321pyKkn1hC+Hy5cs8luJ nfjRdqXcl2FRvyLiIATFM5mAvOqkLQKL1o80R+Fc7P5dZfgulwOVwKYVyU0tWTGdSOEV 9+R6xM9iElQkp/hKTMYDtES0XMJPqWAW6/IISESEZQkRqvy/51IHq2WtsILnBUIdNZ+7 D2QbquQ/Hm5FCFOvPwICqsh+BVMRrMIvk6XN1NvqNIJuqXiO9WRTe/y+g3J6I7b2nWVJ sKiuXKBrHWRet8mfBSF8oUiPrdQkIRf/Dl0IXSYtgBYzRmafXO/s9XT7FPmoK7wsT+8d KLvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=R+TAH/otDOv57v7/YPeFnIgXjUgUyyTDfClJWX27O8o=; b=KijB17D7rCbutTTpeBv3IAJLRbvIxC/DP4HIHktebgat8x4931KFvk7GUrPZIkFrY4 OV7XoAM4DDnhg6LvGUEhAFP6k2s+CYjVlAU0KZ98o2GFn0Ro8SuieDto8Y4f9WUSL8uy O5l2wr0nPTiGwIoJmoxhNHxKaFPYRqswARBIl4MTU7eYR9mw6SBK7T/TGfx/W+n3PMew hDPESTJqjq1e2Dht2OrgW/xrYQCAGZ68QzK2o++AebTYhAX54+Xaly6Tcf9dZNEAxegM 9R0sJWUjd5PKit+RlknS+hyelCia5Bq1JAaOB7Bwojn+huSjUNRlpcEeb2JpxHeictax HQ8w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id 39si310292edq.222.2019.02.21.23.05.15; Thu, 21 Feb 2019 23:05:15 -0800 (PST) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 579172C38; Fri, 22 Feb 2019 08:05:09 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id 6C11B2C2B for ; Fri, 22 Feb 2019 08:05:07 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D6D0480D; Thu, 21 Feb 2019 23:05:06 -0800 (PST) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.12.104]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 537E33F690; Thu, 21 Feb 2019 23:05:06 -0800 (PST) From: Honnappa Nagarahalli To: konstantin.ananyev@intel.com, stephen@networkplumber.org, paulmck@linux.ibm.com, dev@dpdk.org, honnappa.nagarahalli@arm.com Cc: gavin.hu@arm.com, dharmik.thakkar@arm.com, malvika.gupta@arm.com, nd@arm.com Date: Fri, 22 Feb 2019 01:04:24 -0600 Message-Id: <20190222070427.22866-3-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190222070427.22866-1-honnappa.nagarahalli@arm.com> References: <20181222021420.5114-1-honnappa.nagarahalli@arm.com> <20190222070427.22866-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [RFC v3 2/5] test/rcu_qsbr: add API and functional tests X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Dharmik Thakkar Add API positive/negative test cases, functional tests and performance tests. Signed-off-by: Malvika Gupta Signed-off-by: Dharmik Thakkar Signed-off-by: Honnappa Nagarahalli Reviewed-by: Gavin Hu --- test/test/Makefile | 2 + test/test/autotest_data.py | 12 + test/test/meson.build | 7 +- test/test/test_rcu_qsbr.c | 831 +++++++++++++++++++++++++++++++++ test/test/test_rcu_qsbr_perf.c | 272 +++++++++++ 5 files changed, 1123 insertions(+), 1 deletion(-) create mode 100644 test/test/test_rcu_qsbr.c create mode 100644 test/test/test_rcu_qsbr_perf.c -- 2.17.1 diff --git a/test/test/Makefile b/test/test/Makefile index 89949c2bb..6b6dfefc2 100644 --- a/test/test/Makefile +++ b/test/test/Makefile @@ -213,6 +213,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_KVARGS) += test_kvargs.c SRCS-$(CONFIG_RTE_LIBRTE_BPF) += test_bpf.c +SRCS-$(CONFIG_RTE_LIBRTE_RCU) += test_rcu_qsbr.c test_rcu_qsbr_perf.c + SRCS-$(CONFIG_RTE_LIBRTE_IPSEC) += test_ipsec.c ifeq ($(CONFIG_RTE_LIBRTE_IPSEC),y) LDLIBS += -lrte_ipsec diff --git a/test/test/autotest_data.py b/test/test/autotest_data.py index 5f87bb94d..c26ec889c 100644 --- a/test/test/autotest_data.py +++ b/test/test/autotest_data.py @@ -694,6 +694,18 @@ "Func": default_autotest, "Report": None, }, + { + "Name": "RCU QSBR autotest", + "Command": "rcu_qsbr_autotest", + "Func": default_autotest, + "Report": None, + }, + { + "Name": "RCU QSBR performance autotest", + "Command": "rcu_qsbr_perf_autotest", + "Func": default_autotest, + "Report": None, + }, # # Please always make sure that ring_perf is the last test! # diff --git a/test/test/meson.build b/test/test/meson.build index 05e5ddeb0..4df8e337b 100644 --- a/test/test/meson.build +++ b/test/test/meson.build @@ -107,6 +107,8 @@ test_sources = files('commands.c', 'test_timer.c', 'test_timer_perf.c', 'test_timer_racecond.c', + 'test_rcu_qsbr.c', + 'test_rcu_qsbr_perf.c', 'test_version.c', 'virtual_pmd.c' ) @@ -132,7 +134,8 @@ test_deps = ['acl', 'port', 'reorder', 'ring', - 'timer' + 'timer', + 'rcu' ] # All test cases in fast_parallel_test_names list are parallel @@ -171,6 +174,7 @@ fast_parallel_test_names = [ 'ring_autotest', 'ring_pmd_autotest', 'rwlock_autotest', + 'rcu_qsbr_autotest', 'sched_autotest', 'spinlock_autotest', 'string_autotest', @@ -236,6 +240,7 @@ perf_test_names = [ 'red_perf', 'distributor_perf_autotest', 'ring_pmd_perf_autotest', + 'rcu_qsbr_perf_autotest', 'pmd_perf_autotest', ] diff --git a/test/test/test_rcu_qsbr.c b/test/test/test_rcu_qsbr.c new file mode 100644 index 000000000..ae60614af --- /dev/null +++ b/test/test/test_rcu_qsbr.c @@ -0,0 +1,831 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2018 Arm Limited + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "test.h" + +/* Check condition and return an error if true. */ +#define RCU_QSBR_RETURN_IF_ERROR(cond, str, ...) do { \ + if (cond) { \ + printf("ERROR file %s, line %d: " str "\n", __FILE__, \ + __LINE__, ##__VA_ARGS__); \ + return -1; \ + } \ +} while (0) + +#define RTE_RCU_MAX_LCORE 128 +uint16_t enabled_core_ids[RTE_RCU_MAX_LCORE]; +uint8_t num_cores; +uint16_t num_1qs = 1; /* Number of quiescent states = 1 */ +uint16_t num_2qs = 2; /* Number of quiescent states = 2 */ +uint16_t num_3qs = 3; /* Number of quiescent states = 3 */ + +static uint32_t *keys; +#define TOTAL_ENTRY (1024 * 8) +#define COUNTER_VALUE 4096 +uint32_t *hash_data[RTE_RCU_MAX_LCORE][TOTAL_ENTRY]; +uint8_t writer_done; + +struct rte_rcu_qsbr t[RTE_RCU_MAX_LCORE]; +struct rte_hash *h[RTE_RCU_MAX_LCORE]; +char hash_name[RTE_RCU_MAX_LCORE][8]; + +static inline int +get_enabled_cores_mask(void) +{ + uint16_t core_id; + uint32_t max_cores = rte_lcore_count(); + if (max_cores > RTE_RCU_MAX_LCORE) { + printf("Number of cores exceed %d\n", RTE_RCU_MAX_LCORE); + return -1; + } + + core_id = 0; + num_cores = 0; + RTE_LCORE_FOREACH_SLAVE(core_id) { + enabled_core_ids[num_cores] = core_id; + num_cores++; + } + + return 0; +} + +/* + * rte_rcu_qsbr_register_thread: Add a reader thread, to the list of threads + * reporting their quiescent state on a QS variable. + */ +static int +test_rcu_qsbr_register_thread(void) +{ + printf("\nTest rte_rcu_qsbr_register_thread()\n"); + + rte_rcu_qsbr_init(&t[0]); + + rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[0]); + return 0; +} + +/* + * rte_rcu_qsbr_unregister_thread: Remove a reader thread, from the list of + * threads reporting their quiescent state on a QS variable. + */ +static int +test_rcu_qsbr_unregister_thread(void) +{ + int i, j, ret; + uint64_t token; + uint8_t num_threads[3] = {1, RTE_RCU_MAX_THREADS, 1}; + + printf("\nTest rte_rcu_qsbr_unregister_thread()\n"); + + rte_rcu_qsbr_init(&t[0]); + + rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[0]); + + /* Find first disabled core */ + for (i = 0; i < RTE_RCU_MAX_LCORE; i++) { + if (enabled_core_ids[i] == 0) + break; + } + /* Test with disabled lcore */ + rte_rcu_qsbr_unregister_thread(&t[0], i); + + /* Test with enabled lcore */ + rte_rcu_qsbr_unregister_thread(&t[0], enabled_core_ids[0]); + + /* + * Test with different thread_ids: + * 1 - thread_id = 0 + * 2 - All possible thread_ids, from 0 to RTE_RCU_MAX_THREADS + * 3 - thread_id = RTE_RCU_MAX_THREADS - 1 + */ + for (j = 0; j < 3; j++) { + rte_rcu_qsbr_init(&t[0]); + + for (i = 0; i < num_threads[j]; i++) + rte_rcu_qsbr_register_thread(&t[0], + (j == 2) ? (RTE_RCU_MAX_THREADS - 1) : i); + + rte_rcu_qsbr_start(&t[0], 1, &token); + RCU_QSBR_RETURN_IF_ERROR((token != 1), "QSBR Start"); + /* Update quiescent state counter */ + for (i = 0; i < num_threads[j]; i++) { + /* Skip one update */ + if (i == (RTE_RCU_MAX_THREADS - 10)) + continue; + rte_rcu_qsbr_update(&t[0], + (j == 2) ? (RTE_RCU_MAX_THREADS - 1) : i); + } + + if (j == 1) { + /* Validate the updates */ + ret = rte_rcu_qsbr_check(&t[0], token, false); + RCU_QSBR_RETURN_IF_ERROR((ret == 1), "Non-blocking QSBR check"); + /* Update the previously skipped thread */ + rte_rcu_qsbr_update(&t[0], RTE_RCU_MAX_THREADS - 10); + } + + /* Validate the updates */ + ret = rte_rcu_qsbr_check(&t[0], token, false); + RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Non-blocking QSBR check"); + + for (i = 0; i < num_threads[j]; i++) + rte_rcu_qsbr_unregister_thread(&t[0], + (j == 2) ? (RTE_RCU_MAX_THREADS - 1) : i); + + /* Check with no thread registered */ + ret = rte_rcu_qsbr_check(&t[0], token, true); + RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Blocking QSBR check"); + } + return 0; +} + +/* + * rte_rcu_qsbr_start: Trigger the worker threads to report the quiescent state + * status. + */ +static int +test_rcu_qsbr_start(void) +{ + uint64_t token; + int i; + + printf("\nTest rte_rcu_qsbr_start()\n"); + + rte_rcu_qsbr_init(&t[0]); + + for (i = 0; i < 3; i++) + rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[i]); + + rte_rcu_qsbr_start(&t[0], 1, &token); + RCU_QSBR_RETURN_IF_ERROR((token != 1), "QSBR Start"); + return 0; +} + +static int +test_rcu_qsbr_check_reader(void *arg) +{ + struct rte_rcu_qsbr *temp; + uint8_t read_type = (uint8_t)((uintptr_t)arg); + temp = &t[read_type]; + + /* Update quiescent state counter */ + rte_rcu_qsbr_update(temp, enabled_core_ids[0]); + rte_rcu_qsbr_update(temp, enabled_core_ids[1]); + rte_rcu_qsbr_unregister_thread(temp, enabled_core_ids[2]); + rte_rcu_qsbr_update(temp, enabled_core_ids[3]); + return 0; +} +/* + * rte_rcu_qsbr_check: Checks if all the worker threads have entered the queis- + * cent state 'n' number of times. 'n' is provided in rte_rcu_qsbr_start API. + */ +static int +test_rcu_qsbr_check(void) +{ + int i, ret; + uint64_t token; + + printf("\nTest rte_rcu_qsbr_check()\n"); + + rte_rcu_qsbr_init(&t[0]); + + rte_rcu_qsbr_start(&t[0], 1, &token); + RCU_QSBR_RETURN_IF_ERROR((token != 1), "QSBR Start"); + + + ret = rte_rcu_qsbr_check(&t[0], 0, false); + RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Token = 0"); + + ret = rte_rcu_qsbr_check(&t[0], token, true); + RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Blocking QSBR check"); + + for (i = 0; i < 3; i++) + rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[i]); + + ret = rte_rcu_qsbr_check(&t[0], token, false); + RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Non-blocking QSBR check"); + + rte_rcu_qsbr_start(&t[0], 1, &token); + RCU_QSBR_RETURN_IF_ERROR((token != 2), "QSBR Start"); + + ret = rte_rcu_qsbr_check(&t[0], token, false); + RCU_QSBR_RETURN_IF_ERROR((ret == 1), "Non-blocking QSBR check"); + + for (i = 0; i < 3; i++) + rte_rcu_qsbr_unregister_thread(&t[0], enabled_core_ids[i]); + + ret = rte_rcu_qsbr_check(&t[0], token, true); + RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Blocking QSBR check"); + + rte_rcu_qsbr_init(&t[0]); + + for (i = 0; i < 4; i++) + rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[i]); + + rte_rcu_qsbr_start(&t[0], 1, &token); + RCU_QSBR_RETURN_IF_ERROR((token != 1), "QSBR Start"); + + rte_eal_remote_launch(test_rcu_qsbr_check_reader, NULL, + enabled_core_ids[0]); + + rte_eal_mp_wait_lcore(); + ret = rte_rcu_qsbr_check(&t[0], token, true); + RCU_QSBR_RETURN_IF_ERROR((ret != 1), "Blocking QSBR check"); + + return 0; +} + +/* + * rte_rcu_qsbr_dump: Dump status of a single QS variable to a file + */ +static int +test_rcu_qsbr_dump(void) +{ + int i; + + printf("\nTest rte_rcu_qsbr_dump()\n"); + + rte_rcu_qsbr_init(&t[0]); + rte_rcu_qsbr_init(&t[1]); + + /* QS variable with 0 core mask */ + rte_rcu_qsbr_dump(stdout, &t[0]); + + rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[0]); + + for (i = 1; i < 3; i++) + rte_rcu_qsbr_register_thread(&t[1], enabled_core_ids[i]); + + rte_rcu_qsbr_dump(stdout, &t[0]); + rte_rcu_qsbr_dump(stdout, &t[1]); + printf("\n"); + return 0; +} + +static int +test_rcu_qsbr_reader(void *arg) +{ + struct rte_rcu_qsbr *temp; + struct rte_hash *hash = NULL; + int i; + uint32_t lcore_id = rte_lcore_id(); + uint8_t read_type = (uint8_t)((uintptr_t)arg); + uint32_t *pdata; + + temp = &t[read_type]; + hash = h[read_type]; + + do { + rte_rcu_qsbr_register_thread(temp, lcore_id); + for (i = 0; i < TOTAL_ENTRY; i++) { + if (rte_hash_lookup_data(hash, keys+i, + (void **)&pdata) != -ENOENT) { + *pdata = 0; + while (*pdata < COUNTER_VALUE) + ++*pdata; + } + } + /* Update quiescent state counter */ + rte_rcu_qsbr_update(temp, lcore_id); + rte_rcu_qsbr_unregister_thread(temp, lcore_id); + } while (!writer_done); + + return 0; +} + +static int +test_rcu_qsbr_writer(void *arg) +{ + uint64_t token; + int32_t pos; + struct rte_rcu_qsbr *temp; + struct rte_hash *hash = NULL; + uint8_t writer_type = (uint8_t)((uintptr_t)arg); + + temp = &t[(writer_type/2) % RTE_RCU_MAX_LCORE]; + hash = h[(writer_type/2) % RTE_RCU_MAX_LCORE]; + + /* Delete element from the shared data structure */ + pos = rte_hash_del_key(hash, keys + (writer_type % TOTAL_ENTRY)); + if (pos < 0) { + printf("Delete key failed #%d\n", + keys[writer_type % TOTAL_ENTRY]); + return -1; + } + /* + * Start the quiescent state query process + * Note: Expected Quiescent states kept greater than 1 for test only + */ + rte_rcu_qsbr_start(temp, writer_type + 1, &token); + /* Check the quiescent state status */ + rte_rcu_qsbr_check(temp, token, true); + if (*hash_data[(writer_type/2) % RTE_RCU_MAX_LCORE] + [writer_type % TOTAL_ENTRY] != COUNTER_VALUE && + *hash_data[(writer_type/2) % RTE_RCU_MAX_LCORE] + [writer_type % TOTAL_ENTRY] != 0) { + printf("Reader did not complete #%d = %d\t", writer_type, + *hash_data[(writer_type/2) % RTE_RCU_MAX_LCORE] + [writer_type % TOTAL_ENTRY]); + return -1; + } + + if (rte_hash_free_key_with_position(hash, pos) < 0) { + printf("Failed to free the key #%d\n", + keys[writer_type % TOTAL_ENTRY]); + return -1; + } + rte_free(hash_data[(writer_type/2) % RTE_RCU_MAX_LCORE] + [writer_type % TOTAL_ENTRY]); + hash_data[(writer_type/2) % RTE_RCU_MAX_LCORE] + [writer_type % TOTAL_ENTRY] = NULL; + + return 0; +} + +static struct rte_hash * +init_hash(int hash_id) +{ + int i; + struct rte_hash *h = NULL; + sprintf(hash_name[hash_id], "hash%d", hash_id); + struct rte_hash_parameters hash_params = { + .entries = TOTAL_ENTRY, + .key_len = sizeof(uint32_t), + .hash_func_init_val = 0, + .socket_id = rte_socket_id(), + .hash_func = rte_hash_crc, + .extra_flag = + RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF, + .name = hash_name[hash_id], + }; + + h = rte_hash_create(&hash_params); + if (h == NULL) { + printf("Hash create Failed\n"); + return NULL; + } + + for (i = 0; i < TOTAL_ENTRY; i++) { + hash_data[hash_id][i] = rte_zmalloc(NULL, sizeof(uint32_t), 0); + if (hash_data[hash_id][i] == NULL) { + printf("No memory\n"); + return NULL; + } + } + keys = rte_malloc(NULL, sizeof(uint32_t) * TOTAL_ENTRY, 0); + if (keys == NULL) { + printf("No memory\n"); + return NULL; + } + + for (i = 0; i < TOTAL_ENTRY; i++) + keys[i] = i; + + for (i = 0; i < TOTAL_ENTRY; i++) { + if (rte_hash_add_key_data(h, keys + i, + (void *)((uintptr_t)hash_data[hash_id][i])) + < 0) { + printf("Hash key add Failed #%d\n", i); + return NULL; + } + } + return h; +} + +/* + * Functional test: + * Single writer, Single QS variable Single QSBR query, Blocking rcu_qsbr_check + */ +static int +test_rcu_qsbr_sw_sv_1qs(void) +{ + uint64_t token; + int i; + int32_t pos; + writer_done = 0; + + printf("\nTest: 1 writer, 1 QSBR variable, 1 QSBR Query, " + "Blocking QSBR Check\n"); + + /* QS variable is initialized */ + rte_rcu_qsbr_init(&t[0]); + + /* Register worker threads on 4 cores */ + for (i = 0; i < 4; i++) + rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[i]); + + /* Shared data structure created */ + h[0] = init_hash(0); + if (h[0] == NULL) { + printf("Hash init failed\n"); + goto error; + } + + /* Reader threads are launched */ + for (i = 0; i < 4; i++) + rte_eal_remote_launch(test_rcu_qsbr_reader, NULL, + enabled_core_ids[i]); + + for (i = 0; i < TOTAL_ENTRY; i++) { + /* Delete elements from the shared data structure */ + pos = rte_hash_del_key(h[0], keys + i); + if (pos < 0) { + printf("Delete key failed #%d\n", keys[i]); + goto error; + } + /* Start the quiescent state query process */ + rte_rcu_qsbr_start(&t[0], num_1qs, &token); + + /* Check the quiescent state status */ + rte_rcu_qsbr_check(&t[0], token, true); + if (*hash_data[0][i] != COUNTER_VALUE && + *hash_data[0][i] != 0) { + printf("Reader did not complete #%d = %d\n", i, + *hash_data[0][i]); + goto error; + } + + if (rte_hash_free_key_with_position(h[0], pos) < 0) { + printf("Failed to free the key #%d\n", keys[i]); + goto error; + } + rte_free(hash_data[0][i]); + hash_data[0][i] = NULL; + } + writer_done = 1; + /* Wait until all readers have exited */ + rte_eal_mp_wait_lcore(); + /* Check return value from threads */ + for (i = 0; i < 4; i++) + if (lcore_config[enabled_core_ids[i]].ret < 0) + goto error; + rte_hash_free(h[0]); + rte_free(keys); + + return 0; + +error: + writer_done = 1; + /* Wait until all readers have exited */ + rte_eal_mp_wait_lcore(); + + rte_hash_free(h[0]); + rte_free(keys); + for (i = 0; i < TOTAL_ENTRY; i++) + rte_free(hash_data[0][i]); + + return -1; +} + +/* + * Functional test: + * Single writer, Single QS variable, Single QSBR query, + * Non-blocking rcu_qsbr_check + */ +static int +test_rcu_qsbr_sw_sv_1qs_non_blocking(void) +{ + uint64_t token; + int i, ret; + int32_t pos; + writer_done = 0; + + printf("Test: 1 writer, 1 QSBR variable, 1 QSBR Query, " + "Non-Blocking QSBR check\n"); + + rte_rcu_qsbr_init(&t[0]); + /* Register worker threads on 4 cores */ + for (i = 0; i < 4; i++) + rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[i]); + + /* Shared data structure created */ + h[0] = init_hash(0); + if (h[0] == NULL) { + printf("Hash init failed\n"); + goto error; + } + + /* Reader threads are launched */ + for (i = 0; i < 4; i++) + rte_eal_remote_launch(test_rcu_qsbr_reader, NULL, + enabled_core_ids[i]); + + for (i = 0; i < TOTAL_ENTRY; i++) { + /* Delete elements from the shared data structure */ + pos = rte_hash_del_key(h[0], keys + i); + if (pos < 0) { + printf("Delete key failed #%d\n", keys[i]); + goto error; + } + /* Start the quiescent state query process */ + rte_rcu_qsbr_start(&t[0], num_1qs, &token); + + /* Check the quiescent state status */ + do { + ret = rte_rcu_qsbr_check(&t[0], token, false); + } while (ret == 0); + if (*hash_data[0][i] != COUNTER_VALUE && + *hash_data[0][i] != 0) { + printf("Reader did not complete #%d = %d\n", i, + *hash_data[0][i]); + goto error; + } + + if (rte_hash_free_key_with_position(h[0], pos) < 0) { + printf("Failed to free the key #%d\n", keys[i]); + goto error; + } + rte_free(hash_data[0][i]); + hash_data[0][i] = NULL; + } + writer_done = 1; + /* Wait until all readers have exited */ + rte_eal_mp_wait_lcore(); + /* Check return value from threads */ + for (i = 0; i < 4; i++) + if (lcore_config[enabled_core_ids[i]].ret < 0) + goto error; + rte_hash_free(h[0]); + rte_free(keys); + + return 0; + +error: + writer_done = 1; + /* Wait until all readers have exited */ + rte_eal_mp_wait_lcore(); + + rte_hash_free(h[0]); + rte_free(keys); + for (i = 0; i < TOTAL_ENTRY; i++) + rte_free(hash_data[0][i]); + + return -1; +} + +/* + * Functional test: + * Single writer, Single QS variable, simultaneous QSBR Queries + */ +static int +test_rcu_qsbr_sw_sv_3qs(void) +{ + uint64_t token[3]; + int i; + int32_t pos[3]; + writer_done = 0; + + printf("Test: 1 writer, 1 QSBR variable, simultaneous QSBR queries\n"); + + rte_rcu_qsbr_init(&t[0]); + + /* Register worker threads on 4 cores */ + for (i = 0; i < 4; i++) + rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[i]); + + /* Shared data structure created */ + h[0] = init_hash(0); + if (h[0] == NULL) { + printf("Hash init failed\n"); + goto error; + } + + /* Reader threads are launched */ + for (i = 0; i < 4; i++) + rte_eal_remote_launch(test_rcu_qsbr_reader, NULL, + enabled_core_ids[i]); + + /* Delete element from the shared data structure */ + pos[0] = rte_hash_del_key(h[0], keys + 0); + if (pos[0] < 0) { + printf("Delete key failed #%d\n", keys[0]); + goto error; + } + /* Start the quiescent state query process */ + rte_rcu_qsbr_start(&t[0], num_1qs, &token[0]); + + /* Delete element from the shared data structure */ + pos[1] = rte_hash_del_key(h[0], keys + 3); + if (pos[1] < 0) { + printf("Delete key failed #%d\n", keys[3]); + goto error; + } + /* + * Start the quiescent state query process + * Note: num_2qs kept greater than 1 for test only + */ + rte_rcu_qsbr_start(&t[0], num_2qs, &token[1]); + + /* Delete element from the shared data structure */ + pos[2] = rte_hash_del_key(h[0], keys + 6); + if (pos[2] < 0) { + printf("Delete key failed #%d\n", keys[6]); + goto error; + } + /* + * Start the quiescent state query process + * Note: num_3qs kept greater than 1 for test only + */ + rte_rcu_qsbr_start(&t[0], num_3qs, &token[2]); + + /* Check the quiescent state status */ + rte_rcu_qsbr_check(&t[0], token[0], true); + if (*hash_data[0][0] != COUNTER_VALUE && *hash_data[0][0] != 0) { + printf("Reader did not complete #0 = %d\n", *hash_data[0][0]); + goto error; + } + + if (rte_hash_free_key_with_position(h[0], pos[0]) < 0) { + printf("Failed to free the key #%d\n", keys[0]); + goto error; + } + rte_free(hash_data[0][0]); + hash_data[0][0] = NULL; + + /* Check the quiescent state status */ + rte_rcu_qsbr_check(&t[0], token[1], true); + if (*hash_data[0][3] != COUNTER_VALUE && *hash_data[0][3] != 0) { + printf("Reader did not complete #3 = %d\n", *hash_data[0][3]); + goto error; + } + + if (rte_hash_free_key_with_position(h[0], pos[1]) < 0) { + printf("Failed to free the key #%d\n", keys[3]); + goto error; + } + rte_free(hash_data[0][3]); + hash_data[0][3] = NULL; + + /* Check the quiescent state status */ + rte_rcu_qsbr_check(&t[0], token[2], true); + if (*hash_data[0][6] != COUNTER_VALUE && *hash_data[0][6] != 0) { + printf("Reader did not complete #6 = %d\n", *hash_data[0][6]); + goto error; + } + + if (rte_hash_free_key_with_position(h[0], pos[2]) < 0) { + printf("Failed to free the key #%d\n", keys[6]); + goto error; + } + rte_free(hash_data[0][6]); + hash_data[0][6] = NULL; + + writer_done = 1; + /* Wait until all readers have exited */ + rte_eal_mp_wait_lcore(); + /* Check return value from threads */ + for (i = 0; i < 4; i++) + if (lcore_config[enabled_core_ids[i]].ret < 0) + goto error; + rte_hash_free(h[0]); + rte_free(keys); + + return 0; + +error: + writer_done = 1; + /* Wait until all readers have exited */ + rte_eal_mp_wait_lcore(); + + rte_hash_free(h[0]); + rte_free(keys); + for (i = 0; i < TOTAL_ENTRY; i++) + rte_free(hash_data[0][i]); + + return -1; +} + +/* + * Multi writer, Multiple QS variable, simultaneous QSBR queries + */ +static int +test_rcu_qsbr_mw_mv_mqs(void) +{ + int i, j; + writer_done = 0; + uint8_t test_cores; + test_cores = num_cores / 4; + test_cores = test_cores * 4; + + printf("Test: %d writers, %d QSBR variable, Simultaneous QSBR queries\n" + , test_cores / 2, test_cores / 4); + + for (i = 0; i < num_cores / 4; i++) { + rte_rcu_qsbr_init(&t[i]); + h[i] = init_hash(i); + if (h[i] == NULL) { + printf("Hash init failed\n"); + goto error; + } + } + + /* Register worker threads on 2 cores */ + for (i = 0; i < test_cores / 2; i += 2) { + rte_rcu_qsbr_register_thread(&t[i / 2], enabled_core_ids[i]); + rte_rcu_qsbr_register_thread(&t[i / 2], + enabled_core_ids[i + 1]); + } + + /* Reader threads are launched */ + for (i = 0; i < test_cores / 2; i++) + rte_eal_remote_launch(test_rcu_qsbr_reader, + (void *)(uintptr_t)(i / 2), + enabled_core_ids[i]); + + /* Writer threads are launched */ + for (; i < test_cores; i++) + rte_eal_remote_launch(test_rcu_qsbr_writer, + (void *)(uintptr_t)(i - (test_cores / 2)), + enabled_core_ids[i]); + /* Wait for writers to complete */ + for (i = test_cores / 2; i < test_cores; i++) + rte_eal_wait_lcore(enabled_core_ids[i]); + + writer_done = 1; + /* Wait for readers to complete */ + rte_eal_mp_wait_lcore(); + + /* Check return value from threads */ + for (i = 0; i < test_cores; i++) + if (lcore_config[enabled_core_ids[i]].ret < 0) + goto error; + + for (i = 0; i < num_cores / 4; i++) + rte_hash_free(h[i]); + + rte_free(keys); + + return 0; + +error: + writer_done = 1; + /* Wait until all readers have exited */ + rte_eal_mp_wait_lcore(); + + for (i = 0; i < num_cores / 4; i++) + rte_hash_free(h[i]); + rte_free(keys); + for (j = 0; j < RTE_RCU_MAX_LCORE; j++) + for (i = 0; i < TOTAL_ENTRY; i++) + rte_free(hash_data[j][i]); + + return -1; +} + +static int +test_rcu_qsbr_main(void) +{ + if (get_enabled_cores_mask() != 0) + return -1; + /* Error-checking test cases */ + if (test_rcu_qsbr_register_thread() < 0) + goto test_fail; + + if (test_rcu_qsbr_unregister_thread() < 0) + goto test_fail; + + if (test_rcu_qsbr_start() < 0) + goto test_fail; + + if (test_rcu_qsbr_check() < 0) + goto test_fail; + + if (test_rcu_qsbr_dump() < 0) + goto test_fail; + + /* Functional test cases */ + if (num_cores < 4) { + printf("Test failed! Need 4 or more cores\n"); + goto test_fail; + } + if (test_rcu_qsbr_sw_sv_1qs() < 0) + goto test_fail; + + if (test_rcu_qsbr_sw_sv_1qs_non_blocking() < 0) + goto test_fail; + + if (test_rcu_qsbr_sw_sv_3qs() < 0) + goto test_fail; + + if (test_rcu_qsbr_mw_mv_mqs() < 0) + goto test_fail; + + printf("\n"); + return 0; + +test_fail: + return -1; +} + +REGISTER_TEST_COMMAND(rcu_qsbr_autotest, test_rcu_qsbr_main); diff --git a/test/test/test_rcu_qsbr_perf.c b/test/test/test_rcu_qsbr_perf.c new file mode 100644 index 000000000..51af4961d --- /dev/null +++ b/test/test/test_rcu_qsbr_perf.c @@ -0,0 +1,272 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2018 Arm Limited + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "test.h" + +/* Check condition and return an error if true. */ +#define TEST_RCU_MAX_LCORE 128 +uint16_t enabled_core_ids[TEST_RCU_MAX_LCORE]; +uint8_t num_cores; + +#define TOTAL_ENTRY (1024 * 8) +#define COUNTER_VALUE 4096 +uint32_t *hash_data[TEST_RCU_MAX_LCORE][TOTAL_ENTRY]; +uint8_t writer_done; + +static struct rte_rcu_qsbr t; +struct rte_hash *h[TEST_RCU_MAX_LCORE]; +char hash_name[TEST_RCU_MAX_LCORE][8]; +static rte_atomic64_t updates, checks; +static rte_atomic64_t update_cycles, check_cycles; + +/* Scale down results to 1000 operations to support lower + * granularity clocks. + */ +#define RCU_SCALE_DOWN 1000 + +extern int test_rcu_qsbr_get_memsize(void); + +static inline int +get_enabled_cores_mask(void) +{ + uint16_t core_id; + uint32_t max_cores = rte_lcore_count(); + if (max_cores > TEST_RCU_MAX_LCORE) { + printf("Number of cores exceed %d\n", TEST_RCU_MAX_LCORE); + return -1; + } + + core_id = 0; + num_cores = 0; + RTE_LCORE_FOREACH_SLAVE(core_id) { + enabled_core_ids[num_cores] = core_id; + num_cores++; + } + + return 0; +} + +static int +test_rcu_qsbr_reader_perf(void *arg) +{ + bool writer_present = (bool)arg; + uint32_t lcore_id = rte_lcore_id(); + uint64_t loop_cnt = 0; + uint64_t begin, cycles; + + begin = rte_rdtsc_precise(); + + if (writer_present) { + while (!writer_done) { + /* Update quiescent state counter */ + rte_rcu_qsbr_update(&t, lcore_id); + loop_cnt++; + } + } else { + while (loop_cnt < 100000000) { + /* Update quiescent state counter */ + rte_rcu_qsbr_update(&t, lcore_id); + loop_cnt++; + } + } + + cycles = rte_rdtsc_precise() - begin; + rte_atomic64_add(&update_cycles, cycles); + rte_atomic64_add(&updates, loop_cnt); + + /* Unregister before exiting to avoid writer from waiting */ + rte_rcu_qsbr_unregister_thread(&t, lcore_id); + + return 0; +} + +static int +test_rcu_qsbr_writer_perf(void *arg) +{ + bool wait = (bool)arg; + uint64_t token = 0; + uint64_t loop_cnt = 0; + uint64_t begin, cycles; + + begin = rte_rdtsc_precise(); + + do { + /* Start the quiescent state query process */ + if (wait) rte_rcu_qsbr_start(&t, 1, &token); + + /* Check quiescent state status */ + rte_rcu_qsbr_check(&t, token, wait); + loop_cnt++; + } while (loop_cnt < 20000000); + + cycles = rte_rdtsc_precise() - begin; + rte_atomic64_add(&check_cycles, cycles); + rte_atomic64_add(&checks, loop_cnt); + return 0; +} + +/* + * Perf test: Reader/writer + * Single writer, Multiple Readers, Single QS var, Non-Blocking rcu_qsbr_check + */ +static int +test_rcu_qsbr_perf(void) +{ + int i; + writer_done = 0; + + rte_atomic64_clear(&updates); + rte_atomic64_clear(&update_cycles); + rte_atomic64_clear(&checks); + rte_atomic64_clear(&check_cycles); + + printf("\nPerf Test: 6 Readers/1 Writer('wait' in qsbr_check == true)\n"); + + /* QS variable is initialized */ + rte_rcu_qsbr_init(&t); + + /* Register worker threads on 6 cores */ + for (i = 0; i < 6; i++) + rte_rcu_qsbr_register_thread(&t, enabled_core_ids[i]); + + /* Reader threads are launched */ + for (i = 0; i < 6; i++) + rte_eal_remote_launch(test_rcu_qsbr_reader_perf, (void *)1, + enabled_core_ids[i]); + + /* Writer thread is launched */ + rte_eal_remote_launch(test_rcu_qsbr_writer_perf, + (void *)1, enabled_core_ids[i]); + + /* Wait for the writer thread */ + rte_eal_wait_lcore(enabled_core_ids[i]); + writer_done = 1; + + /* Wait until all readers have exited */ + rte_eal_mp_wait_lcore(); + + printf("Total RCU updates = %ld\n", rte_atomic64_read(&updates)); + printf("Cycles per %d updates: %lu\n", RCU_SCALE_DOWN, + rte_atomic64_read(&update_cycles) / + (rte_atomic64_read(&updates) / RCU_SCALE_DOWN)); + printf("Total RCU checks = %ld\n", rte_atomic64_read(&checks)); + printf("Cycles per %d checks: %lu\n", RCU_SCALE_DOWN, + rte_atomic64_read(&check_cycles) / + (rte_atomic64_read(&checks) / RCU_SCALE_DOWN)); + + return 0; +} + +/* + * Perf test: Readers + * Single writer, Multiple readers, Single QS variable + */ +static int +test_rcu_qsbr_rperf(void) +{ + int i; + + rte_atomic64_clear(&updates); + rte_atomic64_clear(&update_cycles); + rte_atomic64_clear(&checks); + rte_atomic64_clear(&check_cycles); + + printf("\nPerf Test: 8 Readers\n"); + + /* QS variable is initialized */ + rte_rcu_qsbr_init(&t); + + /* Register worker threads on 8 cores */ + for (i = 0; i < 8; i++) + rte_rcu_qsbr_register_thread(&t, enabled_core_ids[i]); + + /* Reader threads are launched */ + for (i = 0; i < 8; i++) + rte_eal_remote_launch(test_rcu_qsbr_reader_perf, NULL, + enabled_core_ids[i]); + + /* Wait until all readers have exited */ + rte_eal_mp_wait_lcore(); + + printf("Total RCU updates = %ld\n", rte_atomic64_read(&updates)); + printf("Cycles per %d updates: %lu\n", RCU_SCALE_DOWN, + rte_atomic64_read(&update_cycles) / + (rte_atomic64_read(&updates) / RCU_SCALE_DOWN)); + + return 0; +} + +/* + * Perf test: + * Multiple writer, Single QS variable, Non-blocking rcu_qsbr_check + */ +static int +test_rcu_qsbr_wperf(void) +{ + int i; + + rte_atomic64_clear(&updates); + rte_atomic64_clear(&update_cycles); + rte_atomic64_clear(&checks); + rte_atomic64_clear(&check_cycles); + + printf("\nPerf test: 8 Writers ('wait' in qsbr_check == false)\n"); + + /* QS variable is initialized */ + rte_rcu_qsbr_init(&t); + + /* Writer threads are launched */ + for (i = 0; i < 8; i++) + rte_eal_remote_launch(test_rcu_qsbr_writer_perf, + (void *)0, enabled_core_ids[i]); + + /* Wait until all readers have exited */ + rte_eal_mp_wait_lcore(); + + printf("Total RCU checks = %ld\n", rte_atomic64_read(&checks)); + printf("Cycles per %d checks: %lu\n", RCU_SCALE_DOWN, + rte_atomic64_read(&check_cycles) / + (rte_atomic64_read(&checks) / RCU_SCALE_DOWN)); + + return 0; +} + +static int +test_rcu_qsbr_main(void) +{ + rte_atomic64_init(&updates); + rte_atomic64_init(&update_cycles); + rte_atomic64_init(&checks); + rte_atomic64_init(&check_cycles); + + if (get_enabled_cores_mask() != 0) + return -1; + + if (test_rcu_qsbr_perf() < 0) + goto test_fail; + + if (test_rcu_qsbr_rperf() < 0) + goto test_fail; + + if (test_rcu_qsbr_wperf() < 0) + goto test_fail; + + printf("\n"); + return 0; + +test_fail: + return -1; +} + +REGISTER_TEST_COMMAND(rcu_qsbr_perf_autotest, test_rcu_qsbr_main); From patchwork Fri Feb 22 07:04:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 158986 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp1395469jaa; Thu, 21 Feb 2019 23:05:29 -0800 (PST) X-Google-Smtp-Source: AHgI3IahHe+Gi8TL0+4HxlWne0LvAi9Uswn+SV/zTniniUfFrxBhy0omejh3Hwjl/BjOBuU2x/ix X-Received: by 2002:a05:6402:12d4:: with SMTP id k20mr2034657edx.71.1550819129866; Thu, 21 Feb 2019 23:05:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550819129; cv=none; d=google.com; s=arc-20160816; b=g6/5cJl4XiXVz4omc20BSAUkF3JsHE6dTvC3fvD7Pwm8m1/XX/WuthCVp1e4rITwlq vfki9Ol6BCT2EA2Eu2zd8timvKFYuS2KFdSV92abH7jZWf54ygs1Bh/dbB/1C7ssQf13 lSX2Kj3h0d7PEEe1pH6+de1XXApgOazbI60uTL/RDiMA3BTMadRtbA8CgTt0nNIPMiMd PXPZu3yb836v3SqGiBopVNAWOL0mrfI3yGGl/rem3DGlgX1dPZfhQZNSTWOH6Hk883aE 3oVhZO+clocI+TcBzpFALOYtRhMzXMpf/5rHs44iuMkCMuixGyRsXFOONzBd5mrnTeKs TBww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=LCXAIAkw3h54jfLbVaAlpq7RYhnfbgEV91WV09iChDw=; b=PKSagtB88LUwo1FLNFpFiADdx5Po+xdkduSUTyI32lu8WYZmbidlvZp13CM37vPX4b j/czo+TwNF1DGtsUKvhwuuD3RsTT4b/avLx7VwFI31/2yhsQKUbO27iMW6DL2upMaTdX muuFaMNgCrsgvPL5gWM1lfML+ahz/2/S0Ct9KfVDt64E0MFoa4RHItPcgJ4XTq79U66C DQyYAc+HkxccKFULOovhip/3oPJeSiQZVmSv8hu0yxTRq671jUF+T9i86A/uGefdsbTN JNzfvRHkyJu+O915A9Pgr5LITn/N31iP7fe6DZoTlQDixGF5QqnG5xtsPfoyZH9hLZo2 62aA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id cb7si293464ejb.11.2019.02.21.23.05.29; Thu, 21 Feb 2019 23:05:29 -0800 (PST) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4D410316B; Fri, 22 Feb 2019 08:05:12 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id 76AD52C39 for ; Fri, 22 Feb 2019 08:05:10 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E593280D; Thu, 21 Feb 2019 23:05:09 -0800 (PST) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.12.104]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 747B43F690; Thu, 21 Feb 2019 23:05:09 -0800 (PST) From: Honnappa Nagarahalli To: konstantin.ananyev@intel.com, stephen@networkplumber.org, paulmck@linux.ibm.com, dev@dpdk.org, honnappa.nagarahalli@arm.com Cc: gavin.hu@arm.com, dharmik.thakkar@arm.com, malvika.gupta@arm.com, nd@arm.com Date: Fri, 22 Feb 2019 01:04:25 -0600 Message-Id: <20190222070427.22866-4-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190222070427.22866-1-honnappa.nagarahalli@arm.com> References: <20181222021420.5114-1-honnappa.nagarahalli@arm.com> <20190222070427.22866-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [RFC v3 3/5] lib/rcu: add dynamic memory allocation capability X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" rte_rcu_qsbr_get_memsize API is introduced. This will allow the user to controll the amount of memory used based on the maximum number of threads present in the application. Signed-off-by: Honnappa Nagarahalli --- lib/librte_rcu/rte_rcu_qsbr.c | 51 ++++++++++++--- lib/librte_rcu/rte_rcu_qsbr.h | 118 +++++++++++++++++++++------------- 2 files changed, 118 insertions(+), 51 deletions(-) -- 2.17.1 diff --git a/lib/librte_rcu/rte_rcu_qsbr.c b/lib/librte_rcu/rte_rcu_qsbr.c index 3c2577ee2..02464fdba 100644 --- a/lib/librte_rcu/rte_rcu_qsbr.c +++ b/lib/librte_rcu/rte_rcu_qsbr.c @@ -21,11 +21,39 @@ #include "rte_rcu_qsbr.h" +/* Get the memory size of QSBR variable */ +unsigned int __rte_experimental +rte_rcu_qsbr_get_memsize(uint32_t max_threads) +{ + int n; + ssize_t sz; + + RTE_ASSERT(max_threads == 0); + + sz = sizeof(struct rte_rcu_qsbr); + + /* Add the size of the registered thread ID bitmap array */ + n = RTE_ALIGN(max_threads, RTE_QSBR_THRID_ARRAY_ELM_SIZE); + sz += RTE_QSBR_THRID_ARRAY_SIZE(n); + + /* Add the size of quiescent state counter array */ + sz += sizeof(struct rte_rcu_qsbr_cnt) * max_threads; + + return RTE_ALIGN(sz, RTE_CACHE_LINE_SIZE); +} + /* Initialize a quiescent state variable */ void __rte_experimental -rte_rcu_qsbr_init(struct rte_rcu_qsbr *v) +rte_rcu_qsbr_init(struct rte_rcu_qsbr *v, uint32_t max_threads) { - memset(v, 0, sizeof(struct rte_rcu_qsbr)); + RTE_ASSERT(v == NULL); + + memset(v, 0, rte_rcu_qsbr_get_memsize(max_threads)); + v->m_threads = max_threads; + v->ma_threads = RTE_ALIGN(max_threads, RTE_QSBR_THRID_ARRAY_ELM_SIZE); + + v->num_elems = v->ma_threads/RTE_QSBR_THRID_ARRAY_ELM_SIZE; + v->thrid_array_size = RTE_QSBR_THRID_ARRAY_SIZE(v->ma_threads); } /* Dump the details of a single quiescent state variable to a file. */ @@ -39,9 +67,15 @@ rte_rcu_qsbr_dump(FILE *f, struct rte_rcu_qsbr *v) fprintf(f, "\nQuiescent State Variable @%p\n", v); + fprintf(f, " QS variable memory size = %u\n", + rte_rcu_qsbr_get_memsize(v->m_threads)); + fprintf(f, " Given # max threads = %u\n", v->m_threads); + fprintf(f, " Adjusted # max threads = %u\n", v->ma_threads); + fprintf(f, " Registered thread ID mask = 0x"); - for (i = 0; i < RTE_QSBR_BIT_MAP_ELEMS; i++) - fprintf(f, "%lx", __atomic_load_n(&v->reg_thread_id[i], + for (i = 0; i < v->num_elems; i++) + fprintf(f, "%lx", __atomic_load_n( + RTE_QSBR_THRID_ARRAY_ELM(v, i), __ATOMIC_ACQUIRE)); fprintf(f, "\n"); @@ -49,14 +83,15 @@ rte_rcu_qsbr_dump(FILE *f, struct rte_rcu_qsbr *v) __atomic_load_n(&v->token, __ATOMIC_ACQUIRE)); fprintf(f, "Quiescent State Counts for readers:\n"); - for (i = 0; i < RTE_QSBR_BIT_MAP_ELEMS; i++) { - bmap = __atomic_load_n(&v->reg_thread_id[i], + for (i = 0; i < v->num_elems; i++) { + bmap = __atomic_load_n(RTE_QSBR_THRID_ARRAY_ELM(v, i), __ATOMIC_ACQUIRE); while (bmap) { t = __builtin_ctzl(bmap); fprintf(f, "thread ID = %d, count = %lu\n", t, - __atomic_load_n(&v->w[i].cnt, - __ATOMIC_RELAXED)); + __atomic_load_n( + &RTE_QSBR_CNT_ARRAY_ELM(v, i)->cnt, + __ATOMIC_RELAXED)); bmap &= ~(1UL << t); } } diff --git a/lib/librte_rcu/rte_rcu_qsbr.h b/lib/librte_rcu/rte_rcu_qsbr.h index 53e00488b..21fa2c198 100644 --- a/lib/librte_rcu/rte_rcu_qsbr.h +++ b/lib/librte_rcu/rte_rcu_qsbr.h @@ -29,46 +29,71 @@ extern "C" { #include #include -/**< Maximum number of reader threads supported. */ -#define RTE_RCU_MAX_THREADS 128 - -#if !RTE_IS_POWER_OF_2(RTE_RCU_MAX_THREADS) -#error RTE_RCU_MAX_THREADS must be a power of 2 -#endif - -/**< Number of array elements required for the bit-map */ -#define RTE_QSBR_BIT_MAP_ELEMS (RTE_RCU_MAX_THREADS/(sizeof(uint64_t) * 8)) - -/* Thread IDs are stored as a bitmap of 64b element array. Given thread id - * needs to be converted to index into the array and the id within - * the array element. +/* Registered thread IDs are stored as a bitmap of 64b element array. + * Given thread id needs to be converted to index into the array and + * the id within the array element. + */ +/* Thread ID array size + * @param ma_threads + * num of threads aligned to 64 */ -#define RTE_QSBR_THR_INDEX_SHIFT 6 -#define RTE_QSBR_THR_ID_MASK 0x3f +#define RTE_QSBR_THRID_ARRAY_SIZE(ma_threads) \ + RTE_ALIGN((ma_threads) >> 3, RTE_CACHE_LINE_SIZE) +#define RTE_QSBR_THRID_ARRAY_ELM_SIZE (sizeof(uint64_t) * 8) +#define RTE_QSBR_THRID_ARRAY_ELM(v, i) ((uint64_t *)(v + 1) + i) +#define RTE_QSBR_THRID_INDEX_SHIFT 6 +#define RTE_QSBR_THRID_MASK 0x3f /* Worker thread counter */ struct rte_rcu_qsbr_cnt { uint64_t cnt; /**< Quiescent state counter. */ } __rte_cache_aligned; +#define RTE_QSBR_CNT_ARRAY_ELM(v, i) ((struct rte_rcu_qsbr_cnt *) \ + ((uint8_t *)(v + 1) + v->thrid_array_size) + i) + /** * RTE thread Quiescent State structure. + * The following data, which is dependent on the maximum number of + * threads using this variable, is stored in memory immediately + * following this structure. + * + * 1) registered thread ID bitmap array + * This is a uint64_t array enough to hold 'ma_threads' number + * of thread IDs. + * 2) quiescent state counter array + * This is an array of 'struct rte_rcu_qsbr_cnt' with + * 'm_threads' number of elements. */ struct rte_rcu_qsbr { - uint64_t reg_thread_id[RTE_QSBR_BIT_MAP_ELEMS] __rte_cache_aligned; - /**< Registered reader thread IDs - reader threads reporting - * on this QS variable represented in a bit map. - */ - uint64_t token __rte_cache_aligned; /**< Counter to allow for multiple simultaneous QS queries */ - struct rte_rcu_qsbr_cnt w[RTE_RCU_MAX_THREADS] __rte_cache_aligned; - /**< QS counter for each reader thread, counts upto - * current value of token. - */ + uint32_t thrid_array_size __rte_cache_aligned; + /**< Registered thread ID bitmap array size in bytes */ + uint32_t num_elems; + /**< Number of elements in the thread ID array */ + + uint32_t m_threads; + /**< Maximum number of threads this RCU variable will use */ + uint32_t ma_threads; + /**< Maximum number of threads aligned to 32 */ } __rte_cache_aligned; +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Return the size of the memory occupied by a Quiescent State (QS) variable. + * + * @param max_threads + * Maximum number of threads reporting QS on this variable. + * @return + * Size of memory in bytes required for this QS variable. + */ +unsigned int __rte_experimental +rte_rcu_qsbr_get_memsize(uint32_t max_threads); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice @@ -77,10 +102,12 @@ struct rte_rcu_qsbr { * * @param v * QS variable + * @param max_threads + * Maximum number of threads reporting QS on this variable. * */ void __rte_experimental -rte_rcu_qsbr_init(struct rte_rcu_qsbr *v); +rte_rcu_qsbr_init(struct rte_rcu_qsbr *v, uint32_t max_threads); /** * @warning @@ -108,24 +135,25 @@ rte_rcu_qsbr_register_thread(struct rte_rcu_qsbr *v, unsigned int thread_id) { unsigned int i, id; - RTE_ASSERT(v == NULL || thread_id >= RTE_RCU_MAX_THREADS); + RTE_ASSERT(v == NULL || thread_id >= v->max_threads); - id = thread_id & RTE_QSBR_THR_ID_MASK; - i = thread_id >> RTE_QSBR_THR_INDEX_SHIFT; + id = thread_id & RTE_QSBR_THRID_MASK; + i = thread_id >> RTE_QSBR_THRID_INDEX_SHIFT; /* Worker thread has to count the quiescent states * only from the current value of token. * __atomic_store_n(cnt, __ATOMIC_RELAXED) is used to ensure * 'cnt' (64b) is accessed atomically. */ - __atomic_store_n(&v->w[thread_id].cnt, + __atomic_store_n(&RTE_QSBR_CNT_ARRAY_ELM(v, thread_id)->cnt, __atomic_load_n(&v->token, __ATOMIC_ACQUIRE), __ATOMIC_RELAXED); /* Release the store to initial TQS count so that readers * can use it immediately after this function returns. */ - __atomic_fetch_or(&v->reg_thread_id[i], 1UL << id, __ATOMIC_RELEASE); + __atomic_fetch_or(RTE_QSBR_THRID_ARRAY_ELM(v, i), + 1UL << id, __ATOMIC_RELEASE); } /** @@ -151,16 +179,16 @@ rte_rcu_qsbr_unregister_thread(struct rte_rcu_qsbr *v, unsigned int thread_id) { unsigned int i, id; - RTE_ASSERT(v == NULL || thread_id >= RTE_RCU_MAX_THREADS); + RTE_ASSERT(v == NULL || thread_id >= v->max_threads); - id = thread_id & RTE_QSBR_THR_ID_MASK; - i = thread_id >> RTE_QSBR_THR_INDEX_SHIFT; + id = thread_id & RTE_QSBR_THRID_MASK; + i = thread_id >> RTE_QSBR_THRID_INDEX_SHIFT; /* Make sure the removal of the thread from the list of * reporting threads is visible before the thread * does anything else. */ - __atomic_fetch_and(&v->reg_thread_id[i], + __atomic_fetch_and(RTE_QSBR_THRID_ARRAY_ELM(v, i), ~(1UL << id), __ATOMIC_RELEASE); } @@ -212,7 +240,7 @@ rte_rcu_qsbr_update(struct rte_rcu_qsbr *v, unsigned int thread_id) { uint64_t t; - RTE_ASSERT(v == NULL || thread_id >= RTE_RCU_MAX_THREADS); + RTE_ASSERT(v == NULL || thread_id >= v->max_threads); /* Load the token before the reader thread loads any other * (lock-free) data structure. This ensures that updates @@ -228,8 +256,10 @@ rte_rcu_qsbr_update(struct rte_rcu_qsbr *v, unsigned int thread_id) * Copy the current token value. This will end grace period * of multiple concurrent writers. */ - if (__atomic_load_n(&v->w[thread_id].cnt, __ATOMIC_RELAXED) != t) - __atomic_store_n(&v->w[thread_id].cnt, t, __ATOMIC_RELAXED); + if (__atomic_load_n(&RTE_QSBR_CNT_ARRAY_ELM(v, thread_id)->cnt, + __ATOMIC_RELAXED) != t) + __atomic_store_n(&RTE_QSBR_CNT_ARRAY_ELM(v, thread_id)->cnt, + t, __ATOMIC_RELAXED); } /** @@ -268,18 +298,20 @@ rte_rcu_qsbr_check(struct rte_rcu_qsbr *v, uint64_t t, bool wait) /* Load the current registered thread bit map before * loading the reader thread quiescent state counters. */ - bmap = __atomic_load_n(&v->reg_thread_id[i], __ATOMIC_ACQUIRE); - id = i << RTE_QSBR_THR_INDEX_SHIFT; + bmap = __atomic_load_n(RTE_QSBR_THRID_ARRAY_ELM(v, i), + __ATOMIC_ACQUIRE); + id = i << RTE_QSBR_THRID_INDEX_SHIFT; while (bmap) { j = __builtin_ctzl(bmap); -/* printf ("Status check: token = %lu, wait = %d, Bit Map = 0x%x, Thread ID = %d\n", t, wait, bmap, id+j); */ +/* printf ("Status check: token = %lu, wait = %d, Bit Map = 0x%lx, Thread ID = %d\n", t, wait, bmap, id+j); */ /* __atomic_load_n(cnt, __ATOMIC_RELAXED) * is used to ensure 'cnt' (64b) is accessed * atomically. */ - if (unlikely(__atomic_load_n(&v->w[id + j].cnt, + if (unlikely(__atomic_load_n( + &RTE_QSBR_CNT_ARRAY_ELM(v, id + j)->cnt, __ATOMIC_RELAXED) < t)) { /* printf ("Status not in QS: token = %lu, Wait = %d, Thread QS cnt = %lu, Thread ID = %d\n", t, wait, RTE_QSBR_CNT_ARRAY_ELM(v, id + j)->cnt, id+j); */ @@ -292,7 +324,7 @@ rte_rcu_qsbr_check(struct rte_rcu_qsbr *v, uint64_t t, bool wait) * Re-read the bitmap. */ bmap = __atomic_load_n( - &v->reg_thread_id[i], + RTE_QSBR_THRID_ARRAY_ELM(v, i), __ATOMIC_ACQUIRE); continue; @@ -302,7 +334,7 @@ rte_rcu_qsbr_check(struct rte_rcu_qsbr *v, uint64_t t, bool wait) } i++; - } while (i < RTE_QSBR_BIT_MAP_ELEMS); + } while (i < v->num_elems); return 1; } From patchwork Fri Feb 22 07:04:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 158987 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp1395594jaa; Thu, 21 Feb 2019 23:05:39 -0800 (PST) X-Google-Smtp-Source: AHgI3IZNdsQstf6QdT0Fi4Ih+/PK87QRrYPGBaRdIv1TOnmHzexbzPlk60vO3TPIAE26v+wAGH/3 X-Received: by 2002:a17:906:19c4:: with SMTP id h4mr1781450ejd.30.1550819139160; Thu, 21 Feb 2019 23:05:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550819139; cv=none; d=google.com; s=arc-20160816; b=uMhG2JfS5v3OCTVa+wn30DKOQsxf/3z1z3zDshfDVhPJ+E9KuqERMe5gNlgZxPgk0D IqSWlBYJfgf+VL8ErKleWBoUGIfPtscABsKNYzvY2p2adb2HFsO12GZBGcQ40AIxcLYx 22cRvmUx3U8z690t5lSo0i5Ax3Of3KarsQ93VUC6VyJSZxzhWxSXuuS7/ZRJ+S+GpP5v KKbUSDIbZUwLqmo+MegRH3HD3SrkdxqlwBrWKIO+IQaFnrhkE9bFqU89Pc/5OLY5Z8Wx dPk42g2mUwi4qr2khANK0OK8aKWk2ze/UmcMx9oUou+1RU8xK4raKqXP+Hn8pM88nnHh tbZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=0tXpevRt7QK+n29WGMAG8dCd9E9wjR+vaekMbrGn/fE=; b=E/zCZ6kLSfhnjIqZ+zl6+5JOvhPLOrZUple3GidKPdl4TDEpVsnau9n9Cm94DfxbLA bh2abokp8NSMNxw1VR5qFcVhrV78EsUAGaxVPXsqTCwiSJ8MsslR8SWl3kz4pFpI5TbB u/hTWoUxjIqeudUko//JHvOUWg1YF54ASRNkS0N1Aqew0AKCjO44/SyfNO7WqQffhvmJ OxOZaexaCVMXAmoSFU1TVpXQu7hPEVlFOZKU7aUkzxHum8AxHcWIUEyfzWUY9Cq0lIGE S74UGoEjHfJs6TpbRMMFPV2iYHk3GOaoUhrtYHfH+mUfv74smNIErbc/Ypm8nHpyXJXc 0DmQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id bq19si290574ejb.45.2019.02.21.23.05.38; Thu, 21 Feb 2019 23:05:39 -0800 (PST) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id ADF522C54; Fri, 22 Feb 2019 08:05:13 +0100 (CET) Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id 454132C6D for ; Fri, 22 Feb 2019 08:05:12 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9DBD115AB; Thu, 21 Feb 2019 23:05:11 -0800 (PST) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.12.104]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2534C3F690; Thu, 21 Feb 2019 23:05:11 -0800 (PST) From: Honnappa Nagarahalli To: konstantin.ananyev@intel.com, stephen@networkplumber.org, paulmck@linux.ibm.com, dev@dpdk.org, honnappa.nagarahalli@arm.com Cc: gavin.hu@arm.com, dharmik.thakkar@arm.com, malvika.gupta@arm.com, nd@arm.com Date: Fri, 22 Feb 2019 01:04:26 -0600 Message-Id: <20190222070427.22866-5-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190222070427.22866-1-honnappa.nagarahalli@arm.com> References: <20181222021420.5114-1-honnappa.nagarahalli@arm.com> <20190222070427.22866-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [RFC v3 4/5] test/rcu_qsbr: modify test cases for dynamic memory allocation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Modify the test cases to allocate the QSBR variable dynamically. Signed-off-by: Honnappa Nagarahalli --- test/test/test_rcu_qsbr.c | 225 ++++++++++++++++++--------------- test/test/test_rcu_qsbr_perf.c | 33 ++--- 2 files changed, 144 insertions(+), 114 deletions(-) -- 2.17.1 diff --git a/test/test/test_rcu_qsbr.c b/test/test/test_rcu_qsbr.c index ae60614af..09744279f 100644 --- a/test/test/test_rcu_qsbr.c +++ b/test/test/test_rcu_qsbr.c @@ -15,7 +15,7 @@ #include "test.h" /* Check condition and return an error if true. */ -#define RCU_QSBR_RETURN_IF_ERROR(cond, str, ...) do { \ +#define TEST_RCU_QSBR_RETURN_IF_ERROR(cond, str, ...) do { \ if (cond) { \ printf("ERROR file %s, line %d: " str "\n", __FILE__, \ __LINE__, ##__VA_ARGS__); \ @@ -23,8 +23,8 @@ } \ } while (0) -#define RTE_RCU_MAX_LCORE 128 -uint16_t enabled_core_ids[RTE_RCU_MAX_LCORE]; +#define TEST_RCU_MAX_LCORE 128 +uint16_t enabled_core_ids[TEST_RCU_MAX_LCORE]; uint8_t num_cores; uint16_t num_1qs = 1; /* Number of quiescent states = 1 */ uint16_t num_2qs = 2; /* Number of quiescent states = 2 */ @@ -33,20 +33,20 @@ uint16_t num_3qs = 3; /* Number of quiescent states = 3 */ static uint32_t *keys; #define TOTAL_ENTRY (1024 * 8) #define COUNTER_VALUE 4096 -uint32_t *hash_data[RTE_RCU_MAX_LCORE][TOTAL_ENTRY]; +uint32_t *hash_data[TEST_RCU_MAX_LCORE][TOTAL_ENTRY]; uint8_t writer_done; -struct rte_rcu_qsbr t[RTE_RCU_MAX_LCORE]; -struct rte_hash *h[RTE_RCU_MAX_LCORE]; -char hash_name[RTE_RCU_MAX_LCORE][8]; +static struct rte_rcu_qsbr *t[TEST_RCU_MAX_LCORE]; +struct rte_hash *h[TEST_RCU_MAX_LCORE]; +char hash_name[TEST_RCU_MAX_LCORE][8]; static inline int get_enabled_cores_mask(void) { uint16_t core_id; uint32_t max_cores = rte_lcore_count(); - if (max_cores > RTE_RCU_MAX_LCORE) { - printf("Number of cores exceed %d\n", RTE_RCU_MAX_LCORE); + if (max_cores > TEST_RCU_MAX_LCORE) { + printf("Number of cores exceed %d\n", TEST_RCU_MAX_LCORE); return -1; } @@ -60,6 +60,30 @@ get_enabled_cores_mask(void) return 0; } +/* + * rte_rcu_qsbr_register_thread: Add a reader thread, to the list of threads + * reporting their quiescent state on a QS variable. + */ +static int +test_rcu_qsbr_get_memsize(void) +{ + uint32_t sz; + + printf("\nTest rte_rcu_qsbr_register_thread()\n"); + + sz = rte_rcu_qsbr_get_memsize(TEST_RCU_MAX_LCORE); + t[0] = (struct rte_rcu_qsbr *)rte_zmalloc("rcu0", sz, + RTE_CACHE_LINE_SIZE); + t[1] = (struct rte_rcu_qsbr *)rte_zmalloc("rcu1", sz, + RTE_CACHE_LINE_SIZE); + t[2] = (struct rte_rcu_qsbr *)rte_zmalloc("rcu2", sz, + RTE_CACHE_LINE_SIZE); + t[3] = (struct rte_rcu_qsbr *)rte_zmalloc("rcu3", sz, + RTE_CACHE_LINE_SIZE); + + return 0; +} + /* * rte_rcu_qsbr_register_thread: Add a reader thread, to the list of threads * reporting their quiescent state on a QS variable. @@ -69,9 +93,9 @@ test_rcu_qsbr_register_thread(void) { printf("\nTest rte_rcu_qsbr_register_thread()\n"); - rte_rcu_qsbr_init(&t[0]); + rte_rcu_qsbr_init(t[0], TEST_RCU_MAX_LCORE); - rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[0]); + rte_rcu_qsbr_register_thread(t[0], enabled_core_ids[0]); return 0; } @@ -84,68 +108,68 @@ test_rcu_qsbr_unregister_thread(void) { int i, j, ret; uint64_t token; - uint8_t num_threads[3] = {1, RTE_RCU_MAX_THREADS, 1}; + uint8_t num_threads[3] = {1, TEST_RCU_MAX_LCORE, 1}; printf("\nTest rte_rcu_qsbr_unregister_thread()\n"); - rte_rcu_qsbr_init(&t[0]); + rte_rcu_qsbr_init(t[0], TEST_RCU_MAX_LCORE); - rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[0]); + rte_rcu_qsbr_register_thread(t[0], enabled_core_ids[0]); /* Find first disabled core */ - for (i = 0; i < RTE_RCU_MAX_LCORE; i++) { + for (i = 0; i < TEST_RCU_MAX_LCORE; i++) { if (enabled_core_ids[i] == 0) break; } /* Test with disabled lcore */ - rte_rcu_qsbr_unregister_thread(&t[0], i); + rte_rcu_qsbr_unregister_thread(t[0], i); /* Test with enabled lcore */ - rte_rcu_qsbr_unregister_thread(&t[0], enabled_core_ids[0]); + rte_rcu_qsbr_unregister_thread(t[0], enabled_core_ids[0]); /* * Test with different thread_ids: * 1 - thread_id = 0 - * 2 - All possible thread_ids, from 0 to RTE_RCU_MAX_THREADS - * 3 - thread_id = RTE_RCU_MAX_THREADS - 1 + * 2 - All possible thread_ids, from 0 to TEST_RCU_MAX_LCORE + * 3 - thread_id = TEST_RCU_MAX_LCORE - 1 */ for (j = 0; j < 3; j++) { - rte_rcu_qsbr_init(&t[0]); + rte_rcu_qsbr_init(t[0], TEST_RCU_MAX_LCORE); for (i = 0; i < num_threads[j]; i++) - rte_rcu_qsbr_register_thread(&t[0], - (j == 2) ? (RTE_RCU_MAX_THREADS - 1) : i); + rte_rcu_qsbr_register_thread(t[0], + (j == 2) ? (TEST_RCU_MAX_LCORE - 1) : i); - rte_rcu_qsbr_start(&t[0], 1, &token); - RCU_QSBR_RETURN_IF_ERROR((token != 1), "QSBR Start"); + rte_rcu_qsbr_start(t[0], 1, &token); + TEST_RCU_QSBR_RETURN_IF_ERROR((token != 1), "QSBR Start"); /* Update quiescent state counter */ for (i = 0; i < num_threads[j]; i++) { /* Skip one update */ - if (i == (RTE_RCU_MAX_THREADS - 10)) + if (i == (TEST_RCU_MAX_LCORE - 10)) continue; - rte_rcu_qsbr_update(&t[0], - (j == 2) ? (RTE_RCU_MAX_THREADS - 1) : i); + rte_rcu_qsbr_update(t[0], + (j == 2) ? (TEST_RCU_MAX_LCORE - 1) : i); } if (j == 1) { /* Validate the updates */ - ret = rte_rcu_qsbr_check(&t[0], token, false); - RCU_QSBR_RETURN_IF_ERROR((ret == 1), "Non-blocking QSBR check"); + ret = rte_rcu_qsbr_check(t[0], token, false); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 1), "Non-blocking QSBR check"); /* Update the previously skipped thread */ - rte_rcu_qsbr_update(&t[0], RTE_RCU_MAX_THREADS - 10); + rte_rcu_qsbr_update(t[0], TEST_RCU_MAX_LCORE - 10); } /* Validate the updates */ - ret = rte_rcu_qsbr_check(&t[0], token, false); - RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Non-blocking QSBR check"); + ret = rte_rcu_qsbr_check(t[0], token, false); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Non-blocking QSBR check"); for (i = 0; i < num_threads[j]; i++) - rte_rcu_qsbr_unregister_thread(&t[0], - (j == 2) ? (RTE_RCU_MAX_THREADS - 1) : i); + rte_rcu_qsbr_unregister_thread(t[0], + (j == 2) ? (TEST_RCU_MAX_LCORE - 1) : i); /* Check with no thread registered */ - ret = rte_rcu_qsbr_check(&t[0], token, true); - RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Blocking QSBR check"); + ret = rte_rcu_qsbr_check(t[0], token, true); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Blocking QSBR check"); } return 0; } @@ -162,13 +186,13 @@ test_rcu_qsbr_start(void) printf("\nTest rte_rcu_qsbr_start()\n"); - rte_rcu_qsbr_init(&t[0]); + rte_rcu_qsbr_init(t[0], TEST_RCU_MAX_LCORE); for (i = 0; i < 3; i++) - rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[i]); + rte_rcu_qsbr_register_thread(t[0], enabled_core_ids[i]); - rte_rcu_qsbr_start(&t[0], 1, &token); - RCU_QSBR_RETURN_IF_ERROR((token != 1), "QSBR Start"); + rte_rcu_qsbr_start(t[0], 1, &token); + TEST_RCU_QSBR_RETURN_IF_ERROR((token != 1), "QSBR Start"); return 0; } @@ -177,7 +201,7 @@ test_rcu_qsbr_check_reader(void *arg) { struct rte_rcu_qsbr *temp; uint8_t read_type = (uint8_t)((uintptr_t)arg); - temp = &t[read_type]; + temp = t[read_type]; /* Update quiescent state counter */ rte_rcu_qsbr_update(temp, enabled_core_ids[0]); @@ -198,50 +222,50 @@ test_rcu_qsbr_check(void) printf("\nTest rte_rcu_qsbr_check()\n"); - rte_rcu_qsbr_init(&t[0]); + rte_rcu_qsbr_init(t[0], TEST_RCU_MAX_LCORE); - rte_rcu_qsbr_start(&t[0], 1, &token); - RCU_QSBR_RETURN_IF_ERROR((token != 1), "QSBR Start"); + rte_rcu_qsbr_start(t[0], 1, &token); + TEST_RCU_QSBR_RETURN_IF_ERROR((token != 1), "QSBR Start"); - ret = rte_rcu_qsbr_check(&t[0], 0, false); - RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Token = 0"); + ret = rte_rcu_qsbr_check(t[0], 0, false); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Token = 0"); - ret = rte_rcu_qsbr_check(&t[0], token, true); - RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Blocking QSBR check"); + ret = rte_rcu_qsbr_check(t[0], token, true); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Blocking QSBR check"); for (i = 0; i < 3; i++) - rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[i]); + rte_rcu_qsbr_register_thread(t[0], enabled_core_ids[i]); - ret = rte_rcu_qsbr_check(&t[0], token, false); - RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Non-blocking QSBR check"); + ret = rte_rcu_qsbr_check(t[0], token, false); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Non-blocking QSBR check"); - rte_rcu_qsbr_start(&t[0], 1, &token); - RCU_QSBR_RETURN_IF_ERROR((token != 2), "QSBR Start"); + rte_rcu_qsbr_start(t[0], 1, &token); + TEST_RCU_QSBR_RETURN_IF_ERROR((token != 2), "QSBR Start"); - ret = rte_rcu_qsbr_check(&t[0], token, false); - RCU_QSBR_RETURN_IF_ERROR((ret == 1), "Non-blocking QSBR check"); + ret = rte_rcu_qsbr_check(t[0], token, false); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 1), "Non-blocking QSBR check"); for (i = 0; i < 3; i++) - rte_rcu_qsbr_unregister_thread(&t[0], enabled_core_ids[i]); + rte_rcu_qsbr_unregister_thread(t[0], enabled_core_ids[i]); - ret = rte_rcu_qsbr_check(&t[0], token, true); - RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Blocking QSBR check"); + ret = rte_rcu_qsbr_check(t[0], token, true); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 0), "Blocking QSBR check"); - rte_rcu_qsbr_init(&t[0]); + rte_rcu_qsbr_init(t[0], TEST_RCU_MAX_LCORE); for (i = 0; i < 4; i++) - rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[i]); + rte_rcu_qsbr_register_thread(t[0], enabled_core_ids[i]); - rte_rcu_qsbr_start(&t[0], 1, &token); - RCU_QSBR_RETURN_IF_ERROR((token != 1), "QSBR Start"); + rte_rcu_qsbr_start(t[0], 1, &token); + TEST_RCU_QSBR_RETURN_IF_ERROR((token != 1), "QSBR Start"); rte_eal_remote_launch(test_rcu_qsbr_check_reader, NULL, enabled_core_ids[0]); rte_eal_mp_wait_lcore(); - ret = rte_rcu_qsbr_check(&t[0], token, true); - RCU_QSBR_RETURN_IF_ERROR((ret != 1), "Blocking QSBR check"); + ret = rte_rcu_qsbr_check(t[0], token, true); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret != 1), "Blocking QSBR check"); return 0; } @@ -256,19 +280,19 @@ test_rcu_qsbr_dump(void) printf("\nTest rte_rcu_qsbr_dump()\n"); - rte_rcu_qsbr_init(&t[0]); - rte_rcu_qsbr_init(&t[1]); + rte_rcu_qsbr_init(t[0], TEST_RCU_MAX_LCORE); + rte_rcu_qsbr_init(t[1], TEST_RCU_MAX_LCORE); /* QS variable with 0 core mask */ - rte_rcu_qsbr_dump(stdout, &t[0]); + rte_rcu_qsbr_dump(stdout, t[0]); - rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[0]); + rte_rcu_qsbr_register_thread(t[0], enabled_core_ids[0]); for (i = 1; i < 3; i++) - rte_rcu_qsbr_register_thread(&t[1], enabled_core_ids[i]); + rte_rcu_qsbr_register_thread(t[1], enabled_core_ids[i]); - rte_rcu_qsbr_dump(stdout, &t[0]); - rte_rcu_qsbr_dump(stdout, &t[1]); + rte_rcu_qsbr_dump(stdout, t[0]); + rte_rcu_qsbr_dump(stdout, t[1]); printf("\n"); return 0; } @@ -283,7 +307,7 @@ test_rcu_qsbr_reader(void *arg) uint8_t read_type = (uint8_t)((uintptr_t)arg); uint32_t *pdata; - temp = &t[read_type]; + temp = t[read_type]; hash = h[read_type]; do { @@ -313,8 +337,8 @@ test_rcu_qsbr_writer(void *arg) struct rte_hash *hash = NULL; uint8_t writer_type = (uint8_t)((uintptr_t)arg); - temp = &t[(writer_type/2) % RTE_RCU_MAX_LCORE]; - hash = h[(writer_type/2) % RTE_RCU_MAX_LCORE]; + temp = t[(writer_type/2) % TEST_RCU_MAX_LCORE]; + hash = h[(writer_type/2) % TEST_RCU_MAX_LCORE]; /* Delete element from the shared data structure */ pos = rte_hash_del_key(hash, keys + (writer_type % TOTAL_ENTRY)); @@ -330,12 +354,12 @@ test_rcu_qsbr_writer(void *arg) rte_rcu_qsbr_start(temp, writer_type + 1, &token); /* Check the quiescent state status */ rte_rcu_qsbr_check(temp, token, true); - if (*hash_data[(writer_type/2) % RTE_RCU_MAX_LCORE] + if (*hash_data[(writer_type/2) % TEST_RCU_MAX_LCORE] [writer_type % TOTAL_ENTRY] != COUNTER_VALUE && - *hash_data[(writer_type/2) % RTE_RCU_MAX_LCORE] + *hash_data[(writer_type/2) % TEST_RCU_MAX_LCORE] [writer_type % TOTAL_ENTRY] != 0) { printf("Reader did not complete #%d = %d\t", writer_type, - *hash_data[(writer_type/2) % RTE_RCU_MAX_LCORE] + *hash_data[(writer_type/2) % TEST_RCU_MAX_LCORE] [writer_type % TOTAL_ENTRY]); return -1; } @@ -345,9 +369,9 @@ test_rcu_qsbr_writer(void *arg) keys[writer_type % TOTAL_ENTRY]); return -1; } - rte_free(hash_data[(writer_type/2) % RTE_RCU_MAX_LCORE] + rte_free(hash_data[(writer_type/2) % TEST_RCU_MAX_LCORE] [writer_type % TOTAL_ENTRY]); - hash_data[(writer_type/2) % RTE_RCU_MAX_LCORE] + hash_data[(writer_type/2) % TEST_RCU_MAX_LCORE] [writer_type % TOTAL_ENTRY] = NULL; return 0; @@ -419,11 +443,11 @@ test_rcu_qsbr_sw_sv_1qs(void) "Blocking QSBR Check\n"); /* QS variable is initialized */ - rte_rcu_qsbr_init(&t[0]); + rte_rcu_qsbr_init(t[0], TEST_RCU_MAX_LCORE); /* Register worker threads on 4 cores */ for (i = 0; i < 4; i++) - rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[i]); + rte_rcu_qsbr_register_thread(t[0], enabled_core_ids[i]); /* Shared data structure created */ h[0] = init_hash(0); @@ -445,10 +469,10 @@ test_rcu_qsbr_sw_sv_1qs(void) goto error; } /* Start the quiescent state query process */ - rte_rcu_qsbr_start(&t[0], num_1qs, &token); + rte_rcu_qsbr_start(t[0], num_1qs, &token); /* Check the quiescent state status */ - rte_rcu_qsbr_check(&t[0], token, true); + rte_rcu_qsbr_check(t[0], token, true); if (*hash_data[0][i] != COUNTER_VALUE && *hash_data[0][i] != 0) { printf("Reader did not complete #%d = %d\n", i, @@ -504,10 +528,10 @@ test_rcu_qsbr_sw_sv_1qs_non_blocking(void) printf("Test: 1 writer, 1 QSBR variable, 1 QSBR Query, " "Non-Blocking QSBR check\n"); - rte_rcu_qsbr_init(&t[0]); + rte_rcu_qsbr_init(t[0], TEST_RCU_MAX_LCORE); /* Register worker threads on 4 cores */ for (i = 0; i < 4; i++) - rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[i]); + rte_rcu_qsbr_register_thread(t[0], enabled_core_ids[i]); /* Shared data structure created */ h[0] = init_hash(0); @@ -529,11 +553,11 @@ test_rcu_qsbr_sw_sv_1qs_non_blocking(void) goto error; } /* Start the quiescent state query process */ - rte_rcu_qsbr_start(&t[0], num_1qs, &token); + rte_rcu_qsbr_start(t[0], num_1qs, &token); /* Check the quiescent state status */ do { - ret = rte_rcu_qsbr_check(&t[0], token, false); + ret = rte_rcu_qsbr_check(t[0], token, false); } while (ret == 0); if (*hash_data[0][i] != COUNTER_VALUE && *hash_data[0][i] != 0) { @@ -588,11 +612,11 @@ test_rcu_qsbr_sw_sv_3qs(void) printf("Test: 1 writer, 1 QSBR variable, simultaneous QSBR queries\n"); - rte_rcu_qsbr_init(&t[0]); + rte_rcu_qsbr_init(t[0], TEST_RCU_MAX_LCORE); /* Register worker threads on 4 cores */ for (i = 0; i < 4; i++) - rte_rcu_qsbr_register_thread(&t[0], enabled_core_ids[i]); + rte_rcu_qsbr_register_thread(t[0], enabled_core_ids[i]); /* Shared data structure created */ h[0] = init_hash(0); @@ -613,7 +637,7 @@ test_rcu_qsbr_sw_sv_3qs(void) goto error; } /* Start the quiescent state query process */ - rte_rcu_qsbr_start(&t[0], num_1qs, &token[0]); + rte_rcu_qsbr_start(t[0], num_1qs, &token[0]); /* Delete element from the shared data structure */ pos[1] = rte_hash_del_key(h[0], keys + 3); @@ -625,7 +649,7 @@ test_rcu_qsbr_sw_sv_3qs(void) * Start the quiescent state query process * Note: num_2qs kept greater than 1 for test only */ - rte_rcu_qsbr_start(&t[0], num_2qs, &token[1]); + rte_rcu_qsbr_start(t[0], num_2qs, &token[1]); /* Delete element from the shared data structure */ pos[2] = rte_hash_del_key(h[0], keys + 6); @@ -637,10 +661,10 @@ test_rcu_qsbr_sw_sv_3qs(void) * Start the quiescent state query process * Note: num_3qs kept greater than 1 for test only */ - rte_rcu_qsbr_start(&t[0], num_3qs, &token[2]); + rte_rcu_qsbr_start(t[0], num_3qs, &token[2]); /* Check the quiescent state status */ - rte_rcu_qsbr_check(&t[0], token[0], true); + rte_rcu_qsbr_check(t[0], token[0], true); if (*hash_data[0][0] != COUNTER_VALUE && *hash_data[0][0] != 0) { printf("Reader did not complete #0 = %d\n", *hash_data[0][0]); goto error; @@ -654,7 +678,7 @@ test_rcu_qsbr_sw_sv_3qs(void) hash_data[0][0] = NULL; /* Check the quiescent state status */ - rte_rcu_qsbr_check(&t[0], token[1], true); + rte_rcu_qsbr_check(t[0], token[1], true); if (*hash_data[0][3] != COUNTER_VALUE && *hash_data[0][3] != 0) { printf("Reader did not complete #3 = %d\n", *hash_data[0][3]); goto error; @@ -668,7 +692,7 @@ test_rcu_qsbr_sw_sv_3qs(void) hash_data[0][3] = NULL; /* Check the quiescent state status */ - rte_rcu_qsbr_check(&t[0], token[2], true); + rte_rcu_qsbr_check(t[0], token[2], true); if (*hash_data[0][6] != COUNTER_VALUE && *hash_data[0][6] != 0) { printf("Reader did not complete #6 = %d\n", *hash_data[0][6]); goto error; @@ -722,7 +746,7 @@ test_rcu_qsbr_mw_mv_mqs(void) , test_cores / 2, test_cores / 4); for (i = 0; i < num_cores / 4; i++) { - rte_rcu_qsbr_init(&t[i]); + rte_rcu_qsbr_init(t[i], TEST_RCU_MAX_LCORE); h[i] = init_hash(i); if (h[i] == NULL) { printf("Hash init failed\n"); @@ -732,8 +756,8 @@ test_rcu_qsbr_mw_mv_mqs(void) /* Register worker threads on 2 cores */ for (i = 0; i < test_cores / 2; i += 2) { - rte_rcu_qsbr_register_thread(&t[i / 2], enabled_core_ids[i]); - rte_rcu_qsbr_register_thread(&t[i / 2], + rte_rcu_qsbr_register_thread(t[i / 2], enabled_core_ids[i]); + rte_rcu_qsbr_register_thread(t[i / 2], enabled_core_ids[i + 1]); } @@ -776,7 +800,7 @@ test_rcu_qsbr_mw_mv_mqs(void) for (i = 0; i < num_cores / 4; i++) rte_hash_free(h[i]); rte_free(keys); - for (j = 0; j < RTE_RCU_MAX_LCORE; j++) + for (j = 0; j < TEST_RCU_MAX_LCORE; j++) for (i = 0; i < TOTAL_ENTRY; i++) rte_free(hash_data[j][i]); @@ -788,6 +812,9 @@ test_rcu_qsbr_main(void) { if (get_enabled_cores_mask() != 0) return -1; + +test_rcu_qsbr_get_memsize(); + /* Error-checking test cases */ if (test_rcu_qsbr_register_thread() < 0) goto test_fail; diff --git a/test/test/test_rcu_qsbr_perf.c b/test/test/test_rcu_qsbr_perf.c index 51af4961d..89c5030bb 100644 --- a/test/test/test_rcu_qsbr_perf.c +++ b/test/test/test_rcu_qsbr_perf.c @@ -24,7 +24,7 @@ uint8_t num_cores; uint32_t *hash_data[TEST_RCU_MAX_LCORE][TOTAL_ENTRY]; uint8_t writer_done; -static struct rte_rcu_qsbr t; +static struct rte_rcu_qsbr *t; struct rte_hash *h[TEST_RCU_MAX_LCORE]; char hash_name[TEST_RCU_MAX_LCORE][8]; static rte_atomic64_t updates, checks; @@ -70,13 +70,13 @@ test_rcu_qsbr_reader_perf(void *arg) if (writer_present) { while (!writer_done) { /* Update quiescent state counter */ - rte_rcu_qsbr_update(&t, lcore_id); + rte_rcu_qsbr_update(t, lcore_id); loop_cnt++; } } else { while (loop_cnt < 100000000) { /* Update quiescent state counter */ - rte_rcu_qsbr_update(&t, lcore_id); + rte_rcu_qsbr_update(t, lcore_id); loop_cnt++; } } @@ -86,7 +86,7 @@ test_rcu_qsbr_reader_perf(void *arg) rte_atomic64_add(&updates, loop_cnt); /* Unregister before exiting to avoid writer from waiting */ - rte_rcu_qsbr_unregister_thread(&t, lcore_id); + rte_rcu_qsbr_unregister_thread(t, lcore_id); return 0; } @@ -103,10 +103,10 @@ test_rcu_qsbr_writer_perf(void *arg) do { /* Start the quiescent state query process */ - if (wait) rte_rcu_qsbr_start(&t, 1, &token); + if (wait) rte_rcu_qsbr_start(t, 1, &token); /* Check quiescent state status */ - rte_rcu_qsbr_check(&t, token, wait); + rte_rcu_qsbr_check(t, token, wait); loop_cnt++; } while (loop_cnt < 20000000); @@ -134,11 +134,11 @@ test_rcu_qsbr_perf(void) printf("\nPerf Test: 6 Readers/1 Writer('wait' in qsbr_check == true)\n"); /* QS variable is initialized */ - rte_rcu_qsbr_init(&t); + rte_rcu_qsbr_init(t, TEST_RCU_MAX_LCORE); /* Register worker threads on 6 cores */ for (i = 0; i < 6; i++) - rte_rcu_qsbr_register_thread(&t, enabled_core_ids[i]); + rte_rcu_qsbr_register_thread(t, enabled_core_ids[i]); /* Reader threads are launched */ for (i = 0; i < 6; i++) @@ -179,17 +179,15 @@ test_rcu_qsbr_rperf(void) rte_atomic64_clear(&updates); rte_atomic64_clear(&update_cycles); - rte_atomic64_clear(&checks); - rte_atomic64_clear(&check_cycles); printf("\nPerf Test: 8 Readers\n"); /* QS variable is initialized */ - rte_rcu_qsbr_init(&t); + rte_rcu_qsbr_init(t, TEST_RCU_MAX_LCORE); /* Register worker threads on 8 cores */ for (i = 0; i < 8; i++) - rte_rcu_qsbr_register_thread(&t, enabled_core_ids[i]); + rte_rcu_qsbr_register_thread(t, enabled_core_ids[i]); /* Reader threads are launched */ for (i = 0; i < 8; i++) @@ -200,6 +198,8 @@ test_rcu_qsbr_rperf(void) rte_eal_mp_wait_lcore(); printf("Total RCU updates = %ld\n", rte_atomic64_read(&updates)); + printf("Total update cycles = %ld\n", + rte_atomic64_read(&update_cycles)); printf("Cycles per %d updates: %lu\n", RCU_SCALE_DOWN, rte_atomic64_read(&update_cycles) / (rte_atomic64_read(&updates) / RCU_SCALE_DOWN)); @@ -216,15 +216,13 @@ test_rcu_qsbr_wperf(void) { int i; - rte_atomic64_clear(&updates); - rte_atomic64_clear(&update_cycles); rte_atomic64_clear(&checks); rte_atomic64_clear(&check_cycles); printf("\nPerf test: 8 Writers ('wait' in qsbr_check == false)\n"); /* QS variable is initialized */ - rte_rcu_qsbr_init(&t); + rte_rcu_qsbr_init(t, TEST_RCU_MAX_LCORE); /* Writer threads are launched */ for (i = 0; i < 8; i++) @@ -245,6 +243,8 @@ test_rcu_qsbr_wperf(void) static int test_rcu_qsbr_main(void) { + uint32_t sz; + rte_atomic64_init(&updates); rte_atomic64_init(&update_cycles); rte_atomic64_init(&checks); @@ -253,6 +253,9 @@ test_rcu_qsbr_main(void) if (get_enabled_cores_mask() != 0) return -1; + sz = rte_rcu_qsbr_get_memsize(TEST_RCU_MAX_LCORE); + t = (struct rte_rcu_qsbr *)rte_zmalloc("rcu", sz, RTE_CACHE_LINE_SIZE); + if (test_rcu_qsbr_perf() < 0) goto test_fail; From patchwork Fri Feb 22 07:04:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 158988 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp1395741jaa; Thu, 21 Feb 2019 23:05:50 -0800 (PST) X-Google-Smtp-Source: AHgI3IZBz5hnh9ytrirSpE0D19uqo1ZVMHld6mr9VA1lnP43zx3g7NR94Q9lGYCzwXpRcikWbXf9 X-Received: by 2002:a17:906:5586:: with SMTP id y6mr1821252ejp.197.1550819150778; Thu, 21 Feb 2019 23:05:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550819150; cv=none; d=google.com; s=arc-20160816; b=suRMPLeVrokOq+s1Yg2VN9DeVRZpwEF1/WRLqvgtWrLFJ7Nd7t3EB7ZYN9UDkfs0k6 CHleeUhhqmsqNF13gMFOnCV+VHItRXmlNS/eqSLivC8Ye/xgs9Kqkh7o4Y7wjrU/9BnJ AJUHTdMB9fnBWNi/O5XeEjrDgTA47NUoXmATz1dttSmi07Nq9An+/dO1ghBgWpq5UQqK A3+QbUriRp4Uz2GUGc1D0sZAZDWUPJ1Rs4n8QFlc8hklwd43S/b1c3ZMY1eTgrSkPRA6 yraN17o6txfJ4Z2Raa/ccBUWCKZn+O09q4QY5mKXFY8D4OaqgHx0p20LP3PO0HDAygYZ qYjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=4qKfs+V9frFuHzD68wEUtvl26urxT1OM6KpY/2KPpGU=; b=RWzEqELNNsaZX97IhSWqmTFMvU1Ss/dYeDze0XVfTtWgN849OpnMK2AvK9XD8SiKoT tg1uILwevGcJiH5dpN+GjvgLNprbQ7u31jRLftgtiU2ZMT6ZLFVn1EZ2UVYXQ+mAf0Fs rTvFgnV0BGoqXOwo04K/qqEdPXFH7xG54cxPHWKXrNkgI75ywjR2dZpYTPZkOmJK2AHN xgWGX4CmSg7V5LQ/TwREjQaT8+tOoACI8MU/9kHWevERTHzEDfKBvrTbAhWNa31/yofh lufSLVUHtqnPLyWJ5waQ1I9B+vUJTAfKEOOoldjBX6m4qSor9hiCjXttH2R/nB8LgPhU 60+A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id f10si320275edb.207.2019.02.21.23.05.50; Thu, 21 Feb 2019 23:05:50 -0800 (PST) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 51CC6374E; Fri, 22 Feb 2019 08:05:16 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id E63D831FC for ; Fri, 22 Feb 2019 08:05:13 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5F63580D; Thu, 21 Feb 2019 23:05:13 -0800 (PST) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.12.104]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E3D3A3F690; Thu, 21 Feb 2019 23:05:12 -0800 (PST) From: Honnappa Nagarahalli To: konstantin.ananyev@intel.com, stephen@networkplumber.org, paulmck@linux.ibm.com, dev@dpdk.org, honnappa.nagarahalli@arm.com Cc: gavin.hu@arm.com, dharmik.thakkar@arm.com, malvika.gupta@arm.com, nd@arm.com Date: Fri, 22 Feb 2019 01:04:27 -0600 Message-Id: <20190222070427.22866-6-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190222070427.22866-1-honnappa.nagarahalli@arm.com> References: <20181222021420.5114-1-honnappa.nagarahalli@arm.com> <20190222070427.22866-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [RFC v3 5/5] lib/rcu: fix the size of register thread ID array size X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Keeping the register thread ID size dependent on the max threads is resulting in performance drops due to address calculations at run time. Fixing the size of the thread ID registration array reduces the complexity of address calculation. This change fixes the maximum number of threads supported to 512(1 cache line size of 64B). However, the memory required for QS counters is still dependent on the max threads parameter. This change provides both flexibility and addresses performance as well. Signed-off-by: Honnappa Nagarahalli --- lib/librte_rcu/rte_rcu_qsbr.c | 13 ++----------- lib/librte_rcu/rte_rcu_qsbr.h | 29 ++++++++++------------------- 2 files changed, 12 insertions(+), 30 deletions(-) -- 2.17.1 diff --git a/lib/librte_rcu/rte_rcu_qsbr.c b/lib/librte_rcu/rte_rcu_qsbr.c index 02464fdba..3cff82121 100644 --- a/lib/librte_rcu/rte_rcu_qsbr.c +++ b/lib/librte_rcu/rte_rcu_qsbr.c @@ -25,17 +25,12 @@ unsigned int __rte_experimental rte_rcu_qsbr_get_memsize(uint32_t max_threads) { - int n; ssize_t sz; RTE_ASSERT(max_threads == 0); sz = sizeof(struct rte_rcu_qsbr); - /* Add the size of the registered thread ID bitmap array */ - n = RTE_ALIGN(max_threads, RTE_QSBR_THRID_ARRAY_ELM_SIZE); - sz += RTE_QSBR_THRID_ARRAY_SIZE(n); - /* Add the size of quiescent state counter array */ sz += sizeof(struct rte_rcu_qsbr_cnt) * max_threads; @@ -51,9 +46,7 @@ rte_rcu_qsbr_init(struct rte_rcu_qsbr *v, uint32_t max_threads) memset(v, 0, rte_rcu_qsbr_get_memsize(max_threads)); v->m_threads = max_threads; v->ma_threads = RTE_ALIGN(max_threads, RTE_QSBR_THRID_ARRAY_ELM_SIZE); - v->num_elems = v->ma_threads/RTE_QSBR_THRID_ARRAY_ELM_SIZE; - v->thrid_array_size = RTE_QSBR_THRID_ARRAY_SIZE(v->ma_threads); } /* Dump the details of a single quiescent state variable to a file. */ @@ -74,8 +67,7 @@ rte_rcu_qsbr_dump(FILE *f, struct rte_rcu_qsbr *v) fprintf(f, " Registered thread ID mask = 0x"); for (i = 0; i < v->num_elems; i++) - fprintf(f, "%lx", __atomic_load_n( - RTE_QSBR_THRID_ARRAY_ELM(v, i), + fprintf(f, "%lx", __atomic_load_n(&v->reg_thread_id[i], __ATOMIC_ACQUIRE)); fprintf(f, "\n"); @@ -84,8 +76,7 @@ rte_rcu_qsbr_dump(FILE *f, struct rte_rcu_qsbr *v) fprintf(f, "Quiescent State Counts for readers:\n"); for (i = 0; i < v->num_elems; i++) { - bmap = __atomic_load_n(RTE_QSBR_THRID_ARRAY_ELM(v, i), - __ATOMIC_ACQUIRE); + bmap = __atomic_load_n(&v->reg_thread_id[i], __ATOMIC_ACQUIRE); while (bmap) { t = __builtin_ctzl(bmap); fprintf(f, "thread ID = %d, count = %lu\n", t, diff --git a/lib/librte_rcu/rte_rcu_qsbr.h b/lib/librte_rcu/rte_rcu_qsbr.h index 21fa2c198..1147f11f2 100644 --- a/lib/librte_rcu/rte_rcu_qsbr.h +++ b/lib/librte_rcu/rte_rcu_qsbr.h @@ -33,14 +33,9 @@ extern "C" { * Given thread id needs to be converted to index into the array and * the id within the array element. */ -/* Thread ID array size - * @param ma_threads - * num of threads aligned to 64 - */ -#define RTE_QSBR_THRID_ARRAY_SIZE(ma_threads) \ - RTE_ALIGN((ma_threads) >> 3, RTE_CACHE_LINE_SIZE) +#define RTE_RCU_MAX_THREADS 512 +#define RTE_QSBR_THRID_ARRAY_ELEMS (RTE_RCU_MAX_THREADS/(sizeof(uint64_t) * 8)) #define RTE_QSBR_THRID_ARRAY_ELM_SIZE (sizeof(uint64_t) * 8) -#define RTE_QSBR_THRID_ARRAY_ELM(v, i) ((uint64_t *)(v + 1) + i) #define RTE_QSBR_THRID_INDEX_SHIFT 6 #define RTE_QSBR_THRID_MASK 0x3f @@ -49,8 +44,7 @@ struct rte_rcu_qsbr_cnt { uint64_t cnt; /**< Quiescent state counter. */ } __rte_cache_aligned; -#define RTE_QSBR_CNT_ARRAY_ELM(v, i) ((struct rte_rcu_qsbr_cnt *) \ - ((uint8_t *)(v + 1) + v->thrid_array_size) + i) +#define RTE_QSBR_CNT_ARRAY_ELM(v, i) (((struct rte_rcu_qsbr_cnt *)(v + 1)) + i) /** * RTE thread Quiescent State structure. @@ -69,15 +63,14 @@ struct rte_rcu_qsbr { uint64_t token __rte_cache_aligned; /**< Counter to allow for multiple simultaneous QS queries */ - uint32_t thrid_array_size __rte_cache_aligned; - /**< Registered thread ID bitmap array size in bytes */ - uint32_t num_elems; + uint32_t num_elems __rte_cache_aligned; /**< Number of elements in the thread ID array */ - uint32_t m_threads; /**< Maximum number of threads this RCU variable will use */ uint32_t ma_threads; /**< Maximum number of threads aligned to 32 */ + + uint64_t reg_thread_id[RTE_QSBR_THRID_ARRAY_ELEMS] __rte_cache_aligned; } __rte_cache_aligned; /** @@ -152,8 +145,7 @@ rte_rcu_qsbr_register_thread(struct rte_rcu_qsbr *v, unsigned int thread_id) /* Release the store to initial TQS count so that readers * can use it immediately after this function returns. */ - __atomic_fetch_or(RTE_QSBR_THRID_ARRAY_ELM(v, i), - 1UL << id, __ATOMIC_RELEASE); + __atomic_fetch_or(&v->reg_thread_id[i], 1UL << id, __ATOMIC_RELEASE); } /** @@ -188,7 +180,7 @@ rte_rcu_qsbr_unregister_thread(struct rte_rcu_qsbr *v, unsigned int thread_id) * reporting threads is visible before the thread * does anything else. */ - __atomic_fetch_and(RTE_QSBR_THRID_ARRAY_ELM(v, i), + __atomic_fetch_and(&v->reg_thread_id[i], ~(1UL << id), __ATOMIC_RELEASE); } @@ -298,8 +290,7 @@ rte_rcu_qsbr_check(struct rte_rcu_qsbr *v, uint64_t t, bool wait) /* Load the current registered thread bit map before * loading the reader thread quiescent state counters. */ - bmap = __atomic_load_n(RTE_QSBR_THRID_ARRAY_ELM(v, i), - __ATOMIC_ACQUIRE); + bmap = __atomic_load_n(&v->reg_thread_id[i], __ATOMIC_ACQUIRE); id = i << RTE_QSBR_THRID_INDEX_SHIFT; while (bmap) { @@ -324,7 +315,7 @@ rte_rcu_qsbr_check(struct rte_rcu_qsbr *v, uint64_t t, bool wait) * Re-read the bitmap. */ bmap = __atomic_load_n( - RTE_QSBR_THRID_ARRAY_ELM(v, i), + &v->reg_thread_id[i], __ATOMIC_ACQUIRE); continue;