From patchwork Thu Nov 22 03:30:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 151713 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp219001ljp; Wed, 21 Nov 2018 19:31:53 -0800 (PST) X-Google-Smtp-Source: AFSGD/UFH1G633sUbnTQiUf6ZULKLnMPF/UTMtVBK0A2jp5/pguCQMhM9Ig0bAh7H/rBaNea5/81 X-Received: by 2002:a50:9996:: with SMTP id m22mr7923646edb.7.1542857513658; Wed, 21 Nov 2018 19:31:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542857513; cv=none; d=google.com; s=arc-20160816; b=Rh+UTCmiBxqT6TLS/OH4MRcxillPMFW/Tm6Y5i/xHVfrC/ZYQTf3OOCT6MnWTgDvxs i2LjwdGAXiim2jBMlcHxSEjZQt1K6ESA7dqtiYzoQ2BwpBydHx27mqmb15r/RGWD/fKN MNtTnIMvauxq6pNL/vH1JNAE1h39bWFdultvwSILs6/4ttC7bhVbd6RXbmzDrngfmAEy MCyov80nDqRur9W51JlvjqPA5xj27LabwGkl86wS/hGVMELYCDE+SlzuZzbqajRUqli2 G+K0tZESOOUQhHNufZx3F4V+8+cZ6CkiUL57j6qau0/RaiAd8uTMW1gq488tW84GQz1M Obfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=P3q/R+hE4rC8L3xgWW53TFZzxCHKVbakHZ8OlDkonl0=; b=uQ4vnal+fXxkpCbNnb5/MCpWZiRnvGPY4p5iE3bu2qV4Iz+E+hVaAIbTPTa/KfnRae 92sAqRnklqWNnNUoDTTWXO/IUN3/M+oQGLa04sxtRIPUYviSYkkKzl0TCeyLzWuOgAZx bhtNWz/l6WZZYGlQp5WJPRwH4fUZDaHaEE0RmSKUBCcbuLace1bCbrBEcaAmhI0Wijbe tFftHp1FnParXfx82hb9cXkwfQF8gZ2P/RhuHVXIQWGvhltc2QVyixZOQboq5upaXOOT fleiejpZ6SWovYf+Y3uqpBVFp2aCHDh0ORjpxv3dt7lI48aISk8X8eMLcbCaMO0iWZMW aRkQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id l1si2919660edn.1.2018.11.21.19.31.53; Wed, 21 Nov 2018 19:31:53 -0800 (PST) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 07D721B1FD; Thu, 22 Nov 2018 04:31:52 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id 84B4D1B1F9 for ; Thu, 22 Nov 2018 04:31:50 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E21993617; Wed, 21 Nov 2018 19:31:49 -0800 (PST) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.14.34]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8E04B3F5CF; Wed, 21 Nov 2018 19:31:49 -0800 (PST) From: Honnappa Nagarahalli To: dev@dpdk.org Cc: nd@arm.com, honnappa.nagarahalli@arm.com, dharmik.thakkar@arm.com, malvika.gupta@arm.com, gavin.hu@arm.com Date: Wed, 21 Nov 2018 21:30:53 -0600 Message-Id: <20181122033055.3431-2-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181122033055.3431-1-honnappa.nagarahalli@arm.com> References: <20181122033055.3431-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [RFC 1/3] log: add TQS log type X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Honnappa Nagarahalli Reviewed-by: Steve Capper Reviewed-by: Gavin Hu --- lib/librte_eal/common/include/rte_log.h | 1 + 1 file changed, 1 insertion(+) -- 2.17.1 diff --git a/lib/librte_eal/common/include/rte_log.h b/lib/librte_eal/common/include/rte_log.h index 2f789cb90..b4e91a4a5 100644 --- a/lib/librte_eal/common/include/rte_log.h +++ b/lib/librte_eal/common/include/rte_log.h @@ -61,6 +61,7 @@ extern struct rte_logs rte_logs; #define RTE_LOGTYPE_EFD 18 /**< Log related to EFD. */ #define RTE_LOGTYPE_EVENTDEV 19 /**< Log related to eventdev. */ #define RTE_LOGTYPE_GSO 20 /**< Log related to GSO. */ +#define RTE_LOGTYPE_TQS 21 /**< Log related to Thread Quiescent State. */ /* these log types can be used in an application */ #define RTE_LOGTYPE_USER1 24 /**< User-defined log type 1. */ From patchwork Thu Nov 22 03:30:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 151714 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp219111ljp; Wed, 21 Nov 2018 19:32:02 -0800 (PST) X-Google-Smtp-Source: AFSGD/VQeWiE2d3/Cg7sIng2K2ejeBXES82yXc2ABOvnNkTfijgt4SqAs6CC5DaGqJbobxCYl0I4 X-Received: by 2002:a50:9ac4:: with SMTP id p62mr7868270edb.179.1542857521956; Wed, 21 Nov 2018 19:32:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542857521; cv=none; d=google.com; s=arc-20160816; b=E0BRXsNhiaciUIsCfqskXJwxuuGMj9O3WZND3FnKKE+dhGv4ibj7CTtDNHslmTU5lW JKspI0WfR0A9/ITmkkL/Da63Cy0bZwxlAdS1j5hjPrgaLAuFOx5gvitR+JBil8Kf6qJN OXIQwooKob/xNpa4K0KE9hG+EaggFHIUGITYOSSQNDFPoWZtIx0HWo/iHouKh+l9v9H+ lZ4JXfyxz4wCZHZNLtEGxrwUcCCR3EBlQQmS29NYTmLtFnX7RbnEYFdkNX613pzv0EHx ZAPfczfNniqa6DPp7cDBcLKaZs/sUyFUXLbY5x+l3XmsV/Y3lH9Erny8GFETZMsv+s1B E2Bw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=bKI0jvN283Rhnjs0FmeZVBKCSEdBxJt9iIlRmWgh2rg=; b=vumUEAuSjWwDW6VWl8OJ4eHYFiGjYCreRHQkRhc0W5PqQBOJCcXlXTIqC+Vi8ZtrTB 0cEEUiX8jNIblUpcQW0idBblW9+URPctjJwGnu6PbVx9K0quNLHEl3fXWndEnYzMGlW+ v2ZFB244ewiVTv5PGtl8gXySBEImpUr85NArG+pUyea1cAKkIP5gaYX3/OXWzxFekAuF 84F8QWEdpffGBOZsngVwxp4btUlF/UBPnHokFD3WAPeq1pQZFojT2WlytXf79WxlKpsZ qi9r/c0B1Gbq1siz4TmH8Nb5S4Al2t1xJpKFykgtPD54tRHGDgZJoKtQlVBbEo6tGV0H +P7Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id e20-v6si8433213ejq.231.2018.11.21.19.32.01; Wed, 21 Nov 2018 19:32:01 -0800 (PST) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BA4491B204; Thu, 22 Nov 2018 04:31:55 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id 9CE461B202 for ; Thu, 22 Nov 2018 04:31:53 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E3D933617; Wed, 21 Nov 2018 19:31:52 -0800 (PST) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.14.34]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 86B203F5CF; Wed, 21 Nov 2018 19:31:52 -0800 (PST) From: Honnappa Nagarahalli To: dev@dpdk.org Cc: nd@arm.com, honnappa.nagarahalli@arm.com, dharmik.thakkar@arm.com, malvika.gupta@arm.com, gavin.hu@arm.com Date: Wed, 21 Nov 2018 21:30:54 -0600 Message-Id: <20181122033055.3431-3-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181122033055.3431-1-honnappa.nagarahalli@arm.com> References: <20181122033055.3431-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [RFC 2/3] tqs: add thread quiescent state library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Thread Quiescent State (TQS) library. This library helps identify the quiescent state of the reader threads so that the writers can free the memory associated with the lock less data structures. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Steve Capper Reviewed-by: Gavin Hu --- config/common_base | 6 + lib/Makefile | 2 + lib/librte_tqs/Makefile | 23 ++ lib/librte_tqs/meson.build | 5 + lib/librte_tqs/rte_tqs.c | 249 ++++++++++++++++++++ lib/librte_tqs/rte_tqs.h | 352 +++++++++++++++++++++++++++++ lib/librte_tqs/rte_tqs_version.map | 16 ++ lib/meson.build | 2 +- mk/rte.app.mk | 1 + 9 files changed, 655 insertions(+), 1 deletion(-) create mode 100644 lib/librte_tqs/Makefile create mode 100644 lib/librte_tqs/meson.build create mode 100644 lib/librte_tqs/rte_tqs.c create mode 100644 lib/librte_tqs/rte_tqs.h create mode 100644 lib/librte_tqs/rte_tqs_version.map -- 2.17.1 diff --git a/config/common_base b/config/common_base index d12ae98bc..af40a9f81 100644 --- a/config/common_base +++ b/config/common_base @@ -792,6 +792,12 @@ CONFIG_RTE_LIBRTE_LATENCY_STATS=y # CONFIG_RTE_LIBRTE_TELEMETRY=n +# +# Compile librte_tqs +# +CONFIG_RTE_LIBRTE_TQS=y +CONFIG_RTE_LIBRTE_TQS_DEBUG=n + # # Compile librte_lpm # diff --git a/lib/Makefile b/lib/Makefile index b7370ef97..7095eac88 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -108,6 +108,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_BPF) += librte_bpf DEPDIRS-librte_bpf := librte_eal librte_mempool librte_mbuf librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry DEPDIRS-librte_telemetry := librte_eal librte_metrics librte_ethdev +DIRS-$(CONFIG_RTE_LIBRTE_TQS) += librte_tqs +DEPDIRS-librte_tqs := librte_eal ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y) DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni diff --git a/lib/librte_tqs/Makefile b/lib/librte_tqs/Makefile new file mode 100644 index 000000000..059de53e2 --- /dev/null +++ b/lib/librte_tqs/Makefile @@ -0,0 +1,23 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Arm Limited + +include $(RTE_SDK)/mk/rte.vars.mk + +# library name +LIB = librte_tqs.a + +CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 +LDLIBS += -lrte_eal + +EXPORT_MAP := rte_tqs_version.map + +LIBABIVER := 1 + +# all source are stored in SRCS-y +SRCS-$(CONFIG_RTE_LIBRTE_RING) := rte_tqs.c + +# install includes +SYMLINK-$(CONFIG_RTE_LIBRTE_RING)-include := rte_tqs.h + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_tqs/meson.build b/lib/librte_tqs/meson.build new file mode 100644 index 000000000..dd696ab07 --- /dev/null +++ b/lib/librte_tqs/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Arm Limited + +sources = files('rte_tqs.c') +headers = files('rte_tqs.h') diff --git a/lib/librte_tqs/rte_tqs.c b/lib/librte_tqs/rte_tqs.c new file mode 100644 index 000000000..cbc36864e --- /dev/null +++ b/lib/librte_tqs/rte_tqs.c @@ -0,0 +1,249 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * + * Copyright (c) 2018 Arm Limited + */ + +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "rte_tqs.h" + +TAILQ_HEAD(rte_tqs_list, rte_tailq_entry); + +static struct rte_tailq_elem rte_tqs_tailq = { + .name = RTE_TAILQ_TQS_NAME, +}; +EAL_REGISTER_TAILQ(rte_tqs_tailq) + +/* Allocate a new TQS variable with the name *name* in memory. */ +struct rte_tqs * __rte_experimental +rte_tqs_alloc(const char *name, int socket_id, uint64_t lcore_mask) +{ + char tqs_name[RTE_TQS_NAMESIZE]; + struct rte_tailq_entry *te, *tmp_te; + struct rte_tqs_list *tqs_list; + struct rte_tqs *v, *tmp_v; + int ret; + + if (name == NULL) { + RTE_LOG(ERR, TQS, "Invalid input parameters\n"); + rte_errno = -EINVAL; + return NULL; + } + + te = rte_zmalloc("TQS_TAILQ_ENTRY", sizeof(*te), 0); + if (te == NULL) { + RTE_LOG(ERR, TQS, "Cannot reserve memory for tailq\n"); + rte_errno = -ENOMEM; + return NULL; + } + + snprintf(tqs_name, sizeof(tqs_name), "%s", name); + v = rte_zmalloc_socket(tqs_name, sizeof(struct rte_tqs), + RTE_CACHE_LINE_SIZE, socket_id); + if (v == NULL) { + RTE_LOG(ERR, TQS, "Cannot reserve memory for TQS variable\n"); + rte_errno = -ENOMEM; + goto alloc_error; + } + + ret = snprintf(v->name, sizeof(v->name), "%s", name); + if (ret < 0 || ret >= (int)sizeof(v->name)) { + rte_errno = -ENAMETOOLONG; + goto alloc_error; + } + + te->data = (void *) v; + v->lcore_mask = lcore_mask; + + rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + + tqs_list = RTE_TAILQ_CAST(rte_tqs_tailq.head, rte_tqs_list); + + /* Search if a TQS variable with the same name exists already */ + TAILQ_FOREACH(tmp_te, tqs_list, next) { + tmp_v = (struct rte_tqs *) tmp_te->data; + if (strncmp(name, tmp_v->name, RTE_TQS_NAMESIZE) == 0) + break; + } + + if (tmp_te != NULL) { + rte_errno = -EEXIST; + goto tqs_exist; + } + + TAILQ_INSERT_TAIL(tqs_list, te, next); + + rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + + return v; + +tqs_exist: + rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + +alloc_error: + rte_free(te); + rte_free(v); + return NULL; +} + +/* De-allocate all the memory used by a TQS variable. */ +void __rte_experimental +rte_tqs_free(struct rte_tqs *v) +{ + struct rte_tqs_list *tqs_list; + struct rte_tailq_entry *te; + + /* Search for the TQS variable in tailq */ + tqs_list = RTE_TAILQ_CAST(rte_tqs_tailq.head, rte_tqs_list); + rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK); + + TAILQ_FOREACH(te, tqs_list, next) { + if (te->data == (void *) v) + break; + } + + if (te != NULL) + TAILQ_REMOVE(tqs_list, te, next); + else + RTE_LOG(ERR, TQS, "TQS variable %s not found\n", v->name); + + rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK); + + rte_free(te); + rte_free(v); +} + +/* Add a reader thread, running on an lcore, to the list of threads + * reporting their quiescent state on a TQS variable. + */ +int __rte_experimental +rte_tqs_register_lcore(struct rte_tqs *v, unsigned int lcore_id) +{ + TQS_RETURN_IF_TRUE((v == NULL || lcore_id >= RTE_TQS_MAX_LCORE), + -EINVAL); + + /* Worker thread has to count the quiescent states + * only from the current value of token. + */ + v->w[lcore_id].cnt = v->token; + + /* Release the store to initial TQS count so that workers + * can use it immediately after this function returns. + */ + __atomic_fetch_or(&v->lcore_mask, (1UL << lcore_id), __ATOMIC_RELEASE); + + return 0; +} + +/* Remove a reader thread, running on an lcore, from the list of threads + * reporting their quiescent state on a TQS variable. + */ +int __rte_experimental +rte_tqs_unregister_lcore(struct rte_tqs *v, unsigned int lcore_id) +{ + TQS_RETURN_IF_TRUE((v == NULL || + lcore_id >= RTE_TQS_MAX_LCORE), -EINVAL); + + /* This can be a relaxed store. Since this is an API, make sure + * the store is not reordered with other memory operations. + */ + __atomic_fetch_and(&v->lcore_mask, + ~(1UL << lcore_id), __ATOMIC_RELEASE); + + return 0; +} + +/* Search a TQS variable, given its name. */ +struct rte_tqs * __rte_experimental +rte_tqs_lookup(const char *name) +{ + struct rte_tqs_list *tqs_list; + struct rte_tailq_entry *te; + struct rte_tqs *v; + + if (name == NULL) { + RTE_LOG(ERR, TQS, "Invalid input parameters\n"); + rte_errno = -EINVAL; + return NULL; + } + + v = NULL; + tqs_list = RTE_TAILQ_CAST(rte_tqs_tailq.head, rte_tqs_list); + + rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + + /* find out tailq entry */ + TAILQ_FOREACH(te, tqs_list, next) { + v = (struct rte_tqs *) te->data; + if (strncmp(name, v->name, RTE_TQS_NAMESIZE) == 0) + break; + } + + if (te == NULL) { + rte_errno = -ENOENT; + v = NULL; + } + + rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); + + return v; +} + +/* Dump the details of a single TQS variables to a file. */ +void __rte_experimental +rte_tqs_dump(FILE *f, struct rte_tqs *v) +{ + uint64_t tmp_mask; + uint32_t i; + + TQS_ERR_LOG_IF_TRUE(v == NULL || f == NULL); + + fprintf(f, "\nTQS <%s>@%p\n", v->name, v); + fprintf(f, " lcore mask = 0x%lx\n", v->lcore_mask); + fprintf(f, " token = %u\n", v->token); + + if (v->lcore_mask != 0) { + fprintf(f, "Quiescent State Counts for readers:\n"); + tmp_mask = v->lcore_mask; + while (tmp_mask) { + i = __builtin_ctz(tmp_mask); + fprintf(f, "lcore # = %d, count = %u\n", + i, v->w[i].cnt); + tmp_mask &= ~(1UL << i); + } + } +} + +/* Dump the details of all the TQS variables to a file. */ +void __rte_experimental +rte_tqs_list_dump(FILE *f) +{ + const struct rte_tailq_entry *te; + struct rte_tqs_list *tqs_list; + + TQS_ERR_LOG_IF_TRUE(f == NULL); + + tqs_list = RTE_TAILQ_CAST(rte_tqs_tailq.head, rte_tqs_list); + + rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK); + + TAILQ_FOREACH(te, tqs_list, next) { + rte_tqs_dump(f, (struct rte_tqs *) te->data); + } + + rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK); +} diff --git a/lib/librte_tqs/rte_tqs.h b/lib/librte_tqs/rte_tqs.h new file mode 100644 index 000000000..9136418d2 --- /dev/null +++ b/lib/librte_tqs/rte_tqs.h @@ -0,0 +1,352 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2018 Arm Limited + */ + +#ifndef _RTE_TQS_H_ +#define _RTE_TQS_H_ + +/** + * @file + * RTE Thread Quiescent State + * + * Thread Quiescent State (TQS) is any point in the thread execution + * where the thread does not hold a reference to shared memory, i.e. + * a non-critical section. A critical section for a data structure can + * be a quiescent state for another data structure. + * + * An application can identify the quiescent state according to its + * needs. It can identify 1 quiescent state for each data structure or + * 1 quiescent state for a group/all of data structures. + * + * This library provides the flexibility for these use cases. + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include +#include +#include +#include + +#define RTE_TAILQ_TQS_NAME "RTE_TQS" + +/**< The maximum length of a TQS variable name. */ +#define RTE_TQS_NAMESIZE 32 + +/**< Maximum number of lcores supported. */ +#if (RTE_MAX_LCORE > 64) +#define RTE_TQS_MAX_LCORE 64 +#else +#define RTE_TQS_MAX_LCORE RTE_MAX_LCORE +#endif + +/* Macro for run-time checking of function parameters */ +#if defined(RTE_LIBRTE_TQS_DEBUG) +#define TQS_RETURN_IF_TRUE(cond, retval) do { \ + if ((cond)) \ + return retval; \ +} while (0) +#else +#define TQS_RETURN_IF_TRUE(cond, retval) +#endif + +/* Macro to log error message */ +#define TQS_ERR_LOG_IF_TRUE(cond) do { \ + if ((cond)) { \ + RTE_LOG(ERR, TQS, "Invalid parameters\n"); \ + return; \ + } \ +} while (0) + +/* Worker thread counter */ +struct rte_tqs_cnt { + volatile uint32_t cnt; /**< Quiescent state counter. */ +} __rte_cache_aligned; + +/** + * RTE Thread Quiescent State structure. + */ +struct rte_tqs { + char name[RTE_TQS_NAMESIZE]; + /**< Name of the ring. */ + uint64_t lcore_mask; + /**< Worker lcores reporting on this TQS */ + + uint32_t token __rte_cache_aligned; + /**< Counter to allow for multiple simultaneous TQS queries */ + + struct rte_tqs_cnt w[RTE_TQS_MAX_LCORE] __rte_cache_aligned; + /**< TQS counter for each worker thread, counts upto + * current value of token. + */ +} __rte_cache_aligned; + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Allocate a new TQS variable with the name *name* in memory. + * + * The TQS variable is added in RTE_TAILQ_TQS list. + * + * @param name + * The name of the TQS variable. + * @param socket_id + * The *socket_id* argument is the socket identifier in case of + * NUMA. The value can be *SOCKET_ID_ANY* if there is no NUMA + * constraint for the reserved zone. + * @param lcore_mask + * Data plane reader threads in this mask will report their quiescent + * state on this TQS variable. + * @return + * On success, the pointer to the new allocated TQS variable. NULL on + * error with rte_errno set appropriately. Possible errno values include: + * - EINVAL - invalid input parameters + * - ENAMETOOLONG - TQS variable name is longer than RTE_TQS_NAMESIZE + * - EEXIST - a TQS variable with the same name already exists + * - ENOMEM - no appropriate memory area found in which to create memzone + */ +struct rte_tqs * __rte_experimental +rte_tqs_alloc(const char *name, int socket_id, uint64_t lcore_mask); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * De-allocate all the memory used by a TQS variable. It is the + * application's responsibility to make sure that no other thread + * is using the TQS variable. + * + * The TQS variable is removed from RTE_TAILQ_TQS list. + * + * @param v + * TQS variable to free. + */ +void __rte_experimental rte_tqs_free(struct rte_tqs *v); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Add a worker thread, running on an lcore, to the list of threads + * reporting their quiescent state on a TQS variable. + * + * This is implemented as a lock-free function. It is multi-thread + * safe. This API can be called from the worker threads during + * initialization. Any ongoing TQS queries may wait for the + * status from this registered worker thread. + * + * @param v + * TQS variable + * @param lcore_id + * Worker thread on this lcore will report its quiescent state on + * this TQS variable. + * @return + * - 0 if registered successfully. + * - -EINVAL if the parameters are invalid (debug mode compilation only). + */ +int __rte_experimental +rte_tqs_register_lcore(struct rte_tqs *v, unsigned int lcore_id); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Remove a worker thread, running on an lcore, from the list of threads + * reporting their quiescent state on a TQS variable. + * + * This is implemented as a lock-free function. It is multi-thread safe. + * This API can be called from the worker threads during shutdown. + * Any ongoing TQS queries may stop waiting for the status from this + * unregistered worker thread. + * + * @param v + * TQS variable + * @param lcore_id + * Worker thread on this lcore will stop reporting its quiescent state + * on this TQS variable. + * @return + * - 0 if un-registered successfully. + * - -EINVAL if the parameters are invalid (debug mode compilation only). + */ +int __rte_experimental +rte_tqs_unregister_lcore(struct rte_tqs *v, unsigned int lcore_id); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Trigger the worker threads to report the quiescent state + * status. + * + * This is implemented as a lock-free function. It is multi-thread + * safe and can be called from the worker threads as well. + * + * @param v + * TQS variable + * @param n + * Expected number of times the quiescent state is entered + * @param t + * - If successful, this is the token for this call of the API. + * This should be passed to rte_tqs_check API. + * @return + * - -EINVAL if the parameters are invalid (debug mode compilation only). + * - 0 Otherwise and always (non-debug mode compilation). + */ +static __rte_always_inline int __rte_experimental +rte_tqs_start(struct rte_tqs *v, unsigned int n, uint32_t *t) +{ + TQS_RETURN_IF_TRUE((v == NULL || t == NULL), -EINVAL); + + /* This store release will ensure that changes to any data + * structure are visible to the workers before the token + * update is visible. + */ + *t = __atomic_add_fetch(&v->token, n, __ATOMIC_RELEASE); + + return 0; +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Update quiescent state for the worker thread on a lcore. + * + * This is implemented as a lock-free function. It is multi-thread safe. + * All the worker threads registered to report their quiescent state + * on the TQS variable must call this API. + * + * @param v + * TQS variable + */ +static __rte_always_inline void __rte_experimental +rte_tqs_update(struct rte_tqs *v, unsigned int lcore_id) +{ + uint32_t t; + + TQS_ERR_LOG_IF_TRUE(v == NULL || lcore_id >= RTE_TQS_MAX_LCORE); + + /* Load the token before the worker thread loads any other + * (lock-free) data structure. This ensures that updates + * to the data structures are visible if the update + * to token is visible. + */ + t = __atomic_load_n(&v->token, __ATOMIC_ACQUIRE); + if (v->w[lcore_id].cnt != t) + v->w[lcore_id].cnt++; +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Checks if all the worker threads have entered the quiescent state + * 'n' number of times. 'n' is provided in rte_tqs_start API. + * + * This is implemented as a lock-free function. It is multi-thread + * safe and can be called from the worker threads as well. + * + * @param v + * TQS variable + * @param t + * Token returned by rte_tqs_start API + * @param wait + * If true, block till all the worker threads have completed entering + * the quiescent state 'n' number of times + * @return + * - 0 if all worker threads have NOT passed through specified number + * of quiescent states. + * - 1 if all worker threads have passed through specified number + * of quiescent states. + * - -EINVAL if the parameters are invalid (debug mode compilation only). + */ +static __rte_always_inline int __rte_experimental +rte_tqs_check(struct rte_tqs *v, uint32_t t, bool wait) +{ + uint64_t l; + uint64_t lcore_mask; + + TQS_RETURN_IF_TRUE((v == NULL), -EINVAL); + + do { + /* Load the current lcore_mask before loading the + * worker thread quiescent state counters. + */ + lcore_mask = __atomic_load_n(&v->lcore_mask, __ATOMIC_ACQUIRE); + + while (lcore_mask) { + l = __builtin_ctz(lcore_mask); + if (v->w[l].cnt != t) + break; + + lcore_mask &= ~(1UL << l); + } + + if (lcore_mask == 0) + return 1; + + rte_pause(); + } while (wait); + + return 0; +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Search a TQS variable, given its name. + * + * It is multi-thread safe. + * + * @param name + * The name of the TQS variable. + * @return + * On success, the pointer to the TQS variable. NULL on + * error with rte_errno set appropriately. Possible errno values include: + * - EINVAL - invalid input parameters. + * - ENOENT - entry not found. + */ +struct rte_tqs * __rte_experimental +rte_tqs_lookup(const char *name); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Dump the details of a single TQS variables to a file. + * + * It is NOT multi-thread safe. + * + * @param f + * A pointer to a file for output + * @param tqs + * TQS variable + */ +void __rte_experimental +rte_tqs_dump(FILE *f, struct rte_tqs *v); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Dump the details of all the TQS variables to a file. + * + * It is multi-thread safe. + * + * @param f + * A pointer to a file for output + */ +void __rte_experimental +rte_tqs_list_dump(FILE *f); + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_TQS_H_ */ diff --git a/lib/librte_tqs/rte_tqs_version.map b/lib/librte_tqs/rte_tqs_version.map new file mode 100644 index 000000000..2e4d5c094 --- /dev/null +++ b/lib/librte_tqs/rte_tqs_version.map @@ -0,0 +1,16 @@ +EXPERIMENTAL { + global: + + rte_tqs_alloc; + rte_tqs_free; + rte_tqs_register_lcore; + rte_tqs_unregister_lcore; + rte_tqs_start; + rte_tqs_update; + rte_tqs_check; + rte_tqs_lookup; + rte_tqs_list_dump; + rte_tqs_dump; + + local: *; +}; diff --git a/lib/meson.build b/lib/meson.build index bb7f443f9..ee19c483e 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -21,7 +21,7 @@ libraries = [ 'compat', # just a header, used for versioning 'gro', 'gso', 'ip_frag', 'jobstats', 'kni', 'latencystats', 'lpm', 'member', 'meter', 'power', 'pdump', 'rawdev', - 'reorder', 'sched', 'security', 'vhost', + 'reorder', 'sched', 'security', 'tqs', 'vhost', # add pkt framework libs which use other libs from above 'port', 'table', 'pipeline', # flow_classify lib depends on pkt framework table lib diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 3ebc4e64c..6e19e669a 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -92,6 +92,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrte_sched +_LDLIBS-$(CONFIG_RTE_LIBRTE_TQS) += -lrte_tqs ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_KNI) += -lrte_kni From patchwork Thu Nov 22 03:30:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 151715 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp219275ljp; Wed, 21 Nov 2018 19:32:15 -0800 (PST) X-Google-Smtp-Source: AFSGD/VczhPHZZkHXhnHebDdsi0+Vbecf0f60yi8tf9OVMNO/3RADVbY2JqjtXX6f2WHd9PUf40w X-Received: by 2002:a50:9665:: with SMTP id y92mr7522242eda.282.1542857535663; Wed, 21 Nov 2018 19:32:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542857535; cv=none; d=google.com; s=arc-20160816; b=I6QP0d09AOUW7A1bbGvFc9jQo8RH2XPxR90iMzzB40k161IpJ9NnOJa9hZTjQ5xC6L AtBtP2r1aRXxWhi/29FZDt5gT0MLyoH8W1MPOJR12RNLEaPLb0JWwXyASv9+49t+4aAu HfG3oiHrlFGjuVqe+VQjUCrPghbufqzrVMXLm/sFI/2wnSM8tZfFs4uS//xOcrwz49zB kvlvNFHbMsXX0gRvb7zTkgoh3qS/Ub9xD53niOfO4bxWYRSeUJINU/piVlXYZD1a5a7I MxIUHPM9lJZU38Fsgjr9BFPW8eRogkHe5AZfKMM8sCOtbRmZ9Cy+DsP3/KD8j8be7pSP I8JA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=LJr8zQ6dW41Tjgz1khjKu/c9oFcgZihHl7DWIKp6eRI=; b=xhvkuYmwAPlu03cVY8FUlEJg2DJvXt1qlIcbYDVtnSVePHJmtjoNAGhrqr0QHmzCR1 ClGijtK1ICLm3TTl0qgB0j1pwia265q+J6VOjRFslCOmpbxYXa2GORco2Hw4Ob5egqoj 2z23uI7PvnrUVB00/y9GreWM7LNqCNrzjeQu0qkjqqK6pZ8iYgiqLs9KBRiAO+kzbkcj pED88rE6cnNoLcwLFWfRozUvguemv9HtInZ61sXLiuWi8+lypmbUAZiXe34bTlly5CAV 4hwAQOhfxgoqxiny8rs5aQGDhoRrJ2g9kCnQUxF1PiTNwOg0BGnmyWcPFRoNupA3oWTz izDg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id j29-v6si8348188ejo.195.2018.11.21.19.32.15; Wed, 21 Nov 2018 19:32:15 -0800 (PST) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BA20E1B216; Thu, 22 Nov 2018 04:31:57 +0100 (CET) Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id ADBDD1B1FC for ; Thu, 22 Nov 2018 04:31:55 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 282173617; Wed, 21 Nov 2018 19:31:55 -0800 (PST) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.14.34]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C52DB3F5CF; Wed, 21 Nov 2018 19:31:54 -0800 (PST) From: Honnappa Nagarahalli To: dev@dpdk.org Cc: nd@arm.com, honnappa.nagarahalli@arm.com, dharmik.thakkar@arm.com, malvika.gupta@arm.com, gavin.hu@arm.com Date: Wed, 21 Nov 2018 21:30:55 -0600 Message-Id: <20181122033055.3431-4-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181122033055.3431-1-honnappa.nagarahalli@arm.com> References: <20181122033055.3431-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [RFC 3/3] test/tqs: Add API and functional tests X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Dharmik Thakkar Add API positive/negative test cases and functional tests. Signed-off-by: Malvika Gupta Signed-off-by: Dharmik Thakkar Signed-off-by: Honnappa Nagarahalli Reviewed-by: Gavin Hu --- test/test/Makefile | 2 + test/test/autotest_data.py | 6 + test/test/meson.build | 5 +- test/test/test_tqs.c | 540 +++++++++++++++++++++++++++++++++++++ 4 files changed, 552 insertions(+), 1 deletion(-) create mode 100644 test/test/test_tqs.c -- 2.17.1 diff --git a/test/test/Makefile b/test/test/Makefile index ab4fec34a..7a07039e7 100644 --- a/test/test/Makefile +++ b/test/test/Makefile @@ -207,6 +207,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_KVARGS) += test_kvargs.c SRCS-$(CONFIG_RTE_LIBRTE_BPF) += test_bpf.c +SRCS-$(CONFIG_RTE_LIBRTE_TQS) += test_tqs.c + CFLAGS += -DALLOW_EXPERIMENTAL_API CFLAGS += -O3 diff --git a/test/test/autotest_data.py b/test/test/autotest_data.py index 0fb7866db..e676757cd 100644 --- a/test/test/autotest_data.py +++ b/test/test/autotest_data.py @@ -676,6 +676,12 @@ "Func": default_autotest, "Report": None, }, + { + "Name": "Thread Quiescent State autotest", + "Command": "tqs_autotest", + "Func": default_autotest, + "Report": None, + }, # # Please always make sure that ring_perf is the last test! # diff --git a/test/test/meson.build b/test/test/meson.build index 554e9945f..b80a449ad 100644 --- a/test/test/meson.build +++ b/test/test/meson.build @@ -100,6 +100,7 @@ test_sources = files('commands.c', 'test_timer.c', 'test_timer_perf.c', 'test_timer_racecond.c', + 'tet_tqs.c', 'test_version.c', 'virtual_pmd.c' ) @@ -122,7 +123,8 @@ test_deps = ['acl', 'port', 'reorder', 'ring', - 'timer' + 'timer', + 'tqs' ] test_names = [ @@ -228,6 +230,7 @@ test_names = [ 'timer_autotest', 'timer_perf__autotest', 'timer_racecond_autotest', + 'tqs_autotest', 'user_delay_us', 'version_autotest', ] diff --git a/test/test/test_tqs.c b/test/test/test_tqs.c new file mode 100644 index 000000000..8633a80a6 --- /dev/null +++ b/test/test/test_tqs.c @@ -0,0 +1,540 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2018 Arm Limited + */ + +#include +#include +#include +#include +#include +#include +#include + +#include "test.h" + +/* Check condition and return an error if true. */ +#define TQS_RETURN_IF_ERROR(tqs, cond, str, ...) do { \ + if (cond) { \ + printf("ERROR file %s, line %d: " str "\n", __FILE__, \ + __LINE__, ##__VA_ARGS__); \ + if (tqs) \ + rte_tqs_free(tqs); \ + return -1; \ + } \ +} while (0) + +#define RTE_TQS_MAX_LCORE 64 +uint16_t enabled_core_ids[RTE_TQS_MAX_LCORE]; +uint64_t core_mask; +uint32_t sw_token; +uint16_t num_1qs = 1; /* Number of quiescent states = 1 */ + +struct node { + int data; + struct node *next; +}; + +struct node *head = NULL, *lastNode; + +static inline int +get_enabled_cores_mask(void) +{ + uint32_t i; + uint16_t core_id; + uint32_t max_cores = rte_lcore_count(); + + if (max_cores > RTE_TQS_MAX_LCORE) { + printf("Number of cores exceed %d\n", RTE_TQS_MAX_LCORE); + return -1; + } + + core_id = 0; + core_mask = 0; + i = 0; + RTE_LCORE_FOREACH_SLAVE(core_id) { + enabled_core_ids[i] = core_id; + i++; + core_mask |= 1UL << core_id; + } + + return 0; +} + +/* + * rte_tqs_alloc: Allocate a TQS variable + */ +static int +test_tqs_alloc(void) +{ + const char name32[] = "xyzxyzxyzxyzxyzxyzxyzxyzxyzx123"; + const char name33[] = "xyzxyizxyzxyzxyzxyzxyzxyzxyzxyzxyz"; + const char name3[] = "xyz"; + struct rte_tqs *t; + + printf("Test rte_tqs_alloc()\n"); + + t = rte_tqs_alloc(NULL, SOCKET_ID_ANY, core_mask); + TQS_RETURN_IF_ERROR(t, (t != NULL), "NULL TQS variable"); + + t = rte_tqs_alloc(name3, SOCKET_ID_ANY, core_mask); + TQS_RETURN_IF_ERROR(t, (t == NULL), "Variable name < %d", + RTE_TQS_NAMESIZE); + rte_tqs_free(t); + + t = rte_tqs_alloc(name32, SOCKET_ID_ANY, core_mask); + TQS_RETURN_IF_ERROR(t, (t == NULL), "Variable name < %d", + RTE_TQS_NAMESIZE); + rte_tqs_free(t); + + t = rte_tqs_alloc(name33, SOCKET_ID_ANY, core_mask); + TQS_RETURN_IF_ERROR(t, (t != NULL), "Variable name > %d", + RTE_TQS_NAMESIZE); + + t = rte_tqs_alloc(name3, 0, core_mask); + TQS_RETURN_IF_ERROR(t, (t == NULL), "Valid socket ID"); + rte_tqs_free(t); + + t = rte_tqs_alloc(name3, 10000, core_mask); + TQS_RETURN_IF_ERROR(t, (t != NULL), "Invalid socket ID"); + + t = rte_tqs_alloc(name3, SOCKET_ID_ANY, 0); + TQS_RETURN_IF_ERROR(t, (t == NULL), "0 core mask"); + rte_tqs_free(t); + + return 0; +} + +/* + * rte_tqs_register_lcore: Register threads + */ +static int +test_tqs_register_lcore(void) +{ + struct rte_tqs *t; + const char *name = "TQS"; + int ret; + + printf("\nTest rte_tqs_register_lcore()\n"); + + t = rte_tqs_alloc(name, SOCKET_ID_ANY, 0); + TQS_RETURN_IF_ERROR(t, (t == NULL), "Failed to alloc TQS variable"); + + ret = rte_tqs_register_lcore(t, enabled_core_ids[0]); + TQS_RETURN_IF_ERROR(t, (ret == -EINVAL), + "lcore_id < RTE_TQS_MAX_LCORE"); + rte_tqs_free(t); + return 0; +} + +/* + * rte_tqs_unregister_lcore: Unregister threads + */ +static int +test_tqs_unregister_lcore(void) +{ + struct rte_tqs *t; + const char *name = "TQS"; + int i, ret; + + printf("\nTest rte_tqs_unregister_lcore()\n"); + + t = rte_tqs_alloc(name, SOCKET_ID_ANY, 0); + TQS_RETURN_IF_ERROR(t, (t == NULL), "Failed to alloc TQS variable"); + + ret = rte_tqs_register_lcore(t, enabled_core_ids[0]); + TQS_RETURN_IF_ERROR(t, (ret == -EINVAL), "Register lcore failed"); + + /* Find first disabled core */ + for (i = 0; i < RTE_TQS_MAX_LCORE; i++) { + if (enabled_core_ids[i] == 0) + break; + } + ret = rte_tqs_unregister_lcore(t, i); + TQS_RETURN_IF_ERROR(t, (ret != 0), "lcore disabled"); + + ret = rte_tqs_unregister_lcore(t, enabled_core_ids[0]); + TQS_RETURN_IF_ERROR(t, (ret == -EINVAL), "Valid lcore_id"); + + rte_tqs_free(t); + return 0; +} + +/* + * rte_tqs_start: Trigger reader threads to count the number of times they enter + * the quiescent state + */ +static int +test_tqs_start(void) +{ + struct rte_tqs *t; + const char *name = "TQS"; + uint32_t token; + int i, ret; + + printf("\nTest rte_tqs_start()\n"); + + t = rte_tqs_alloc(name, SOCKET_ID_ANY, 0); + TQS_RETURN_IF_ERROR(t, (t == NULL), "Failed to alloc TQS variable"); + + for (i = 0; i < 3; i++) { + ret = rte_tqs_register_lcore(t, enabled_core_ids[i]); + TQS_RETURN_IF_ERROR(t, (ret == -EINVAL), + "Register lcore failed"); + } + + ret = rte_tqs_start(t, 1, &token); + TQS_RETURN_IF_ERROR(t, (ret == -EINVAL), "1 quiescent states"); + + rte_tqs_free(t); + return 0; +} + +/* + * rte_tqs_check: Check if all reader threads have completed entering the + * quiescent state 'n' times + */ +static int +test_tqs_check(void) +{ + + struct rte_tqs *t; + const char *name = "TQS"; + int i, ret; + uint32_t token; + + printf("\nTest rte_tqs_check()\n"); + + t = rte_tqs_alloc(name, SOCKET_ID_ANY, 0); + TQS_RETURN_IF_ERROR(t, (t == NULL), "Failed to alloc TQS variable"); + + ret = rte_tqs_start(t, 1, &token); + TQS_RETURN_IF_ERROR(t, (ret == -EINVAL), "TQS Start failed"); + + ret = rte_tqs_check(t, 0, true); + TQS_RETURN_IF_ERROR(t, (ret == -EINVAL), "Token = 0"); + + ret = rte_tqs_check(t, token, true); + TQS_RETURN_IF_ERROR(t, (ret == -EINVAL), "Blocking TQS check"); + + for (i = 0; i < 3; i++) { + ret = rte_tqs_register_lcore(t, enabled_core_ids[i]); + TQS_RETURN_IF_ERROR(t, (ret == -EINVAL), + "Register lcore failed"); + } + + ret = rte_tqs_check(t, token, false); + TQS_RETURN_IF_ERROR(t, (ret == -EINVAL), "Non-blocking TQS check"); + + rte_tqs_free(t); + return 0; +} + +/* + * rte_tqs_lookup: Lookup a TQS variable by its name + */ +static int +test_tqs_lookup(void) +{ + struct rte_tqs *t, *ret; + const char *name1 = "TQS"; + const char *name2 = "NO_TQS"; + + printf("\nTest rte_tqs_lookup()\n"); + + t = rte_tqs_alloc(name1, SOCKET_ID_ANY, 0); + TQS_RETURN_IF_ERROR(t, (t == NULL), "Failed to alloc TQS variable"); + + ret = rte_tqs_lookup(name1); + TQS_RETURN_IF_ERROR(t, (ret != t), "Allocated TQS variable name"); + + ret = rte_tqs_lookup(name2); + TQS_RETURN_IF_ERROR(t, (ret != NULL), "Unallocated TQS variable name"); + + rte_tqs_free(t); + return 0; +} + +/* + * rte_tqs_list_dump: Dump the status of all TQS variables to a file + */ +static int +test_tqs_list_dump(void) +{ + struct rte_tqs *t1, *t2; + const char name1[] = "TQS_1"; + const char name2[] = "TQS_2"; + int i, ret; + + printf("\nTest rte_tqs_list_dump()\n"); + + /* File pointer NULL */ + rte_tqs_list_dump(NULL); + + /* Dump an empty list */ + rte_tqs_list_dump(stdout); + + t1 = rte_tqs_alloc(name1, SOCKET_ID_ANY, 0); + TQS_RETURN_IF_ERROR(t1, (t1 == NULL), "Failed to alloc TQS variable"); + + /* Dump a list with TQS variable that has no readers */ + rte_tqs_list_dump(stdout); + + t2 = rte_tqs_alloc(name2, SOCKET_ID_ANY, 0); + TQS_RETURN_IF_ERROR(t2, (t2 == NULL), "Failed to alloc TQS variable"); + + ret = rte_tqs_register_lcore(t1, enabled_core_ids[0]); + if (ret != 0) { + printf("ERROR file %s, line %d: Failed to register lcore\n", + __FILE__, __LINE__); + return -1; + } + + for (i = 1; i < 3; i++) { + ret = rte_tqs_register_lcore(t2, enabled_core_ids[i]); + if (ret != 0) { + printf("ERROR file %s, line %d: Failed to register lcore\n", + __FILE__, __LINE__); + return -1; + } + } + + rte_tqs_list_dump(stdout); + + rte_tqs_free(t1); + rte_tqs_free(t2); + + return 0; +} + +/* + * rte_tqs_dump: Dump status of a single TQS variable to a file + */ +static int +test_tqs_dump(void) +{ + struct rte_tqs *t1, *t2; + const char name1[] = "TQS_1"; + const char name2[] = "TQS_2"; + int i, ret; + + printf("\nTest rte_tqs_dump()\n"); + + /* NULL TQS variable */ + rte_tqs_dump(stdout, NULL); + + t1 = rte_tqs_alloc(name1, SOCKET_ID_ANY, 0); + TQS_RETURN_IF_ERROR(t1, (t1 == NULL), "Failed to alloc TQS variable"); + + /* NULL file pointer */ + rte_tqs_dump(NULL, t1); + + /* TQS variable with 0 core mask */ + rte_tqs_dump(stdout, t1); + + t2 = rte_tqs_alloc(name2, SOCKET_ID_ANY, 0); + TQS_RETURN_IF_ERROR(t2, (t2 == NULL), "Failed to alloc TQS variable"); + + ret = rte_tqs_register_lcore(t1, enabled_core_ids[0]); + if (ret != 0) { + printf("ERROR file %s, line %d: Failed to register lcore\n", + __FILE__, __LINE__); + return -1; + } + + for (i = 1; i < 3; i++) { + ret = rte_tqs_register_lcore(t2, enabled_core_ids[i]); + if (ret != 0) { + printf("ERROR file %s, line %d: Failed to register lcore\n", + __FILE__, __LINE__); + return -1; + } + } + + rte_tqs_dump(stdout, t1); + rte_tqs_dump(stdout, t2); + + return 0; +} + +struct rte_hash *handle; +static uint32_t *keys; +#define TOTAL_ENTRY (1 * 8) +#define COUNTER_VALUE 4096 +uint32_t *hash_data[TOTAL_ENTRY]; +uint8_t writer_done; +static int + +test_tqs_reader(__attribute__((unused)) void *arg) +{ + struct rte_tqs *t; + int i, ret; + uint32_t lcore_id = rte_lcore_id(); + + t = rte_tqs_lookup("TQS"); + TQS_RETURN_IF_ERROR(t, (t == NULL), "TQS variable lookup failed"); + + do { + for (i = 0; i < TOTAL_ENTRY; i++) { + uint32_t *pdata; + ret = rte_hash_lookup_data(handle, keys+i, + (void **)&pdata); + if (ret != -ENOENT) + while (*pdata < COUNTER_VALUE) + ++*pdata; + } + + /* Update quiescent state counter */ + rte_tqs_update(t, lcore_id); + } while (!writer_done); + + return 0; +} + +static int +init_hash(void) +{ + int i, ret; + struct rte_hash_parameters hash_params = { + .entries = TOTAL_ENTRY, + .key_len = sizeof(uint32_t), + .hash_func_init_val = 0, + .socket_id = rte_socket_id(), + .hash_func = rte_hash_crc, + .extra_flag = + RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF, + .name = "tests", + }; + + handle = rte_hash_create(&hash_params); + TQS_RETURN_IF_ERROR(NULL, (handle == NULL), "Hash create Failed"); + + for (i = 0; i < TOTAL_ENTRY; i++) { + hash_data[i] = rte_zmalloc(NULL, sizeof(uint32_t), 0); + TQS_RETURN_IF_ERROR(NULL, (hash_data[i] == NULL), "No memory"); + } + keys = rte_malloc(NULL, sizeof(uint32_t) * TOTAL_ENTRY, 0); + TQS_RETURN_IF_ERROR(NULL, (keys == NULL), "No memory"); + + for (i = 0; i < TOTAL_ENTRY; i++) + keys[i] = i; + + for (i = 0; i < TOTAL_ENTRY; i++) { + ret = rte_hash_add_key_data(handle, keys + i, + (void *)((uintptr_t)hash_data[i])); + TQS_RETURN_IF_ERROR(NULL, (ret < 0), + "Hash key add Failed #%d\n", i); + } + return 0; +} + +/* + * Single writer, 1 TQS variable, 1 quiescent state + */ +static int +test_tqs_sw_sv_1qs(void) +{ + struct rte_tqs *t; + const char *name = "TQS"; + static uint32_t token; + int i, ret; + int32_t pos; + writer_done = 0; + + printf("\nTest Single writer, 1 TQS variable, pass 1 quiescent state\n"); + + /* TQS variable is allocated */ + t = rte_tqs_alloc(name, SOCKET_ID_ANY, 0); + TQS_RETURN_IF_ERROR(t, (t == NULL), "Failed to alloc TQS variable"); + + /* Register worker threads on 4 cores */ + for (i = 0; i < 4; i++) { + ret = rte_tqs_register_lcore(t, enabled_core_ids[i]); + TQS_RETURN_IF_ERROR(t, (ret == -EINVAL), + "Register lcore failed"); + } + + /* Shared data structure created */ + ret = init_hash(); + TQS_RETURN_IF_ERROR(t, (ret != 0), "Hash init failed"); + + /* Reader threads are launched */ + for (i = 0; i < 4; i++) + rte_eal_remote_launch(test_tqs_reader, NULL, + enabled_core_ids[i]); + + for (i = 0; i < TOTAL_ENTRY; i++) { + /* Delete elements from the shared data structure */ + pos = rte_hash_del_key(handle, keys + i); + TQS_RETURN_IF_ERROR(t, (pos < 0), "Delete key failed #%d", + keys[i]); + /* Start the quiescent state query process */ + ret = rte_tqs_start(t, num_1qs, &token); + TQS_RETURN_IF_ERROR(t, (ret != 0), "TQS Start Failed"); + + /* Check the quiescent state status */ + rte_tqs_check(t, token, true); + TQS_RETURN_IF_ERROR(t, (*hash_data[i] != COUNTER_VALUE), + "Reader did not complete %d", + *hash_data[i]); + + rte_hash_free_key_with_position(handle, pos); + TQS_RETURN_IF_ERROR(t, (ret < 0), + "Failed to free the key #%d", keys[i]); + rte_free(hash_data[i]); + } + writer_done = 1; + /* Wait until all readers have exited */ + rte_eal_mp_wait_lcore(); + + /* Free TQS variable */ + rte_tqs_free(t); + rte_hash_free(handle); + rte_free(keys); + + return 0; +} + +static int +test_tqs_main(void) +{ + if (get_enabled_cores_mask() != 0) + return -1; + + /* Error-checking test cases */ + if (test_tqs_alloc() < 0) + goto test_fail; + + if (test_tqs_register_lcore() < 0) + goto test_fail; + + if (test_tqs_unregister_lcore() < 0) + goto test_fail; + + if (test_tqs_start() < 0) + goto test_fail; + + if (test_tqs_check() < 0) + goto test_fail; + + if (test_tqs_lookup() < 0) + goto test_fail; + + if (test_tqs_list_dump() < 0) + goto test_fail; + + if (test_tqs_dump() < 0) + goto test_fail; + + + /* Functional test cases */ + if (test_tqs_sw_sv_1qs() < 0) + goto test_fail; + + printf("\n"); + return 0; + + test_fail: + return -1; +} + +REGISTER_TEST_COMMAND(tqs_autotest, test_tqs_main);