From patchwork Wed Apr 22 03:30:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 185654 Delivered-To: patch@linaro.org Received: by 2002:a92:3d9a:0:0:0:0:0 with SMTP id k26csp252171ilf; Tue, 21 Apr 2020 20:31:06 -0700 (PDT) X-Google-Smtp-Source: APiQypKTTDl88d9i/4jeX1y0iS9BElJ6pnGhepWAbXPgCbRE2tXn0cqL0ctJMLhsnFD6HsJ06NbM X-Received: by 2002:a17:906:504f:: with SMTP id e15mr10885878ejk.45.1587526266620; Tue, 21 Apr 2020 20:31:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1587526266; cv=none; d=google.com; s=arc-20160816; b=wJdm3nkkDcECgZL7hWGnA+xNgGRGBVEupVgr/D/TatLTtvXMLR3hTCpprKDLKBsQo8 +S6nr8Y9AgAtavHDHyGTql1bglLgv34KIW5g084+vxTD/+KWCeUAm1xCEEvtQtzjzTiI /kUsCVU2iqFeV0aprhAj3GHqzr4F5fFJah/HBWfcECV1uArQ5yXxyj72ooRnSUv6HfYj UexxmYP28Ao8UtMTP5P+KR0JvwqDbFwQb63JzHZpbhEVxso27QBYvZfIsEcvBVxHSt8z jFO9iHZWdtjrCwu/DDEmACdvvq3U9h0Qok1wyZl/QwHBkC9x0oMaR92uXudmXpIPUX9e 1Klw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=iAB47r6rxYkkYFHG9J1csrZSLsgQ0belhtkRU1UblA4=; b=B3qqPikWYSBaWWpA8E4oho61wrAdQnOONYXQ4L7mMKoUzZzTZ32iYkWsQCKvcILtM3 Oo0pt77+T+4G4oFrhh0lXbeKS6V0iR0BdnYNulXNCEC+rp9mwVzJe/24QtNLpzkjgDKj d+wwN84MsNdkYoyUzwB3W8sugDPu/o3OivKn+TQU5jfC3TMXumwHbTU4vz1DpTO7RZe6 yFVJ9bRnQE9VjFSGQZbm3enyroMmuKJa+o9tHg/WQI7le6PI0muQ6lxXWp+2F3PWLjyU lq8Qn82Sb7UrmQCLRgGoXmRmEjR0MXR+g5Cd+ci21rJFuhVqY4AKHF8NR0AxQ1WWltUQ yb8w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id ck5si2739822ejb.12.2020.04.21.20.31.06; Tue, 21 Apr 2020 20:31:06 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 12B9B1D176; Wed, 22 Apr 2020 05:30:43 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 7ADFC1C2F7 for ; Wed, 22 Apr 2020 05:30:38 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D843130E; Tue, 21 Apr 2020 20:30:32 -0700 (PDT) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.14.48]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A45E63F6CF; Tue, 21 Apr 2020 20:30:32 -0700 (PDT) From: Honnappa Nagarahalli To: konstantin.ananyev@intel.com, stephen@networkplumber.org, vladimir.medvedkin@intel.com, dev@dpdk.org Cc: honnappa.nagarahalli@arm.com, david.marchand@redhat.com, ruifeng.wang@arm.com, dharmik.thakkar@arm.com, nd@arm.com Date: Tue, 21 Apr 2020 22:30:03 -0500 Message-Id: <20200422033006.1124-2-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200422033006.1124-1-honnappa.nagarahalli@arm.com> References: <20191001062917.35578-1-honnappa.nagarahalli@arm.com> <20200422033006.1124-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [PATCH v5 1/4] lib/rcu: add resource reclamation APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add resource reclamation APIs to make it simple for applications and libraries to integrate rte_rcu library. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Ola Liljedhal Reviewed-by: Ruifeng Wang Acked-by: Konstantin Ananyev --- lib/librte_rcu/Makefile | 2 +- lib/librte_rcu/meson.build | 7 + lib/librte_rcu/rcu_qsbr_pvt.h | 66 +++++++++ lib/librte_rcu/rte_rcu_qsbr.c | 227 ++++++++++++++++++++++++++++- lib/librte_rcu/rte_rcu_qsbr.h | 194 +++++++++++++++++++++++- lib/librte_rcu/rte_rcu_version.map | 4 + lib/meson.build | 6 +- 7 files changed, 501 insertions(+), 5 deletions(-) create mode 100644 lib/librte_rcu/rcu_qsbr_pvt.h -- 2.17.1 Acked-by: Konstantin Ananyev diff --git a/lib/librte_rcu/Makefile b/lib/librte_rcu/Makefile index 728669975..553bca2ef 100644 --- a/lib/librte_rcu/Makefile +++ b/lib/librte_rcu/Makefile @@ -7,7 +7,7 @@ include $(RTE_SDK)/mk/rte.vars.mk LIB = librte_rcu.a CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -LDLIBS += -lrte_eal +LDLIBS += -lrte_eal -lrte_ring EXPORT_MAP := rte_rcu_version.map diff --git a/lib/librte_rcu/meson.build b/lib/librte_rcu/meson.build index c009ae4b7..3eb2ace17 100644 --- a/lib/librte_rcu/meson.build +++ b/lib/librte_rcu/meson.build @@ -3,3 +3,10 @@ sources = files('rte_rcu_qsbr.c') headers = files('rte_rcu_qsbr.h') + +# for clang 32-bit compiles we need libatomic for 64-bit atomic ops +if cc.get_id() == 'clang' and dpdk_conf.get('RTE_ARCH_64') == false + ext_deps += cc.find_library('atomic') +endif + +deps += ['ring'] diff --git a/lib/librte_rcu/rcu_qsbr_pvt.h b/lib/librte_rcu/rcu_qsbr_pvt.h new file mode 100644 index 000000000..63f7a5fff --- /dev/null +++ b/lib/librte_rcu/rcu_qsbr_pvt.h @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2020 Arm Limited + */ + +#ifndef _RTE_RCU_QSBR_PVT_H_ +#define _RTE_RCU_QSBR_PVT_H_ + +/** + * This file is private to the RCU library. It should not be included + * by the user of this library. + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include + +#include "rte_rcu_qsbr.h" + +/* Defer queue structure. + * This structure holds the defer queue. The defer queue is used to + * hold the deleted entries from the data structure that are not + * yet freed. + */ +struct rte_rcu_qsbr_dq { + struct rte_rcu_qsbr *v; /**< RCU QSBR variable used by this queue.*/ + struct rte_ring *r; /**< RCU QSBR defer queue. */ + uint32_t size; + /**< Number of elements in the defer queue */ + uint32_t esize; + /**< Size (in bytes) of data, including the token, stored on the + * defer queue. + */ + uint32_t trigger_reclaim_limit; + /**< Trigger automatic reclamation after the defer queue + * has atleast these many resources waiting. + */ + uint32_t max_reclaim_size; + /**< Reclaim at the max these many resources during auto + * reclamation. + */ + rte_rcu_qsbr_free_resource_t free_fn; + /**< Function to call to free the resource. */ + void *p; + /**< Pointer passed to the free function. Typically, this is the + * pointer to the data structure to which the resource to free + * belongs. + */ +}; + +/* Internal structure to represent the element on the defer queue. + * Use alias as a character array is type casted to a variable + * of this structure type. + */ +typedef struct { + uint64_t token; /**< Token */ + uint8_t elem[0]; /**< Pointer to user element */ +} __attribute__((__may_alias__)) __rte_rcu_qsbr_dq_elem_t; + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_RCU_QSBR_PVT_H_ */ diff --git a/lib/librte_rcu/rte_rcu_qsbr.c b/lib/librte_rcu/rte_rcu_qsbr.c index 2f3fad776..6a429d8b3 100644 --- a/lib/librte_rcu/rte_rcu_qsbr.c +++ b/lib/librte_rcu/rte_rcu_qsbr.c @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: BSD-3-Clause * - * Copyright (c) 2018 Arm Limited + * Copyright (c) 2018-2020 Arm Limited */ #include @@ -18,8 +18,10 @@ #include #include #include +#include #include "rte_rcu_qsbr.h" +#include "rcu_qsbr_pvt.h" /* Get the memory size of QSBR variable */ size_t @@ -270,6 +272,229 @@ rte_rcu_qsbr_dump(FILE *f, struct rte_rcu_qsbr *v) return 0; } +/* Create a queue used to store the data structure elements that can + * be freed later. This queue is referred to as 'defer queue'. + */ +struct rte_rcu_qsbr_dq * +rte_rcu_qsbr_dq_create(const struct rte_rcu_qsbr_dq_parameters *params) +{ + struct rte_rcu_qsbr_dq *dq; + uint32_t qs_fifo_size; + unsigned int flags; + + if (params == NULL || params->free_fn == NULL || + params->v == NULL || params->name == NULL || + params->size == 0 || params->esize == 0 || + (params->esize % 4 != 0)) { + rte_log(RTE_LOG_ERR, rte_rcu_log_type, + "%s(): Invalid input parameter\n", __func__); + rte_errno = EINVAL; + + return NULL; + } + /* If auto reclamation is configured, reclaim limit + * should be a valid value. + */ + if ((params->trigger_reclaim_limit <= params->size) && + (params->max_reclaim_size == 0)) { + rte_log(RTE_LOG_ERR, rte_rcu_log_type, + "%s(): Invalid input parameter, size = %u, trigger_reclaim_limit = %u, max_reclaim_size = %u\n", + __func__, params->size, params->trigger_reclaim_limit, + params->max_reclaim_size); + rte_errno = EINVAL; + + return NULL; + } + + dq = rte_zmalloc(NULL, sizeof(struct rte_rcu_qsbr_dq), + RTE_CACHE_LINE_SIZE); + if (dq == NULL) { + rte_errno = ENOMEM; + + return NULL; + } + + /* Decide the flags for the ring. + * If MT safety is requested, use RTS for ring enqueue as most + * use cases involve dq-enqueue happening on the control plane. + * Ring dequeue is always HTS due to the possibility of revert. + */ + flags = RING_F_MP_RTS_ENQ; + if (params->flags & RTE_RCU_QSBR_DQ_MT_UNSAFE) + flags = RING_F_SP_ENQ; + flags |= RING_F_MC_HTS_DEQ; + /* round up qs_fifo_size to next power of two that is not less than + * max_size. + */ + qs_fifo_size = rte_align32pow2(params->size + 1); + /* Add token size to ring element size */ + dq->r = rte_ring_create_elem(params->name, + __RTE_QSBR_TOKEN_SIZE + params->esize, + qs_fifo_size, SOCKET_ID_ANY, flags); + if (dq->r == NULL) { + rte_log(RTE_LOG_ERR, rte_rcu_log_type, + "%s(): defer queue create failed\n", __func__); + rte_free(dq); + return NULL; + } + + dq->v = params->v; + dq->size = params->size; + dq->esize = __RTE_QSBR_TOKEN_SIZE + params->esize; + dq->trigger_reclaim_limit = params->trigger_reclaim_limit; + dq->max_reclaim_size = params->max_reclaim_size; + dq->free_fn = params->free_fn; + dq->p = params->p; + + return dq; +} + +/* Enqueue one resource to the defer queue to free after the grace + * period is over. + */ +int rte_rcu_qsbr_dq_enqueue(struct rte_rcu_qsbr_dq *dq, void *e) +{ + __rte_rcu_qsbr_dq_elem_t *dq_elem; + uint32_t cur_size; + + if (dq == NULL || e == NULL) { + rte_log(RTE_LOG_ERR, rte_rcu_log_type, + "%s(): Invalid input parameter\n", __func__); + rte_errno = EINVAL; + + return 1; + } + + char data[dq->esize]; + dq_elem = (__rte_rcu_qsbr_dq_elem_t *)data; + /* Start the grace period */ + dq_elem->token = rte_rcu_qsbr_start(dq->v); + + /* Reclaim resources if the queue size has hit the reclaim + * limit. This helps the queue from growing too large and + * allows time for reader threads to report their quiescent state. + */ + cur_size = rte_ring_count(dq->r); + if (cur_size > dq->trigger_reclaim_limit) { + rte_log(RTE_LOG_INFO, rte_rcu_log_type, + "%s(): Triggering reclamation\n", __func__); + rte_rcu_qsbr_dq_reclaim(dq, dq->max_reclaim_size, + NULL, NULL, NULL); + } + + /* Enqueue the token and resource. Generating the token and + * enqueuing (token + resource) on the queue is not an + * atomic operation. When the defer queue is shared by multiple + * writers, this might result in tokens enqueued out of order + * on the queue. So, some tokens might wait longer than they + * are required to be reclaimed. + */ + memcpy(dq_elem->elem, e, dq->esize - __RTE_QSBR_TOKEN_SIZE); + /* Check the status as enqueue might fail since the other threads + * might have used up the freed space. + * Enqueue uses the configured flags when the DQ was created. + */ + if (rte_ring_enqueue_elem(dq->r, data, dq->esize) != 0) { + rte_log(RTE_LOG_ERR, rte_rcu_log_type, + "%s(): Enqueue failed\n", __func__); + /* Note that the token generated above is not used. + * Other than wasting tokens, it should not cause any + * other issues. + */ + rte_log(RTE_LOG_INFO, rte_rcu_log_type, + "%s(): Skipped enqueuing token = %"PRIu64"\n", + __func__, dq_elem->token); + + rte_errno = ENOSPC; + return 1; + } + + rte_log(RTE_LOG_INFO, rte_rcu_log_type, + "%s(): Enqueued token = %"PRIu64"\n", __func__, dq_elem->token); + + return 0; +} + +/* Reclaim resources from the defer queue. */ +int +rte_rcu_qsbr_dq_reclaim(struct rte_rcu_qsbr_dq *dq, unsigned int n, + unsigned int *freed, unsigned int *pending, + unsigned int *available) +{ + uint32_t cnt; + __rte_rcu_qsbr_dq_elem_t *dq_elem; + + if (dq == NULL || n == 0) { + rte_log(RTE_LOG_ERR, rte_rcu_log_type, + "%s(): Invalid input parameter\n", __func__); + rte_errno = EINVAL; + + return 1; + } + + cnt = 0; + + char data[dq->esize]; + /* Check reader threads quiescent state and reclaim resources */ + while (cnt < n && + rte_ring_dequeue_bulk_elem_start(dq->r, &data, + dq->esize, 1, available) != 0) { + dq_elem = (__rte_rcu_qsbr_dq_elem_t *)data; + + /* Reclaim the resource */ + if (rte_rcu_qsbr_check(dq->v, dq_elem->token, false) != 1) { + rte_ring_dequeue_elem_finish(dq->r, 0); + break; + } + rte_ring_dequeue_elem_finish(dq->r, 1); + + rte_log(RTE_LOG_INFO, rte_rcu_log_type, + "%s(): Reclaimed token = %"PRIu64"\n", + __func__, dq_elem->token); + + dq->free_fn(dq->p, dq_elem->elem, 1); + + cnt++; + } + + rte_log(RTE_LOG_INFO, rte_rcu_log_type, + "%s(): Reclaimed %u resources\n", __func__, cnt); + + if (freed != NULL) + *freed = cnt; + if (pending != NULL) + *pending = rte_ring_count(dq->r); + + return 0; +} + +/* Delete a defer queue. */ +int +rte_rcu_qsbr_dq_delete(struct rte_rcu_qsbr_dq *dq) +{ + unsigned int pending; + + if (dq == NULL) { + rte_log(RTE_LOG_DEBUG, rte_rcu_log_type, + "%s(): Invalid input parameter\n", __func__); + + return 0; + } + + /* Reclaim all the resources */ + rte_rcu_qsbr_dq_reclaim(dq, ~0, NULL, &pending, NULL); + if (pending != 0) { + rte_errno = EAGAIN; + + return 1; + } + + rte_ring_free(dq->r); + rte_free(dq); + + return 0; +} + int rte_rcu_log_type; RTE_INIT(rte_rcu_register) diff --git a/lib/librte_rcu/rte_rcu_qsbr.h b/lib/librte_rcu/rte_rcu_qsbr.h index 0b5585925..e2fc7f83e 100644 --- a/lib/librte_rcu/rte_rcu_qsbr.h +++ b/lib/librte_rcu/rte_rcu_qsbr.h @@ -1,5 +1,5 @@ /* SPDX-License-Identifier: BSD-3-Clause - * Copyright (c) 2018 Arm Limited + * Copyright (c) 2018-2020 Arm Limited */ #ifndef _RTE_RCU_QSBR_H_ @@ -34,6 +34,7 @@ extern "C" { #include #include #include +#include extern int rte_rcu_log_type; @@ -84,6 +85,7 @@ struct rte_rcu_qsbr_cnt { #define __RTE_QSBR_CNT_THR_OFFLINE 0 #define __RTE_QSBR_CNT_INIT 1 #define __RTE_QSBR_CNT_MAX ((uint64_t)~0) +#define __RTE_QSBR_TOKEN_SIZE sizeof(uint64_t) /* RTE Quiescent State variable structure. * This structure has two elements that vary in size based on the @@ -114,6 +116,86 @@ struct rte_rcu_qsbr { */ } __rte_cache_aligned; +/** + * Call back function called to free the resources. + * + * @param p + * Pointer provided while creating the defer queue + * @param e + * Pointer to the resource data stored on the defer queue + * @param n + * Number of resources to free. Currently, this is set to 1. + * + * @return + * None + */ +typedef void (*rte_rcu_qsbr_free_resource_t)(void *p, void *e, unsigned int n); + +#define RTE_RCU_QSBR_DQ_NAMESIZE RTE_RING_NAMESIZE + +/** + * Various flags supported. + */ +/**< Enqueue and reclaim operations are multi-thread safe by default. + * The call back functions registered to free the resources are + * assumed to be multi-thread safe. + * Set this flag is multi-thread safety is not required. + */ +#define RTE_RCU_QSBR_DQ_MT_UNSAFE 1 + +/** + * Parameters used when creating the defer queue. + */ +struct rte_rcu_qsbr_dq_parameters { + const char *name; + /**< Name of the queue. */ + uint32_t flags; + /**< Flags to control API behaviors */ + uint32_t size; + /**< Number of entries in queue. Typically, this will be + * the same as the maximum number of entries supported in the + * lock free data structure. + * Data structures with unbounded number of entries is not + * supported currently. + */ + uint32_t esize; + /**< Size (in bytes) of each element in the defer queue. + * This has to be multiple of 4B. + */ + uint32_t trigger_reclaim_limit; + /**< Trigger automatic reclamation after the defer queue + * has atleast these many resources waiting. This auto + * reclamation is triggered in rte_rcu_qsbr_dq_enqueue API + * call. + * If this is greater than 'size', auto reclamation is + * not triggered. + * If this is set to 0, auto reclamation is triggered + * in every call to rte_rcu_qsbr_dq_enqueue API. + */ + uint32_t max_reclaim_size; + /**< When automatic reclamation is enabled, reclaim at the max + * these many resources. This should contain a valid value, if + * auto reclamation is on. Setting this to 'size' or greater will + * reclaim all possible resources currently on the defer queue. + */ + rte_rcu_qsbr_free_resource_t free_fn; + /**< Function to call to free the resource. */ + void *p; + /**< Pointer passed to the free function. Typically, this is the + * pointer to the data structure to which the resource to free + * belongs. This can be NULL. + */ + struct rte_rcu_qsbr *v; + /**< RCU QSBR variable to use for this defer queue */ +}; + +/* RTE defer queue structure. + * This structure holds the defer queue. The defer queue is used to + * hold the deleted entries from the data structure that are not + * yet freed. + */ +struct rte_rcu_qsbr_dq; + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice @@ -692,6 +774,116 @@ __rte_experimental int rte_rcu_qsbr_dump(FILE *f, struct rte_rcu_qsbr *v); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Create a queue used to store the data structure elements that can + * be freed later. This queue is referred to as 'defer queue'. + * + * @param params + * Parameters to create a defer queue. + * @return + * On success - Valid pointer to defer queue + * On error - NULL + * Possible rte_errno codes are: + * - EINVAL - NULL parameters are passed + * - ENOMEM - Not enough memory + */ +__rte_experimental +struct rte_rcu_qsbr_dq * +rte_rcu_qsbr_dq_create(const struct rte_rcu_qsbr_dq_parameters *params); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Enqueue one resource to the defer queue and start the grace period. + * The resource will be freed later after at least one grace period + * is over. + * + * If the defer queue is full, it will attempt to reclaim resources. + * It will also reclaim resources at regular intervals to avoid + * the defer queue from growing too big. + * + * Multi-thread safety is provided as the defer queue configuration. + * When multi-thread safety is requested, it is possible that the + * resources are not stored in their order of deletion. This results + * in resources being held in the defer queue longer than they should. + * + * @param dq + * Defer queue to allocate an entry from. + * @param e + * Pointer to resource data to copy to the defer queue. The size of + * the data to copy is equal to the element size provided when the + * defer queue was created. + * @return + * On success - 0 + * On error - 1 with rte_errno set to + * - EINVAL - NULL parameters are passed + * - ENOSPC - Defer queue is full. This condition can not happen + * if the defer queue size is equal (or larger) than the + * number of elements in the data structure. + */ +__rte_experimental +int +rte_rcu_qsbr_dq_enqueue(struct rte_rcu_qsbr_dq *dq, void *e); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Free quesed resources from the defer queue. + * + * This API is multi-thread safe. + * + * @param dq + * Defer queue to free an entry from. + * @param n + * Maximum number of resources to free. + * @param freed + * Number of resources that were freed. + * @param pending + * Number of resources pending on the defer queue. This number might not + * be acurate if multi-thread safety is configured. + * @param available + * Number of resources that can be added to the defer queue. + * This number might not be acurate if multi-thread safety is configured. + * @return + * On successful reclamation of at least 1 resource - 0 + * On error - 1 with rte_errno set to + * - EINVAL - NULL parameters are passed + */ +__rte_experimental +int +rte_rcu_qsbr_dq_reclaim(struct rte_rcu_qsbr_dq *dq, unsigned int n, + unsigned int *freed, unsigned int *pending, unsigned int *available); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Delete a defer queue. + * + * It tries to reclaim all the resources on the defer queue. + * If any of the resources have not completed the grace period + * the reclamation stops and returns immediately. The rest of + * the resources are not reclaimed and the defer queue is not + * freed. + * + * @param dq + * Defer queue to delete. + * @return + * On success - 0 + * On error - 1 + * Possible rte_errno codes are: + * - EAGAIN - Some of the resources have not completed at least 1 grace + * period, try again. + */ +__rte_experimental +int +rte_rcu_qsbr_dq_delete(struct rte_rcu_qsbr_dq *dq); + #ifdef __cplusplus } #endif diff --git a/lib/librte_rcu/rte_rcu_version.map b/lib/librte_rcu/rte_rcu_version.map index f8b9ef2ab..dfac88a37 100644 --- a/lib/librte_rcu/rte_rcu_version.map +++ b/lib/librte_rcu/rte_rcu_version.map @@ -8,6 +8,10 @@ EXPERIMENTAL { rte_rcu_qsbr_synchronize; rte_rcu_qsbr_thread_register; rte_rcu_qsbr_thread_unregister; + rte_rcu_qsbr_dq_create; + rte_rcu_qsbr_dq_enqueue; + rte_rcu_qsbr_dq_reclaim; + rte_rcu_qsbr_dq_delete; local: *; }; diff --git a/lib/meson.build b/lib/meson.build index 63c17ee75..c28b8df83 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -11,7 +11,9 @@ libraries = [ 'kvargs', # eal depends on kvargs 'eal', # everything depends on eal - 'ring', 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core + 'ring', + 'rcu', # rcu depends on ring + 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core 'cmdline', 'metrics', # bitrate/latency stats depends on this 'hash', # efd depends on this @@ -22,7 +24,7 @@ libraries = [ 'gro', 'gso', 'ip_frag', 'jobstats', 'kni', 'latencystats', 'lpm', 'member', 'power', 'pdump', 'rawdev', - 'rcu', 'rib', 'reorder', 'sched', 'security', 'stack', 'vhost', + 'rib', 'reorder', 'sched', 'security', 'stack', 'vhost', # ipsec lib depends on net, crypto and security 'ipsec', #fib lib depends on rib From patchwork Wed Apr 22 03:30:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 185651 Delivered-To: patch@linaro.org Received: by 2002:a92:3d9a:0:0:0:0:0 with SMTP id k26csp251741ilf; Tue, 21 Apr 2020 20:30:36 -0700 (PDT) X-Google-Smtp-Source: APiQypKsrcYXc2DrhisnQ7zF046w6skttqulBpmgomTAMpY3LlZLzAG9WeGwOBbbKDjxDK/zTwRG X-Received: by 2002:aa7:c886:: with SMTP id p6mr9460475eds.97.1587526236076; Tue, 21 Apr 2020 20:30:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1587526236; cv=none; d=google.com; s=arc-20160816; b=VDvPAKPDfuKoIkKYQBNzJEy3y7tyFRqlVDxdf624WTxRXlnUkI33XPDszwESLHw+bj 4aiI959GLIIjvVfDYBjc4nRijemGUNhafWmahLnEplO/suapGaOyddciQedDhM86r8lV yTxbWfkLbtNJpqSqPOhUq321AIIB8f3qCfhiPb9scS8VxChisbqnynu8HET4gxxDr5yt Ff9rokDcuKME8vCvowT2WjjfIJKjq7LywBRNWRqMcT02mLlj9HhMnrf3aMAC/3FmVVY6 3H7duiOERHCPq4ucWEaljAGICPtI3hcjE3kmgmACMppRpO2VmD0zccGZ+4fEXkUpLSu0 5lDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=Pu3AoAAef2KWZEPdNhNLWtU99jChFLpA4AsfjqUxse0=; b=yEJ3acqz7VF0VYdpHbypX9y90PH+wxN78/VD/PiaygVCO0bkMBJve8JWXHWr2v61kf 2ma1MH5KKBA7m8gh2dpXmg3m6fm8gNPgXq0E9PzZGPeJufezHcgtSMk/9QJ3fPjn9FTR 9iAU+mokNniiDMkSQVWHV3gaH9gSdbO1ZHEHRhNLMZQ8tA4eZw36V/VnER+qFqoRM/s/ ISeFwlDmamZWHn9xKnjkR/lobI65zeKjrvsJpIL6lDb5QoCRPO74U0/drsNqnTsEwKG0 gh0bnt/Y+oDVaeCkSmCkL2ASpRhhl0qWcPqUOniV+/V/RECjQv3BQBspo8NB3nU/tmfC jpbA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id t14si2684780edi.331.2020.04.21.20.30.35; Tue, 21 Apr 2020 20:30:36 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A567A1C1DF; Wed, 22 Apr 2020 05:30:35 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 6D5111C1DF for ; Wed, 22 Apr 2020 05:30:34 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CBF9C1042; Tue, 21 Apr 2020 20:30:33 -0700 (PDT) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.14.48]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9A4C03F6CF; Tue, 21 Apr 2020 20:30:33 -0700 (PDT) From: Honnappa Nagarahalli To: konstantin.ananyev@intel.com, stephen@networkplumber.org, vladimir.medvedkin@intel.com, dev@dpdk.org Cc: honnappa.nagarahalli@arm.com, david.marchand@redhat.com, ruifeng.wang@arm.com, dharmik.thakkar@arm.com, nd@arm.com Date: Tue, 21 Apr 2020 22:30:04 -0500 Message-Id: <20200422033006.1124-3-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200422033006.1124-1-honnappa.nagarahalli@arm.com> References: <20191001062917.35578-1-honnappa.nagarahalli@arm.com> <20200422033006.1124-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [PATCH v5 2/4] test/rcu: test cases for RCU defer queue APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add test cases for RCU defer queue APIs. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Ola Liljedhal Reviewed-by: Ruifeng Wang --- app/test/test_rcu_qsbr.c | 365 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 363 insertions(+), 2 deletions(-) -- 2.17.1 Acked-by: Konstantin Ananyev diff --git a/app/test/test_rcu_qsbr.c b/app/test/test_rcu_qsbr.c index b60dc5099..ae73bdab8 100644 --- a/app/test/test_rcu_qsbr.c +++ b/app/test/test_rcu_qsbr.c @@ -1,8 +1,9 @@ /* SPDX-License-Identifier: BSD-3-Clause - * Copyright (c) 2018 Arm Limited + * Copyright (c) 2019-2020 Arm Limited */ #include +#include #include #include #include @@ -15,7 +16,8 @@ #include "test.h" /* Check condition and return an error if true. */ -#define TEST_RCU_QSBR_RETURN_IF_ERROR(cond, str, ...) do { \ +#define TEST_RCU_QSBR_RETURN_IF_ERROR(cond, str, ...) \ +do { \ if (cond) { \ printf("ERROR file %s, line %d: " str "\n", __FILE__, \ __LINE__, ##__VA_ARGS__); \ @@ -23,6 +25,16 @@ } \ } while (0) +/* Check condition and go to label if true. */ +#define TEST_RCU_QSBR_GOTO_IF_ERROR(label, cond, str, ...) \ +do { \ + if (cond) { \ + printf("ERROR file %s, line %d: " str "\n", __FILE__, \ + __LINE__, ##__VA_ARGS__); \ + goto label; \ + } \ +} while (0) + /* Make sure that this has the same value as __RTE_QSBR_CNT_INIT */ #define TEST_RCU_QSBR_CNT_INIT 1 @@ -34,6 +46,7 @@ static uint32_t *keys; #define COUNTER_VALUE 4096 static uint32_t *hash_data[RTE_MAX_LCORE][TOTAL_ENTRY]; static uint8_t writer_done; +static uint8_t cb_failed; static struct rte_rcu_qsbr *t[RTE_MAX_LCORE]; static struct rte_hash *h[RTE_MAX_LCORE]; @@ -585,6 +598,330 @@ test_rcu_qsbr_thread_offline(void) return 0; } +static void +test_rcu_qsbr_free_resource1(void *p, void *e, unsigned int n) +{ + if (p != NULL || e != NULL || n != 1) { + printf("%s: Test failed\n", __func__); + cb_failed = 1; + } +} + +static void +test_rcu_qsbr_free_resource2(void *p, void *e, unsigned int n) +{ + if (p != NULL || e == NULL || n != 1) { + printf("%s: Test failed\n", __func__); + cb_failed = 1; + } +} + +/* + * rte_rcu_qsbr_dq_create: create a queue used to store the data structure + * elements that can be freed later. This queue is referred to as 'defer queue'. + */ +static int +test_rcu_qsbr_dq_create(void) +{ + char rcu_dq_name[RTE_RCU_QSBR_DQ_NAMESIZE]; + struct rte_rcu_qsbr_dq_parameters params; + struct rte_rcu_qsbr_dq *dq; + + printf("\nTest rte_rcu_qsbr_dq_create()\n"); + + /* Pass invalid parameters */ + dq = rte_rcu_qsbr_dq_create(NULL); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + memset(¶ms, 0, sizeof(struct rte_rcu_qsbr_dq_parameters)); + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + snprintf(rcu_dq_name, sizeof(rcu_dq_name), "TEST_RCU"); + params.name = rcu_dq_name; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + params.free_fn = test_rcu_qsbr_free_resource1; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + rte_rcu_qsbr_init(t[0], RTE_MAX_LCORE); + params.v = t[0]; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + params.size = 1; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + params.esize = 3; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + params.trigger_reclaim_limit = 0; + params.max_reclaim_size = 0; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + /* Pass all valid parameters */ + params.esize = 16; + params.trigger_reclaim_limit = 0; + params.max_reclaim_size = params.size; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq == NULL), "dq create valid params"); + rte_rcu_qsbr_dq_delete(dq); + + params.esize = 16; + params.flags = RTE_RCU_QSBR_DQ_MT_UNSAFE; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq == NULL), "dq create valid params"); + rte_rcu_qsbr_dq_delete(dq); + + return 0; +} + +/* + * rte_rcu_qsbr_dq_enqueue: enqueue one resource to the defer queue, + * to be freed later after atleast one grace period is over. + */ +static int +test_rcu_qsbr_dq_enqueue(void) +{ + int ret; + uint64_t r; + char rcu_dq_name[RTE_RCU_QSBR_DQ_NAMESIZE]; + struct rte_rcu_qsbr_dq_parameters params; + struct rte_rcu_qsbr_dq *dq; + + printf("\nTest rte_rcu_qsbr_dq_enqueue()\n"); + + /* Create a queue with simple parameters */ + memset(¶ms, 0, sizeof(struct rte_rcu_qsbr_dq_parameters)); + snprintf(rcu_dq_name, sizeof(rcu_dq_name), "TEST_RCU"); + params.name = rcu_dq_name; + params.free_fn = test_rcu_qsbr_free_resource1; + rte_rcu_qsbr_init(t[0], RTE_MAX_LCORE); + params.v = t[0]; + params.size = 1; + params.esize = 16; + params.trigger_reclaim_limit = 0; + params.max_reclaim_size = params.size; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq == NULL), "dq create valid params"); + + /* Pass invalid parameters */ + ret = rte_rcu_qsbr_dq_enqueue(NULL, NULL); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 0), "dq enqueue invalid params"); + + ret = rte_rcu_qsbr_dq_enqueue(dq, NULL); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 0), "dq enqueue invalid params"); + + ret = rte_rcu_qsbr_dq_enqueue(NULL, &r); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 0), "dq enqueue invalid params"); + + ret = rte_rcu_qsbr_dq_delete(dq); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 1), "dq delete valid params"); + + return 0; +} + +/* + * rte_rcu_qsbr_dq_reclaim: Reclaim resources from the defer queue. + */ +static int +test_rcu_qsbr_dq_reclaim(void) +{ + int ret; + char rcu_dq_name[RTE_RCU_QSBR_DQ_NAMESIZE]; + struct rte_rcu_qsbr_dq_parameters params; + struct rte_rcu_qsbr_dq *dq; + + printf("\nTest rte_rcu_qsbr_dq_reclaim()\n"); + + /* Pass invalid parameters */ + ret = rte_rcu_qsbr_dq_reclaim(NULL, 10, NULL, NULL, NULL); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret != 1), "dq reclaim invalid params"); + + /* Pass invalid parameters */ + memset(¶ms, 0, sizeof(struct rte_rcu_qsbr_dq_parameters)); + snprintf(rcu_dq_name, sizeof(rcu_dq_name), "TEST_RCU"); + params.name = rcu_dq_name; + params.free_fn = test_rcu_qsbr_free_resource1; + rte_rcu_qsbr_init(t[0], RTE_MAX_LCORE); + params.v = t[0]; + params.size = 1; + params.esize = 3; + params.trigger_reclaim_limit = 0; + params.max_reclaim_size = params.size; + dq = rte_rcu_qsbr_dq_create(¶ms); + ret = rte_rcu_qsbr_dq_reclaim(dq, 0, NULL, NULL, NULL); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret != 1), "dq reclaim invalid params"); + + return 0; +} + +/* + * rte_rcu_qsbr_dq_delete: Delete a defer queue. + */ +static int +test_rcu_qsbr_dq_delete(void) +{ + int ret; + char rcu_dq_name[RTE_RCU_QSBR_DQ_NAMESIZE]; + struct rte_rcu_qsbr_dq_parameters params; + struct rte_rcu_qsbr_dq *dq; + + printf("\nTest rte_rcu_qsbr_dq_delete()\n"); + + /* Pass invalid parameters */ + ret = rte_rcu_qsbr_dq_delete(NULL); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret != 0), "dq delete invalid params"); + + memset(¶ms, 0, sizeof(struct rte_rcu_qsbr_dq_parameters)); + snprintf(rcu_dq_name, sizeof(rcu_dq_name), "TEST_RCU"); + params.name = rcu_dq_name; + params.free_fn = test_rcu_qsbr_free_resource1; + rte_rcu_qsbr_init(t[0], RTE_MAX_LCORE); + params.v = t[0]; + params.size = 1; + params.esize = 16; + params.trigger_reclaim_limit = 0; + params.max_reclaim_size = params.size; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq == NULL), "dq create valid params"); + ret = rte_rcu_qsbr_dq_delete(dq); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret != 0), "dq delete valid params"); + + return 0; +} + +/* + * rte_rcu_qsbr_dq_enqueue: enqueue one resource to the defer queue, + * to be freed later after atleast one grace period is over. + */ +static int +test_rcu_qsbr_dq_functional(int32_t size, int32_t esize, uint32_t flags) +{ + int i, j, ret; + char rcu_dq_name[RTE_RCU_QSBR_DQ_NAMESIZE]; + struct rte_rcu_qsbr_dq_parameters params; + struct rte_rcu_qsbr_dq *dq; + uint64_t *e; + uint64_t sc = 200; + int max_entries; + + printf("\nTest rte_rcu_qsbr_dq_xxx functional tests()\n"); + printf("Size = %d, esize = %d, flags = 0x%x\n", size, esize, flags); + + e = (uint64_t *)rte_zmalloc(NULL, esize, RTE_CACHE_LINE_SIZE); + if (e == NULL) + return 0; + cb_failed = 0; + + /* Initialize the RCU variable. No threads are registered */ + rte_rcu_qsbr_init(t[0], RTE_MAX_LCORE); + + /* Create a queue with simple parameters */ + memset(¶ms, 0, sizeof(struct rte_rcu_qsbr_dq_parameters)); + snprintf(rcu_dq_name, sizeof(rcu_dq_name), "TEST_RCU"); + params.name = rcu_dq_name; + params.flags = flags; + params.free_fn = test_rcu_qsbr_free_resource2; + params.v = t[0]; + params.size = size; + params.esize = esize; + params.trigger_reclaim_limit = size >> 3; + params.max_reclaim_size = (size >> 4)?(size >> 4):1; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq == NULL), "dq create valid params"); + + /* Given the size calculate the maximum number of entries + * that can be stored on the defer queue (look at the logic used + * in capacity calculation of rte_ring). + */ + max_entries = rte_align32pow2(size + 1) - 1; + printf("max_entries = %d\n", max_entries); + + /* Enqueue few counters starting with the value 'sc' */ + /* The queue size will be rounded up to 2. The enqueue API also + * reclaims if the queue size is above certain limit. Since, there + * are no threads registered, reclamation succedes. Hence, it should + * be possible to enqueue more than the provided queue size. + */ + for (i = 0; i < 10; i++) { + ret = rte_rcu_qsbr_dq_enqueue(dq, e); + TEST_RCU_QSBR_GOTO_IF_ERROR(end, (ret != 0), + "dq enqueue functional, i = %d", i); + for (j = 0; j < esize/8; j++) + e[j] = sc++; + } + + /* Validate that call back function did not return any error */ + TEST_RCU_QSBR_GOTO_IF_ERROR(end, (cb_failed == 1), "CB failed"); + + /* Register a thread on the RCU QSBR variable. Reclamation will not + * succeed. It should not be possible to enqueue more than the size + * number of resources. + */ + rte_rcu_qsbr_thread_register(t[0], 1); + rte_rcu_qsbr_thread_online(t[0], 1); + + for (i = 0; i < max_entries; i++) { + ret = rte_rcu_qsbr_dq_enqueue(dq, e); + TEST_RCU_QSBR_GOTO_IF_ERROR(end, (ret != 0), + "dq enqueue functional, max_entries = %d, i = %d", + max_entries, i); + for (j = 0; j < esize/8; j++) + e[j] = sc++; + } + + /* Enqueue fails as queue is full */ + ret = rte_rcu_qsbr_dq_enqueue(dq, e); + TEST_RCU_QSBR_GOTO_IF_ERROR(end, (ret == 0), "defer queue is not full"); + + /* Delete should fail as there are elements in defer queue which + * cannot be reclaimed. + */ + ret = rte_rcu_qsbr_dq_delete(dq); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 0), "dq delete valid params"); + + /* Report quiescent state, enqueue should succeed */ + rte_rcu_qsbr_quiescent(t[0], 1); + for (i = 0; i < max_entries; i++) { + ret = rte_rcu_qsbr_dq_enqueue(dq, e); + TEST_RCU_QSBR_GOTO_IF_ERROR(end, (ret != 0), + "dq enqueue functional"); + for (j = 0; j < esize/8; j++) + e[j] = sc++; + } + + /* Validate that call back function did not return any error */ + TEST_RCU_QSBR_GOTO_IF_ERROR(end, (cb_failed == 1), "CB failed"); + + /* Queue is full */ + ret = rte_rcu_qsbr_dq_enqueue(dq, e); + TEST_RCU_QSBR_GOTO_IF_ERROR(end, (ret == 0), "defer queue is not full"); + + /* Report quiescent state, delete should succeed */ + rte_rcu_qsbr_quiescent(t[0], 1); + ret = rte_rcu_qsbr_dq_delete(dq); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret != 0), "dq delete valid params"); + + rte_free(e); + + /* Validate that call back function did not return any error */ + TEST_RCU_QSBR_RETURN_IF_ERROR((cb_failed == 1), "CB failed"); + + return 0; + +end: + rte_free(e); + ret = rte_rcu_qsbr_dq_delete(dq); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret != 0), "dq delete valid params"); + return -1; +} + /* * rte_rcu_qsbr_dump: Dump status of a single QS variable to a file */ @@ -1028,6 +1365,18 @@ test_rcu_qsbr_main(void) if (test_rcu_qsbr_thread_offline() < 0) goto test_fail; + if (test_rcu_qsbr_dq_create() < 0) + goto test_fail; + + if (test_rcu_qsbr_dq_reclaim() < 0) + goto test_fail; + + if (test_rcu_qsbr_dq_delete() < 0) + goto test_fail; + + if (test_rcu_qsbr_dq_enqueue() < 0) + goto test_fail; + printf("\nFunctional tests\n"); if (test_rcu_qsbr_sw_sv_3qs() < 0) @@ -1036,6 +1385,18 @@ test_rcu_qsbr_main(void) if (test_rcu_qsbr_mw_mv_mqs() < 0) goto test_fail; + if (test_rcu_qsbr_dq_functional(1, 8, 0) < 0) + goto test_fail; + + if (test_rcu_qsbr_dq_functional(2, 8, RTE_RCU_QSBR_DQ_MT_UNSAFE) < 0) + goto test_fail; + + if (test_rcu_qsbr_dq_functional(303, 16, 0) < 0) + goto test_fail; + + if (test_rcu_qsbr_dq_functional(7, 128, RTE_RCU_QSBR_DQ_MT_UNSAFE) < 0) + goto test_fail; + free_rcu(); printf("\n"); From patchwork Wed Apr 22 03:30:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 185652 Delivered-To: patch@linaro.org Received: by 2002:a92:3d9a:0:0:0:0:0 with SMTP id k26csp251842ilf; Tue, 21 Apr 2020 20:30:44 -0700 (PDT) X-Google-Smtp-Source: APiQypIejCd3qnAkUf3tMHx68MDv8SzYhQmzvfDV2/w6MXT0R+TFQPzSwh0KuMy+4++k14NaipYV X-Received: by 2002:a05:6402:1757:: with SMTP id v23mr22378093edx.335.1587526244573; Tue, 21 Apr 2020 20:30:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1587526244; cv=none; d=google.com; s=arc-20160816; b=fuLXSUOl7Rm4yT/0JwjShyR9mHBvLsPCG6W5e839AlWqiHBh1XfRxCRkiEZjVVNWJt 6BRqZH1p8Hs+m7VkYxPFxAIPyVr1XbC8X1lRmACeoF8U7/bB3JZh06iXlvKOM81tk1Gd LwV+d2lHOtfSc6vaFWNkwqwak7IMF59k7Nr0fhHV1CRRYaGxvAkiAiaNEDPkmKNn8FaS sbk0zthCrJOcBH2bDm2Yhe7hDVpeHNHUEUUCSlyIEd9W3e/X7sx4gpH0c3+aomxsJvI2 F0cKYYZljSFmdWu9q1YdiVp4IsxgTubSPpsqCKeVA0elsCYvjx+7dmUIypAp4bDDWWBo JDaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=YL9ZdZGE1Du239JZwls0MEYmphnyWITfOPaomcgGl9Y=; b=IEXOCzdwB+n2kDISRG35/N5oOtCsWyzqbuhJno4MW8FnL/3oc5A6NIkimZVEo0BhnZ xyIABvXDva++rIY9gNs/hMP9MdVqNhghgzjs5KF58aDerKK+bBiGk0g50sN5QkjFKrAs FPoFGAofdion3Vib/+oZ8YMGj28BsRxP1WdRqlM8Vio9lBbuDu8NuHwj1deq2fW3YZRL T626NA1cfy0uE3pcFDRDzpDc5T2ss9ckC7cpwDX72LW5jBNXjG/jhP+cgkCyW1S/MRF0 ZOzxt99J5MSgXve7jpPnnX2ipTkLvYgdDDnS83fjUupRUXCfxM9bw1HqyWq4RuoLq2x6 N2aA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id oh24si2818530ejb.531.2020.04.21.20.30.44; Tue, 21 Apr 2020 20:30:44 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2CFF01C2EF; Wed, 22 Apr 2020 05:30:38 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 461F01C1DF for ; Wed, 22 Apr 2020 05:30:35 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9B4181045; Tue, 21 Apr 2020 20:30:34 -0700 (PDT) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.14.48]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8933A3F6CF; Tue, 21 Apr 2020 20:30:34 -0700 (PDT) From: Honnappa Nagarahalli To: konstantin.ananyev@intel.com, stephen@networkplumber.org, vladimir.medvedkin@intel.com, dev@dpdk.org Cc: honnappa.nagarahalli@arm.com, david.marchand@redhat.com, ruifeng.wang@arm.com, dharmik.thakkar@arm.com, nd@arm.com Date: Tue, 21 Apr 2020 22:30:05 -0500 Message-Id: <20200422033006.1124-4-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200422033006.1124-1-honnappa.nagarahalli@arm.com> References: <20191001062917.35578-1-honnappa.nagarahalli@arm.com> <20200422033006.1124-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [PATCH v5 3/4] doc/rcu: add RCU integration design details X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ruifeng Wang Add a section to describe a design to integrate QSBR RCU library with other libraries in DPDK. Signed-off-by: Honnappa Nagarahalli Signed-off-by: Ruifeng Wang Reviewed-by: Gavin Hu Reviewed-by: Ruifeng Wang --- doc/guides/prog_guide/rcu_lib.rst | 59 +++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) -- 2.17.1 diff --git a/doc/guides/prog_guide/rcu_lib.rst b/doc/guides/prog_guide/rcu_lib.rst index 9b0bf138f..60468f143 100644 --- a/doc/guides/prog_guide/rcu_lib.rst +++ b/doc/guides/prog_guide/rcu_lib.rst @@ -190,3 +190,62 @@ However, when ``CONFIG_RTE_LIBRTE_RCU_DEBUG`` is enabled, these APIs aid in debugging issues. One can mark the access to shared data structures on the reader side using these APIs. The ``rte_rcu_qsbr_quiescent()`` will check if all the locks are unlocked. + +Resource reclamation framework for DPDK +--------------------------------------- + +Lock-free algorithms place additional burden of resource reclamation on +the application. When a writer deletes an entry from a data structure, the writer: + +#. Has to start the grace period +#. Has to store a reference to the deleted resources in a FIFO +#. Should check if the readers have completed a grace period and free the resources. + +There are several APIs provided to help with this process. The writer +can create a FIFO to store the references to deleted resources using ``rte_rcu_qsbr_dq_create()``. +The resources can be enqueued to this FIFO using ``rte_rcu_qsbr_dq_enqueue()``. +If the FIFO is full, ``rte_rcu_qsbr_dq_enqueue`` will reclaim the resources before enqueuing. It will also reclaim resources on regular basis to keep the FIFO from growing too large. If the writer runs out of resources, the writer can call ``rte_rcu_qsbr_dq_reclaim`` API to reclaim resources. ``rte_rcu_qsbr_dq_delete`` is provided to reclaim any remaining resources and free the FIFO while shutting down. + +However, if this resource reclamation process were to be integrated in lock-free data structure libraries, it +hides this complexity from the application and makes it easier for the application to adopt lock-free algorithms. The following paragraphs discuss how the reclamation process can be integrated in DPDK libraries. + +In any DPDK application, the resource reclamation process using QSBR can be split into 4 parts: + +#. Initialization +#. Quiescent State Reporting +#. Reclaiming Resources +#. Shutdown + +The design proposed here assigns different parts of this process to client libraries and applications. The term 'client library' refers to lock-free data structure libraries such at rte_hash, rte_lpm etc. in DPDK or similar libraries outside of DPDK. The term 'application' refers to the packet processing application that makes use of DPDK such as L3 Forwarding example application, OVS, VPP etc.. + +The application has to handle 'Initialization' and 'Quiescent State Reporting'. So, + +* the application has to create the RCU variable and register the reader threads to report their quiescent state. +* the application has to register the same RCU variable with the client library. +* reader threads in the application have to report the quiescent state. This allows for the application to control the length of the critical section/how frequently the application wants to report the quiescent state. + +The client library will handle 'Reclaiming Resources' part of the process. The +client libraries will make use of the writer thread context to execute the memory +reclamation algorithm. So, + +* client library should provide an API to register a RCU variable that it will use. It should call ``rte_rcu_qsbr_dq_create()`` to create the FIFO to store the references to deleted entries. +* client library should use ``rte_rcu_qsbr_dq_enqueue`` to enqueue the deleted resources on the FIFO and start the grace period. +* if the library runs out of resources while adding entries, it should call ``rte_rcu_qsbr_dq_reclaim`` to reclaim the resources and try the resource allocation again. + +The 'Shutdown' process needs to be shared between the application and the +client library. + +* the application should make sure that the reader threads are not using the shared data structure, unregister the reader threads from the QSBR variable before calling the client library's shutdown function. + +* client library should call ``rte_rcu_qsbr_dq_delete`` to reclaim any remaining resources and free the FIFO. + +Integrating the resource reclamation with client libraries removes the burden from +the application and makes it easy to use lock-free algorithms. + +This design has several advantages over currently known methods. + +#. Application does not need a dedicated thread to reclaim resources. Memory + reclamation happens as part of the writer thread with little impact on + performance. +#. The client library has better control over the resources. For ex: the client + library can attempt to reclaim when it has run out of resources. From patchwork Wed Apr 22 03:30:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 185653 Delivered-To: patch@linaro.org Received: by 2002:a92:3d9a:0:0:0:0:0 with SMTP id k26csp251993ilf; Tue, 21 Apr 2020 20:30:53 -0700 (PDT) X-Google-Smtp-Source: APiQypKyp9wju5PAM4bX/HxxI7gbTieh3x/2nP34lda+vROTh1ZUQv6ElAkL99ZhAfin2iebCLSQ X-Received: by 2002:a17:906:11c9:: with SMTP id o9mr24893961eja.64.1587526253691; Tue, 21 Apr 2020 20:30:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1587526253; cv=none; d=google.com; s=arc-20160816; b=baaAi9I/cf8tn8Wh5PVsVMspXJqclv1OdWjDcQwWpNU8B+ChyDzLEJ2TKqXZdQItq3 c8/jPeymeoR3/4uwo9HIQ43+wJvOWFlKmjl+kkmoVzuqouilrWZ4ASwXNhxytxV7GNuY FQhxAnGqsrZcgzhP4Y6XpoMxD7nsTuwVf0uqD7DS7weepkLCUmFBLULK4NzH4Sl8U8qK 8+wHDmYBlok6/uenKOaBnfgH/QMIMDrYyRXFdUHrYAerR3m3tL0zWtgsJOUHA5EpC8Zx tfAsGwJ+y3f6bXL68t6RRcxlFwx8MYnPomxO+pX/O0JLR+oLmitVaTFy/gKeRVcsjHDj Bq5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=mDRz4aUz6639xLrp/zZDvVhLTX5R3OsgL7wbE1gXZK0=; b=IBfXKz5jpVN868l3iL69CFncP/bqlPRp2hw/rY2WQ1flytahtK5o+FL0Sq4c3PJ/HW gq3Z7EaKg7QR8hCLmvHcF8i0O8g6ZbRUqkcaVZLe4ZFBvdXbjATMz6auDWe+0NpRQrSn 1moDZhA7+S4ldM+RJ0oYvO6h4lUF2Gap2l0h/FyRtvdde94MLOjcMCSQoMUMEtPxJECt 0sbSt58EsmYsoLM2s49C087MDr3KhrenfT9cMLLI5uKrsw1yox7N1YmYl7s9U6iBKy7N nDSIukyptNmm1OwPcIC5YM3TobWZpv9P+BJ7jBDSIHL7dh7g5yqTLOkzj6S/hY5no25c h4BA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id e16si2654086ejk.0.2020.04.21.20.30.53; Tue, 21 Apr 2020 20:30:53 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B10201C434; Wed, 22 Apr 2020 05:30:39 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 27D1D1C2DD for ; Wed, 22 Apr 2020 05:30:36 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 721BE1063; Tue, 21 Apr 2020 20:30:35 -0700 (PDT) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.14.48]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 611913F6CF; Tue, 21 Apr 2020 20:30:35 -0700 (PDT) From: Honnappa Nagarahalli To: konstantin.ananyev@intel.com, stephen@networkplumber.org, vladimir.medvedkin@intel.com, dev@dpdk.org Cc: honnappa.nagarahalli@arm.com, david.marchand@redhat.com, ruifeng.wang@arm.com, dharmik.thakkar@arm.com, nd@arm.com Date: Tue, 21 Apr 2020 22:30:06 -0500 Message-Id: <20200422033006.1124-5-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200422033006.1124-1-honnappa.nagarahalli@arm.com> References: <20191001062917.35578-1-honnappa.nagarahalli@arm.com> <20200422033006.1124-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [PATCH v5 4/4] lib/rcu: add additional debug logs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Added additional debug logs. These helped in debugging RCU defer APIs. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Gavin Hu Reviewed-by: Ruifeng Wang --- lib/librte_rcu/rte_rcu_qsbr.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) -- 2.17.1 Acked-by: Konstantin Ananyev diff --git a/lib/librte_rcu/rte_rcu_qsbr.h b/lib/librte_rcu/rte_rcu_qsbr.h index e2fc7f83e..9524915fa 100644 --- a/lib/librte_rcu/rte_rcu_qsbr.h +++ b/lib/librte_rcu/rte_rcu_qsbr.h @@ -718,8 +718,15 @@ rte_rcu_qsbr_check(struct rte_rcu_qsbr *v, uint64_t t, bool wait) RTE_ASSERT(v != NULL); /* Check if all the readers have already acknowledged this token */ - if (likely(t <= v->acked_token)) + if (likely(t <= v->acked_token)) { + __RTE_RCU_DP_LOG(DEBUG, + "%s: check: token = %"PRIu64", wait = %d", + __func__, t, wait); + __RTE_RCU_DP_LOG(DEBUG, + "%s: status: least acked token = %"PRIu64"", + __func__, v->acked_token); return 1; + } if (likely(v->num_threads == v->max_threads)) return __rte_rcu_qsbr_check_all(v, t, wait);