From patchwork Fri Apr 3 18:41:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 185174 Delivered-To: patch@linaro.org Received: by 2002:a92:de47:0:0:0:0:0 with SMTP id e7csp882176ilr; Fri, 3 Apr 2020 11:42:01 -0700 (PDT) X-Google-Smtp-Source: APiQypIxXKjhVlT8S/iz7k1w3jT/IbtrgXdSe23a5fUZ1oaj7XAbhpOHG84+/b4fE2bLLN7/fwxS X-Received: by 2002:adf:f1ce:: with SMTP id z14mr10451008wro.68.1585939321742; Fri, 03 Apr 2020 11:42:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1585939321; cv=none; d=google.com; s=arc-20160816; b=Pu5bYWBu0auH0uMDq3bvh6YQq45i8sOOONWxGHUDUvtrW/y7VJKi3O+Kep2c3e3HBj NrDN/IHtgu2ZbZ0vcwO+xeIQXeYIED5oL/AOM40uMqOJGu8/lepxXlhqWRhcE0xpo8X1 pDWVv589xoQLfW6BpP+NDKz4QoVmCm4hl/AUvac2RMOxM2nWTx6pkBEpjzBRwruYPX+y 6C32ws2AGp6qh8zEiCns+BMvg6KJZEh3oE9CIeoRETy9jEoFt/bmwnciED0QiQwUyNxf bEJUe6RfBpmbKH08cikUNSEhDW8NFgl4ujL2gsZ6+xa/cQsXLRyMUCMqbhvWgpArnJWE vMEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=5pyabD3Oftj5vbrR8ec9BhZRulcDkpWk7aO1kJCK62k=; b=NdiyYROnW17XFmXdqLE7B2NRGJvbveqCNCd15hlyhBGVdBMSl3BmNziOMgBxDnV06y BTPTFDZJ0f7V3rlcjmY5EA5TvqNi91/ELQsCIR0N1U3/dc/lldl0XmLsNK6l6igk/JBg ZQxRJGvs/vA/elSmEv1gHRmib4RfkYEQ5qJp4o+AAB4AnKYFUPldeYahHlkevjeQKTe0 3SufNXobbvwcqcn+W5PiP+qVVA3H3l29TG58yaIUtgiR82ma7XcTCIqASSVyy5bnySnW Xcyy+e6gcbehcXhpfxqMpq5/zUnkDtY+Nl06ZRmZ04KpddsHK/Zdi9IY+P2L1WWSpvwz Wtpg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id s20si8732490wmc.43.2020.04.03.11.42.01; Fri, 03 Apr 2020 11:42:01 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C05CB1C1C9; Fri, 3 Apr 2020 20:41:57 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id B6D3A1C1C1 for ; Fri, 3 Apr 2020 20:41:55 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3ACFE1FB; Fri, 3 Apr 2020 11:41:55 -0700 (PDT) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.14.48]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2BC1F3F71E; Fri, 3 Apr 2020 11:41:55 -0700 (PDT) From: Honnappa Nagarahalli To: konstantin.ananyev@intel.com, stephen@networkplumber.org, vladimir.medvedkin@intel.com Cc: dev@dpdk.org, honnappa.nagarahalli@arm.com, ruifeng.wang@arm.com, dharmik.thakkar@arm.com, nd@arm.com Date: Fri, 3 Apr 2020 13:41:39 -0500 Message-Id: <20200403184142.7729-2-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200403184142.7729-1-honnappa.nagarahalli@arm.com> References: <20191001062917.35578-1-honnappa.nagarahalli@arm.com> <20200403184142.7729-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [PATCH v4 1/4] lib/rcu: add resource reclamation APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add resource reclamation APIs to make it simple for applications and libraries to integrate rte_rcu library. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Ola Liljedhal Reviewed-by: Ruifeng Wang --- lib/librte_rcu/Makefile | 2 +- lib/librte_rcu/meson.build | 2 + lib/librte_rcu/rcu_qsbr_pvt.h | 57 +++++++ lib/librte_rcu/rte_rcu_qsbr.c | 243 ++++++++++++++++++++++++++++- lib/librte_rcu/rte_rcu_qsbr.h | 188 ++++++++++++++++++++++ lib/librte_rcu/rte_rcu_version.map | 4 + lib/meson.build | 6 +- 7 files changed, 498 insertions(+), 4 deletions(-) create mode 100644 lib/librte_rcu/rcu_qsbr_pvt.h -- 2.17.1 Acked-by: Konstantin Ananyev diff --git a/lib/librte_rcu/Makefile b/lib/librte_rcu/Makefile index c4bb28d77..95f8a57e2 100644 --- a/lib/librte_rcu/Makefile +++ b/lib/librte_rcu/Makefile @@ -8,7 +8,7 @@ LIB = librte_rcu.a CFLAGS += -DALLOW_EXPERIMENTAL_API CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -LDLIBS += -lrte_eal +LDLIBS += -lrte_eal -lrte_ring EXPORT_MAP := rte_rcu_version.map diff --git a/lib/librte_rcu/meson.build b/lib/librte_rcu/meson.build index 62920ba02..e280b29c1 100644 --- a/lib/librte_rcu/meson.build +++ b/lib/librte_rcu/meson.build @@ -10,3 +10,5 @@ headers = files('rte_rcu_qsbr.h') if cc.get_id() == 'clang' and dpdk_conf.get('RTE_ARCH_64') == false ext_deps += cc.find_library('atomic') endif + +deps += ['ring'] diff --git a/lib/librte_rcu/rcu_qsbr_pvt.h b/lib/librte_rcu/rcu_qsbr_pvt.h new file mode 100644 index 000000000..413f28587 --- /dev/null +++ b/lib/librte_rcu/rcu_qsbr_pvt.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2019 Arm Limited + */ + +#ifndef _RTE_RCU_QSBR_PVT_H_ +#define _RTE_RCU_QSBR_PVT_H_ + +/** + * This file is private to the RCU library. It should not be included + * by the user of this library. + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include + +#include "rte_rcu_qsbr.h" + +/* RTE defer queue structure. + * This structure holds the defer queue. The defer queue is used to + * hold the deleted entries from the data structure that are not + * yet freed. + */ +struct rte_rcu_qsbr_dq { + struct rte_rcu_qsbr *v; /**< RCU QSBR variable used by this queue.*/ + struct rte_ring *r; /**< RCU QSBR defer queue. */ + uint32_t size; + /**< Number of elements in the defer queue */ + uint32_t esize; + /**< Size (in bytes) of data, including the token, stored on the + * defer queue. + */ + uint32_t trigger_reclaim_limit; + /**< Trigger automatic reclamation after the defer queue + * has atleast these many resources waiting. + */ + uint32_t max_reclaim_size; + /**< Reclaim at the max these many resources during auto + * reclamation. + */ + rte_rcu_qsbr_free_resource_t free_fn; + /**< Function to call to free the resource. */ + void *p; + /**< Pointer passed to the free function. Typically, this is the + * pointer to the data structure to which the resource to free + * belongs. + */ +}; + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_RCU_QSBR_PVT_H_ */ diff --git a/lib/librte_rcu/rte_rcu_qsbr.c b/lib/librte_rcu/rte_rcu_qsbr.c index 2f3fad776..e8c1e386f 100644 --- a/lib/librte_rcu/rte_rcu_qsbr.c +++ b/lib/librte_rcu/rte_rcu_qsbr.c @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: BSD-3-Clause * - * Copyright (c) 2018 Arm Limited + * Copyright (c) 2018-2019 Arm Limited */ #include @@ -18,8 +18,10 @@ #include #include #include +#include #include "rte_rcu_qsbr.h" +#include "rcu_qsbr_pvt.h" /* Get the memory size of QSBR variable */ size_t @@ -270,6 +272,245 @@ rte_rcu_qsbr_dump(FILE *f, struct rte_rcu_qsbr *v) return 0; } +/* Create a queue used to store the data structure elements that can + * be freed later. This queue is referred to as 'defer queue'. + */ +struct rte_rcu_qsbr_dq * +rte_rcu_qsbr_dq_create(const struct rte_rcu_qsbr_dq_parameters *params) +{ + struct rte_rcu_qsbr_dq *dq; + uint32_t qs_fifo_size; + unsigned int flags; + + if (params == NULL || params->free_fn == NULL || + params->v == NULL || params->name == NULL || + params->size == 0 || params->esize == 0 || + (params->esize % 4 != 0)) { + rte_log(RTE_LOG_ERR, rte_rcu_log_type, + "%s(): Invalid input parameter\n", __func__); + rte_errno = EINVAL; + + return NULL; + } + /* If auto reclamation is configured, reclaim limit + * should be a valid value. + */ + if ((params->trigger_reclaim_limit <= params->size) && + (params->max_reclaim_size == 0)) { + rte_log(RTE_LOG_ERR, rte_rcu_log_type, + "%s(): Invalid input parameter, size = %u, trigger_reclaim_limit = %u, max_reclaim_size = %u\n", + __func__, params->size, params->trigger_reclaim_limit, + params->max_reclaim_size); + rte_errno = EINVAL; + + return NULL; + } + + dq = rte_zmalloc(NULL, sizeof(struct rte_rcu_qsbr_dq), + RTE_CACHE_LINE_SIZE); + if (dq == NULL) { + rte_errno = ENOMEM; + + return NULL; + } + + /* Decide the flags for the ring. + * If MT safety is requested, use RTS for ring enqueue as most + * use cases involve dq-enqueue happening on the control plane. + * Ring dequeue is always HTS due to the possibility of revert. + */ + flags = RING_F_MP_RTS_ENQ; + if (params->flags & RTE_RCU_QSBR_DQ_MT_UNSAFE) + flags = RING_F_SP_ENQ; + flags |= RING_F_MC_HTS_DEQ; + /* round up qs_fifo_size to next power of two that is not less than + * max_size. + */ + qs_fifo_size = rte_align32pow2(params->size + 1); + /* Add token size to ring element size */ + dq->r = rte_ring_create_elem(params->name, + __RTE_QSBR_TOKEN_SIZE + params->esize, + qs_fifo_size, SOCKET_ID_ANY, flags); + if (dq->r == NULL) { + rte_log(RTE_LOG_ERR, rte_rcu_log_type, + "%s(): defer queue create failed\n", __func__); + rte_free(dq); + return NULL; + } + + dq->v = params->v; + dq->size = params->size; + dq->esize = __RTE_QSBR_TOKEN_SIZE + params->esize; + dq->trigger_reclaim_limit = params->trigger_reclaim_limit; + dq->max_reclaim_size = params->max_reclaim_size; + dq->free_fn = params->free_fn; + dq->p = params->p; + + return dq; +} + +/* Enqueue one resource to the defer queue to free after the grace + * period is over. + */ +int rte_rcu_qsbr_dq_enqueue(struct rte_rcu_qsbr_dq *dq, void *e) +{ + uint64_t token; + uint32_t cur_size, free_size; + + if (dq == NULL || e == NULL) { + rte_log(RTE_LOG_ERR, rte_rcu_log_type, + "%s(): Invalid input parameter\n", __func__); + rte_errno = EINVAL; + + return 1; + } + + /* Start the grace period */ + token = rte_rcu_qsbr_start(dq->v); + + /* Reclaim resources if the queue is 1/8th full. This helps + * the queue from growing too large and allows time for reader + * threads to report their quiescent state. + */ + cur_size = rte_ring_count(dq->r); + if (cur_size > dq->trigger_reclaim_limit) { + rte_log(RTE_LOG_INFO, rte_rcu_log_type, + "%s(): Triggering reclamation\n", __func__); + rte_rcu_qsbr_dq_reclaim(dq, dq->max_reclaim_size, NULL, NULL); + } + + /* Check if there is space for atleast 1 resource */ + free_size = rte_ring_free_count(dq->r); + if (!free_size) { + rte_log(RTE_LOG_ERR, rte_rcu_log_type, + "%s(): Defer queue is full\n", __func__); + /* Note that the token generated above is not used. + * Other than wasting tokens, it should not cause any + * other issues. + */ + rte_log(RTE_LOG_INFO, rte_rcu_log_type, + "%s(): Skipped enqueuing token = %"PRIu64"\n", + __func__, token); + + rte_errno = ENOSPC; + return 1; + } + + /* Enqueue the token and resource. Generating the token + * and enqueuing (token + resource) on the queue is not an + * atomic operation. This might result in tokens enqueued + * out of order on the queue. So, some tokens might wait + * longer than they are required to be reclaimed. + */ + char data[dq->esize]; + memcpy(data, &token, __RTE_QSBR_TOKEN_SIZE); + memcpy(data + __RTE_QSBR_TOKEN_SIZE, e, + dq->esize - __RTE_QSBR_TOKEN_SIZE); + /* Check the status as enqueue might fail since the other thread + * might have used up the freed space. + * Enqueue uses the configured flags when the DQ was created. + */ + if (rte_ring_enqueue_elem(dq->r, data, dq->esize) != 0) { + rte_log(RTE_LOG_ERR, rte_rcu_log_type, + "%s(): Enqueue failed\n", __func__); + /* Note that the token generated above is not used. + * Other than wasting tokens, it should not cause any + * other issues. + */ + rte_log(RTE_LOG_INFO, rte_rcu_log_type, + "%s(): Skipped enqueuing token = %"PRIu64"\n", + __func__, token); + + rte_errno = ENOSPC; + return 1; + } + + rte_log(RTE_LOG_INFO, rte_rcu_log_type, + "%s(): Enqueued token = %"PRIu64"\n", __func__, token); + + return 0; +} + +/* Reclaim resources from the defer queue. */ +int +rte_rcu_qsbr_dq_reclaim(struct rte_rcu_qsbr_dq *dq, unsigned int n, + unsigned int *freed, unsigned int *pending) +{ + uint32_t cnt; + uint64_t token; + + if (dq == NULL || n == 0) { + rte_log(RTE_LOG_ERR, rte_rcu_log_type, + "%s(): Invalid input parameter\n", __func__); + rte_errno = EINVAL; + + return 1; + } + + cnt = 0; + + char e[dq->esize]; + /* Check reader threads quiescent state and reclaim resources */ + while ((cnt < n) && + (rte_ring_dequeue_bulk_elem_start(dq->r, e, + dq->esize, 1, NULL) != 0)) { + memcpy(&token, e, sizeof(uint64_t)); + + /* Reclaim the resource */ + if (rte_rcu_qsbr_check(dq->v, token, false) != 1) { + rte_ring_dequeue_finish(dq->r, 0); + break; + } + rte_ring_dequeue_finish(dq->r, 1); + + rte_log(RTE_LOG_INFO, rte_rcu_log_type, + "%s(): Reclaimed token = %"PRIu64"\n", + __func__, *(uint64_t *)e); + + dq->free_fn(dq->p, e + __RTE_QSBR_TOKEN_SIZE); + + cnt++; + } + + rte_log(RTE_LOG_INFO, rte_rcu_log_type, + "%s(): Reclaimed %u resources\n", __func__, cnt); + + if (freed != NULL) + *freed = cnt; + if (pending != NULL) + *pending = rte_ring_count(dq->r); + + return 0; +} + +/* Delete a defer queue. */ +int +rte_rcu_qsbr_dq_delete(struct rte_rcu_qsbr_dq *dq) +{ + unsigned int pending; + + if (dq == NULL) { + rte_log(RTE_LOG_ERR, rte_rcu_log_type, + "%s(): Invalid input parameter\n", __func__); + rte_errno = EINVAL; + + return 1; + } + + /* Reclaim all the resources */ + rte_rcu_qsbr_dq_reclaim(dq, ~0, NULL, &pending); + if (pending != 0) { + rte_errno = EAGAIN; + + return 1; + } + + rte_ring_free(dq->r); + rte_free(dq); + + return 0; +} + int rte_rcu_log_type; RTE_INIT(rte_rcu_register) diff --git a/lib/librte_rcu/rte_rcu_qsbr.h b/lib/librte_rcu/rte_rcu_qsbr.h index 0b5585925..213f9b029 100644 --- a/lib/librte_rcu/rte_rcu_qsbr.h +++ b/lib/librte_rcu/rte_rcu_qsbr.h @@ -34,6 +34,7 @@ extern "C" { #include #include #include +#include extern int rte_rcu_log_type; @@ -84,6 +85,7 @@ struct rte_rcu_qsbr_cnt { #define __RTE_QSBR_CNT_THR_OFFLINE 0 #define __RTE_QSBR_CNT_INIT 1 #define __RTE_QSBR_CNT_MAX ((uint64_t)~0) +#define __RTE_QSBR_TOKEN_SIZE sizeof(uint64_t) /* RTE Quiescent State variable structure. * This structure has two elements that vary in size based on the @@ -114,6 +116,84 @@ struct rte_rcu_qsbr { */ } __rte_cache_aligned; +/** + * Call back function called to free the resources. + * + * @param p + * Pointer provided while creating the defer queue + * @param e + * Pointer to the resource data stored on the defer queue + * + * @return + * None + */ +typedef void (*rte_rcu_qsbr_free_resource_t)(void *p, void *e); + +#define RTE_RCU_QSBR_DQ_NAMESIZE RTE_RING_NAMESIZE + +/** + * Various flags supported. + */ +/**< Enqueue and reclaim operations are multi-thread safe by default. + * The call back functions registered to free the resources are + * assumed to be multi-thread safe. + * Set this flag is multi-thread safety is not required. + */ +#define RTE_RCU_QSBR_DQ_MT_UNSAFE 1 + +/** + * Parameters used when creating the defer queue. + */ +struct rte_rcu_qsbr_dq_parameters { + const char *name; + /**< Name of the queue. */ + uint32_t flags; + /**< Flags to control API behaviors */ + uint32_t size; + /**< Number of entries in queue. Typically, this will be + * the same as the maximum number of entries supported in the + * lock free data structure. + * Data structures with unbounded number of entries is not + * supported currently. + */ + uint32_t esize; + /**< Size (in bytes) of each element in the defer queue. + * This has to be multiple of 4B. + */ + uint32_t trigger_reclaim_limit; + /**< Trigger automatic reclamation after the defer queue + * has atleast these many resources waiting. This auto + * reclamation is triggered in rte_rcu_qsbr_dq_enqueue API + * call. + * If this is greater than 'size', auto reclamation is + * not triggered. + * If this is set to 0, auto reclamation is triggered + * in every call to rte_rcu_qsbr_dq_enqueue API. + */ + uint32_t max_reclaim_size; + /**< When automatic reclamation is enabled, reclaim at the max + * these many resources. This should contain a valid value, if + * auto reclamation is on. Setting this to 'size' or greater will + * reclaim all possible resources currently on the defer queue. + */ + rte_rcu_qsbr_free_resource_t free_fn; + /**< Function to call to free the resource. */ + void *p; + /**< Pointer passed to the free function. Typically, this is the + * pointer to the data structure to which the resource to free + * belongs. This can be NULL. + */ + struct rte_rcu_qsbr *v; + /**< RCU QSBR variable to use for this defer queue */ +}; + +/* RTE defer queue structure. + * This structure holds the defer queue. The defer queue is used to + * hold the deleted entries from the data structure that are not + * yet freed. + */ +struct rte_rcu_qsbr_dq; + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice @@ -692,6 +772,114 @@ __rte_experimental int rte_rcu_qsbr_dump(FILE *f, struct rte_rcu_qsbr *v); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Create a queue used to store the data structure elements that can + * be freed later. This queue is referred to as 'defer queue'. + * + * @param params + * Parameters to create a defer queue. + * @return + * On success - Valid pointer to defer queue + * On error - NULL + * Possible rte_errno codes are: + * - EINVAL - NULL parameters are passed + * - ENOMEM - Not enough memory + */ +__rte_experimental +struct rte_rcu_qsbr_dq * +rte_rcu_qsbr_dq_create(const struct rte_rcu_qsbr_dq_parameters *params); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Enqueue one resource to the defer queue and start the grace period. + * The resource will be freed later after at least one grace period + * is over. + * + * If the defer queue is full, it will attempt to reclaim resources. + * It will also reclaim resources at regular intervals to avoid + * the defer queue from growing too big. + * + * Multi-thread safety is provided as the defer queue configuration. + * When multi-thread safety is requested, it is possible that the + * resources are not stored in their order of deletion. This results + * in resources being held in the defer queue longer than they should. + * + * @param dq + * Defer queue to allocate an entry from. + * @param e + * Pointer to resource data to copy to the defer queue. The size of + * the data to copy is equal to the element size provided when the + * defer queue was created. + * @return + * On success - 0 + * On error - 1 with rte_errno set to + * - EINVAL - NULL parameters are passed + * - ENOSPC - Defer queue is full. This condition can not happen + * if the defer queue size is equal (or larger) than the + * number of elements in the data structure. + */ +__rte_experimental +int +rte_rcu_qsbr_dq_enqueue(struct rte_rcu_qsbr_dq *dq, void *e); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Free quesed resources from the defer queue. + * + * This API is multi-thread safe. + * + * @param dq + * Defer queue to free an entry from. + * @param n + * Maximum number of resources to free. + * @param freed + * Number of resources that were freed. + * @param pending + * Number of resources pending on the defer queue. This number might not + * be acurate if multi-thread safety is configured. + * @return + * On successful reclamation of at least 1 resource - 0 + * On error - 1 with rte_errno set to + * - EINVAL - NULL parameters are passed + */ +__rte_experimental +int +rte_rcu_qsbr_dq_reclaim(struct rte_rcu_qsbr_dq *dq, unsigned int n, + unsigned int *freed, unsigned int *pending); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Delete a defer queue. + * + * It tries to reclaim all the resources on the defer queue. + * If any of the resources have not completed the grace period + * the reclamation stops and returns immediately. The rest of + * the resources are not reclaimed and the defer queue is not + * freed. + * + * @param dq + * Defer queue to delete. + * @return + * On success - 0 + * On error - 1 + * Possible rte_errno codes are: + * - EINVAL - NULL parameters are passed + * - EAGAIN - Some of the resources have not completed at least 1 grace + * period, try again. + */ +__rte_experimental +int +rte_rcu_qsbr_dq_delete(struct rte_rcu_qsbr_dq *dq); + #ifdef __cplusplus } #endif diff --git a/lib/librte_rcu/rte_rcu_version.map b/lib/librte_rcu/rte_rcu_version.map index f8b9ef2ab..dfac88a37 100644 --- a/lib/librte_rcu/rte_rcu_version.map +++ b/lib/librte_rcu/rte_rcu_version.map @@ -8,6 +8,10 @@ EXPERIMENTAL { rte_rcu_qsbr_synchronize; rte_rcu_qsbr_thread_register; rte_rcu_qsbr_thread_unregister; + rte_rcu_qsbr_dq_create; + rte_rcu_qsbr_dq_enqueue; + rte_rcu_qsbr_dq_reclaim; + rte_rcu_qsbr_dq_delete; local: *; }; diff --git a/lib/meson.build b/lib/meson.build index 9c3cc55d5..15e91a303 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -11,7 +11,9 @@ libraries = [ 'kvargs', # eal depends on kvargs 'eal', # everything depends on eal - 'ring', 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core + 'ring', + 'rcu', # rcu depends on ring + 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core 'cmdline', 'metrics', # bitrate/latency stats depends on this 'hash', # efd depends on this @@ -22,7 +24,7 @@ libraries = [ 'gro', 'gso', 'ip_frag', 'jobstats', 'kni', 'latencystats', 'lpm', 'member', 'power', 'pdump', 'rawdev', - 'rcu', 'rib', 'reorder', 'sched', 'security', 'stack', 'vhost', + 'rib', 'reorder', 'sched', 'security', 'stack', 'vhost', # ipsec lib depends on net, crypto and security 'ipsec', #fib lib depends on rib From patchwork Fri Apr 3 18:41:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 185175 Delivered-To: patch@linaro.org Received: by 2002:a92:de47:0:0:0:0:0 with SMTP id e7csp882375ilr; Fri, 3 Apr 2020 11:42:11 -0700 (PDT) X-Google-Smtp-Source: APiQypJpBqdc0f+AMe7cdWTwCWFiQMTrWL279vwxMDvt/iX81MHjVkFUjOvm3KJ2TykXOyI1O0d5 X-Received: by 2002:a7b:c24a:: with SMTP id b10mr10056951wmj.84.1585939331347; Fri, 03 Apr 2020 11:42:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1585939331; cv=none; d=google.com; s=arc-20160816; b=MXjy3PfqrBKuYYH5osHSMjaOx3H4X/6CPFNJOHczClE25hyEAoq1GhsDa+6OqIV/ld BK1r2ubLrlwogip8YM4SEMhvTsS3NuSHAnesmlSvmpdq05EYWLlPfVKFIrgX8Rdc5449 zcThS/pofsdIz4cC3KPUnmLekotPoF6oZEBhR9t9yJhI4rmJU9Z/HuAFs600y4lrxJUU f0i+HnrwUpbnYK3IJyRoOr0vpYwoYbscd2jON7mcntWswYM5Wux5CDlhfqgsKBD5Mm+m cfj7Q0gPJwFTF7C9P+HC2/JFrBpNGU+tPrA0OqhzpUVYZnftnzdiuIn9AgRBWWm7++Pg gGYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=bmjduD6X67I+CbflJPah1uhEWxX2mvoycEEHo9xbSN4=; b=amMPzcr3qa0HhS4XJLUQ/l4GapHPFxYUWCBCaZ0cQZHoufKQBZQ4GjI4EzcxBmrueA I3UqqQvyuAVO6+NEs7ekWeXQMaYNlT+tKV3cWbN/2raHFUYmUMdGa2P6OUnRQLD8Q3Zx sBwrxx5WyYLRJev4TBd8HvX06NYhLrmGTRnxLksNcSRBaswUHzbU8FG2kgy6udAUakuZ aZ8Jrp1nxtOBAgf62CUlMgqWbO3Q0kycbyppLEOMewmMJ+L92i653eAv+3Xm2UDWhWeR WG43v2WA0F/ZHxpO6RGUYEJzoeG8kSHTVi9pJNi8Gk4d3KdfDLso9ahgyasmpsdenfOy tIgA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id h4si8692331wru.482.2020.04.03.11.42.11; Fri, 03 Apr 2020 11:42:11 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 12B951C1DC; Fri, 3 Apr 2020 20:42:00 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 04A511C1D4 for ; Fri, 3 Apr 2020 20:41:58 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 700071045; Fri, 3 Apr 2020 11:41:58 -0700 (PDT) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.14.48]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 598B83F71E; Fri, 3 Apr 2020 11:41:58 -0700 (PDT) From: Honnappa Nagarahalli To: konstantin.ananyev@intel.com, stephen@networkplumber.org, vladimir.medvedkin@intel.com Cc: dev@dpdk.org, honnappa.nagarahalli@arm.com, ruifeng.wang@arm.com, dharmik.thakkar@arm.com, nd@arm.com Date: Fri, 3 Apr 2020 13:41:40 -0500 Message-Id: <20200403184142.7729-3-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200403184142.7729-1-honnappa.nagarahalli@arm.com> References: <20191001062917.35578-1-honnappa.nagarahalli@arm.com> <20200403184142.7729-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [PATCH v4 2/4] test/rcu: test cases for RCU defer queue APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add test cases for RCU defer queue APIs. Signed-off-by: Honnappa Nagarahalli --- app/test/test_rcu_qsbr.c | 365 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 363 insertions(+), 2 deletions(-) -- 2.17.1 diff --git a/app/test/test_rcu_qsbr.c b/app/test/test_rcu_qsbr.c index b60dc5099..94869b7b9 100644 --- a/app/test/test_rcu_qsbr.c +++ b/app/test/test_rcu_qsbr.c @@ -1,8 +1,9 @@ /* SPDX-License-Identifier: BSD-3-Clause - * Copyright (c) 2018 Arm Limited + * Copyright (c) 2019 Arm Limited */ #include +#include #include #include #include @@ -15,7 +16,8 @@ #include "test.h" /* Check condition and return an error if true. */ -#define TEST_RCU_QSBR_RETURN_IF_ERROR(cond, str, ...) do { \ +#define TEST_RCU_QSBR_RETURN_IF_ERROR(cond, str, ...) \ +do { \ if (cond) { \ printf("ERROR file %s, line %d: " str "\n", __FILE__, \ __LINE__, ##__VA_ARGS__); \ @@ -23,6 +25,16 @@ } \ } while (0) +/* Check condition and go to label if true. */ +#define TEST_RCU_QSBR_GOTO_IF_ERROR(label, cond, str, ...) \ +do { \ + if (cond) { \ + printf("ERROR file %s, line %d: " str "\n", __FILE__, \ + __LINE__, ##__VA_ARGS__); \ + goto label; \ + } \ +} while (0) + /* Make sure that this has the same value as __RTE_QSBR_CNT_INIT */ #define TEST_RCU_QSBR_CNT_INIT 1 @@ -34,6 +46,7 @@ static uint32_t *keys; #define COUNTER_VALUE 4096 static uint32_t *hash_data[RTE_MAX_LCORE][TOTAL_ENTRY]; static uint8_t writer_done; +static uint8_t cb_failed; static struct rte_rcu_qsbr *t[RTE_MAX_LCORE]; static struct rte_hash *h[RTE_MAX_LCORE]; @@ -585,6 +598,330 @@ test_rcu_qsbr_thread_offline(void) return 0; } +static void +test_rcu_qsbr_free_resource1(void *p, void *e) +{ + if (p != NULL || e != NULL) { + printf("%s: Test failed\n", __func__); + cb_failed = 1; + } +} + +static void +test_rcu_qsbr_free_resource2(void *p, void *e) +{ + if (p != NULL || e == NULL) { + printf("%s: Test failed\n", __func__); + cb_failed = 1; + } +} + +/* + * rte_rcu_qsbr_dq_create: create a queue used to store the data structure + * elements that can be freed later. This queue is referred to as 'defer queue'. + */ +static int +test_rcu_qsbr_dq_create(void) +{ + char rcu_dq_name[RTE_RCU_QSBR_DQ_NAMESIZE]; + struct rte_rcu_qsbr_dq_parameters params; + struct rte_rcu_qsbr_dq *dq; + + printf("\nTest rte_rcu_qsbr_dq_create()\n"); + + /* Pass invalid parameters */ + dq = rte_rcu_qsbr_dq_create(NULL); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + memset(¶ms, 0, sizeof(struct rte_rcu_qsbr_dq_parameters)); + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + snprintf(rcu_dq_name, sizeof(rcu_dq_name), "TEST_RCU"); + params.name = rcu_dq_name; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + params.free_fn = test_rcu_qsbr_free_resource1; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + rte_rcu_qsbr_init(t[0], RTE_MAX_LCORE); + params.v = t[0]; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + params.size = 1; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + params.esize = 3; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + params.trigger_reclaim_limit = 0; + params.max_reclaim_size = 0; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq != NULL), "dq create invalid params"); + + /* Pass all valid parameters */ + params.esize = 16; + params.trigger_reclaim_limit = 0; + params.max_reclaim_size = params.size; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq == NULL), "dq create valid params"); + rte_rcu_qsbr_dq_delete(dq); + + params.esize = 16; + params.flags = RTE_RCU_QSBR_DQ_MT_UNSAFE; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq == NULL), "dq create valid params"); + rte_rcu_qsbr_dq_delete(dq); + + return 0; +} + +/* + * rte_rcu_qsbr_dq_enqueue: enqueue one resource to the defer queue, + * to be freed later after atleast one grace period is over. + */ +static int +test_rcu_qsbr_dq_enqueue(void) +{ + int ret; + uint64_t r; + char rcu_dq_name[RTE_RCU_QSBR_DQ_NAMESIZE]; + struct rte_rcu_qsbr_dq_parameters params; + struct rte_rcu_qsbr_dq *dq; + + printf("\nTest rte_rcu_qsbr_dq_enqueue()\n"); + + /* Create a queue with simple parameters */ + memset(¶ms, 0, sizeof(struct rte_rcu_qsbr_dq_parameters)); + snprintf(rcu_dq_name, sizeof(rcu_dq_name), "TEST_RCU"); + params.name = rcu_dq_name; + params.free_fn = test_rcu_qsbr_free_resource1; + rte_rcu_qsbr_init(t[0], RTE_MAX_LCORE); + params.v = t[0]; + params.size = 1; + params.esize = 16; + params.trigger_reclaim_limit = 0; + params.max_reclaim_size = params.size; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq == NULL), "dq create valid params"); + + /* Pass invalid parameters */ + ret = rte_rcu_qsbr_dq_enqueue(NULL, NULL); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 0), "dq enqueue invalid params"); + + ret = rte_rcu_qsbr_dq_enqueue(dq, NULL); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 0), "dq enqueue invalid params"); + + ret = rte_rcu_qsbr_dq_enqueue(NULL, &r); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 0), "dq enqueue invalid params"); + + ret = rte_rcu_qsbr_dq_delete(dq); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 1), "dq delete valid params"); + + return 0; +} + +/* + * rte_rcu_qsbr_dq_reclaim: Reclaim resources from the defer queue. + */ +static int +test_rcu_qsbr_dq_reclaim(void) +{ + int ret; + char rcu_dq_name[RTE_RCU_QSBR_DQ_NAMESIZE]; + struct rte_rcu_qsbr_dq_parameters params; + struct rte_rcu_qsbr_dq *dq; + + printf("\nTest rte_rcu_qsbr_dq_reclaim()\n"); + + /* Pass invalid parameters */ + ret = rte_rcu_qsbr_dq_reclaim(NULL, 10, NULL, NULL); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret != 1), "dq reclaim invalid params"); + + /* Pass invalid parameters */ + memset(¶ms, 0, sizeof(struct rte_rcu_qsbr_dq_parameters)); + snprintf(rcu_dq_name, sizeof(rcu_dq_name), "TEST_RCU"); + params.name = rcu_dq_name; + params.free_fn = test_rcu_qsbr_free_resource1; + rte_rcu_qsbr_init(t[0], RTE_MAX_LCORE); + params.v = t[0]; + params.size = 1; + params.esize = 3; + params.trigger_reclaim_limit = 0; + params.max_reclaim_size = params.size; + dq = rte_rcu_qsbr_dq_create(¶ms); + ret = rte_rcu_qsbr_dq_reclaim(dq, 0, NULL, NULL); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret != 1), "dq reclaim invalid params"); + + return 0; +} + +/* + * rte_rcu_qsbr_dq_delete: Delete a defer queue. + */ +static int +test_rcu_qsbr_dq_delete(void) +{ + int ret; + char rcu_dq_name[RTE_RCU_QSBR_DQ_NAMESIZE]; + struct rte_rcu_qsbr_dq_parameters params; + struct rte_rcu_qsbr_dq *dq; + + printf("\nTest rte_rcu_qsbr_dq_delete()\n"); + + /* Pass invalid parameters */ + ret = rte_rcu_qsbr_dq_delete(NULL); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret != 1), "dq delete invalid params"); + + memset(¶ms, 0, sizeof(struct rte_rcu_qsbr_dq_parameters)); + snprintf(rcu_dq_name, sizeof(rcu_dq_name), "TEST_RCU"); + params.name = rcu_dq_name; + params.free_fn = test_rcu_qsbr_free_resource1; + rte_rcu_qsbr_init(t[0], RTE_MAX_LCORE); + params.v = t[0]; + params.size = 1; + params.esize = 16; + params.trigger_reclaim_limit = 0; + params.max_reclaim_size = params.size; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq == NULL), "dq create valid params"); + ret = rte_rcu_qsbr_dq_delete(dq); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret != 0), "dq delete valid params"); + + return 0; +} + +/* + * rte_rcu_qsbr_dq_enqueue: enqueue one resource to the defer queue, + * to be freed later after atleast one grace period is over. + */ +static int +test_rcu_qsbr_dq_functional(int32_t size, int32_t esize, uint32_t flags) +{ + int i, j, ret; + char rcu_dq_name[RTE_RCU_QSBR_DQ_NAMESIZE]; + struct rte_rcu_qsbr_dq_parameters params; + struct rte_rcu_qsbr_dq *dq; + uint64_t *e; + uint64_t sc = 200; + int max_entries; + + printf("\nTest rte_rcu_qsbr_dq_xxx functional tests()\n"); + printf("Size = %d, esize = %d, flags = 0x%x\n", size, esize, flags); + + e = (uint64_t *)rte_zmalloc(NULL, esize, RTE_CACHE_LINE_SIZE); + if (e == NULL) + return 0; + cb_failed = 0; + + /* Initialize the RCU variable. No threads are registered */ + rte_rcu_qsbr_init(t[0], RTE_MAX_LCORE); + + /* Create a queue with simple parameters */ + memset(¶ms, 0, sizeof(struct rte_rcu_qsbr_dq_parameters)); + snprintf(rcu_dq_name, sizeof(rcu_dq_name), "TEST_RCU"); + params.name = rcu_dq_name; + params.flags = flags; + params.free_fn = test_rcu_qsbr_free_resource2; + params.v = t[0]; + params.size = size; + params.esize = esize; + params.trigger_reclaim_limit = size >> 3; + params.max_reclaim_size = (size >> 4)?(size >> 4):1; + dq = rte_rcu_qsbr_dq_create(¶ms); + TEST_RCU_QSBR_RETURN_IF_ERROR((dq == NULL), "dq create valid params"); + + /* Given the size calculate the maximum number of entries + * that can be stored on the defer queue (look at the logic used + * in capacity calculation of rte_ring). + */ + max_entries = rte_align32pow2(size + 1) - 1; + printf("max_entries = %d\n", max_entries); + + /* Enqueue few counters starting with the value 'sc' */ + /* The queue size will be rounded up to 2. The enqueue API also + * reclaims if the queue size is above certain limit. Since, there + * are no threads registered, reclamation succedes. Hence, it should + * be possible to enqueue more than the provided queue size. + */ + for (i = 0; i < 10; i++) { + ret = rte_rcu_qsbr_dq_enqueue(dq, e); + TEST_RCU_QSBR_GOTO_IF_ERROR(end, (ret != 0), + "dq enqueue functional, i = %d", i); + for (j = 0; j < esize/8; j++) + e[j] = sc++; + } + + /* Validate that call back function did not return any error */ + TEST_RCU_QSBR_GOTO_IF_ERROR(end, (cb_failed == 1), "CB failed"); + + /* Register a thread on the RCU QSBR variable. Reclamation will not + * succeed. It should not be possible to enqueue more than the size + * number of resources. + */ + rte_rcu_qsbr_thread_register(t[0], 1); + rte_rcu_qsbr_thread_online(t[0], 1); + + for (i = 0; i < max_entries; i++) { + ret = rte_rcu_qsbr_dq_enqueue(dq, e); + TEST_RCU_QSBR_GOTO_IF_ERROR(end, (ret != 0), + "dq enqueue functional, max_entries = %d, i = %d", + max_entries, i); + for (j = 0; j < esize/8; j++) + e[j] = sc++; + } + + /* Enqueue fails as queue is full */ + ret = rte_rcu_qsbr_dq_enqueue(dq, e); + TEST_RCU_QSBR_GOTO_IF_ERROR(end, (ret == 0), "defer queue is not full"); + + /* Delete should fail as there are elements in defer queue which + * cannot be reclaimed. + */ + ret = rte_rcu_qsbr_dq_delete(dq); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret == 0), "dq delete valid params"); + + /* Report quiescent state, enqueue should succeed */ + rte_rcu_qsbr_quiescent(t[0], 1); + for (i = 0; i < max_entries; i++) { + ret = rte_rcu_qsbr_dq_enqueue(dq, e); + TEST_RCU_QSBR_GOTO_IF_ERROR(end, (ret != 0), + "dq enqueue functional"); + for (j = 0; j < esize/8; j++) + e[j] = sc++; + } + + /* Validate that call back function did not return any error */ + TEST_RCU_QSBR_GOTO_IF_ERROR(end, (cb_failed == 1), "CB failed"); + + /* Queue is full */ + ret = rte_rcu_qsbr_dq_enqueue(dq, e); + TEST_RCU_QSBR_GOTO_IF_ERROR(end, (ret == 0), "defer queue is not full"); + + /* Report quiescent state, delete should succeed */ + rte_rcu_qsbr_quiescent(t[0], 1); + ret = rte_rcu_qsbr_dq_delete(dq); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret != 0), "dq delete valid params"); + + rte_free(e); + + /* Validate that call back function did not return any error */ + TEST_RCU_QSBR_RETURN_IF_ERROR((cb_failed == 1), "CB failed"); + + return 0; + +end: + rte_free(e); + ret = rte_rcu_qsbr_dq_delete(dq); + TEST_RCU_QSBR_RETURN_IF_ERROR((ret != 0), "dq delete valid params"); + return -1; +} + /* * rte_rcu_qsbr_dump: Dump status of a single QS variable to a file */ @@ -1028,6 +1365,18 @@ test_rcu_qsbr_main(void) if (test_rcu_qsbr_thread_offline() < 0) goto test_fail; + if (test_rcu_qsbr_dq_create() < 0) + goto test_fail; + + if (test_rcu_qsbr_dq_reclaim() < 0) + goto test_fail; + + if (test_rcu_qsbr_dq_delete() < 0) + goto test_fail; + + if (test_rcu_qsbr_dq_enqueue() < 0) + goto test_fail; + printf("\nFunctional tests\n"); if (test_rcu_qsbr_sw_sv_3qs() < 0) @@ -1036,6 +1385,18 @@ test_rcu_qsbr_main(void) if (test_rcu_qsbr_mw_mv_mqs() < 0) goto test_fail; + if (test_rcu_qsbr_dq_functional(1, 8, 0) < 0) + goto test_fail; + + if (test_rcu_qsbr_dq_functional(2, 8, RTE_RCU_QSBR_DQ_MT_UNSAFE) < 0) + goto test_fail; + + if (test_rcu_qsbr_dq_functional(303, 16, 0) < 0) + goto test_fail; + + if (test_rcu_qsbr_dq_functional(7, 128, RTE_RCU_QSBR_DQ_MT_UNSAFE) < 0) + goto test_fail; + free_rcu(); printf("\n"); From patchwork Fri Apr 3 18:41:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 185176 Delivered-To: patch@linaro.org Received: by 2002:a92:de47:0:0:0:0:0 with SMTP id e7csp882541ilr; Fri, 3 Apr 2020 11:42:21 -0700 (PDT) X-Google-Smtp-Source: APiQypLsCxg2NgBYU7SjVPznD4IP2Jnuq7CRA/2wQJrnrzvZONVP3lr5jaeDJZYZIHYM7LOjLoiM X-Received: by 2002:adf:e6c8:: with SMTP id y8mr9985479wrm.279.1585939341863; Fri, 03 Apr 2020 11:42:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1585939341; cv=none; d=google.com; s=arc-20160816; b=W7WTf9ZWuj8cr8Kcl1HFGzIRxz9sAkLDKTgu6LFejztb+iAvr+zOkh/uT8tN2Jkc9S xhXnen4i5nXxTpi91LbPAD4u9UNJHbN8+hPZuc2FfHrmTVHwvcH/uA76kNaYU3AYCVKy hOiQBjTSm5cApR3lAx3Z1GyLA/1GuKfxDiWcC35VvJGWLlnFtgLyOSll5PudfJ9tHmtH bW0NRBJPGaiQ5Y2bbu2X9CpEla/Lj2RrmTKNJa5hE9HuwWh48o817OZDl2o1h1tjDEml nCd0rFMzxwHYUEJQSmTTdQo2P/58xAhQUQ86fiXZcATb9DqIDiH4Blel9rnWFe9x0f9I +QzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=ZrqSEMp/DAwtZXcEjmBy5lcyb5iGyUMo+qQH90tNDOY=; b=oBPzHVh7jpaoVFKI2EUv3OH5j2zPbZ5IaR0tq9ZjCq5PzsSH5PVzWV5gegJOQ+2mk4 J1eYgY7ylvF+yFmb6Tc96eW4EeIJmG5o+Dcyk3xOV7k2l5UEKEnI6hXGF97LqTXQadms 6IFPxoGob7sJKbRcwa4MYqBgZF0gxTil1o6C8loNI5dxcuAXGJvNzFLBBJZ0paqOMBVB i0HcFOEWm+roadgBo6gNKrRbbeFn9gbrml1mhIpAS4+aYP9RGtmR1+bKDANtk8f5BekJ eTGnc9Wi++abyqJQazFaJVH0mS7V//S80qf6G8JdIVcY8uwC8TcCLtcMUbwTudXtkz6K 4dGg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id b2si8570894wmc.62.2020.04.03.11.42.21; Fri, 03 Apr 2020 11:42:21 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D4CA31C1EB; Fri, 3 Apr 2020 20:42:01 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id D96FA1C1D4 for ; Fri, 3 Apr 2020 20:41:59 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 636891FB; Fri, 3 Apr 2020 11:41:59 -0700 (PDT) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.14.48]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5192B3F71E; Fri, 3 Apr 2020 11:41:59 -0700 (PDT) From: Honnappa Nagarahalli To: konstantin.ananyev@intel.com, stephen@networkplumber.org, vladimir.medvedkin@intel.com Cc: dev@dpdk.org, honnappa.nagarahalli@arm.com, ruifeng.wang@arm.com, dharmik.thakkar@arm.com, nd@arm.com Date: Fri, 3 Apr 2020 13:41:41 -0500 Message-Id: <20200403184142.7729-4-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200403184142.7729-1-honnappa.nagarahalli@arm.com> References: <20191001062917.35578-1-honnappa.nagarahalli@arm.com> <20200403184142.7729-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [PATCH v4 3/4] doc/rcu: add RCU integration design details X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ruifeng Wang Add a section to describe a design to integrate QSBR RCU library with other libraries in DPDK. Signed-off-by: Honnappa Nagarahalli --- doc/guides/prog_guide/rcu_lib.rst | 59 +++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) -- 2.17.1 diff --git a/doc/guides/prog_guide/rcu_lib.rst b/doc/guides/prog_guide/rcu_lib.rst index 9b0bf138f..484c7b882 100644 --- a/doc/guides/prog_guide/rcu_lib.rst +++ b/doc/guides/prog_guide/rcu_lib.rst @@ -190,3 +190,62 @@ However, when ``CONFIG_RTE_LIBRTE_RCU_DEBUG`` is enabled, these APIs aid in debugging issues. One can mark the access to shared data structures on the reader side using these APIs. The ``rte_rcu_qsbr_quiescent()`` will check if all the locks are unlocked. + +Resource reclamation framework for DPDK +--------------------------------------- + +Lock-free algorithms place additional burden of resource reclamation on +the application. When a writer deletes an entry from a data structure, the writer: + +#. Has to start the grace period +#. Has to store a reference to the deleted resources in a FIFO +#. Should check if the readers have completed a grace period and free the resources. This can also be done when the writer runs out of free resources. + +There are several APIs provided to help with this process. The writer +can create a FIFO to store the references to deleted resources using ``rte_rcu_qsbr_dq_create()``. +The resources can be enqueued to this FIFO using ``rte_rcu_qsbr_dq_enqueue()``. +If the FIFO is full, ``rte_rcu_qsbr_dq_enqueue`` will reclaim the resources before enqueuing. It will also reclaim resources on regular basis to keep the FIFO from growing too large. If the writer runs out of resources, the writer can call ``rte_rcu_qsbr_dq_reclaim`` API to reclaim resources. ``rte_rcu_qsbr_dq_delete`` is provided to reclaim any remaining resources and free the FIFO while shutting down. + +However, if this resource reclamation process were to be integrated in lock-free data structure libraries, it +hides this complexity from the application and makes it easier for the application to adopt lock-free algorithms. The following paragraphs discuss how the reclamation process can be integrated in DPDK libraries. + +In any DPDK application, the resource reclamation process using QSBR can be split into 4 parts: + +#. Initialization +#. Quiescent State Reporting +#. Reclaiming Resources +#. Shutdown + +The design proposed here assigns different parts of this process to client libraries and applications. The term 'client library' refers to lock-free data structure libraries such at rte_hash, rte_lpm etc. in DPDK or similar libraries outside of DPDK. The term 'application' refers to the packet processing application that makes use of DPDK such as L3 Forwarding example application, OVS, VPP etc.. + +The application has to handle 'Initialization' and 'Quiescent State Reporting'. So, + +* the application has to create the RCU variable and register the reader threads to report their quiescent state. +* the application has to register the same RCU variable with the client library. +* reader threads in the application have to report the quiescent state. This allows for the application to control the length of the critical section/how frequently the application wants to report the quiescent state. + +The client library will handle 'Reclaiming Resources' part of the process. The +client libraries will make use of the writer thread context to execute the memory +reclamation algorithm. So, + +* client library should provide an API to register a RCU variable that it will use. It should call ``rte_rcu_qsbr_dq_create()`` to create the FIFO to store the references to deleted entries. +* client library should use ``rte_rcu_qsbr_dq_enqueue`` to enqueue the deleted resources on the FIFO and start the grace period. +* if the library runs out of resources while adding entries, it should call ``rte_rcu_qsbr_dq_reclaim`` to reclaim the resources and try the resource allocation again. + +The 'Shutdown' process needs to be shared between the application and the +client library. + +* the application should make sure that the reader threads are not using the shared data structure, unregister the reader threads from the QSBR variable before calling the client library's shutdown function. + +* client library should call ``rte_rcu_qsbr_dq_delete`` to reclaim any remaining resources and free the FIFO. + +Integrating the resource reclamation with client libraries removes the burden from +the application and makes it easy to use lock-free algorithms. + +This design has several advantages over currently known methods. + +#. Application does not need a dedicated thread to reclaim resources. Memory + reclamation happens as part of the writer thread with little impact on + performance. +#. The client library has better control over the resources. For ex: the client + library can attempt to reclaim when it has run out of resources. From patchwork Fri Apr 3 18:41:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 185177 Delivered-To: patch@linaro.org Received: by 2002:a92:de47:0:0:0:0:0 with SMTP id e7csp882663ilr; Fri, 3 Apr 2020 11:42:30 -0700 (PDT) X-Google-Smtp-Source: APiQypJXSzGyLUGuiLw9Zv0HVAOxqlJtGjiE4qn1uQqE+I0Yo56T0wDPDWbVnmUgd0Np5NSDe5HE X-Received: by 2002:a5d:400d:: with SMTP id n13mr10473019wrp.396.1585939350814; Fri, 03 Apr 2020 11:42:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1585939350; cv=none; d=google.com; s=arc-20160816; b=ykYPZeEWWQNuJ4+SL+CjRq+6vJsIr/G3nfpSZEm8XDmdt9WaGex/3do9fwQJIfdG74 4HCLSdY4I2gIbgHD2rPbSJt1L5bLNtjW1lR6YWYdSlyKV7rsam8Vpoo7ZdJgEfXkfEvv h0ogcawRL5/GAGS0FG4zxm4WPhYNfLEQEL3vq/C3EPCm6VA+NBiP6jMJMAvZDT+Cla5q P/gxrcgDxjzTPUQohqntjVXRlQsx6QDcE0RsY5v+SEaTVyFmRVHgRK/9VpHmmXvm58fS 4XbQVjcCSfNis2vo309afSJMhATCazsLjz8CzS9WnBc+hJINVdFHLUa9yPvxrUoy+BHf HzhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=IQQ9bDsWhiU7su9Cmq1YEO3OcuxDo7YXhJRXqIpkUUk=; b=0uSXlAchRpVslNhrKPeHlNUMiLgfEMCuf4AiiU9QNUWcMCYl8nGGOfxu7uoA+E/fRu jZHCQaIAJECyNC/y6McGc3u5sNmKxWiWF+VzWPG2i4wRdEYAL1tqv637HKRc8yFf3ogV GIH+2yVI3jsTW0J0PKB73WmDJSXsLA0QKy4NChlaUjAVmadMaJBgcyUDROvszVCrUwrk Qcblb2dK06RCXIIbzu5s/LD/Rc9YxeaOOVApTHDankRYQl702RTsiUAYdRkg8A/4Q3UK QY5OYkfZ0Ko/HLMzDmaTg0nEbfqMZJ6xn2kBnvZOrJvcAH4/hxdVegIy8EZoSOwpwFM+ em/g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id c2si4037512wru.415.2020.04.03.11.42.30; Fri, 03 Apr 2020 11:42:30 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3EE391C1F6; Fri, 3 Apr 2020 20:42:03 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id B8D391C1DF for ; Fri, 3 Apr 2020 20:42:00 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 434C01045; Fri, 3 Apr 2020 11:42:00 -0700 (PDT) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.14.48]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 32DD53F71E; Fri, 3 Apr 2020 11:42:00 -0700 (PDT) From: Honnappa Nagarahalli To: konstantin.ananyev@intel.com, stephen@networkplumber.org, vladimir.medvedkin@intel.com Cc: dev@dpdk.org, honnappa.nagarahalli@arm.com, ruifeng.wang@arm.com, dharmik.thakkar@arm.com, nd@arm.com Date: Fri, 3 Apr 2020 13:41:42 -0500 Message-Id: <20200403184142.7729-5-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200403184142.7729-1-honnappa.nagarahalli@arm.com> References: <20191001062917.35578-1-honnappa.nagarahalli@arm.com> <20200403184142.7729-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [PATCH v4 4/4] lib/rcu: add additional debug logs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Added additional debug logs. These helped in debugging RCU defer APIs. Signed-off-by: Honnappa Nagarahalli --- lib/librte_rcu/rte_rcu_qsbr.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) -- 2.17.1 diff --git a/lib/librte_rcu/rte_rcu_qsbr.h b/lib/librte_rcu/rte_rcu_qsbr.h index 213f9b029..dd51e7e35 100644 --- a/lib/librte_rcu/rte_rcu_qsbr.h +++ b/lib/librte_rcu/rte_rcu_qsbr.h @@ -716,8 +716,15 @@ rte_rcu_qsbr_check(struct rte_rcu_qsbr *v, uint64_t t, bool wait) RTE_ASSERT(v != NULL); /* Check if all the readers have already acknowledged this token */ - if (likely(t <= v->acked_token)) + if (likely(t <= v->acked_token)) { + __RTE_RCU_DP_LOG(DEBUG, + "%s: check: token = %"PRIu64", wait = %d", + __func__, t, wait); + __RTE_RCU_DP_LOG(DEBUG, + "%s: status: least acked token = %"PRIu64"", + __func__, v->acked_token); return 1; + } if (likely(v->num_threads == v->max_threads)) return __rte_rcu_qsbr_check_all(v, t, wait);