From patchwork Mon Aug 30 16:34:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 504335 Delivered-To: patch@linaro.org Received: by 2002:a02:8629:0:0:0:0:0 with SMTP id e38csp1948592jai; Mon, 30 Aug 2021 09:34:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyws0qSAU4vdYnQdDoeJMeKD19XbNGHQ04YkvkSAJRwS8kR4mQJcrm8F/fccRDvEvS+3F0k X-Received: by 2002:a5d:61c1:: with SMTP id q1mr26576663wrv.87.1630341272404; Mon, 30 Aug 2021 09:34:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630341272; cv=none; d=google.com; s=arc-20160816; b=Hlsr4XWyh56UBehDMCrcZBhS2lq9hhDNKkeOnbUOnz51MSvCUoH74fsNSupRV3jKsk Ri1JQmTbb7aaYUPOjhn7sLDZcX+Ai8+Hi+k1BIkCR/EaUae3mWof77C1HYJFKmXPmX0W a1R/7jMGDnplKIuHdjM2/ZdypqXQEEEt3avPlCt87JL7Xwmu6SoHq6YHbUmh1nqgRdUw Z3V0JlcfsxLq9kgwuleF2QHcxsbj9ArkVIi5wNGOAry8Me01XFKnpmUfnU3ap7sG4doO C+8IvePs+OtUxOY0vnqcoeNymuiD28CiIxlYFTphW55zs9NPqi2+fajf2P4chFDliNfy cvng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=YBmuuVtaCi3uO4iuedqNl4Idha9Ip+9NOMdI5T4/pTk=; b=z3AfAhoyvbKN0/cJiABip1cF4hoX7jtZu2P/umu4BLpGss+FxnTzcQIt44lLtppsxx tPYdfbzpyRVLbstXSa27Akeq1LM/1euDmmoLEE+CbNUOikmle6xCiEf5+uV7KkumCuNE 5tpynMmhDuTKcuFX0cMz+ZTFLORmcTMy1sLpJKB6PEoip3iE363f4gp1E9f9h+N4/FYm gjly7LlmMWFVLbm9iUYSYqR2YIpejH+IS2KqTvZ2V0YVUVZ8a3R1wfLyvf8vzRrgZG5F hgYceasRvsD7Lte6qUXlG2MPK1cNQFwRFFNfNuSIEQ6i6SkCxJUhZCPcSr6nmBhYD2IU pOPA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from mails.dpdk.org (mails.dpdk.org. [217.70.189.124]) by mx.google.com with ESMTP id m5si16579527wrs.43.2021.08.30.09.34.32; Mon, 30 Aug 2021 09:34:32 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) client-ip=217.70.189.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D431C410FE; Mon, 30 Aug 2021 18:34:31 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id A6540410F2 for ; Mon, 30 Aug 2021 18:34:30 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 204861FB; Mon, 30 Aug 2021 09:34:30 -0700 (PDT) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.12.44]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 137A53F766; Mon, 30 Aug 2021 09:34:30 -0700 (PDT) From: Honnappa Nagarahalli To: dev@dpdk.org, honnappa.nagarahalli@arm.com, olivier.matz@6wind.com, lucp.at.work@gmail.com, stephen@networkplumber.org, david.marchand@redhat.com, thomas@monjalon.net Cc: ruifeng.wang@arm.com, nd@arm.com Date: Mon, 30 Aug 2021 11:34:12 -0500 Message-Id: <20210830163413.31576-1-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210730213709.19400-1-honnappa.nagarahalli@arm.com> References: <20210730213709.19400-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [PATCH v3 1/2] eal: simplify the implementation of rte_ctrl_thread_create X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Remove the usage of pthread barrier and replace it with synchronization using atomic variable. This also removes the use of reference count required to synchronize freeing the memory. Signed-off-by: Honnappa Nagarahalli --- v3: 1) RFC to Patch 2) Changed commit log and API description [Olivier] 3) Used sched_yield while spinning [Stephen] 4) Fixed use after free 5) Added a test case v2: 1) Used compiler's C++11 atomic built-ins to access the shared variable. lib/eal/common/eal_common_thread.c | 83 +++++++++++++++--------------- lib/eal/include/rte_lcore.h | 9 ++-- 2 files changed, 46 insertions(+), 46 deletions(-) -- 2.25.1 diff --git a/lib/eal/common/eal_common_thread.c b/lib/eal/common/eal_common_thread.c index 1a52f42a2b..b0af965b7b 100644 --- a/lib/eal/common/eal_common_thread.c +++ b/lib/eal/common/eal_common_thread.c @@ -166,38 +166,46 @@ __rte_thread_uninit(void) RTE_PER_LCORE(_lcore_id) = LCORE_ID_ANY; } +#define _RTE_CTRL_THREAD_LAUNCHING 0 +#define _RTE_CTRL_THREAD_RUNNING 1 +#define _RTE_CTRL_THREAD_ERROR 2 + struct rte_thread_ctrl_params { void *(*start_routine)(void *); void *arg; - pthread_barrier_t configured; - unsigned int refcnt; + int ret; + /* Control thread status. + * + * _RTE_CTRL_THREAD_LAUNCHING - Yet to call pthread_create function + * _RTE_CTRL_THREAD_RUNNING - Control thread is running successfully + * _RTE_CTRL_THREAD_ERROR - Control thread encountered an error. + * 'ret' has the error code. + */ + unsigned int ctrl_thread_status; }; -static void ctrl_params_free(struct rte_thread_ctrl_params *params) -{ - if (__atomic_sub_fetch(¶ms->refcnt, 1, __ATOMIC_ACQ_REL) == 0) { - (void)pthread_barrier_destroy(¶ms->configured); - free(params); - } -} - static void *ctrl_thread_init(void *arg) { struct internal_config *internal_conf = eal_get_internal_configuration(); rte_cpuset_t *cpuset = &internal_conf->ctrl_cpuset; struct rte_thread_ctrl_params *params = arg; - void *(*start_routine)(void *); + void *(*start_routine)(void *) = params->start_routine; void *routine_arg = params->arg; __rte_thread_init(rte_lcore_id(), cpuset); - - pthread_barrier_wait(¶ms->configured); - start_routine = params->start_routine; - ctrl_params_free(params); - - if (start_routine == NULL) + params->ret = pthread_setaffinity_np(pthread_self(), + sizeof(*cpuset), cpuset); + if (params->ret != 0) { + __atomic_store_n(¶ms->ctrl_thread_status, + _RTE_CTRL_THREAD_ERROR, + __ATOMIC_RELEASE); return NULL; + } + + __atomic_store_n(¶ms->ctrl_thread_status, + _RTE_CTRL_THREAD_RUNNING, + __ATOMIC_RELEASE); return start_routine(routine_arg); } @@ -207,10 +215,8 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name, const pthread_attr_t *attr, void *(*start_routine)(void *), void *arg) { - struct internal_config *internal_conf = - eal_get_internal_configuration(); - rte_cpuset_t *cpuset = &internal_conf->ctrl_cpuset; struct rte_thread_ctrl_params *params; + unsigned int ctrl_thread_status; int ret; params = malloc(sizeof(*params)); @@ -219,15 +225,14 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name, params->start_routine = start_routine; params->arg = arg; - params->refcnt = 2; - - ret = pthread_barrier_init(¶ms->configured, NULL, 2); - if (ret != 0) - goto fail_no_barrier; + params->ret = 0; + params->ctrl_thread_status = _RTE_CTRL_THREAD_LAUNCHING; ret = pthread_create(thread, attr, ctrl_thread_init, (void *)params); - if (ret != 0) - goto fail_with_barrier; + if (ret != 0) { + free(params); + return -ret; + } if (name != NULL) { ret = rte_thread_setname(*thread, name); @@ -236,24 +241,18 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name, "Cannot set name for ctrl thread\n"); } - ret = pthread_setaffinity_np(*thread, sizeof(*cpuset), cpuset); - if (ret != 0) - params->start_routine = NULL; - - pthread_barrier_wait(¶ms->configured); - ctrl_params_free(params); + /* Wait for the control thread to initialize successfully */ + while ((ctrl_thread_status = + __atomic_load_n(¶ms->ctrl_thread_status, __ATOMIC_ACQUIRE)) + == _RTE_CTRL_THREAD_LAUNCHING) + sched_yield(); - if (ret != 0) - /* start_routine has been set to NULL above; */ - /* ctrl thread will exit immediately */ + /* Check if the control thread encountered an error */ + if (ctrl_thread_status == _RTE_CTRL_THREAD_ERROR) + /* ctrl thread is exiting */ pthread_join(*thread, NULL); - return -ret; - -fail_with_barrier: - (void)pthread_barrier_destroy(¶ms->configured); - -fail_no_barrier: + ret = params->ret; free(params); return -ret; diff --git a/lib/eal/include/rte_lcore.h b/lib/eal/include/rte_lcore.h index 1550b75da0..b81c8b2685 100644 --- a/lib/eal/include/rte_lcore.h +++ b/lib/eal/include/rte_lcore.h @@ -420,10 +420,11 @@ rte_thread_unregister(void); /** * Create a control thread. * - * Wrapper to pthread_create(), pthread_setname_np() and - * pthread_setaffinity_np(). The affinity of the new thread is based - * on the CPU affinity retrieved at the time rte_eal_init() was called, - * the dataplane and service lcores are then excluded. + * Creates a control thread with the given name and attributes. The + * affinity of the new thread is based on the CPU affinity retrieved + * at the time rte_eal_init() was called, the dataplane and service + * lcores are then excluded. If setting the name of the thread fails, + * the error is ignored and a debug message is logged. * * @param thread * Filled with the thread id of the new created thread.