From patchwork Thu Sep 9 23:13:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 508372 Delivered-To: patch@linaro.org Received: by 2002:a02:8629:0:0:0:0:0 with SMTP id e38csp767594jai; Thu, 9 Sep 2021 16:13:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyssOeFSbGg/zE0M+v9lL0vnQ2HwrUoBSyYZ2hwtP3aFQdTXmuDimp40zSmFBOjeeHPkO6x X-Received: by 2002:adf:ec81:: with SMTP id z1mr6254614wrn.181.1631229222144; Thu, 09 Sep 2021 16:13:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1631229222; cv=none; d=google.com; s=arc-20160816; b=srkJPtELME25Lf2eZfIS9ySZZU/T83q/rjSyQxURftu/6QkA3PDpO8dHcFBMUmYdrS XwnxrPGZyiz0ji/79uXVeWxvU+KfZXmPBFAd0KnssFpWSBa5zVKKQOfXrW+4PHRitHv/ xqYnDeD3PoksEtg/YUrHfl6Cghpq/rZIjNzk+wWVbMh8cmpwSHP0/vzz8WADM2Wkr0GF 65WXpemCbEfU5ohe0vWTcaQZkX5hg+dlR1UjOypy/exrBDBkIbZGTb4TmuWsfoPq952z Z1o6ln7LSoc0DRbgNO2i248ZE7MTntNHeyrvWDIAyGJzh5SBYB3v3s3KrSQ97smeKgwO G9pw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:cc:to:from; bh=I/+22NAQTIWj20kbmyY1/GuQKqGyUglilmzXAX+qdUA=; b=ACVyGGCDN1gQlFZQ7QnNT5PmxHQ54pQ1DJKDTwfPoKFJEkL3twQItTcCZ7/X6fiAiO P8Pb2qc24RN6grFmjxeH9vPrIrEDCdgny5IUONpE1HKj/9vEgCtuJPmK807XSuj/lqaN YLd8fDKjqeoaJEhxzSjMzhdRnig6gLQA6TdoDAz4ZaNBu5bHv6OVIP+5HmD0n1TSpGfB wQCudw6dyLDt4xu8DCpzLzOooKcaB/o9KJ8nMUXQMN2/D05TLMLelZvCloaIXhIISAlB +0hQt/JsKZc07zmu2R5U1wd/PafYtQJQoFiwQlNDaRhI4sgaShINwkgEfwm2FYbrlrH0 Zx6A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from mails.dpdk.org (mails.dpdk.org. [217.70.189.124]) by mx.google.com with ESMTP id a9si3039783wrr.416.2021.09.09.16.13.41; Thu, 09 Sep 2021 16:13:42 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) client-ip=217.70.189.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DCEE240DF7; Fri, 10 Sep 2021 01:13:41 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 8B3DB40DF7 for ; Fri, 10 Sep 2021 01:13:40 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1137A31B; Thu, 9 Sep 2021 16:13:40 -0700 (PDT) Received: from 2p2660v4-1.austin.arm.com (2p2660v4-1.austin.arm.com [10.118.13.211]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0A5CA3F73D; Thu, 9 Sep 2021 16:13:40 -0700 (PDT) From: Honnappa Nagarahalli To: dev@dpdk.org, honnappa.nagarahalli@arm.com, konstantin.ananyev@intel.com, david.marchand@redhat.com, feifei.wang2@arm.com Cc: ruifeng.wang@arm.com, nd@arm.com, Feifei Wang Date: Thu, 9 Sep 2021 18:13:07 -0500 Message-Id: <20210909231312.2572006-2-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210909231312.2572006-1-honnappa.nagarahalli@arm.com> References: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> <20210909231312.2572006-1-honnappa.nagarahalli@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 1/6] eal: reset lcore function pointer and argument X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In the rte_eal_remote_launch function, the lcore function pointer is checked for NULL. However, the pointer is never reset to NULL. Reset the lcore function pointer and argument after the worker has completed executing the lcore function. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Ruifeng Wang Reviewed-by: Feifei Wang --- lib/eal/freebsd/eal_thread.c | 2 ++ lib/eal/linux/eal_thread.c | 2 ++ lib/eal/windows/eal_thread.c | 2 ++ 3 files changed, 6 insertions(+) -- 2.25.1 diff --git a/lib/eal/freebsd/eal_thread.c b/lib/eal/freebsd/eal_thread.c index 1dce9b04f2..bbc3a8e985 100644 --- a/lib/eal/freebsd/eal_thread.c +++ b/lib/eal/freebsd/eal_thread.c @@ -126,6 +126,8 @@ eal_thread_loop(__rte_unused void *arg) fct_arg = lcore_config[lcore_id].arg; ret = lcore_config[lcore_id].f(fct_arg); lcore_config[lcore_id].ret = ret; + lcore_config[lcore_id].f = NULL; + lcore_config[lcore_id].arg = NULL; rte_wmb(); lcore_config[lcore_id].state = FINISHED; } diff --git a/lib/eal/linux/eal_thread.c b/lib/eal/linux/eal_thread.c index 83c2034b93..8f3c0dafd6 100644 --- a/lib/eal/linux/eal_thread.c +++ b/lib/eal/linux/eal_thread.c @@ -126,6 +126,8 @@ eal_thread_loop(__rte_unused void *arg) fct_arg = lcore_config[lcore_id].arg; ret = lcore_config[lcore_id].f(fct_arg); lcore_config[lcore_id].ret = ret; + lcore_config[lcore_id].f = NULL; + lcore_config[lcore_id].arg = NULL; rte_wmb(); /* when a service core returns, it should go directly to WAIT diff --git a/lib/eal/windows/eal_thread.c b/lib/eal/windows/eal_thread.c index 9c3f6d69fd..df1df5d02c 100644 --- a/lib/eal/windows/eal_thread.c +++ b/lib/eal/windows/eal_thread.c @@ -110,6 +110,8 @@ eal_thread_loop(void *arg __rte_unused) fct_arg = lcore_config[lcore_id].arg; ret = lcore_config[lcore_id].f(fct_arg); lcore_config[lcore_id].ret = ret; + lcore_config[lcore_id].f = NULL; + lcore_config[lcore_id].arg = NULL; rte_wmb(); /* when a service core returns, it should go directly to WAIT From patchwork Thu Sep 9 23:13:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 508374 Delivered-To: patch@linaro.org Received: by 2002:a02:8629:0:0:0:0:0 with SMTP id e38csp767730jai; Thu, 9 Sep 2021 16:13:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzW99DA9o8p4GM8+JvPmxsRvqaO+93SWGOLB63CsMNk1itWZK26YJ6aXFlJjtAMg8LyQS5J X-Received: by 2002:a05:600c:3790:: with SMTP id o16mr5397796wmr.157.1631229232489; Thu, 09 Sep 2021 16:13:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1631229232; cv=none; d=google.com; s=arc-20160816; b=XQp26fTM2qAgcndCDRdZ9UCxx6gAH2h/N1tenmMrTBTgPZsGmHBEVG8qVrkCjGoMJI dTlp3jwljMnlgs2GGS579+1mNGa9HiRMVyMx51JpYAZHjj9HjIUIIMlEZ109zbxD1Qnz 6MNA6zZz9+rWLRB+Ju9FqCaY42yOAp9qD/O/LcYmzAB87vSB5SQozWDxz9QD52dWQSHu LB3Tw9hk3K4XWnNQMiGju+bIzm+iaVCpHG0ZyvPKB2qrAzTQnDx3GinCryTbXWxDS9Ge ZraN9w24Kr3LWIZhXF2Srv4MUYmqlatzPKg9jGHrgkuvIZGO8oFAaA/lqf4t6y7Fike4 PLQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:cc:to:from; bh=HF9bVxswE+JWzdLpvQLHiXnF3FxJiHkrE5rOpSPZIdA=; b=QezOcjtQCNPI41C2BnoP5Gmx+sWsu878bxGp5lLaXsVAIHt/6081LGVMzUMZtRHk5H LFyQD8QqTmkxMyHbe3kLZcmGjKWw66KEYnEsFjXlG4tStXqUfO2kALiXEDNboGW1GKoR 8yrrNa2RmBe4ZhzhliQm7+giwiE6BSUca2rSMFf4YkTlz9xedaX1zrpINQMoWO8u7gET Bp95LZfkRFUkmPXB6T58Q8bzIYsI+L1zitqv75dozJIdz158VfGdGZJ1Nylkw8VV18rV 2n2PRzVkBfYd5dIXKxatoR4vgYucm0EYVKj6sCzpSgCtV95CZqzCYRBaDMvT57gvyv89 z/gQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from mails.dpdk.org (mails.dpdk.org. [217.70.189.124]) by mx.google.com with ESMTP id n26si3297949wmk.27.2021.09.09.16.13.52; Thu, 09 Sep 2021 16:13:52 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) client-ip=217.70.189.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 843094112E; Fri, 10 Sep 2021 01:13:45 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id A1EFA410FE for ; Fri, 10 Sep 2021 01:13:42 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 13D3D31B; Thu, 9 Sep 2021 16:13:42 -0700 (PDT) Received: from 2p2660v4-1.austin.arm.com (2p2660v4-1.austin.arm.com [10.118.13.211]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0C1DD3F73D; Thu, 9 Sep 2021 16:13:42 -0700 (PDT) From: Honnappa Nagarahalli To: dev@dpdk.org, honnappa.nagarahalli@arm.com, konstantin.ananyev@intel.com, david.marchand@redhat.com, feifei.wang2@arm.com Cc: ruifeng.wang@arm.com, nd@arm.com, Ola Liljedahl , Feifei Wang Date: Thu, 9 Sep 2021 18:13:08 -0500 Message-Id: <20210909231312.2572006-3-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210909231312.2572006-1-honnappa.nagarahalli@arm.com> References: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> <20210909231312.2572006-1-honnappa.nagarahalli@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 2/6] eal: ensure memory operations are visible to worker X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Ensure that the memory operations before the call to rte_eal_remote_launch are visible to the worker thread. Use the function pointer to execute in worker thread as the guard variable. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Ola Liljedahl Reviewed-by: Feifei Wang --- lib/eal/freebsd/eal_thread.c | 19 +++++++++++++++---- lib/eal/linux/eal_thread.c | 19 +++++++++++++++---- lib/eal/windows/eal_thread.c | 19 +++++++++++++++---- 3 files changed, 45 insertions(+), 12 deletions(-) -- 2.25.1 diff --git a/lib/eal/freebsd/eal_thread.c b/lib/eal/freebsd/eal_thread.c index bbc3a8e985..17b8f39966 100644 --- a/lib/eal/freebsd/eal_thread.c +++ b/lib/eal/freebsd/eal_thread.c @@ -42,8 +42,12 @@ rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned worker_id) if (lcore_config[worker_id].state != WAIT) goto finish; - lcore_config[worker_id].f = f; lcore_config[worker_id].arg = arg; + /* Ensure that all the memory operations are completed + * before the worker thread starts running the function. + * Use worker thread function as the guard variable. + */ + __atomic_store_n(&lcore_config[worker_id].f, f, __ATOMIC_RELEASE); /* send message */ n = 0; @@ -100,6 +104,7 @@ eal_thread_loop(__rte_unused void *arg) /* read on our pipe to get commands */ while (1) { + lcore_function_t *f; void *fct_arg; /* wait command */ @@ -119,12 +124,18 @@ eal_thread_loop(__rte_unused void *arg) if (n < 0) rte_panic("cannot write on configuration pipe\n"); - if (lcore_config[lcore_id].f == NULL) - rte_panic("NULL function pointer\n"); + /* Load 'f' with acquire order to ensure that + * the memory operations from the main thread + * are accessed only after update to 'f' is visible. + * Wait till the update to 'f' is visible to the worker. + */ + while ((f = __atomic_load_n(&lcore_config[lcore_id].f, + __ATOMIC_ACQUIRE)) == NULL) + rte_pause(); /* call the function and store the return value */ fct_arg = lcore_config[lcore_id].arg; - ret = lcore_config[lcore_id].f(fct_arg); + ret = f(fct_arg); lcore_config[lcore_id].ret = ret; lcore_config[lcore_id].f = NULL; lcore_config[lcore_id].arg = NULL; diff --git a/lib/eal/linux/eal_thread.c b/lib/eal/linux/eal_thread.c index 8f3c0dafd6..a0a0091040 100644 --- a/lib/eal/linux/eal_thread.c +++ b/lib/eal/linux/eal_thread.c @@ -42,8 +42,12 @@ rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned int worker_id) if (lcore_config[worker_id].state != WAIT) goto finish; - lcore_config[worker_id].f = f; lcore_config[worker_id].arg = arg; + /* Ensure that all the memory operations are completed + * before the worker thread starts running the function. + * Use worker thread function as the guard variable. + */ + __atomic_store_n(&lcore_config[worker_id].f, f, __ATOMIC_RELEASE); /* send message */ n = 0; @@ -100,6 +104,7 @@ eal_thread_loop(__rte_unused void *arg) /* read on our pipe to get commands */ while (1) { + lcore_function_t *f; void *fct_arg; /* wait command */ @@ -119,12 +124,18 @@ eal_thread_loop(__rte_unused void *arg) if (n < 0) rte_panic("cannot write on configuration pipe\n"); - if (lcore_config[lcore_id].f == NULL) - rte_panic("NULL function pointer\n"); + /* Load 'f' with acquire order to ensure that + * the memory operations from the main thread + * are accessed only after update to 'f' is visible. + * Wait till the update to 'f' is visible to the worker. + */ + while ((f = __atomic_load_n(&lcore_config[lcore_id].f, + __ATOMIC_ACQUIRE)) == NULL) + rte_pause(); /* call the function and store the return value */ fct_arg = lcore_config[lcore_id].arg; - ret = lcore_config[lcore_id].f(fct_arg); + ret = f(fct_arg); lcore_config[lcore_id].ret = ret; lcore_config[lcore_id].f = NULL; lcore_config[lcore_id].arg = NULL; diff --git a/lib/eal/windows/eal_thread.c b/lib/eal/windows/eal_thread.c index df1df5d02c..e08abcf21f 100644 --- a/lib/eal/windows/eal_thread.c +++ b/lib/eal/windows/eal_thread.c @@ -32,8 +32,12 @@ rte_eal_remote_launch(lcore_function_t *f, void *arg, unsigned int worker_id) if (lcore_config[worker_id].state != WAIT) return -EBUSY; - lcore_config[worker_id].f = f; lcore_config[worker_id].arg = arg; + /* Ensure that all the memory operations are completed + * before the worker thread starts running the function. + * Use worker thread function as the guard variable. + */ + __atomic_store_n(&lcore_config[worker_id].f, f, __ATOMIC_RELEASE); /* send message */ n = 0; @@ -84,6 +88,7 @@ eal_thread_loop(void *arg __rte_unused) /* read on our pipe to get commands */ while (1) { + lcore_function_t *f; void *fct_arg; /* wait command */ @@ -103,12 +108,18 @@ eal_thread_loop(void *arg __rte_unused) if (n < 0) rte_panic("cannot write on configuration pipe\n"); - if (lcore_config[lcore_id].f == NULL) - rte_panic("NULL function pointer\n"); + /* Load 'f' with acquire order to ensure that + * the memory operations from the main thread + * are accessed only after update to 'f' is visible. + * Wait till the update to 'f' is visible to the worker. + */ + while ((f = __atomic_load_n(&lcore_config[lcore_id].f, + __ATOMIC_ACQUIRE)) == NULL) + rte_pause(); /* call the function and store the return value */ fct_arg = lcore_config[lcore_id].arg; - ret = lcore_config[lcore_id].f(fct_arg); + ret = f(fct_arg); lcore_config[lcore_id].ret = ret; lcore_config[lcore_id].f = NULL; lcore_config[lcore_id].arg = NULL; From patchwork Thu Sep 9 23:13:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 508375 Delivered-To: patch@linaro.org Received: by 2002:a02:8629:0:0:0:0:0 with SMTP id e38csp767802jai; Thu, 9 Sep 2021 16:13:57 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy4YSGqhLfGsqgdiDwOUMt0UqPLkdQLmUx47Z9nBWfM4GuJaI63TdYi/aSkQFcWaCdvbbfG X-Received: by 2002:adf:e34a:: with SMTP id n10mr6312386wrj.320.1631229237678; Thu, 09 Sep 2021 16:13:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1631229237; cv=none; d=google.com; s=arc-20160816; b=bHEqTyTe+6NPD6dVZgWd/pERs42FNGF8aPvAURHRpJGNAdmqBKQQriX9iwgTxiHVea Jmmb1cwzC/ehpLkaejPSo33vXr4WF1zycxC928evaiMLdatDHIT3FTo2wEPm22ojGFgF eWTheG3YA9R9ETjBA+pMl7sRBYA3DIYqC5LORKg2RUNPVi+HZOf8ftrsWsRt4y3LbO13 qYbJXWv+zjtc5L4g11d1ozyx53NIDO+NZNMTUYW/tzehye2d3yDGPONbZiq2TwKlMi/k jVYmXL2h7bG3C3dwHsAq90vh8P3K9oxYdvFADsRCojHPawPSi0cN2edeBhZyT0hMqvN7 rpug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:cc:to:from; bh=coFVG7AjdPJrnkC/3j6RCK2BXEU61yxSV2n+E4P3rOM=; b=QOHtf9UZFOuqAwkX6S2y1myZMrMYn2LA7szwGG2q6f994o38CE7pxGzGdd3iO2boR7 O/xhL8MfylpT3oZyf2zI9cgpz1MBivfmnMSqXNxbr4Wefl49USw96qDeH2pF7RVAKMCO izqtrgI0dgYfk/6z/QytUR1Lhul0MFz1cfj0O2IEXkc5RN1yvix18NQYT99qvObN5JtH dsaId7SsSO4OtQokV2iA/AqWgIwuwQ2y2wodiV6zFdWNKsc8SV5xaZD8SOjcG81c8ApW UICKvYiL/nAPPggiNdTZRim8s+8DLkRdj7kfpLc/IzYTavqozj40PkwPSZNINnp3ttOs MlHw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from mails.dpdk.org (mails.dpdk.org. [217.70.189.124]) by mx.google.com with ESMTP id v23si2922681wmh.172.2021.09.09.16.13.57; Thu, 09 Sep 2021 16:13:57 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) client-ip=217.70.189.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A80CB41139; Fri, 10 Sep 2021 01:13:46 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 9D769410F5 for ; Fri, 10 Sep 2021 01:13:42 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1A105101E; Thu, 9 Sep 2021 16:13:42 -0700 (PDT) Received: from 2p2660v4-1.austin.arm.com (2p2660v4-1.austin.arm.com [10.118.13.211]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 11F883F766; Thu, 9 Sep 2021 16:13:42 -0700 (PDT) From: Honnappa Nagarahalli To: dev@dpdk.org, honnappa.nagarahalli@arm.com, konstantin.ananyev@intel.com, david.marchand@redhat.com, feifei.wang2@arm.com Cc: ruifeng.wang@arm.com, nd@arm.com, Ola Liljedahl , Feifei Wang Date: Thu, 9 Sep 2021 18:13:09 -0500 Message-Id: <20210909231312.2572006-4-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210909231312.2572006-1-honnappa.nagarahalli@arm.com> References: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> <20210909231312.2572006-1-honnappa.nagarahalli@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 3/6] eal: lcore state FINISHED is not required X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" FINISHED state seems to be used to indicate that the worker's update of the 'state' is not visible to other threads. There seems to be no requirement to have such a state. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Ola Liljedahl Reviewed-by: Feifei Wang --- drivers/event/dpaa2/dpaa2_eventdev_selftest.c | 2 +- drivers/event/octeontx/ssovf_evdev_selftest.c | 2 +- drivers/event/sw/sw_evdev_selftest.c | 4 ++-- examples/l2fwd-keepalive/main.c | 3 +-- lib/eal/common/eal_common_launch.c | 7 ++----- lib/eal/freebsd/eal_thread.c | 4 ++-- lib/eal/include/rte_launch.h | 9 +++++---- lib/eal/include/rte_service.h | 4 +--- lib/eal/linux/eal_thread.c | 10 ++-------- lib/eal/windows/eal_thread.c | 10 ++-------- 10 files changed, 19 insertions(+), 36 deletions(-) -- 2.25.1 diff --git a/drivers/event/dpaa2/dpaa2_eventdev_selftest.c b/drivers/event/dpaa2/dpaa2_eventdev_selftest.c index cd7311a94d..bbbd20951f 100644 --- a/drivers/event/dpaa2/dpaa2_eventdev_selftest.c +++ b/drivers/event/dpaa2/dpaa2_eventdev_selftest.c @@ -468,7 +468,7 @@ wait_workers_to_join(int lcore, const rte_atomic32_t *count) RTE_SET_USED(count); print_cycles = cycles = rte_get_timer_cycles(); - while (rte_eal_get_lcore_state(lcore) != FINISHED) { + while (rte_eal_get_lcore_state(lcore) != WAIT) { uint64_t new_cycles = rte_get_timer_cycles(); if (new_cycles - print_cycles > rte_get_timer_hz()) { diff --git a/drivers/event/octeontx/ssovf_evdev_selftest.c b/drivers/event/octeontx/ssovf_evdev_selftest.c index 528f99dd84..d7b0d22111 100644 --- a/drivers/event/octeontx/ssovf_evdev_selftest.c +++ b/drivers/event/octeontx/ssovf_evdev_selftest.c @@ -579,7 +579,7 @@ wait_workers_to_join(int lcore, const rte_atomic32_t *count) RTE_SET_USED(count); print_cycles = cycles = rte_get_timer_cycles(); - while (rte_eal_get_lcore_state(lcore) != FINISHED) { + while (rte_eal_get_lcore_state(lcore) != WAIT) { uint64_t new_cycles = rte_get_timer_cycles(); if (new_cycles - print_cycles > rte_get_timer_hz()) { diff --git a/drivers/event/sw/sw_evdev_selftest.c b/drivers/event/sw/sw_evdev_selftest.c index d53e903129..9768d3a0c7 100644 --- a/drivers/event/sw/sw_evdev_selftest.c +++ b/drivers/event/sw/sw_evdev_selftest.c @@ -3140,8 +3140,8 @@ worker_loopback(struct test *t, uint8_t disable_implicit_release) rte_eal_remote_launch(worker_loopback_worker_fn, t, w_lcore); print_cycles = cycles = rte_get_timer_cycles(); - while (rte_eal_get_lcore_state(p_lcore) != FINISHED || - rte_eal_get_lcore_state(w_lcore) != FINISHED) { + while (rte_eal_get_lcore_state(p_lcore) != WAIT || + rte_eal_get_lcore_state(w_lcore) != WAIT) { rte_service_run_iter_on_app_lcore(t->service_id, 1); diff --git a/examples/l2fwd-keepalive/main.c b/examples/l2fwd-keepalive/main.c index 4e1a17cfe4..b4f9d7334b 100644 --- a/examples/l2fwd-keepalive/main.c +++ b/examples/l2fwd-keepalive/main.c @@ -507,8 +507,7 @@ dead_core(__rte_unused void *ptr_data, const int id_core) if (terminate_signal_received) return; printf("Dead core %i - restarting..\n", id_core); - if (rte_eal_get_lcore_state(id_core) == FINISHED) { - rte_eal_wait_lcore(id_core); + if (rte_eal_get_lcore_state(id_core) == WAIT) { rte_eal_remote_launch(l2fwd_launch_one_lcore, NULL, id_core); } else { printf("..false positive!\n"); diff --git a/lib/eal/common/eal_common_launch.c b/lib/eal/common/eal_common_launch.c index 34f854ad80..78fd940267 100644 --- a/lib/eal/common/eal_common_launch.c +++ b/lib/eal/common/eal_common_launch.c @@ -26,14 +26,11 @@ rte_eal_wait_lcore(unsigned worker_id) if (lcore_config[worker_id].state == WAIT) return 0; - while (lcore_config[worker_id].state != WAIT && - lcore_config[worker_id].state != FINISHED) + while (lcore_config[worker_id].state != WAIT) rte_pause(); rte_rmb(); - /* we are in finished state, go to wait state */ - lcore_config[worker_id].state = WAIT; return lcore_config[worker_id].ret; } @@ -62,7 +59,7 @@ rte_eal_mp_remote_launch(int (*f)(void *), void *arg, if (call_main == CALL_MAIN) { lcore_config[main_lcore].ret = f(arg); - lcore_config[main_lcore].state = FINISHED; + lcore_config[main_lcore].state = WAIT; } return 0; diff --git a/lib/eal/freebsd/eal_thread.c b/lib/eal/freebsd/eal_thread.c index 17b8f39966..e5e3edf79b 100644 --- a/lib/eal/freebsd/eal_thread.c +++ b/lib/eal/freebsd/eal_thread.c @@ -28,7 +28,7 @@ /* * Send a message to a worker lcore identified by worker_id to call a * function f with argument arg. Once the execution is done, the - * remote lcore switch in FINISHED state. + * remote lcore switches to WAIT state. */ int rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned worker_id) @@ -140,7 +140,7 @@ eal_thread_loop(__rte_unused void *arg) lcore_config[lcore_id].f = NULL; lcore_config[lcore_id].arg = NULL; rte_wmb(); - lcore_config[lcore_id].state = FINISHED; + lcore_config[lcore_id].state = WAIT; } /* never reached */ diff --git a/lib/eal/include/rte_launch.h b/lib/eal/include/rte_launch.h index 22a901ce62..ed0bb4762a 100644 --- a/lib/eal/include/rte_launch.h +++ b/lib/eal/include/rte_launch.h @@ -19,9 +19,10 @@ extern "C" { * State of an lcore. */ enum rte_lcore_state_t { - WAIT, /**< waiting a new command */ - RUNNING, /**< executing command */ - FINISHED, /**< command executed */ + WAIT, + /**< waiting for new command */ + RUNNING, + /**< executing command */ }; /** @@ -41,7 +42,7 @@ typedef int (lcore_function_t)(void *); * * When the remote lcore receives the message, it switches to * the RUNNING state, then calls the function f with argument arg. Once the - * execution is done, the remote lcore switches to a FINISHED state and + * execution is done, the remote lcore switches to WAIT state and * the return value of f is stored in a local variable to be read using * rte_eal_wait_lcore(). * diff --git a/lib/eal/include/rte_service.h b/lib/eal/include/rte_service.h index c7d037d862..646533e871 100644 --- a/lib/eal/include/rte_service.h +++ b/lib/eal/include/rte_service.h @@ -321,9 +321,7 @@ int32_t rte_service_lcore_count(void); * from duty, just unmaps all services / cores, and stops() the service cores. * The runstate of services is not modified. * - * The cores that are stopped with this call, are in FINISHED state and - * the application must take care of bringing them back to a launchable state: - * e.g. call *rte_eal_lcore_wait* on the lcore_id. + * The cores that are stopped with this call, are in WAIT state. * * @retval 0 Success */ diff --git a/lib/eal/linux/eal_thread.c b/lib/eal/linux/eal_thread.c index a0a0091040..c79e09c9ad 100644 --- a/lib/eal/linux/eal_thread.c +++ b/lib/eal/linux/eal_thread.c @@ -28,7 +28,7 @@ /* * Send a message to a worker lcore identified by worker_id to call a * function f with argument arg. Once the execution is done, the - * remote lcore switch in FINISHED state. + * remote lcore switches to WAIT state. */ int rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned int worker_id) @@ -141,13 +141,7 @@ eal_thread_loop(__rte_unused void *arg) lcore_config[lcore_id].arg = NULL; rte_wmb(); - /* when a service core returns, it should go directly to WAIT - * state, because the application will not lcore_wait() for it. - */ - if (lcore_config[lcore_id].core_role == ROLE_SERVICE) - lcore_config[lcore_id].state = WAIT; - else - lcore_config[lcore_id].state = FINISHED; + lcore_config[lcore_id].state = WAIT; } /* never reached */ diff --git a/lib/eal/windows/eal_thread.c b/lib/eal/windows/eal_thread.c index e08abcf21f..977911905e 100644 --- a/lib/eal/windows/eal_thread.c +++ b/lib/eal/windows/eal_thread.c @@ -19,7 +19,7 @@ /* * Send a message to a worker lcore identified by worker_id to call a * function f with argument arg. Once the execution is done, the - * remote lcore switch in FINISHED state. + * remote lcore switches to WAIT state. */ int rte_eal_remote_launch(lcore_function_t *f, void *arg, unsigned int worker_id) @@ -125,13 +125,7 @@ eal_thread_loop(void *arg __rte_unused) lcore_config[lcore_id].arg = NULL; rte_wmb(); - /* when a service core returns, it should go directly to WAIT - * state, because the application will not lcore_wait() for it. - */ - if (lcore_config[lcore_id].core_role == ROLE_SERVICE) - lcore_config[lcore_id].state = WAIT; - else - lcore_config[lcore_id].state = FINISHED; + lcore_config[lcore_id].state = WAIT; } } From patchwork Thu Sep 9 23:13:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 508373 Delivered-To: patch@linaro.org Received: by 2002:a02:8629:0:0:0:0:0 with SMTP id e38csp767655jai; Thu, 9 Sep 2021 16:13:47 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyqOj715Wsjp4hDutz4kVHd3IT1oR/wIGhJgMBtVzEoqKwPWb0NP71CoxwRqOYPhm4BM0+L X-Received: by 2002:a05:600c:230c:: with SMTP id 12mr5421039wmo.41.1631229227370; Thu, 09 Sep 2021 16:13:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1631229227; cv=none; d=google.com; s=arc-20160816; b=LacB6VLBwgeNJ89M+/ZGUFZHLgurO7bAq+NVwIPHpn1wFaxeH7ZMZXJnC9e+Pfz91/ Yh68obExX7VbOmwONM6TC3H0mUYsnYbCn4gOztcYkSHKw/+5y3KeFLAUib4zDTeaDlQL rrJZnKvec02kmQjYq47h20O70iDzaxcXEph57YJs+AEpXquBL9WfuyF6EtRwMb+tGuZh fG1YesSuGjQv/iO7ArhNgRJ6MrmyyldP8qb9Dy/7EkTRgtGDnaX4XAYRGahtzhq7L/3n c6xXKRhxz4gYQhOxQwIvKjHvtfO0dXDXCwEDdFUs4fkXzewRWBptDbFxCV9GK0Geaipe mtWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:cc:to:from; bh=7T9TCviEo8Wff6VUamtpdRdbav4xXCCDjLscCizPj14=; b=Yty56lunwOp5ZzfA2L/y53h6hIc9gvC/yG6AiL2kANZStnwivPgPT6xPdYGIAh3K8l H4nrLR5ss8dTn0i6Ul1MteCH+JD5LDhkCwvp8jRKBVkRZRwwJx2jJSJBrB0RAP9Rv52j s4YTqIo6fy2kVi3CrWCXhIeh7r7ETWzk5HfFiEjCo8w1LvRsTi0O3ufPwD/VwFZX5neZ zifOnRHUar8kGUogKpkcs0Vo/UX4rWkpzsy3RLC9uDz7cFVeEuUMfrDUM/Y2UAAHMTzc jXlUEFXD7+dUzpYj/PoTYbhbKlaFN3gmv09E3G1mgKxp3mXLj0u7BTf//q6r/9OxXjWb AVWQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from mails.dpdk.org (mails.dpdk.org. [217.70.189.124]) by mx.google.com with ESMTP id o13si504374wmc.177.2021.09.09.16.13.47; Thu, 09 Sep 2021 16:13:47 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) client-ip=217.70.189.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 44D4E410FE; Fri, 10 Sep 2021 01:13:44 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id A4997410DA for ; Fri, 10 Sep 2021 01:13:42 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1E7651042; Thu, 9 Sep 2021 16:13:42 -0700 (PDT) Received: from 2p2660v4-1.austin.arm.com (2p2660v4-1.austin.arm.com [10.118.13.211]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 17E2E3F73D; Thu, 9 Sep 2021 16:13:42 -0700 (PDT) From: Honnappa Nagarahalli To: dev@dpdk.org, honnappa.nagarahalli@arm.com, konstantin.ananyev@intel.com, david.marchand@redhat.com, feifei.wang2@arm.com Cc: ruifeng.wang@arm.com, nd@arm.com, Feifei Wang Date: Thu, 9 Sep 2021 18:13:10 -0500 Message-Id: <20210909231312.2572006-5-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210909231312.2572006-1-honnappa.nagarahalli@arm.com> References: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> <20210909231312.2572006-1-honnappa.nagarahalli@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 4/6] eal: update rte_eal_wait_lcore definition X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Since the FINISHED state is removed, the API rte_eal_wait_lcore is updated to always return the status of the last function that ran in the worker core. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Feifei Wang --- lib/eal/common/eal_common_launch.c | 6 ++---- lib/eal/include/rte_launch.h | 12 +++++------- 2 files changed, 7 insertions(+), 11 deletions(-) -- 2.25.1 diff --git a/lib/eal/common/eal_common_launch.c b/lib/eal/common/eal_common_launch.c index 78fd940267..4bc842417a 100644 --- a/lib/eal/common/eal_common_launch.c +++ b/lib/eal/common/eal_common_launch.c @@ -23,10 +23,8 @@ int rte_eal_wait_lcore(unsigned worker_id) { - if (lcore_config[worker_id].state == WAIT) - return 0; - - while (lcore_config[worker_id].state != WAIT) + while (__atomic_load_n(&lcore_config[worker_id].state, + __ATOMIC_ACQUIRE) != WAIT) rte_pause(); rte_rmb(); diff --git a/lib/eal/include/rte_launch.h b/lib/eal/include/rte_launch.h index ed0bb4762a..f2d386e6e2 100644 --- a/lib/eal/include/rte_launch.h +++ b/lib/eal/include/rte_launch.h @@ -119,18 +119,16 @@ enum rte_lcore_state_t rte_eal_get_lcore_state(unsigned int worker_id); * * To be executed on the MAIN lcore only. * - * If the worker lcore identified by the worker_id is in a FINISHED state, - * switch to the WAIT state. If the lcore is in RUNNING state, wait until - * the lcore finishes its job and moves to the FINISHED state. + * If the lcore identified by the worker_id is in RUNNING state, wait until + * the lcore finishes its job and moves to the WAIT state. * * @param worker_id * The identifier of the lcore. * @return - * - 0: If the lcore identified by the worker_id is in a WAIT state. + * - 0: If the remote launch function was never called on the lcore + * identified by the worker_id. * - The value that was returned by the previous remote launch - * function call if the lcore identified by the worker_id was in a - * FINISHED or RUNNING state. In this case, it changes the state - * of the lcore to WAIT. + * function call. */ int rte_eal_wait_lcore(unsigned worker_id); From patchwork Thu Sep 9 23:13:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 508377 Delivered-To: patch@linaro.org Received: by 2002:a02:8629:0:0:0:0:0 with SMTP id e38csp767919jai; Thu, 9 Sep 2021 16:14:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwsJoRxbB6twkuiO3eW4KHG558JU+oOWlj0mOfW8wmjAF9/wG1eUqXdgfDfZxGlCEFMveYL X-Received: by 2002:a5d:6392:: with SMTP id p18mr6489335wru.10.1631229247940; Thu, 09 Sep 2021 16:14:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1631229247; cv=none; d=google.com; s=arc-20160816; b=sJeWfwt6tPKbJgQAtJVj44qmHv3OIMCoIpfm0cR8SIRmykGdyXpndapXm82oNnaEUV a01094HPzmMoNvKN4kNt01Oz75SEMh96r8cbgVWzNZoEuhH06KJqEM6kDLBEaHN2wrpD YF+U3du950iUzO8ZvoCp0LqyjLhqeFVctoLc7G6ksxmcTVW3t5pe/toqiWFxXp/2Qnki 0QGvXEpnDufg+DHp4+4YYpdvGW1pV6sxbkrXh92WPrC7uWg45t5lleTp3TAInRBbgbBU N6jTffMvCyvkGUBMPnQ/Ih3lnH7pILUeMUYDmvfi3C4ycQQuXAFbAyKs2k4Rk8+WV2Wb /c4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:cc:to:from; bh=U2BFRS+OcxDhkbyG2amam6svg3HTnbLIv0EE6F1f+SM=; b=pj+c2yD5BQqfEJdXdIhdhYBmdIfWRjOp0Waz+/Dks9MD7szeaIJsussVnQVnKz3brB xBUOaVsCMuadCqEBs1LbCHqAfF0HWcSr0k2my6yYRd+CU8cJ6J44wcmfDC3qxLUG8BJQ gEmEgi+YFA8EqhOjuYyLfkFbO//jM/t4Dl+SMWDPU/76WqsLw429WeTO0ZqNe1nCqUwz lcN4T+uSM2y7NjmAOm/sFcz/7DSaHM6o8v/Bbee2HjtRI3jx+ZPq6gbhdGXZhelmNwQW PoOunWD/3ZiILhVPQ/EsPP0dGONCdgVnJjlg4V9Y09wUqj7egLZpjXrQWAbCTkkAYyHO rPHw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from mails.dpdk.org (mails.dpdk.org. [217.70.189.124]) by mx.google.com with ESMTP id y6si3081948wrp.91.2021.09.09.16.14.07; Thu, 09 Sep 2021 16:14:07 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) client-ip=217.70.189.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F16734114F; Fri, 10 Sep 2021 01:13:48 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id AF3A541103 for ; Fri, 10 Sep 2021 01:13:42 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 23C211063; Thu, 9 Sep 2021 16:13:42 -0700 (PDT) Received: from 2p2660v4-1.austin.arm.com (2p2660v4-1.austin.arm.com [10.118.13.211]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1C7583F766; Thu, 9 Sep 2021 16:13:42 -0700 (PDT) From: Honnappa Nagarahalli To: dev@dpdk.org, honnappa.nagarahalli@arm.com, konstantin.ananyev@intel.com, david.marchand@redhat.com, feifei.wang2@arm.com Cc: ruifeng.wang@arm.com, nd@arm.com, Feifei Wang Date: Thu, 9 Sep 2021 18:13:11 -0500 Message-Id: <20210909231312.2572006-6-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210909231312.2572006-1-honnappa.nagarahalli@arm.com> References: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> <20210909231312.2572006-1-honnappa.nagarahalli@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 5/6] eal: ensure memory operations are visible to main X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Ensure that the memory operations in worker thread, that happen before it returns the status of the assigned function, are visible to the main thread. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Feifei Wang --- lib/eal/common/eal_common_launch.c | 2 -- lib/eal/freebsd/eal_thread.c | 22 ++++++++++++++++++---- lib/eal/linux/eal_thread.c | 23 ++++++++++++++++++----- lib/eal/windows/eal_thread.c | 21 +++++++++++++++++---- 4 files changed, 53 insertions(+), 15 deletions(-) -- 2.25.1 diff --git a/lib/eal/common/eal_common_launch.c b/lib/eal/common/eal_common_launch.c index 4bc842417a..e95dadffb3 100644 --- a/lib/eal/common/eal_common_launch.c +++ b/lib/eal/common/eal_common_launch.c @@ -27,8 +27,6 @@ rte_eal_wait_lcore(unsigned worker_id) __ATOMIC_ACQUIRE) != WAIT) rte_pause(); - rte_rmb(); - return lcore_config[worker_id].ret; } diff --git a/lib/eal/freebsd/eal_thread.c b/lib/eal/freebsd/eal_thread.c index e5e3edf79b..3b18030d73 100644 --- a/lib/eal/freebsd/eal_thread.c +++ b/lib/eal/freebsd/eal_thread.c @@ -39,7 +39,11 @@ rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned worker_id) int w2m = lcore_config[worker_id].pipe_worker2main[0]; int rc = -EBUSY; - if (lcore_config[worker_id].state != WAIT) + /* Check if the worker is in 'WAIT' state. Use acquire order + * since 'state' variable is used as the guard variable. + */ + if (__atomic_load_n(&lcore_config[worker_id].state, + __ATOMIC_ACQUIRE) != WAIT) goto finish; lcore_config[worker_id].arg = arg; @@ -115,7 +119,11 @@ eal_thread_loop(__rte_unused void *arg) if (n <= 0) rte_panic("cannot read on configuration pipe\n"); - lcore_config[lcore_id].state = RUNNING; + /* Set the state to 'RUNNING'. Use release order + * since 'state' variable is used as the guard variable. + */ + __atomic_store_n(&lcore_config[lcore_id].state, RUNNING, + __ATOMIC_RELEASE); /* send ack */ n = 0; @@ -139,8 +147,14 @@ eal_thread_loop(__rte_unused void *arg) lcore_config[lcore_id].ret = ret; lcore_config[lcore_id].f = NULL; lcore_config[lcore_id].arg = NULL; - rte_wmb(); - lcore_config[lcore_id].state = WAIT; + + /* Store the state with release order to ensure that + * the memory operations from the worker thread + * are completed before the state is updated. + * Use 'state' as the guard variable. + */ + __atomic_store_n(&lcore_config[lcore_id].state, WAIT, + __ATOMIC_RELEASE); } /* never reached */ diff --git a/lib/eal/linux/eal_thread.c b/lib/eal/linux/eal_thread.c index c79e09c9ad..c7f0f9b2f7 100644 --- a/lib/eal/linux/eal_thread.c +++ b/lib/eal/linux/eal_thread.c @@ -39,13 +39,17 @@ rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned int worker_id) int w2m = lcore_config[worker_id].pipe_worker2main[0]; int rc = -EBUSY; - if (lcore_config[worker_id].state != WAIT) + /* Check if the worker is in 'WAIT' state. Use acquire order + * since 'state' variable is used as the guard variable. + */ + if (__atomic_load_n(&lcore_config[worker_id].state, + __ATOMIC_ACQUIRE) != WAIT) goto finish; lcore_config[worker_id].arg = arg; /* Ensure that all the memory operations are completed * before the worker thread starts running the function. - * Use worker thread function as the guard variable. + * Use worker thread function pointer as the guard variable. */ __atomic_store_n(&lcore_config[worker_id].f, f, __ATOMIC_RELEASE); @@ -115,7 +119,11 @@ eal_thread_loop(__rte_unused void *arg) if (n <= 0) rte_panic("cannot read on configuration pipe\n"); - lcore_config[lcore_id].state = RUNNING; + /* Set the state to 'RUNNING'. Use release order + * since 'state' variable is used as the guard variable. + */ + __atomic_store_n(&lcore_config[lcore_id].state, RUNNING, + __ATOMIC_RELEASE); /* send ack */ n = 0; @@ -139,9 +147,14 @@ eal_thread_loop(__rte_unused void *arg) lcore_config[lcore_id].ret = ret; lcore_config[lcore_id].f = NULL; lcore_config[lcore_id].arg = NULL; - rte_wmb(); - lcore_config[lcore_id].state = WAIT; + /* Store the state with release order to ensure that + * the memory operations from the worker thread + * are completed before the state is updated. + * Use 'state' as the guard variable. + */ + __atomic_store_n(&lcore_config[lcore_id].state, WAIT, + __ATOMIC_RELEASE); } /* never reached */ diff --git a/lib/eal/windows/eal_thread.c b/lib/eal/windows/eal_thread.c index 977911905e..54fa93fa62 100644 --- a/lib/eal/windows/eal_thread.c +++ b/lib/eal/windows/eal_thread.c @@ -29,7 +29,11 @@ rte_eal_remote_launch(lcore_function_t *f, void *arg, unsigned int worker_id) int m2w = lcore_config[worker_id].pipe_main2worker[1]; int w2m = lcore_config[worker_id].pipe_worker2main[0]; - if (lcore_config[worker_id].state != WAIT) + /* Check if the worker is in 'WAIT' state. Use acquire order + * since 'state' variable is used as the guard variable. + */ + if (__atomic_load_n(&lcore_config[worker_id].state, + __ATOMIC_ACQUIRE) != WAIT) return -EBUSY; lcore_config[worker_id].arg = arg; @@ -99,7 +103,11 @@ eal_thread_loop(void *arg __rte_unused) if (n <= 0) rte_panic("cannot read on configuration pipe\n"); - lcore_config[lcore_id].state = RUNNING; + /* Set the state to 'RUNNING'. Use release order + * since 'state' variable is used as the guard variable. + */ + __atomic_store_n(&lcore_config[lcore_id].state, RUNNING, + __ATOMIC_RELEASE); /* send ack */ n = 0; @@ -123,9 +131,14 @@ eal_thread_loop(void *arg __rte_unused) lcore_config[lcore_id].ret = ret; lcore_config[lcore_id].f = NULL; lcore_config[lcore_id].arg = NULL; - rte_wmb(); - lcore_config[lcore_id].state = WAIT; + /* Store the state with release order to ensure that + * the memory operations from the worker thread + * are completed before the state is updated. + * Use 'state' as the guard variable. + */ + __atomic_store_n(&lcore_config[lcore_id].state, WAIT, + __ATOMIC_RELEASE); } } From patchwork Thu Sep 9 23:13:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 508376 Delivered-To: patch@linaro.org Received: by 2002:a02:8629:0:0:0:0:0 with SMTP id e38csp767851jai; Thu, 9 Sep 2021 16:14:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzLI0fWOR8cUTpa/Ngwj2oOSzWNuojzVTHDfftp6oyXYNmdNpPGdhe4zmjIu6tb+zf7MzSY X-Received: by 2002:a5d:6781:: with SMTP id v1mr6343482wru.249.1631229242888; Thu, 09 Sep 2021 16:14:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1631229242; cv=none; d=google.com; s=arc-20160816; b=XqbTGosVMapchfq4rgiWr9LgvOnaztAR5I57PgSewkpK/gAw/NM2WCWlXJ2ECNXF18 I3ShKuMMP44AEu2dHyVDlOfMaE+SlZjPQ0OCxX7fl8DoihHs/V+tSerYHxCtFta8XA9Z qBOr1L0PERs1DJGcZUaqUIz2Q+/g8nv5eNmCFhreSSPgC4MTXmCDMLut6pw2EOeOKjXT ZrAnLEFTlTIpIzpCFjt86+9Fb2ZW8PNMQ0B9zIEvSo9QmMv4/LU9AJRUevR9yV9KOFFT EflFh9whd4LUzCZeZw7Wt+w9c+8sD0wQrKzEZvGcP4t6ftlLr1+pv5uNHvwZiFG5NRGw Q37A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:cc:to:from; bh=xZC03eVHiRLWhj8duFzhkKEjizGRYylA/AyAuZKS2G8=; b=a39/wAHEABT9S28y1Ns9tBoOBWx+uC9wEi2ukCod/vgBiHqoXGhZDAPysqBwP8FGQ7 XzBlbmQ7c2Avw7bO9QJVAO/kiK8kzTqRY8egAKt++lJ18f3oV8n7iB9RpD763yy/70DP OtmRZWT2xZqhMcyMwIdsTjalDsnGQJzenFbJY8L9fHSeBtDQIIXEZXSqbx6hTGU0KB2I 8PJiYWFzDeFa5ru2V5LgvBnjHCVv3fKviapq6O69B9YdHKwyFmcUq4w0jBSuI8Mio4mO p8buFRkbPXWuBA/MPl9djBXmnf0786kNwnoAcMR++Q9e9sic9SN+kVUF3TxCRp1P31Hs 76og== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from mails.dpdk.org (mails.dpdk.org. [217.70.189.124]) by mx.google.com with ESMTP id 138si3172299wmb.107.2021.09.09.16.14.02; Thu, 09 Sep 2021 16:14:02 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) client-ip=217.70.189.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BD6FE41142; Fri, 10 Sep 2021 01:13:47 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id B3BF941120 for ; Fri, 10 Sep 2021 01:13:42 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 29CAB106F; Thu, 9 Sep 2021 16:13:42 -0700 (PDT) Received: from 2p2660v4-1.austin.arm.com (2p2660v4-1.austin.arm.com [10.118.13.211]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 21C553F73D; Thu, 9 Sep 2021 16:13:42 -0700 (PDT) From: Honnappa Nagarahalli To: dev@dpdk.org, honnappa.nagarahalli@arm.com, konstantin.ananyev@intel.com, david.marchand@redhat.com, feifei.wang2@arm.com Cc: ruifeng.wang@arm.com, nd@arm.com, Ola Liljedahl , Feifei Wang Date: Thu, 9 Sep 2021 18:13:12 -0500 Message-Id: <20210909231312.2572006-7-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210909231312.2572006-1-honnappa.nagarahalli@arm.com> References: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> <20210909231312.2572006-1-honnappa.nagarahalli@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 6/6] test/ring: use relaxed barriers for ring stress test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" wrk_cmd variable is used to signal the worker thread to start or stop the stress test loop. Relaxed barriers are used to achieve the same. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Ola Liljedahl Reviewed-by: Feifei Wang --- app/test/test_ring_stress_impl.h | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) -- 2.25.1 diff --git a/app/test/test_ring_stress_impl.h b/app/test/test_ring_stress_impl.h index f9ca63b908..ee8293bb04 100644 --- a/app/test/test_ring_stress_impl.h +++ b/app/test/test_ring_stress_impl.h @@ -22,7 +22,7 @@ enum { WRK_CMD_RUN, }; -static volatile uint32_t wrk_cmd __rte_cache_aligned; +static volatile uint32_t wrk_cmd __rte_cache_aligned = WRK_CMD_STOP; /* test run-time in seconds */ static const uint32_t run_time = 60; @@ -197,10 +197,12 @@ test_worker(void *arg, const char *fname, int32_t prcs) fill_ring_elm(&def_elm, UINT32_MAX); fill_ring_elm(&loc_elm, lc); - while (wrk_cmd != WRK_CMD_RUN) { - rte_smp_rmb(); + /* Acquire ordering is not required as the main is not + * really releasing any data through 'wrk_cmd' to + * the worker. + */ + while (__atomic_load_n(&wrk_cmd, __ATOMIC_RELAXED) != WRK_CMD_RUN) rte_pause(); - } cl = rte_rdtsc_precise(); @@ -242,7 +244,7 @@ test_worker(void *arg, const char *fname, int32_t prcs) lcore_stat_update(&la->stats, 1, num, tm0 + tm1, prcs); - } while (wrk_cmd == WRK_CMD_RUN); + } while (__atomic_load_n(&wrk_cmd, __ATOMIC_RELAXED) == WRK_CMD_RUN); cl = rte_rdtsc_precise() - cl; if (prcs == 0) @@ -356,14 +358,12 @@ test_mt1(int (*test)(void *)) } /* signal worker to start test */ - wrk_cmd = WRK_CMD_RUN; - rte_smp_wmb(); + __atomic_store_n(&wrk_cmd, WRK_CMD_RUN, __ATOMIC_RELEASE); usleep(run_time * US_PER_S); /* signal worker to start test */ - wrk_cmd = WRK_CMD_STOP; - rte_smp_wmb(); + __atomic_store_n(&wrk_cmd, WRK_CMD_STOP, __ATOMIC_RELEASE); /* wait for workers and collect stats. */ mc = rte_lcore_id();