From patchwork Wed Feb 24 21:20:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 386876 Delivered-To: patch@linaro.org Received: by 2002:a02:290e:0:0:0:0:0 with SMTP id p14csp697728jap; Wed, 24 Feb 2021 13:20:42 -0800 (PST) X-Google-Smtp-Source: ABdhPJxfywlCL02vgFSqX46eY9M7d7iKMI5XeDTQPPw4EUbUmXCUPB6WZK4B1n2CQHFCl3DBYmfw X-Received: by 2002:aa7:dc17:: with SMTP id b23mr36054412edu.139.1614201642404; Wed, 24 Feb 2021 13:20:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1614201642; cv=none; d=google.com; s=arc-20160816; b=eDS95QOnjoUMJpFk8gIuZpXUBuj71/elJJKdTTPYFstT7w9YlRLnUZkmnjL6qAtQV0 4bck81SO0VKVFOg9dDCJCce4HnP0Km2Eb1Xri+8OHN45X2w9jV9mGP+LeFMzpUQAy9GT xi510qfSWHiGtNwaVJ7AXTU03WPUyRj26hEzg8raNjlpgch+EJ8Uj+VOcOwBRSlJmdtr oxulxS3IvHslsmISLz3oK1zlGPi8xDcSlWHAbFrQolzCTTXO4ULwXrilThUq1ZEZ6zAF pGWUlbWM8FzqWTCQFZMb/yBvna7EuMcr/g6bcvioKrdOPJt6CNE6pbuEDP0jbZzC7BAe MyTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=YgTgusxH26PPbSznZDfZrOqbbacVtfyxG94DGHfcDcs=; b=NtBSHRhkW8r2SpQ68/4Dt8Ewdg0S4c1Em/UyTaYduVQjqYQaOpZnVNQGr0n81Yc1KC +Hgb+40N/MNB3voIMgXy3+P05LAMNhrWe9ks+ygClEUzkAB7Hf7rUpp+J440c60yQIum 1oZhedGsp0S6GVokaEkittcH6N89GPE6nZS7GkIkmbO1yOK+dUPPFDjgqBk5mB/6SYGh 0/Ubb5TadbztADy993bNCicZxzuq6Y1WcMaisT1fvmKp2JjmuzBVBGC0cD5gGQpJFxwL KXcI0FzkmnHpWaRfeRWh9Z095lzUSvH/dPnZKAfSirBTxpwsJLpJn7ALCm/OUJtBFEpu 3cfA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from mails.dpdk.org (mails.dpdk.org. [217.70.189.124]) by mx.google.com with ESMTP id lz19si2201414ejb.88.2021.02.24.13.20.42; Wed, 24 Feb 2021 13:20:42 -0800 (PST) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) client-ip=217.70.189.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0675C16074F; Wed, 24 Feb 2021 22:20:42 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 315D640040 for ; Wed, 24 Feb 2021 22:20:40 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 877C9ED1; Wed, 24 Feb 2021 13:20:39 -0800 (PST) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.12.44]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7BB8B3F73D; Wed, 24 Feb 2021 13:20:39 -0800 (PST) From: Honnappa Nagarahalli To: Bruce Richardson , Dmitry Kozlyuk , Narcisa Ana Maria Vasile , Dmitry Malloy , Pallavi Kadam Cc: dev@dpdk.org, honnappa.nagarahalli@arm.com, ruifeng.wang@arm.com, feifei.wang@arm.com, nd@arm.com Date: Wed, 24 Feb 2021 15:20:14 -0600 Message-Id: <20210224212018.17576-2-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> References: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [RFC 1/5] eal: reset lcore function pointer and argument X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In the rte_eal_remote_launch function, the lcore function pointer is checked for NULL. However, the pointer is never reset to NULL. Reset the lcore function pointer and argument after the worker has completed executing the lcore function. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Ruifeng Wang --- lib/librte_eal/freebsd/eal_thread.c | 2 ++ lib/librte_eal/linux/eal_thread.c | 2 ++ lib/librte_eal/windows/eal_thread.c | 2 ++ 3 files changed, 6 insertions(+) -- 2.17.1 diff --git a/lib/librte_eal/freebsd/eal_thread.c b/lib/librte_eal/freebsd/eal_thread.c index 1dce9b04f..bbc3a8e98 100644 --- a/lib/librte_eal/freebsd/eal_thread.c +++ b/lib/librte_eal/freebsd/eal_thread.c @@ -126,6 +126,8 @@ eal_thread_loop(__rte_unused void *arg) fct_arg = lcore_config[lcore_id].arg; ret = lcore_config[lcore_id].f(fct_arg); lcore_config[lcore_id].ret = ret; + lcore_config[lcore_id].f = NULL; + lcore_config[lcore_id].arg = NULL; rte_wmb(); lcore_config[lcore_id].state = FINISHED; } diff --git a/lib/librte_eal/linux/eal_thread.c b/lib/librte_eal/linux/eal_thread.c index 83c2034b9..8f3c0dafd 100644 --- a/lib/librte_eal/linux/eal_thread.c +++ b/lib/librte_eal/linux/eal_thread.c @@ -126,6 +126,8 @@ eal_thread_loop(__rte_unused void *arg) fct_arg = lcore_config[lcore_id].arg; ret = lcore_config[lcore_id].f(fct_arg); lcore_config[lcore_id].ret = ret; + lcore_config[lcore_id].f = NULL; + lcore_config[lcore_id].arg = NULL; rte_wmb(); /* when a service core returns, it should go directly to WAIT diff --git a/lib/librte_eal/windows/eal_thread.c b/lib/librte_eal/windows/eal_thread.c index 908e726d1..b69672fe0 100644 --- a/lib/librte_eal/windows/eal_thread.c +++ b/lib/librte_eal/windows/eal_thread.c @@ -110,6 +110,8 @@ eal_thread_loop(void *arg __rte_unused) fct_arg = lcore_config[lcore_id].arg; ret = lcore_config[lcore_id].f(fct_arg); lcore_config[lcore_id].ret = ret; + lcore_config[lcore_id].f = NULL; + lcore_config[lcore_id].arg = NULL; rte_wmb(); /* when a service core returns, it should go directly to WAIT From patchwork Wed Feb 24 21:20:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 386877 Delivered-To: patch@linaro.org Received: by 2002:a02:290e:0:0:0:0:0 with SMTP id p14csp697800jap; Wed, 24 Feb 2021 13:20:48 -0800 (PST) X-Google-Smtp-Source: ABdhPJyxdWTGkb7ttSKoX2rJsBxeAoGvP+t9vwxVDJpD0D49l2ZV9oDg6eMMQZp8euEp3ou/YdrS X-Received: by 2002:a05:6402:d1:: with SMTP id i17mr35247880edu.85.1614201648152; Wed, 24 Feb 2021 13:20:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1614201648; cv=none; d=google.com; s=arc-20160816; b=la7KP6HZAmiqM20JOkl4PA3a8HfOy4LuxxTkFoeZ+0cAZsGRh0d0rbdZb1CJvGngkV zrmsKAdluYZGSqizKo2/3ZuJFotleoIu7+4OlNzq8tunLXTKG4HxZ3VZJ2E/wrFDLDYX Y4YXG7bb+GTozH/nTm7QCFJE3d1mdZxl6iNGlGgxl6djwWEGOZSBw7FnQJUTnzB9eZVA b1KNGpI6K9xU+pWcf9P8m7ZdFZrYMX5dT1np6wsrTfWwbCDUXEwCtUFMlVP5cjwGyfJT LGzxQ6wKD/xYqoh+jNhK6BxOCcqk6A+oYZSMcW3yJl/gC7FgGbPW2cr2Jx5U6Hr23qzY yNJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=Jd7kq+5p9VKzlPYiAcPyYtRrSQcijTX49PjfgVerQLw=; b=zJ0Vn1W6GPNepTf9MXv2HYmkcliAUd6swewcjzFZ3wON39qbjXqRlOgeKyVss46X74 BgMMee1B/rvIUxSO3b9cdjGeWhOjIi1WI3q4SPjOieO+ftMhDMt23RaYQX1VI8dV71kl kZTuw+irIF6Xzy3lFtUzWQWgYrAVvS9KWPsKukIzN2Pd0Doi9bGkVeusP2D4CRva5CL4 /Se9JCMpHIWfu8JLLl/U7CZADmswTHXiWrK1E/2YAbLWvQMDCLsSNq4bXQMTasq/VBwn Rp3m3MBEB+VxEyzaUwtP06Er+DCFKslYNTEyesL7C6LWqhinLuyoUTeivbYPJ0/MY9qY A6ew== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from mails.dpdk.org (mails.dpdk.org. [217.70.189.124]) by mx.google.com with ESMTP id t14si2055085eju.664.2021.02.24.13.20.47; Wed, 24 Feb 2021 13:20:48 -0800 (PST) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) client-ip=217.70.189.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4407C16075A; Wed, 24 Feb 2021 22:20:45 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 5EECD160758 for ; Wed, 24 Feb 2021 22:20:43 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D57C7101E; Wed, 24 Feb 2021 13:20:42 -0800 (PST) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.12.44]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C0F3D3F73D; Wed, 24 Feb 2021 13:20:42 -0800 (PST) From: Honnappa Nagarahalli To: Bruce Richardson , Dmitry Kozlyuk , Narcisa Ana Maria Vasile , Dmitry Malloy , Pallavi Kadam Cc: dev@dpdk.org, honnappa.nagarahalli@arm.com, ruifeng.wang@arm.com, feifei.wang@arm.com, nd@arm.com Date: Wed, 24 Feb 2021 15:20:15 -0600 Message-Id: <20210224212018.17576-3-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> References: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [RFC 2/5] eal: ensure memory operations are visible to worker X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Ensure that the memory operations before the call to rte_eal_remote_launch are visible to the worker thread. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Ola Liljedahl --- lib/librte_eal/freebsd/eal_thread.c | 19 +++++++++++++++---- lib/librte_eal/linux/eal_thread.c | 19 +++++++++++++++---- lib/librte_eal/windows/eal_thread.c | 19 +++++++++++++++---- 3 files changed, 45 insertions(+), 12 deletions(-) -- 2.17.1 diff --git a/lib/librte_eal/freebsd/eal_thread.c b/lib/librte_eal/freebsd/eal_thread.c index bbc3a8e98..17b8f3996 100644 --- a/lib/librte_eal/freebsd/eal_thread.c +++ b/lib/librte_eal/freebsd/eal_thread.c @@ -42,8 +42,12 @@ rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned worker_id) if (lcore_config[worker_id].state != WAIT) goto finish; - lcore_config[worker_id].f = f; lcore_config[worker_id].arg = arg; + /* Ensure that all the memory operations are completed + * before the worker thread starts running the function. + * Use worker thread function as the guard variable. + */ + __atomic_store_n(&lcore_config[worker_id].f, f, __ATOMIC_RELEASE); /* send message */ n = 0; @@ -100,6 +104,7 @@ eal_thread_loop(__rte_unused void *arg) /* read on our pipe to get commands */ while (1) { + lcore_function_t *f; void *fct_arg; /* wait command */ @@ -119,12 +124,18 @@ eal_thread_loop(__rte_unused void *arg) if (n < 0) rte_panic("cannot write on configuration pipe\n"); - if (lcore_config[lcore_id].f == NULL) - rte_panic("NULL function pointer\n"); + /* Load 'f' with acquire order to ensure that + * the memory operations from the main thread + * are accessed only after update to 'f' is visible. + * Wait till the update to 'f' is visible to the worker. + */ + while ((f = __atomic_load_n(&lcore_config[lcore_id].f, + __ATOMIC_ACQUIRE)) == NULL) + rte_pause(); /* call the function and store the return value */ fct_arg = lcore_config[lcore_id].arg; - ret = lcore_config[lcore_id].f(fct_arg); + ret = f(fct_arg); lcore_config[lcore_id].ret = ret; lcore_config[lcore_id].f = NULL; lcore_config[lcore_id].arg = NULL; diff --git a/lib/librte_eal/linux/eal_thread.c b/lib/librte_eal/linux/eal_thread.c index 8f3c0dafd..a0a009104 100644 --- a/lib/librte_eal/linux/eal_thread.c +++ b/lib/librte_eal/linux/eal_thread.c @@ -42,8 +42,12 @@ rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned int worker_id) if (lcore_config[worker_id].state != WAIT) goto finish; - lcore_config[worker_id].f = f; lcore_config[worker_id].arg = arg; + /* Ensure that all the memory operations are completed + * before the worker thread starts running the function. + * Use worker thread function as the guard variable. + */ + __atomic_store_n(&lcore_config[worker_id].f, f, __ATOMIC_RELEASE); /* send message */ n = 0; @@ -100,6 +104,7 @@ eal_thread_loop(__rte_unused void *arg) /* read on our pipe to get commands */ while (1) { + lcore_function_t *f; void *fct_arg; /* wait command */ @@ -119,12 +124,18 @@ eal_thread_loop(__rte_unused void *arg) if (n < 0) rte_panic("cannot write on configuration pipe\n"); - if (lcore_config[lcore_id].f == NULL) - rte_panic("NULL function pointer\n"); + /* Load 'f' with acquire order to ensure that + * the memory operations from the main thread + * are accessed only after update to 'f' is visible. + * Wait till the update to 'f' is visible to the worker. + */ + while ((f = __atomic_load_n(&lcore_config[lcore_id].f, + __ATOMIC_ACQUIRE)) == NULL) + rte_pause(); /* call the function and store the return value */ fct_arg = lcore_config[lcore_id].arg; - ret = lcore_config[lcore_id].f(fct_arg); + ret = f(fct_arg); lcore_config[lcore_id].ret = ret; lcore_config[lcore_id].f = NULL; lcore_config[lcore_id].arg = NULL; diff --git a/lib/librte_eal/windows/eal_thread.c b/lib/librte_eal/windows/eal_thread.c index b69672fe0..7a9277c51 100644 --- a/lib/librte_eal/windows/eal_thread.c +++ b/lib/librte_eal/windows/eal_thread.c @@ -32,8 +32,12 @@ rte_eal_remote_launch(lcore_function_t *f, void *arg, unsigned int worker_id) if (lcore_config[worker_id].state != WAIT) return -EBUSY; - lcore_config[worker_id].f = f; lcore_config[worker_id].arg = arg; + /* Ensure that all the memory operations are completed + * before the worker thread starts running the function. + * Use worker thread function as the guard variable. + */ + __atomic_store_n(&lcore_config[worker_id].f, f, __ATOMIC_RELEASE); /* send message */ n = 0; @@ -84,6 +88,7 @@ eal_thread_loop(void *arg __rte_unused) /* read on our pipe to get commands */ while (1) { + lcore_function_t *f; void *fct_arg; /* wait command */ @@ -103,12 +108,18 @@ eal_thread_loop(void *arg __rte_unused) if (n < 0) rte_panic("cannot write on configuration pipe\n"); - if (lcore_config[lcore_id].f == NULL) - rte_panic("NULL function pointer\n"); + /* Load 'f' with acquire order to ensure that + * the memory operations from the main thread + * are accessed only after update to 'f' is visible. + * Wait till the update to 'f' is visible to the worker. + */ + while ((f = __atomic_load_n(&lcore_config[lcore_id].f, + __ATOMIC_ACQUIRE)) == NULL) + rte_pause(); /* call the function and store the return value */ fct_arg = lcore_config[lcore_id].arg; - ret = lcore_config[lcore_id].f(fct_arg); + ret = f(fct_arg); lcore_config[lcore_id].ret = ret; lcore_config[lcore_id].f = NULL; lcore_config[lcore_id].arg = NULL; From patchwork Wed Feb 24 21:20:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 386878 Delivered-To: patch@linaro.org Received: by 2002:a02:290e:0:0:0:0:0 with SMTP id p14csp697843jap; Wed, 24 Feb 2021 13:20:53 -0800 (PST) X-Google-Smtp-Source: ABdhPJzCGmMZ7qT/FkR89vQRKhgrvdyK6w3kCSd2Xz95efTO9U/urzhaTQvLSx4hvgfiS04Ic4KB X-Received: by 2002:a17:906:f288:: with SMTP id gu8mr33241175ejb.512.1614201653263; Wed, 24 Feb 2021 13:20:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1614201653; cv=none; d=google.com; s=arc-20160816; b=SlSkJzIXoh2jORIs3lwqb4PCO4oHt26X3UYBbCPQGx9Mt865dwojatgiwWuMeKYbjt de79MQ5mtHT7Ox8ZCYllY7hGF2t7wiQE3Jb33/B80JMVAe46Tud8aw31OsDBrvoMfsmh yuZbXYGzJ3Ikt9Ret3SJ3UXaqoUJFPtj/9GAWLTbqx6/AIVOIKz4L/5dyMtcEIYnl6Sk 7tHG92CLJW8pWiCaMR8AYnEm1yHqTxymqRsuTEWu+vnGvdCGN5fY1wu2+kftPdCGskqq mXQP4SVmOxV2QoRtadCwCFJ34I+/SgCgDfivkUwTAS4IkOG8XezY/KJ/gppDtWK3BaiR mpUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=uMMUtRAy+XKQkzWXw0lSAOCM63m8ihLGUmzFJrwVSQo=; b=b2Vp1YMhvKvPGD0fG5HpaP9GcUWTjxgINPKnTyUwti+3Loa/L4GzbFsHNnxS6PoL2m 0Kjz5sD/7FPHHyoP+iexJTpNHIJyBPCSRrqKx9dzmUzhNaDFcZJZitumku4vF2+aCTz3 KrLkQtQIc8oCbejqg2NYDPO+8ntpcBmJAcDD+7ak+Q7ax980Q0CijmX1HRfuvY3dxMnG 0Pc+2lKlvQ8N0WGrnNdrvvAArUaY0ONsGR/HmQhDiFZDVodECfb/JlG+28hr/Q6r3HAK V6dFTl/gvYTP+4qgESJoLFCszBlXpsBqOQNBheRDoKxpBe1fpxswSp47ZchsA5inshBE RJPA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from mails.dpdk.org (mails.dpdk.org. [217.70.189.124]) by mx.google.com with ESMTP id a3si2119249ejd.310.2021.02.24.13.20.53; Wed, 24 Feb 2021 13:20:53 -0800 (PST) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) client-ip=217.70.189.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8A05516076E; Wed, 24 Feb 2021 22:20:47 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 8452F16076E for ; Wed, 24 Feb 2021 22:20:46 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 08C381063; Wed, 24 Feb 2021 13:20:46 -0800 (PST) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.12.44]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EFC343F73D; Wed, 24 Feb 2021 13:20:45 -0800 (PST) From: Honnappa Nagarahalli To: Hemant Agrawal , Nipun Gupta , Jerin Jacob , Harry van Haaren , Bruce Richardson , Dmitry Kozlyuk , Narcisa Ana Maria Vasile , Dmitry Malloy , Pallavi Kadam Cc: dev@dpdk.org, honnappa.nagarahalli@arm.com, ruifeng.wang@arm.com, feifei.wang@arm.com, nd@arm.com Date: Wed, 24 Feb 2021 15:20:16 -0600 Message-Id: <20210224212018.17576-4-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> References: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [RFC 3/5] eal: lcore state FINISHED is not required X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" FINISHED state seems to be used to indicate that the worker's update of the 'state' is not visible to other threads. There seems to be no requirement to have such a state. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Ola Liljedahl --- drivers/event/dpaa2/dpaa2_eventdev_selftest.c | 2 +- drivers/event/octeontx/ssovf_evdev_selftest.c | 2 +- drivers/event/sw/sw_evdev_selftest.c | 4 ++-- examples/l2fwd-keepalive/main.c | 2 +- lib/librte_eal/common/eal_common_launch.c | 7 ++----- lib/librte_eal/freebsd/eal_thread.c | 2 +- lib/librte_eal/linux/eal_thread.c | 8 +------- lib/librte_eal/windows/eal_thread.c | 8 +------- 8 files changed, 10 insertions(+), 25 deletions(-) -- 2.17.1 diff --git a/drivers/event/dpaa2/dpaa2_eventdev_selftest.c b/drivers/event/dpaa2/dpaa2_eventdev_selftest.c index cd7311a94..bbbd20951 100644 --- a/drivers/event/dpaa2/dpaa2_eventdev_selftest.c +++ b/drivers/event/dpaa2/dpaa2_eventdev_selftest.c @@ -468,7 +468,7 @@ wait_workers_to_join(int lcore, const rte_atomic32_t *count) RTE_SET_USED(count); print_cycles = cycles = rte_get_timer_cycles(); - while (rte_eal_get_lcore_state(lcore) != FINISHED) { + while (rte_eal_get_lcore_state(lcore) != WAIT) { uint64_t new_cycles = rte_get_timer_cycles(); if (new_cycles - print_cycles > rte_get_timer_hz()) { diff --git a/drivers/event/octeontx/ssovf_evdev_selftest.c b/drivers/event/octeontx/ssovf_evdev_selftest.c index 528f99dd8..d7b0d2211 100644 --- a/drivers/event/octeontx/ssovf_evdev_selftest.c +++ b/drivers/event/octeontx/ssovf_evdev_selftest.c @@ -579,7 +579,7 @@ wait_workers_to_join(int lcore, const rte_atomic32_t *count) RTE_SET_USED(count); print_cycles = cycles = rte_get_timer_cycles(); - while (rte_eal_get_lcore_state(lcore) != FINISHED) { + while (rte_eal_get_lcore_state(lcore) != WAIT) { uint64_t new_cycles = rte_get_timer_cycles(); if (new_cycles - print_cycles > rte_get_timer_hz()) { diff --git a/drivers/event/sw/sw_evdev_selftest.c b/drivers/event/sw/sw_evdev_selftest.c index e4bfb3a0f..7847a8645 100644 --- a/drivers/event/sw/sw_evdev_selftest.c +++ b/drivers/event/sw/sw_evdev_selftest.c @@ -3138,8 +3138,8 @@ worker_loopback(struct test *t, uint8_t disable_implicit_release) rte_eal_remote_launch(worker_loopback_worker_fn, t, w_lcore); print_cycles = cycles = rte_get_timer_cycles(); - while (rte_eal_get_lcore_state(p_lcore) != FINISHED || - rte_eal_get_lcore_state(w_lcore) != FINISHED) { + while (rte_eal_get_lcore_state(p_lcore) != WAIT || + rte_eal_get_lcore_state(w_lcore) != WAIT) { rte_service_run_iter_on_app_lcore(t->service_id, 1); diff --git a/examples/l2fwd-keepalive/main.c b/examples/l2fwd-keepalive/main.c index e4c2b2793..dd777c46a 100644 --- a/examples/l2fwd-keepalive/main.c +++ b/examples/l2fwd-keepalive/main.c @@ -506,7 +506,7 @@ dead_core(__rte_unused void *ptr_data, const int id_core) if (terminate_signal_received) return; printf("Dead core %i - restarting..\n", id_core); - if (rte_eal_get_lcore_state(id_core) == FINISHED) { + if (rte_eal_get_lcore_state(id_core) == WAIT) { rte_eal_wait_lcore(id_core); rte_eal_remote_launch(l2fwd_launch_one_lcore, NULL, id_core); } else { diff --git a/lib/librte_eal/common/eal_common_launch.c b/lib/librte_eal/common/eal_common_launch.c index 34f854ad8..78fd94026 100644 --- a/lib/librte_eal/common/eal_common_launch.c +++ b/lib/librte_eal/common/eal_common_launch.c @@ -26,14 +26,11 @@ rte_eal_wait_lcore(unsigned worker_id) if (lcore_config[worker_id].state == WAIT) return 0; - while (lcore_config[worker_id].state != WAIT && - lcore_config[worker_id].state != FINISHED) + while (lcore_config[worker_id].state != WAIT) rte_pause(); rte_rmb(); - /* we are in finished state, go to wait state */ - lcore_config[worker_id].state = WAIT; return lcore_config[worker_id].ret; } @@ -62,7 +59,7 @@ rte_eal_mp_remote_launch(int (*f)(void *), void *arg, if (call_main == CALL_MAIN) { lcore_config[main_lcore].ret = f(arg); - lcore_config[main_lcore].state = FINISHED; + lcore_config[main_lcore].state = WAIT; } return 0; diff --git a/lib/librte_eal/freebsd/eal_thread.c b/lib/librte_eal/freebsd/eal_thread.c index 17b8f3996..6d6f1e2fd 100644 --- a/lib/librte_eal/freebsd/eal_thread.c +++ b/lib/librte_eal/freebsd/eal_thread.c @@ -140,7 +140,7 @@ eal_thread_loop(__rte_unused void *arg) lcore_config[lcore_id].f = NULL; lcore_config[lcore_id].arg = NULL; rte_wmb(); - lcore_config[lcore_id].state = FINISHED; + lcore_config[lcore_id].state = WAIT; } /* never reached */ diff --git a/lib/librte_eal/linux/eal_thread.c b/lib/librte_eal/linux/eal_thread.c index a0a009104..7b9463df3 100644 --- a/lib/librte_eal/linux/eal_thread.c +++ b/lib/librte_eal/linux/eal_thread.c @@ -141,13 +141,7 @@ eal_thread_loop(__rte_unused void *arg) lcore_config[lcore_id].arg = NULL; rte_wmb(); - /* when a service core returns, it should go directly to WAIT - * state, because the application will not lcore_wait() for it. - */ - if (lcore_config[lcore_id].core_role == ROLE_SERVICE) - lcore_config[lcore_id].state = WAIT; - else - lcore_config[lcore_id].state = FINISHED; + lcore_config[lcore_id].state = WAIT; } /* never reached */ diff --git a/lib/librte_eal/windows/eal_thread.c b/lib/librte_eal/windows/eal_thread.c index 7a9277c51..35d059a30 100644 --- a/lib/librte_eal/windows/eal_thread.c +++ b/lib/librte_eal/windows/eal_thread.c @@ -125,13 +125,7 @@ eal_thread_loop(void *arg __rte_unused) lcore_config[lcore_id].arg = NULL; rte_wmb(); - /* when a service core returns, it should go directly to WAIT - * state, because the application will not lcore_wait() for it. - */ - if (lcore_config[lcore_id].core_role == ROLE_SERVICE) - lcore_config[lcore_id].state = WAIT; - else - lcore_config[lcore_id].state = FINISHED; + lcore_config[lcore_id].state = WAIT; } } From patchwork Wed Feb 24 21:20:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 386879 Delivered-To: patch@linaro.org Received: by 2002:a02:290e:0:0:0:0:0 with SMTP id p14csp697917jap; Wed, 24 Feb 2021 13:21:00 -0800 (PST) X-Google-Smtp-Source: ABdhPJw4sappw+uGbOdryAW7bQb51VDS+0Ly+tCRlFd8iGZ/BB6tKMXNLSX8ehGUBe3RmWIFtfqs X-Received: by 2002:a17:906:8a5c:: with SMTP id gx28mr25516237ejc.51.1614201660161; Wed, 24 Feb 2021 13:21:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1614201660; cv=none; d=google.com; s=arc-20160816; b=cwtNs4Q29Ze0dcogLu9EXSS6DFE7KoT64V5xeAxVxBQI6yt3BP8pVJm8c54QrwQJIi ZJJuSC3uv5LV9NFld2NrsZoRElk/KtcW+jjy5uJY9dZtncnBOrrON+PyyElwgOVEG5YA Ti4m7XXyc3CmXhiDlF8XG8M0OMcRAmIdInmILbeyCauEhIIjrmlFmiE/rtjBTUMMlMe+ rHl1G7up4O/zXI3BDdTsisfcvC21Chd6jm2dVhogXpIlfPGYUdSFDgMPDUNzCV28RIDb 4YcxMR5yWszu7GWzy5fBwwNR0WBNuVIkdzXq/X4vLn2EsA2drYM9Z9c8FxfuvxIb3Ro4 Hasg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=01kzJnM7ENLR4aCpRWXTHLFM49N1F1jQtJaGPHAvmLY=; b=NaOT1RJk5kL1vZCQGAnLTg16MmIs2pM5XR9uC78YMZ8VzC6ZFDXY2ZQNrmYkp45dMv J0/ukkAEGbFqE/sP5Nn6MPqZa+9pgyIals4MQ4/p+oYPlE/QWKaLv3WhXrSpXgyylorl TqKjmwwr9t1Zq6p/KpOFbg7wzBqIBoyOjBq6up1s+cRE3OAhqyk/kqVZjpwAFBqd24Vg VndZTQw9pdhi0cNAH2pvRJXioaaOJ/FgNomQw72bgHAMC7YE5nmcGAqhcEB36NhvpvzT SQDS0DSKLvsPXDVP+Wu9SX4z/dmo3DQt/9VbQpd4U+ui+I0M/HWnJw2f6b45MEljioNy Z4Wg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from mails.dpdk.org (mails.dpdk.org. [217.70.189.124]) by mx.google.com with ESMTP id m6si2075018edp.192.2021.02.24.13.20.59; Wed, 24 Feb 2021 13:21:00 -0800 (PST) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) client-ip=217.70.189.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C5300160778; Wed, 24 Feb 2021 22:20:50 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 1665D160774 for ; Wed, 24 Feb 2021 22:20:49 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 88CB8ED1; Wed, 24 Feb 2021 13:20:48 -0800 (PST) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.12.44]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7BFE33F73D; Wed, 24 Feb 2021 13:20:48 -0800 (PST) From: Honnappa Nagarahalli To: Bruce Richardson , Dmitry Kozlyuk , Narcisa Ana Maria Vasile , Dmitry Malloy , Pallavi Kadam Cc: dev@dpdk.org, honnappa.nagarahalli@arm.com, ruifeng.wang@arm.com, feifei.wang@arm.com, nd@arm.com Date: Wed, 24 Feb 2021 15:20:17 -0600 Message-Id: <20210224212018.17576-5-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> References: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [RFC 4/5] eal: ensure memory operations are visible to main X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Ensure that the memory operations in worker thread, that happen before it returns the status of the assigned function, are visible to the main thread. Signed-off-by: Honnappa Nagarahalli --- lib/librte_eal/common/eal_common_launch.c | 8 ++++---- lib/librte_eal/freebsd/eal_thread.c | 10 ++++++++-- lib/librte_eal/linux/eal_thread.c | 17 ++++++++++++----- lib/librte_eal/windows/eal_thread.c | 9 +++++++-- 4 files changed, 31 insertions(+), 13 deletions(-) -- 2.17.1 diff --git a/lib/librte_eal/common/eal_common_launch.c b/lib/librte_eal/common/eal_common_launch.c index 78fd94026..9cc71801a 100644 --- a/lib/librte_eal/common/eal_common_launch.c +++ b/lib/librte_eal/common/eal_common_launch.c @@ -23,14 +23,14 @@ int rte_eal_wait_lcore(unsigned worker_id) { - if (lcore_config[worker_id].state == WAIT) + if (__atomic_load_n(&lcore_config[worker_id].state, + __ATOMIC_ACQUIRE) == WAIT) return 0; - while (lcore_config[worker_id].state != WAIT) + while (__atomic_load_n(&lcore_config[worker_id].state, + __ATOMIC_ACQUIRE) != WAIT) rte_pause(); - rte_rmb(); - return lcore_config[worker_id].ret; } diff --git a/lib/librte_eal/freebsd/eal_thread.c b/lib/librte_eal/freebsd/eal_thread.c index 6d6f1e2fd..58c8502de 100644 --- a/lib/librte_eal/freebsd/eal_thread.c +++ b/lib/librte_eal/freebsd/eal_thread.c @@ -139,8 +139,14 @@ eal_thread_loop(__rte_unused void *arg) lcore_config[lcore_id].ret = ret; lcore_config[lcore_id].f = NULL; lcore_config[lcore_id].arg = NULL; - rte_wmb(); - lcore_config[lcore_id].state = WAIT; + + /* Store the state with release order to ensure that + * the memory operations from the worker thread + * are completed before the state is updated. + * Use 'state' as the guard variable. + */ + __atomic_store_n(&lcore_config[lcore_id].state, WAIT, + __ATOMIC_RELEASE); } /* never reached */ diff --git a/lib/librte_eal/linux/eal_thread.c b/lib/librte_eal/linux/eal_thread.c index 7b9463df3..eab6fa652 100644 --- a/lib/librte_eal/linux/eal_thread.c +++ b/lib/librte_eal/linux/eal_thread.c @@ -39,13 +39,14 @@ rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned int worker_id) int w2m = lcore_config[worker_id].pipe_worker2main[0]; int rc = -EBUSY; - if (lcore_config[worker_id].state != WAIT) + if (__atomic_load_n(&lcore_config[worker_id].state, + __ATOMIC_ACQUIRE) != WAIT) goto finish; lcore_config[worker_id].arg = arg; /* Ensure that all the memory operations are completed * before the worker thread starts running the function. - * Use worker thread function as the guard variable. + * Use worker thread function pointer as the guard variable. */ __atomic_store_n(&lcore_config[worker_id].f, f, __ATOMIC_RELEASE); @@ -115,7 +116,8 @@ eal_thread_loop(__rte_unused void *arg) if (n <= 0) rte_panic("cannot read on configuration pipe\n"); - lcore_config[lcore_id].state = RUNNING; + __atomic_store_n(&lcore_config[lcore_id].state, RUNNING, + __ATOMIC_RELEASE); /* send ack */ n = 0; @@ -139,9 +141,14 @@ eal_thread_loop(__rte_unused void *arg) lcore_config[lcore_id].ret = ret; lcore_config[lcore_id].f = NULL; lcore_config[lcore_id].arg = NULL; - rte_wmb(); - lcore_config[lcore_id].state = WAIT; + /* Store the state with release order to ensure that + * the memory operations from the worker thread + * are completed before the state is updated. + * Use 'state' as the guard variable. + */ + __atomic_store_n(&lcore_config[lcore_id].state, WAIT, + __ATOMIC_RELEASE); } /* never reached */ diff --git a/lib/librte_eal/windows/eal_thread.c b/lib/librte_eal/windows/eal_thread.c index 35d059a30..fb1ec4b4f 100644 --- a/lib/librte_eal/windows/eal_thread.c +++ b/lib/librte_eal/windows/eal_thread.c @@ -123,9 +123,14 @@ eal_thread_loop(void *arg __rte_unused) lcore_config[lcore_id].ret = ret; lcore_config[lcore_id].f = NULL; lcore_config[lcore_id].arg = NULL; - rte_wmb(); - lcore_config[lcore_id].state = WAIT; + /* Store the state with release order to ensure that + * the memory operations from the worker thread + * are completed before the state is updated. + * Use 'state' as the guard variable. + */ + __atomic_store_n(&lcore_config[lcore_id].state, WAIT, + __ATOMIC_RELEASE); } } From patchwork Wed Feb 24 21:20:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 386880 Delivered-To: patch@linaro.org Received: by 2002:a02:290e:0:0:0:0:0 with SMTP id p14csp697981jap; Wed, 24 Feb 2021 13:21:06 -0800 (PST) X-Google-Smtp-Source: ABdhPJyE4aTxhumFcl28AEgQGvpDWTFFmEITadxUGnoh/Xi4L/iyeVeWTroeV6FaFWVglHssAZTR X-Received: by 2002:a17:906:9259:: with SMTP id c25mr33198729ejx.37.1614201666810; Wed, 24 Feb 2021 13:21:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1614201666; cv=none; d=google.com; s=arc-20160816; b=e+J1iNOzlC0XNupFgkJMwkg7ncs76n1/Q8wJaTS+YdIkka9Cn9x9WuELEMXN+CXAiB t0R3LvT4ijLElJm3++Ezysq78nUPNvnwCsPBGPSLlKWRfn9r7qwYbcm3xcConPveAdwO bq+/xWRSwkzkCb9E1PM/BIyTpbjEusu4/tAuowqvbMTgk1RUI/D0R04fKtO/p0JtupDC U7UaQ1WjB15Q+73BxM205QLq0+QUQa5TGxsfYP5kigLM/rdQEfU5/UMJV57Q3ZcTqgD8 48Do485Gdji28dvpQJd19gmVbvyyzEh26ihtmsplOLI0m1DL0iwwspu00RXVFqgFz7tj 9Wpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=/h8PysL3obZArLafWYOhnt8yRwXj7SizuxB9R0sPXfo=; b=QFxbesemYFK17OZNHLOjADZzh8sVRjjYU88pw6BzMErMkEQUMtH0NOam8fQredtz/r yX8sVsIZpDTlRNPSolpz0o4DILdomM4w5obGe/JUzrg8OLug+AcK2uts2Z4LGHG+AvxG taCE13r62GP8Rz8bYhqbjon25HTgO48Rm2OfwAZzlI+3UIGK4/xUzdTv+E+fwL+cjHcC xQmYKsHIy53+huDB6HcfyqbwuOQgRX9cGpIebMXiYoxUh8GecJlKlBhM8BimerbL/4Xx pG15gYK9FRTo+CWmhlbb4F3R38+VwGk7enNbMExa32+sYH/6wkE0uuRuTQOz0LZsl+g2 j+aw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from mails.dpdk.org (mails.dpdk.org. [217.70.189.124]) by mx.google.com with ESMTP id v1si2211512ejg.518.2021.02.24.13.21.06; Wed, 24 Feb 2021 13:21:06 -0800 (PST) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) client-ip=217.70.189.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 02E5516077E; Wed, 24 Feb 2021 22:20:53 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 8719216077C for ; Wed, 24 Feb 2021 22:20:51 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EC595101E; Wed, 24 Feb 2021 13:20:50 -0800 (PST) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.12.44]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E0E843F73D; Wed, 24 Feb 2021 13:20:50 -0800 (PST) From: Honnappa Nagarahalli To: Honnappa Nagarahalli , Konstantin Ananyev Cc: dev@dpdk.org, ruifeng.wang@arm.com, feifei.wang@arm.com, nd@arm.com Date: Wed, 24 Feb 2021 15:20:18 -0600 Message-Id: <20210224212018.17576-6-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> References: <20210224212018.17576-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [RFC 5/5] test/ring: use relaxed barriers for ring stress test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" wrk_cmd variable is used to signal the worker thread to start or stop the stress test loop. Relaxed barriers are used to achieve the same. Signed-off-by: Honnappa Nagarahalli Reviewed-by: Ola Liljedahl --- app/test/test_ring_stress_impl.h | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) -- 2.17.1 diff --git a/app/test/test_ring_stress_impl.h b/app/test/test_ring_stress_impl.h index f9ca63b90..ee8293bb0 100644 --- a/app/test/test_ring_stress_impl.h +++ b/app/test/test_ring_stress_impl.h @@ -22,7 +22,7 @@ enum { WRK_CMD_RUN, }; -static volatile uint32_t wrk_cmd __rte_cache_aligned; +static volatile uint32_t wrk_cmd __rte_cache_aligned = WRK_CMD_STOP; /* test run-time in seconds */ static const uint32_t run_time = 60; @@ -197,10 +197,12 @@ test_worker(void *arg, const char *fname, int32_t prcs) fill_ring_elm(&def_elm, UINT32_MAX); fill_ring_elm(&loc_elm, lc); - while (wrk_cmd != WRK_CMD_RUN) { - rte_smp_rmb(); + /* Acquire ordering is not required as the main is not + * really releasing any data through 'wrk_cmd' to + * the worker. + */ + while (__atomic_load_n(&wrk_cmd, __ATOMIC_RELAXED) != WRK_CMD_RUN) rte_pause(); - } cl = rte_rdtsc_precise(); @@ -242,7 +244,7 @@ test_worker(void *arg, const char *fname, int32_t prcs) lcore_stat_update(&la->stats, 1, num, tm0 + tm1, prcs); - } while (wrk_cmd == WRK_CMD_RUN); + } while (__atomic_load_n(&wrk_cmd, __ATOMIC_RELAXED) == WRK_CMD_RUN); cl = rte_rdtsc_precise() - cl; if (prcs == 0) @@ -356,14 +358,12 @@ test_mt1(int (*test)(void *)) } /* signal worker to start test */ - wrk_cmd = WRK_CMD_RUN; - rte_smp_wmb(); + __atomic_store_n(&wrk_cmd, WRK_CMD_RUN, __ATOMIC_RELEASE); usleep(run_time * US_PER_S); /* signal worker to start test */ - wrk_cmd = WRK_CMD_STOP; - rte_smp_wmb(); + __atomic_store_n(&wrk_cmd, WRK_CMD_STOP, __ATOMIC_RELEASE); /* wait for workers and collect stats. */ mc = rte_lcore_id();