From patchwork Tue Mar 10 17:49:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Phil Yang X-Patchwork-Id: 184358 Delivered-To: patch@linaro.org Received: by 2002:a92:1f12:0:0:0:0:0 with SMTP id i18csp985253ile; Tue, 10 Mar 2020 10:51:11 -0700 (PDT) X-Google-Smtp-Source: ADFU+vv4+Id3zp/HPeBGz2UmFBGnCCZL/W+9StTC6vQRIA5sNDHyUGyaLC8i47pbMMzPuTPEWy5Z X-Received: by 2002:a05:6402:78c:: with SMTP id d12mr22236231edy.203.1583862671768; Tue, 10 Mar 2020 10:51:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1583862671; cv=none; d=google.com; s=arc-20160816; b=J0vX3T0egZOTZVN1k9WrEmAdJ0DiGfeLPhjttp03TF4MI67OzWRSVu8TLJRQy16dHy Xrh+KYHdI397XRzQhYtuvzvEHpDlmogGYljiRfuA8cov1F5y4cFGxx1CU1u8lQerpek1 m6JZO4EZK7tMhcow5tMAFlvWJV2NJMaTA+BCbpJkXnqcdlynJf/ssNImj2N3Ps+uGqrz u/czYQRRNmDYdsTDsobbLIZWa17mO1OO9xjYHQrQXc1LQwRLlfFn75TKlGHEnaWfatsT 3jdLPdtqlGdaRRr/ae3NTDAkpKpLq36iipQD3Co2rNXeYOOVW/n7UMhHyuzUPqfK2yBD 1NfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=ge3jErxjbLqAoZevpnqfMQ5lOkHWiJOjTfHnpjsVb2Y=; b=dU1YMvCmJ2NLL4t0JfDbghBRZqLMxGXxV7ZRHXhBwKCAYl+7WvPpmLM84FianxfrRv H+KIW5AfOnAGKm9M+dp9yY9BYyu6yKyrWWhMB1Kh45xERVD6dYO2Iu9aIRyyMHob8uCU UyYOCUHxFT7N7YoJOnfIzHMMLMWhhJf8+/MpddxzAUMIhTivblbuRfYRFGJiRPNgfJng PTz/BPbV0Ct8a9NiHQwxjTz6awmRtQX55twtW1WqVCXq/QyAApmGCIls7MQA+91bS9PT y2E/vOuNHg3tqcKP8BRiZAcPZ7d+82939HsVWIvi6IreHyo8tII5BlDe4W0iXwuvXgO0 NT/A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id j14si3839162edl.42.2020.03.10.10.51.11; Tue, 10 Mar 2020 10:51:11 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 78AC61C0BC; Tue, 10 Mar 2020 18:50:24 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id CFA3F1C0B7; Tue, 10 Mar 2020 18:50:22 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 624DA1FB; Tue, 10 Mar 2020 10:50:22 -0700 (PDT) Received: from phil-VirtualBox.arm.com (A010647.Arm.com [10.170.243.28]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 964293F534; Tue, 10 Mar 2020 10:50:18 -0700 (PDT) From: Phil Yang To: thomas@monjalon.net, harry.van.haaren@intel.com, konstantin.ananyev@intel.com, stephen@networkplumber.org, maxime.coquelin@redhat.com, dev@dpdk.org Cc: david.marchand@redhat.com, jerinj@marvell.com, hemant.agrawal@nxp.com, Honnappa.Nagarahalli@arm.com, gavin.hu@arm.com, ruifeng.wang@arm.com, joyce.kong@arm.com, nd@arm.com, Honnappa Nagarahalli , stable@dpdk.org Date: Wed, 11 Mar 2020 01:49:08 +0800 Message-Id: <1583862551-2049-8-git-send-email-phil.yang@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1583862551-2049-1-git-send-email-phil.yang@arm.com> References: <1583862551-2049-1-git-send-email-phil.yang@arm.com> Subject: [dpdk-dev] [PATCH 07/10] service: avoid race condition for MT unsafe service X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Honnappa Nagarahalli There has possible that a MT unsafe service might get configured to run on another core while the service is running currently. This might result in the MT unsafe service running on multiple cores simultaneously. Use 'execute_lock' always when the service is MT unsafe. Fixes: e9139a32f6e8 ("service: add function to run on app lcore") Cc: stable@dpdk.org Signed-off-by: Honnappa Nagarahalli Reviewed-by: Phil Yang --- lib/librte_eal/common/rte_service.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) -- 2.7.4 diff --git a/lib/librte_eal/common/rte_service.c b/lib/librte_eal/common/rte_service.c index 6990dc2..b37fc56 100644 --- a/lib/librte_eal/common/rte_service.c +++ b/lib/librte_eal/common/rte_service.c @@ -50,6 +50,10 @@ struct rte_service_spec_impl { uint8_t internal_flags; /* per service statistics */ + /* Indicates how many cores the service is mapped to run on. + * It does not indicate the number of cores the service is running + * on currently. + */ rte_atomic32_t num_mapped_cores; uint64_t calls; uint64_t cycles_spent; @@ -367,12 +371,7 @@ service_run(uint32_t i, struct core_state *cs, uint64_t service_mask, cs->service_active_on_lcore[i] = 1; - /* check do we need cmpset, if MT safe or <= 1 core - * mapped, atomic ops are not required. - */ - const int use_atomics = (service_mt_safe(s) == 0) && - (rte_atomic32_read(&s->num_mapped_cores) > 1); - if (use_atomics) { + if (service_mt_safe(s) == 0) { if (!rte_atomic32_cmpset((uint32_t *)&s->execute_lock, 0, 1)) return -EBUSY; From patchwork Tue Mar 10 17:49:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Phil Yang X-Patchwork-Id: 184359 Delivered-To: patch@linaro.org Received: by 2002:a92:1f12:0:0:0:0:0 with SMTP id i18csp985469ile; Tue, 10 Mar 2020 10:51:25 -0700 (PDT) X-Google-Smtp-Source: ADFU+vtULBVTTNAXFJozwrjAioQb0XZXpvgvJjtnOmpscZ5IbLG8cRR4ppJYuNlYc3wrH3vjpPSD X-Received: by 2002:a17:906:5ad4:: with SMTP id x20mr19546864ejs.293.1583862685276; Tue, 10 Mar 2020 10:51:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1583862685; cv=none; d=google.com; s=arc-20160816; b=odgRJTlW57pFFN9YiD5z9Ybck5VXRLncbiScCpUxubj350w/+w/sB1b1mDzi6Hcyvm 259uyxx5L2Fv4o6vRSvfN+CeXA2vA0vql8ftZMq6Y19tHHNAjhdlGdVt1npgDaJBzgCr 8S+J1iyDC1l2DxMN2Qf1viyAK7UKKYHNcXscULWln1uS0lksV1M3s371lCwfIJRKoVCd yoCLaZnKkCK8gXeFXhIU1qiHYS2VDbAwSvqr/ejXfgqJu6KmH9T/QlKU5Og7SPJ6DQUs El6+zY4pPaev21JPrSnLD1H2WFJCivRZ2Bv1UmY5dRJHzrgHh5HVS6wVX/GE1PDCjvoR 3N5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=RZHFv/Qv7vl2sNj8+4vVKdprI4GzcG6ZQgRHYGc2vhM=; b=nK9sVR4t2DNTe0J6l+Ef6qONIuUngk3fuqJssseedTo2kb8xWeKrChWY1Yw+nLVsjt eis2nWtNFhXg3D8C7FSyo8dcyi0VxNtsd1przr/i/7QcqsRnY1hPC0F6aKq8chzHqy4O rLhMqEL76O/wx8xeRu35dPeC5AEgzw0uVo0cMFpFTK3oIIcXYqsf2kH2ilxY674kyWVh 1pxDsFEo1LjLY6ypQHApZRfWIlxB7wjo4F0EGPGqGmklwCHwHhRbdfneQucvsG44tJEw Nnh4hzXFyzy2OAJqj4Bc/y//aZs82O3MV+0yO5OVVwCpwFPGYHad+rsbXagp/JZzST0J G9yQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id f10si9138638eja.201.2020.03.10.10.51.25; Tue, 10 Mar 2020 10:51:25 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 16AA41C0D0; Tue, 10 Mar 2020 18:50:29 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 8C5271C0D0; Tue, 10 Mar 2020 18:50:27 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1F2921FB; Tue, 10 Mar 2020 10:50:27 -0700 (PDT) Received: from phil-VirtualBox.arm.com (A010647.Arm.com [10.170.243.28]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D1D383F534; Tue, 10 Mar 2020 10:50:22 -0700 (PDT) From: Phil Yang To: thomas@monjalon.net, harry.van.haaren@intel.com, konstantin.ananyev@intel.com, stephen@networkplumber.org, maxime.coquelin@redhat.com, dev@dpdk.org Cc: david.marchand@redhat.com, jerinj@marvell.com, hemant.agrawal@nxp.com, Honnappa.Nagarahalli@arm.com, gavin.hu@arm.com, ruifeng.wang@arm.com, joyce.kong@arm.com, nd@arm.com, Honnappa Nagarahalli , stable@dpdk.org Date: Wed, 11 Mar 2020 01:49:09 +0800 Message-Id: <1583862551-2049-9-git-send-email-phil.yang@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1583862551-2049-1-git-send-email-phil.yang@arm.com> References: <1583862551-2049-1-git-send-email-phil.yang@arm.com> Subject: [dpdk-dev] [PATCH 08/10] service: identify service running on another core correctly X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Honnappa Nagarahalli The logic to identify if the MT unsafe service is running on another core can return -EBUSY spuriously. In such cases, running the service becomes more costlier than using atomic operations. Assume that the application passes the right parameters and reduce the number of instructions for all cases. Cc: stable@dpdk.org Signed-off-by: Honnappa Nagarahalli Reviewed-by: Phil Yang --- lib/librte_eal/common/rte_service.c | 26 ++++++++------------------ 1 file changed, 8 insertions(+), 18 deletions(-) -- 2.7.4 diff --git a/lib/librte_eal/common/rte_service.c b/lib/librte_eal/common/rte_service.c index b37fc56..0186024 100644 --- a/lib/librte_eal/common/rte_service.c +++ b/lib/librte_eal/common/rte_service.c @@ -357,7 +357,7 @@ service_runner_do_callback(struct rte_service_spec_impl *s, /* Expects the service 's' is valid. */ static int32_t service_run(uint32_t i, struct core_state *cs, uint64_t service_mask, - struct rte_service_spec_impl *s) + struct rte_service_spec_impl *s, uint32_t serialize_mt_unsafe) { if (!s) return -EINVAL; @@ -371,7 +371,7 @@ service_run(uint32_t i, struct core_state *cs, uint64_t service_mask, cs->service_active_on_lcore[i] = 1; - if (service_mt_safe(s) == 0) { + if ((service_mt_safe(s) == 0) && (serialize_mt_unsafe == 1)) { if (!rte_atomic32_cmpset((uint32_t *)&s->execute_lock, 0, 1)) return -EBUSY; @@ -409,24 +409,14 @@ rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe) SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL); - /* Atomically add this core to the mapped cores first, then examine if - * we can run the service. This avoids a race condition between - * checking the value, and atomically adding to the mapped count. + /* Increment num_mapped_cores to indicate that the service is + * is running on a core. */ - if (serialize_mt_unsafe) - rte_atomic32_inc(&s->num_mapped_cores); + rte_atomic32_inc(&s->num_mapped_cores); - if (service_mt_safe(s) == 0 && - rte_atomic32_read(&s->num_mapped_cores) > 1) { - if (serialize_mt_unsafe) - rte_atomic32_dec(&s->num_mapped_cores); - return -EBUSY; - } - - int ret = service_run(id, cs, UINT64_MAX, s); + int ret = service_run(id, cs, UINT64_MAX, s, serialize_mt_unsafe); - if (serialize_mt_unsafe) - rte_atomic32_dec(&s->num_mapped_cores); + rte_atomic32_dec(&s->num_mapped_cores); return ret; } @@ -446,7 +436,7 @@ service_runner_func(void *arg) if (!service_valid(i)) continue; /* return value ignored as no change to code flow */ - service_run(i, cs, service_mask, service_get(i)); + service_run(i, cs, service_mask, service_get(i), 1); } cs->loops++;