From patchwork Sat May 2 00:02:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 186050 Delivered-To: patch@linaro.org Received: by 2002:a50:3a95:0:0:0:0:0 with SMTP id v21csp791974ecc; Fri, 1 May 2020 17:03:57 -0700 (PDT) X-Google-Smtp-Source: APiQypIqBIraCgLwijI+b73m56sCJyAIsuw5UJhGX4u7PwBWXRg27hS/YBFjsjNxNPCTt+g/BI7h X-Received: by 2002:a17:906:f90d:: with SMTP id lc13mr5322348ejb.367.1588377837220; Fri, 01 May 2020 17:03:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1588377837; cv=none; d=google.com; s=arc-20160816; b=vPJKPywJ+TxY2W2bvzV9BibnrV4uxHJHDk/DqTdY7rpVd2tnKjWr/he77P1uEgrAWy /W5Lxifo1ahNguyQV99gbgliuf6yJhVZ3Lp+yleuUKBXfL6V1PRUKgFww2TH4Ie4gs0g G1MTphmjGxH1lngRzpqPv/SpAySz2EoJuVJ1bU9BgC+DlGKIl0jIVaHFeWDEpSrMRsNF e39vJm1Ze9bvtYHE8dSh+uAFDuInxlRVeQ7/32EIFkItBS6lPzDn5CsJCPfmRnq1h3gr jHC0YacuDq4JjsHpNcMtg75HeT0nIFq7ZtgU2kBu7DctdDaL+ZZwHAkgkq8dmjAchh9K MQcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=P/G36I8Evq79wsJVuWeXmXA7U4aw+YDKEDoVLCvW0Sk=; b=V/480e2IM3nRXgSIQhwEB/2weRkTUkDN1hEtS57faWayqurpEJuTPp9iP+H04sfrOO GF+XJDr6E+LNJ2s/cidUWVNhpdYdYsVTz5+dYIwf+Z4obSmJ7o8RePHCWlZ17rb6XK9f lwyY22qnbkthGroKC0mVSXTqCCWph4LHzPCMJi0rHCFgUO3hJ7sOFJMW5hz5Dn5mPoF1 6QJHyyjmkQ8q6eK2tHvDEcboOBziWrARdsSvpTYva+cA1hMHOSyOmIdYAq6D5WG82baZ 5jMKlFSRrWbG8TDcI1h36wIPzYYiGTSW7rxrTheUyZbwcyX/6qN6HP82ZNGhn/WEvxot SI6w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id oy15si2800183ejb.9.2020.05.01.17.03.57; Fri, 01 May 2020 17:03:57 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 755461DA1D; Sat, 2 May 2020 02:03:11 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id CC17B1D9B2 for ; Sat, 2 May 2020 02:03:06 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 565B9106F; Fri, 1 May 2020 17:03:06 -0700 (PDT) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.14.48]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4A22A3F305; Fri, 1 May 2020 17:03:06 -0700 (PDT) From: Honnappa Nagarahalli To: dev@dpdk.org, phil.yang@arm.com, harry.van.haaren@intel.com Cc: thomas@monjalon.net, david.marchand@redhat.com, konstantin.ananyev@intel.com, jerinj@marvell.com, hemant.agrawal@nxp.com, gage.eads@intel.com, bruce.richardson@intel.com, honnappa.nagarahalli@arm.com, nd@arm.com Date: Fri, 1 May 2020 19:02:45 -0500 Message-Id: <20200502000245.11071-7-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200502000245.11071-1-honnappa.nagarahalli@arm.com> References: <1587659482-27133-1-git-send-email-phil.yang@arm.com> <20200502000245.11071-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [PATCH v3 6/6] service: relax barriers with C11 atomics X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Phil Yang The runstate, comp_runstate and app_runstate are used as guard variables in the service core lib. To guarantee the inter-threads visibility of these guard variables, it uses rte_smp_r/wmb. This patch use c11 atomic built-ins to relax these barriers. Signed-off-by: Phil Yang Reviewed-by: Honnappa Nagarahalli --- lib/librte_eal/common/rte_service.c | 115 ++++++++++++++++++++-------- 1 file changed, 84 insertions(+), 31 deletions(-) -- 2.17.1 Acked-by: Harry van Haaren diff --git a/lib/librte_eal/common/rte_service.c b/lib/librte_eal/common/rte_service.c index 8cac265c9..dbb821139 100644 --- a/lib/librte_eal/common/rte_service.c +++ b/lib/librte_eal/common/rte_service.c @@ -265,7 +265,6 @@ rte_service_component_register(const struct rte_service_spec *spec, s->spec = *spec; s->internal_flags |= SERVICE_F_REGISTERED | SERVICE_F_START_CHECK; - rte_smp_wmb(); rte_service_count++; if (id_ptr) @@ -282,7 +281,6 @@ rte_service_component_unregister(uint32_t id) SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL); rte_service_count--; - rte_smp_wmb(); s->internal_flags &= ~(SERVICE_F_REGISTERED); @@ -301,12 +299,17 @@ rte_service_component_runstate_set(uint32_t id, uint32_t runstate) struct rte_service_spec_impl *s; SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL); + /* comp_runstate act as the guard variable. Use store-release + * memory order. This synchronizes with load-acquire in + * service_run and service_runstate_get function. + */ if (runstate) - s->comp_runstate = RUNSTATE_RUNNING; + __atomic_store_n(&s->comp_runstate, RUNSTATE_RUNNING, + __ATOMIC_RELEASE); else - s->comp_runstate = RUNSTATE_STOPPED; + __atomic_store_n(&s->comp_runstate, RUNSTATE_STOPPED, + __ATOMIC_RELEASE); - rte_smp_wmb(); return 0; } @@ -316,12 +319,17 @@ rte_service_runstate_set(uint32_t id, uint32_t runstate) struct rte_service_spec_impl *s; SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL); + /* app_runstate act as the guard variable. Use store-release + * memory order. This synchronizes with load-acquire in + * service_run runstate_get function. + */ if (runstate) - s->app_runstate = RUNSTATE_RUNNING; + __atomic_store_n(&s->app_runstate, RUNSTATE_RUNNING, + __ATOMIC_RELEASE); else - s->app_runstate = RUNSTATE_STOPPED; + __atomic_store_n(&s->app_runstate, RUNSTATE_STOPPED, + __ATOMIC_RELEASE); - rte_smp_wmb(); return 0; } @@ -330,15 +338,24 @@ rte_service_runstate_get(uint32_t id) { struct rte_service_spec_impl *s; SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL); - rte_smp_rmb(); - int check_disabled = !(s->internal_flags & SERVICE_F_START_CHECK); - int lcore_mapped = (__atomic_load_n(&s->num_mapped_cores, + /* comp_runstate and app_runstate act as the guard variables. + * Use load-acquire memory order. This synchronizes with + * store-release in service state set functions. + */ + if (__atomic_load_n(&s->comp_runstate, + __ATOMIC_ACQUIRE) == RUNSTATE_RUNNING && + __atomic_load_n(&s->app_runstate, + __ATOMIC_ACQUIRE) == RUNSTATE_RUNNING) { + int check_disabled = !(s->internal_flags & + SERVICE_F_START_CHECK); + int lcore_mapped = (__atomic_load_n(&s->num_mapped_cores, __ATOMIC_RELAXED) > 0); - return (s->app_runstate == RUNSTATE_RUNNING) && - (s->comp_runstate == RUNSTATE_RUNNING) && - (check_disabled | lcore_mapped); + return (check_disabled | lcore_mapped); + } else + return 0; + } static inline void @@ -367,9 +384,15 @@ service_run(uint32_t i, struct core_state *cs, uint64_t service_mask, if (!s) return -EINVAL; - if (s->comp_runstate != RUNSTATE_RUNNING || - s->app_runstate != RUNSTATE_RUNNING || - !(service_mask & (UINT64_C(1) << i))) { + /* comp_runstate and app_runstate act as the guard variables. + * Use load-acquire memory order. This synchronizes with + * store-release in service state set functions. + */ + if (__atomic_load_n(&s->comp_runstate, + __ATOMIC_ACQUIRE) != RUNSTATE_RUNNING || + __atomic_load_n(&s->app_runstate, + __ATOMIC_ACQUIRE) != RUNSTATE_RUNNING || + !(service_mask & (UINT64_C(1) << i))) { cs->service_active_on_lcore[i] = 0; return -ENOEXEC; } @@ -434,7 +457,12 @@ service_runner_func(void *arg) const int lcore = rte_lcore_id(); struct core_state *cs = &lcore_states[lcore]; - while (cs->runstate == RUNSTATE_RUNNING) { + /* runstate act as the guard variable. Use load-acquire + * memory order here to synchronize with store-release + * in runstate update functions. + */ + while (__atomic_load_n(&cs->runstate, + __ATOMIC_ACQUIRE) == RUNSTATE_RUNNING) { const uint64_t service_mask = cs->service_mask; for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) { @@ -445,8 +473,6 @@ service_runner_func(void *arg) } cs->loops++; - - rte_smp_rmb(); } lcore_config[lcore].state = WAIT; @@ -614,15 +640,18 @@ rte_service_lcore_reset_all(void) if (lcore_states[i].is_service_core) { lcore_states[i].service_mask = 0; set_lcore_state(i, ROLE_RTE); - lcore_states[i].runstate = RUNSTATE_STOPPED; + /* runstate act as guard variable Use + * store-release memory order here to synchronize + * with load-acquire in runstate read functions. + */ + __atomic_store_n(&lcore_states[i].runstate, + RUNSTATE_STOPPED, __ATOMIC_RELEASE); } } for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) __atomic_store_n(&rte_services[i].num_mapped_cores, 0, __ATOMIC_RELAXED); - rte_smp_wmb(); - return 0; } @@ -638,9 +667,11 @@ rte_service_lcore_add(uint32_t lcore) /* ensure that after adding a core the mask and state are defaults */ lcore_states[lcore].service_mask = 0; - lcore_states[lcore].runstate = RUNSTATE_STOPPED; - - rte_smp_wmb(); + /* Use store-release memory order here to synchronize with + * load-acquire in runstate read functions. + */ + __atomic_store_n(&lcore_states[lcore].runstate, RUNSTATE_STOPPED, + __ATOMIC_RELEASE); return rte_eal_wait_lcore(lcore); } @@ -655,7 +686,12 @@ rte_service_lcore_del(uint32_t lcore) if (!cs->is_service_core) return -EINVAL; - if (cs->runstate != RUNSTATE_STOPPED) + /* runstate act as the guard variable. Use load-acquire + * memory order here to synchronize with store-release + * in runstate update functions. + */ + if (__atomic_load_n(&cs->runstate, + __ATOMIC_ACQUIRE) != RUNSTATE_STOPPED) return -EBUSY; set_lcore_state(lcore, ROLE_RTE); @@ -674,13 +710,21 @@ rte_service_lcore_start(uint32_t lcore) if (!cs->is_service_core) return -EINVAL; - if (cs->runstate == RUNSTATE_RUNNING) + /* runstate act as the guard variable. Use load-acquire + * memory order here to synchronize with store-release + * in runstate update functions. + */ + if (__atomic_load_n(&cs->runstate, + __ATOMIC_ACQUIRE) == RUNSTATE_RUNNING) return -EALREADY; /* set core to run state first, and then launch otherwise it will * return immediately as runstate keeps it in the service poll loop */ - cs->runstate = RUNSTATE_RUNNING; + /* Use load-acquire memory order here to synchronize with + * store-release in runstate update functions. + */ + __atomic_store_n(&cs->runstate, RUNSTATE_RUNNING, __ATOMIC_RELEASE); int ret = rte_eal_remote_launch(service_runner_func, 0, lcore); /* returns -EBUSY if the core is already launched, 0 on success */ @@ -693,7 +737,12 @@ rte_service_lcore_stop(uint32_t lcore) if (lcore >= RTE_MAX_LCORE) return -EINVAL; - if (lcore_states[lcore].runstate == RUNSTATE_STOPPED) + /* runstate act as the guard variable. Use load-acquire + * memory order here to synchronize with store-release + * in runstate update functions. + */ + if (__atomic_load_n(&lcore_states[lcore].runstate, + __ATOMIC_ACQUIRE) == RUNSTATE_STOPPED) return -EALREADY; uint32_t i; @@ -713,7 +762,11 @@ rte_service_lcore_stop(uint32_t lcore) return -EBUSY; } - lcore_states[lcore].runstate = RUNSTATE_STOPPED; + /* Use store-release memory order here to synchronize with + * load-acquire in runstate read functions. + */ + __atomic_store_n(&lcore_states[lcore].runstate, RUNSTATE_STOPPED, + __ATOMIC_RELEASE); return 0; }