From patchwork Mon Nov 27 04:21:00 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 119653 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp1253102qgn; Sun, 26 Nov 2017 20:24:46 -0800 (PST) X-Google-Smtp-Source: AGs4zMaldcgSjbFJpL9MxUL7C7nZ61FpXT5COiqCkAP34wGNneghxsefIFDPYmPOPqWW3/Pb37GB X-Received: by 10.99.158.18 with SMTP id s18mr5627406pgd.176.1511756686118; Sun, 26 Nov 2017 20:24:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1511756686; cv=none; d=google.com; s=arc-20160816; b=uiArzKnlmyrJhT7zMcnJnfxbVTS08RkOmIXOSGXteReI0Vc93wpIordZusfeVT90ZN pHIyRXy7H8Ul81Aef0hTB/dUkQHqF5wA5avkwjPxUgMZzHk8YlJoE87e75YMhVWYqE3p 7uK6a3Pu3mhjphYViHrgPs4JddV6ihZDMXiGcYHK6u4MWp4ZDkG8WY+zPIKebKaUBk59 bOd5rbZw08Y76IxPTMU5f7s0Zpf1/hcrSDQf2aSQYTBBZqyZxbyiOtP4HmSW08jmEiIE oaRY3nhu4zqQKJO+9kW15o9NJcQ7QjizkzyJdB7H67ieX4pcrQ9U97sNWeF7o4zo9VUd DyRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=Egeo+IAz+pjR/4WodQ+ePreMvKKeWeF087eM7b85QpY=; b=VsAq2rTaZp0G3PlaH+RO660vdado+L8Xkx+aDpd1VmcN+ZyZEXklOyVwKyzCdSIye8 uNH2ILv65cuXu2o2/69NBO/8N41gLw8AOeyGEXQRtEuJLmQegxzOWDw47oFd9C1TXcie dKbzgy4WWqM2mNr+Q5idCFV2YX7TSogG7qmMSOMsyjZOaPT6a3D10blRhNa1TJp11sGg tnw5oLKQ3kCGIoLdp8rYGhu7VaRWdKkQYL5ESZmTxf2MivY6X29ZwfsUayLNeejmMe4b LiablMwQEZ7NCNVORb1F0nTA8zrXWcD1Rc8JCGWs/QMIyr0L6qFcvrZ71HalIRY4DO9w eWRg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h82si24952559pfa.196.2017.11.26.20.24.45; Sun, 26 Nov 2017 20:24:46 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751367AbdK0EYn (ORCPT + 28 others); Sun, 26 Nov 2017 23:24:43 -0500 Received: from smtp2.provo.novell.com ([137.65.250.81]:34087 "EHLO smtp2.provo.novell.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751227AbdK0EYk (ORCPT ); Sun, 26 Nov 2017 23:24:40 -0500 Received: from localhost.localdomain (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by smtp2.provo.novell.com with ESMTP (TLS encrypted); Sun, 26 Nov 2017 21:24:27 -0700 From: Davidlohr Bueso To: acme@kernel.org Cc: james.yang@arm.com, kim.phillips@arm.com, dave@stgolabs.net, linux-kernel@vger.kernel.org, Kim Phillips , Davidlohr Bueso Subject: [PATCH 2/3] perf bench futex: Add --affine-wakers option to wake-parallel Date: Sun, 26 Nov 2017 20:21:00 -0800 Message-Id: <20171127042101.3659-3-dave@stgolabs.net> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20171127042101.3659-1-dave@stgolabs.net> References: <20171127042101.3659-1-dave@stgolabs.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: James Yang The waker threads' processor affinity is not specified, so the result has run-to-run variability as the scheduler decides on which CPUs they are to run. So we add a -W/--affine-wakers flag to stripe the affinity of the waker threads across the online CPUs instead of having the scheduler place them. Cc: Kim Phillips Signed-off-by: James Yang Signed-off-by: Davidlohr Bueso --- tools/perf/bench/futex-wake-parallel.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) -- 2.13.6 diff --git a/tools/perf/bench/futex-wake-parallel.c b/tools/perf/bench/futex-wake-parallel.c index 979e303e4797..c04e207ea37c 100644 --- a/tools/perf/bench/futex-wake-parallel.c +++ b/tools/perf/bench/futex-wake-parallel.c @@ -39,6 +39,7 @@ static u_int32_t futex = 0; static pthread_t *blocked_worker; static bool done = false, silent = false, fshared = false; +static bool affine_wakers = false; static unsigned int nblocked_threads = 0, nwaking_threads = 0; static pthread_mutex_t thread_lock; static pthread_cond_t thread_parent, thread_worker; @@ -51,6 +52,7 @@ static const struct option options[] = { OPT_UINTEGER('w', "nwakers", &nwaking_threads, "Specify amount of waking threads"), OPT_BOOLEAN( 's', "silent", &silent, "Silent mode: do not display data/details"), OPT_BOOLEAN( 'S', "shared", &fshared, "Use shared futexes instead of private ones"), + OPT_BOOLEAN( 'W', "affine-wakers", &affine_wakers, "Stripe affinity of waker threads across CPUs"), OPT_END() }; @@ -78,7 +80,8 @@ static void *waking_workerfn(void *arg) return NULL; } -static void wakeup_threads(struct thread_data *td, pthread_attr_t thread_attr) +static void wakeup_threads(struct thread_data *td, pthread_attr_t thread_attr, + struct cpu_map *cpu) { unsigned int i; @@ -91,6 +94,17 @@ static void wakeup_threads(struct thread_data *td, pthread_attr_t thread_attr) * as it will affect the order to acquire the hb spinlock. * For now let the scheduler decide. */ + if (affine_wakers) { + cpu_set_t cpuset; + CPU_ZERO(&cpuset); + CPU_SET(cpu->map[(i + 1) % cpu->nr], &cpuset); + + if (pthread_attr_setaffinity_np(&thread_attr, + sizeof(cpu_set_t), + &cpuset)) + err(EXIT_FAILURE, "pthread_attr_setaffinity_np"); + } + if (pthread_create(&td[i].worker, &thread_attr, waking_workerfn, (void *)&td[i])) err(EXIT_FAILURE, "pthread_create"); @@ -276,7 +290,7 @@ int bench_futex_wake_parallel(int argc, const char **argv) usleep(100000); /* Ok, all threads are patiently blocked, start waking folks up */ - wakeup_threads(waking_worker, thread_attr); + wakeup_threads(waking_worker, thread_attr, cpu); for (i = 0; i < nblocked_threads; i++) { ret = pthread_join(blocked_worker[i], NULL); From patchwork Mon Nov 27 04:21:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 119654 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp1253204qgn; Sun, 26 Nov 2017 20:24:56 -0800 (PST) X-Google-Smtp-Source: AGs4zMbGmkzYZBCiJLX4Sa9DGl1dmhOZwmaQYr25OMjVa32sVtY5Ua/aCBXkDwNdiZiEZqCP4f/O X-Received: by 10.98.108.200 with SMTP id h191mr34335244pfc.146.1511756696638; Sun, 26 Nov 2017 20:24:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1511756696; cv=none; d=google.com; s=arc-20160816; b=sJYlIe0lI7ZBkb2OJ6NGrmyJX31djqswAVe0uf4zeLmwogiD/uGscr0TXqwQRsarrk SRvXbsahZzilPalqo/rJdM0xN+ZPbf4mllfdjD+Wc5g++RoJ9PcRx3Dy++eBH0WK8Pbv 0kp8SeOPzU64AsCTZcLZAX7hxqH70UBjbIr6DDw5yrtDMLTbPo5z107hv3rqVHOEKEbH VK+g9ndNdc/0nltuv5bpS3QCmrQlsUIBw2ClEc2sN6VQI7M9ttNL6+Gz5wvcOy1Sy79i yD71gCn5BHv2SOob/UaCvOxYtSHSoRlQZ0GUG6xHC/60mGbFUuY8rXN0eb7k+el3xXgp R8bA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=ER7sIZBwb++gpjsz5unX1ak/9HJs9T7C6CO9M8kAd2Q=; b=ytBygPcGgszJG2mQTp1rDGqlbVnNKROiX34AroMdgMrqG2mla1l9ekUaluh3pVtbrw uQYHQrwcFKS83lUieEHmYMnNIIjXEb+KhTlQQ2rD4xUqtyxPz7OegBJYTjIYyS9eKkou YaYbQEv/K0Z75TWYyu+lWm06vgciLpy7tuyy1N/Jiaa7IG0FdCXuCq8dMWTE7JfNOUGz lrKXqL+dn4JYSnyjukfrQjP1zobvyEVgNCuNCPB7V9YPXXYErkqMqkMWOantIB9s00oP 6oFplVQgKsrp45EKC0hjxks+r4tlxbHwD75WauTnyAGQ/lUxDCK+xk4MsJ6FnJCVdCXN hrxQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x21si22158998pge.567.2017.11.26.20.24.56; Sun, 26 Nov 2017 20:24:56 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751313AbdK0EYm (ORCPT + 28 others); Sun, 26 Nov 2017 23:24:42 -0500 Received: from smtp2.provo.novell.com ([137.65.250.81]:34220 "EHLO smtp2.provo.novell.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751231AbdK0EYk (ORCPT ); Sun, 26 Nov 2017 23:24:40 -0500 Received: from localhost.localdomain (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by smtp2.provo.novell.com with ESMTP (TLS encrypted); Sun, 26 Nov 2017 21:24:29 -0700 From: Davidlohr Bueso To: acme@kernel.org Cc: james.yang@arm.com, kim.phillips@arm.com, dave@stgolabs.net, linux-kernel@vger.kernel.org, Kim Phillips Subject: [PATCH 3/3] perf bench futex: sync waker threads Date: Sun, 26 Nov 2017 20:21:01 -0800 Message-Id: <20171127042101.3659-4-dave@stgolabs.net> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20171127042101.3659-1-dave@stgolabs.net> References: <20171127042101.3659-1-dave@stgolabs.net> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: James Yang Waker threads in the futex wake-parallel benchmark are started by a loop using pthread_create(). However, there is no synchronization for when the waker threads wake the waiting threads. Comparison of the waker threads' measurement timestamps show they are not all running concurrently because older waker threads finish their task before newer waker threads even start. This patch uses a barrier to better synchronize the waker threads. Cc: Kim Phillips Signed-off-by: James Yang --- tools/perf/bench/futex-wake-parallel.c | 9 +++++++++ 1 file changed, 9 insertions(+) -- 2.13.6 diff --git a/tools/perf/bench/futex-wake-parallel.c b/tools/perf/bench/futex-wake-parallel.c index c04e207ea37c..55bff8673d31 100644 --- a/tools/perf/bench/futex-wake-parallel.c +++ b/tools/perf/bench/futex-wake-parallel.c @@ -43,6 +43,7 @@ static bool affine_wakers = false; static unsigned int nblocked_threads = 0, nwaking_threads = 0; static pthread_mutex_t thread_lock; static pthread_cond_t thread_parent, thread_worker; +static pthread_barrier_t barrier; static struct stats waketime_stats, wakeup_stats; static unsigned int threads_starting; static int futex_flag = 0; @@ -66,6 +67,8 @@ static void *waking_workerfn(void *arg) struct thread_data *waker = (struct thread_data *) arg; struct timeval start, end; + pthread_barrier_wait(&barrier); + gettimeofday(&start, NULL); waker->nwoken = futex_wake(&futex, nwakes, futex_flag); @@ -87,6 +90,8 @@ static void wakeup_threads(struct thread_data *td, pthread_attr_t thread_attr, pthread_attr_setdetachstate(&thread_attr, PTHREAD_CREATE_JOINABLE); + pthread_barrier_init(&barrier, NULL, nwaking_threads + 1); + /* create and block all threads */ for (i = 0; i < nwaking_threads; i++) { /* @@ -110,9 +115,13 @@ static void wakeup_threads(struct thread_data *td, pthread_attr_t thread_attr, err(EXIT_FAILURE, "pthread_create"); } + pthread_barrier_wait(&barrier); + for (i = 0; i < nwaking_threads; i++) if (pthread_join(td[i].worker, NULL)) err(EXIT_FAILURE, "pthread_join"); + + pthread_barrier_destroy(&barrier); } static void *blocked_workerfn(void *arg __maybe_unused)