From patchwork Fri Jul 3 09:57:47 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 50621 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-wi0-f197.google.com (mail-wi0-f197.google.com [209.85.212.197]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 7B4FE214B3 for ; Fri, 3 Jul 2015 09:58:29 +0000 (UTC) Received: by wiwz6 with SMTP id z6sf31227964wiw.0 for ; Fri, 03 Jul 2015 02:58:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:mime-version:in-reply-to:references :from:date:message-id:subject:to:cc:content-type:sender:precedence :list-id:x-original-sender:x-original-authentication-results :mailing-list:list-post:list-help:list-archive:list-unsubscribe; bh=dNh3wCcm45EVFlfZ4pulTMyyl9EjVGHsua0uAf5toQg=; b=DQGGF5JskaqPhv8gFwzYZAG/jnGpMl0V9ySvvBrwFi5dE+CSQKG2wI4xsy3vZtkfg5 r+l5sddca4NSF4+sNpUT9JTX8xSjOe2UWXS0WxPZel/y3CBX3jXqzN1gwtzWzbWle/6z EI1QxxNIdh5vjnli5mR7ozEZIrWeKOqDMvnjEPUlBywyAn84/pRbS/jQRQ2EAKOTGgjv wi0wK6oY3r3g5RxPv1u3SV1pxxXR+yZDVjguEm4xU/BHfsyzmCjUNWd7A905aHfCaNoY 0v5zZc5lwNNcojwzy6m1FbnQ/HE910BKQ9aVTq8453obn3tOIxpvI5h58tGdjd7cAHkB CLFQ== X-Gm-Message-State: ALoCoQkoZsxx31I80SncfZW6IoXOa3zdc/Pig1wTjP2zigSpblS7mAOaz5h9GmhwChLaPq4bQFcf X-Received: by 10.112.85.69 with SMTP id f5mr22256972lbz.23.1435917508796; Fri, 03 Jul 2015 02:58:28 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.87.198 with SMTP id ba6ls475285lab.40.gmail; Fri, 03 Jul 2015 02:58:28 -0700 (PDT) X-Received: by 10.152.44.193 with SMTP id g1mr35061540lam.15.1435917508403; Fri, 03 Jul 2015 02:58:28 -0700 (PDT) Received: from mail-la0-f49.google.com (mail-la0-f49.google.com. [209.85.215.49]) by mx.google.com with ESMTPS id a3si6802310laa.64.2015.07.03.02.58.28 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Jul 2015 02:58:28 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.49 as permitted sender) client-ip=209.85.215.49; Received: by laar3 with SMTP id r3so79269661laa.0 for ; Fri, 03 Jul 2015 02:58:28 -0700 (PDT) X-Received: by 10.152.4.163 with SMTP id l3mr35651046lal.35.1435917508161; Fri, 03 Jul 2015 02:58:28 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.108.230 with SMTP id hn6csp1057190lbb; Fri, 3 Jul 2015 02:58:26 -0700 (PDT) X-Received: by 10.68.219.1 with SMTP id pk1mr74786285pbc.18.1435917505630; Fri, 03 Jul 2015 02:58:25 -0700 (PDT) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id fc10si13554881pad.170.2015.07.03.02.58.22; Fri, 03 Jul 2015 02:58:25 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-pm-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754788AbbGCJ6T (ORCPT + 11 others); Fri, 3 Jul 2015 05:58:19 -0400 Received: from mail-la0-f41.google.com ([209.85.215.41]:35989 "EHLO mail-la0-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754594AbbGCJ6J (ORCPT ); Fri, 3 Jul 2015 05:58:09 -0400 Received: by lagc2 with SMTP id c2so79220316lag.3 for ; Fri, 03 Jul 2015 02:58:07 -0700 (PDT) X-Received: by 10.152.87.13 with SMTP id t13mr35349816laz.66.1435917487560; Fri, 03 Jul 2015 02:58:07 -0700 (PDT) MIME-Version: 1.0 Received: by 10.112.198.7 with HTTP; Fri, 3 Jul 2015 02:57:47 -0700 (PDT) In-Reply-To: <1435362824-26734-1-git-send-email-mturquette@linaro.org> References: <1435362824-26734-1-git-send-email-mturquette@linaro.org> From: Vincent Guittot Date: Fri, 3 Jul 2015 11:57:47 +0200 Message-ID: Subject: Re: [PATCH v3 0/4] scheduler-driven cpu frequency selection To: Michael Turquette Cc: Peter Zijlstra , Ingo Molnar , linux-kernel , Preeti U Murthy , Morten Rasmussen , Rik van Riel , Mike Galbraith , Nicolas Pitre , Daniel Lezcano , Dietmar Eggemann , Amit Kucheria , Juri Lelli , "rjw@rjwysocki.net" , Viresh Kumar , ashwin.chaugule@linaro.org, Alex Shi , "linux-pm@vger.kernel.org" , abelvesa@gmail.com, pebolle@tiscali.nl, Michael Turquette Sender: linux-pm-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: vincent.guittot@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.49 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , Hi Mike, I have tried to evaluate the performance and the power consumption of the various policy that we have in the kernel to manage the DVFS of cpus including your sched-dvfs proposal. For this purpose, i have used rt-app and a script that are available here: https://git.linaro.org/power/rt-app.git/shortlog/refs/heads/master. The script is in the directory: doc/examples/cpufreq_governor_efficiency/ For evaluating the performance of DVFS policy, I use rt-app to run a simple sequence that alternates run phase and sleep phase. The run phase is done by looping a defined number of time on a function that burns cycles (The number of loop is calibrated to be equivalent to a specified duration at max frequency of the cpu). At the end of the test, we know how many time is needed by each policy to run the same pattern and we can compare policies. I have also done some power consumption measurement so we can compare both the performance/responsiveness and power consumption (the power figure only reflects the power consumption of the A53 cluster). The test has been done on cortex-A53 platofrm so i have ported the frequency invariance patch [1/4] on arm64 (attached to the email for reference) The performance figures has been normalized; so 100% means as fast as the performance governor that always use the max frequency and 0% means as slow as the powersave governor that always uses the min frequency. The power consumption figures have been normalized vs the performance governor power consumption. For all governor i have used the default parameters which means a sampling rate of 10ms and a sampling_down_factor of 1 for ondemand and a sampling rate of 200ms for conservative governor performance powersave ondemand sched conservative run/sleep perf energy perf energy perf energy perf energy perf energy 50ms/1000ms 100% 100% 0% 34% 95% 62% 49% 35% 72% 33% 100ms/1000ms 100% 100% 0% 41% 98% 78% 69% 41% 92% 45% 200ms/1000ms 100% 100% 0% 51% 98% 78% 86% 62% 93% 62% 400ms/1000ms 100% 100% 0% 55% 99% 83% 93% 63% 99% 77% 1000ms/100ms 100% 100% 0% 74% 99% 100% 96% 97% 99% 99% We can see that the responsiveness of the sched governor is not that good for short running duration but this can probably can be explained with the responsiveness of the load tracking which needs around 75ms to reach 80% of max usage. there are several proposal to add the blocked tasks in the usage of a cpu, this can may be improved the responsiveness for some pattern. Regards, Vincent On 27 June 2015 at 01:53, Michael Turquette wrote: > This series addresses the comments from v2 and rearranges the code to > separate the bits that are safe to merge from the bits that are not. In > particular the simplified governor no longer implements any policy of > its own. That code is migrated to fair.c and marked as RFC. The > capacity-selection code in fair.c (patch #4) is a placeholder and can be > replaced by something more sophisticated, and it illustrates the use of > the new governor api's for anyone that wants to play around with > crafting a new policy. > > Patch #1 is a re-post from Morten, as it is the only dependency these > patches have on his EAS series. Please consider merging patches #2 and > #3 if they do not appear too controversial. Without enabling the feature > in Kconfig will be no impact on existing code. > > Michael Turquette (3): > cpufreq: introduce cpufreq_driver_might_sleep > sched: scheduler-driven cpu frequency selection > [RFC] sched: cfs: cpu frequency scaling policy > > Morten Rasmussen (1): > arm: Frequency invariant scheduler load-tracking support > > arch/arm/include/asm/topology.h | 7 + > arch/arm/kernel/smp.c | 53 ++++++- > arch/arm/kernel/topology.c | 17 +++ > drivers/cpufreq/Kconfig | 24 ++++ > drivers/cpufreq/cpufreq.c | 6 + > include/linux/cpufreq.h | 12 ++ > kernel/sched/Makefile | 1 + > kernel/sched/cpufreq_sched.c | 308 ++++++++++++++++++++++++++++++++++++++++ > kernel/sched/fair.c | 41 ++++++ > kernel/sched/sched.h | 8 ++ > 10 files changed, 475 insertions(+), 2 deletions(-) > create mode 100644 kernel/sched/cpufreq_sched.c > > -- > 1.9.1 > >From 1be4a1e126cdb1b47b4d8a5dd156f8bca04715c0 Mon Sep 17 00:00:00 2001 From: Vincent Guittot Date: Fri, 3 Jul 2015 10:02:53 +0200 Subject: [PATCH] arm64: Frequency invariant scheduler load-tracking support Implements arch-specific function to provide the scheduler with a frequency scaling correction factor for more accurate load-tracking. This patch is based on the implementation that has been done for arm arch by Morten Rasmussen Signed-off-by: Vincent Guittot --- arch/arm64/include/asm/topology.h | 11 ++++++ arch/arm64/kernel/smp.c | 80 +++++++++++++++++++++++++++++++++++++++ arch/arm64/kernel/topology.c | 19 ++++++++++ 3 files changed, 110 insertions(+) diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h index 7ebcd31..887835a 100644 --- a/arch/arm64/include/asm/topology.h +++ b/arch/arm64/include/asm/topology.h @@ -24,6 +24,17 @@ void init_cpu_topology(void); void store_cpu_topology(unsigned int cpuid); const struct cpumask *cpu_coregroup_mask(int cpu); +#ifdef CONFIG_CPU_FREQ + +#define arch_scale_freq_capacity arm64_arch_scale_freq_capacity +struct sched_domain; +extern unsigned long arm64_arch_scale_freq_capacity(struct sched_domain *sd, + int cpu); + +DECLARE_PER_CPU(atomic_long_t, cpu_freq_capacity); + +#endif /* CONFIG_CPU_FREQ */ + #else static inline void init_cpu_topology(void) { } diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 328b8ce..45b88cb 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include @@ -656,3 +657,82 @@ int setup_profiling_timer(unsigned int multiplier) { return -EINVAL; } + +#ifdef CONFIG_CPU_FREQ + +static DEFINE_PER_CPU(atomic_long_t, cpu_max_freq); +DEFINE_PER_CPU(atomic_long_t, cpu_freq_capacity); + +/* + * Scheduler load-tracking scale-invariance + * + * Provides the scheduler with a scale-invariance correction factor that + * compensates for frequency scaling through arch_scale_freq_capacity() + * (implemented in topology.c). + */ +static inline +void scale_freq_capacity(int cpu, unsigned long curr, unsigned long max) +{ + unsigned long capacity; + + if (!max) + return; + + capacity = (curr << SCHED_CAPACITY_SHIFT) / max; + atomic_long_set(&per_cpu(cpu_freq_capacity, cpu), capacity); +} + +static int cpufreq_callback(struct notifier_block *nb, + unsigned long val, void *data) +{ + struct cpufreq_freqs *freq = data; + int cpu = freq->cpu; + unsigned long max = atomic_long_read(&per_cpu(cpu_max_freq, cpu)); + + if (freq->flags & CPUFREQ_CONST_LOOPS) + return NOTIFY_OK; + + scale_freq_capacity(cpu, freq->new, max); + + return NOTIFY_OK; +} + +static struct notifier_block cpufreq_notifier = { + .notifier_call = cpufreq_callback, +}; + +static int cpufreq_policy_callback(struct notifier_block *nb, + unsigned long val, void *data) +{ + struct cpufreq_policy *policy = data; + int i; + + if (val != CPUFREQ_NOTIFY) + return NOTIFY_OK; + + for_each_cpu(i, policy->cpus) { + scale_freq_capacity(i, policy->cur, policy->max); + atomic_long_set(&per_cpu(cpu_max_freq, i), policy->max); + } + + return NOTIFY_OK; +} + +static struct notifier_block cpufreq_policy_notifier = { + .notifier_call = cpufreq_policy_callback, +}; + +static int __init register_cpufreq_notifier(void) +{ + int ret; + + ret = cpufreq_register_notifier(&cpufreq_notifier, + CPUFREQ_TRANSITION_NOTIFIER); + if (ret) + return ret; + + return cpufreq_register_notifier(&cpufreq_policy_notifier, + CPUFREQ_POLICY_NOTIFIER); +} +core_initcall(register_cpufreq_notifier); +#endif diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index fcb8f7b..6387f65 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -200,6 +200,25 @@ static int __init parse_dt_topology(void) return ret; } +#ifdef CONFIG_CPU_FREQ +/* + * Scheduler load-tracking scale-invariance + * + * Provides the scheduler with a scale-invariance correction factor that + * compensates for frequency scaling (arch_scale_freq_capacity()). The scaling + * factor is updated in smp.c + */ +unsigned long arm64_arch_scale_freq_capacity(struct sched_domain *sd, int cpu) +{ + unsigned long curr = atomic_long_read(&per_cpu(cpu_freq_capacity, cpu)); + + if (!curr) + return SCHED_CAPACITY_SCALE; + + return curr; +} +#endif + /* * cpu topology table */ -- 1.9.1