From patchwork Thu Aug 24 18:08:56 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Bellasi X-Patchwork-Id: 110944 Delivered-To: patch@linaro.org Received: by 10.140.95.78 with SMTP id h72csp6048090qge; Thu, 24 Aug 2017 11:09:49 -0700 (PDT) X-Received: by 10.98.130.206 with SMTP id w197mr7109697pfd.310.1503598189042; Thu, 24 Aug 2017 11:09:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1503598189; cv=none; d=google.com; s=arc-20160816; b=Ef/UZEPpEZnH92AfScw9Z6/NBUeN1Wpmk/R9vH3VXhq7olSIy34pud3u6ytsmk2Ofj n3P7XQTu6IKVHg5g69i7HQPGwkwvbvkEb+MlnBXPgT+dlmhbmLtHsOvPPSj56ql8POpX +9NhvEeDdhKrUsPsc75LyhtEV+UB+AyACqKpmKIglhH+dPub4dqm60pb3F6k6YedRABO netPpQZ6HaS5HUdvMwQ+nV4TpDbD2q6GpskU/7FPcMLrm8u2iV8zhkf9lM44tgJ5ix+o PxgYOOxgebPmiE1WGd5Iq0LYf/wHMJF+p24sCVndq3IrWj8Qnc3/XG7vY01t2P+P1MDe Kikw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=AZcf77aSqAfVkdux+yJERo9Z9HLpPxEYS3KX2b9vdl8=; b=T2BCNbcEYTZP6YieGHaMt0NP+wx+WzGGGPTTq02yRDD04KQUjbDqIZgu0ZC36DdEBS Cc6aAy4fThAMOkNnL65gr1xxSi/wlbvGRMsUTBQ5If8dYNyTrpCz802GVz/ARkzJ+kxc PdWAzgjL46T+AHAHxh80Rh7/gH2ey5mESVF+WG0zPf5ApwFid9wsjiGnC7c3AfTAYZZF D70UuLVdEz98TRtwcDBO6kxUmUG43zEJL7Bo9n+7prsYD4nsLg5qVlkWoDE+9yTYRolY LPtGNkA3CxG6HBR2WzyhupUrVQ8njoPKQCf0m1Sl+ugbNuXtnwlfK0S3lzBbCsBLUJOs RgLg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n1si3173462pga.824.2017.08.24.11.09.48; Thu, 24 Aug 2017 11:09:49 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753683AbdHXSJp (ORCPT + 26 others); Thu, 24 Aug 2017 14:09:45 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:45494 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753519AbdHXSJj (ORCPT ); Thu, 24 Aug 2017 14:09:39 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7E1F81715; Thu, 24 Aug 2017 11:09:39 -0700 (PDT) Received: from e110439-lin.cambridge.arm.com (e110439-lin.cambridge.arm.com [10.1.210.68]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 86E943F3E1; Thu, 24 Aug 2017 11:09:36 -0700 (PDT) From: Patrick Bellasi To: linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org Cc: Ingo Molnar , Peter Zijlstra , Tejun Heo , "Rafael J . Wysocki" , Paul Turner , Vincent Guittot , John Stultz , Morten Rasmussen , Dietmar Eggemann , Juri Lelli , Tim Murray , Todd Kjos , Andres Oportus , Joel Fernandes , Viresh Kumar Subject: [RFCv4 5/6] cpufreq: schedutil: add util clamp for FAIR tasks Date: Thu, 24 Aug 2017 19:08:56 +0100 Message-Id: <20170824180857.32103-6-patrick.bellasi@arm.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20170824180857.32103-1-patrick.bellasi@arm.com> References: <20170824180857.32103-1-patrick.bellasi@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Each time a frequency update is required via schedutil, we must grant the util_{min,max} constraints enforced in the current CPU by its set of currently RUNNABLE tasks. This patch adds the required support to clamp the utilization generated by FAIR tasks within the boundaries defined by their aggregated utilization clamp constraints. The clamped utilization is then used to select the frequency thus allowing, for example, to: - boost tasks which are directly affecting the user experience by running them at least at a minimum "required" frequency - cap low priority tasks not directly affecting the user experience by running them only up to a maximum "allowed" frequency The default values for boosting and capping are defined to be: - util_min: 0 - util_max: SCHED_CAPACITY_SCALE which means that by default no boosting/capping is enforced. Signed-off-by: Patrick Bellasi Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Rafael J. Wysocki Cc: linux-kernel@vger.kernel.org Cc: linux-pm@vger.kernel.org --- kernel/sched/cpufreq_schedutil.c | 33 ++++++++++++++++++++++ kernel/sched/sched.h | 60 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 93 insertions(+) -- 2.14.1 diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index 29a397067ffa..f67c26bbade4 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -231,6 +231,7 @@ static void sugov_update_single(struct update_util_data *hook, u64 time, } else { sugov_get_util(&util, &max); sugov_iowait_boost(sg_cpu, &util, &max); + util = uclamp_util(smp_processor_id(), util); next_f = get_next_freq(sg_policy, util, max); /* * Do not reduce the frequency if the CPU has not been idle @@ -246,9 +247,18 @@ static unsigned int sugov_next_freq_shared(struct sugov_cpu *sg_cpu, u64 time) { struct sugov_policy *sg_policy = sg_cpu->sg_policy; struct cpufreq_policy *policy = sg_policy->policy; + unsigned long max_util, min_util; unsigned long util = 0, max = 1; unsigned int j; + /* Initialize clamp values based on caller CPU constraints */ + if (uclamp_enabled) { + int cpu = smp_processor_id(); + + max_util = uclamp_value(cpu, UCLAMP_MAX); + min_util = uclamp_value(cpu, UCLAMP_MIN); + } + for_each_cpu(j, policy->cpus) { struct sugov_cpu *j_sg_cpu = &per_cpu(sugov_cpu, j); unsigned long j_util, j_max; @@ -277,8 +287,31 @@ static unsigned int sugov_next_freq_shared(struct sugov_cpu *sg_cpu, u64 time) } sugov_iowait_boost(j_sg_cpu, &util, &max); + + /* + * Update clamping range based on j-CPUs constraints, but only + * if active. Idle CPUs do not enforce constraints in a shared + * frequency domain. + */ + if (uclamp_enabled && !idle_cpu(j)) { + unsigned long j_max_util, j_min_util; + + j_max_util = uclamp_value(j, UCLAMP_MAX); + j_min_util = uclamp_value(j, UCLAMP_MIN); + + /* + * Clamp values are MAX aggregated among all the + * different CPUs in the shared frequency domain. + */ + max_util = max(max_util, j_max_util); + min_util = max(min_util, j_min_util); + } } + /* Clamp utilization based on aggregated uclamp constraints */ + if (uclamp_enabled) + util = clamp(util, min_util, max_util); + return get_next_freq(sg_policy, util, max); } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 164a8ac152b3..4a235c4a0762 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2224,6 +2224,66 @@ static inline void cpufreq_update_util(struct rq *rq, unsigned int flags) {} static inline void cpufreq_update_this_cpu(struct rq *rq, unsigned int flags) {} #endif /* CONFIG_CPU_FREQ */ +#ifdef CONFIG_UTIL_CLAMP +/* Enable clamping code at compile time by constant propagation */ +#define uclamp_enabled true + +/** + * uclamp_value: get the current CPU's utilization clamp value + * @cpu: the CPU to consider + * @clamp_id: the utilization clamp index (i.e. min or max utilization) + * + * The utilization clamp value for a CPU depends on its set of currently + * active tasks and their specific util_{min,max} constraints. + * A max aggregated value is tracked for each CPU and returned by this + * function. An IDLE CPU never enforces a clamp value. + * + * Return: the current value for the specified CPU and clamp index + */ +static inline unsigned int uclamp_value(unsigned int cpu, int clamp_id) +{ + struct uclamp_cpu *uc_cpu = &cpu_rq(cpu)->uclamp[clamp_id]; + int clamp_value = uclamp_none(clamp_id); + + /* Update min utilization clamp */ + if (uc_cpu->value != UCLAMP_NONE) + clamp_value = uc_cpu->value; + + return clamp_value; +} + +/** + * clamp_util: clamp a utilization value for a specified CPU + * @cpu: the CPU to get the clamp values from + * @util: the utilization signal to clamp + * + * Each CPU tracks util_{min,max} clamp values depending on the set of its + * currently active tasks. Given a utilization signal, i.e a signal in the + * [0..SCHED_CAPACITY_SCALE] range, this function returns a clamped + * utilization signal considering the current clamp values for the + * specified CPU. + * + * Return: a clamped utilization signal for a given CPU. + */ +static inline int uclamp_util(unsigned int cpu, unsigned int util) +{ + unsigned int min_util = uclamp_value(cpu, UCLAMP_MIN); + unsigned int max_util = uclamp_value(cpu, UCLAMP_MAX); + + return clamp(util, min_util, max_util); +} +#else +/* Disable clamping code at compile time by constant propagation */ +#define uclamp_enabled false +#define uclamp_util(cpu, util) util +static inline unsigned int uclamp_value(unsigned int cpu, int clamp_id) +{ + if (clamp_id == UCLAMP_MIN) + return 0; + return SCHED_CAPACITY_SCALE; +} +#endif /* CONFIG_UTIL_CLAMP */ + #ifdef arch_scale_freq_capacity #ifndef arch_scale_freq_invariant #define arch_scale_freq_invariant() (true)