From patchwork Fri Jun 25 15:26:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Luba X-Patchwork-Id: 467185 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B7CCC2B9F4 for ; Fri, 25 Jun 2021 15:26:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 85D1D6193D for ; Fri, 25 Jun 2021 15:26:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229816AbhFYP3E (ORCPT ); Fri, 25 Jun 2021 11:29:04 -0400 Received: from foss.arm.com ([217.140.110.172]:58636 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229445AbhFYP3D (ORCPT ); Fri, 25 Jun 2021 11:29:03 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B3C6F12FC; Fri, 25 Jun 2021 08:26:42 -0700 (PDT) Received: from e123648.arm.com (unknown [10.57.7.232]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 90EF23F694; Fri, 25 Jun 2021 08:26:39 -0700 (PDT) From: Lukasz Luba To: linux-kernel@vger.kernel.org Cc: Chris.Redpath@arm.com, lukasz.luba@arm.com, dietmar.eggemann@arm.com, morten.rasmussen@arm.com, qperret@google.com, linux-pm@vger.kernel.org, peterz@infradead.org, rjw@rjwysocki.net, viresh.kumar@linaro.org, vincent.guittot@linaro.org, mingo@redhat.com, juri.lelli@redhat.com, rostedt@goodmis.org, segall@google.com, mgorman@suse.de, bristot@redhat.com, CCj.Yeh@mediatek.com Subject: [PATCH 1/3] sched/fair: Prepare variables for increased precision of EAS estimated energy Date: Fri, 25 Jun 2021 16:26:01 +0100 Message-Id: <20210625152603.25960-2-lukasz.luba@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210625152603.25960-1-lukasz.luba@arm.com> References: <20210625152603.25960-1-lukasz.luba@arm.com> Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org The Energy Aware Scheduler (EAS) tries to find best CPU for a waking up task. It probes many possibilities and compares the estimated energy values for different scenarios. For calculating those energy values it relies on Energy Model (EM) data and em_cpu_energy(). The precision which is used in EM data is in milli-Watts (or abstract scale), which sometimes is not sufficient. In some cases it might happen that two CPUs from different Performance Domains (PDs) get the same calculated value for a given task placement, but in more precised scale, they might differ. This rounding error has to be addressed. This patch prepares EAS code for better precision in the coming EM improvements. Signed-off-by: Lukasz Luba --- kernel/sched/fair.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 7b8990fd4896..b517c9e79768 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6582,7 +6582,7 @@ static unsigned long cpu_util_next(int cpu, struct task_struct *p, int dst_cpu) * to compute what would be the energy if we decided to actually migrate that * task. */ -static long +static u64 compute_energy(struct task_struct *p, int dst_cpu, struct perf_domain *pd) { struct cpumask *pd_mask = perf_domain_span(pd); @@ -6689,12 +6689,13 @@ compute_energy(struct task_struct *p, int dst_cpu, struct perf_domain *pd) */ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu) { - unsigned long prev_delta = ULONG_MAX, best_delta = ULONG_MAX; struct root_domain *rd = cpu_rq(smp_processor_id())->rd; + u64 prev_delta = ULLONG_MAX, best_delta = ULLONG_MAX; int cpu, best_energy_cpu = prev_cpu, target = -1; - unsigned long cpu_cap, util, base_energy = 0; + unsigned long cpu_cap, util; struct sched_domain *sd; struct perf_domain *pd; + u64 base_energy = 0; rcu_read_lock(); pd = rcu_dereference(rd->pd); @@ -6718,9 +6719,9 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu) goto unlock; for (; pd; pd = pd->next) { - unsigned long cur_delta, spare_cap, max_spare_cap = 0; + unsigned long spare_cap, max_spare_cap = 0; bool compute_prev_delta = false; - unsigned long base_energy_pd; + u64 base_energy_pd, cur_delta; int max_spare_cap_cpu = -1; for_each_cpu_and(cpu, perf_domain_span(pd), sched_domain_span(sd)) { @@ -6790,7 +6791,7 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu) * Pick the best CPU if prev_cpu cannot be used, or if it saves at * least 6% of the energy used by prev_cpu. */ - if ((prev_delta == ULONG_MAX) || + if ((prev_delta == ULLONG_MAX) || (prev_delta - best_delta) > ((prev_delta + base_energy) >> 4)) target = best_energy_cpu; From patchwork Fri Jun 25 15:26:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Luba X-Patchwork-Id: 467665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAAA9C2B9F4 for ; Fri, 25 Jun 2021 15:26:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B35226193F for ; Fri, 25 Jun 2021 15:26:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229844AbhFYP3H (ORCPT ); Fri, 25 Jun 2021 11:29:07 -0400 Received: from foss.arm.com ([217.140.110.172]:58660 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229445AbhFYP3H (ORCPT ); Fri, 25 Jun 2021 11:29:07 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 399591396; Fri, 25 Jun 2021 08:26:46 -0700 (PDT) Received: from e123648.arm.com (unknown [10.57.7.232]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0C50B3F694; Fri, 25 Jun 2021 08:26:42 -0700 (PDT) From: Lukasz Luba To: linux-kernel@vger.kernel.org Cc: Chris.Redpath@arm.com, lukasz.luba@arm.com, dietmar.eggemann@arm.com, morten.rasmussen@arm.com, qperret@google.com, linux-pm@vger.kernel.org, peterz@infradead.org, rjw@rjwysocki.net, viresh.kumar@linaro.org, vincent.guittot@linaro.org, mingo@redhat.com, juri.lelli@redhat.com, rostedt@goodmis.org, segall@google.com, mgorman@suse.de, bristot@redhat.com, CCj.Yeh@mediatek.com Subject: [PATCH 2/3] PM: EM: Make em_cpu_energy() able to return bigger values Date: Fri, 25 Jun 2021 16:26:02 +0100 Message-Id: <20210625152603.25960-3-lukasz.luba@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210625152603.25960-1-lukasz.luba@arm.com> References: <20210625152603.25960-1-lukasz.luba@arm.com> Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org The Energy Model (EM) em_cpu_energy() is responsible for providing good estimation regarding CPUs energy. It contains proper data structures which are then used during calculation. The values stored in there are in milli-Watts precision (or in abstract scale) smaller that 0xffff, which use sufficient unsigned long even on 32-bit machines. There are scenarios where we would like to provide calculated estimations in a better precision and the values might be 1000 times bigger. This patch makes possible to use quite big values for also 32-bit machines. Signed-off-by: Lukasz Luba --- include/linux/energy_model.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/include/linux/energy_model.h b/include/linux/energy_model.h index 3f221dbf5f95..2016f5a706e0 100644 --- a/include/linux/energy_model.h +++ b/include/linux/energy_model.h @@ -101,7 +101,7 @@ void em_dev_unregister_perf_domain(struct device *dev); * Return: the sum of the energy consumed by the CPUs of the domain assuming * a capacity state satisfying the max utilization of the domain. */ -static inline unsigned long em_cpu_energy(struct em_perf_domain *pd, +static inline u64 em_cpu_energy(struct em_perf_domain *pd, unsigned long max_util, unsigned long sum_util, unsigned long allowed_cpu_cap) { @@ -180,7 +180,7 @@ static inline unsigned long em_cpu_energy(struct em_perf_domain *pd, * pd_nrg = ------------------------ (4) * scale_cpu */ - return ps->cost * sum_util / scale_cpu; + return div_u64((u64)ps->cost * sum_util, scale_cpu); } /** @@ -217,7 +217,7 @@ static inline struct em_perf_domain *em_pd_get(struct device *dev) { return NULL; } -static inline unsigned long em_cpu_energy(struct em_perf_domain *pd, +static inline u64 em_cpu_energy(struct em_perf_domain *pd, unsigned long max_util, unsigned long sum_util, unsigned long allowed_cpu_cap) { From patchwork Fri Jun 25 15:26:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukasz Luba X-Patchwork-Id: 467184 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82ED6C2B9F4 for ; Fri, 25 Jun 2021 15:26:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6926A6193F for ; Fri, 25 Jun 2021 15:26:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229930AbhFYP3L (ORCPT ); Fri, 25 Jun 2021 11:29:11 -0400 Received: from foss.arm.com ([217.140.110.172]:58684 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229445AbhFYP3K (ORCPT ); Fri, 25 Jun 2021 11:29:10 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AF19413A1; Fri, 25 Jun 2021 08:26:49 -0700 (PDT) Received: from e123648.arm.com (unknown [10.57.7.232]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 83E683F694; Fri, 25 Jun 2021 08:26:46 -0700 (PDT) From: Lukasz Luba To: linux-kernel@vger.kernel.org Cc: Chris.Redpath@arm.com, lukasz.luba@arm.com, dietmar.eggemann@arm.com, morten.rasmussen@arm.com, qperret@google.com, linux-pm@vger.kernel.org, peterz@infradead.org, rjw@rjwysocki.net, viresh.kumar@linaro.org, vincent.guittot@linaro.org, mingo@redhat.com, juri.lelli@redhat.com, rostedt@goodmis.org, segall@google.com, mgorman@suse.de, bristot@redhat.com, CCj.Yeh@mediatek.com Subject: [PATCH 3/3] PM: EM: Increase energy calculation precision Date: Fri, 25 Jun 2021 16:26:03 +0100 Message-Id: <20210625152603.25960-4-lukasz.luba@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210625152603.25960-1-lukasz.luba@arm.com> References: <20210625152603.25960-1-lukasz.luba@arm.com> Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org The Energy Model (EM) provides useful information about device power in each performance state to other subsystems like: Energy Aware Scheduler (EAS). The energy calculation in EAS does arithmetic operation based on the EM em_cpu_energy(). Current implementation of that function uses em_perf_state::cost as a pre-computed cost coefficient equal to: cost = power * max_frequency / frequency. The 'power' is expressed in milli-Watts (or in abstract scale). There are corner cases then the EAS energy calculation for two Performance Domains (PDs) return the same value, e.g. 10mW. The EAS compares these values to choose smaller one. It might happen that this values are equal due to rounding error. In such scenario, we need better precision, e.g. 10000 times better. To provide this possibility increase the precision on the em_perf_state::cost. This patch allows to avoid the rounding to milli-Watt errors, which might occur in EAS energy estimation for each Performance Domains (PD). The rounding error is common for small tasks which have small utilization values. The rest of the EM code doesn't change, em_perf_state::power is still expressed in milli-Watts (or in abstract scale). Thus, all existing platforms don't have to change their reported power. The same applies to EM clients, like thermal or DTPM (they use em_perf_state::power). Reported-by: CCJ Yeh Suggested-by: CCJ Yeh Signed-off-by: Lukasz Luba --- include/linux/energy_model.h | 5 ++++- kernel/power/energy_model.c | 3 ++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/include/linux/energy_model.h b/include/linux/energy_model.h index 2016f5a706e0..91037dd57e61 100644 --- a/include/linux/energy_model.h +++ b/include/linux/energy_model.h @@ -16,7 +16,10 @@ * @power: The power consumed at this level (by 1 CPU or by a registered * device). It can be a total power: static and dynamic. * @cost: The cost coefficient associated with this level, used during - * energy calculation. Equal to: power * max_frequency / frequency + * energy calculation. Equal to: + power * 10000 * max_frequency / frequency + * To increase the energy estimation presision use different + * scale in this coefficient than in @power field. */ struct em_perf_state { unsigned long frequency; diff --git a/kernel/power/energy_model.c b/kernel/power/energy_model.c index 0f4530b3a8cd..2724f0ac417d 100644 --- a/kernel/power/energy_model.c +++ b/kernel/power/energy_model.c @@ -170,7 +170,8 @@ static int em_create_perf_table(struct device *dev, struct em_perf_domain *pd, /* Compute the cost of each performance state. */ fmax = (u64) table[nr_states - 1].frequency; for (i = 0; i < nr_states; i++) { - table[i].cost = div64_u64(fmax * table[i].power, + u64 power_res = (u64)table[i].power * 10000; + table[i].cost = div64_u64(fmax * power_res, table[i].frequency); }