From patchwork Thu Nov 9 16:41:16 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Bellasi X-Patchwork-Id: 118465 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp6816370qgn; Thu, 9 Nov 2017 08:42:12 -0800 (PST) X-Google-Smtp-Source: ABhQp+RQymCtuf/X3QUvreawTd+mQHtFezqHlk23SpoIVmqvpniW0gFj4/9xof4tBssQcaXSn7t+ X-Received: by 10.101.80.132 with SMTP id r4mr1009530pgp.428.1510245732800; Thu, 09 Nov 2017 08:42:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510245732; cv=none; d=google.com; s=arc-20160816; b=vHxARhoQcOt/r7uD5kZtheF3A8kbeVCPW5HRICJhnBavqocOnPJ7RAHS9p0MQZg/K4 Wo8MjMnShd5ZOPGWEknul8m7+FZaayJ7WPEn7byQRpgBxutylnIIuFBY59nYizpPbMj5 FWN2I//Ync8iXNJ4syd5SyS4X8CapuwPOXyDkFZjEqf5aX0WaYI5Bg5h0jdLmn/qSZZs 6BP2mbwysSqAtrGX7gort12qvw5vX4vSWO/N6k0sqmLuBIYZG1JLtqrwYj0KRbpl+g1T fD8ACVMD0vUjPxZ9UPyFud1V1j5yyvMarkY+IVEgh3Bc9WPf6HtJv0vW3xt2EsxrGxEW AKXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=qVUVZ2cS6UQYIOapviHUXO3bznUFJivkodQcsWFjHXw=; b=XcBlRamabEyhGoGESZv4oaXZxNy+d6FOMXSEdFmeorM8Q6aPtGRKiNP/iQMa2hqSap 1ok65LP8JsdkuBk850V4A/PxEIkMDkhFx64uf3g06UkjX/sL69ASjXqvFrtGZxwt9q7e j934cCfahjSJAisX9IrHKA3euIKAZ654O92KcVAcxVIwDoH26R9tkdlDq9t+2fCNs6wi 1Gtc0nn5dDC5SkY521j64A1Apt+ynY3lEOZaSrGdNnD7O2jfE4euNFDO0UPAzE7q1lz6 LqcqFctcN+N+7UmWSJgDM/vRq/h9rsxxFDuCRCgjaGvHR12h26X1eS0lJg9wt2Z01LbA YcbA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w12si6816071pgp.482.2017.11.09.08.42.12; Thu, 09 Nov 2017 08:42:12 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753726AbdKIQmL (ORCPT + 23 others); Thu, 9 Nov 2017 11:42:11 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:49242 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753288AbdKIQlo (ORCPT ); Thu, 9 Nov 2017 11:41:44 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9C0BF164F; Thu, 9 Nov 2017 08:41:44 -0800 (PST) Received: from e110439-lin.cambridge.arm.com (e110439-lin.cambridge.arm.com [10.1.210.68]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 42C2B3F3DF; Thu, 9 Nov 2017 08:41:42 -0800 (PST) From: Patrick Bellasi To: linux-kernel@vger.kernel.org Cc: Ingo Molnar , Peter Zijlstra , "Rafael J . Wysocki" , Viresh Kumar , Vincent Guittot , Paul Turner , Dietmar Eggemann , Morten Rasmussen , Juri Lelli , Todd Kjos , Joel Fernandes , linux-pm@vger.kernel.org Subject: [PATCH 3/4] sched/fair: use util_est in LB and WU paths Date: Thu, 9 Nov 2017 16:41:16 +0000 Message-Id: <20171109164117.19401-4-patrick.bellasi@arm.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20171109164117.19401-1-patrick.bellasi@arm.com> References: <20171109164117.19401-1-patrick.bellasi@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org When the scheduler looks at the CPU utilization, the current PELT value for a CPU is returned straight away. In certain scenarios this can have undesired side effects on task placement. For example, since the task utilization is decayed at wakeup time, when a long sleeping big task is enqueued it does not add immediately a significant contribution to the target CPU. As a result we generate a race condition where other tasks can be placed on the same CPU while is still considered relatively empty. In order to reduce these kind of race conditions, this patch introduces the required support to integrate the usage of the CPU's estimated utilization in cpu_util_wake as well as in update_sg_lb_stats. The estimated utilization of a CPU is defined to be the maximum between its PELT's utilization and the sum of the estimated utilization of the tasks currently RUNNABLE on that CPU. This allows to properly represent the expected utilization of a CPU which, for example it has just got a big task running since a long sleep period. Signed-off-by: Patrick Bellasi Reviewed-by: Brendan Jackman Reviewed-by: Dietmar Eggemann Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Rafael J. Wysocki Cc: Viresh Kumar Cc: Paul Turner Cc: Vincent Guittot Cc: Morten Rasmussen Cc: Dietmar Eggemann Cc: linux-kernel@vger.kernel.org Cc: linux-pm@vger.kernel.org --- kernel/sched/fair.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 68 insertions(+), 6 deletions(-) -- 2.14.1 diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index f14d199e81ed..76626e1a7645 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -5973,6 +5973,41 @@ static unsigned long cpu_util(int cpu) return (util >= capacity) ? capacity : util; } +/** + * cpu_util_est: estimated utilization for the specified CPU + * @cpu: the CPU to get the estimated utilization for + * + * The estimated utilization of a CPU is defined to be the maximum between its + * PELT's utilization and the sum of the estimated utilization of the tasks + * currently RUNNABLE on that CPU. + * + * This allows to properly represent the expected utilization of a CPU which + * has just got a big task running since a long sleep period. At the same time + * however it preserves the benefits of the "blocked load" in describing the + * potential for other tasks waking up on the same CPU. + * + * Return: the estimated utlization for the specified CPU + */ +static inline unsigned long cpu_util_est(int cpu) +{ + unsigned long util, util_est; + unsigned long capacity; + struct cfs_rq *cfs_rq; + + if (!sched_feat(UTIL_EST)) + return cpu_util(cpu); + + cfs_rq = &cpu_rq(cpu)->cfs; + util = cfs_rq->avg.util_avg; + util_est = cfs_rq->util_est_runnable; + util_est = max(util, util_est); + + capacity = capacity_orig_of(cpu); + util_est = min(util_est, capacity); + + return util_est; +} + static inline unsigned long task_util(struct task_struct *p) { return p->se.avg.util_avg; @@ -5989,16 +6024,43 @@ static inline unsigned long task_util_est(struct task_struct *p) */ static unsigned long cpu_util_wake(int cpu, struct task_struct *p) { - unsigned long util, capacity; + long util, util_est; /* Task has no contribution or is new */ if (cpu != task_cpu(p) || !p->se.avg.last_update_time) - return cpu_util(cpu); + return cpu_util_est(cpu); - capacity = capacity_orig_of(cpu); - util = max_t(long, cpu_rq(cpu)->cfs.avg.util_avg - task_util(p), 0); + /* Discount task's blocked util from CPU's util */ + util = cpu_util(cpu) - task_util(p); + util = max(util, 0L); - return (util >= capacity) ? capacity : util; + if (!sched_feat(UTIL_EST)) + return util; + + /* + * These are the main cases covered: + * - if *p is the only task sleeping on this CPU, then: + * cpu_util (== task_util) > util_est (== 0) + * and thus we return: + * cpu_util_wake = (cpu_util - task_util) = 0 + * + * - if other tasks are SLEEPING on the same CPU, which is just waking + * up, then: + * cpu_util >= task_util + * cpu_util > util_est (== 0) + * and thus we discount *p's blocked utilization to return: + * cpu_util_wake = (cpu_util - task_util) >= 0 + * + * - if other tasks are RUNNABLE on that CPU and + * util_est > cpu_util + * then we use util_est since it returns a more restrictive + * estimation of the spare capacity on that CPU, by just considering + * the expected utilization of tasks already runnable on that CPU. + */ + util_est = cpu_rq(cpu)->cfs.util_est_runnable; + util = max(util, util_est); + + return util; } /* @@ -7525,7 +7587,7 @@ static inline void update_sg_lb_stats(struct lb_env *env, load = source_load(i, load_idx); sgs->group_load += load; - sgs->group_util += cpu_util(i); + sgs->group_util += cpu_util_est(i); sgs->sum_nr_running += rq->cfs.h_nr_running; nr_running = rq->nr_running;