From patchwork Tue Dec 5 17:10:17 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Bellasi X-Patchwork-Id: 120718 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp6004500qgn; Tue, 5 Dec 2017 09:11:18 -0800 (PST) X-Google-Smtp-Source: AGs4zMbs/O5lUYbG5NDF9StfEYjEyYqHF+qDVG9u4Qk2fpzlqJXAQ3g+MKzjI/E6wkQJVEuJUSe3 X-Received: by 10.84.130.104 with SMTP id 95mr19201962plc.151.1512493878485; Tue, 05 Dec 2017 09:11:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512493878; cv=none; d=google.com; s=arc-20160816; b=zXx7WgCsJje6woLHCf9r1RRDolXs+/P5KodNHqaOqB+CbAOX56RTKj+93c9OE6JPol sBm4CutkZ4BpMhH/zlcprJQpTjUhIIwxosddTrmJ+7ciD95alQBDGCICvUwwWO8hap/z e0kSPdUWh0ekw3TdSvrm1rImQDPReSLcAAzIHT35S7H88BX+fmyxGsUrryqdvnkEqbxg 4e2HjjggehjidBGjul+JXrw+Dwi/nupRPIdlHCq8s5qy37bnUKOYASEGteWwTYaoVzW7 TX3Zd3y3KGSEB//eilWYN5fbiQqp61dEK2WS83EY3XgnZg1pUkn87s7bJYz+FKckyVlJ NxsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=79tChW3Lz5g9S3JN9l2Dvldyefk0jw9wFGuTkyJvaVI=; b=0MWTUDSZ3+DgQxFCGraqdgp0a6QB270m6RL8viNUGosm7OZoCt0vXByldhETUdESUW eAiT1t9EKJmVhD1hjrZOrehD/VA8eOgq016AsMrz6nzAFWn9GlG6HZEC7zC+jwEC9CS6 MAmMG6o2US9D6hLEUPTnmNNTuoqTUm6k7ZTN7ToBeVa75ni3ckI/S27iB7ykVrIi8X8K QlzIcAfhuhA0y4X9HGt0axS+o/ysN/DJbAxMNKndgHake8w9VKsFtNp/dLsbQ6M6OiDo ssAhaZzsEPjwwTNvoOoq91CazKJlxBC1CxO7jo3yG5Re2wderZExcgpJD2HSZUFX8tCq ZdZw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-pm-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-pm-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f4si331093plr.643.2017.12.05.09.11.18; Tue, 05 Dec 2017 09:11:18 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-pm-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-pm-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-pm-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752780AbdLERLQ (ORCPT + 11 others); Tue, 5 Dec 2017 12:11:16 -0500 Received: from foss.arm.com ([217.140.101.70]:52338 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753063AbdLERKp (ORCPT ); Tue, 5 Dec 2017 12:10:45 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 564E515BF; Tue, 5 Dec 2017 09:10:45 -0800 (PST) Received: from e110439-lin.cambridge.arm.com (e110439-lin.cambridge.arm.com [10.1.210.68]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id F09623F246; Tue, 5 Dec 2017 09:10:42 -0800 (PST) From: Patrick Bellasi To: linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org Cc: Ingo Molnar , Peter Zijlstra , "Rafael J . Wysocki" , Viresh Kumar , Vincent Guittot , Paul Turner , Dietmar Eggemann , Morten Rasmussen , Juri Lelli , Todd Kjos , Joel Fernandes Subject: [PATCH v2 3/4] sched/fair: use util_est in LB and WU paths Date: Tue, 5 Dec 2017 17:10:17 +0000 Message-Id: <20171205171018.9203-4-patrick.bellasi@arm.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20171205171018.9203-1-patrick.bellasi@arm.com> References: <20171205171018.9203-1-patrick.bellasi@arm.com> Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org When the scheduler looks at the CPU utilization, the current PELT value for a CPU is returned straight away. In certain scenarios this can have undesired side effects on task placement. For example, since the task utilization is decayed at wakeup time, when a long sleeping big task is enqueued it does not add immediately a significant contribution to the target CPU. As a result we generate a race condition where other tasks can be placed on the same CPU while is still considered relatively empty. In order to reduce these kind of race conditions, this patch introduces the required support to integrate the usage of the CPU's estimated utilization in cpu_util_wake as well as in update_sg_lb_stats. The estimated utilization of a CPU is defined to be the maximum between its PELT's utilization and the sum of the estimated utilization of the tasks currently RUNNABLE on that CPU. This allows to properly represent the expected utilization of a CPU which, for example, has just got a big task running since a long sleep period. Signed-off-by: Patrick Bellasi Reviewed-by: Brendan Jackman Reviewed-by: Dietmar Eggemann Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Rafael J. Wysocki Cc: Viresh Kumar Cc: Paul Turner Cc: Vincent Guittot Cc: Morten Rasmussen Cc: Dietmar Eggemann Cc: linux-kernel@vger.kernel.org Cc: linux-pm@vger.kernel.org --- Changes v1->v2: - rebase on top of v4.15-rc2 - tested that overhauled PELT code does not affect the util_est --- kernel/sched/fair.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 68 insertions(+), 6 deletions(-) -- 2.14.1 diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d8f3ed71010b..373d631efa91 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6306,6 +6306,41 @@ static unsigned long cpu_util(int cpu) return (util >= capacity) ? capacity : util; } +/** + * cpu_util_est: estimated utilization for the specified CPU + * @cpu: the CPU to get the estimated utilization for + * + * The estimated utilization of a CPU is defined to be the maximum between its + * PELT's utilization and the sum of the estimated utilization of the tasks + * currently RUNNABLE on that CPU. + * + * This allows to properly represent the expected utilization of a CPU which + * has just got a big task running since a long sleep period. At the same time + * however it preserves the benefits of the "blocked load" in describing the + * potential for other tasks waking up on the same CPU. + * + * Return: the estimated utlization for the specified CPU + */ +static inline unsigned long cpu_util_est(int cpu) +{ + unsigned long util, util_est; + unsigned long capacity; + struct cfs_rq *cfs_rq; + + if (!sched_feat(UTIL_EST)) + return cpu_util(cpu); + + cfs_rq = &cpu_rq(cpu)->cfs; + util = cfs_rq->avg.util_avg; + util_est = cfs_rq->util_est_runnable; + util_est = max(util, util_est); + + capacity = capacity_orig_of(cpu); + util_est = min(util_est, capacity); + + return util_est; +} + static inline unsigned long task_util(struct task_struct *p) { return p->se.avg.util_avg; @@ -6322,16 +6357,43 @@ static inline unsigned long task_util_est(struct task_struct *p) */ static unsigned long cpu_util_wake(int cpu, struct task_struct *p) { - unsigned long util, capacity; + long util, util_est; /* Task has no contribution or is new */ if (cpu != task_cpu(p) || !p->se.avg.last_update_time) - return cpu_util(cpu); + return cpu_util_est(cpu); - capacity = capacity_orig_of(cpu); - util = max_t(long, cpu_rq(cpu)->cfs.avg.util_avg - task_util(p), 0); + /* Discount task's blocked util from CPU's util */ + util = cpu_util(cpu) - task_util(p); + util = max(util, 0L); - return (util >= capacity) ? capacity : util; + if (!sched_feat(UTIL_EST)) + return util; + + /* + * These are the main cases covered: + * - if *p is the only task sleeping on this CPU, then: + * cpu_util (== task_util) > util_est (== 0) + * and thus we return: + * cpu_util_wake = (cpu_util - task_util) = 0 + * + * - if other tasks are SLEEPING on the same CPU, which is just waking + * up, then: + * cpu_util >= task_util + * cpu_util > util_est (== 0) + * and thus we discount *p's blocked utilization to return: + * cpu_util_wake = (cpu_util - task_util) >= 0 + * + * - if other tasks are RUNNABLE on that CPU and + * util_est > cpu_util + * then we use util_est since it returns a more restrictive + * estimation of the spare capacity on that CPU, by just considering + * the expected utilization of tasks already runnable on that CPU. + */ + util_est = cpu_rq(cpu)->cfs.util_est_runnable; + util = max(util, util_est); + + return util; } /* @@ -7857,7 +7919,7 @@ static inline void update_sg_lb_stats(struct lb_env *env, load = source_load(i, load_idx); sgs->group_load += load; - sgs->group_util += cpu_util(i); + sgs->group_util += cpu_util_est(i); sgs->sum_nr_running += rq->cfs.h_nr_running; nr_running = rq->nr_running;