From patchwork Fri May 25 13:12:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 136864 Delivered-To: patch@linaro.org Received: by 2002:a2e:9706:0:0:0:0:0 with SMTP id r6-v6csp3574430lji; Fri, 25 May 2018 06:13:44 -0700 (PDT) X-Google-Smtp-Source: AB8JxZqAk4SDdG4eUdu8k6DoFp/+mqOFjcB21LpGzb9AYVBC3y2S9194cBnDctvcgkD9uY2SRPAR X-Received: by 2002:a62:c95c:: with SMTP id k89-v6mr2536488pfg.47.1527254023933; Fri, 25 May 2018 06:13:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527254023; cv=none; d=google.com; s=arc-20160816; b=thCbOLiiFlmltD62MODw4ZMf96YEzJ4Xns3bRUCCzzrNDCGe2EM7q0a1yZVu8ARcGY Xt10ug+idPu0yUdeo2juRdNJZWvL21XOz2JHQ1m6EPqdd9c6xt27gvGcq4MtPs9QPero SK9ZL5Czgkcs0FrJZoeVOUS6CoH+bKwDj6hC0YCn5UsFBlQcnI48OW65IHgHggfeA/Uy Qym7VwHFcz//aXjd2iPwQMUAqNvFf6+YDRSDxha0El5VcTd+aELEbiMGjr53kzDPYBAM EV/PXotflp3xJINuTNBK0VfbSFeAY7tGC3OBfWcnK3k0E//py5SBmeyntkuEYqTwF+yl 9szA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=lC1ixMvyDPZwLn7MlQaAOGGi+E1VlbohLHb+TH0L1UU=; b=h3NOO/tS/MtPCaiZhywutw8A0Tkfc0oonM+MTtpQokXakFVOnrXwGEdFaNfgk2C2gH yeMEfFI05kl3sIeboj9mUYBcZq218YZOlhT3qiqwwDcLoM0Sh9fHRNTHuyt7HRV/dkaW KSpqdLCmGrAge9Fj75UeP8v1if/xtmvbAXW8Eghx4C0T6ZgdA0kB6mgYbAGXHFMZqkls 5jyqGwnZfBvPnhHzTKJyWg5KHKEx65fZ8RhNSinK71NBNGslRxW0NGjthPqvKRgYrJNl s1soba46mvIkQ0/mCey7M8NjvSfRUP47YZllORFFIgbUdQ5sRGqk9JvH9p8cpJPLByZw 45Yg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iwosJfnP; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j19-v6si22685994pll.518.2018.05.25.06.13.43; Fri, 25 May 2018 06:13:43 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iwosJfnP; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S935767AbeEYNMt (ORCPT + 30 others); Fri, 25 May 2018 09:12:49 -0400 Received: from mail-wm0-f65.google.com ([74.125.82.65]:34978 "EHLO mail-wm0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752819AbeEYNMn (ORCPT ); Fri, 25 May 2018 09:12:43 -0400 Received: by mail-wm0-f65.google.com with SMTP id o78-v6so14561013wmg.0 for ; Fri, 25 May 2018 06:12:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=lC1ixMvyDPZwLn7MlQaAOGGi+E1VlbohLHb+TH0L1UU=; b=iwosJfnPmzOJvt4iGIS2SR6kogQFSOafV1sbux+Ucpje7r8OF5ngpeVxqKpKNu6i28 zgxyZJu8ZPWFq86zOAkwHVpl7hDyYGa7aW7f0xIwMsla30nSWeQm+UO+G/i7eKKCsTFq 9gNfnYXznmIX2Scy0+FA5bhrQyi2kdLLmxDHc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=lC1ixMvyDPZwLn7MlQaAOGGi+E1VlbohLHb+TH0L1UU=; b=ZxzbvBFKuHed0vClD4IWY3xuQLsrquFU5se+RTFivbF1r/phgUoq5yfJ9XdzxLGEqZ H7ZZNAsgJKxbFwAQ9yKlB5Z4QCa29VvyTiHP9NJ87+DaNaRJWAJjkSnBRVXXeOHyjJ8A rRwE7Xq+TiwZ0zjnvr07zWATG7ssuvuBhHovgFoz0eONB3bJ+OKJ0+83DahX8L0a9AHr gKNHmeoucbpn7VKW/FoVaqnzE+mvSIBGTvyLqQiQcW04TNQqRY7yB4WPYa9K1N5P88MI OO5nTvfj3GDeIuU2UtOe9skOMwStVUUoJIx7OZvqIivx8FPeaD9EFOCYPfqhg2Gepuuk 0qsg== X-Gm-Message-State: ALKqPwdv+YG+mjOVyapduXV+Jey0uCBMpAp6p7ZLCNV8M+U7pJu4rZYw 4XMK5pMX/t6CAKtShWjDxiPjh6pvavE= X-Received: by 2002:a1c:b595:: with SMTP id e143-v6mr1627309wmf.66.1527253961558; Fri, 25 May 2018 06:12:41 -0700 (PDT) Received: from localhost.localdomain ([2a01:e0a:f:6020:a860:64b4:335b:c763]) by smtp.gmail.com with ESMTPSA id 4-v6sm9690948wmg.8.2018.05.25.06.12.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 25 May 2018 06:12:40 -0700 (PDT) From: Vincent Guittot To: peterz@infradead.org, mingo@kernel.org, linux-kernel@vger.kernel.org, rjw@rjwysocki.net Cc: juri.lelli@redhat.com, dietmar.eggemann@arm.com, Morten.Rasmussen@arm.com, viresh.kumar@linaro.org, valentin.schneider@arm.com, quentin.perret@arm.com, Vincent Guittot Subject: [PATCH v5 02/10] sched/rt: add rt_rq utilization tracking Date: Fri, 25 May 2018 15:12:23 +0200 Message-Id: <1527253951-22709-3-git-send-email-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1527253951-22709-1-git-send-email-vincent.guittot@linaro.org> References: <1527253951-22709-1-git-send-email-vincent.guittot@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org schedutil governor relies on cfs_rq's util_avg to choose the OPP when cfs tasks are running. When the CPU is overloaded by cfs and rt tasks, cfs tasks are preempted by rt tasks and in this case util_avg reflects the remaining capacity but not what cfs want to use. In such case, schedutil can select a lower OPP whereas the CPU is overloaded. In order to have a more accurate view of the utilization of the CPU, we track the utilization that is "stolen" by rt tasks. rt_rq uses rq_clock_task and cfs_rq uses cfs_rq_clock_task but they are the same at the root group level, so the PELT windows of the util_sum are aligned. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 15 ++++++++++++++- kernel/sched/pelt.c | 23 +++++++++++++++++++++++ kernel/sched/pelt.h | 7 +++++++ kernel/sched/rt.c | 8 ++++++++ kernel/sched/sched.h | 7 +++++++ 5 files changed, 59 insertions(+), 1 deletion(-) -- 2.7.4 diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 6390c66..fb18bcc 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7290,6 +7290,14 @@ static inline bool cfs_rq_has_blocked(struct cfs_rq *cfs_rq) return false; } +static inline bool rt_rq_has_blocked(struct rq *rq) +{ + if (rq->avg_rt.util_avg) + return true; + + return false; +} + #ifdef CONFIG_FAIR_GROUP_SCHED static inline bool cfs_rq_is_decayed(struct cfs_rq *cfs_rq) @@ -7349,6 +7357,10 @@ static void update_blocked_averages(int cpu) if (cfs_rq_has_blocked(cfs_rq)) done = false; } + update_rt_rq_load_avg(rq_clock_task(rq), rq, 0); + /* Don't need periodic decay once load/util_avg are null */ + if (rt_rq_has_blocked(rq)) + done = false; #ifdef CONFIG_NO_HZ_COMMON rq->last_blocked_load_update_tick = jiffies; @@ -7414,9 +7426,10 @@ static inline void update_blocked_averages(int cpu) rq_lock_irqsave(rq, &rf); update_rq_clock(rq); update_cfs_rq_load_avg(cfs_rq_clock_task(cfs_rq), cfs_rq); + update_rt_rq_load_avg(rq_clock_task(rq), rq, 0); #ifdef CONFIG_NO_HZ_COMMON rq->last_blocked_load_update_tick = jiffies; - if (!cfs_rq_has_blocked(cfs_rq)) + if (!cfs_rq_has_blocked(cfs_rq) && !rt_rq_has_blocked(rq)) rq->has_blocked_load = 0; #endif rq_unlock_irqrestore(rq, &rf); diff --git a/kernel/sched/pelt.c b/kernel/sched/pelt.c index e6ecbb2..213b922 100644 --- a/kernel/sched/pelt.c +++ b/kernel/sched/pelt.c @@ -309,3 +309,26 @@ int __update_load_avg_cfs_rq(u64 now, int cpu, struct cfs_rq *cfs_rq) return 0; } + +/* + * rt_rq: + * + * util_sum = \Sum se->avg.util_sum but se->avg.util_sum is not tracked + * util_sum = cpu_scale * load_sum + * runnable_load_sum = load_sum + * + */ + +int update_rt_rq_load_avg(u64 now, struct rq *rq, int running) +{ + if (___update_load_sum(now, rq->cpu, &rq->avg_rt, + running, + running, + running)) { + + ___update_load_avg(&rq->avg_rt, 1, 1); + return 1; + } + + return 0; +} diff --git a/kernel/sched/pelt.h b/kernel/sched/pelt.h index 9cac73e..b2983b7 100644 --- a/kernel/sched/pelt.h +++ b/kernel/sched/pelt.h @@ -3,6 +3,7 @@ int __update_load_avg_blocked_se(u64 now, int cpu, struct sched_entity *se); int __update_load_avg_se(u64 now, int cpu, struct cfs_rq *cfs_rq, struct sched_entity *se); int __update_load_avg_cfs_rq(u64 now, int cpu, struct cfs_rq *cfs_rq); +int update_rt_rq_load_avg(u64 now, struct rq *rq, int running); /* * When a task is dequeued, its estimated utilization should not be update if @@ -38,6 +39,12 @@ update_cfs_rq_load_avg(u64 now, struct cfs_rq *cfs_rq) return 0; } +static inline int +update_rt_rq_load_avg(u64 now, struct rq *rq, int running) +{ + return 0; +} + #endif diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index ef3c4e6..b4148a9 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -5,6 +5,8 @@ */ #include "sched.h" +#include "pelt.h" + int sched_rr_timeslice = RR_TIMESLICE; int sysctl_sched_rr_timeslice = (MSEC_PER_SEC / HZ) * RR_TIMESLICE; @@ -1572,6 +1574,9 @@ pick_next_task_rt(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) rt_queue_push_tasks(rq); + update_rt_rq_load_avg(rq_clock_task(rq), rq, + rq->curr->sched_class == &rt_sched_class); + return p; } @@ -1579,6 +1584,8 @@ static void put_prev_task_rt(struct rq *rq, struct task_struct *p) { update_curr_rt(rq); + update_rt_rq_load_avg(rq_clock_task(rq), rq, 1); + /* * The previous task needs to be made eligible for pushing * if it is still active @@ -2308,6 +2315,7 @@ static void task_tick_rt(struct rq *rq, struct task_struct *p, int queued) struct sched_rt_entity *rt_se = &p->rt; update_curr_rt(rq); + update_rt_rq_load_avg(rq_clock_task(rq), rq, 1); watchdog(rq, p); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 757a3ee..7a16de9 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -592,6 +592,7 @@ struct rt_rq { unsigned long rt_nr_total; int overloaded; struct plist_head pushable_tasks; + #endif /* CONFIG_SMP */ int rt_queued; @@ -847,6 +848,7 @@ struct rq { u64 rt_avg; u64 age_stamp; + struct sched_avg avg_rt; u64 idle_stamp; u64 avg_idle; @@ -2205,4 +2207,9 @@ static inline unsigned long cpu_util_cfs(struct rq *rq) return util; } + +static inline unsigned long cpu_util_rt(struct rq *rq) +{ + return rq->avg_rt.util_avg; +} #endif