From patchwork Fri Mar 16 11:25:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 131918 Delivered-To: patch@linaro.org Received: by 10.46.84.17 with SMTP id i17csp617542ljb; Fri, 16 Mar 2018 04:26:02 -0700 (PDT) X-Google-Smtp-Source: AG47ELtV9ua7Eb/bBi7OH4mqF7sLL1EdeZEpzd/P46NAjOOLgkOOEtGP6Gmhgwd91JXeHGQz7Fql X-Received: by 10.98.102.155 with SMTP id s27mr1299865pfj.198.1521199562595; Fri, 16 Mar 2018 04:26:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1521199562; cv=none; d=google.com; s=arc-20160816; b=gJjhgH4LzVCX02UnuiJMYoch3sLph5ZkbrUW4EZSDA+xtVkH6Sl0gSsCCINjKQX4x2 Zxyydne7KgpYg6z/0FEwHGio1I2bosrMfPW47IIW1FQ5jT47ok++be9ukA88gX2WfDMD xnu7/3vsxl8W0EVErk4dkQJ2AVz30DuoIKAVuUhHUWq0/Q2zZzirV4ZNfuQAkxQedTY2 twFn6p/njH4QgqUQZucBNQJw3yOWi9Cev0GGejB4bP/vGVmMjz3TXLelN8bueA2HqKKg m12A0x0X03bo8W9JvLiIPTUbDDqW7UC/qFiz8qCBSafqnca+6eL5pPl6VUYBcZAq2PiN XMtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=dDHwCU13IdFGmm7buZPl3Ymx7q2j3VwNA6EtjObtB60=; b=E8EV1n5tywdHkw8q6UIbZOFXLI01K7b2kdJFTCMgwcluxUopZbfulLlY6+0E/7vI2t 2XZsQgVEPIJBIwBE/RHVA8g7Li+D+r/u0FaM+431akmpR+2fwNisKDaY18x1HLiGPNWB Ugevo0eB2pIsNj8rVUf7kL0j33a37TLGWejXfTx1kK3NLlYRfRQ17JEoHaRURGGxyriL ZLcxyL5AT7/Dz8e4HvtYlps60BDVzyWBxpmE3AgrvE4wQmhJBKxRBGbmNQsCermJhHpp kovD0/CHJiG4UdHL8qPN8++IiUgw/9SsCWU0QAWw5JDOxGWtod/ntKD7PDkT17+ug4zX qEww== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CN4DBfVt; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f12-v6si6007009plo.91.2018.03.16.04.26.02; Fri, 16 Mar 2018 04:26:02 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CN4DBfVt; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753903AbeCPLZ6 (ORCPT + 28 others); Fri, 16 Mar 2018 07:25:58 -0400 Received: from mail-wr0-f194.google.com ([209.85.128.194]:36933 "EHLO mail-wr0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753777AbeCPLZz (ORCPT ); Fri, 16 Mar 2018 07:25:55 -0400 Received: by mail-wr0-f194.google.com with SMTP id z12so11372850wrg.4 for ; Fri, 16 Mar 2018 04:25:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dDHwCU13IdFGmm7buZPl3Ymx7q2j3VwNA6EtjObtB60=; b=CN4DBfVtR36FHsnZhGLVQw1WDNdCVC4S+0ZzDiUgYh3/g0NyrhV17c/O3Kjo7XWEdE KBAgkHK1v5a5/tG/IykXgbWlqpnwMuEMBk+FMBz9QDMkbVz3Zzw8xaB+4P44LFHJcwom cqnoIcQnVgqkytds6NRJBhX0zdqtPaADM1hDA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dDHwCU13IdFGmm7buZPl3Ymx7q2j3VwNA6EtjObtB60=; b=XWpb6i/Ppv8IJHON460SXrQXkwWO5M80COTVfEu0eYq96IJQhxZbHxaxJGd8U8SL2z zOrRdopaRgSx3jarTN+7Gw5X4lB4+sxj8BDJMXpimdDeWhmZdYNXD7b/fn2ZB8dNGAZ1 1OCmojZtMgc266ghW0cTWpTPc/SMvCBABzGGG9BgPesK2YkCNgZJZNX45ZD+N8gvL3wt 9wVU8xcf+5mEMJz8IKyrfhnFVExWLYdP+wnIypwRpw2+MIS+Aen8sR42da5MEAdV8B/k Cnta3b/vV+04T4CIyt9/kWFGA0ooMvh6/UxNjJEdfexXwfIIsugMkmgrmqsZGMf2vjoM Dp/g== X-Gm-Message-State: AElRT7HGmv9fMD/89WXpTU+TJECWoKP5mCQKZsgsPNA2wWizaOQKIo/Q ljyxH986lkLcj4+xge//beFjZA== X-Received: by 10.223.171.79 with SMTP id r15mr1250778wrc.208.1521199553266; Fri, 16 Mar 2018 04:25:53 -0700 (PDT) Received: from localhost.localdomain ([2a01:e0a:f:6020:9471:2e09:99e5:f59]) by smtp.gmail.com with ESMTPSA id m15sm6785292wrb.58.2018.03.16.04.25.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 16 Mar 2018 04:25:52 -0700 (PDT) From: Vincent Guittot To: peterz@infradead.org, mingo@kernel.org, linux-kernel@vger.kernel.org, rjw@rjwysocki.net Cc: juri.lelli@redhat.com, dietmar.eggemann@arm.com, Morten.Rasmussen@arm.com, viresh.kumar@linaro.org, valentin.schneider@arm.com, Vincent Guittot Subject: [PATCH 2/4 v4] sched/rt: add rt_rq utilization tracking Date: Fri, 16 Mar 2018 12:25:39 +0100 Message-Id: <1521199541-15308-3-git-send-email-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1521199541-15308-1-git-send-email-vincent.guittot@linaro.org> References: <1521199541-15308-1-git-send-email-vincent.guittot@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org schedutil governor relies on cfs_rq's util_avg to choose the OPP when cfs tasks are running. When the CPU is overloaded by cfs and rt tasks, cfs tasks are preempted by rt tasks and in this case util_avg reflects the remaining capacity that is used by cfs task but not what cfs want to use. In such case, schedutil can select a lower OPP whereas the CPU is overloaded. In order to have a more accurate view of the utilization of the CPU, we track the utilization that is "stolen" by RT tasks. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 2 ++ kernel/sched/pelt.c | 23 +++++++++++++++++++++++ kernel/sched/pelt.h | 7 +++++++ kernel/sched/rt.c | 8 ++++++++ kernel/sched/sched.h | 2 ++ 5 files changed, 42 insertions(+) -- 2.7.4 diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index bfd56bc..60e3c4b 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7190,6 +7190,7 @@ static void update_blocked_averages(int cpu) if (cfs_rq_has_blocked(cfs_rq)) done = false; } + update_rt_rq_load_avg(rq_clock_task(rq), cpu, &rq->rt, 0); #ifdef CONFIG_NO_HZ_COMMON rq->last_blocked_load_update_tick = jiffies; @@ -7255,6 +7256,7 @@ static inline void update_blocked_averages(int cpu) rq_lock_irqsave(rq, &rf); update_rq_clock(rq); update_cfs_rq_load_avg(cfs_rq_clock_task(cfs_rq), cfs_rq); + update_rt_rq_load_avg(rq_clock_task(rq), cpu, &rq->rt, 0); #ifdef CONFIG_NO_HZ_COMMON rq->last_blocked_load_update_tick = jiffies; if (!cfs_rq_has_blocked(cfs_rq)) diff --git a/kernel/sched/pelt.c b/kernel/sched/pelt.c index d693e5e..cd51576 100644 --- a/kernel/sched/pelt.c +++ b/kernel/sched/pelt.c @@ -306,3 +306,26 @@ int __update_load_avg_cfs_rq(u64 now, int cpu, struct cfs_rq *cfs_rq) return 0; } + +/* + * rt_rq: + * + * util_sum = \Sum se->avg.util_sum but se->avg.util_sum is not tracked + * util_sum = cpu_scale * load_sum + * runnable_load_sum = load_sum + * + */ + +int update_rt_rq_load_avg(u64 now, int cpu, struct rt_rq *rt_rq, int running) +{ + if (___update_load_sum(now, cpu, &rt_rq->avg, + running, + running, + running)) { + + ___update_load_avg(&rt_rq->avg, 1, 1); + return 1; + } + + return 0; +} diff --git a/kernel/sched/pelt.h b/kernel/sched/pelt.h index c312d8c..78a2107 100644 --- a/kernel/sched/pelt.h +++ b/kernel/sched/pelt.h @@ -3,6 +3,7 @@ int __update_load_avg_blocked_se(u64 now, int cpu, struct sched_entity *se); int __update_load_avg_se(u64 now, int cpu, struct cfs_rq *cfs_rq, struct sched_entity *se); int __update_load_avg_cfs_rq(u64 now, int cpu, struct cfs_rq *cfs_rq); +int update_rt_rq_load_avg(u64 now, int cpu, struct rt_rq *rt_rq, int running); #else @@ -12,6 +13,12 @@ update_cfs_rq_load_avg(u64 now, struct cfs_rq *cfs_rq) return 0; } +static inline int +update_rt_rq_load_avg(u64 now, int cpu, struct rt_rq *rt_rq, int running) +{ + return 0; +} + #endif diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 86b7798..c48078e 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -5,6 +5,8 @@ */ #include "sched.h" +#include "pelt.h" + int sched_rr_timeslice = RR_TIMESLICE; int sysctl_sched_rr_timeslice = (MSEC_PER_SEC / HZ) * RR_TIMESLICE; @@ -1570,6 +1572,9 @@ pick_next_task_rt(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) rt_queue_push_tasks(rq); + update_rt_rq_load_avg(rq_clock_task(rq), cpu_of(rq), rt_rq, + rq->curr->sched_class == &rt_sched_class); + return p; } @@ -1577,6 +1582,8 @@ static void put_prev_task_rt(struct rq *rq, struct task_struct *p) { update_curr_rt(rq); + update_rt_rq_load_avg(rq_clock_task(rq), cpu_of(rq), &rq->rt, 1); + /* * The previous task needs to be made eligible for pushing * if it is still active @@ -2306,6 +2313,7 @@ static void task_tick_rt(struct rq *rq, struct task_struct *p, int queued) struct sched_rt_entity *rt_se = &p->rt; update_curr_rt(rq); + update_rt_rq_load_avg(rq_clock_task(rq), cpu_of(rq), &rq->rt, 1); watchdog(rq, p); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 783eacf..a8003a9 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -592,6 +592,8 @@ struct rt_rq { unsigned long rt_nr_total; int overloaded; struct plist_head pushable_tasks; + + struct sched_avg avg; #endif /* CONFIG_SMP */ int rt_queued;