From patchwork Tue Jan 9 13:53:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 123963 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp4048059qgn; Tue, 9 Jan 2018 05:54:24 -0800 (PST) X-Google-Smtp-Source: ACJfBosCqHsDSUAuqK55jpoNRfQd6EkkZ2Tuqn2NiPu+obRLZv4Buoy9MrWuGLit283rbt5sAuWh X-Received: by 10.159.250.143 with SMTP id k15mr15424579pls.368.1515506064432; Tue, 09 Jan 2018 05:54:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515506064; cv=none; d=google.com; s=arc-20160816; b=Ce4mlaguFCtUGFdTm2ACH6LSLmCSty7X8l3RdETPVvleZ7r6ALfNUAre0L6/s73zq8 oocTefSOjseX5979O9oyETTIISp8gBqGxNuZEd6A6Db6PuagEkBVj7Zihn/kXo08D31C P+ls4Eb6ajEfTf7anFB9IVTQt9FCxe/psufchCSPXf6/a/mUJ3cFYKkeEftrx/C4pk2v sB9iIClbDrcCk2Ug16PhyqQkyrMrSTLsFHYtTiO+Hk8QFb71ICkpumXvRFyLImS/NdJt DpMCgDS1liaWAO56upcFJLPbg9mKdcnxOHs0rLEsGQuOS5P5jA7BHe8oDBirY+s1tZCj UOqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=mNK3AUux9TSziOkLaMHrhq4Z0tH7JkhYDwzkRdjRpII=; b=0YeBZlx7seJ3z5tYSXPY00+WxwFA90R2wB0PMzPCS6/iZN/vmfYAmW3i3Ni+KYz5rW VCY+38UwlzbAL5Xslgpw9fqhxLKCyvPCpMx6kyZ/ZS9Pnvzg+jfmcpmDeN/LSfIIQVE+ P7vQ04KzPQ6JVq8XVIPCWNiLF4LNiifMbG9Mj2vKotNrbPwttOb5rFgm/G+8B/Kf0AQ3 X55FoiE7XS3S0m1zFfJIyhywzhgZjkd6EZNuhU5EkKOsm5uY8uTh4veikSQiZtxVF2o8 dMygP9pI72Z+phIOrIptXNyf+kLKwBHHX7m+05azWSFb174Gi/1oyCEMCPKVBlI77d3Z mwhg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MqCB6ofn; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d25si2518416plj.188.2018.01.09.05.54.24; Tue, 09 Jan 2018 05:54:24 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MqCB6ofn; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758182AbeAINyG (ORCPT + 28 others); Tue, 9 Jan 2018 08:54:06 -0500 Received: from mail-wm0-f67.google.com ([74.125.82.67]:44982 "EHLO mail-wm0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752175AbeAINxS (ORCPT ); Tue, 9 Jan 2018 08:53:18 -0500 Received: by mail-wm0-f67.google.com with SMTP id t8so20509568wmc.3 for ; Tue, 09 Jan 2018 05:53:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=mNK3AUux9TSziOkLaMHrhq4Z0tH7JkhYDwzkRdjRpII=; b=MqCB6ofnTMQX16ly39dNhTtr0BOzl7oyRjf79jEg1uuvyoaAo5uL69hj9M/wynt8Oc OTlrAN7pxy9nOsz4cH3LgiPzIrv1vpnAnfa7mJwXV4Ac2hGpdJt1C4ALTDvuzFdg7iVi HxKAKfK5gC5qzn3SSaCvJx9xz4WlVbKFNBnpg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=mNK3AUux9TSziOkLaMHrhq4Z0tH7JkhYDwzkRdjRpII=; b=gwaDy3MXf9Jc3Yo3Ow+xelkaa9B18h26I84o5XjGhoUTAbj5CC/B2l0oglKtg2r5GE 6e6zB8/85fIDNIpYWe2WbLibUhC88EQS6qf7NQ34ohU135QMG1kUgtwOSUDOyKFLWpFC PLUzIJNZdE60p0xsa+zm/Xf6QrlwcaWiFl0qdlbWonkkOFZOT0pakMT0Y5q7Bj2Mq9ju dcuBvf5/05UYDdYPokvzrZUIvx48LiWRaJifXHKy+cHQCJrRdYedpUI0uFIz10RFbJaQ oHwQ2s5NaEh0pKuFfFngV0IpjTFDlKwvvybO9jDJf32x3jUCc4ATmQdVMNzMMbxbedO5 kSFg== X-Gm-Message-State: AKwxyte5D3kJhLEjFzOWSrPOAepO+f8YrgXaxY3JKhd/IQOqcnIZ7ohx B40GwH2z2Dwej6FyNGQ0iga2fw== X-Received: by 10.28.167.215 with SMTP id q206mr1703790wme.48.1515505996545; Tue, 09 Jan 2018 05:53:16 -0800 (PST) Received: from localhost.localdomain ([2a01:e0a:f:6020:f51d:768f:aab1:18e6]) by smtp.gmail.com with ESMTPSA id e7sm15070597wrd.82.2018.01.09.05.53.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 09 Jan 2018 05:53:15 -0800 (PST) From: Vincent Guittot To: peterz@infradead.org, mingo@kernel.org, linux-kernel@vger.kernel.org, rjw@rjwysocki.net Cc: juri.lelli@arm.com, dietmar.eggemann@arm.com, Morten.Rasmussen@arm.com, viresh.kumar@linaro.org, Vincent Guittot Subject: [PATCH RESEND v3 1/3] sched/pelt: Move pelt related code in a dedicated file Date: Tue, 9 Jan 2018 14:53:04 +0100 Message-Id: <1515505986-21997-2-git-send-email-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1515505986-21997-1-git-send-email-vincent.guittot@linaro.org> References: <1515505986-21997-1-git-send-email-vincent.guittot@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We want to track rt_rq's utilization as a part of the estimation of the whole rq's utilization. This is necessary because rt tasks can steal utilization to cfs tasks and make them lighter than they are. As we want to use the same load tracking mecanism for both and prevent useless dependency between cfs and rt code, pelt code is moved in a dedicated file. Signed-off-by: Vincent Guittot --- kernel/sched/Makefile | 2 +- kernel/sched/fair.c | 308 +------------------------------------------------- kernel/sched/pelt.c | 308 ++++++++++++++++++++++++++++++++++++++++++++++++++ kernel/sched/pelt.h | 17 +++ kernel/sched/sched.h | 20 ++++ 5 files changed, 347 insertions(+), 308 deletions(-) create mode 100644 kernel/sched/pelt.c create mode 100644 kernel/sched/pelt.h -- 2.7.4 diff --git a/kernel/sched/Makefile b/kernel/sched/Makefile index e2f9d4f..5a6d1c1 100644 --- a/kernel/sched/Makefile +++ b/kernel/sched/Makefile @@ -19,7 +19,7 @@ endif obj-y += core.o loadavg.o clock.o cputime.o obj-y += idle_task.o fair.o rt.o deadline.o obj-y += wait.o wait_bit.o swait.o completion.o idle.o -obj-$(CONFIG_SMP) += cpupri.o cpudeadline.o topology.o stop_task.o +obj-$(CONFIG_SMP) += cpupri.o cpudeadline.o topology.o stop_task.o pelt.o obj-$(CONFIG_SCHED_AUTOGROUP) += autogroup.o obj-$(CONFIG_SCHEDSTATS) += stats.o obj-$(CONFIG_SCHED_DEBUG) += debug.o diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 2915c0d..1abcf6e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -270,9 +270,6 @@ static inline struct rq *rq_of(struct cfs_rq *cfs_rq) return cfs_rq->rq; } -/* An entity is a task if it doesn't "own" a runqueue */ -#define entity_is_task(se) (!se->my_q) - static inline struct task_struct *task_of(struct sched_entity *se) { SCHED_WARN_ON(!entity_is_task(se)); @@ -434,7 +431,6 @@ static inline struct rq *rq_of(struct cfs_rq *cfs_rq) return container_of(cfs_rq, struct rq, cfs); } -#define entity_is_task(se) 1 #define for_each_sched_entity(se) \ for (; se; se = NULL) @@ -707,7 +703,7 @@ static u64 sched_vslice(struct cfs_rq *cfs_rq, struct sched_entity *se) } #ifdef CONFIG_SMP - +#include "pelt.h" #include "sched-pelt.h" static int select_idle_sibling(struct task_struct *p, int prev_cpu, int cpu); @@ -2723,19 +2719,6 @@ account_entity_dequeue(struct cfs_rq *cfs_rq, struct sched_entity *se) } while (0) #ifdef CONFIG_SMP -/* - * XXX we want to get rid of these helpers and use the full load resolution. - */ -static inline long se_weight(struct sched_entity *se) -{ - return scale_load_down(se->load.weight); -} - -static inline long se_runnable(struct sched_entity *se) -{ - return scale_load_down(se->runnable_weight); -} - static inline void enqueue_runnable_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se) { @@ -3038,289 +3021,6 @@ static inline void cfs_rq_util_change(struct cfs_rq *cfs_rq) } #ifdef CONFIG_SMP -/* - * Approximate: - * val * y^n, where y^32 ~= 0.5 (~1 scheduling period) - */ -static u64 decay_load(u64 val, u64 n) -{ - unsigned int local_n; - - if (unlikely(n > LOAD_AVG_PERIOD * 63)) - return 0; - - /* after bounds checking we can collapse to 32-bit */ - local_n = n; - - /* - * As y^PERIOD = 1/2, we can combine - * y^n = 1/2^(n/PERIOD) * y^(n%PERIOD) - * With a look-up table which covers y^n (n= LOAD_AVG_PERIOD)) { - val >>= local_n / LOAD_AVG_PERIOD; - local_n %= LOAD_AVG_PERIOD; - } - - val = mul_u64_u32_shr(val, runnable_avg_yN_inv[local_n], 32); - return val; -} - -static u32 __accumulate_pelt_segments(u64 periods, u32 d1, u32 d3) -{ - u32 c1, c2, c3 = d3; /* y^0 == 1 */ - - /* - * c1 = d1 y^p - */ - c1 = decay_load((u64)d1, periods); - - /* - * p-1 - * c2 = 1024 \Sum y^n - * n=1 - * - * inf inf - * = 1024 ( \Sum y^n - \Sum y^n - y^0 ) - * n=0 n=p - */ - c2 = LOAD_AVG_MAX - decay_load(LOAD_AVG_MAX, periods) - 1024; - - return c1 + c2 + c3; -} - -#define cap_scale(v, s) ((v)*(s) >> SCHED_CAPACITY_SHIFT) - -/* - * Accumulate the three separate parts of the sum; d1 the remainder - * of the last (incomplete) period, d2 the span of full periods and d3 - * the remainder of the (incomplete) current period. - * - * d1 d2 d3 - * ^ ^ ^ - * | | | - * |<->|<----------------->|<--->| - * ... |---x---|------| ... |------|-----x (now) - * - * p-1 - * u' = (u + d1) y^p + 1024 \Sum y^n + d3 y^0 - * n=1 - * - * = u y^p + (Step 1) - * - * p-1 - * d1 y^p + 1024 \Sum y^n + d3 y^0 (Step 2) - * n=1 - */ -static __always_inline u32 -accumulate_sum(u64 delta, int cpu, struct sched_avg *sa, - unsigned long load, unsigned long runnable, int running) -{ - unsigned long scale_freq, scale_cpu; - u32 contrib = (u32)delta; /* p == 0 -> delta < 1024 */ - u64 periods; - - scale_freq = arch_scale_freq_capacity(NULL, cpu); - scale_cpu = arch_scale_cpu_capacity(NULL, cpu); - - delta += sa->period_contrib; - periods = delta / 1024; /* A period is 1024us (~1ms) */ - - /* - * Step 1: decay old *_sum if we crossed period boundaries. - */ - if (periods) { - sa->load_sum = decay_load(sa->load_sum, periods); - sa->runnable_load_sum = - decay_load(sa->runnable_load_sum, periods); - sa->util_sum = decay_load((u64)(sa->util_sum), periods); - - /* - * Step 2 - */ - delta %= 1024; - contrib = __accumulate_pelt_segments(periods, - 1024 - sa->period_contrib, delta); - } - sa->period_contrib = delta; - - contrib = cap_scale(contrib, scale_freq); - if (load) - sa->load_sum += load * contrib; - if (runnable) - sa->runnable_load_sum += runnable * contrib; - if (running) - sa->util_sum += contrib * scale_cpu; - - return periods; -} - -/* - * We can represent the historical contribution to runnable average as the - * coefficients of a geometric series. To do this we sub-divide our runnable - * history into segments of approximately 1ms (1024us); label the segment that - * occurred N-ms ago p_N, with p_0 corresponding to the current period, e.g. - * - * [<- 1024us ->|<- 1024us ->|<- 1024us ->| ... - * p0 p1 p2 - * (now) (~1ms ago) (~2ms ago) - * - * Let u_i denote the fraction of p_i that the entity was runnable. - * - * We then designate the fractions u_i as our co-efficients, yielding the - * following representation of historical load: - * u_0 + u_1*y + u_2*y^2 + u_3*y^3 + ... - * - * We choose y based on the with of a reasonably scheduling period, fixing: - * y^32 = 0.5 - * - * This means that the contribution to load ~32ms ago (u_32) will be weighted - * approximately half as much as the contribution to load within the last ms - * (u_0). - * - * When a period "rolls over" and we have new u_0`, multiplying the previous - * sum again by y is sufficient to update: - * load_avg = u_0` + y*(u_0 + u_1*y + u_2*y^2 + ... ) - * = u_0 + u_1*y + u_2*y^2 + ... [re-labeling u_i --> u_{i+1}] - */ -static __always_inline int -___update_load_sum(u64 now, int cpu, struct sched_avg *sa, - unsigned long load, unsigned long runnable, int running) -{ - u64 delta; - - delta = now - sa->last_update_time; - /* - * This should only happen when time goes backwards, which it - * unfortunately does during sched clock init when we swap over to TSC. - */ - if ((s64)delta < 0) { - sa->last_update_time = now; - return 0; - } - - /* - * Use 1024ns as the unit of measurement since it's a reasonable - * approximation of 1us and fast to compute. - */ - delta >>= 10; - if (!delta) - return 0; - - sa->last_update_time += delta << 10; - - /* - * running is a subset of runnable (weight) so running can't be set if - * runnable is clear. But there are some corner cases where the current - * se has been already dequeued but cfs_rq->curr still points to it. - * This means that weight will be 0 but not running for a sched_entity - * but also for a cfs_rq if the latter becomes idle. As an example, - * this happens during idle_balance() which calls - * update_blocked_averages() - */ - if (!load) - runnable = running = 0; - - /* - * Now we know we crossed measurement unit boundaries. The *_avg - * accrues by two steps: - * - * Step 1: accumulate *_sum since last_update_time. If we haven't - * crossed period boundaries, finish. - */ - if (!accumulate_sum(delta, cpu, sa, load, runnable, running)) - return 0; - - return 1; -} - -static __always_inline void -___update_load_avg(struct sched_avg *sa, unsigned long load, unsigned long runnable) -{ - u32 divider = LOAD_AVG_MAX - 1024 + sa->period_contrib; - - /* - * Step 2: update *_avg. - */ - sa->load_avg = div_u64(load * sa->load_sum, divider); - sa->runnable_load_avg = div_u64(runnable * sa->runnable_load_sum, divider); - sa->util_avg = sa->util_sum / divider; -} - -/* - * sched_entity: - * - * task: - * se_runnable() == se_weight() - * - * group: [ see update_cfs_group() ] - * se_weight() = tg->weight * grq->load_avg / tg->load_avg - * se_runnable() = se_weight(se) * grq->runnable_load_avg / grq->load_avg - * - * load_sum := runnable_sum - * load_avg = se_weight(se) * runnable_avg - * - * runnable_load_sum := runnable_sum - * runnable_load_avg = se_runnable(se) * runnable_avg - * - * XXX collapse load_sum and runnable_load_sum - * - * cfq_rs: - * - * load_sum = \Sum se_weight(se) * se->avg.load_sum - * load_avg = \Sum se->avg.load_avg - * - * runnable_load_sum = \Sum se_runnable(se) * se->avg.runnable_load_sum - * runnable_load_avg = \Sum se->avg.runable_load_avg - */ - -static int -__update_load_avg_blocked_se(u64 now, int cpu, struct sched_entity *se) -{ - if (entity_is_task(se)) - se->runnable_weight = se->load.weight; - - if (___update_load_sum(now, cpu, &se->avg, 0, 0, 0)) { - ___update_load_avg(&se->avg, se_weight(se), se_runnable(se)); - return 1; - } - - return 0; -} - -static int -__update_load_avg_se(u64 now, int cpu, struct cfs_rq *cfs_rq, struct sched_entity *se) -{ - if (entity_is_task(se)) - se->runnable_weight = se->load.weight; - - if (___update_load_sum(now, cpu, &se->avg, !!se->on_rq, !!se->on_rq, - cfs_rq->curr == se)) { - - ___update_load_avg(&se->avg, se_weight(se), se_runnable(se)); - return 1; - } - - return 0; -} - -static int -__update_load_avg_cfs_rq(u64 now, int cpu, struct cfs_rq *cfs_rq) -{ - if (___update_load_sum(now, cpu, &cfs_rq->avg, - scale_load_down(cfs_rq->load.weight), - scale_load_down(cfs_rq->runnable_weight), - cfs_rq->curr != NULL)) { - - ___update_load_avg(&cfs_rq->avg, 1, 1); - return 1; - } - - return 0; -} - #ifdef CONFIG_FAIR_GROUP_SCHED /** * update_tg_load_avg - update the tg's load avg @@ -3875,12 +3575,6 @@ static int idle_balance(struct rq *this_rq, struct rq_flags *rf); #else /* CONFIG_SMP */ -static inline int -update_cfs_rq_load_avg(u64 now, struct cfs_rq *cfs_rq) -{ - return 0; -} - #define UPDATE_TG 0x0 #define SKIP_AGE_LOAD 0x0 #define DO_ATTACH 0x0 diff --git a/kernel/sched/pelt.c b/kernel/sched/pelt.c new file mode 100644 index 0000000..da6d84f --- /dev/null +++ b/kernel/sched/pelt.c @@ -0,0 +1,308 @@ +/* + * Per Entity Load Tracking + * + * Copyright (C) 2007 Red Hat, Inc., Ingo Molnar + * + * Interactivity improvements by Mike Galbraith + * (C) 2007 Mike Galbraith + * + * Various enhancements by Dmitry Adamushko. + * (C) 2007 Dmitry Adamushko + * + * Group scheduling enhancements by Srivatsa Vaddagiri + * Copyright IBM Corporation, 2007 + * Author: Srivatsa Vaddagiri + * + * Scaled math optimizations by Thomas Gleixner + * Copyright (C) 2007, Thomas Gleixner + * + * Adaptive scheduling granularity, math enhancements by Peter Zijlstra + * Copyright (C) 2007 Red Hat, Inc., Peter Zijlstra + * + * Move PELT related code from fair.c into this pelt.c file + * Author: Vincent Guittot + */ + +#include +#include "sched.h" +#include "sched-pelt.h" + +/* + * Approximate: + * val * y^n, where y^32 ~= 0.5 (~1 scheduling period) + */ +static u64 decay_load(u64 val, u64 n) +{ + unsigned int local_n; + + if (unlikely(n > LOAD_AVG_PERIOD * 63)) + return 0; + + /* after bounds checking we can collapse to 32-bit */ + local_n = n; + + /* + * As y^PERIOD = 1/2, we can combine + * y^n = 1/2^(n/PERIOD) * y^(n%PERIOD) + * With a look-up table which covers y^n (n= LOAD_AVG_PERIOD)) { + val >>= local_n / LOAD_AVG_PERIOD; + local_n %= LOAD_AVG_PERIOD; + } + + val = mul_u64_u32_shr(val, runnable_avg_yN_inv[local_n], 32); + return val; +} + +static u32 __accumulate_pelt_segments(u64 periods, u32 d1, u32 d3) +{ + u32 c1, c2, c3 = d3; /* y^0 == 1 */ + + /* + * c1 = d1 y^p + */ + c1 = decay_load((u64)d1, periods); + + /* + * p-1 + * c2 = 1024 \Sum y^n + * n=1 + * + * inf inf + * = 1024 ( \Sum y^n - \Sum y^n - y^0 ) + * n=0 n=p + */ + c2 = LOAD_AVG_MAX - decay_load(LOAD_AVG_MAX, periods) - 1024; + + return c1 + c2 + c3; +} + +#define cap_scale(v, s) ((v)*(s) >> SCHED_CAPACITY_SHIFT) + +/* + * Accumulate the three separate parts of the sum; d1 the remainder + * of the last (incomplete) period, d2 the span of full periods and d3 + * the remainder of the (incomplete) current period. + * + * d1 d2 d3 + * ^ ^ ^ + * | | | + * |<->|<----------------->|<--->| + * ... |---x---|------| ... |------|-----x (now) + * + * p-1 + * u' = (u + d1) y^p + 1024 \Sum y^n + d3 y^0 + * n=1 + * + * = u y^p + (Step 1) + * + * p-1 + * d1 y^p + 1024 \Sum y^n + d3 y^0 (Step 2) + * n=1 + */ +static __always_inline u32 +accumulate_sum(u64 delta, int cpu, struct sched_avg *sa, + unsigned long load, unsigned long runnable, int running) +{ + unsigned long scale_freq, scale_cpu; + u32 contrib = (u32)delta; /* p == 0 -> delta < 1024 */ + u64 periods; + + scale_freq = arch_scale_freq_capacity(NULL, cpu); + scale_cpu = arch_scale_cpu_capacity(NULL, cpu); + + delta += sa->period_contrib; + periods = delta / 1024; /* A period is 1024us (~1ms) */ + + /* + * Step 1: decay old *_sum if we crossed period boundaries. + */ + if (periods) { + sa->load_sum = decay_load(sa->load_sum, periods); + sa->runnable_load_sum = + decay_load(sa->runnable_load_sum, periods); + sa->util_sum = decay_load((u64)(sa->util_sum), periods); + + /* + * Step 2 + */ + delta %= 1024; + contrib = __accumulate_pelt_segments(periods, + 1024 - sa->period_contrib, delta); + } + sa->period_contrib = delta; + + contrib = cap_scale(contrib, scale_freq); + if (load) + sa->load_sum += load * contrib; + if (runnable) + sa->runnable_load_sum += runnable * contrib; + if (running) + sa->util_sum += contrib * scale_cpu; + + return periods; +} + +/* + * We can represent the historical contribution to runnable average as the + * coefficients of a geometric series. To do this we sub-divide our runnable + * history into segments of approximately 1ms (1024us); label the segment that + * occurred N-ms ago p_N, with p_0 corresponding to the current period, e.g. + * + * [<- 1024us ->|<- 1024us ->|<- 1024us ->| ... + * p0 p1 p2 + * (now) (~1ms ago) (~2ms ago) + * + * Let u_i denote the fraction of p_i that the entity was runnable. + * + * We then designate the fractions u_i as our co-efficients, yielding the + * following representation of historical load: + * u_0 + u_1*y + u_2*y^2 + u_3*y^3 + ... + * + * We choose y based on the with of a reasonably scheduling period, fixing: + * y^32 = 0.5 + * + * This means that the contribution to load ~32ms ago (u_32) will be weighted + * approximately half as much as the contribution to load within the last ms + * (u_0). + * + * When a period "rolls over" and we have new u_0`, multiplying the previous + * sum again by y is sufficient to update: + * load_avg = u_0` + y*(u_0 + u_1*y + u_2*y^2 + ... ) + * = u_0 + u_1*y + u_2*y^2 + ... [re-labeling u_i --> u_{i+1}] + */ +static __always_inline int +___update_load_sum(u64 now, int cpu, struct sched_avg *sa, + unsigned long load, unsigned long runnable, int running) +{ + u64 delta; + + delta = now - sa->last_update_time; + /* + * This should only happen when time goes backwards, which it + * unfortunately does during sched clock init when we swap over to TSC. + */ + if ((s64)delta < 0) { + sa->last_update_time = now; + return 0; + } + + /* + * Use 1024ns as the unit of measurement since it's a reasonable + * approximation of 1us and fast to compute. + */ + delta >>= 10; + if (!delta) + return 0; + + sa->last_update_time += delta << 10; + + /* + * running is a subset of runnable (weight) so running can't be set if + * runnable is clear. But there are some corner cases where the current + * se has been already dequeued but cfs_rq->curr still points to it. + * This means that weight will be 0 but not running for a sched_entity + * but also for a cfs_rq if the latter becomes idle. As an example, + * this happens during idle_balance() which calls + * update_blocked_averages() + */ + if (!load) + runnable = running = 0; + + /* + * Now we know we crossed measurement unit boundaries. The *_avg + * accrues by two steps: + * + * Step 1: accumulate *_sum since last_update_time. If we haven't + * crossed period boundaries, finish. + */ + if (!accumulate_sum(delta, cpu, sa, load, runnable, running)) + return 0; + + return 1; +} + +static __always_inline void +___update_load_avg(struct sched_avg *sa, unsigned long load, unsigned long runnable) +{ + u32 divider = LOAD_AVG_MAX - 1024 + sa->period_contrib; + + /* + * Step 2: update *_avg. + */ + sa->load_avg = div_u64(load * sa->load_sum, divider); + sa->runnable_load_avg = div_u64(runnable * sa->runnable_load_sum, divider); + sa->util_avg = sa->util_sum / divider; +} + +/* + * sched_entity: + * + * task: + * se_runnable() == se_weight() + * + * group: [ see update_cfs_group() ] + * se_weight() = tg->weight * grq->load_avg / tg->load_avg + * se_runnable() = se_weight(se) * grq->runnable_load_avg / grq->load_avg + * + * load_sum := runnable_sum + * load_avg = se_weight(se) * runnable_avg + * + * runnable_load_sum := runnable_sum + * runnable_load_avg = se_runnable(se) * runnable_avg + * + * XXX collapse load_sum and runnable_load_sum + * + * cfq_rs: + * + * load_sum = \Sum se_weight(se) * se->avg.load_sum + * load_avg = \Sum se->avg.load_avg + * + * runnable_load_sum = \Sum se_runnable(se) * se->avg.runnable_load_sum + * runnable_load_avg = \Sum se->avg.runable_load_avg + */ + +int __update_load_avg_blocked_se(u64 now, int cpu, struct sched_entity *se) +{ + if (entity_is_task(se)) + se->runnable_weight = se->load.weight; + + if (___update_load_sum(now, cpu, &se->avg, 0, 0, 0)) { + ___update_load_avg(&se->avg, se_weight(se), se_runnable(se)); + return 1; + } + + return 0; +} + +int __update_load_avg_se(u64 now, int cpu, struct cfs_rq *cfs_rq, struct sched_entity *se) +{ + if (entity_is_task(se)) + se->runnable_weight = se->load.weight; + + if (___update_load_sum(now, cpu, &se->avg, !!se->on_rq, !!se->on_rq, + cfs_rq->curr == se)) { + + ___update_load_avg(&se->avg, se_weight(se), se_runnable(se)); + return 1; + } + + return 0; +} + +int __update_load_avg_cfs_rq(u64 now, int cpu, struct cfs_rq *cfs_rq) +{ + if (___update_load_sum(now, cpu, &cfs_rq->avg, + scale_load_down(cfs_rq->load.weight), + scale_load_down(cfs_rq->runnable_weight), + cfs_rq->curr != NULL)) { + + ___update_load_avg(&cfs_rq->avg, 1, 1); + return 1; + } + + return 0; +} diff --git a/kernel/sched/pelt.h b/kernel/sched/pelt.h new file mode 100644 index 0000000..c312d8c --- /dev/null +++ b/kernel/sched/pelt.h @@ -0,0 +1,17 @@ +#ifdef CONFIG_SMP + +int __update_load_avg_blocked_se(u64 now, int cpu, struct sched_entity *se); +int __update_load_avg_se(u64 now, int cpu, struct cfs_rq *cfs_rq, struct sched_entity *se); +int __update_load_avg_cfs_rq(u64 now, int cpu, struct cfs_rq *cfs_rq); + +#else + +static inline int +update_cfs_rq_load_avg(u64 now, struct cfs_rq *cfs_rq) +{ + return 0; +} + +#endif + + diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index b19552a2..9c56e73 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -528,6 +528,7 @@ struct rt_rq { unsigned long rt_nr_total; int overloaded; struct plist_head pushable_tasks; + #endif /* CONFIG_SMP */ int rt_queued; @@ -602,7 +603,26 @@ struct dl_rq { u64 bw_ratio; }; +#ifdef CONFIG_FAIR_GROUP_SCHED +/* An entity is a task if it doesn't "own" a runqueue */ +#define entity_is_task(se) (!se->my_q) +#else +#define entity_is_task(se) 1 +#endif + #ifdef CONFIG_SMP +/* + * XXX we want to get rid of these helpers and use the full load resolution. + */ +static inline long se_weight(struct sched_entity *se) +{ + return scale_load_down(se->load.weight); +} + +static inline long se_runnable(struct sched_entity *se) +{ + return scale_load_down(se->runnable_weight); +} static inline bool sched_asym_prefer(int a, int b) { From patchwork Tue Jan 9 13:53:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 123962 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp4047830qgn; Tue, 9 Jan 2018 05:54:11 -0800 (PST) X-Google-Smtp-Source: ACJfBotYpxxN3dFvlDM0XyLWDDEWColQ+lBBmuUgPp+5OJoA1xcA/GhS+ZtzDTcm0ktk+Qh9p8A2 X-Received: by 10.84.128.74 with SMTP id 68mr5897035pla.228.1515506051236; Tue, 09 Jan 2018 05:54:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515506051; cv=none; d=google.com; s=arc-20160816; b=sy0I4eBad8dmghHQWoFWh2KzsD1ikmCPZxnmzKBFJvNKs/2QRxIUVqODqn8gZRTNND taTKSNJEFR1+WsAi2Bh8BHNKgBqjUInjWeSzYL9MrKlI6WGr2xkXQiPQ2yHg9jmpAB/1 Hk8R0kXlfcTA+Z5eX+R7AwuJu98LNLCsC8CiJMVglzu0DdZlyNo5TruOBQ4GQYButnA6 9n5FeX482qotCQSWnYiGMy+0qkfQHOygM6TuAflgJj+cbkLv7STChlo46tImcm6bCfdp +x+R1X2NWLW0kQHry/gxWNMtj1aV9p5LMVVxaKbJq/EqG8D+ax/K8NYqKMl9++nbLyq3 0hQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=5+s+8sTwMQVSPlVy9iMynL2IG5jJA2tkCOxOzC9yL9g=; b=FGVpUpk/SfUH+PJiuMkOOuEYc47ys8ItPunVUCViK5ZbviO4WFhkpufypSAYqN89qX Cen0ShN62yiM9wkUwc82Ro2I2fji6hB4Zd1bKwtcx/rc9NPPbu1Vuf8k3VR3tR+cRUny 8kUWH3VvfVRDcdkMDBnDtnW7GLmJu6OlvzoVrWcbSiZcgAFV/W+mfhQm+XdUf8u42rys zRpSo1pypgnIaJa9+yPooQ7129gH59mhOOE6DhldlQB1UBuuY+PwiA3qg4AehG/qkUwY /BU3P3jFxQkmSCgRlVFDwbxw0ipWoTojVMMY1TSPbG3BdMKdy/ps/NhFYkHJvycNNxas XsAg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=K49/QOHM; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u5si9042745pgp.237.2018.01.09.05.54.10; Tue, 09 Jan 2018 05:54:11 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=K49/QOHM; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758205AbeAINyJ (ORCPT + 28 others); Tue, 9 Jan 2018 08:54:09 -0500 Received: from mail-wr0-f195.google.com ([209.85.128.195]:39615 "EHLO mail-wr0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750786AbeAINxT (ORCPT ); Tue, 9 Jan 2018 08:53:19 -0500 Received: by mail-wr0-f195.google.com with SMTP id z48so8461466wrz.6 for ; Tue, 09 Jan 2018 05:53:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=5+s+8sTwMQVSPlVy9iMynL2IG5jJA2tkCOxOzC9yL9g=; b=K49/QOHMnSlLkgCdPerRMab88tKqFGKyM6qW2d0KOwQYB5+4y7V7yRKB0nZ++azJhW sBD8kQaVf6D9MgfYbXxaVZT5EtU0yZmlIHfqDssX3OvxssSt92WmR73XLhSbIYtzqskj XFe9dVh7YGpdZw1fcMlMLTstA7fO6m+2aO3ms= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5+s+8sTwMQVSPlVy9iMynL2IG5jJA2tkCOxOzC9yL9g=; b=DCmAUc3pMwvdPB4TiPPwGBgtrTDdHW/ZGHmueff9FnuIhxCP5+Yba6EFRdItJCsT7S t5KBKG+QGfUB9sJGT/+1EYe6NqpKvK8w/QE+XdUTZKl2okpJdvG4BqDnyOpK2Y9810fr 0X9g7t53aHrMmMfg5oHJPfj2dNKjLRTlSzLbRIBadyAhx7Bx6TC2kv2u0cc6f+3Jyj7V EYrZqSUpdCa3xruzXeGCXudyU7C3eOVGCfh1F/KAAtEPelfnX7XvGKSO2fvnKfJZQ4WX ARpWlB/vKgJJPedCmzANIbjL3NEZNHmSb8joCJ3XiyWbfgLq6V6WvZpjDa6uTE1+gSWM iWlg== X-Gm-Message-State: AKGB3mLmz8iYA6+AwBz8Y3TafNk1lOYV5AoY9v6SxCOcQHHDKeVDWwTh HEm/tJ1EjWjK92olZqMRLss/qw== X-Received: by 10.223.195.197 with SMTP id d5mr13856425wrg.86.1515505998148; Tue, 09 Jan 2018 05:53:18 -0800 (PST) Received: from localhost.localdomain ([2a01:e0a:f:6020:f51d:768f:aab1:18e6]) by smtp.gmail.com with ESMTPSA id e7sm15070597wrd.82.2018.01.09.05.53.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 09 Jan 2018 05:53:16 -0800 (PST) From: Vincent Guittot To: peterz@infradead.org, mingo@kernel.org, linux-kernel@vger.kernel.org, rjw@rjwysocki.net Cc: juri.lelli@arm.com, dietmar.eggemann@arm.com, Morten.Rasmussen@arm.com, viresh.kumar@linaro.org, Vincent Guittot Subject: [PATCH RESEND v3 2/3] sched/rt: add rt_rq utilization tracking Date: Tue, 9 Jan 2018 14:53:05 +0100 Message-Id: <1515505986-21997-3-git-send-email-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1515505986-21997-1-git-send-email-vincent.guittot@linaro.org> References: <1515505986-21997-1-git-send-email-vincent.guittot@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org schedutil governor relies on cfs_rq's util_avg to choose the OPP when cfs tasks are running. When the CPU is overloaded by cfs and rt tasks, cfs tasks are preempted by rt tasks and in this case util_avg reflects the remaining capacity that is used by cfs task but not what cfs want to use. In such case, schedutil can select a lower OPP whereas the CPU is overloaded. In order to have a more accurate view of the utilization of the CPU, we track the utilization that is "stolen" by RT tasks. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 2 ++ kernel/sched/pelt.c | 23 +++++++++++++++++++++++ kernel/sched/pelt.h | 7 +++++++ kernel/sched/rt.c | 8 ++++++++ kernel/sched/sched.h | 1 + 5 files changed, 41 insertions(+) -- 2.7.4 diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 1abcf6e..6844fb7 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7072,6 +7072,7 @@ static void update_blocked_averages(int cpu) if (cfs_rq_is_decayed(cfs_rq)) list_del_leaf_cfs_rq(cfs_rq); } + update_rt_rq_load_avg(rq_clock_task(rq), cpu, &rq->rt, 0); rq_unlock_irqrestore(rq, &rf); } @@ -7131,6 +7132,7 @@ static inline void update_blocked_averages(int cpu) rq_lock_irqsave(rq, &rf); update_rq_clock(rq); update_cfs_rq_load_avg(cfs_rq_clock_task(cfs_rq), cfs_rq); + update_rt_rq_load_avg(rq_clock_task(rq), cpu, &rq->rt, 0); rq_unlock_irqrestore(rq, &rf); } diff --git a/kernel/sched/pelt.c b/kernel/sched/pelt.c index da6d84f..ac09b69 100644 --- a/kernel/sched/pelt.c +++ b/kernel/sched/pelt.c @@ -306,3 +306,26 @@ int __update_load_avg_cfs_rq(u64 now, int cpu, struct cfs_rq *cfs_rq) return 0; } + +/* + * rt_rq: + * + * util_sum = \Sum se->avg.util_sum but se->avg.util_sum is not tracked + * util_sum = cpu_scale * load_sum + * runnable_load_sum = load_sum + * + */ + +int update_rt_rq_load_avg(u64 now, int cpu, struct rt_rq *rt_rq, int running) +{ + if (___update_load_sum(now, cpu, &rt_rq->avg, + running, + running, + running)) { + + ___update_load_avg(&rt_rq->avg, 1, 1); + return 1; + } + + return 0; +} diff --git a/kernel/sched/pelt.h b/kernel/sched/pelt.h index c312d8c..78a2107 100644 --- a/kernel/sched/pelt.h +++ b/kernel/sched/pelt.h @@ -3,6 +3,7 @@ int __update_load_avg_blocked_se(u64 now, int cpu, struct sched_entity *se); int __update_load_avg_se(u64 now, int cpu, struct cfs_rq *cfs_rq, struct sched_entity *se); int __update_load_avg_cfs_rq(u64 now, int cpu, struct cfs_rq *cfs_rq); +int update_rt_rq_load_avg(u64 now, int cpu, struct rt_rq *rt_rq, int running); #else @@ -12,6 +13,12 @@ update_cfs_rq_load_avg(u64 now, struct cfs_rq *cfs_rq) return 0; } +static inline int +update_rt_rq_load_avg(u64 now, int cpu, struct rt_rq *rt_rq, int running) +{ + return 0; +} + #endif diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 4056c19..09c2616 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -9,6 +9,8 @@ #include #include +#include "pelt.h" + int sched_rr_timeslice = RR_TIMESLICE; int sysctl_sched_rr_timeslice = (MSEC_PER_SEC / HZ) * RR_TIMESLICE; @@ -1569,6 +1571,9 @@ pick_next_task_rt(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) queue_push_tasks(rq); + update_rt_rq_load_avg(rq_clock_task(rq), cpu_of(rq), rt_rq, + rq->curr->sched_class == &rt_sched_class); + return p; } @@ -1576,6 +1581,8 @@ static void put_prev_task_rt(struct rq *rq, struct task_struct *p) { update_curr_rt(rq); + update_rt_rq_load_avg(rq_clock_task(rq), cpu_of(rq), &rq->rt, 1); + /* * The previous task needs to be made eligible for pushing * if it is still active @@ -2283,6 +2290,7 @@ static void task_tick_rt(struct rq *rq, struct task_struct *p, int queued) struct sched_rt_entity *rt_se = &p->rt; update_curr_rt(rq); + update_rt_rq_load_avg(rq_clock_task(rq), cpu_of(rq), &rq->rt, 1); watchdog(rq, p); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 9c56e73..7eaa103 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -529,6 +529,7 @@ struct rt_rq { int overloaded; struct plist_head pushable_tasks; + struct sched_avg avg; #endif /* CONFIG_SMP */ int rt_queued; From patchwork Tue Jan 9 13:53:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 123961 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp4047474qgn; Tue, 9 Jan 2018 05:53:48 -0800 (PST) X-Google-Smtp-Source: ACJfBosXJyvFftKX8imPAi8aVKwEgS6XIg4l5CpkKYMqodVhh19zZqrmuGB3fxyfEvT6nnIDHKPq X-Received: by 10.99.125.70 with SMTP id m6mr12260608pgn.415.1515506028867; Tue, 09 Jan 2018 05:53:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515506028; cv=none; d=google.com; s=arc-20160816; b=nnLIbVOt0GD0CxORu7EHgpvz6SUZaGKEKdiyqKUETHxAxgjVQ+5DFgfqxlVoByhjRs jZkwpoOXmYhyEmgizqKzJAxjW57t0Ak55Oq8Hlb3f64t+GXXt9vsljtFtpw5vDhVWUUH LWOiGGXXFyjiy5XuqRbV74fC6HOkoe26iF0GPa+pQOtBfahrS1xrL23WbW7thYfvLvwo R/ZfxHW3c4yzeRXpY0ehfyTulL8cPF45urC8z+ryXa1/dOmYgr1cZ3i6ke3A4OCnWgKU yUIvhTCFVdnKAaAMs02Ii1JkQAZvv7CMr3a53Cbawq+SJ5F+S4rkQjez9jUWpxTV/g4r o7xw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=2OVBF5+dc/kVtiUqLAsVzLBkf0TykvdDQ/YCTgDvEGs=; b=0tdyfFonoKiXJ7n+x98c0E3KS6UM+a0bIspZHr9cMxjLoIn2dSawMKVBzkGb5HSCwf l3IXTfPHVewr+/q2HQGrz29B+cCmDdcgGYHSfuTbDa/17eY7KNhGG1lJtmeF8ckkV15A NQ+dEbM6ChFdb7pGtJJwZZNvj0T9TQsGOpQS2Ldau1nwx0k8ZAL7IjdX3UxGfqWVRBQw 6vuVqOwDETU1pcJXc4pv1fW4uSzvvhGB5qRYHVE6/9hCNJIXRbe6uhYysFCo5Gn2zcmZ z2fRB8K1ZaJbNx4a9Y2e81JKgVbgytrNjKA439Zeg5cBMQGWDeejjCuzmSVZ6FAdtgLi lyRA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Fj1zWBqk; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q77si5335608pfq.400.2018.01.09.05.53.48; Tue, 09 Jan 2018 05:53:48 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Fj1zWBqk; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758154AbeAINxq (ORCPT + 28 others); Tue, 9 Jan 2018 08:53:46 -0500 Received: from mail-wm0-f68.google.com ([74.125.82.68]:46795 "EHLO mail-wm0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758091AbeAINxU (ORCPT ); Tue, 9 Jan 2018 08:53:20 -0500 Received: by mail-wm0-f68.google.com with SMTP id r78so20649934wme.5 for ; Tue, 09 Jan 2018 05:53:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=2OVBF5+dc/kVtiUqLAsVzLBkf0TykvdDQ/YCTgDvEGs=; b=Fj1zWBqkjL7rkjwq7LbWguAJbobcSwlLAQc16KeRJH37aA/KcBu6HEEury01WvCmV1 ML5vARGQ+7p1aE/BhCSFVMuuO0KtGyZY2odpjYfJ9q6TDtUZcLMeDMuHc6mlWEHmx3zd iUYsoYp1clzwDq4a8QUHU/yCW4AXJd8fg9sdg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=2OVBF5+dc/kVtiUqLAsVzLBkf0TykvdDQ/YCTgDvEGs=; b=bDwONSbdFS+YRNIHgI5gOdT7FRN+oow8mY9XqSTpdF27Mz7c9s+YLF/PTlaOt7Nrg4 XWsOpxnfc9m4nSjtT0pExt7WaSk0QyLek0v1isv2j5ah/dJ5/ckvvdq83sDB+4S7xLwx YBrPukf51AVe2tJDhZ2Hu1YtTQIW4DileCmrLUmhP1q/EOoOjxQ9hInKvnLe2uT1HzXb NJRTcYAysgRwtPVGyGaoE3M+PIg+y6fMIph9LSi4wSSPvM2PtrvY8OHQRI7RPhZyoxk4 0bvjdairg8zDsYBuk0rIWRVjXZyjh3TkbMqdFZOT9qg42oWUVYD1PEuTaT1X1aqbCAkH JO4w== X-Gm-Message-State: AKGB3mIyNOrloMxupXV3fkERSlQcjXZdIQqpdKKQ5txfDJTaAjBNW/i7 CC04HcgGxBlXvK2WrvyOoWVxNA== X-Received: by 10.28.218.207 with SMTP id r198mr11676098wmg.37.1515505999344; Tue, 09 Jan 2018 05:53:19 -0800 (PST) Received: from localhost.localdomain ([2a01:e0a:f:6020:f51d:768f:aab1:18e6]) by smtp.gmail.com with ESMTPSA id e7sm15070597wrd.82.2018.01.09.05.53.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 09 Jan 2018 05:53:18 -0800 (PST) From: Vincent Guittot To: peterz@infradead.org, mingo@kernel.org, linux-kernel@vger.kernel.org, rjw@rjwysocki.net Cc: juri.lelli@arm.com, dietmar.eggemann@arm.com, Morten.Rasmussen@arm.com, viresh.kumar@linaro.org, Vincent Guittot Subject: [PATCH RESEND v3 3/3] cpufreq/schedutil: add rt utilization tracking Date: Tue, 9 Jan 2018 14:53:06 +0100 Message-Id: <1515505986-21997-4-git-send-email-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1515505986-21997-1-git-send-email-vincent.guittot@linaro.org> References: <1515505986-21997-1-git-send-email-vincent.guittot@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org add both cfs and rt utilization when selecting an OPP as rt can preempt and steal cfs's running time Signed-off-by: Vincent Guittot --- kernel/sched/cpufreq_schedutil.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.7.4 diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index 2f52ec0..d8ba31b 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -183,7 +183,7 @@ static void sugov_get_util(unsigned long *util, unsigned long *max, int cpu) cfs_max = arch_scale_cpu_capacity(NULL, cpu); - *util = min(rq->cfs.avg.util_avg, cfs_max); + *util = min(rq->cfs.avg.util_avg + rq->rt.avg.util_avg, cfs_max); *max = cfs_max; }