From patchwork Fri Oct 31 08:47:31 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 39868 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-lb0-f200.google.com (mail-lb0-f200.google.com [209.85.217.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id D36BB202FE for ; Fri, 31 Oct 2014 08:49:17 +0000 (UTC) Received: by mail-lb0-f200.google.com with SMTP id f15sf3892244lbj.7 for ; Fri, 31 Oct 2014 01:49:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:sender:precedence:list-id :x-original-sender:x-original-authentication-results:mailing-list :list-post:list-help:list-archive:list-unsubscribe; bh=NvISRHKQgal9oa7SRu6k21a7/qFEA4hzEqrun6dMTYQ=; b=jsIVvvI6VNh3l4Ip1ifZSiKWugPpXtwW0vMll3MYwKePa4Gp8Ia5eSn+sA3TKlYqxs 8mZ4GGBhIpa2tTCh+2s+H0ONMeNX7FVD2D9Z2kNJxq9i/OEH8mBhbiz3nILk5AvyaJ+I tUEfO9/+k2lcSi1AKXFotQUYCy1d5WfXoo82TPxliPPpfVl16PgHZlQ4+6VQID9noICr pxk+RGQlYQFgFA1fb3v2NmneBFtAlOmvt+bBy8phuwDfOz8Jpi0ZIw2CN3GvK1GnNedV /ePjh4rIl9N3wFdoFDfh8jjz12gJImGL7NeeOdtc+vxws59BTXMx5CdEQa24rK27Vtke NUZg== X-Gm-Message-State: ALoCoQmAfAj3P3C82gJ+JCvYEOlYZXtNxq6k9gAtZQrufo9SWronkVekqn2dbXoym8oBbRl/r1mF X-Received: by 10.152.42.171 with SMTP id p11mr3751035lal.4.1414745356645; Fri, 31 Oct 2014 01:49:16 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.7.66 with SMTP id h2ls358279laa.58.gmail; Fri, 31 Oct 2014 01:49:16 -0700 (PDT) X-Received: by 10.152.21.226 with SMTP id y2mr24714500lae.70.1414745356347; Fri, 31 Oct 2014 01:49:16 -0700 (PDT) Received: from mail-lb0-f174.google.com (mail-lb0-f174.google.com. [209.85.217.174]) by mx.google.com with ESMTPS id y2si15633477lae.49.2014.10.31.01.49.16 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 31 Oct 2014 01:49:16 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.174 as permitted sender) client-ip=209.85.217.174; Received: by mail-lb0-f174.google.com with SMTP id z11so3637767lbi.5 for ; Fri, 31 Oct 2014 01:49:16 -0700 (PDT) X-Received: by 10.112.14.69 with SMTP id n5mr24518287lbc.34.1414745356258; Fri, 31 Oct 2014 01:49:16 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.84.229 with SMTP id c5csp168149lbz; Fri, 31 Oct 2014 01:49:15 -0700 (PDT) X-Received: by 10.68.175.196 with SMTP id cc4mr22769254pbc.147.1414745354448; Fri, 31 Oct 2014 01:49:14 -0700 (PDT) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id ey5si8744483pdb.122.2014.10.31.01.49.12 for ; Fri, 31 Oct 2014 01:49:14 -0700 (PDT) Received-SPF: none (google.com: linux-kernel-owner@vger.kernel.org does not designate permitted sender hosts) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757470AbaJaItH (ORCPT + 26 others); Fri, 31 Oct 2014 04:49:07 -0400 Received: from mail-wi0-f169.google.com ([209.85.212.169]:37250 "EHLO mail-wi0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757175AbaJaIsT (ORCPT ); Fri, 31 Oct 2014 04:48:19 -0400 Received: by mail-wi0-f169.google.com with SMTP id n3so707923wiv.4 for ; Fri, 31 Oct 2014 01:48:18 -0700 (PDT) X-Received: by 10.180.90.237 with SMTP id bz13mr2409965wib.50.1414745298418; Fri, 31 Oct 2014 01:48:18 -0700 (PDT) Received: from lmenx30s.st.com. (pas72-3-88-189-71-117.fbx.proxad.net. [88.189.71.117]) by mx.google.com with ESMTPSA id u5sm11636547wiz.9.2014.10.31.01.48.16 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 31 Oct 2014 01:48:17 -0700 (PDT) From: Vincent Guittot To: peterz@infradead.org, mingo@kernel.org, linux-kernel@vger.kernel.org, preeti@linux.vnet.ibm.com, Morten.Rasmussen@arm.com, kamalesh@linux.vnet.ibm.com, linux@arm.linux.org.uk, linux-arm-kernel@lists.infradead.org Cc: riel@redhat.com, efault@gmx.de, nicolas.pitre@linaro.org, linaro-kernel@lists.linaro.org, Morten Rasmussen , Paul Turner , Ben Segall , Vincent Guittot Subject: [PATCH v8 09/10] sched: Make sched entity usage tracking scale-invariant Date: Fri, 31 Oct 2014 09:47:31 +0100 Message-Id: <1414745252-4895-10-git-send-email-vincent.guittot@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1414745252-4895-1-git-send-email-vincent.guittot@linaro.org> References: <1414745252-4895-1-git-send-email-vincent.guittot@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: vincent.guittot@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.174 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , From: Morten Rasmussen Apply frequency scale-invariance correction factor to usage tracking. Each segment of the running_load_avg geometric series is now scaled by the current frequency so the utilization_avg_contrib of each entity will be invariant with frequency scaling. As a result, utilization_load_avg which is the sum of utilization_avg_contrib, becomes invariant too. So the usage level that is returned by get_cpu_usage, stays relative to the max frequency as the cpu_capacity which is is compared against. Then, we want the keep the load tracking values in a 32bits type, which implies that the max value of {runnable|running}_avg_sum must be lower than 2^32/88761=48388 (88761 is the max weigth of a task). As LOAD_AVG_MAX = 47742, arch_scale_freq_capacity must return a value less than (48388/47742) << SCHED_CAPACITY_SHIFT = 1037 (SCHED_SCALE_CAPACITY = 1024). So we define the range to [0..SCHED_SCALE_CAPACITY] in order to avoid overflow. cc: Paul Turner cc: Ben Segall Signed-off-by: Morten Rasmussen Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index c04f2b2..31c3ec2 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -2266,6 +2266,8 @@ static u32 __compute_runnable_contrib(u64 n) return contrib + runnable_avg_yN_sum[n]; } +unsigned long __weak arch_scale_freq_capacity(struct sched_domain *sd, int cpu); + /* * We can represent the historical contribution to runnable average as the * coefficients of a geometric series. To do this we sub-divide our runnable @@ -2294,7 +2296,7 @@ static u32 __compute_runnable_contrib(u64 n) * load_avg = u_0` + y*(u_0 + u_1*y + u_2*y^2 + ... ) * = u_0 + u_1*y + u_2*y^2 + ... [re-labeling u_i --> u_{i+1}] */ -static __always_inline int __update_entity_runnable_avg(u64 now, +static __always_inline int __update_entity_runnable_avg(u64 now, int cpu, struct sched_avg *sa, int runnable, int running) @@ -2302,6 +2304,7 @@ static __always_inline int __update_entity_runnable_avg(u64 now, u64 delta, periods; u32 runnable_contrib; int delta_w, decayed = 0; + unsigned long scale_freq = arch_scale_freq_capacity(NULL, cpu); delta = now - sa->last_runnable_update; /* @@ -2337,7 +2340,8 @@ static __always_inline int __update_entity_runnable_avg(u64 now, if (runnable) sa->runnable_avg_sum += delta_w; if (running) - sa->running_avg_sum += delta_w; + sa->running_avg_sum += delta_w * scale_freq + >> SCHED_CAPACITY_SHIFT; sa->avg_period += delta_w; delta -= delta_w; @@ -2358,7 +2362,8 @@ static __always_inline int __update_entity_runnable_avg(u64 now, if (runnable) sa->runnable_avg_sum += runnable_contrib; if (running) - sa->running_avg_sum += runnable_contrib; + sa->running_avg_sum += runnable_contrib * scale_freq + >> SCHED_CAPACITY_SHIFT; sa->avg_period += runnable_contrib; } @@ -2366,7 +2371,8 @@ static __always_inline int __update_entity_runnable_avg(u64 now, if (runnable) sa->runnable_avg_sum += delta; if (running) - sa->running_avg_sum += delta; + sa->running_avg_sum += delta * scale_freq + >> SCHED_CAPACITY_SHIFT; sa->avg_period += delta; return decayed; @@ -2474,8 +2480,8 @@ static inline void __update_group_entity_contrib(struct sched_entity *se) static inline void update_rq_runnable_avg(struct rq *rq, int runnable) { - __update_entity_runnable_avg(rq_clock_task(rq), &rq->avg, runnable, - runnable); + __update_entity_runnable_avg(rq_clock_task(rq), cpu_of(rq), &rq->avg, + runnable, runnable); __update_tg_runnable_avg(&rq->avg, &rq->cfs); } #else /* CONFIG_FAIR_GROUP_SCHED */ @@ -2553,6 +2559,7 @@ static inline void update_entity_load_avg(struct sched_entity *se, { struct cfs_rq *cfs_rq = cfs_rq_of(se); long contrib_delta, utilization_delta; + int cpu = cpu_of(rq_of(cfs_rq)); u64 now; /* @@ -2564,7 +2571,7 @@ static inline void update_entity_load_avg(struct sched_entity *se, else now = cfs_rq_clock_task(group_cfs_rq(se)); - if (!__update_entity_runnable_avg(now, &se->avg, se->on_rq, + if (!__update_entity_runnable_avg(now, cpu, &se->avg, se->on_rq, cfs_rq->curr == se)) return;