From patchwork Wed Feb 4 18:30:44 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Morten Rasmussen X-Patchwork-Id: 44381 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-la0-f72.google.com (mail-la0-f72.google.com [209.85.215.72]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 94A1A21173 for ; Wed, 4 Feb 2015 18:44:07 +0000 (UTC) Received: by mail-la0-f72.google.com with SMTP id gq15sf2519839lab.3 for ; Wed, 04 Feb 2015 10:44:06 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:sender:precedence:list-id :x-original-sender:x-original-authentication-results:mailing-list :list-post:list-help:list-archive:list-unsubscribe; bh=EbYH1WNT1m2mO4cR8u59tUraTFPYVP8kYafJDBRw8jE=; b=XDVnYV+xZdn8rpv8NzDn2O/UXXWxW5coqDBKP8cZ20NV0XaDZvpyZaGzj7OTRhcRS8 mQBGFY8WBo9wWPwb63PDKYS6xp1gIOPt2w1JYpIjwb/4GpiYV1zcc8bE7LEY8rEmUVWN P+s1EtSSibOyAv3lY6c7ZWB1+M78sHm3QHCqC7M4nwjX7h1WdyvjMHYQBOaTSKB/QAvN 11wSj57LRfpGE1uZlPF0MpvOPZlAg8euE6MKsxQKx26n7RZwaT0pb3FF8WWKpF/hWnFJ 7X5yvPIqiVm12JXtpS/23gNDsTjv99j0PYpaajjgjvpMSwWCufYx91quAFShIGFGUeUw /Huw== X-Gm-Message-State: ALoCoQmO9R4rbK2/5IVponUylNC+cfNqQzonct35AysWmx6EYrFGw/4mdbBr5lT7O/4TrSdogAVV X-Received: by 10.180.105.228 with SMTP id gp4mr2933575wib.6.1423075446526; Wed, 04 Feb 2015 10:44:06 -0800 (PST) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.27.9 with SMTP id p9ls71030lag.86.gmail; Wed, 04 Feb 2015 10:44:06 -0800 (PST) X-Received: by 10.152.88.44 with SMTP id bd12mr32153126lab.86.1423075446289; Wed, 04 Feb 2015 10:44:06 -0800 (PST) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com. [209.85.215.54]) by mx.google.com with ESMTPS id a9si2112442lbc.44.2015.02.04.10.44.05 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Feb 2015 10:44:05 -0800 (PST) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.54 as permitted sender) client-ip=209.85.215.54; Received: by mail-la0-f54.google.com with SMTP id s18so2136927lam.13 for ; Wed, 04 Feb 2015 10:44:05 -0800 (PST) X-Received: by 10.152.203.230 with SMTP id kt6mr31539012lac.38.1423075445017; Wed, 04 Feb 2015 10:44:05 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.35.133 with SMTP id h5csp670997lbj; Wed, 4 Feb 2015 10:44:04 -0800 (PST) X-Received: by 10.66.253.197 with SMTP id ac5mr47923787pad.152.1423075443186; Wed, 04 Feb 2015 10:44:03 -0800 (PST) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id uo3si3225216pac.48.2015.02.04.10.44.02; Wed, 04 Feb 2015 10:44:03 -0800 (PST) Received-SPF: none (google.com: linux-kernel-owner@vger.kernel.org does not designate permitted sender hosts) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1162255AbbBDSnp (ORCPT + 29 others); Wed, 4 Feb 2015 13:43:45 -0500 Received: from foss-mx-na.foss.arm.com ([217.140.108.86]:41486 "EHLO foss-mx-na.foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934088AbbBDSav (ORCPT ); Wed, 4 Feb 2015 13:30:51 -0500 Received: from foss-smtp-na-1.foss.arm.com (unknown [10.80.61.8]) by foss-mx-na.foss.arm.com (Postfix) with ESMTP id 5F2B3517; Wed, 4 Feb 2015 12:30:46 -0600 (CST) Received: from collaborate-mta1.arm.com (highbank-bc01-b06.austin.arm.com [10.112.81.134]) by foss-smtp-na-1.foss.arm.com (Postfix) with ESMTP id DC0E85FAC1; Wed, 4 Feb 2015 12:30:43 -0600 (CST) Received: from e105550-lin.cambridge.arm.com (e105550-lin.cambridge.arm.com [10.2.131.193]) by collaborate-mta1.arm.com (Postfix) with ESMTP id 5062013F6EE; Wed, 4 Feb 2015 12:30:42 -0600 (CST) From: Morten Rasmussen To: peterz@infradead.org, mingo@redhat.com Cc: vincent.guittot@linaro.org, dietmar.eggemann@arm.com, yuyang.du@intel.com, preeti@linux.vnet.ibm.com, mturquette@linaro.org, nico@linaro.org, rjw@rjwysocki.net, juri.lelli@arm.com, linux-kernel@vger.kernel.org Subject: [RFCv3 PATCH 07/48] sched: get CPU's usage statistic Date: Wed, 4 Feb 2015 18:30:44 +0000 Message-Id: <1423074685-6336-8-git-send-email-morten.rasmussen@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1423074685-6336-1-git-send-email-morten.rasmussen@arm.com> References: <1423074685-6336-1-git-send-email-morten.rasmussen@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: morten.rasmussen@arm.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.54 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , From: Vincent Guittot Monitor the usage level of each group of each sched_domain level. The usage is the portion of cpu_capacity_orig that is currently used on a CPU or group of CPUs. We use the utilization_load_avg to evaluate the usage level of each group. The utilization_load_avg only takes into account the running time of the CFS tasks on a CPU with a maximum value of SCHED_LOAD_SCALE when the CPU is fully utilized. Nevertheless, we must cap utilization_load_avg which can be temporaly greater than SCHED_LOAD_SCALE after the migration of a task on this CPU and until the metrics are stabilized. The utilization_load_avg is in the range [0..SCHED_LOAD_SCALE] to reflect the running load on the CPU whereas the available capacity for the CFS task is in the range [0..cpu_capacity_orig]. In order to test if a CPU is fully utilized by CFS tasks, we have to scale the utilization in the cpu_capacity_orig range of the CPU to get the usage of the latter. The usage can then be compared with the available capacity (ie cpu_capacity) to deduct the usage level of a CPU. The frequency scaling invariance of the usage is not taken into account in this patch, it will be solved in another patch which will deal with frequency scaling invariance on the running_load_avg. Signed-off-by: Vincent Guittot Acked-by: Morten Rasmussen --- kernel/sched/fair.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 3fdad38..7ec48db 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4769,6 +4769,33 @@ static int select_idle_sibling(struct task_struct *p, int target) done: return target; } +/* + * get_cpu_usage returns the amount of capacity of a CPU that is used by CFS + * tasks. The unit of the return value must capacity so we can compare the + * usage with the capacity of the CPU that is available for CFS task (ie + * cpu_capacity). + * cfs.utilization_load_avg is the sum of running time of runnable tasks on a + * CPU. It represents the amount of utilization of a CPU in the range + * [0..SCHED_LOAD_SCALE]. The usage of a CPU can't be higher than the full + * capacity of the CPU because it's about the running time on this CPU. + * Nevertheless, cfs.utilization_load_avg can be higher than SCHED_LOAD_SCALE + * because of unfortunate rounding in avg_period and running_load_avg or just + * after migrating tasks until the average stabilizes with the new running + * time. So we need to check that the usage stays into the range + * [0..cpu_capacity_orig] and cap if necessary. + * Without capping the usage, a group could be seen as overloaded (CPU0 usage + * at 121% + CPU1 usage at 80%) whereas CPU1 has 20% of available capacity/ + */ +static int get_cpu_usage(int cpu) +{ + unsigned long usage = cpu_rq(cpu)->cfs.utilization_load_avg; + unsigned long capacity = capacity_orig_of(cpu); + + if (usage >= SCHED_LOAD_SCALE) + return capacity; + + return (usage * capacity) >> SCHED_LOAD_SHIFT; +} /* * select_task_rq_fair: Select target runqueue for the waking task in domains @@ -5895,6 +5922,7 @@ struct sg_lb_stats { unsigned long sum_weighted_load; /* Weighted load of group's tasks */ unsigned long load_per_task; unsigned long group_capacity; + unsigned long group_usage; /* Total usage of the group */ unsigned int sum_nr_running; /* Nr tasks running in the group */ unsigned int group_capacity_factor; unsigned int idle_cpus; @@ -6243,6 +6271,7 @@ static inline void update_sg_lb_stats(struct lb_env *env, load = source_load(i, load_idx); sgs->group_load += load; + sgs->group_usage += get_cpu_usage(i); sgs->sum_nr_running += rq->cfs.h_nr_running; if (rq->nr_running > 1)