From patchwork Wed Aug 16 21:20:39 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Poirier X-Patchwork-Id: 110280 Delivered-To: patch@linaro.org Received: by 10.140.95.78 with SMTP id h72csp1323380qge; Wed, 16 Aug 2017 14:21:01 -0700 (PDT) X-Received: by 10.99.8.194 with SMTP id 185mr2876994pgi.273.1502918461056; Wed, 16 Aug 2017 14:21:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1502918461; cv=none; d=google.com; s=arc-20160816; b=F4jEGRDWfWz46dJvtLiM7IuWRk1X6j3/RpZOccbIlkYCdmgM7Sb4Coiasv3JeV7+Rk TsQgpwrEDZi0ox1d9kSLwXHlVrd1gUXK8ymU0qxTu6OVRg50IwwVR39QvVPTMOx4xLo5 oI9JpEaOhB4Jynb8J7r6nsl+UWXvLrHy8tCar5HnfsVW6l7JjUvJc8MHV1QsWkSWVQfA jMXOPxcUYmIGxn38oST5n9PV8rpud83VL8JfUBZQ0ouxO9qyPz/kG9zvNXTXj3j29xzg QyIjJsUlp/6xjiR8m5bjRz7AVqrLbN2XyE7eEVCTNL+KmESN3t7Lem2xKbktTYi4PWbr Oezg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=QeJmxk+PFZ/Zq+5ovG1dI5IeCKluOyAxo20TVy5b650=; b=UYhmR3zKWQLnna+CULGZYRHj/yYSe+yXytvkeJsEy+y5kFblB9foKSn4L/pgblsfJz q2uPDGG0qWTnqmv7wYMuWarRwYZb/g00ojh9GAwhjsX8qr6MI0/mgEPm3/Y9cvvBXRiL h7BbDoGoROW1MojuNIDKCtYyO/95Zjicyo0Jc0ZPumIDXtfhdUjoVVOOA3m6vGf+YYHN XLEkQuSUuxbEco7Oa9S0zl5bQuxbR7nc29enPa9deat2J5rs1RFRTV0MvngF/ND7ZoGR JUvnT5DoeyhcQEmxrVIua+5r1GBTSMzgeIoP6jOG1fGWJacMeNA30VqJumqhM6Z1AcZM VpjQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Q4o99BBs; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id p81si1043425pfd.11.2017.08.16.14.21.00; Wed, 16 Aug 2017 14:21:01 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Q4o99BBs; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752555AbdHPVU5 (ORCPT + 26 others); Wed, 16 Aug 2017 17:20:57 -0400 Received: from mail-io0-f180.google.com ([209.85.223.180]:38472 "EHLO mail-io0-f180.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752488AbdHPVUy (ORCPT ); Wed, 16 Aug 2017 17:20:54 -0400 Received: by mail-io0-f180.google.com with SMTP id g71so17299611ioe.5 for ; Wed, 16 Aug 2017 14:20:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=QeJmxk+PFZ/Zq+5ovG1dI5IeCKluOyAxo20TVy5b650=; b=Q4o99BBsColzz0xVTM/mMurf8EVhjTvc5xV5pzvRvVQJRfA4p87MMSLgosG53UEwua wkAnnmGGaeY0mcgmgaL7KjNcQkFnpawmj3wlvnWSkzyJPJySpgCAVwFcQlPjsjWFJj7o S3cCXRv5nFi7wcmrfveNfSYvXSQkfRJAhlzC0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=QeJmxk+PFZ/Zq+5ovG1dI5IeCKluOyAxo20TVy5b650=; b=WS6o/iBfdnnb/aNmdScm+TPy+aMxtVAJAJMl//Nn+gEYAXqTnRPb86U5khbAPt0heF B4R/fcwzNB5GDiPJBXZZ0Y9IsuMsxQdr+/6NwBVLNv7t/NP3Ktoqnn8PJicKx660blDv Knk5KmEnsM02+0C5UUoWq5+vTIgZTnnobE/4fwHNxmboRAHt5+ZO/y4m2RQjoiCVrw2W 41DZZXWsccgHFAd7RR2F28cgCXYyWRLZDlstOcMT6D/VRd9F8+ko0EgeHG3eU0ZpaNWC pVyxGlIrWBo+Sro85Dx2fGSZ5KW4bXpZzs/VI8NMDy4clfkQQ5vnyyP+UHfTD9olZhu7 C9UA== X-Gm-Message-State: AHYfb5ja6LnRctx25vI2fMI/LHSyr1R90/QscpdWbUicM7Vl0jCtJt7L lP8dmNsaAF05WcKc X-Received: by 10.107.28.78 with SMTP id c75mr2662028ioc.282.1502918453336; Wed, 16 Aug 2017 14:20:53 -0700 (PDT) Received: from xps15.cg.shawcable.net (S0106002369de4dac.cg.shawcable.net. [68.147.8.254]) by smtp.gmail.com with ESMTPSA id 80sm918281itk.11.2017.08.16.14.20.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 16 Aug 2017 14:20:52 -0700 (PDT) From: Mathieu Poirier To: mingo@redhat.com, peterz@infradead.org Cc: tj@kernel.org, vbabka@suse.cz, lizefan@huawei.com, akpm@linux-foundation.org, weiyongjun1@huawei.com, juri.lelli@arm.com, rostedt@goodmis.org, claudio@evidence.eu.com, luca.abeni@santannapisa.it, bristot@redhat.com, linux-kernel@vger.kernel.org, mathieu.poirier@linaro.org Subject: [PATCH 3/7] sched/deadline: Keep new DL task within root domain's boundary Date: Wed, 16 Aug 2017 15:20:39 -0600 Message-Id: <1502918443-30169-4-git-send-email-mathieu.poirier@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1502918443-30169-1-git-send-email-mathieu.poirier@linaro.org> References: <1502918443-30169-1-git-send-email-mathieu.poirier@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org When considering to move a task to the DL policy we need to make sure the CPUs it is allowed to run on matches the CPUs of the root domains of the runqueue it is currently assigned to. Otherwise the task will be allowed to roam on CPUs outside of this root domain, something that will skew system deadline statistics and potentially lead to over selling DL bandwidth. For example say we have a 4 core system split in 2 cpuset: set1 has CPU 0 and 1 while set2 has CPU 2 and 3. This results in 3 cpuset - the default set that has all 4 CPUs along with set1 and set2 as just depicted. We also have task A that hasn't been assigned to any CPUset and as such, is part of the default CPUset. At the time we want to move task A to a DL policy it has been assigned to CPU1. Since CPU1 is part of set1 the root domain will have 2 CPUs in it and the bandwidth constraint checked against the current DL bandwidth allotment of those 2 CPUs. If task A is promoted to a DL policy it's 'cpus_allowed' mask is still equal to the CPUs in the default CPUset, making it possible for the scheduler to move it to CPU2 and CPU3, which could also be running DL tasks of their own. This patch makes sure that a task's cpus_allowed mask matches the CPUs in the root domain associated to the runqueue it has been assigned to. Signed-off-by: Mathieu Poirier --- kernel/sched/deadline.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) -- 2.7.4 diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index ba64a5b8f40b..2c0405d74367 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -2474,6 +2474,7 @@ int sched_dl_overflow(struct task_struct *p, int policy, const struct sched_attr *attr) { struct dl_bw *dl_b = dl_bw_of(task_cpu(p)); + struct root_domain *rd = cpu_rq(task_cpu(p))->rd; u64 period = attr->sched_period ?: attr->sched_deadline; u64 runtime = attr->sched_runtime; u64 new_bw = dl_policy(policy) ? to_ratio(period, runtime) : 0; @@ -2484,6 +2485,19 @@ int sched_dl_overflow(struct task_struct *p, int policy, return 0; /* + * By default a task is set to run on all the CPUs the system + * knows about. This is fine for as long as we don't deal with cpusets + * where runqueues are split between root domains. The computation + * below is based on root domain information, as such the task must be + * constrained to run within that root domain. It is the user's + * responsability to constrain the task to specific CPUs by either + * assigning the task to a cpuset or run the taskset utility. Here we + * simply make sure things are coherent. + */ + if (!cpumask_equal(&p->cpus_allowed, rd->span)) + goto out; + + /* * Either if a task, enters, leave, or stays -deadline but changes * its parameters, we may need to update accordingly the total * allocated bandwidth of the container. @@ -2518,7 +2532,7 @@ int sched_dl_overflow(struct task_struct *p, int policy, err = 0; } raw_spin_unlock(&dl_b->lock); - +out: return err; }