From patchwork Wed Aug 16 21:20:38 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Poirier X-Patchwork-Id: 110285 Delivered-To: patch@linaro.org Received: by 10.140.95.78 with SMTP id h72csp1325065qge; Wed, 16 Aug 2017 14:22:46 -0700 (PDT) X-Received: by 10.98.74.199 with SMTP id c68mr2980798pfj.242.1502918566301; Wed, 16 Aug 2017 14:22:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1502918566; cv=none; d=google.com; s=arc-20160816; b=wm/4hGae7VpXnZWTGLJdw/asdXtHGsSA/GVD40PeMBchH1QGIGQpqcgMMYCzQXLTYx 6SMjTAhEGe86jxbQLu68XDiIojs0YVKs/B3jGRxgKQt/Zy8+u74Uy1fE6hshzPMEnjgN DNvru0lS2eKAewTMuMcaTUC0AZJ+x7ttGp47CCx9z4+bT3HP/km0tjZkvryPl3oB9iXh tC3RWxJzHlVkRW73OHQ2DnXEPPUhToFkRIahVRpXQUbhNeOKnUvcYUG1QAqMSLlyeVq4 bvhFF8siDJS3HWVHUKFEVH91/esWLeOqloc4GWN0AEnmJVL8WKPxUa1SIoQyTZ1Jzn61 NR4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=vgj72HD53CNQquqxWf8RMoN2Zr9dsyVDBBSeHiY5OJE=; b=FeRawCLgkFzLg1xq493qkMRRL7d9khYghVyVb6jrRHMgmSdsUZchcvKt6prxrylXfJ dLRG2cBmAQBBDgO3MuHHpoKIBUsFQBsjPX+lkDQptxJQEgNxr9bMNgFXO/dVXpg8RWWM WC9fCFWKybdixQo+XuUXr15G7tEjMkXiKq8L6vuJwo1IgIl/9BDHrZYiEDTZs/7uXH1X c1UUhAGPitc5JfdGHQl4QmDyVe+y3jlsObBog5yIz5J5Tym9UEhFaRJF+kDvaqI6V6OD +3BE9OGGaB6TWF/jMjfoVyoVeCrfaj/6bhWIuD8oG6jXaGnximvzrwJNSPMYbGDoO5ON vgMg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=O8Jb/22e; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d8si1118344pln.20.2017.08.16.14.22.45; Wed, 16 Aug 2017 14:22:46 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=O8Jb/22e; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752744AbdHPVWm (ORCPT + 26 others); Wed, 16 Aug 2017 17:22:42 -0400 Received: from mail-it0-f42.google.com ([209.85.214.42]:33090 "EHLO mail-it0-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752428AbdHPVUv (ORCPT ); Wed, 16 Aug 2017 17:20:51 -0400 Received: by mail-it0-f42.google.com with SMTP id f16so21543518itb.0 for ; Wed, 16 Aug 2017 14:20:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vgj72HD53CNQquqxWf8RMoN2Zr9dsyVDBBSeHiY5OJE=; b=O8Jb/22ePi8ZkqVfZxXbYwtS6Tazl6Q4J7xvfCyWRPRVmiony9Uw76sN9Z9C7w93BX Eu3XG4k1raJ1+5KA+o45OzcXHiy/KkpllCaUMV9qpITiWtwkvo2C4NWsmRlJWZUTNALI wu6+UKDNqCg4lRz+DRLtI/Z5gqY0TOF91XUXs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vgj72HD53CNQquqxWf8RMoN2Zr9dsyVDBBSeHiY5OJE=; b=RKiGruvZiJmUJwb7s5TbnA4atV5tnJBhJ5N37DGwZd+blpP57zQwlvhyDfhVCjDERT YdppXDXD7vrnXo9LHSjqu7Wx/ysHw/gPfDHRbVd5smfIxVjpPqweWHjlmyyTpcWnxH4v NOteRkKbBl1zc6Nj5ISITPgYEeoZaSLzaO4dHtqqKWp7frOrRJQggtKQzCc+1uQotEwe N97jiJHrcNjwmpplQIlfZmT/JEyGSznAGhIUVmAKawQbMJnWl/w9JYjhnd3+I7RTHJBL PCYAOvLGgGMWmTUj4iMokSX5t3+fD194X5miq2pwMnKq/+rQ6DadamLxGVmZKZBFFiN7 +OzA== X-Gm-Message-State: AHYfb5g2cfvaJKFjiM1JiyTilziWDAdhm1kLFLT9NHJblj6wrG7VIs7p fenaWDCz29eh3zPJ X-Received: by 10.36.51.212 with SMTP id k203mr403537itk.136.1502918450982; Wed, 16 Aug 2017 14:20:50 -0700 (PDT) Received: from xps15.cg.shawcable.net (S0106002369de4dac.cg.shawcable.net. [68.147.8.254]) by smtp.gmail.com with ESMTPSA id 80sm918281itk.11.2017.08.16.14.20.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 16 Aug 2017 14:20:49 -0700 (PDT) From: Mathieu Poirier To: mingo@redhat.com, peterz@infradead.org Cc: tj@kernel.org, vbabka@suse.cz, lizefan@huawei.com, akpm@linux-foundation.org, weiyongjun1@huawei.com, juri.lelli@arm.com, rostedt@goodmis.org, claudio@evidence.eu.com, luca.abeni@santannapisa.it, bristot@redhat.com, linux-kernel@vger.kernel.org, mathieu.poirier@linaro.org Subject: [PATCH 2/7] cpuset: Rebuild root domain deadline accounting information Date: Wed, 16 Aug 2017 15:20:38 -0600 Message-Id: <1502918443-30169-3-git-send-email-mathieu.poirier@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1502918443-30169-1-git-send-email-mathieu.poirier@linaro.org> References: <1502918443-30169-1-git-send-email-mathieu.poirier@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org When the topology of root domains is modified by CPUset or CPUhotplug operations information about the current deadline bandwidth held in the root domain is lost. This patch address the issue by recalculating the lost deadline bandwidth information by circling through the deadline tasks held in CPUsets and adding their current load to the root domain they are associated with. Signed-off-by: Mathieu Poirier --- include/linux/sched.h | 3 ++ include/linux/sched/deadline.h | 8 ++++++ kernel/cgroup/cpuset.c | 63 +++++++++++++++++++++++++++++++++++++++++- kernel/sched/deadline.c | 31 +++++++++++++++++++++ kernel/sched/sched.h | 3 -- kernel/sched/topology.c | 14 +++++++++- 6 files changed, 117 insertions(+), 5 deletions(-) -- 2.7.4 diff --git a/include/linux/sched.h b/include/linux/sched.h index 8337e2db0bb2..bb07b194d951 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -241,6 +241,9 @@ struct vtime { u64 gtime; }; +extern struct root_domain def_root_domain; +extern struct mutex sched_domains_mutex; + struct sched_info { #ifdef CONFIG_SCHED_INFO /* Cumulative counters: */ diff --git a/include/linux/sched/deadline.h b/include/linux/sched/deadline.h index 975be862e083..fcef5565dc94 100644 --- a/include/linux/sched/deadline.h +++ b/include/linux/sched/deadline.h @@ -28,4 +28,12 @@ static inline bool dl_time_before(u64 a, u64 b) return (s64)(a - b) < 0; } +#ifdef CONFIG_SMP + +struct root_domain; +extern void dl_add_task_root_domain(struct task_struct *p); +extern void dl_clear_root_domain(struct root_domain *rd); + +#endif /* CONFIG_SMP */ + #endif /* _LINUX_SCHED_DEADLINE_H */ diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 63ce8bbb476e..f6d1e485dc2d 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -44,6 +44,7 @@ #include #include #include +#include #include #include #include @@ -800,6 +801,66 @@ static int generate_sched_domains(cpumask_var_t **domains, return ndoms; } +static void update_tasks_root_domain(struct cpuset *cs) +{ + struct css_task_iter it; + struct task_struct *task; + + css_task_iter_start(&cs->css, &it); + + while ((task = css_task_iter_next(&it))) + dl_add_task_root_domain(task); + + css_task_iter_end(&it); +} + +/* + * Called with cpuset_mutex held (rebuild_sched_domains()) + * Called with hotplug lock held (rebuild_sched_domains_locked()) + * Called with sched_domains_mutex held (partition_and_rebuild_domains()) + */ +static void rebuild_root_domains(void) +{ + struct cpuset *cs = NULL; + struct cgroup_subsys_state *pos_css; + + rcu_read_lock(); + + /* + * Clear default root domain DL accounting, it will be computed again + * if a task belongs to it. + */ + dl_clear_root_domain(&def_root_domain); + + cpuset_for_each_descendant_pre(cs, pos_css, &top_cpuset) { + + if (cpumask_empty(cs->effective_cpus)) { + pos_css = css_rightmost_descendant(pos_css); + continue; + } + + css_get(&cs->css); + + rcu_read_unlock(); + + update_tasks_root_domain(cs); + + rcu_read_lock(); + css_put(&cs->css); + } + rcu_read_unlock(); +} + +static void +partition_and_rebuild_domains(int ndoms_new, cpumask_var_t doms_new[], + struct sched_domain_attr *dattr_new) +{ + mutex_lock(&sched_domains_mutex); + partition_sched_domains_locked(ndoms_new, doms_new, dattr_new); + rebuild_root_domains(); + mutex_unlock(&sched_domains_mutex); +} + /* * Rebuild scheduler domains. * @@ -832,7 +893,7 @@ static void rebuild_sched_domains_locked(void) ndoms = generate_sched_domains(&doms, &attr); /* Have scheduler rebuild the domains */ - partition_sched_domains(ndoms, doms, attr); + partition_and_rebuild_domains(ndoms, doms, attr); out: put_online_cpus(); } diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 755bd3f1a1a9..ba64a5b8f40b 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -2214,6 +2214,37 @@ void __init init_sched_dl_class(void) GFP_KERNEL, cpu_to_node(i)); } +void dl_add_task_root_domain(struct task_struct *p) +{ + unsigned long flags; + struct rq_flags rf; + struct rq *rq; + struct dl_bw *dl_b; + + rq = task_rq_lock(p, &rf); + if (!dl_task(p)) + goto unlock; + + dl_b = &rq->rd->dl_bw; + raw_spin_lock_irqsave(&dl_b->lock, flags); + + dl_b->total_bw += p->dl.dl_bw; + + raw_spin_unlock_irqrestore(&dl_b->lock, flags); + +unlock: + task_rq_unlock(rq, p, &rf); +} + +void dl_clear_root_domain(struct root_domain *rd) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&rd->dl_bw.lock, flags); + rd->dl_bw.total_bw = 0; + raw_spin_unlock_irqrestore(&rd->dl_bw.lock, flags); +} + #endif /* CONFIG_SMP */ static void switched_from_dl(struct rq *rq, struct task_struct *p) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index eeef1a3086d1..cfd3b5b5fe7f 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -650,9 +650,6 @@ struct root_domain { unsigned long max_cpu_capacity; }; -extern struct root_domain def_root_domain; -extern struct mutex sched_domains_mutex; - extern void init_defrootdomain(void); extern int sched_init_domains(const struct cpumask *cpu_map); extern void rq_attach_root(struct rq *rq, struct root_domain *rd); diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 58133f2ce889..3522dfc2566f 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -2,6 +2,7 @@ * Scheduler topology setup/handling methods */ #include +#include #include #include #include @@ -1862,8 +1863,19 @@ void partition_sched_domains_locked(int ndoms_new, cpumask_var_t doms_new[], for (i = 0; i < ndoms_cur; i++) { for (j = 0; j < n && !new_topology; j++) { if (cpumask_equal(doms_cur[i], doms_new[j]) - && dattrs_equal(dattr_cur, i, dattr_new, j)) + && dattrs_equal(dattr_cur, i, dattr_new, j)) { + struct root_domain *rd; + + /* + * This domain won't be destroyed and as such + * its dl_bw->total_bw needs to be cleared. It + * will be recomputed in function + * update_tasks_root_domain(). + */ + rd = cpu_rq(cpumask_any(doms_cur[i]))->rd; + dl_clear_root_domain(rd); goto match1; + } } /* No match - a current sched domain not in new doms_new[] */ detach_destroy_domains(doms_cur[i]);