From patchwork Wed Jun 13 12:17:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juri Lelli X-Patchwork-Id: 138446 Delivered-To: patch@linaro.org Received: by 2002:a2e:970d:0:0:0:0:0 with SMTP id r13-v6csp645100lji; Wed, 13 Jun 2018 05:17:54 -0700 (PDT) X-Google-Smtp-Source: ADUXVKK0pZMcyDZN2FR0aK2b4NVPfjPP/NJT/JvPSsxpxbM42sGag+8RLd0QPSJxKcftbR8SmO4G X-Received: by 2002:a62:5788:: with SMTP id i8-v6mr4651194pfj.175.1528892274361; Wed, 13 Jun 2018 05:17:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1528892274; cv=none; d=google.com; s=arc-20160816; b=k0sReut42FLVBHQ/B6pGzEVlWcdL5fRctTmIWgp3pGRs2ri+lh7PB9ulXprZP29ZZq 16I7o4LWGrhkel9rQNV62YQGL7P9BqoUr+sGAs9g6vGbdo7SR7Uu5qr/57HzCo/eX/nB jD3f3RmVE5bb2CXz8LpFVgSHUTcvJeX2zCICeEvKgGsxPX6m9CO6f16+FdWKxgkj2AEB CPIiJkSzPE1sWZGdeO/09BKb4WVGJN6LJ5ByrJxrzOIO0BK0sLZBxwmqK/GQRcYfwwD6 ZRbAnveiUdWKrp8SinCccFPlFMsAtEuJQ5/FFGPY7lisvaudz/x0UHRmF+i+IsTDtG0q FHJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=+v7ZtHB+S1/k+VzF7nBNUx06ZPwRVh8ls5JWBGUzTgo=; b=sit694SBMYzC+x5XVerTwWZvo5nGCVLsSjlF/agKuw3c913nDtiUPWOfHar6cp0g1p eQT1jbaQTuF38CL6FwJI3Hq9c557mZ9ARf1VaEXvKyHMPsyO5hMLj0ttXTCroEQyGd49 QQi1XV2GaLsl23i4jqgz8fomOotlenSBarAOt1ElaSLesstc1BHO3mP0viVFRJRs4Vts POboblfftdd0y9vh2dDONmyi6iNlD5ysJw3Ok3a65tFiTj1CzKHimT2xuzrjpXV4MVaC 9ghyJ7KigBybAeUYPAsCfG0OxvKCWMWc5mBKr2rPobX0FFwuKRUI9zdMvKybDI0pZ+dC eLDg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id z23-v6si2573373pfe.296.2018.06.13.05.17.54; Wed, 13 Jun 2018 05:17:54 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S964791AbeFMMRw (ORCPT + 30 others); Wed, 13 Jun 2018 08:17:52 -0400 Received: from mail-wm0-f66.google.com ([74.125.82.66]:38505 "EHLO mail-wm0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S935497AbeFMMRd (ORCPT ); Wed, 13 Jun 2018 08:17:33 -0400 Received: by mail-wm0-f66.google.com with SMTP id 69-v6so4971729wmf.3 for ; Wed, 13 Jun 2018 05:17:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=+v7ZtHB+S1/k+VzF7nBNUx06ZPwRVh8ls5JWBGUzTgo=; b=srV1iS8KWzmBsVFOOKkJVmEh0I0N4GpDgUvanZQJ7CUmAH7ukNiBb21rPKUeAtIGQu xZLHvsp047a0p6NMWpcG65I0nnd2tU+Z8IdX2cfoGQ/WmaJgmWJZxTn+FdBhtBiGjzsB Wt9mnvawEguTaWp+HHHMHXBMynVCNESKcUZTR+M/u0qj5NF4dGpNOPfuU35r0qSGKHbo aWxFwCUJy4U1RkCs8uP2a5v5tEXL326gUjG/Fl0ZjiIpILucBCvI86Xqgj9F7kzaMDMp JNzxI3VaYOZIbcqdSMxauXmbx9AFYiSVuUr2LFqAe9IZlqRm5MKv/Uu/dL6b7n9d3Oym 5tGA== X-Gm-Message-State: APt69E0ZREGei/kGRul+aPmObfgpSzVDT3LOjK5MvreeCBQw33LbY3u8 8LJGdMtWifOvqzSIv1a2iIOc/w== X-Received: by 2002:a7b:c01a:: with SMTP id c26-v6mr3441099wmb.33.1528892252483; Wed, 13 Jun 2018 05:17:32 -0700 (PDT) Received: from localhost.localdomain.com ([151.15.207.242]) by smtp.gmail.com with ESMTPSA id 137-v6sm4943673wmv.28.2018.06.13.05.17.31 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 13 Jun 2018 05:17:31 -0700 (PDT) From: Juri Lelli To: peterz@infradead.org, mingo@redhat.com, rostedt@goodmis.org Cc: linux-kernel@vger.kernel.org, luca.abeni@santannapisa.it, claudio@evidence.eu.com, tommaso.cucinotta@santannapisa.it, bristot@redhat.com, mathieu.poirier@linaro.org, lizefan@huawei.com, cgroups@vger.kernel.org Subject: [PATCH v4 5/5] cpuset: Rebuild root domain deadline accounting information Date: Wed, 13 Jun 2018 14:17:11 +0200 Message-Id: <20180613121711.5018-6-juri.lelli@redhat.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180613121711.5018-1-juri.lelli@redhat.com> References: <20180613121711.5018-1-juri.lelli@redhat.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Mathieu Poirier When the topology of root domains is modified by CPUset or CPUhotplug operations information about the current deadline bandwidth held in the root domain is lost. This patch address the issue by recalculating the lost deadline bandwidth information by circling through the deadline tasks held in CPUsets and adding their current load to the root domain they are associated with. Signed-off-by: Mathieu Poirier --- include/linux/sched.h | 5 ++++ include/linux/sched/deadline.h | 8 ++++++ kernel/cgroup/cpuset.c | 63 +++++++++++++++++++++++++++++++++++++++++- kernel/sched/deadline.c | 31 +++++++++++++++++++++ kernel/sched/sched.h | 3 -- kernel/sched/topology.c | 13 ++++++++- 6 files changed, 118 insertions(+), 5 deletions(-) -- 2.14.3 diff --git a/include/linux/sched.h b/include/linux/sched.h index 28ff3ca9f752..f7fcc903e1a1 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -278,6 +278,11 @@ struct vtime { u64 gtime; }; +#ifdef CONFIG_SMP +extern struct root_domain def_root_domain; +extern struct mutex sched_domains_mutex; +#endif + struct sched_info { #ifdef CONFIG_SCHED_INFO /* Cumulative counters: */ diff --git a/include/linux/sched/deadline.h b/include/linux/sched/deadline.h index 0cb034331cbb..1aff00b65f3c 100644 --- a/include/linux/sched/deadline.h +++ b/include/linux/sched/deadline.h @@ -24,3 +24,11 @@ static inline bool dl_time_before(u64 a, u64 b) { return (s64)(a - b) < 0; } + +#ifdef CONFIG_SMP + +struct root_domain; +extern void dl_add_task_root_domain(struct task_struct *p); +extern void dl_clear_root_domain(struct root_domain *rd); + +#endif /* CONFIG_SMP */ diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index d26fd4795aa3..0ca10418ddf6 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -44,6 +44,7 @@ #include #include #include +#include #include #include #include @@ -812,6 +813,66 @@ static int generate_sched_domains(cpumask_var_t **domains, return ndoms; } +static void update_tasks_root_domain(struct cpuset *cs) +{ + struct css_task_iter it; + struct task_struct *task; + + css_task_iter_start(&cs->css, 0, &it); + + while ((task = css_task_iter_next(&it))) + dl_add_task_root_domain(task); + + css_task_iter_end(&it); +} + +/* + * Called with cpuset_mutex held (rebuild_sched_domains()) + * Called with hotplug lock held (rebuild_sched_domains_locked()) + * Called with sched_domains_mutex held (partition_and_rebuild_domains()) + */ +static void rebuild_root_domains(void) +{ + struct cpuset *cs = NULL; + struct cgroup_subsys_state *pos_css; + + rcu_read_lock(); + + /* + * Clear default root domain DL accounting, it will be computed again + * if a task belongs to it. + */ + dl_clear_root_domain(&def_root_domain); + + cpuset_for_each_descendant_pre(cs, pos_css, &top_cpuset) { + + if (cpumask_empty(cs->effective_cpus)) { + pos_css = css_rightmost_descendant(pos_css); + continue; + } + + css_get(&cs->css); + + rcu_read_unlock(); + + update_tasks_root_domain(cs); + + rcu_read_lock(); + css_put(&cs->css); + } + rcu_read_unlock(); +} + +static void +partition_and_rebuild_sched_domains(int ndoms_new, cpumask_var_t doms_new[], + struct sched_domain_attr *dattr_new) +{ + mutex_lock(&sched_domains_mutex); + partition_sched_domains_locked(ndoms_new, doms_new, dattr_new); + rebuild_root_domains(); + mutex_unlock(&sched_domains_mutex); +} + /* * Rebuild scheduler domains. * @@ -844,7 +905,7 @@ static void rebuild_sched_domains_locked(void) ndoms = generate_sched_domains(&doms, &attr); /* Have scheduler rebuild the domains */ - partition_sched_domains(ndoms, doms, attr); + partition_and_rebuild_sched_domains(ndoms, doms, attr); out: put_online_cpus(); } diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 1356afd1eeb6..7be2c89b0645 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -2275,6 +2275,37 @@ void __init init_sched_dl_class(void) GFP_KERNEL, cpu_to_node(i)); } +void dl_add_task_root_domain(struct task_struct *p) +{ + unsigned long flags; + struct rq_flags rf; + struct rq *rq; + struct dl_bw *dl_b; + + rq = task_rq_lock(p, &rf); + if (!dl_task(p)) + goto unlock; + + dl_b = &rq->rd->dl_bw; + raw_spin_lock_irqsave(&dl_b->lock, flags); + + dl_b->total_bw += p->dl.dl_bw; + + raw_spin_unlock_irqrestore(&dl_b->lock, flags); + +unlock: + task_rq_unlock(rq, p, &rf); +} + +void dl_clear_root_domain(struct root_domain *rd) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&rd->dl_bw.lock, flags); + rd->dl_bw.total_bw = 0; + raw_spin_unlock_irqrestore(&rd->dl_bw.lock, flags); +} + #endif /* CONFIG_SMP */ static void switched_from_dl(struct rq *rq, struct task_struct *p) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 67702b4d9ac7..83bfacbd13d4 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -723,9 +723,6 @@ struct root_domain { unsigned long max_cpu_capacity; }; -extern struct root_domain def_root_domain; -extern struct mutex sched_domains_mutex; - extern void init_defrootdomain(void); extern int sched_init_domains(const struct cpumask *cpu_map); extern void rq_attach_root(struct rq *rq, struct root_domain *rd); diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 25a5727d3b48..fb39cee51b14 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -1884,8 +1884,19 @@ void partition_sched_domains_locked(int ndoms_new, cpumask_var_t doms_new[], for (i = 0; i < ndoms_cur; i++) { for (j = 0; j < n && !new_topology; j++) { if (cpumask_equal(doms_cur[i], doms_new[j]) - && dattrs_equal(dattr_cur, i, dattr_new, j)) + && dattrs_equal(dattr_cur, i, dattr_new, j)) { + struct root_domain *rd; + + /* + * This domain won't be destroyed and as such + * its dl_bw->total_bw needs to be cleared. It + * will be recomputed in function + * update_tasks_root_domain(). + */ + rd = cpu_rq(cpumask_any(doms_cur[i]))->rd; + dl_clear_root_domain(rd); goto match1; + } } /* No match - a current sched domain not in new doms_new[] */ detach_destroy_domains(doms_cur[i]);