From patchwork Thu Feb 1 16:51:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Poirier X-Patchwork-Id: 126557 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp1878127ljc; Thu, 1 Feb 2018 08:51:30 -0800 (PST) X-Google-Smtp-Source: AH8x224RNRHV04TpOf6T0RGf7hNEKMlZAQQ+Ffa6koU6BTZBhg/tjkCFL8DMKUjBUwTstTqrpKto X-Received: by 2002:a17:902:5a8c:: with SMTP id r12-v6mr15994744pli.87.1517503890694; Thu, 01 Feb 2018 08:51:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1517503890; cv=none; d=google.com; s=arc-20160816; b=PzKtxdnKFN5dnORD6AcE1iZPbU923vXWOYQfHdZvTpEugFR3VXt/Fpu7IItk7e2s7R 4XE9KRMl1YI/ECggL5jH+DPq+bqdfsMHWZDeLhbZPdWyYYNG3cY8RdIcHtBvvnAolLAW 2rCfTReEXX7n2WkGuUHbgwC6Ddwq/y9JOf9surZewG0BEf1GP+GNFspkx1blNhAONDCK uuC45B5TfGpSCrhs1aoSAHDREc8IfONjtXHSMLJNZppS31zcRLaAUXw0/uoMQeqmXk4/ riSuirOrbgeLRg+xH7niB6W0jbnoiOT6TgwXUJmmJAQfp+y5ZbxPsSWJOzPw4ZFIhrju n+rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=V//kdI3WH+mWZEDeznJVIv7RJ7NLlJ52gNLFJllNrSg=; b=DaaFhIyr5iW8/HP6LAAmEz5ByuHto43jlaQcEtYZPaNHH3Up3JEVSR5YAlFq4ZXYFq 1rpR5Jr6v6qDknW+nX50NRx74fFlt+nCC2vC5MesgRTXzQyyaYOmTx7bc1SbdCdzP/Nc ZPQBjBBFhdSUPMyAcuOYp9sKoJtl3aW/Fj5WwTWbOatfc4AuRHRKpUHrCBkxTceLqsdB NJ+hqgQeuAoPGP7MTOIqee7Za7AwUebrzt0m1NpZOgH564pFYDUPQUnzwjZhwDd4VNkx woJ40terAfVYm3QdwJUTxTj26aX4WtgLSi/GPVC5cekxgniUQ0O1guERXa7Eh8G/ADPT Es+w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MZQNgjP1; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j11-v6si2108303pll.485.2018.02.01.08.51.30; Thu, 01 Feb 2018 08:51:30 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MZQNgjP1; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752833AbeBAQv2 (ORCPT + 28 others); Thu, 1 Feb 2018 11:51:28 -0500 Received: from mail-it0-f65.google.com ([209.85.214.65]:53485 "EHLO mail-it0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752782AbeBAQvU (ORCPT ); Thu, 1 Feb 2018 11:51:20 -0500 Received: by mail-it0-f65.google.com with SMTP id b5so4992567itc.3 for ; Thu, 01 Feb 2018 08:51:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=V//kdI3WH+mWZEDeznJVIv7RJ7NLlJ52gNLFJllNrSg=; b=MZQNgjP1d+ldoO57fcZE6UIhjPSGyeoz/YViqCMwd97eCQEuz5BDtdkIR+wJAufNJw Ll+RD02wFIuqIwvB8MaZGEIHvDXCqIz2f+MXM4F53Ya3InN+la2iSvjLbUbg3qBN3Jr3 gIDxiUwVzpZ/m4hXqhU0hRx/sqCfSGTcVxEsw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=V//kdI3WH+mWZEDeznJVIv7RJ7NLlJ52gNLFJllNrSg=; b=Y6k/Pg4rE45DIgRe6paq5ZIgmwZGFHF3wLzcVUHvaqFavoZKf8N7PrgxAP4XtBy7DB yvjj2LASqpadp/JcFk/2wGJtSGA68JjUWMsrPd0XvE+Wh/L1rtoJcWpjzoVoiqLOVI07 sTqZUIBgPKG+u3RrBV3lOTMEZT1Aod339Pv6p7Tm7/r7Qck6YTDlINV3+JAkupXOI9zO F+MRVPKMVdcbk3o4M1z5e/gzYZpMktXXs8B8jmcrmPG8zQfaP3AJtf/bduRKzgdJPtht J72RNCIUWwe3Dc48yvK5773KFZUG3yC5kf/OOlvpo9K/dJaP5pOxKj1a1eS8oHPkD1+E qCbg== X-Gm-Message-State: AKwxytc+FILjMD4z0GlOZwZTJQgI1KXnNIGcv5LPQNtGzp3gPFXVGa75 dz7XICsrdUcL/NdGc0NYPamMAQ== X-Received: by 10.36.250.195 with SMTP id v186mr19418039ith.151.1517503879328; Thu, 01 Feb 2018 08:51:19 -0800 (PST) Received: from xps15.cg.shawcable.net (S0106002369de4dac.cg.shawcable.net. [68.147.8.254]) by smtp.gmail.com with ESMTPSA id e83sm9270773iof.71.2018.02.01.08.51.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Feb 2018 08:51:18 -0800 (PST) From: Mathieu Poirier To: peterz@infradead.org Cc: lizefan@huawei.com, mingo@redhat.com, rostedt@goodmis.org, claudio@evidence.eu.com, bristot@redhat.com, tommaso.cucinotta@santannapisa.it, juri.lelli@redhat.com, luca.abeni@santannapisa.it, linux-kernel@vger.kernel.org Subject: [PATCH V2 4/7] cgroup: Constrain 'sched_load_balance' flag when DL tasks are present Date: Thu, 1 Feb 2018 09:51:06 -0700 Message-Id: <1517503869-3179-5-git-send-email-mathieu.poirier@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1517503869-3179-1-git-send-email-mathieu.poirier@linaro.org> References: <1517503869-3179-1-git-send-email-mathieu.poirier@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch prevents the 'sched_load_balance' flag from being set to 0 when DL tasks are present in a CPUset. Otherwise we end up with the DL tasks using CPUs belonging to different root domains, something that breaks the mathematical model behind DL bandwidth management. For example on a 4 core system CPUset "set1" has been created and CPUs 0 and 1 assigned to it. A DL task has also been spun off. By default the DL task can use all the CPUs in the default CPUset. If we set the base CPUset's cpuset.sched_load_balance to 0, CPU 0 and 1 are added to a newly created root domain while CPU 2 and 3 endup in the default root domain. But the DL task is still part of the base CPUset and as such can use CPUs 0 to 3, spanning at the same time more than one root domain. Signed-off-by: Mathieu Poirier --- kernel/cgroup/cpuset.c | 104 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) -- 2.7.4 diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 6942c4652f31..daa1b2bc7e11 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -458,6 +458,106 @@ static void free_trial_cpuset(struct cpuset *trial) kfree(trial); } +static bool cpuset_has_dl_tasks(struct cpuset *cs) +{ + bool dl_tasks = false; + struct css_task_iter it; + struct task_struct *task; + + /* Go through each task in @cs looking for a DL task */ + css_task_iter_start(&cs->css, 0, &it); + + while (!dl_tasks && (task = css_task_iter_next(&it))) { + if (dl_task(task)) + dl_tasks = true; + } + + css_task_iter_end(&it); + + return dl_tasks; +} + +/* + * Assumes RCU read lock and cpuset_mutex are held. + */ +static int +validate_change_load_balance(struct cpuset *cur, struct cpuset *trial) +{ + bool populated = false, dl_tasks = false; + int ret = -EBUSY; + struct cgroup_subsys_state *pos_css; + struct cpuset *cs; + + /* Bail out if nothing has changed. */ + if (is_sched_load_balance(cur) == + is_sched_load_balance(trial)) { + ret = 0; + goto out; + } + + /* + * First deal with the generic case that applies when + * cpuset.sched_load_balance gets flipped on a cpuset, + * regardless of the value. + */ + cpuset_for_each_descendant_pre(cs, pos_css, cur) { + if (cpuset_has_dl_tasks(cs)) + dl_tasks = true; + + /* Skip the top cpuset since it obviously exists */ + if (cs == cur) + continue; + + /* Children without CPUs are not important */ + if (cpumask_empty(cs->cpus_allowed)) { + pos_css = css_rightmost_descendant(pos_css); + continue; + } + + /* CPUs have been assigned to this cpuset. */ + populated = true; + + /* + * Go no further if both conditions are true so that we + * don't end up in a situation where a DL task is + * spanning more than one root domain or only assigned + * to a subset of the CPUs in a root domain. + */ + if (populated && dl_tasks) + goto out; + } + + /* + * Things get very complicated when dealing with children cpuset, + * resulting in hard to maintain code and low confidence that + * all cases are handled properly. As such prevent the + * cpuset.sched_load_balance from being modified on children cpuset + * where DL tasks have been assigned (or any of its children). + */ + if (dl_tasks && parent_cs(cur)) + goto out; + + ret = 0; +out: + return ret; +} + +/* + * Assumes RCU read lock and cpuset_mutex are held. + */ +static int +validate_dl_change(struct cpuset *cur, struct cpuset *trial) +{ + int ret = 0; + + /* Check if the sched_load_balance flag has been changed */ + ret = validate_change_load_balance(cur, trial); + if (ret) + return ret; + + return ret; +} + /* * validate_change() - Used to validate that any proposed cpuset change * follows the structural rules for cpusets. @@ -492,6 +592,10 @@ static int validate_change(struct cpuset *cur, struct cpuset *trial) if (!is_cpuset_subset(c, trial)) goto out; + /* Make sure changes are compatible with deadline scheduling class */ + if (validate_dl_change(cur, trial)) + goto out; + /* Remaining checks don't apply to root cpuset */ ret = 0; if (cur == &top_cpuset)