From patchwork Wed Mar 18 13:24:30 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xunlei Pang X-Patchwork-Id: 45941 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-we0-f197.google.com (mail-we0-f197.google.com [74.125.82.197]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id CE2472153C for ; Wed, 18 Mar 2015 13:26:58 +0000 (UTC) Received: by wesk11 with SMTP id k11sf2545545wes.3 for ; Wed, 18 Mar 2015 06:26:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:sender:precedence:list-id :x-original-sender:x-original-authentication-results:mailing-list :list-post:list-help:list-archive:list-unsubscribe; bh=0W3if4h+AMmjB5yxDlCtIEoWyWLwSpMxz50YB7Vm9rM=; b=LOP/ABqGa5NabRNidcya9DLodCdEkBpdAuNNbIpo51y+LmzeXT2nP8B3yCStdrcZDj W77sZ5W7krmH1IwnaDCeRvItwHC6l4j/Gx3488lJUERb5OCM9xc4fAkAIdZMzxbvtUoj 5cC0cpYM0teKbwHYgMF2KZ+sk5Q6NMeHkb/MhpQ8hVS8U6o+zvpOzkvLdpH80byGFNaX pyPyz0kMNzvScEsqUDsTOdnqlChqDFSRKVcGQDAZyE3FN6GXVM8eyxDHsNA2TJ1DUo4M NyN9hS9AxxD1H2rB1jv7s45AjrCCz3NE/OzftfUa3ksPNLIbf4mxvxRuPQWznUqbTpC1 PKpw== X-Gm-Message-State: ALoCoQnxg33hhhxPheCRYICALxVkSC55IsagD8O0zB/8ZLAQa9OQHyBeN8gpxECyBssLbUy+tpPw X-Received: by 10.180.150.115 with SMTP id uh19mr752520wib.7.1426685218160; Wed, 18 Mar 2015 06:26:58 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.20.67 with SMTP id l3ls155092lae.93.gmail; Wed, 18 Mar 2015 06:26:57 -0700 (PDT) X-Received: by 10.112.167.4 with SMTP id zk4mr62062244lbb.74.1426685217818; Wed, 18 Mar 2015 06:26:57 -0700 (PDT) Received: from mail-lb0-x22e.google.com (mail-lb0-x22e.google.com. [2a00:1450:4010:c04::22e]) by mx.google.com with ESMTPS id w10si12916308lbp.44.2015.03.18.06.26.57 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Mar 2015 06:26:57 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 2a00:1450:4010:c04::22e as permitted sender) client-ip=2a00:1450:4010:c04::22e; Received: by lbbsy1 with SMTP id sy1so29509947lbb.1 for ; Wed, 18 Mar 2015 06:26:57 -0700 (PDT) X-Received: by 10.112.130.100 with SMTP id od4mr41697229lbb.86.1426685217656; Wed, 18 Mar 2015 06:26:57 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.35.133 with SMTP id h5csp1155942lbj; Wed, 18 Mar 2015 06:26:56 -0700 (PDT) X-Received: by 10.67.24.35 with SMTP id if3mr160543562pad.99.1426685214547; Wed, 18 Mar 2015 06:26:54 -0700 (PDT) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id lx1si17959946pdb.68.2015.03.18.06.26.53; Wed, 18 Mar 2015 06:26:54 -0700 (PDT) Received-SPF: none (google.com: linux-kernel-owner@vger.kernel.org does not designate permitted sender hosts) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756216AbbCRN0r (ORCPT + 27 others); Wed, 18 Mar 2015 09:26:47 -0400 Received: from m50-110.126.com ([123.125.50.110]:47870 "EHLO m50-110.126.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755898AbbCRN0o (ORCPT ); Wed, 18 Mar 2015 09:26:44 -0400 Received: from localhost.localdomain (unknown [210.21.223.3]) by smtp4 (Coremail) with SMTP id jdKowAD3aGCWfAlVIG56AQ--.331S3; Wed, 18 Mar 2015 21:24:55 +0800 (CST) From: Xunlei Pang To: linux-kernel@vger.kernel.org Cc: Peter Zijlstra , Steven Rostedt , Juri Lelli , Xunlei Pang Subject: [PATCH RESEND 2/2] sched/rt: Consider deadline tasks in cpupri_find() Date: Wed, 18 Mar 2015 21:24:30 +0800 Message-Id: <1426685070-4419-2-git-send-email-xlpang@126.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1426685070-4419-1-git-send-email-xlpang@126.com> References: <1426685070-4419-1-git-send-email-xlpang@126.com> X-CM-TRANSID: jdKowAD3aGCWfAlVIG56AQ--.331S3 X-Coremail-Antispam: 1Uf129KBjvJXoWxKF15Aw4fGrW5WF1xZw4ktFb_yoW7KryUpF yq93yUAF4DJFy7W3s5Aw4jkwn5Kw1vgas8tw1rta4rtF9xtF10vF1qqr9xZryY9r4kuF13 tF4vyrW29F1jyFJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07ji189UUUUU= X-Originating-IP: [210.21.223.3] X-CM-SenderInfo: p0ost0bj6rjloofrz/1tbi7wfAv00vbiOUYQABsM Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Original-Sender: xlpang@126.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 2a00:1450:4010:c04::22e as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org; dkim=neutral (body hash did not verify) header.i=@; dmarc=fail (p=NONE dis=NONE) header.from=126.com Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , From: Xunlei Pang Currently, RT global scheduling doesn't factor deadline tasks, this may cause some problems. See a case below: On a 3 CPU system, CPU0 has one running deadline task, CPU1 has one running low priority RT task or idle, CPU3 has one running high priority RT task. When another mid priority RT task is woken on CPU3, it will be pushed to CPU0, while it is reasonable to put it on CPU1. Worse still, this disturbs(causing sched IPI, rq locking, etc) higher-class deadline tasks on CPU0. This patch eliminates this issue by filtering CPUs that have runnable deadline tasks, using cpudl->free_cpus in cpupri_find() if there's any deadline task. NOTE: We want to make the most use of percpu local_cpu_mask to save an extra mask allocation, so always passing a non-NULL lowest_mask to cpupri_find(). Signed-off-by: Xunlei Pang --- kernel/sched/core.c | 3 ++- kernel/sched/cpupri.c | 29 +++++++++++------------------ kernel/sched/cpupri.h | 3 ++- kernel/sched/rt.c | 14 ++++++++++---- 4 files changed, 25 insertions(+), 24 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 17141da..86450df 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5644,8 +5644,9 @@ static int init_rootdomain(struct root_domain *rd) if (cpudl_init(&rd->cpudl) != 0) goto free_dlo_mask; - if (cpupri_init(&rd->cpupri) != 0) + if (cpupri_init(&rd->cpupri, &rd->cpudl) != 0) goto free_rto_mask; + return 0; free_rto_mask: diff --git a/kernel/sched/cpupri.c b/kernel/sched/cpupri.c index 981fcd7..1a1e38d 100644 --- a/kernel/sched/cpupri.c +++ b/kernel/sched/cpupri.c @@ -31,6 +31,7 @@ #include #include #include +#include "cpudeadline.h" #include "cpupri.h" /* Convert between a 140 based task->prio, and our 102 based cpupri */ @@ -54,7 +55,7 @@ static int convert_prio(int prio) * cpupri_find - find the best (lowest-pri) CPU in the system * @cp: The cpupri context * @p: The task - * @lowest_mask: A mask to fill in with selected CPUs (or NULL) + * @lowest_mask: A mask to fill in with selected CPUs (not NULL) * * Note: This function returns the recommended CPUs as calculated during the * current invocation. By the time the call returns, the CPUs may have in @@ -75,6 +76,7 @@ int cpupri_find(struct cpupri *cp, struct task_struct *p, for (idx = 0; idx < task_pri; idx++) { struct cpupri_vec *vec = &cp->pri_to_cpu[idx]; + struct cpudl *dl = cp->cpudl; int skip = 0; if (!atomic_read(&(vec)->count)) @@ -103,24 +105,12 @@ int cpupri_find(struct cpupri *cp, struct task_struct *p, if (skip) continue; - if (cpumask_any_and(&p->cpus_allowed, vec->mask) >= nr_cpu_ids) + cpumask_and(lowest_mask, &p->cpus_allowed, vec->mask); + if (dl->size) + cpumask_and(lowest_mask, lowest_mask, dl->free_cpus); + if (cpumask_any(lowest_mask) >= nr_cpu_ids) continue; - if (lowest_mask) { - cpumask_and(lowest_mask, &p->cpus_allowed, vec->mask); - - /* - * We have to ensure that we have at least one bit - * still set in the array, since the map could have - * been concurrently emptied between the first and - * second reads of vec->mask. If we hit this - * condition, simply act as though we never hit this - * priority level and continue on. - */ - if (cpumask_any(lowest_mask) >= nr_cpu_ids) - continue; - } - return 1; } @@ -202,10 +192,11 @@ void cpupri_set(struct cpupri *cp, int cpu, int newpri) /** * cpupri_init - initialize the cpupri structure * @cp: The cpupri context + * @cpudl: The cpudl context of the same root domain * * Return: -ENOMEM on memory allocation failure. */ -int cpupri_init(struct cpupri *cp) +int cpupri_init(struct cpupri *cp, struct cpudl *cpudl) { int i; @@ -226,6 +217,8 @@ int cpupri_init(struct cpupri *cp) for_each_possible_cpu(i) cp->cpu_to_pri[i] = CPUPRI_INVALID; + cp->cpudl = cpudl; + return 0; cleanup: diff --git a/kernel/sched/cpupri.h b/kernel/sched/cpupri.h index 63cbb9c..6fee80b 100644 --- a/kernel/sched/cpupri.h +++ b/kernel/sched/cpupri.h @@ -18,13 +18,14 @@ struct cpupri_vec { struct cpupri { struct cpupri_vec pri_to_cpu[CPUPRI_NR_PRIORITIES]; int *cpu_to_pri; + struct cpudl *cpudl; }; #ifdef CONFIG_SMP int cpupri_find(struct cpupri *cp, struct task_struct *p, struct cpumask *lowest_mask); void cpupri_set(struct cpupri *cp, int cpu, int pri); -int cpupri_init(struct cpupri *cp); +int cpupri_init(struct cpupri *cp, struct cpudl *cpudl); void cpupri_cleanup(struct cpupri *cp); #endif diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index f4d4b07..564b19a 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1354,14 +1354,22 @@ out: return cpu; } +static DEFINE_PER_CPU(cpumask_var_t, local_cpu_mask); + static void check_preempt_equal_prio(struct rq *rq, struct task_struct *p) { + struct cpumask *lowest_mask = this_cpu_cpumask_var_ptr(local_cpu_mask); + + /* Make sure the mask is initialized first */ + if (unlikely(!lowest_mask)) + return; + /* * Current can't be migrated, useless to reschedule, * let's hope p can move out. */ if (rq->curr->nr_cpus_allowed == 1 || - !cpupri_find(&rq->rd->cpupri, rq->curr, NULL)) + !cpupri_find(&rq->rd->cpupri, rq->curr, lowest_mask)) return; /* @@ -1369,7 +1377,7 @@ static void check_preempt_equal_prio(struct rq *rq, struct task_struct *p) * see if it is pushed or pulled somewhere else. */ if (p->nr_cpus_allowed != 1 - && cpupri_find(&rq->rd->cpupri, p, NULL)) + && cpupri_find(&rq->rd->cpupri, p, lowest_mask)) return; /* @@ -1531,8 +1539,6 @@ static struct task_struct *pick_highest_pushable_task(struct rq *rq, int cpu) return NULL; } -static DEFINE_PER_CPU(cpumask_var_t, local_cpu_mask); - static int find_lowest_rq(struct task_struct *task) { struct sched_domain *sd;