From patchwork Thu Feb 5 15:59:33 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xunlei Pang X-Patchwork-Id: 44465 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-wg0-f72.google.com (mail-wg0-f72.google.com [74.125.82.72]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 8958121521 for ; Thu, 5 Feb 2015 16:01:34 +0000 (UTC) Received: by mail-wg0-f72.google.com with SMTP id k14sf6053204wgh.3 for ; Thu, 05 Feb 2015 08:01:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:sender:precedence:list-id:x-original-sender :x-original-authentication-results:mailing-list:list-post:list-help :list-archive:list-unsubscribe; bh=yL4dAHs8qQSL6CgBDokp+LkEI4sL7FWtUAWvQpyFzCE=; b=hkSWy9RC6d39KbNubm4jOpObx6TTOr2S2ltgAMX8oBqJ4lbrgRvCFJGre9XzPQn3w/ FowqajEpTQm3uJa604Fwgeh1b68CtLHjvLSPO9AKFEzHeK3VR6K/IfQB9c+rIApRXo3K 61WkSur8tjnSoNxNk7fYl4cJif9d3YBDJxq/hywvoNidanJuf/rivRIon4Gb9o6k/ltH dU61wEo/eZQ3EXDwXVTh0QbOgmBFulTj5sx2XMD/RYO9LE2XNAiobYRldDGVsp7N6yY9 FLXBaBLSFCTQDdAT/gsdkQ7rzLMXsiiwY5qLsHlt7k2eI0ljVGSuCkidYbuq1sGUou60 n5Ug== X-Gm-Message-State: ALoCoQk9m8dgPPsz/3XzHDYlLhU0mkRXXIRwjkmupRAlSFpeWVRccT8VnWA+e0L6IMN4nY5TCb8d X-Received: by 10.152.121.65 with SMTP id li1mr617681lab.0.1423152093746; Thu, 05 Feb 2015 08:01:33 -0800 (PST) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.7.226 with SMTP id m2ls178196laa.82.gmail; Thu, 05 Feb 2015 08:01:33 -0800 (PST) X-Received: by 10.152.45.100 with SMTP id l4mr922254lam.112.1423152093371; Thu, 05 Feb 2015 08:01:33 -0800 (PST) Received: from mail-lb0-x229.google.com (mail-lb0-x229.google.com. [2a00:1450:4010:c04::229]) by mx.google.com with ESMTPS id p2si308423lbh.163.2015.02.05.08.01.33 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 05 Feb 2015 08:01:33 -0800 (PST) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 2a00:1450:4010:c04::229 as permitted sender) client-ip=2a00:1450:4010:c04::229; Received: by mail-lb0-f169.google.com with SMTP id f15so8728725lbj.0 for ; Thu, 05 Feb 2015 08:01:33 -0800 (PST) X-Received: by 10.152.43.193 with SMTP id y1mr1101000lal.35.1423152093222; Thu, 05 Feb 2015 08:01:33 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.35.133 with SMTP id h5csp1232043lbj; Thu, 5 Feb 2015 08:01:32 -0800 (PST) X-Received: by 10.68.221.34 with SMTP id qb2mr6629785pbc.47.1423152091047; Thu, 05 Feb 2015 08:01:31 -0800 (PST) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id kp11si6683747pab.94.2015.02.05.08.01.30; Thu, 05 Feb 2015 08:01:31 -0800 (PST) Received-SPF: none (google.com: linux-kernel-owner@vger.kernel.org does not designate permitted sender hosts) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758144AbbBEQBZ (ORCPT + 29 others); Thu, 5 Feb 2015 11:01:25 -0500 Received: from m50-111.126.com ([123.125.50.111]:52370 "EHLO m50-111.126.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753916AbbBEQBT (ORCPT ); Thu, 5 Feb 2015 11:01:19 -0500 Received: from localhost.localdomain (unknown [210.21.223.3]) by smtp5 (Coremail) with SMTP id jtKowAC33y9vk9NUGrIRAQ--.742S2; Thu, 05 Feb 2015 23:59:49 +0800 (CST) From: Xunlei Pang To: linux-kernel@vger.kernel.org Cc: Peter Zijlstra , Steven Rostedt , Juri Lelli , Xunlei Pang Subject: [PATCH v2 1/2] sched/rt: Check to push the task when changing its affinity Date: Thu, 5 Feb 2015 23:59:33 +0800 Message-Id: <1423151974-22557-1-git-send-email-xlpang@126.com> X-Mailer: git-send-email 1.9.1 X-CM-TRANSID: jtKowAC33y9vk9NUGrIRAQ--.742S2 X-Coremail-Antispam: 1Uf129KBjvJXoWxXFy3Jr48KFy5tryrCw15urg_yoWrCw4fpa 1vk39Ygr4UJayagF1fZr4DZr13Kwn3A34rJrnayw1Fkan0qr4Fv3W5tr1ayr9Y9r1j9F4a qr4ktr47WF1UZa7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jk8n5UUUUU= X-Originating-IP: [210.21.223.3] X-CM-SenderInfo: p0ost0bj6rjloofrz/1tbirxWXv0X9v53i3QAAs3 Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Original-Sender: xlpang@126.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 2a00:1450:4010:c04::229 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org; dkim=neutral (body hash did not verify) header.i=@; dmarc=fail (p=NONE dis=NONE) header.from=126.com Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , From: Xunlei Pang We may suffer from extra rt overload rq due to the affinity, so when the affinity of any runnable rt task is changed, we should check to trigger balancing, otherwise it will cause some unnecessary delayed real-time response. Unfortunately, current RT global scheduler doesn't trigger anything. For example: a 2-cpu system with two runnable FIFO tasks(same rt_priority) bound on CPU0, let's name them rt1(running) and rt2(runnable) respectively; CPU1 has no RTs. Then, someone sets the affinity of rt2 to 0x3(i.e. CPU0 and CPU1), but after this, rt2 still can't be scheduled until rt1 enters schedule(), this definitely causes some/big response latency for rt2. So, when doing set_cpus_allowed_rt(), if detecting such cases, check to trigger a push behaviour. Signed-off-by: Xunlei Pang --- v2: Refine according to Steven Rostedt's comments. kernel/sched/rt.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 70 insertions(+), 10 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index f4d4b07..b1ea9c0 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1428,10 +1428,9 @@ static struct sched_rt_entity *pick_next_rt_entity(struct rq *rq, return next; } -static struct task_struct *_pick_next_task_rt(struct rq *rq) +static struct task_struct *peek_next_task_rt(struct rq *rq) { struct sched_rt_entity *rt_se; - struct task_struct *p; struct rt_rq *rt_rq = &rq->rt; do { @@ -1440,7 +1439,14 @@ static struct task_struct *_pick_next_task_rt(struct rq *rq) rt_rq = group_rt_rq(rt_se); } while (rt_rq); - p = rt_task_of(rt_se); + return rt_task_of(rt_se); +} + +static inline struct task_struct *_pick_next_task_rt(struct rq *rq) +{ + struct task_struct *p; + + p = peek_next_task_rt(rq); p->se.exec_start = rq_clock_task(rq); return p; @@ -1886,28 +1892,73 @@ static void set_cpus_allowed_rt(struct task_struct *p, const struct cpumask *new_mask) { struct rq *rq; - int weight; + int old_weight, new_weight; + int preempt_push = 0, direct_push = 0; BUG_ON(!rt_task(p)); if (!task_on_rq_queued(p)) return; - weight = cpumask_weight(new_mask); + old_weight = p->nr_cpus_allowed; + new_weight = cpumask_weight(new_mask); + + rq = task_rq(p); + + if (new_weight > 1 && + rt_task(rq->curr) && + !test_tsk_need_resched(rq->curr)) { + /* + * Set new mask information which is already valid + * to prepare pushing. + * + * We own p->pi_lock and rq->lock. rq->lock might + * get released when doing direct pushing, however + * p->pi_lock is always held, so it's safe to assign + * the new_mask and new_weight to p. + */ + cpumask_copy(&p->cpus_allowed, new_mask); + p->nr_cpus_allowed = new_weight; + + if (task_running(rq, p) && + cpumask_test_cpu(task_cpu(p), new_mask) && + cpupri_find(&rq->rd->cpupri, p, NULL)) { + /* + * At this point, current task gets migratable most + * likely due to the change of its affinity, let's + * figure out if we can migrate it. + * + * Is there any task with the same priority as that + * of current task? If found one, we should resched. + * NOTE: The target may be unpushable. + */ + if (p->prio == rq->rt.highest_prio.next) { + /* One target just in pushable_tasks list. */ + requeue_task_rt(rq, p, 0); + preempt_push = 1; + } else if (rq->rt.rt_nr_total > 1) { + struct task_struct *next; + + requeue_task_rt(rq, p, 0); + next = peek_next_task_rt(rq); + if (next != p && next->prio == p->prio) + preempt_push = 1; + } + } else if (!task_running(rq, p)) + direct_push = 1; + } /* * Only update if the process changes its state from whether it * can migrate or not. */ - if ((p->nr_cpus_allowed > 1) == (weight > 1)) - return; - - rq = task_rq(p); + if ((old_weight > 1) == (new_weight > 1)) + goto out; /* * The process used to be able to migrate OR it can now migrate */ - if (weight <= 1) { + if (new_weight <= 1) { if (!task_current(rq, p)) dequeue_pushable_task(rq, p); BUG_ON(!rq->rt.rt_nr_migratory); @@ -1919,6 +1970,15 @@ static void set_cpus_allowed_rt(struct task_struct *p, } update_rt_migration(&rq->rt); + +out: + BUG_ON(direct_push == 1 && preempt_push == 1); + + if (direct_push) + push_rt_tasks(rq); + + if (preempt_push) + resched_curr(rq); } /* Assumes rq->lock is held */