From patchwork Thu Aug 30 18:56:18 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 11101 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id DB61E23F70 for ; Thu, 30 Aug 2012 21:43:42 +0000 (UTC) Received: from mail-iy0-f180.google.com (mail-iy0-f180.google.com [209.85.210.180]) by fiordland.canonical.com (Postfix) with ESMTP id EADA6A192FF for ; Thu, 30 Aug 2012 21:43:04 +0000 (UTC) Received: by iafj25 with SMTP id j25so3726599iaf.11 for ; Thu, 30 Aug 2012 14:43:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf:from:to:cc :subject:date:message-id:x-mailer:in-reply-to:references :x-content-scanned:x-cbid:x-gm-message-state; bh=p2FTfqgBsPH1DERx9ieOw3LUq8oBFjbQohQdc/pZrac=; b=JgDbf6bAuosLW7hRBdnZLaEsUPKl0Ic1jQAHV0MoboPei7uZcy7kxakvBi7JFqYMyk 9ehvMIafjOk/u4TrsjPZHk1wJ+d1W/zTkmDgYiEwUjGkCK9bwWDS2JjGDnOFrFVFqqNN ghGbC+Sn8gD36HDsh72lNk6HCsctxVvGL8+Vn6v6sYr5WuSVxdQX17dfb/OU7enPfwGJ RQsvR6E3yM2hm3Vs9fxb48BW+IMWvR4T1SrtPdP1puGFTyZ42ajjBtaY8vWsGE+m9STE lAm78OjJVOzdDlMXK9113Zk54kNJZs7mbnrMJWCSW+zQLNGWsDUSg0fPvQXhfi+nu/vV C7fw== Received: by 10.50.159.196 with SMTP id xe4mr2421487igb.43.1346363020749; Thu, 30 Aug 2012 14:43:40 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.50.184.232 with SMTP id ex8csp32860igc; Thu, 30 Aug 2012 14:43:40 -0700 (PDT) Received: by 10.50.192.199 with SMTP id hi7mr2387294igc.68.1346363020014; Thu, 30 Aug 2012 14:43:40 -0700 (PDT) Received: from e4.ny.us.ibm.com (e4.ny.us.ibm.com. [32.97.182.144]) by mx.google.com with ESMTPS id hl11si2692069igb.20.2012.08.30.14.43.39 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 30 Aug 2012 14:43:40 -0700 (PDT) Received-SPF: pass (google.com: domain of paulmck@linux.vnet.ibm.com designates 32.97.182.144 as permitted sender) client-ip=32.97.182.144; Authentication-Results: mx.google.com; spf=pass (google.com: domain of paulmck@linux.vnet.ibm.com designates 32.97.182.144 as permitted sender) smtp.mail=paulmck@linux.vnet.ibm.com Received: from /spool/local by e4.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 30 Aug 2012 17:43:39 -0400 Received: from d01dlp02.pok.ibm.com (9.56.250.167) by e4.ny.us.ibm.com (192.168.1.104) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 30 Aug 2012 17:43:37 -0400 Received: from d01relay04.pok.ibm.com (d01relay04.pok.ibm.com [9.56.227.236]) by d01dlp02.pok.ibm.com (Postfix) with ESMTP id 61F616E8039; Thu, 30 Aug 2012 17:43:36 -0400 (EDT) Received: from d03av01.boulder.ibm.com (d03av01.boulder.ibm.com [9.17.195.167]) by d01relay04.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q7ULhZkP135104; Thu, 30 Aug 2012 17:43:36 -0400 Received: from d03av01.boulder.ibm.com (loopback [127.0.0.1]) by d03av01.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q7ULhYRO026528; Thu, 30 Aug 2012 15:43:35 -0600 Received: from paulmck-ThinkPad-W500 (sig-9-65-153-225.mts.ibm.com [9.65.153.225]) by d03av01.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q7ULhXSl026416; Thu, 30 Aug 2012 15:43:33 -0600 Received: by paulmck-ThinkPad-W500 (Postfix, from userid 1000) id D2A24EA82A; Thu, 30 Aug 2012 11:56:30 -0700 (PDT) From: "Paul E. McKenney" To: linux-kernel@vger.kernel.org Cc: mingo@elte.hu, laijs@cn.fujitsu.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@polymtl.ca, josh@joshtriplett.org, niv@us.ibm.com, tglx@linutronix.de, peterz@infradead.org, rostedt@goodmis.org, Valdis.Kletnieks@vt.edu, dhowells@redhat.com, eric.dumazet@gmail.com, darren@dvhart.com, fweisbec@gmail.com, sbw@mit.edu, patches@linaro.org, "Paul E. McKenney" , "Paul E. McKenney" Subject: [PATCH tip/core/rcu 05/15] rcu: Improve boost selection when moving tasks to root rcu_node Date: Thu, 30 Aug 2012 11:56:18 -0700 Message-Id: <1346352988-32444-5-git-send-email-paulmck@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.8 In-Reply-To: <1346352988-32444-1-git-send-email-paulmck@linux.vnet.ibm.com> References: <20120830185607.GA32148@linux.vnet.ibm.com> <1346352988-32444-1-git-send-email-paulmck@linux.vnet.ibm.com> X-Content-Scanned: Fidelis XPS MAILER x-cbid: 12083021-3534-0000-0000-00000C0C191F X-Gm-Message-State: ALoCoQma6iYHy0ZoP0HfD0JrtGLzHIfHtHNS/6lr4uefxHGIYs4Dxk9T8hab6VPVrqs97bBPEnxj From: "Paul E. McKenney" The rcu_preempt_offline_tasks() moves all tasks queued on a given leaf rcu_node structure to the root rcu_node, which is done when the last CPU corresponding the the leaf rcu_node structure goes offline. Now that RCU-preempt's synchronize_rcu_expedited() implementation blocks CPU-hotplug operations during the initialization of each rcu_node structure's ->boost_tasks pointer, rcu_preempt_offline_tasks() can do a better job of setting the root rcu_node's ->boost_tasks pointer. The key point is that rcu_preempt_offline_tasks() runs as part of the CPU-hotplug process, so that a concurrent synchronize_rcu_expedited() is guaranteed to either have not started on the one hand (in which case there is no boosting on behalf of the expedited grace period) to be completely initialized on the other (in which case, in absence of other priority boosting, all ->boost_tasks pointers will be initialized). Therefore, if rcu_preempt_offline_tasks() finds that the ->boost_tasks pointer is equal to the ->exp_tasks pointer, it can be sure that it is correcty placed. The case where there was boosting ongoing at the time that the synchronize_rcu_expedited() function started, different nodes might start boosting the tasks blocking the expedited grace period at different times. In this mixed case, the root node will either be boosting tasks for the expedited grace period already, or it will start as soon as it gets done boosting for the normal grace period -- but in this latter case, the root node's tasks needed to be boosted in any case. This commit therefore adds a check of the ->boost_tasks pointer against the ->exp_tasks pointer to the list that prevents updating ->boost_tasks. Signed-off-by: Paul E. McKenney Signed-off-by: Paul E. McKenney Reviewed-by: Josh Triplett --- kernel/rcutree_plugin.h | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h index b1b4851..c930a47 100644 --- a/kernel/rcutree_plugin.h +++ b/kernel/rcutree_plugin.h @@ -591,7 +591,8 @@ static int rcu_preempt_offline_tasks(struct rcu_state *rsp, /* In case root is being boosted and leaf was not. */ raw_spin_lock(&rnp_root->lock); /* irqs already disabled */ if (rnp_root->boost_tasks != NULL && - rnp_root->boost_tasks != rnp_root->gp_tasks) + rnp_root->boost_tasks != rnp_root->gp_tasks && + rnp_root->boost_tasks != rnp_root->exp_tasks) rnp_root->boost_tasks = rnp_root->gp_tasks; raw_spin_unlock(&rnp_root->lock); /* irqs still disabled */ #endif /* #ifdef CONFIG_RCU_BOOST */