diff mbox

[RFC,tip/core/rcu,22/41] rcu: Simplify unboosting checks

Message ID 1328125319-5205-22-git-send-email-paulmck@linux.vnet.ibm.com
State Superseded
Headers show

Commit Message

Paul E. McKenney Feb. 1, 2012, 7:41 p.m. UTC 
From: "Paul E. McKenney" <paul.mckenney@linaro.org>

This is a port of commit #82e78d80 from TREE_PREEMPT_RCU to
TINY_PREEMPT_RCU.

This commit uses the fact that current->rcu_boost_mutex is set
any time that the RCU_READ_UNLOCK_BOOSTED flag is set in the
current->rcu_read_unlock_special bitmask.  This allows tests of
the bit to be changed to tests of the pointer, which in turn allows
the RCU_READ_UNLOCK_BOOSTED flag to be eliminated.

Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 include/linux/sched.h   |    3 +--
 kernel/rcutiny_plugin.h |   10 ++++++----
 2 files changed, 7 insertions(+), 6 deletions(-)

Comments

Josh Triplett Feb. 2, 2012, 2:38 a.m. UTC | #1 
On Wed, Feb 01, 2012 at 11:41:40AM -0800, Paul E. McKenney wrote:
> From: "Paul E. McKenney" <paul.mckenney@linaro.org>
> 
> This is a port of commit #82e78d80 from TREE_PREEMPT_RCU to
> TINY_PREEMPT_RCU.
> 
> This commit uses the fact that current->rcu_boost_mutex is set
> any time that the RCU_READ_UNLOCK_BOOSTED flag is set in the
> current->rcu_read_unlock_special bitmask.  This allows tests of
> the bit to be changed to tests of the pointer, which in turn allows
> the RCU_READ_UNLOCK_BOOSTED flag to be eliminated.

Does this change affect rcu_read_unlock()'s logic to trigger the
slowpath only when special flags get set?

- Josh Triplett
Paul E. McKenney Feb. 2, 2012, 5:48 p.m. UTC | #2 
On Wed, Feb 01, 2012 at 06:38:47PM -0800, Josh Triplett wrote:
> On Wed, Feb 01, 2012 at 11:41:40AM -0800, Paul E. McKenney wrote:
> > From: "Paul E. McKenney" <paul.mckenney@linaro.org>
> > 
> > This is a port of commit #82e78d80 from TREE_PREEMPT_RCU to
> > TINY_PREEMPT_RCU.
> > 
> > This commit uses the fact that current->rcu_boost_mutex is set
> > any time that the RCU_READ_UNLOCK_BOOSTED flag is set in the
> > current->rcu_read_unlock_special bitmask.  This allows tests of
> > the bit to be changed to tests of the pointer, which in turn allows
> > the RCU_READ_UNLOCK_BOOSTED flag to be eliminated.
> 
> Does this change affect rcu_read_unlock()'s logic to trigger the
> slowpath only when special flags get set?

Interestingly enough, it does not.  The only way a task can be subjected
to RCU priority boosting is for that task to block sometime in its
current RCU read-side critical section.  When the task blocks, the
RCU_READ_UNLOCK_BLOCKED flag will be set.  Therefore, any time that the
current->rcu_boost_mutex pointer is non-NULL, the RCU_READ_UNLOCK_BLOCKED
flag will be set, so the current test of current->rcu_read_unlock_special
against zero continues to work correctly.

OK, OK, I will update the commit message with words to this effect.  ;-)

							Thanx, Paul
Josh Triplett Feb. 3, 2012, 4:23 a.m. UTC | #3 
On Thu, Feb 02, 2012 at 09:48:12AM -0800, Paul E. McKenney wrote:
> On Wed, Feb 01, 2012 at 06:38:47PM -0800, Josh Triplett wrote:
> > On Wed, Feb 01, 2012 at 11:41:40AM -0800, Paul E. McKenney wrote:
> > > From: "Paul E. McKenney" <paul.mckenney@linaro.org>
> > > 
> > > This is a port of commit #82e78d80 from TREE_PREEMPT_RCU to
> > > TINY_PREEMPT_RCU.
> > > 
> > > This commit uses the fact that current->rcu_boost_mutex is set
> > > any time that the RCU_READ_UNLOCK_BOOSTED flag is set in the
> > > current->rcu_read_unlock_special bitmask.  This allows tests of
> > > the bit to be changed to tests of the pointer, which in turn allows
> > > the RCU_READ_UNLOCK_BOOSTED flag to be eliminated.
> > 
> > Does this change affect rcu_read_unlock()'s logic to trigger the
> > slowpath only when special flags get set?
> 
> Interestingly enough, it does not.  The only way a task can be subjected
> to RCU priority boosting is for that task to block sometime in its
> current RCU read-side critical section.  When the task blocks, the
> RCU_READ_UNLOCK_BLOCKED flag will be set.  Therefore, any time that the
> current->rcu_boost_mutex pointer is non-NULL, the RCU_READ_UNLOCK_BLOCKED
> flag will be set, so the current test of current->rcu_read_unlock_special
> against zero continues to work correctly.

Makes sense; no sense boosting an RCU reader that already has a CPU to
run on.

> OK, OK, I will update the commit message with words to this effect.  ;-)

Thanks. :)

- Josh Triplett
diff mbox

Patch

diff --git a/include/linux/sched.h b/include/linux/sched.h
index 4032ec1..1db119f 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1864,8 +1864,7 @@  extern void task_clear_jobctl_pending(struct task_struct *task,
 #ifdef CONFIG_PREEMPT_RCU
 
 #define RCU_READ_UNLOCK_BLOCKED (1 << 0) /* blocked while in RCU read-side. */
-#define RCU_READ_UNLOCK_BOOSTED (1 << 1) /* boosted while in RCU read-side. */
-#define RCU_READ_UNLOCK_NEED_QS (1 << 2) /* RCU core needs CPU response. */
+#define RCU_READ_UNLOCK_NEED_QS (1 << 1) /* RCU core needs CPU response. */
 
 static inline void rcu_copy_process(struct task_struct *p)
 {
diff --git a/kernel/rcutiny_plugin.h b/kernel/rcutiny_plugin.h
index 9d7d985..01eac49 100644
--- a/kernel/rcutiny_plugin.h
+++ b/kernel/rcutiny_plugin.h
@@ -318,7 +318,6 @@  static int rcu_boost(void)
 	t = container_of(tb, struct task_struct, rcu_node_entry);
 	rt_mutex_init_proxy_locked(&mtx, t);
 	t->rcu_boost_mutex = &mtx;
-	t->rcu_read_unlock_special |= RCU_READ_UNLOCK_BOOSTED;
 	raw_local_irq_restore(flags);
 	rt_mutex_lock(&mtx);
 	rt_mutex_unlock(&mtx);  /* Keep lockdep happy. */
@@ -550,6 +549,9 @@  static noinline void rcu_read_unlock_special(struct task_struct *t)
 	int empty_exp;
 	unsigned long flags;
 	struct list_head *np;
+#ifdef CONFIG_RCU_BOOST
+	struct rt_mutex *rbmp = NULL;
+#endif /* #ifdef CONFIG_RCU_BOOST */
 	int special;
 
 	/*
@@ -615,10 +617,10 @@  static noinline void rcu_read_unlock_special(struct task_struct *t)
 	}
 #ifdef CONFIG_RCU_BOOST
 	/* Unboost self if was boosted. */
-	if (special & RCU_READ_UNLOCK_BOOSTED) {
-		t->rcu_read_unlock_special &= ~RCU_READ_UNLOCK_BOOSTED;
-		rt_mutex_unlock(t->rcu_boost_mutex);
+	if (t->rcu_boost_mutex != NULL) {
+		rbmp = t->rcu_boost_mutex;
 		t->rcu_boost_mutex = NULL;
+		rt_mutex_unlock(rbmp);
 	}
 #endif /* #ifdef CONFIG_RCU_BOOST */
 	local_irq_restore(flags);