[tip/core/rcu,32/86] rcu: prevent call_rcu() from diving into rcu core if irqs disabled

Message ID 1304256126-26015-32-git-send-email-paulmck@linux.vnet.ibm.com
State New
Headers show

Commit Message

Paul E. McKenney May 1, 2011, 1:21 p.m.
This commit marks a first step towards making call_rcu() have
real-time behavior.  If irqs are disabled, don't dive into the
RCU core.  Later on, this new early exit will wake up the
per-CPU kthread, which first must be modified to handle the
cases involving callback storms.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 kernel/rcutree.c |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

Patch

diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 963b4b1..a6fc20d 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -1839,6 +1839,13 @@  __call_rcu(struct rcu_head *head, void (*func)(struct rcu_head *rcu),
 	/* Add the callback to our list. */
 	*rdp->nxttail[RCU_NEXT_TAIL] = head;
 	rdp->nxttail[RCU_NEXT_TAIL] = &head->next;
+	rdp->qlen++;
+
+	/* If interrupts were disabled, don't dive into RCU core. */
+	if (irqs_disabled_flags(flags)) {
+		local_irq_restore(flags);
+		return;
+	}
 
 	/*
 	 * Force the grace period if too many callbacks or too long waiting.
@@ -1847,7 +1854,7 @@  __call_rcu(struct rcu_head *head, void (*func)(struct rcu_head *rcu),
 	 * invoking force_quiescent_state() if the newly enqueued callback
 	 * is the only one waiting for a grace period to complete.
 	 */
-	if (unlikely(++rdp->qlen > rdp->qlen_last_fqs_check + qhimark)) {
+	if (unlikely(rdp->qlen > rdp->qlen_last_fqs_check + qhimark)) {
 
 		/* Are we ignoring a completed grace period? */
 		rcu_process_gp_end(rsp, rdp);