From patchwork Sun May 1 13:21:12 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 1265 Return-Path: Delivered-To: unknown Received: from imap.gmail.com (74.125.159.109) by localhost6.localdomain6 with IMAP4-SSL; 08 Jun 2011 14:50:38 -0000 Delivered-To: patches@linaro.org Received: by 10.224.2.73 with SMTP id 9cs230872qai; Sun, 1 May 2011 06:22:25 -0700 (PDT) Received: by 10.236.25.130 with SMTP id z2mr1167114yhz.439.1304256145086; Sun, 01 May 2011 06:22:25 -0700 (PDT) Received: from e3.ny.us.ibm.com (e3.ny.us.ibm.com [32.97.182.143]) by mx.google.com with ESMTPS id 46si14504977yhl.225.2011.05.01.06.22.23 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 01 May 2011 06:22:24 -0700 (PDT) Received-SPF: pass (google.com: domain of paulmck@linux.vnet.ibm.com designates 32.97.182.143 as permitted sender) client-ip=32.97.182.143; Authentication-Results: mx.google.com; spf=pass (google.com: domain of paulmck@linux.vnet.ibm.com designates 32.97.182.143 as permitted sender) smtp.mail=paulmck@linux.vnet.ibm.com Received: from d01relay04.pok.ibm.com (d01relay04.pok.ibm.com [9.56.227.236]) by e3.ny.us.ibm.com (8.14.4/8.13.1) with ESMTP id p41D0u75026763 for ; Sun, 1 May 2011 09:00:56 -0400 Received: from d01av01.pok.ibm.com (d01av01.pok.ibm.com [9.56.224.215]) by d01relay04.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id p41DMN2L072848 for ; Sun, 1 May 2011 09:22:23 -0400 Received: from d01av01.pok.ibm.com (loopback [127.0.0.1]) by d01av01.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id p41DMGGo019551 for ; Sun, 1 May 2011 09:22:22 -0400 Received: from paulmck-ThinkPad-W500 (sig-9-65-224-93.mts.ibm.com [9.65.224.93]) by d01av01.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id p41DMBlP019020; Sun, 1 May 2011 09:22:15 -0400 Received: by paulmck-ThinkPad-W500 (Postfix, from userid 1000) id 916A813F80E; Sun, 1 May 2011 06:22:08 -0700 (PDT) From: "Paul E. McKenney" To: linux-kernel@vger.kernel.org Cc: mingo@elte.hu, laijs@cn.fujitsu.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@polymtl.ca, josh@joshtriplett.org, niv@us.ibm.com, tglx@linutronix.de, peterz@infradead.org, rostedt@goodmis.org, Valdis.Kletnieks@vt.edu, dhowells@redhat.com, eric.dumazet@gmail.com, darren@dvhart.com, patches@linaro.org, "Paul E. McKenney" Subject: [PATCH tip/core/rcu 32/86] rcu: prevent call_rcu() from diving into rcu core if irqs disabled Date: Sun, 1 May 2011 06:21:12 -0700 Message-Id: <1304256126-26015-32-git-send-email-paulmck@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.3.2 In-Reply-To: <20110501132142.GA25494@linux.vnet.ibm.com> References: <20110501132142.GA25494@linux.vnet.ibm.com> This commit marks a first step towards making call_rcu() have real-time behavior. If irqs are disabled, don't dive into the RCU core. Later on, this new early exit will wake up the per-CPU kthread, which first must be modified to handle the cases involving callback storms. Signed-off-by: Paul E. McKenney --- kernel/rcutree.c | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-) diff --git a/kernel/rcutree.c b/kernel/rcutree.c index 963b4b1..a6fc20d 100644 --- a/kernel/rcutree.c +++ b/kernel/rcutree.c @@ -1839,6 +1839,13 @@ __call_rcu(struct rcu_head *head, void (*func)(struct rcu_head *rcu), /* Add the callback to our list. */ *rdp->nxttail[RCU_NEXT_TAIL] = head; rdp->nxttail[RCU_NEXT_TAIL] = &head->next; + rdp->qlen++; + + /* If interrupts were disabled, don't dive into RCU core. */ + if (irqs_disabled_flags(flags)) { + local_irq_restore(flags); + return; + } /* * Force the grace period if too many callbacks or too long waiting. @@ -1847,7 +1854,7 @@ __call_rcu(struct rcu_head *head, void (*func)(struct rcu_head *rcu), * invoking force_quiescent_state() if the newly enqueued callback * is the only one waiting for a grace period to complete. */ - if (unlikely(++rdp->qlen > rdp->qlen_last_fqs_check + qhimark)) { + if (unlikely(rdp->qlen > rdp->qlen_last_fqs_check + qhimark)) { /* Are we ignoring a completed grace period? */ rcu_process_gp_end(rsp, rdp);