From patchwork Thu Aug 30 21:05:19 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 11079 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 66AE323E52 for ; Thu, 30 Aug 2012 21:14:12 +0000 (UTC) Received: from mail-iy0-f180.google.com (mail-iy0-f180.google.com [209.85.210.180]) by fiordland.canonical.com (Postfix) with ESMTP id C98B4A18196 for ; Thu, 30 Aug 2012 21:13:35 +0000 (UTC) Received: by mail-iy0-f180.google.com with SMTP id j25so3685139iaf.11 for ; Thu, 30 Aug 2012 14:14:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf:from:to:cc :subject:date:message-id:x-mailer:in-reply-to:references :x-content-scanned:x-cbid:x-gm-message-state; bh=IthHc8gL5niZDzjS7fIWIzAqOScb9t+NPH4X0Ra/phY=; b=X/9fEl/5iJa/uDT558SuchFIIREisXUB7LZVGj2PihQabjV+uXzIjjuNl9mGYQpppy Zxu1JcRUhnmgiAn43V3i3ZYsBDc1u5q/7edljiYX2UOBFTbFhXBe1XTmsDhnDXbznYDx n5okN4kGaBANwL7dw8kEAIFdf7PMRN1sj8YdzIGVBAdNbM7w+ckD2OjkW/2VBSA5YAhQ W77VQ40w/uYgzKJ92wlltkRIwhnkHL0PQBSMA1LLjUszSfzFmRugJqUWYdQQ4psk/QWP kqBvsVjuac4NobsCopusI3NrSwepKL/gYxZ20Z0meQm8DRPS7RYE+zKdUqVhFmEPERaM mHgg== Received: by 10.50.45.162 with SMTP id o2mr2561260igm.0.1346361251858; Thu, 30 Aug 2012 14:14:11 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.50.184.232 with SMTP id ex8csp31641igc; Thu, 30 Aug 2012 14:14:11 -0700 (PDT) Received: by 10.42.163.135 with SMTP id c7mr6307538icy.45.1346361251593; Thu, 30 Aug 2012 14:14:11 -0700 (PDT) Received: from e31.co.us.ibm.com (e31.co.us.ibm.com. [32.97.110.149]) by mx.google.com with ESMTPS id kn1si2595346igc.47.2012.08.30.14.14.11 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 30 Aug 2012 14:14:11 -0700 (PDT) Received-SPF: pass (google.com: domain of paulmck@linux.vnet.ibm.com designates 32.97.110.149 as permitted sender) client-ip=32.97.110.149; Authentication-Results: mx.google.com; spf=pass (google.com: domain of paulmck@linux.vnet.ibm.com designates 32.97.110.149 as permitted sender) smtp.mail=paulmck@linux.vnet.ibm.com Received: from /spool/local by e31.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 30 Aug 2012 15:14:11 -0600 Received: from d03dlp03.boulder.ibm.com (9.17.202.179) by e31.co.us.ibm.com (192.168.1.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 30 Aug 2012 15:14:09 -0600 Received: from d03relay03.boulder.ibm.com (d03relay03.boulder.ibm.com [9.17.195.228]) by d03dlp03.boulder.ibm.com (Postfix) with ESMTP id A833419D8051 for ; Thu, 30 Aug 2012 15:14:08 -0600 (MDT) Received: from d03av01.boulder.ibm.com (d03av01.boulder.ibm.com [9.17.195.167]) by d03relay03.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q7ULE6dp038326 for ; Thu, 30 Aug 2012 15:14:06 -0600 Received: from d03av01.boulder.ibm.com (loopback [127.0.0.1]) by d03av01.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q7ULDYCE009287 for ; Thu, 30 Aug 2012 15:13:44 -0600 Received: from paulmck-ThinkPad-W500 (sig-9-65-153-225.mts.ibm.com [9.65.153.225]) by d03av01.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q7ULDY9T009162; Thu, 30 Aug 2012 15:13:34 -0600 Received: by paulmck-ThinkPad-W500 (Postfix, from userid 1000) id A76C3EA83A; Thu, 30 Aug 2012 14:05:45 -0700 (PDT) From: "Paul E. McKenney" To: linux-kernel@vger.kernel.org Cc: mingo@elte.hu, laijs@cn.fujitsu.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@polymtl.ca, josh@joshtriplett.org, niv@us.ibm.com, tglx@linutronix.de, peterz@infradead.org, rostedt@goodmis.org, Valdis.Kletnieks@vt.edu, dhowells@redhat.com, eric.dumazet@gmail.com, darren@dvhart.com, fweisbec@gmail.com, sbw@mit.edu, patches@linaro.org, "Paul E. McKenney" Subject: [PATCH tip/core/rcu 02/26] rcu: New rcu_user_enter_irq() and rcu_user_exit_irq() APIs Date: Thu, 30 Aug 2012 14:05:19 -0700 Message-Id: <1346360743-3628-2-git-send-email-paulmck@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.8 In-Reply-To: <1346360743-3628-1-git-send-email-paulmck@linux.vnet.ibm.com> References: <20120830210520.GA2824@linux.vnet.ibm.com> <1346360743-3628-1-git-send-email-paulmck@linux.vnet.ibm.com> X-Content-Scanned: Fidelis XPS MAILER x-cbid: 12083021-7282-0000-0000-00000C766137 X-Gm-Message-State: ALoCoQkQq1J/2sfW1iICKo3lz8jWdmRN6ioDbXbE4yXdqIrPyPl8zZI5Wn7f3v385B6yIMKyEwi/ From: Frederic Weisbecker In some cases, it is necessary to enter or exit userspace-RCU-idle mode from an interrupt handler, for example, if some other CPU sends this CPU a resched IPI. In this case, the current CPU would enter the IPI handler in userspace-RCU-idle mode, but would need to exit the IPI handler after having exited that mode. To allow this to work, this commit adds two new APIs to TREE_RCU: - rcu_user_enter_irq(). This must be called from an interrupt between rcu_irq_enter() and rcu_irq_exit(). After the irq calls rcu_irq_exit(), the irq handler will return into an RCU extended quiescent state. In theory, this interrupt is never a nested interrupt, but in practice it might interrupt softirq, which looks to RCU like a nested interrupt. - rcu_user_exit_irq(). This must be called from a non-nesting interrupt, interrupting an RCU extended quiescent state, also between rcu_irq_enter() and rcu_irq_exit(). After the irq calls rcu_irq_exit(), the irq handler will return in an RCU non-quiescent state. [ Combined with "Allow calls to rcu_exit_user_irq from nesting irqs." ] Signed-off-by: Frederic Weisbecker Signed-off-by: Paul E. McKenney --- include/linux/rcupdate.h | 2 ++ kernel/rcutree.c | 43 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+), 0 deletions(-) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 2a7549c..81d3d5c 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -193,6 +193,8 @@ extern void rcu_irq_enter(void); extern void rcu_irq_exit(void); extern void rcu_user_enter(void); extern void rcu_user_exit(void); +extern void rcu_user_enter_irq(void); +extern void rcu_user_exit_irq(void); extern void exit_rcu(void); /** diff --git a/kernel/rcutree.c b/kernel/rcutree.c index c0507b7..8fdea17 100644 --- a/kernel/rcutree.c +++ b/kernel/rcutree.c @@ -440,6 +440,27 @@ EXPORT_SYMBOL_GPL(rcu_user_enter); /** + * rcu_user_enter_irq - inform RCU that we are going to resume userspace + * after the current irq returns. + * + * This is similar to rcu_user_enter() but in the context of a non-nesting + * irq. After this call, RCU enters into idle mode when the interrupt + * returns. + */ +void rcu_user_enter_irq(void) +{ + unsigned long flags; + struct rcu_dynticks *rdtp; + + local_irq_save(flags); + rdtp = &__get_cpu_var(rcu_dynticks); + /* Ensure this irq is interrupting a non-idle RCU state. */ + WARN_ON_ONCE(!(rdtp->dynticks_nesting & DYNTICK_TASK_MASK)); + rdtp->dynticks_nesting = 1; + local_irq_restore(flags); +} + +/** * rcu_irq_exit - inform RCU that current CPU is exiting irq towards idle * * Exit from an interrupt handler, which might possibly result in entering @@ -554,6 +575,28 @@ void rcu_user_exit(void) EXPORT_SYMBOL_GPL(rcu_user_exit); /** + * rcu_user_exit_irq - inform RCU that we won't resume to userspace + * idle mode after the current non-nesting irq returns. + * + * This is similar to rcu_user_exit() but in the context of an irq. + * This is called when the irq has interrupted a userspace RCU idle mode + * context. When the current non-nesting interrupt returns after this call, + * the CPU won't restore the RCU idle mode. + */ +void rcu_user_exit_irq(void) +{ + unsigned long flags; + struct rcu_dynticks *rdtp; + + local_irq_save(flags); + rdtp = &__get_cpu_var(rcu_dynticks); + /* Ensure we are interrupting an RCU idle mode. */ + WARN_ON_ONCE(rdtp->dynticks_nesting & DYNTICK_TASK_NEST_MASK); + rdtp->dynticks_nesting += DYNTICK_TASK_EXIT_IDLE; + local_irq_restore(flags); +} + +/** * rcu_irq_enter - inform RCU that current CPU is entering irq away from idle * * Enter an interrupt handler, which might possibly result in exiting