[tip/core/rcu,02/26] rcu: New rcu_user_enter_irq() and rcu_user_exit_irq() APIs

Message ID 1346360743-3628-2-git-send-email-paulmck@linux.vnet.ibm.com
State New
Headers show

Commit Message

Paul E. McKenney Aug. 30, 2012, 9:05 p.m.
From: Frederic Weisbecker <fweisbec@gmail.com>

In some cases, it is necessary to enter or exit userspace-RCU-idle mode
from an interrupt handler, for example, if some other CPU sends this
CPU a resched IPI.  In this case, the current CPU would enter the IPI
handler in userspace-RCU-idle mode, but would need to exit the IPI handler
after having exited that mode.

To allow this to work, this commit adds two new APIs to TREE_RCU:

- rcu_user_enter_irq(). This must be called from an interrupt between
rcu_irq_enter() and rcu_irq_exit().  After the irq calls rcu_irq_exit(),
the irq handler will return into an RCU extended quiescent state.
In theory, this interrupt is never a nested interrupt, but in practice
it might interrupt softirq, which looks to RCU like a nested interrupt.

- rcu_user_exit_irq(). This must be called from a non-nesting
interrupt, interrupting an RCU extended quiescent state, also
between rcu_irq_enter() and rcu_irq_exit(). After the irq calls
rcu_irq_exit(), the irq handler will return in an RCU non-quiescent
state.

[ Combined with "Allow calls to rcu_exit_user_irq from nesting irqs." ]

Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 include/linux/rcupdate.h |    2 ++
 kernel/rcutree.c         |   43 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 45 insertions(+), 0 deletions(-)

Comments

Josh Triplett Aug. 31, 2012, 7:13 p.m. | #1
On Thu, Aug 30, 2012 at 02:05:19PM -0700, Paul E. McKenney wrote:
> From: Frederic Weisbecker <fweisbec@gmail.com>
> 
> In some cases, it is necessary to enter or exit userspace-RCU-idle mode
> from an interrupt handler, for example, if some other CPU sends this
> CPU a resched IPI.  In this case, the current CPU would enter the IPI
> handler in userspace-RCU-idle mode, but would need to exit the IPI handler
> after having exited that mode.
> 
> To allow this to work, this commit adds two new APIs to TREE_RCU:
> 
> - rcu_user_enter_irq(). This must be called from an interrupt between
> rcu_irq_enter() and rcu_irq_exit().  After the irq calls rcu_irq_exit(),
> the irq handler will return into an RCU extended quiescent state.
> In theory, this interrupt is never a nested interrupt, but in practice
> it might interrupt softirq, which looks to RCU like a nested interrupt.
> 
> - rcu_user_exit_irq(). This must be called from a non-nesting
> interrupt, interrupting an RCU extended quiescent state, also
> between rcu_irq_enter() and rcu_irq_exit(). After the irq calls
> rcu_irq_exit(), the irq handler will return in an RCU non-quiescent
> state.

These names seem a bit confusing.  From the descriptions, it sounds like
you don't always need to pair them; rcu_irq_exit() will return to a
non-quiescent state, unless you call rcu_user_enter_irq and *don't* call
rcu_user_exit_irq.  Did I get that semantic right?

Given that, the "enter" and "exit" names seem confusing.  This seems
more like a flag you can set and clear, rather than a delimited region
as suggested by an enter/exit pair.

How about something vaguely like rcu_user_irq_set_eqs and
rcu_user_irq_clear_eqs?

- Josh Triplett
Frederic Weisbecker Aug. 31, 2012, 7:54 p.m. | #2
2012/8/31 Josh Triplett <josh@joshtriplett.org>:
> On Thu, Aug 30, 2012 at 02:05:19PM -0700, Paul E. McKenney wrote:
>> From: Frederic Weisbecker <fweisbec@gmail.com>
>>
>> In some cases, it is necessary to enter or exit userspace-RCU-idle mode
>> from an interrupt handler, for example, if some other CPU sends this
>> CPU a resched IPI.  In this case, the current CPU would enter the IPI
>> handler in userspace-RCU-idle mode, but would need to exit the IPI handler
>> after having exited that mode.
>>
>> To allow this to work, this commit adds two new APIs to TREE_RCU:
>>
>> - rcu_user_enter_irq(). This must be called from an interrupt between
>> rcu_irq_enter() and rcu_irq_exit().  After the irq calls rcu_irq_exit(),
>> the irq handler will return into an RCU extended quiescent state.
>> In theory, this interrupt is never a nested interrupt, but in practice
>> it might interrupt softirq, which looks to RCU like a nested interrupt.
>>
>> - rcu_user_exit_irq(). This must be called from a non-nesting
>> interrupt, interrupting an RCU extended quiescent state, also
>> between rcu_irq_enter() and rcu_irq_exit(). After the irq calls
>> rcu_irq_exit(), the irq handler will return in an RCU non-quiescent
>> state.
>
> These names seem a bit confusing.  From the descriptions, it sounds like
> you don't always need to pair them; rcu_irq_exit() will return to a
> non-quiescent state, unless you call rcu_user_enter_irq and *don't* call
> rcu_user_exit_irq.  Did I get that semantic right?

Yeah. They indeed don't always need to be paired. We can enter into
user (from rcu POV) with rcu_user_enter_irq() and exit user with
rcu_user_exit().

It's just a matter of context: from where do we set/unset RCU as in
user mode: irq or not. The only thing that is paired is the fact we
enter/exit that RCU user mode. There are just different APIs to do so.

> Given that, the "enter" and "exit" names seem confusing.  This seems
> more like a flag you can set and clear, rather than a delimited region
> as suggested by an enter/exit pair.
>
> How about something vaguely like rcu_user_irq_set_eqs and
> rcu_user_irq_clear_eqs?

I'd rather suggest rcu_user_enter_after_irq and
rcu_user_exit_after_irq. It describes precisely what it does.
>
> - Josh Triplett
Josh Triplett Aug. 31, 2012, 9:38 p.m. | #3
On Fri, Aug 31, 2012 at 09:54:39PM +0200, Frederic Weisbecker wrote:
> 2012/8/31 Josh Triplett <josh@joshtriplett.org>:
> > Given that, the "enter" and "exit" names seem confusing.  This seems
> > more like a flag you can set and clear, rather than a delimited region
> > as suggested by an enter/exit pair.
> >
> > How about something vaguely like rcu_user_irq_set_eqs and
> > rcu_user_irq_clear_eqs?
> 
> I'd rather suggest rcu_user_enter_after_irq and
> rcu_user_exit_after_irq. It describes precisely what it does.

Those names sound reasonable, sure; in the context of "after",
enter/exit sounds less confusing.

- Josh Triplett

Patch

diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
index 2a7549c..81d3d5c 100644
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -193,6 +193,8 @@  extern void rcu_irq_enter(void);
 extern void rcu_irq_exit(void);
 extern void rcu_user_enter(void);
 extern void rcu_user_exit(void);
+extern void rcu_user_enter_irq(void);
+extern void rcu_user_exit_irq(void);
 extern void exit_rcu(void);
 
 /**
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index c0507b7..8fdea17 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -440,6 +440,27 @@  EXPORT_SYMBOL_GPL(rcu_user_enter);
 
 
 /**
+ * rcu_user_enter_irq - inform RCU that we are going to resume userspace
+ * after the current irq returns.
+ *
+ * This is similar to rcu_user_enter() but in the context of a non-nesting
+ * irq. After this call, RCU enters into idle mode when the interrupt
+ * returns.
+ */
+void rcu_user_enter_irq(void)
+{
+	unsigned long flags;
+	struct rcu_dynticks *rdtp;
+
+	local_irq_save(flags);
+	rdtp = &__get_cpu_var(rcu_dynticks);
+	/* Ensure this irq is interrupting a non-idle RCU state.  */
+	WARN_ON_ONCE(!(rdtp->dynticks_nesting & DYNTICK_TASK_MASK));
+	rdtp->dynticks_nesting = 1;
+	local_irq_restore(flags);
+}
+
+/**
  * rcu_irq_exit - inform RCU that current CPU is exiting irq towards idle
  *
  * Exit from an interrupt handler, which might possibly result in entering
@@ -554,6 +575,28 @@  void rcu_user_exit(void)
 EXPORT_SYMBOL_GPL(rcu_user_exit);
 
 /**
+ * rcu_user_exit_irq - inform RCU that we won't resume to userspace
+ * idle mode after the current non-nesting irq returns.
+ *
+ * This is similar to rcu_user_exit() but in the context of an irq.
+ * This is called when the irq has interrupted a userspace RCU idle mode
+ * context. When the current non-nesting interrupt returns after this call,
+ * the CPU won't restore the RCU idle mode.
+ */
+void rcu_user_exit_irq(void)
+{
+	unsigned long flags;
+	struct rcu_dynticks *rdtp;
+
+	local_irq_save(flags);
+	rdtp = &__get_cpu_var(rcu_dynticks);
+	/* Ensure we are interrupting an RCU idle mode. */
+	WARN_ON_ONCE(rdtp->dynticks_nesting & DYNTICK_TASK_NEST_MASK);
+	rdtp->dynticks_nesting += DYNTICK_TASK_EXIT_IDLE;
+	local_irq_restore(flags);
+}
+
+/**
  * rcu_irq_enter - inform RCU that current CPU is entering irq away from idle
  *
  * Enter an interrupt handler, which might possibly result in exiting