diff mbox

[tip/core/rcu,08/23] rcu: Segregate rcu_state fields to improve cache locality

Message ID 1346350718-30937-8-git-send-email-paulmck@linux.vnet.ibm.com
State Superseded
Headers show

Commit Message

Paul E. McKenney Aug. 30, 2012, 6:18 p.m. UTC 
From: Dimitri Sivanich <sivanich@sgi.com>

The fields in the rcu_state structure that are protected by the
root rcu_node structure's ->lock can share a cache line with the
fields protected by ->onofflock.  This can result in excessive
memory contention on large systems, so this commit applies
____cacheline_internodealigned_in_smp to the ->onofflock field in
order to segregate them.

Signed-off-by: Dimitri Sivanich <sivanich@sgi.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Dimitri Sivanich <sivanich@sgi.com>
---
 kernel/rcutree.h |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

Comments

Josh Triplett Sept. 2, 2012, 2:51 a.m. UTC | #1 
On Thu, Aug 30, 2012 at 11:18:23AM -0700, Paul E. McKenney wrote:
> From: Dimitri Sivanich <sivanich@sgi.com>
> 
> The fields in the rcu_state structure that are protected by the
> root rcu_node structure's ->lock can share a cache line with the
> fields protected by ->onofflock.  This can result in excessive
> memory contention on large systems, so this commit applies
> ____cacheline_internodealigned_in_smp to the ->onofflock field in
> order to segregate them.
> 
> Signed-off-by: Dimitri Sivanich <sivanich@sgi.com>
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Tested-by: Dimitri Sivanich <sivanich@sgi.com>

Reviewed-by: Josh Triplett <josh@joshtriplett.org>

>  kernel/rcutree.h |    3 ++-
>  1 files changed, 2 insertions(+), 1 deletions(-)
> 
> diff --git a/kernel/rcutree.h b/kernel/rcutree.h
> index effb273..5d92b80 100644
> --- a/kernel/rcutree.h
> +++ b/kernel/rcutree.h
> @@ -394,7 +394,8 @@ struct rcu_state {
>  
>  	/* End of fields guarded by root rcu_node's lock. */
>  
> -	raw_spinlock_t onofflock;		/* exclude on/offline and */
> +	raw_spinlock_t onofflock ____cacheline_internodealigned_in_smp;
> +						/* exclude on/offline and */
>  						/*  starting new GP. */
>  	struct rcu_head *orphan_nxtlist;	/* Orphaned callbacks that */
>  						/*  need a grace period. */
> -- 
> 1.7.8
>
diff mbox

Patch

diff --git a/kernel/rcutree.h b/kernel/rcutree.h
index effb273..5d92b80 100644
--- a/kernel/rcutree.h
+++ b/kernel/rcutree.h
@@ -394,7 +394,8 @@  struct rcu_state {
 
 	/* End of fields guarded by root rcu_node's lock. */
 
-	raw_spinlock_t onofflock;		/* exclude on/offline and */
+	raw_spinlock_t onofflock ____cacheline_internodealigned_in_smp;
+						/* exclude on/offline and */
 						/*  starting new GP. */
 	struct rcu_head *orphan_nxtlist;	/* Orphaned callbacks that */
 						/*  need a grace period. */