| Message ID | 1395841133-2223-5-git-send-email-ian.campbell@citrix.com |
|---|---|
| State | Accepted |
| Commit | 7486348460b58dfff2540506ee1a8df83fc9a2ef |
| Headers | show |
On 03/26/2014 01:38 PM, Ian Campbell wrote: > Almost because I omitting aed3a4e "ARM: 7868/1: arm/arm64: remove > atomic_clear_mask() ..." which I will apply to both arm32 and arm64 > simultaneously in a later patch. > > This pulls in the following Linux patches: > > commit f38d999c4d16fc0fce4270374f15fbb2d8713c09 > Author: Will Deacon <will.deacon@arm.com> > Date: Thu Jul 4 11:43:18 2013 +0100 > > ARM: atomics: prefetch the destination word for write prior to strex > > The cost of changing a cacheline from shared to exclusive state can be > significant, especially when this is triggered by an exclusive store, > since it may result in having to retry the transaction. > > This patch prefixes our atomic access implementations with pldw > instructions (on CPUs which support them) to try and grab the line in > exclusive state from the start. Only the barrier-less functions are > updated, since memory barriers can limit the usefulness of prefetching > data. > > Acked-by: Nicolas Pitre <nico@linaro.org> > Signed-off-by: Will Deacon <will.deacon@arm.com> > > commit 4dcc1cf7316a26e112f5c9fcca531ff98ef44700 > Author: Chen Gang <gang.chen@asianux.com> > Date: Sat Oct 26 15:07:25 2013 +0100 > > ARM: 7867/1: include: asm: use 'int' instead of 'unsigned long' for 'oldval > > For atomic_cmpxchg(), the type of 'oldval' need be 'int' to match the > type of "*ptr" (used by 'ldrex' instruction) and 'old' (used by 'teq' > instruction). > > Reviewed-by: Will Deacon <will.deacon@arm.com> > Signed-off-by: Chen Gang <gang.chen@asianux.com> > Signed-off-by: Will Deacon <will.deacon@arm.com> > Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk> > > Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Julien Grall <julien.grall@linaro.org>
diff --git a/xen/include/asm-arm/arm32/atomic.h b/xen/include/asm-arm/arm32/atomic.h index 3f024d4..d309f66 100644 --- a/xen/include/asm-arm/arm32/atomic.h +++ b/xen/include/asm-arm/arm32/atomic.h @@ -21,6 +21,7 @@ static inline void atomic_add(int i, atomic_t *v) unsigned long tmp; int result; + prefetchw(&v->counter); __asm__ __volatile__("@ atomic_add\n" "1: ldrex %0, [%3]\n" " add %0, %0, %4\n" @@ -59,6 +60,7 @@ static inline void atomic_sub(int i, atomic_t *v) unsigned long tmp; int result; + prefetchw(&v->counter); __asm__ __volatile__("@ atomic_sub\n" "1: ldrex %0, [%3]\n" " sub %0, %0, %4\n" @@ -94,7 +96,8 @@ static inline int atomic_sub_return(int i, atomic_t *v) static inline int atomic_cmpxchg(atomic_t *ptr, int old, int new) { - unsigned long oldval, res; + int oldval; + unsigned long res; smp_mb(); @@ -118,6 +121,7 @@ static inline void atomic_clear_mask(unsigned long mask, unsigned long *addr) { unsigned long tmp, tmp2; + prefetchw(addr); __asm__ __volatile__("@ atomic_clear_mask\n" "1: ldrex %0, [%3]\n" " bic %0, %0, %4\n" diff --git a/xen/include/asm-arm/atomic.h b/xen/include/asm-arm/atomic.h index 69c8f3f..2c92de9 100644 --- a/xen/include/asm-arm/atomic.h +++ b/xen/include/asm-arm/atomic.h @@ -2,6 +2,7 @@ #define __ARCH_ARM_ATOMIC__ #include <xen/config.h> +#include <xen/prefetch.h> #include <asm/system.h> #define build_atomic_read(name, size, width, type, reg)\
Almost because I omitting aed3a4e "ARM: 7868/1: arm/arm64: remove atomic_clear_mask() ..." which I will apply to both arm32 and arm64 simultaneously in a later patch. This pulls in the following Linux patches: commit f38d999c4d16fc0fce4270374f15fbb2d8713c09 Author: Will Deacon <will.deacon@arm.com> Date: Thu Jul 4 11:43:18 2013 +0100 ARM: atomics: prefetch the destination word for write prior to strex The cost of changing a cacheline from shared to exclusive state can be significant, especially when this is triggered by an exclusive store, since it may result in having to retry the transaction. This patch prefixes our atomic access implementations with pldw instructions (on CPUs which support them) to try and grab the line in exclusive state from the start. Only the barrier-less functions are updated, since memory barriers can limit the usefulness of prefetching data. Acked-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Will Deacon <will.deacon@arm.com> commit 4dcc1cf7316a26e112f5c9fcca531ff98ef44700 Author: Chen Gang <gang.chen@asianux.com> Date: Sat Oct 26 15:07:25 2013 +0100 ARM: 7867/1: include: asm: use 'int' instead of 'unsigned long' for 'oldval For atomic_cmpxchg(), the type of 'oldval' need be 'int' to match the type of "*ptr" (used by 'ldrex' instruction) and 'old' (used by 'teq' instruction). Reviewed-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Chen Gang <gang.chen@asianux.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk> Signed-off-by: Ian Campbell <ian.campbell@citrix.com> --- xen/include/asm-arm/arm32/atomic.h | 6 +++++- xen/include/asm-arm/atomic.h | 1 + 2 files changed, 6 insertions(+), 1 deletion(-)