| Message ID | 1422990417-1783-2-git-send-email-daniel.thompson@linaro.org |
|---|---|
| State | Superseded |
| Headers | show |
On Tue, 3 Feb 2015, Daniel Thompson wrote: > Currently gic_raise_softirq() is locked using upon irq_controller_lock. > This lock is primarily used to make register read-modify-write sequences > atomic but gic_raise_softirq() uses it instead to ensure that the > big.LITTLE migration logic can figure out when it is safe to migrate > interrupts between physical cores. > > This is sub-optimal in closely related ways: > > 1. No locking at all is required on systems where the b.L switcher is > not configured. ACK > 2. Finer grain locking can be used on systems where the b.L switcher is > present. NAK Consider this sequence: CPU 1 CPU 2 ----- ----- gic_raise_softirq() gic_migrate_target() bl_migration_lock() [OK] [...] [...] map |= gic_cpu_map[cpu]; bl_migration_lock() [contended] bl_migration_unlock(flags); bl_migration_lock() [OK] gic_cpu_map[cpu] = 1 << new_cpu_id; bl_migration_unlock(flags); [...] (migrate pending IPI from old CPU) writel_relaxed(map to GIC_DIST_SOFTINT); [this IPI is now lost] Granted, this race is apparently aready possible today. We probably get away with it because the locked sequence in gic_migrate_target() include the retargetting of peripheral interrupts which gives plenti of time for code execution in gic_raise_softirq() to post its IPI before the IPI migration code is executed. So in that sense it could be argued that the reduced lock coverage from your patch doesn't make things any worse. If anything it might even help by letting gic_migrate_target() complete sooner. But removing cpu_map_migration_lock altogether would improve things even further by that logic. I however don't think we should live so dangerously. Therefore, for the lock to be effective, it has to encompass the changing of the CPU map _and_ migration of pending IPIs before new IPIs are allowed again. That means the locked area has to grow not shrink. Oh, and a minor nit: > + * This lock is used by the big.LITTLE migration code to ensure no IPIs > + * can be pended on the old core after the map has been updated. > + */ > +#ifdef CONFIG_BL_SWITCHER > +static DEFINE_RAW_SPINLOCK(cpu_map_migration_lock); > + > +static inline void bl_migration_lock(unsigned long *flags) Please name it gic_migration_lock. "bl_migration_lock" is a bit too generic in this context. Nicolas
On Thu, 2015-02-26 at 15:31 -0500, Nicolas Pitre wrote: > On Tue, 3 Feb 2015, Daniel Thompson wrote: > > > Currently gic_raise_softirq() is locked using upon irq_controller_lock. > > This lock is primarily used to make register read-modify-write sequences > > atomic but gic_raise_softirq() uses it instead to ensure that the > > big.LITTLE migration logic can figure out when it is safe to migrate > > interrupts between physical cores. > > > > This is sub-optimal in closely related ways: > > > > 1. No locking at all is required on systems where the b.L switcher is > > not configured. > > ACK > > > 2. Finer grain locking can be used on systems where the b.L switcher is > > present. > > NAK > > Consider this sequence: > > CPU 1 CPU 2 > ----- ----- > gic_raise_softirq() gic_migrate_target() > bl_migration_lock() [OK] > [...] [...] > map |= gic_cpu_map[cpu]; bl_migration_lock() [contended] > bl_migration_unlock(flags); bl_migration_lock() [OK] > gic_cpu_map[cpu] = 1 << new_cpu_id; > bl_migration_unlock(flags); > [...] > (migrate pending IPI from old CPU) > writel_relaxed(map to GIC_DIST_SOFTINT); Isn't this solved inside gic_raise_softirq? How can the writel_relaxed() escape from the critical section and happen at the end of the sequence? > [this IPI is now lost] > > Granted, this race is apparently aready possible today. We probably get > away with it because the locked sequence in gic_migrate_target() include > the retargetting of peripheral interrupts which gives plenti of time for > code execution in gic_raise_softirq() to post its IPI before the IPI > migration code is executed. So in that sense it could be argued that > the reduced lock coverage from your patch doesn't make things any worse. > If anything it might even help by letting gic_migrate_target() complete > sooner. But removing cpu_map_migration_lock altogether would improve > things even further by that logic. I however don't think we should live > so dangerously. > > Therefore, for the lock to be effective, it has to encompass the > changing of the CPU map _and_ migration of pending IPIs before new IPIs > are allowed again. That means the locked area has to grow not shrink. > > Oh, and a minor nit: > > > + * This lock is used by the big.LITTLE migration code to ensure no IPIs > > + * can be pended on the old core after the map has been updated. > > + */ > > +#ifdef CONFIG_BL_SWITCHER > > +static DEFINE_RAW_SPINLOCK(cpu_map_migration_lock); > > + > > +static inline void bl_migration_lock(unsigned long *flags) > > Please name it gic_migration_lock. "bl_migration_lock" is a bit too > generic in this context. I'll change this. Daniel.
On Thu, 26 Feb 2015, Daniel Thompson wrote: > On Thu, 2015-02-26 at 15:31 -0500, Nicolas Pitre wrote: > > On Tue, 3 Feb 2015, Daniel Thompson wrote: > > > > > Currently gic_raise_softirq() is locked using upon irq_controller_lock. > > > This lock is primarily used to make register read-modify-write sequences > > > atomic but gic_raise_softirq() uses it instead to ensure that the > > > big.LITTLE migration logic can figure out when it is safe to migrate > > > interrupts between physical cores. > > > > > > This is sub-optimal in closely related ways: > > > > > > 1. No locking at all is required on systems where the b.L switcher is > > > not configured. > > > > ACK > > > > > 2. Finer grain locking can be used on systems where the b.L switcher is > > > present. > > > > NAK > > > > Consider this sequence: > > > > CPU 1 CPU 2 > > ----- ----- > > gic_raise_softirq() gic_migrate_target() > > bl_migration_lock() [OK] > > [...] [...] > > map |= gic_cpu_map[cpu]; bl_migration_lock() [contended] > > bl_migration_unlock(flags); bl_migration_lock() [OK] > > gic_cpu_map[cpu] = 1 << new_cpu_id; > > bl_migration_unlock(flags); > > [...] > > (migrate pending IPI from old CPU) > > writel_relaxed(map to GIC_DIST_SOFTINT); > > Isn't this solved inside gic_raise_softirq? How can the writel_relaxed() > escape from the critical section and happen at the end of the sequence? Hmmm... blah. OK I obviously can't read today. The patch is fine of course. > > Oh, and a minor nit: > > > > > + * This lock is used by the big.LITTLE migration code to ensure no IPIs > > > + * can be pended on the old core after the map has been updated. > > > + */ > > > +#ifdef CONFIG_BL_SWITCHER > > > +static DEFINE_RAW_SPINLOCK(cpu_map_migration_lock); > > > + > > > +static inline void bl_migration_lock(unsigned long *flags) > > > > Please name it gic_migration_lock. "bl_migration_lock" is a bit too > > generic in this context. > > I'll change this. Good. You may add my ACK. Nicolas
diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c index d617ee5a3d8a..a9ed64dcc84b 100644 --- a/drivers/irqchip/irq-gic.c +++ b/drivers/irqchip/irq-gic.c @@ -73,6 +73,27 @@ struct gic_chip_data { static DEFINE_RAW_SPINLOCK(irq_controller_lock); /* + * This lock is used by the big.LITTLE migration code to ensure no IPIs + * can be pended on the old core after the map has been updated. + */ +#ifdef CONFIG_BL_SWITCHER +static DEFINE_RAW_SPINLOCK(cpu_map_migration_lock); + +static inline void bl_migration_lock(unsigned long *flags) +{ + raw_spin_lock_irqsave(&cpu_map_migration_lock, *flags); +} + +static inline void bl_migration_unlock(unsigned long flags) +{ + raw_spin_unlock_irqrestore(&cpu_map_migration_lock, flags); +} +#else +static inline void bl_migration_lock(unsigned long *flags) {} +static inline void bl_migration_unlock(unsigned long flags) {} +#endif + +/* * The GIC mapping of CPU interfaces does not necessarily match * the logical CPU numbering. Let's use a mapping as returned * by the GIC itself. @@ -624,7 +645,7 @@ static void gic_raise_softirq(const struct cpumask *mask, unsigned int irq) int cpu; unsigned long flags, map = 0; - raw_spin_lock_irqsave(&irq_controller_lock, flags); + bl_migration_lock(&flags); /* Convert our logical CPU mask into a physical one. */ for_each_cpu(cpu, mask) @@ -639,7 +660,7 @@ static void gic_raise_softirq(const struct cpumask *mask, unsigned int irq) /* this always happens on GIC0 */ writel_relaxed(map << 16 | irq, gic_data_dist_base(&gic_data[0]) + GIC_DIST_SOFTINT); - raw_spin_unlock_irqrestore(&irq_controller_lock, flags); + bl_migration_unlock(flags); } #endif @@ -710,8 +731,17 @@ void gic_migrate_target(unsigned int new_cpu_id) raw_spin_lock(&irq_controller_lock); - /* Update the target interface for this logical CPU */ + /* + * Update the target interface for this logical CPU + * + * From the point we release the cpu_map_migration_lock any new + * SGIs will be pended on the new cpu which makes the set of SGIs + * pending on the old cpu static. That means we can defer the + * migration until after we have released the irq_controller_lock. + */ + raw_spin_lock(&cpu_map_migration_lock); gic_cpu_map[cpu] = 1 << new_cpu_id; + raw_spin_unlock(&cpu_map_migration_lock); /* * Find all the peripheral interrupts targetting the current
Currently gic_raise_softirq() is locked using upon irq_controller_lock. This lock is primarily used to make register read-modify-write sequences atomic but gic_raise_softirq() uses it instead to ensure that the big.LITTLE migration logic can figure out when it is safe to migrate interrupts between physical cores. This is sub-optimal in closely related ways: 1. No locking at all is required on systems where the b.L switcher is not configured. 2. Finer grain locking can be used on systems where the b.L switcher is present. This patch resolves both of the above by introducing a separate finer grain lock and providing conditionally compiled inlines to lock/unlock it. Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Russell King <linux@arm.linux.org.uk> Cc: Marc Zyngier <marc.zyngier@arm.com> --- drivers/irqchip/irq-gic.c | 36 +++++++++++++++++++++++++++++++++--- 1 file changed, 33 insertions(+), 3 deletions(-)