| Message ID | 20230620140625.1001886-6-longman@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | x86/speculation: Disable IBRS when idle | expand |
On Tue, Jun 20, 2023 at 10:06:25AM -0400, Waiman Long wrote: > When a user sets "intel_idle.max_cstate=0", it will disable > intel_idle and fall back to acpi_idle instead. The acpi_idle code > will then call mwait_idle_with_hints() to enter idle state. So when > X86_FEATURE_KERNEL_IBRS is enabled, it is necessary to disable IBRS > within mwait_idle_with_hints() when IRQ was disabled to avoid performance > degradation on silbing thread running user workload. Urgh, no, just no. This is nasty.
On 6/21/23 03:32, Peter Zijlstra wrote: > On Tue, Jun 20, 2023 at 10:06:25AM -0400, Waiman Long wrote: >> When a user sets "intel_idle.max_cstate=0", it will disable >> intel_idle and fall back to acpi_idle instead. The acpi_idle code >> will then call mwait_idle_with_hints() to enter idle state. So when >> X86_FEATURE_KERNEL_IBRS is enabled, it is necessary to disable IBRS >> within mwait_idle_with_hints() when IRQ was disabled to avoid performance >> degradation on silbing thread running user workload. > Urgh, no, just no. This is nasty. OK, will take this out in the next version. Cheers, Longman
diff --git a/arch/x86/include/asm/mwait.h b/arch/x86/include/asm/mwait.h index 778df05f8539..1e36cdc21661 100644 --- a/arch/x86/include/asm/mwait.h +++ b/arch/x86/include/asm/mwait.h @@ -108,15 +108,32 @@ static __always_inline void __sti_mwait(unsigned long eax, unsigned long ecx) static __always_inline void mwait_idle_with_hints(unsigned long eax, unsigned long ecx) { if (static_cpu_has_bug(X86_BUG_MONITOR) || !current_set_polling_and_test()) { + bool ibrs_disabled = false; + u64 spec_ctrl; + if (static_cpu_has_bug(X86_BUG_CLFLUSH_MONITOR)) { mb(); clflush((void *)¤t_thread_info()->flags); mb(); } + if (irqs_disabled() && + cpu_feature_enabled(X86_FEATURE_KERNEL_IBRS)) { + /* NMI always enable IBRS on exception entry */ + ibrs_disabled = true; + spec_ctrl = spec_ctrl_current(); + __this_cpu_write(x86_spec_ctrl_current, 0); + native_wrmsrl(MSR_IA32_SPEC_CTRL, 0); + } + __monitor((void *)¤t_thread_info()->flags, 0, 0); if (!need_resched()) __mwait(eax, ecx); + + if (ibrs_disabled) { + native_wrmsrl(MSR_IA32_SPEC_CTRL, spec_ctrl); + __this_cpu_write(x86_spec_ctrl_current, spec_ctrl); + } } current_clr_polling(); }
When a user sets "intel_idle.max_cstate=0", it will disable intel_idle and fall back to acpi_idle instead. The acpi_idle code will then call mwait_idle_with_hints() to enter idle state. So when X86_FEATURE_KERNEL_IBRS is enabled, it is necessary to disable IBRS within mwait_idle_with_hints() when IRQ was disabled to avoid performance degradation on silbing thread running user workload. Signed-off-by: Waiman Long <longman@redhat.com> --- arch/x86/include/asm/mwait.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)