From patchwork Thu Jan 2 15:01:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 854809 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C58511B413B; Thu, 2 Jan 2025 15:02:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735830130; cv=none; b=Q+4vPgqKhcD7WW+PNswohAgvojqep6vIAmuAIhtW/LR9xr5PeyUa6RLATmBjx1kwr9GjWvGe9QBSsEq1sDgGxNEbArPLeGxuxnBxhnbz0uiholmn0Ly8/uL/7+7vPrpjfX5U2u8D6Fqf/9uzlsSB9RjjHJ6fC2rm7VMrbw0TfNY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735830130; c=relaxed/simple; bh=6M3MwJVPQVJfwZCyUXnKFfzUiBOjUgHdAX5hBQtJ7Q0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nARbfYI+X2Z+hWDcq7T/ASQynDg7G0yH6u3zW70XyacVsQbh+UiqGpMVW0d37YQWjgLkkzN6ByGgZgG8CXu6jMiZ/tqRVH85a/5GLvNa0ofmCgb5yIHsfMoVM/da5fvm766geyw1rBOl1nKhQRUz5Er8AG9HhKKrQvTRa0aIS8s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=L63/ynnU; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="L63/ynnU" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 24548C4CED7; Thu, 2 Jan 2025 15:02:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735830130; bh=6M3MwJVPQVJfwZCyUXnKFfzUiBOjUgHdAX5hBQtJ7Q0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=L63/ynnUEauCbVUAEBzIyL8lnuVzbb6Q6MQwj/R5Qj4zxbUTVGNhAR6BYL59OZa9j PReQL/bPHabbBuhHMfpTFUOziL6CnDFf9bZYRPUc0J6S2PBUsSP2tiQuwkuB+zWRUg t2/nV8Ts607pMLfqadssTmz/4hTUZGNTVgiEsplhGR8qiBB4kGyxo99NH4lR7TTRUc amHMpddZ8QqPLCoicVfitIVrPSyg5XLgt1JMo0LPPPJ+jYA+ZwjFPQlC/Wc1XucIS2 LEYNUGCVJKuVo/cAPMT+Voa1g6ybXWrnoST1nWVWUNqMAfjItkqu9Jcqf0MxGcZRWK 5tQSVEIelL/SA== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , "Rafael J . Wysocki" , Daniel Lezcano , linux-pm@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Marcelo Tosatti Subject: [PATCH 1/6] cpuidle: Remove unnecessary current_clr_polling_and_test() from haltpoll Date: Thu, 2 Jan 2025 16:01:55 +0100 Message-ID: <20250102150201.21639-2-frederic@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250102150201.21639-1-frederic@kernel.org> References: <20250102150201.21639-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When cpuidle drivers ->enter() callback are called, the TIF_NR_POLLING flag is cleared already and TIF_NEED_RESCHED checked by call_cpuidle(). Therefore calling current_clr_polling_and_test() is redundant here and further setting of TIF_NEED_RESCHED will result in an IPI and thus an idle loop exit. This call can be safely removed. Cc: Marcelo Tosatti Acked-by: Rafael J. Wysocki Signed-off-by: Frederic Weisbecker --- drivers/cpuidle/cpuidle-haltpoll.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/drivers/cpuidle/cpuidle-haltpoll.c b/drivers/cpuidle/cpuidle-haltpoll.c index bcd03e893a0a..07dd9f000273 100644 --- a/drivers/cpuidle/cpuidle-haltpoll.c +++ b/drivers/cpuidle/cpuidle-haltpoll.c @@ -28,9 +28,6 @@ static enum cpuhp_state haltpoll_hp_state; static __cpuidle int default_enter_idle(struct cpuidle_device *dev, struct cpuidle_driver *drv, int index) { - if (current_clr_polling_and_test()) - return index; - arch_cpu_idle(); return index; } From patchwork Thu Jan 2 15:01:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 854808 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D546E1B4120; Thu, 2 Jan 2025 15:02:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735830138; cv=none; b=D8lU+5NLTi+DxIh1NZEfISHrzASz47zxYSQ5pn2OuDTZlv9fwV8p+fVtAq6SYsIikR2AJbJDlju92wlyATy0iHeFZEkIh2VgzlJvgUQiTmXmxZYYHgCvM9j/FWSd23wu2pax/d6uDdnY1F5FAHJEFkKRB3n7wvE+0sTqu7jhJ1s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735830138; c=relaxed/simple; bh=F1FmzVp6iCOT8qiM1kjCr6AQCFvi69gi8t0iIwn4s5I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ElFTOrIUP+6o+iC9j7q+rMhvf+w00w+Uz/i+x6k/GhkwcgUvrLbuCYEdO0xbfqcls5NgZB5tWU9cOJu+RdkqcKrFgMzIteoF+SymZhLPUzqS6svM+nMoB07Mp3C7vZhmlGcX89lKPxouQiYCM+xL4/YVAoADsmh1ER9ufDYjtik= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=StxHrsmm; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="StxHrsmm" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D74A6C4CEDD; Thu, 2 Jan 2025 15:02:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735830138; bh=F1FmzVp6iCOT8qiM1kjCr6AQCFvi69gi8t0iIwn4s5I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=StxHrsmmFW/2gj6R3DymlKgYReiQApi2t3zi3CclCNOiILo4eEg/vU4OKWXuPvo67 W6qBi5R1cL28xjBz6dWLY/oCJvJf+2erTsE3GYnQINohLp6oHuJ/rmYVUei1NPTyYL B74GNMHBj+rH6TEPEYB5fRrQ/MQlXabbGDVrhluoWAM7PcooKFdhfL19p+z12ksR1G Qk+hc6C4byaPyg2g3H6JFjB6x84gpPCsodZJzzfXY+uHK14IqCaG0Sp0NjsSAcx23N T6AvpdeOyx/iN4B8Y7U/Q8bd5D5U8R/5Hi34pQ8sgiDiMJiV1G4Y44xFcJR/75711u rRfmZP72UYpZA== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Jacob Pan , Peter Zijlstra , "Rafael J . Wysocki" , Daniel Lezcano , linux-pm@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Russell King , Catalin Marinas , Will Deacon , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Madhavan Srinivasan , Paul Walmsley , Palmer Dabbelt , Albert Ou , Len Brown , Arnd Bergmann , linux-arch@vger.kernel.org Subject: [PATCH 3/6] x86/cpuidle: Move buggy mwait implementations away from CPUIDLE_FLAG_MWAIT Date: Thu, 2 Jan 2025 16:01:57 +0100 Message-ID: <20250102150201.21639-4-frederic@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250102150201.21639-1-frederic@kernel.org> References: <20250102150201.21639-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Buggy MWAIT implementations can't carry the CPUIDLE_FLAG_MWAIT flag because they require IPIs to wake up. Therefore they shouldn't be called with TIF_NR_POLLING. Reported-by: Rafael J. Wysocki Signed-off-by: Frederic Weisbecker --- arch/arm/include/asm/cpuidle.h | 2 ++ arch/arm64/include/asm/cpuidle.h | 3 +++ arch/powerpc/include/asm/cpuidle.h | 4 ++++ arch/riscv/include/asm/cpuidle.h | 2 ++ arch/x86/include/asm/cpuidle.h | 12 ++++++++++++ drivers/acpi/processor_idle.c | 4 +++- drivers/idle/intel_idle.c | 9 +++++++-- include/asm-generic/Kbuild | 1 + include/asm-generic/cpuidle.h | 10 ++++++++++ 9 files changed, 44 insertions(+), 3 deletions(-) create mode 100644 arch/x86/include/asm/cpuidle.h create mode 100644 include/asm-generic/cpuidle.h diff --git a/arch/arm/include/asm/cpuidle.h b/arch/arm/include/asm/cpuidle.h index 397be5ed30e7..0ea1d2ec837d 100644 --- a/arch/arm/include/asm/cpuidle.h +++ b/arch/arm/include/asm/cpuidle.h @@ -55,4 +55,6 @@ struct arm_cpuidle_irq_context { }; #define arm_cpuidle_save_irq_context(c) (void)c #define arm_cpuidle_restore_irq_context(c) (void)c +#include + #endif diff --git a/arch/arm64/include/asm/cpuidle.h b/arch/arm64/include/asm/cpuidle.h index 2047713e097d..ef49124135a7 100644 --- a/arch/arm64/include/asm/cpuidle.h +++ b/arch/arm64/include/asm/cpuidle.h @@ -38,4 +38,7 @@ struct arm_cpuidle_irq_context { }; #define arm_cpuidle_save_irq_context(c) (void)c #define arm_cpuidle_restore_irq_context(c) (void)c #endif + +#include + #endif diff --git a/arch/powerpc/include/asm/cpuidle.h b/arch/powerpc/include/asm/cpuidle.h index 0cce5dc7fb1c..788706bc04ec 100644 --- a/arch/powerpc/include/asm/cpuidle.h +++ b/arch/powerpc/include/asm/cpuidle.h @@ -102,4 +102,8 @@ static inline void report_invalid_psscr_val(u64 psscr_val, int err) #endif +#ifndef __ASSEMBLY__ +#include +#endif + #endif diff --git a/arch/riscv/include/asm/cpuidle.h b/arch/riscv/include/asm/cpuidle.h index 71fdc607d4bc..1f1b24901d86 100644 --- a/arch/riscv/include/asm/cpuidle.h +++ b/arch/riscv/include/asm/cpuidle.h @@ -21,4 +21,6 @@ static inline void cpu_do_idle(void) wait_for_interrupt(); } +#include + #endif diff --git a/arch/x86/include/asm/cpuidle.h b/arch/x86/include/asm/cpuidle.h new file mode 100644 index 000000000000..a59db1a3314a --- /dev/null +++ b/arch/x86/include/asm/cpuidle.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_CPUIDLE_H +#define _ASM_X86_CPUIDLE_H + +#include + +static inline bool arch_cpuidle_mwait_needs_ipi(void) +{ + return boot_cpu_has_bug(X86_BUG_MONITOR); +} + +#endif /* _ASM_X86_CPUIDLE_H */ diff --git a/drivers/acpi/processor_idle.c b/drivers/acpi/processor_idle.c index 66cb5536d91e..0f29dd92b346 100644 --- a/drivers/acpi/processor_idle.c +++ b/drivers/acpi/processor_idle.c @@ -23,6 +23,7 @@ #include #include #include +#include /* * Include the apic definitions for x86 to have the APIC timer related defines @@ -806,7 +807,8 @@ static int acpi_processor_setup_cstates(struct acpi_processor *pr) if (cx->type == ACPI_STATE_C1 || cx->type == ACPI_STATE_C2) drv->safe_state_index = count; - if (cx->entry_method == ACPI_CSTATE_FFH) + if (cx->entry_method == ACPI_CSTATE_FFH && + !arch_cpuidle_mwait_needs_ipi()) state->flags |= CPUIDLE_FLAG_MWAIT; /* diff --git a/drivers/idle/intel_idle.c b/drivers/idle/intel_idle.c index d52723fbeb04..b2f494effd4a 100644 --- a/drivers/idle/intel_idle.c +++ b/drivers/idle/intel_idle.c @@ -56,6 +56,7 @@ #include #include #include +#include #define INTEL_IDLE_VERSION "0.5.1" @@ -1787,7 +1788,10 @@ static void __init intel_idle_init_cstates_acpi(struct cpuidle_driver *drv) if (cx->type > ACPI_STATE_C1) state->target_residency *= 3; - state->flags = MWAIT2flg(cx->address) | CPUIDLE_FLAG_MWAIT; + state->flags = MWAIT2flg(cx->address); + + if (!arch_cpuidle_mwait_needs_ipi()) + state->flags |= CPUIDLE_FLAG_MWAIT; if (cx->type > ACPI_STATE_C2) state->flags |= CPUIDLE_FLAG_TLB_FLUSHED; @@ -2073,7 +2077,8 @@ static bool __init intel_idle_verify_cstate(unsigned int mwait_hint) static void state_update_enter_method(struct cpuidle_state *state, int cstate) { - state->flags |= CPUIDLE_FLAG_MWAIT; + if (!arch_cpuidle_mwait_needs_ipi()) + state->flags |= CPUIDLE_FLAG_MWAIT; if (state->flags & CPUIDLE_FLAG_INIT_XSTATE) { /* diff --git a/include/asm-generic/Kbuild b/include/asm-generic/Kbuild index 1b43c3a77012..7754da499d16 100644 --- a/include/asm-generic/Kbuild +++ b/include/asm-generic/Kbuild @@ -13,6 +13,7 @@ mandatory-y += cacheflush.h mandatory-y += cfi.h mandatory-y += checksum.h mandatory-y += compat.h +mandatory-y += cpuidle.h mandatory-y += current.h mandatory-y += delay.h mandatory-y += device.h diff --git a/include/asm-generic/cpuidle.h b/include/asm-generic/cpuidle.h new file mode 100644 index 000000000000..748a2022ed2a --- /dev/null +++ b/include/asm-generic/cpuidle.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_CPUIDLE_H +#define __ASM_CPUIDLE_H + +static inline bool arch_cpuidle_mwait_needs_ipi(void) +{ + return true; +} + +#endif /* __ASM_CPUIDLE_H */ From patchwork Thu Jan 2 15:01:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 854807 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF1FF1B4137; Thu, 2 Jan 2025 15:02:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735830144; cv=none; b=VaHvH9dzQUYYIahqv62MJR38uA8fPcYc9sJqZQu0uasSJVDdZOmCNyyZ4jkNIki5V9kDsCfmMSgxL2DY6NFSEKOmF3ZhvS75ZgY8d05A8FnMAiUxvCLg8uAYxbGFxRIB3yNd3iJoJ/1pOhGCyJ68IKmvcLO/vdh5FUO+3u95mYY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735830144; c=relaxed/simple; bh=3JoSvgVSyWPdcDhFxJbqeFmv3Z/rfrPtZzh7qZxBCX8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=R1SS52VgUroEt4Vuom4jsH1g/Vl3lOitetHgQdqdQqb3iqcW5/cpgGKtojj/ecSUAoY3iMHKO0qIPV1g7xl+bOQosjoWHnlW8WWJvdk58u0Dp6/OSy/alb+g4xByt0/K1aukoKw09Tm/EEE9MXAzEWqkifmooI9SEyWrlGENlPs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Mv6syPWb; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Mv6syPWb" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 762CBC4CEDC; Thu, 2 Jan 2025 15:02:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735830143; bh=3JoSvgVSyWPdcDhFxJbqeFmv3Z/rfrPtZzh7qZxBCX8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Mv6syPWbiN7Tx5+Yoc4UulxCwUKy4O0bmvLn47nwBqpuOJV3bDPky3UCqfVa00XlZ kuEZMFBO8Q3xLU7yp4lhywRa3sluwbmEfxQeEX37PBU2xkbfewGM/OMC3/JG2NvFAw T0FydsonY8VHD59oNfaO2Gpn68BnxMlTdpvnacSBZa18MYX7fpgEDvyELBkrtOHOaK hEtKr4uI3fyfgOyMrSO2NyU3A125yipjINU232/G4cIE+PkBNgsTTnMcasPxw4uXrq wtHo0PacbNXJtsbqC24dgvA0kzat8tn3sSnrAJnEs2i8i3nZENuoBunLP0SMRTsZ/X QmTtadfXX2y0A== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Peter Zijlstra , "Rafael J . Wysocki" , Daniel Lezcano , linux-pm@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "Rafael J. Wysocki" Subject: [PATCH 5/6] cpuidle: Remove call_cpuidle_s2idle() Date: Thu, 2 Jan 2025 16:01:59 +0100 Message-ID: <20250102150201.21639-6-frederic@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250102150201.21639-1-frederic@kernel.org> References: <20250102150201.21639-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This middle call is unecessary, especially now that its counterpart call_cpuidle() has been removed. Suggested-by: Rafael J. Wysocki Signed-off-by: Frederic Weisbecker --- kernel/sched/idle.c | 23 ++++++----------------- 1 file changed, 6 insertions(+), 17 deletions(-) diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index 9eece3df1080..86b902eb24fe 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -126,19 +126,6 @@ void __cpuidle default_idle_call(void) instrumentation_end(); } -static int call_cpuidle_s2idle(struct cpuidle_driver *drv, - struct cpuidle_device *dev) -{ - int ret; - - if (current_clr_polling_and_test()) - return -EBUSY; - - ret = cpuidle_enter_s2idle(drv, dev); - __current_set_polling(); - return ret; -} - /** * cpuidle_idle_call - the main idle function * @@ -184,10 +171,12 @@ static void cpuidle_idle_call(void) u64 max_latency_ns; if (idle_should_enter_s2idle()) { - - entered_state = call_cpuidle_s2idle(drv, dev); - if (entered_state > 0) - goto exit_idle; + if (!current_clr_polling_and_test()) { + entered_state = cpuidle_enter_s2idle(drv, dev); + __current_set_polling(); + if (entered_state > 0) + goto exit_idle; + } max_latency_ns = U64_MAX; } else {