From patchwork Tue Dec 22 01:37:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 346956 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5AD0C433E6 for ; Tue, 22 Dec 2020 01:38:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B1FCA22B3B for ; Tue, 22 Dec 2020 01:38:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725962AbgLVBiY (ORCPT ); Mon, 21 Dec 2020 20:38:24 -0500 Received: from mail.kernel.org ([198.145.29.99]:51696 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725790AbgLVBiY (ORCPT ); Mon, 21 Dec 2020 20:38:24 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 5C91522CB1; Tue, 22 Dec 2020 01:37:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1608601063; bh=6nMfwrsZb2Jqi0tEbsTkov9Zp4J5F8egvQ1o589NL/8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=V3PETQq8GgEpjNXL3hgjKCORR1bqWVLC01PyGhIQE6oouWvDtydhIC7y5bIi4H5qB EwdPjvUMqn7B48JQvOHaLdK4TQkgAYbMXdrr5u93dljjLn+sKHoN6SxJtLKxC6/PtY s0H/qLHxJxIGiFqpdoW4o2rnCUF1yjRiZoS/wK9qDkCqDH58DGD64P8GzusAHYghy9 P6lfZE2J63uRnJIJEwz39AG9SwClsqtpDjcmC64SWEfIdjxLGGWvJZ38Ty4uU6RYgj yDJqEp4Zi88XO1aiZ9hTRKO6v+iYagc4tAoLzsFDhTBITVBexUhGtRlP52Xgqpi1Rv SHQKENQkt/VWA== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , "Rafael J . Wysocki" , Peter Zijlstra , Ingo Molnar , Fabio Estevam , stable@vger.kernel.org, Thomas Gleixner , "Paul E . McKenney" , Len Brown , Pengutronix Kernel Team , NXP Linux Team , Daniel Lezcano , Shawn Guo , Sascha Hauer Subject: [PATCH 1/4] sched/idle: Fix missing need_resched() check after rcu_idle_enter() Date: Tue, 22 Dec 2020 02:37:09 +0100 Message-Id: <20201222013712.15056-2-frederic@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201222013712.15056-1-frederic@kernel.org> References: <20201222013712.15056-1-frederic@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org Entering RCU idle mode may cause a deferred wake up of an RCU NOCB_GP kthread (rcuog) to be serviced. Usually a wake up happening while running the idle task is spotted in one of the need_resched() checks carefully placed within the idle loop that can break to the scheduler. Unfortunately in default_idle_call(), the call to rcu_idle_enter() is already beyond the last need_resched() check and we may halt the CPU with a resched request unhandled, leaving the task hanging. Fix this with performing a last minute need_resched() check after calling rcu_idle_enter(). Reported-by: Paul E. McKenney Fixes: 96d3fd0d315a (rcu: Break call_rcu() deadlock involving scheduler and perf) Cc: stable@vger.kernel.org Cc: Peter Zijlstra Cc: Rafael J. Wysocki Cc: Thomas Gleixner Cc: Ingo Molnar Signed-off-by: Frederic Weisbecker --- kernel/sched/idle.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index 305727ea0677..1af60dc50beb 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -109,15 +109,21 @@ void __cpuidle default_idle_call(void) rcu_idle_enter(); lockdep_hardirqs_on(_THIS_IP_); - arch_cpu_idle(); + /* + * Last need_resched() check must come after rcu_idle_enter() + * which may wake up RCU internal tasks. + */ + if (!need_resched()) { + arch_cpu_idle(); + raw_local_irq_disable(); + } /* - * OK, so IRQs are enabled here, but RCU needs them disabled to - * turn itself back on.. funny thing is that disabling IRQs - * will cause tracing, which needs RCU. Jump through hoops to - * make it 'work'. + * OK, so IRQs are enabled after arch_cpu_idle(), but RCU needs + * them disabled to turn itself back on.. funny thing is that + * disabling IRQs will cause tracing, which needs RCU. Jump through + * hoops to make it 'work'. */ - raw_local_irq_disable(); lockdep_hardirqs_off(_THIS_IP_); rcu_idle_exit(); lockdep_hardirqs_on(_THIS_IP_); From patchwork Tue Dec 22 01:37:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 346955 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56B1BC433DB for ; Tue, 22 Dec 2020 01:38:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0DFEB22A83 for ; Tue, 22 Dec 2020 01:38:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726081AbgLVBi2 (ORCPT ); Mon, 21 Dec 2020 20:38:28 -0500 Received: from mail.kernel.org ([198.145.29.99]:51744 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725790AbgLVBi2 (ORCPT ); Mon, 21 Dec 2020 20:38:28 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id CCF2322B3B; Tue, 22 Dec 2020 01:37:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1608601066; bh=igI/vQe3iEazWjzSihez2c88DYa3jmo8Cwp6nJTNEJk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qjcXpCaFnWatiCN7WHPSOJRN1Clx7ZlZ2RJdMJQw0p8HFynjEvyOlS7GuDYcT2usk we9C1QdvFVGo5ByiAGdTI++AU5L0q37Yt60EFuWgr30H109ySpm58qA6xwI9YseAs1 zEsu1GlI4BMeIQQdA5LQDTPCoUP60cOE8yb4b8Ob7sSF1Xs4l/29hRJHCZZxhG03Ju DG2oAvPklyr2lMy95/ahSS4GibrKxCxcLSbly/4/0lk8zrRkHAtdgZsdCaAwFB6PmX 2CD7dJGuSfes8V1xtvWeAH+ZAwibHsGfLZEVy9eGjEvFPo064Dt+z4bA8O3MiEFPeP 7qccI0GSdAOvA== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , "Rafael J . Wysocki" , Peter Zijlstra , Ingo Molnar , Fabio Estevam , stable@vger.kernel.org, Thomas Gleixner , "Paul E . McKenney" , Len Brown , Pengutronix Kernel Team , NXP Linux Team , Daniel Lezcano , Shawn Guo , Sascha Hauer Subject: [PATCH 2/4] cpuidle: Fix missing need_resched() check after rcu_idle_enter() Date: Tue, 22 Dec 2020 02:37:10 +0100 Message-Id: <20201222013712.15056-3-frederic@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201222013712.15056-1-frederic@kernel.org> References: <20201222013712.15056-1-frederic@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org Entering RCU idle mode may cause a deferred wake up of an RCU NOCB_GP kthread (rcuog) to be serviced. Usually a wake up happening while running the idle task is spotted in one of the need_resched() checks carefully placed within the idle loop that can break to the scheduler. Unfortunately within cpuidle the call to rcu_idle_enter() is already beyond the last generic need_resched() check. Some drivers may perform their own checks like with mwait_idle_with_hints() but many others don't and we may halt the CPU with a resched request unhandled, leaving the task hanging. Fix this with performing a last minute need_resched() check after calling rcu_idle_enter(). Reported-by: Paul E. McKenney Fixes: 1098582a0f6c (sched,idle,rcu: Push rcu_idle deeper into the idle path) Cc: stable@vger.kernel.org Cc: Daniel Lezcano Cc: Peter Zijlstra Cc: Rafael J. Wysocki Cc: Thomas Gleixner Cc: Ingo Molnar Signed-off-by: Frederic Weisbecker --- drivers/cpuidle/cpuidle.c | 33 +++++++++++++++++++++++++-------- 1 file changed, 25 insertions(+), 8 deletions(-) diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c index ef2ea1b12cd8..4cc1ba49ce05 100644 --- a/drivers/cpuidle/cpuidle.c +++ b/drivers/cpuidle/cpuidle.c @@ -134,8 +134,8 @@ int cpuidle_find_deepest_state(struct cpuidle_driver *drv, } #ifdef CONFIG_SUSPEND -static void enter_s2idle_proper(struct cpuidle_driver *drv, - struct cpuidle_device *dev, int index) +static int enter_s2idle_proper(struct cpuidle_driver *drv, + struct cpuidle_device *dev, int index) { ktime_t time_start, time_end; struct cpuidle_state *target_state = &drv->states[index]; @@ -151,7 +151,14 @@ static void enter_s2idle_proper(struct cpuidle_driver *drv, stop_critical_timings(); if (!(target_state->flags & CPUIDLE_FLAG_RCU_IDLE)) rcu_idle_enter(); - target_state->enter_s2idle(dev, drv, index); + /* + * Last need_resched() check must come after rcu_idle_enter() + * which may wake up RCU internal tasks. + */ + if (!need_resched()) + target_state->enter_s2idle(dev, drv, index); + else + index = -EBUSY; if (WARN_ON_ONCE(!irqs_disabled())) local_irq_disable(); if (!(target_state->flags & CPUIDLE_FLAG_RCU_IDLE)) @@ -159,10 +166,13 @@ static void enter_s2idle_proper(struct cpuidle_driver *drv, tick_unfreeze(); start_critical_timings(); - time_end = ns_to_ktime(local_clock()); + if (index > 0) { + time_end = ns_to_ktime(local_clock()); + dev->states_usage[index].s2idle_time += ktime_us_delta(time_end, time_start); + dev->states_usage[index].s2idle_usage++; + } - dev->states_usage[index].s2idle_time += ktime_us_delta(time_end, time_start); - dev->states_usage[index].s2idle_usage++; + return index; } /** @@ -184,7 +194,7 @@ int cpuidle_enter_s2idle(struct cpuidle_driver *drv, struct cpuidle_device *dev) */ index = find_deepest_state(drv, dev, U64_MAX, 0, true); if (index > 0) { - enter_s2idle_proper(drv, dev, index); + index = enter_s2idle_proper(drv, dev, index); local_irq_enable(); } return index; @@ -234,7 +244,14 @@ int cpuidle_enter_state(struct cpuidle_device *dev, struct cpuidle_driver *drv, stop_critical_timings(); if (!(target_state->flags & CPUIDLE_FLAG_RCU_IDLE)) rcu_idle_enter(); - entered_state = target_state->enter(dev, drv, index); + /* + * Last need_resched() check must come after rcu_idle_enter() + * which may wake up RCU internal tasks. + */ + if (!need_resched()) + entered_state = target_state->enter(dev, drv, index); + else + entered_state = -EBUSY; if (!(target_state->flags & CPUIDLE_FLAG_RCU_IDLE)) rcu_idle_exit(); start_critical_timings();