From patchwork Mon Mar 1 16:13:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 389826 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BA3DC433DB for ; Mon, 1 Mar 2021 17:57:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 324466538E for ; Mon, 1 Mar 2021 17:57:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238592AbhCAR5Y (ORCPT ); Mon, 1 Mar 2021 12:57:24 -0500 Received: from mail.kernel.org ([198.145.29.99]:46584 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238950AbhCARvp (ORCPT ); Mon, 1 Mar 2021 12:51:45 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 3AD4764FE1; Mon, 1 Mar 2021 17:00:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1614618047; bh=CVFtk/G+YCxx7uq3QsEWlaBTR4iacFPY5+Atwc1wH2U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cA1jsiDx9uTDOukN27xWDGu727+qHEj2aCi7Wag0tryfmU8AledJ06x6DlwvaMfQs 4/MdoqWo4Np3xE2TESFM4YCux6OmMaeot4Cv9HVY8HTXhASMAMUFQ7UoUPy9JelU7T MxkOIBka7PdTT6gIufrXJ7MT//7lDEX/z2TabMtU= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Frederic Weisbecker , "Peter Zijlstra (Intel)" , Ingo Molnar Subject: [PATCH 5.4 287/340] rcu: Pull deferred rcuog wake up to rcu_eqs_enter() callers Date: Mon, 1 Mar 2021 17:13:51 +0100 Message-Id: <20210301161102.412420489@linuxfoundation.org> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210301161048.294656001@linuxfoundation.org> References: <20210301161048.294656001@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Frederic Weisbecker commit 54b7429efffc99e845ba9381bee3244f012a06c2 upstream. Deferred wakeup of rcuog kthreads upon RCU idle mode entry is going to be handled differently whether initiated by idle, user or guest. Prepare with pulling that control up to rcu_eqs_enter() callers. Signed-off-by: Frederic Weisbecker Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Cc: stable@vger.kernel.org Link: https://lkml.kernel.org/r/20210131230548.32970-2-frederic@kernel.org Signed-off-by: Greg Kroah-Hartman --- kernel/rcu/tree.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -579,7 +579,6 @@ static void rcu_eqs_enter(bool user) trace_rcu_dyntick(TPS("Start"), rdp->dynticks_nesting, 0, atomic_read(&rdp->dynticks)); WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current)); rdp = this_cpu_ptr(&rcu_data); - do_nocb_deferred_wakeup(rdp); rcu_prepare_for_idle(); rcu_preempt_deferred_qs(current); WRITE_ONCE(rdp->dynticks_nesting, 0); /* Avoid irq-access tearing. */ @@ -600,7 +599,10 @@ static void rcu_eqs_enter(bool user) */ void rcu_idle_enter(void) { + struct rcu_data *rdp = this_cpu_ptr(&rcu_data); + lockdep_assert_irqs_disabled(); + do_nocb_deferred_wakeup(rdp); rcu_eqs_enter(false); } @@ -618,7 +620,14 @@ void rcu_idle_enter(void) */ void rcu_user_enter(void) { + struct rcu_data *rdp = this_cpu_ptr(&rcu_data); + lockdep_assert_irqs_disabled(); + + instrumentation_begin(); + do_nocb_deferred_wakeup(rdp); + instrumentation_end(); + rcu_eqs_enter(true); } #endif /* CONFIG_NO_HZ_FULL */