From patchwork Tue Jun 8 04:37:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Gortmaker X-Patchwork-Id: 456274 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 507A5C4743D for ; Tue, 8 Jun 2021 04:38:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2C33C61263 for ; Tue, 8 Jun 2021 04:38:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229655AbhFHEki (ORCPT ); Tue, 8 Jun 2021 00:40:38 -0400 Received: from mx0a-0064b401.pphosted.com ([205.220.166.238]:10732 "EHLO mx0a-0064b401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229462AbhFHEke (ORCPT ); Tue, 8 Jun 2021 00:40:34 -0400 Received: from pps.filterd (m0250810.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1584c15F014935; Mon, 7 Jun 2021 21:38:02 -0700 Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2177.outbound.protection.outlook.com [104.47.59.177]) by mx0a-0064b401.pphosted.com with ESMTP id 391j7b8m7x-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Jun 2021 21:38:02 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Pvy2lB4qOds/ce0DZRfmkbVyth99C0CW3MJe2wwNG++4Mp/1vl43bOnN85T921dbVCYWpsMhUYe1miAOFU7LfXummKtuP+pd4QRfRU5AuoeJV3w1WmCxJkIyGKUWMtpJqTNLdIea7OCD5bfW/vYf9IppRHzjWJRRYP32ujeXoyWWtGN+abMTVXZzKgpiDt9Xdi1mm3n0v/2ckEVRmSW4IN/nmKrZRXPWXvHR7SleVCGdX/yLq3JezohEkXT4TqWah6z6W7woOm5TC8zbgXjN5QYAaIKYZwKKwX6zlXDnlE3mVFihar70Otq8paDB6eGDhHg0oMufpoC6CbZoRvrOXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Mu/ht2QhqKMdEi9HJHy5gWuGe6JLZQYHZdVFKn1WOWQ=; b=mDvedx1Unr6erRw87s8//TpQ8nAVHvg7OQKh3keEJfcA5MW1UYZ6ONz9pp0sO/ti558O4VFlxqqj9rQaaibg7Rfe0xKyBWXjxgO6B5Tm+OJ8VN66ULAW7z7SLxu45eZsTFhhbgEMBbHKTB/wzOA1sJM1ZF8zL4Q80dsYXEcPApx8pSkKRjrE3rV4fVYhL+dn/kafbtZQ4A1KzH5fatQ4JK96s7W/+MGz4v70C+6xAXjvuiNe8CDQ0PJKlnVQD46viMoXAhcwCNOtQeIYBsQQIwY7lrhY5I/ijYpPBK9WHmdNmIc8SAUgNafMfxMxWuu+5Mv4lbDFDyat9IO8+gtoSw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=windriver.com; dmarc=pass action=none header.from=windriver.com; dkim=pass header.d=windriver.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriversystems.onmicrosoft.com; s=selector2-windriversystems-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Mu/ht2QhqKMdEi9HJHy5gWuGe6JLZQYHZdVFKn1WOWQ=; b=hV99FggQDHv9TxJVzJOvqApcQtunbi5cC0OnR0E//at5Cu8CORBV0vNVEYhC9i/QsAFb9de1EkedNdt2skNIKMbOUEy+26pT7e4v+BI8OrEWzO4K9h70Pd02XIWhmubXWxSv6zu+1t72nIYbk71ucjd+QuQACDsrWPmueYYJmaI= Authentication-Results: goodmis.org; dkim=none (message not signed) header.d=none; goodmis.org; dmarc=none action=none header.from=windriver.com; Received: from DM6PR11MB4545.namprd11.prod.outlook.com (2603:10b6:5:2ae::14) by DM6PR11MB2828.namprd11.prod.outlook.com (2603:10b6:5:c6::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.25; Tue, 8 Jun 2021 04:37:58 +0000 Received: from DM6PR11MB4545.namprd11.prod.outlook.com ([fe80::1caa:f0c2:b584:4aea]) by DM6PR11MB4545.namprd11.prod.outlook.com ([fe80::1caa:f0c2:b584:4aea%3]) with mapi id 15.20.4195.030; Tue, 8 Jun 2021 04:37:58 +0000 From: Paul Gortmaker To: Steven Rostedt , linux-rt-users Cc: Peter Zijlstra , Valentin Schneider Subject: [PATCH 1/7] sched: Fix migration_cpu_stop() requeueing Date: Tue, 8 Jun 2021 00:37:30 -0400 Message-Id: <20210608043736.1102914-2-paul.gortmaker@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210608043736.1102914-1-paul.gortmaker@windriver.com> References: <20210608043736.1102914-1-paul.gortmaker@windriver.com> X-Originating-IP: [128.224.252.2] X-ClientProxiedBy: YT2PR01CA0006.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::11) To DM6PR11MB4545.namprd11.prod.outlook.com (2603:10b6:5:2ae::14) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from yow-cube1.wrs.com (128.224.252.2) by YT2PR01CA0006.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.27 via Frontend Transport; Tue, 8 Jun 2021 04:37:58 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d992c95b-f7f2-49ab-0622-08d92a373102 X-MS-TrafficTypeDiagnostic: DM6PR11MB2828: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4502; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3pShztJ2znM9bk3EWwBmeGUFG0mG2AGMJ//8RiLrSaEnw9RNMaqRB6W5Xt1H3xleJS8uvKRMrx5RRdw4PRBiDaflqz9PkPH7GbhO4pISFWYkxkiZ+ev+JQvrkzDOzBDfnSZUYHg5HGeBSZJqVhMe8lRGNMy0HpTjtIP0SErWHm+fu9RviNKOlpfNwv2hu6TU2xP7rsKRmallRM9oeMqaGB+qye5pTEG6XEwWb2vRK4dTLHp7X1Z/BL2UDSsxBg6EmDlw2thT9PZiX6FqsUPxNcRXUJKaNPimzQDBZgn2A9B//vlgQcHhz4Yu47DfPxJlapQeZww7yqRoHVG8mEcavp9FGtPdmyfiAkje1fcFRrpubiVNlhTrQem3tjhGnxCiXYp0cTkuSi+EpKNmDi/iwn0TKDbQetWUPgM9c9km/IV+BS7x0ZCkHAwWd6GIhZueeCSMvaVZ0gLUszZvUvy/el/O/LEmqX96IWdvzF6S4D5c7z6DChtzrCKSwM2Tksk+Ihw3hPVCcn9fYcQ/fnLO4ogc6Rs2akS2Kku+YSqcry2jTSTHJLBpE+9ZoxcaHkzdpSO4CidA2gwrdO67excap7bJJggqQu2X+5xmx6X7aaFEiVCA/OUsuNIgRrcVHgqwW8QJFgRToQlVoYdW96gIQ550nhBE6jLz2ZNpUtolAX1jhiaDBKWWRQfMQujveMGOVgVIiPc23LkezFL7evWHzSLNoiz0opcz6MuCwum+0h34HpbsWgv351vVniW46DMzs9la8D1sVgJ0rdCL4etsSw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB4545.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(136003)(346002)(39850400004)(396003)(366004)(316002)(54906003)(38350700002)(1076003)(6506007)(478600001)(26005)(4326008)(36756003)(6486002)(66556008)(966005)(8676002)(110136005)(6512007)(66946007)(5660300002)(86362001)(44832011)(16526019)(83380400001)(2616005)(52116002)(956004)(186003)(38100700002)(8936002)(2906002)(6666004)(66476007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: YL2yNO01xfbl0SYVmmBvUev4stAd22AuTWwJMabwVwZK7fKNTGcwmKHq15depdRreHkthK91QmAvPRcMsi3MCWYCFkEkNfDRpNAUE1DwCOpay+zX5bckIYQmlP48wHffVTJqTQBeDO7GPQG7i6/Me0IIzKZtOCsjl2/kZicruojmYv1Xe+bEG/2s4UY/5CsP2J8I0p9+eoC2hZ/p//5pA06OTu8aXKeWGbCvShHjKtCXPdjO2KU+jas7hByctQRcmBJcR9ru9p4ayD/lHR4ktdu00TPFCZl98NKap7Fsh5KvCoUWIgZdqKUITK3VsseJGe8QOOuZjgt+7EpA9ftePzEUdGJi6FCvW6CUZLbr4RjMIOPPP/7z0WSt/mv9uABfRPBm3oZ6iidPh09fvQUv9hAvfwYDZuyYpnfOiNfU7b1LOgghOAM3NR8EIcv4PiiAQGwXIHeIq0inXemkaC8/1xGGRIvNXG7S1wHWb5Z1FsF8JRu6s+qWwLL35+2ddYZvcMvoGmjB460m+P9f6EgIujKZJDLoCHfzSvC0QFjRuerVwn+KUH/x2PB4aJbfs3b2rgeWa/BW0tq5Q6IgKk/rl7LsenfAPCkIdhXqASsj3WaM/J58Bkp/VNdVqtfGmt8HAuH4AZxUKMFBnVQSaMnEI4bDYztZ0cd2X4HAIovwQMucn13Oi7ushjqS1lzt4r5ox+BpjpXYpkZGttI9CzRPEUGPxHh+aJE8r81Pohi133ughFqqk85TmGqakJzgMo5IliEszlygs9cNaVfyNLoSQVPhYhgtYc9oqDP4GxO0LcdvSgdSCOTfg+XH77UGaaEpr/gYmTacl/Ckf8iCMamD3lgV0mvqZL2ZJDnnyxfOZr+QtteUSu7K1CYC6SnBdEEieB6OCLcWZxIs79/9Hc4oO2Ix2DQAkyJU6hMJwkcr5lFWm324BeO5OJnYzDv1F0zfvBXtpvgHk55YuGvq/HqakaELAcATD+XNXvNG6w9Ew29ioeyoyCc60v51o6143DBIKChvwHRJT87G8I7HkVdilQL8j42cWTlUhdvYaLUi4AD6MfH86FE1itorlIUY2KmF4nvtPfj95+XDQ/kyiAkWuf/QHFHqiN5nHfnQMDbs1SyFZ/QWUMpLwyrNYWYsRRRTYzjG9he2UapbOLaZIycW+x+PZ6WLU38N0T+F5/IFT0dCio4QaE++B1mrWMs/UYyZu+wpaP+HI5issgj1VcJtvKgvyp58ipOqXozIwMt+PqhoykT4lH3Fe4GKLNdmpzFf8EZOoV9jLJGcR+Pxq7wHLdQXbOIcMAEPCjIoyP3OahUxprYTBPAMklE2xfCTRkBH X-OriginatorOrg: windriver.com X-MS-Exchange-CrossTenant-Network-Message-Id: d992c95b-f7f2-49ab-0622-08d92a373102 X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4545.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jun 2021 04:37:58.4226 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ddb2873-a1ad-4a18-ae4e-4644631433be X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6qmkKVhlMViHBvtdwFmw3wz5/v5IWhRM7HXu/BHAqg5gc4m07dMUG26wJAtBd00SMiuDknMqwP2dvImhk+LGySoBgLvc+9NZh0mniMrVc9s= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB2828 X-Proofpoint-GUID: YwNq0V_5h9ojvjULbrSJNo6D8ksjnvzZ X-Proofpoint-ORIG-GUID: YwNq0V_5h9ojvjULbrSJNo6D8ksjnvzZ X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-06-08_01:2021-06-04,2021-06-08 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 phishscore=0 spamscore=0 mlxlogscore=999 priorityscore=1501 adultscore=0 lowpriorityscore=0 impostorscore=0 suspectscore=0 malwarescore=0 clxscore=1015 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2106080030 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: Peter Zijlstra commit 8a6edb5257e2a84720fe78cb179eca58ba76126f upstream. When affine_move_task(p) is called on a running task @p, which is not otherwise already changing affinity, we'll first set p->migration_pending and then do: stop_one_cpu(cpu_of_rq(rq), migration_cpu_stop, &arg); This then gets us to migration_cpu_stop() running on the CPU that was previously running our victim task @p. If we find that our task is no longer on that runqueue (this can happen because of a concurrent migration due to load-balance etc.), then we'll end up at the: } else if (dest_cpu < 1 || pending) { branch. Which we'll take because we set pending earlier. Here we first check if the task @p has already satisfied the affinity constraints, if so we bail early [A]. Otherwise we'll reissue migration_cpu_stop() onto the CPU that is now hosting our task @p: stop_one_cpu_nowait(cpu_of(rq), migration_cpu_stop, &pending->arg, &pending->stop_work); Except, we've never initialized pending->arg, which will be all 0s. This then results in running migration_cpu_stop() on the next CPU with arg->p == NULL, which gives the by now obvious result of fireworks. The cure is to change affine_move_task() to always use pending->arg, furthermore we can use the exact same pattern as the SCA_MIGRATE_ENABLE case, since we'll block on the pending->done completion anyway, no point in adding yet another completion in stop_one_cpu(). This then gives a clear distinction between the two migration_cpu_stop() use cases: - sched_exec() / migrate_task_to() : arg->pending == NULL - affine_move_task() : arg->pending != NULL; And we can have it ignore p->migration_pending when !arg->pending. Any stop work from sched_exec() / migrate_task_to() is in addition to stop works from affine_move_task(), which will be sufficient to issue the completion. Fixes: 6d337eab041d ("sched: Fix migrate_disable() vs set_cpus_allowed_ptr()") Cc: stable@kernel.org Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Reviewed-by: Valentin Schneider Link: https://lkml.kernel.org/r/20210224131355.357743989@infradead.org Signed-off-by: Paul Gortmaker --- kernel/sched/core.c | 39 ++++++++++++++++++++++++++++----------- 1 file changed, 28 insertions(+), 11 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 3d3aa9db1548..a3dea38f410a 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1958,6 +1958,24 @@ static int migration_cpu_stop(void *data) rq_lock(rq, &rf); pending = p->migration_pending; + if (pending && !arg->pending) { + /* + * This happens from sched_exec() and migrate_task_to(), + * neither of them care about pending and just want a task to + * maybe move about. + * + * Even if there is a pending, we can ignore it, since + * affine_move_task() will have it's own stop_work's in flight + * which will manage the completion. + * + * Notably, pending doesn't need to match arg->pending. This can + * happen when tripple concurrent affine_move_task() first sets + * pending, then clears pending and eventually sets another + * pending. + */ + pending = NULL; + } + /* * If task_rq(p) != rq, it cannot be migrated here, because we're * holding rq->lock, if p->on_rq == 0 it cannot get enqueued because @@ -2230,10 +2248,6 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag int dest_cpu, unsigned int flags) { struct set_affinity_pending my_pending = { }, *pending = NULL; - struct migration_arg arg = { - .task = p, - .dest_cpu = dest_cpu, - }; bool complete = false; /* Can the task run on the task's current CPU? If so, we're done */ @@ -2271,6 +2285,12 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag /* Install the request */ refcount_set(&my_pending.refs, 1); init_completion(&my_pending.done); + my_pending.arg = (struct migration_arg) { + .task = p, + .dest_cpu = -1, /* any */ + .pending = &my_pending, + }; + p->migration_pending = &my_pending; } else { pending = p->migration_pending; @@ -2301,12 +2321,6 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag p->migration_flags &= ~MDF_PUSH; task_rq_unlock(rq, p, rf); - pending->arg = (struct migration_arg) { - .task = p, - .dest_cpu = -1, - .pending = pending, - }; - stop_one_cpu_nowait(cpu_of(rq), migration_cpu_stop, &pending->arg, &pending->stop_work); @@ -2319,8 +2333,11 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag * is_migration_disabled(p) checks to the stopper, which will * run on the same CPU as said p. */ + refcount_inc(&pending->refs); /* pending->{arg,stop_work} */ task_rq_unlock(rq, p, rf); - stop_one_cpu(cpu_of(rq), migration_cpu_stop, &arg); + + stop_one_cpu_nowait(cpu_of(rq), migration_cpu_stop, + &pending->arg, &pending->stop_work); } else { From patchwork Tue Jun 8 04:37:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Gortmaker X-Patchwork-Id: 456271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37E38C47082 for ; Tue, 8 Jun 2021 04:38:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1B4206124B for ; Tue, 8 Jun 2021 04:38:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230184AbhFHEkn (ORCPT ); Tue, 8 Jun 2021 00:40:43 -0400 Received: from mx0a-0064b401.pphosted.com ([205.220.166.238]:12904 "EHLO mx0a-0064b401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229556AbhFHEki (ORCPT ); Tue, 8 Jun 2021 00:40:38 -0400 Received: from pps.filterd (m0250810.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1584c15D014935; Mon, 7 Jun 2021 21:38:01 -0700 Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2177.outbound.protection.outlook.com [104.47.59.177]) by mx0a-0064b401.pphosted.com with ESMTP id 391j7b8m7x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Jun 2021 21:38:01 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PKPtgXV8FhxGQB8PKirZME0AkJd5L382jSeeMeX2eYzHnNJQ+9jw7AQ42zVXRL4IV2w23Q13yd6spVV6WADFuBhcOzKTV3GQMy3ET7yQ7RMFJdcSEMJDubmW8CfdLXNrOW8KaPrmH0FBFBHuU+aK/iKnKj+O6DDFsU5+qWcsha5mzN0GL+zjjwEuf4gOgiyhfBhLF+meL5QxWONTwe8AS8R1azXK2T+KMjZU/IiGJJF/BsVal4kM+IDAXebkbVEqo6/ys8YP9Xe/1oHG1Q0lFKzOn4gJm2CKW/Zf7yGEmY7d7NpQSzftZ6Nqndjkv8Ea+XKyQ+Q+YbApe2k6xymh6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=++N9btg6zFAD8SjGnV0CwZh+aAJIkFIUu3QlgoOY9VE=; b=Nnrm0l3sgttw8oEscGwZ8WsXTq2Y6lQf6g3nMlnisi/bJRswxNmnnHrryMRRDLKHAX0jotmQvvMOrdu6f8aHkQoodV6eLITCiL1viEn0LgVDyzfAFwTVPyfh4Tan+hkev3JCKk1CyLKmH9hYWSJS9K30fuEcCDpgm3Anf5nlwrLBWvYFjbVirzIFDhXBPoJUA3nhOWdivNyWgW4KjEhn5TKFDfS72ZaTCpxj9G8Iw6bzrHaYDQ333pFKvMDAfpO50bIDfKMijJGNx2HJYxlFnp/G7fmhZn90fZhoa2ufZGK17IYYl7tMBQsbC9EahuyOVAdMoFDmd4/69CCX43/HzA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=windriver.com; dmarc=pass action=none header.from=windriver.com; dkim=pass header.d=windriver.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriversystems.onmicrosoft.com; s=selector2-windriversystems-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=++N9btg6zFAD8SjGnV0CwZh+aAJIkFIUu3QlgoOY9VE=; b=fLEGT9kl/Bqur86ZTKlvqLDc9PlQK94NDijiyXrshSNzIpWj8loqk5DHMw7FIiwO1irs5HS+sW8aSeEoRAx2DbwU1Bse3r6YvBd22u55o9rn+jpqzbQQ8TVWJtIVIBV0oBR/ivl4uW9ld9g6wNKSGWdES6MKLGPzLKM+zxucSPU= Authentication-Results: goodmis.org; dkim=none (message not signed) header.d=none; goodmis.org; dmarc=none action=none header.from=windriver.com; Received: from DM6PR11MB4545.namprd11.prod.outlook.com (2603:10b6:5:2ae::14) by DM6PR11MB2828.namprd11.prod.outlook.com (2603:10b6:5:c6::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.25; Tue, 8 Jun 2021 04:37:59 +0000 Received: from DM6PR11MB4545.namprd11.prod.outlook.com ([fe80::1caa:f0c2:b584:4aea]) by DM6PR11MB4545.namprd11.prod.outlook.com ([fe80::1caa:f0c2:b584:4aea%3]) with mapi id 15.20.4195.030; Tue, 8 Jun 2021 04:37:59 +0000 From: Paul Gortmaker To: Steven Rostedt , linux-rt-users Cc: Peter Zijlstra , Valentin Schneider Subject: [PATCH 2/7] sched: Simplify migration_cpu_stop() Date: Tue, 8 Jun 2021 00:37:31 -0400 Message-Id: <20210608043736.1102914-3-paul.gortmaker@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210608043736.1102914-1-paul.gortmaker@windriver.com> References: <20210608043736.1102914-1-paul.gortmaker@windriver.com> X-Originating-IP: [128.224.252.2] X-ClientProxiedBy: YT2PR01CA0006.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::11) To DM6PR11MB4545.namprd11.prod.outlook.com (2603:10b6:5:2ae::14) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from yow-cube1.wrs.com (128.224.252.2) by YT2PR01CA0006.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.27 via Frontend Transport; Tue, 8 Jun 2021 04:37:58 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3a8d7e1d-67ab-4141-3728-08d92a373160 X-MS-TrafficTypeDiagnostic: DM6PR11MB2828: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4502; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YpNVrMT5f8da1bElBEREK3MmIeHiX8KUNkGRjTIDZGWpnC/nN0an+CxjQobQrXAj7A4wO7YNRVKbZR2j3Pw31+qWvZ86Wtt0ykoEwV20Ja9odEgno4YH/hC7EDBb56BT09wyMmN1bDNwchHIaBvFO699tfiQ8UZrFA7VIVAJcCluu/qVpAGTSr6dnDWI0wguM1a6gUkPi2PmM4FEEsMjYy3elJXZQTyMGdVSaOjssE+H8yhNU1FxYNRNWauWf8zGjUUKqFu3p42/695ssHym3ePFmkaGIqX/JNyH5D621s0DhiXKNVlO7pHzQ6aICZPHx4MWcVqVGmedaNcUq26ztbEV1az6qeHSDRB49NOIYpE4DWomhoirIbkF1K8iM9JOlOdCLpd1BwfZ5rrPFkXVH8Au3AOIDoVYYUkMPeDJvjTehWcEBFF2MpUVuDGQZmUN0rMGpJ9MO2YTRR6TonGG+6fW7xxZG2Oy/mwK8HsLgN+ZN8nocPtuP39kA1cQ1X2eRiHnwgM/gQrrDBJX2w5Gj0t6jenay8Ypks+NxwuaIfb6ceDAWMxC1mAZ/ZkFj4hVty8UUEm52dL4JhntyExWXrhG+EZm/q/0GznQqMomecx8KDcW6jbcrapR9q5R+infXhniX+HDBiiAMfreLOYIkUJgXU9PmVfzOsnt1U3gzGdM4bqYG4xdoqLafIHiZJayGXWpHX40oCclFlOMG6yLJB3VmL3TQ6W+t6s4B6kmOgTeWbPzuRz2DT6H2Z8L7/RlvFhbYO/vwWxaocZrDfCOZw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB4545.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(136003)(346002)(39850400004)(396003)(366004)(316002)(54906003)(38350700002)(1076003)(6506007)(478600001)(26005)(4326008)(36756003)(6486002)(66556008)(966005)(8676002)(110136005)(6512007)(66946007)(5660300002)(86362001)(44832011)(16526019)(83380400001)(2616005)(52116002)(956004)(186003)(38100700002)(8936002)(2906002)(6666004)(66476007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: T0DPtCvTxnFn7pLbuCrY9O1+RT8DgPCSL7jRQY2TqmxGEUI2hXMgwWloe6kshzWZ+5kSuRaZ76aNS39SFI1BKxuqsNg/F9fu/ylpsvR8ZxuGp+afawC1Lvm+n4f/kC/qd4kJQ7aNVhv+KFmp4j3HLhD6FNf571OV1p3eB2WRedJZY/Igz3aL18txKPfRIxV89B9qTioLcJb6y4HyikU6OYZWNobH+qHWS0IOfhn7iELfVSa5BeNJDC/IObHOjMdagjrd3F1/7OwQoD4om1N6oe9A9DOGyywp6oEQXFR2RoJLMWkf+3kAyAvIJT6AroN/ifVj7TFpHKzT587NOgQUGDftOMOwIHUmfEsihSi9RlCy4zidfoVDB2qLzsUeVMtOf6PXax1DDZOcfrw9N8dId//3ENgV+G/jR2PSpEdafJXvhvtTIO9OueTIzY3AEqNPi6QTV1DREE2i8HMo9/2TipYbIC5oo2wIkPN/u9pw3JiFwI77f4T+DNT2ZI8ANUuogjAImad/gfqkaF4ZyC/xi1nDaf8oZ9iSRMXFu6kbDgRg3XJ+Iowzkx7g3ef3NPtdlDrB3B9QzzvWTIPhLzJkB2z4bzIoitcigUa67s5lU8Syp0zg2WGXPqOQ/bw6+gDwf/kPTmSiss7sc7GKm2jyVaXcfiyjnaebARgo60kMZWW7s912DyXyotEbuG+b27uTh4iJN8TVrahu6wZON++c5vdK3eiOMbX9QrYwttxcDKL2ds+uwk72VsZDuRNhLk+c9FAHCl/J5cDKOAkzRC8+eRyEtXvbz23Ey3LONVkWVSmPSWqtLatKOFRsO4puqvAoZ5SIiAcr3q6xmo5upKRUju7SZLSeVlPQhkTPnqq+Y+PPVRUtQCTU5+/TtLDcDK5ES3Az1HNbZ5JSgP//nZ6jstntWoInyGqcNEWy6PsFyY18WrQvTOrq0UKqtqkGaRdRmjVfsVS7/oA1WxtG39jxPSXHxqfiBkqpVeVdtp/sEs3f6ohS3lUrKDGj3i/O9H8Vh0ZRGkXeKMqEatRuSvMBZkCZFzjZvLE+B3QfFfmnwZ1isPkRY4h9amWIKozd8LVRalrufWAysEYL7GM9vL7u8pZ4IsUvyjvXtygwu/ci8I3WJfGiy8kXgpDkyEHezir9R+IBa1QSEyqpSNYENoM391sgEPMeV1wJ8On8Tr83zxtP//bUlor5oecnxjmKZ5g9K85HCOKomW5oDVRutGk/6SsIGoJVVU2JtM9csSeT/QkFTE/a/ZpZ4P4Aj94EbHzhhqmkPqv81f1MkV8u80LQzw0Pcd9/ToXkWP4TnI8KkIUTj7NljzgwuCt5pzAd+2PB X-OriginatorOrg: windriver.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3a8d7e1d-67ab-4141-3728-08d92a373160 X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4545.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jun 2021 04:37:59.0518 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ddb2873-a1ad-4a18-ae4e-4644631433be X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: bYpQHX/iT5vOyrKmlAKar/6jxnlG2Yd+DqcwJ+LmJObLi16OQmMFgFL0uLd1+Om/n27XnVJuL4r3XY+FB5QQ9v0jhnXyCXA7Fo+9RO8bRc8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB2828 X-Proofpoint-GUID: juDy4ApEu0CHDHUKyi2iRGLk1ylPPyaX X-Proofpoint-ORIG-GUID: juDy4ApEu0CHDHUKyi2iRGLk1ylPPyaX X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-06-08_01:2021-06-04,2021-06-08 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 phishscore=0 spamscore=0 mlxlogscore=999 priorityscore=1501 adultscore=0 lowpriorityscore=0 impostorscore=0 suspectscore=0 malwarescore=0 clxscore=1015 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2106080030 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: Peter Zijlstra commit c20cf065d4a619d394d23290093b1002e27dff86 upstream. When affine_move_task() issues a migration_cpu_stop(), the purpose of that function is to complete that @pending, not any random other p->migration_pending that might have gotten installed since. This realization much simplifies migration_cpu_stop() and allows further necessary steps to fix all this as it provides the guarantee that @pending's stopper will complete @pending (and not some random other @pending). Fixes: 6d337eab041d ("sched: Fix migrate_disable() vs set_cpus_allowed_ptr()") Cc: stable@kernel.org Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Reviewed-by: Valentin Schneider Link: https://lkml.kernel.org/r/20210224131355.430014682@infradead.org Signed-off-by: Paul Gortmaker --- kernel/sched/core.c | 56 +++++++-------------------------------------- 1 file changed, 8 insertions(+), 48 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index a3dea38f410a..d497b13efb53 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1934,8 +1934,8 @@ static struct rq *__migrate_task(struct rq *rq, struct rq_flags *rf, */ static int migration_cpu_stop(void *data) { - struct set_affinity_pending *pending; struct migration_arg *arg = data; + struct set_affinity_pending *pending = arg->pending; struct task_struct *p = arg->task; int dest_cpu = arg->dest_cpu; struct rq *rq = this_rq(); @@ -1957,25 +1957,6 @@ static int migration_cpu_stop(void *data) raw_spin_lock(&p->pi_lock); rq_lock(rq, &rf); - pending = p->migration_pending; - if (pending && !arg->pending) { - /* - * This happens from sched_exec() and migrate_task_to(), - * neither of them care about pending and just want a task to - * maybe move about. - * - * Even if there is a pending, we can ignore it, since - * affine_move_task() will have it's own stop_work's in flight - * which will manage the completion. - * - * Notably, pending doesn't need to match arg->pending. This can - * happen when tripple concurrent affine_move_task() first sets - * pending, then clears pending and eventually sets another - * pending. - */ - pending = NULL; - } - /* * If task_rq(p) != rq, it cannot be migrated here, because we're * holding rq->lock, if p->on_rq == 0 it cannot get enqueued because @@ -1986,31 +1967,20 @@ static int migration_cpu_stop(void *data) goto out; if (pending) { - p->migration_pending = NULL; + if (p->migration_pending == pending) + p->migration_pending = NULL; complete = true; } - /* migrate_enable() -- we must not race against SCA */ - if (dest_cpu < 0) { - /* - * When this was migrate_enable() but we no longer - * have a @pending, a concurrent SCA 'fixed' things - * and we should be valid again. Nothing to do. - */ - if (!pending) { - WARN_ON_ONCE(!cpumask_test_cpu(task_cpu(p), &p->cpus_mask)); - goto out; - } - + if (dest_cpu < 0) dest_cpu = cpumask_any_distribute(&p->cpus_mask); - } if (task_on_rq_queued(p)) rq = __migrate_task(rq, &rf, p, dest_cpu); else p->wake_cpu = dest_cpu; - } else if (dest_cpu < 0 || pending) { + } else if (pending) { /* * This happens when we get migrated between migrate_enable()'s * preempt_enable() and scheduling the stopper task. At that @@ -2025,22 +1995,13 @@ static int migration_cpu_stop(void *data) * ->pi_lock, so the allowed mask is stable - if it got * somewhere allowed, we're done. */ - if (pending && cpumask_test_cpu(task_cpu(p), p->cpus_ptr)) { - p->migration_pending = NULL; + if (cpumask_test_cpu(task_cpu(p), p->cpus_ptr)) { + if (p->migration_pending == pending) + p->migration_pending = NULL; complete = true; goto out; } - /* - * When this was migrate_enable() but we no longer have an - * @pending, a concurrent SCA 'fixed' things and we should be - * valid again. Nothing to do. - */ - if (!pending) { - WARN_ON_ONCE(!cpumask_test_cpu(task_cpu(p), &p->cpus_mask)); - goto out; - } - /* * When migrate_enable() hits a rq mis-match we can't reliably * determine is_migration_disabled() and so have to chase after @@ -2058,7 +2019,6 @@ static int migration_cpu_stop(void *data) complete_all(&pending->done); /* For pending->{arg,stop_work} */ - pending = arg->pending; if (pending && refcount_dec_and_test(&pending->refs)) wake_up_var(&pending->refs); From patchwork Tue Jun 8 04:37:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Gortmaker X-Patchwork-Id: 456273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0375C47082 for ; Tue, 8 Jun 2021 04:38:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AEE5861249 for ; Tue, 8 Jun 2021 04:38:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230017AbhFHEkj (ORCPT ); Tue, 8 Jun 2021 00:40:39 -0400 Received: from mx0a-0064b401.pphosted.com ([205.220.166.238]:11514 "EHLO mx0a-0064b401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229526AbhFHEkf (ORCPT ); Tue, 8 Jun 2021 00:40:35 -0400 Received: from pps.filterd (m0250810.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1584c15I014935; Mon, 7 Jun 2021 21:38:03 -0700 Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2177.outbound.protection.outlook.com [104.47.59.177]) by mx0a-0064b401.pphosted.com with ESMTP id 391j7b8m7x-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Jun 2021 21:38:03 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nfGfJhyESEFjiHX5O3kYHUQFjR4S9ZKKf37hnYNYyQWypmLiiYbjc/xDbpdCGA2TmD+aowhmlfEXVqfxV+vM+301WwCBkP2e0O9hIVumSJS07wnJaiaJ7d5JPxrIJq/jssQ1zIAYUB/EJaGxBC34MoXlQK/6flbTWHQRrUxxtHxZTXkrc2RNojsuHeYR2WJif9uaXWtyPW6Af3C74wVWx4Ca6TUblPFgv5VAdUPV9WoyyXOStfMNpLWeFqx0vXwOd088z8LyCtmtkjFiwNgjJ/Vw8aob4uM0VOCx+9fZS1GeQ9EDS5yUD8opsshE4dxnVcr10XH2pQOUe9TXSbbBtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eKtS/gquwVzOfv7el/kv8NfVqxraegnBDDMsHOYrGQA=; b=kNtOw6glhZEhdc1iunDnrTDj1Yjn/rwGhE94Igk3rHxMe8+m56gcws/OqmkKAT88Tmpb0WA/P8YUTZk9AWc1K9aO3FhU/iBG3uhNGrp8W5/HMlLCVNjgXncVjHdEijkmgEl7KvRwg8lqOTwGpQLknQSFvFWFyPhORhX1u1eQflt9LcjgvnwD6FUrAZCDSKbBZOZWc2ybQOKaLgQfdXfZ098oQgkkIS7HbbwmpnWSr8h1xB3GMSx8yIJJSUO5gxuozRHl80n4wUfygpLz8L3F4IxD3j1bI72XaSmm62YjrhY7ViDLS6BRV5RRT/H1yLlmmFSd92ffzZS5G+xDXMNvbA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=windriver.com; dmarc=pass action=none header.from=windriver.com; dkim=pass header.d=windriver.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriversystems.onmicrosoft.com; s=selector2-windriversystems-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eKtS/gquwVzOfv7el/kv8NfVqxraegnBDDMsHOYrGQA=; b=QztcPeaZrWm2Tc2ORcNmu/+tZvMSH6+f08hPCwap9MR+lMFzHCZVWa0PuT6BWV/FimuMX/WGTdkFghfGYg6eStR4PD12HsiYY5O+DAkJKmXoQz0yZe8dZ6A1FavzKiv9NiXmwtNkrjbhFeV8xElzTeWejQSuudCIFGgSH2mixm0= Authentication-Results: goodmis.org; dkim=none (message not signed) header.d=none; goodmis.org; dmarc=none action=none header.from=windriver.com; Received: from DM6PR11MB4545.namprd11.prod.outlook.com (2603:10b6:5:2ae::14) by DM6PR11MB2828.namprd11.prod.outlook.com (2603:10b6:5:c6::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.25; Tue, 8 Jun 2021 04:38:01 +0000 Received: from DM6PR11MB4545.namprd11.prod.outlook.com ([fe80::1caa:f0c2:b584:4aea]) by DM6PR11MB4545.namprd11.prod.outlook.com ([fe80::1caa:f0c2:b584:4aea%3]) with mapi id 15.20.4195.030; Tue, 8 Jun 2021 04:38:01 +0000 From: Paul Gortmaker To: Steven Rostedt , linux-rt-users Cc: Peter Zijlstra , Valentin Schneider Subject: [PATCH 6/7] sched: Simplify set_affinity_pending refcounts Date: Tue, 8 Jun 2021 00:37:35 -0400 Message-Id: <20210608043736.1102914-7-paul.gortmaker@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210608043736.1102914-1-paul.gortmaker@windriver.com> References: <20210608043736.1102914-1-paul.gortmaker@windriver.com> X-Originating-IP: [128.224.252.2] X-ClientProxiedBy: YT2PR01CA0006.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::11) To DM6PR11MB4545.namprd11.prod.outlook.com (2603:10b6:5:2ae::14) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from yow-cube1.wrs.com (128.224.252.2) by YT2PR01CA0006.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.27 via Frontend Transport; Tue, 8 Jun 2021 04:38:01 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ad40ccea-d7e8-4d6e-8903-08d92a3732c5 X-MS-TrafficTypeDiagnostic: DM6PR11MB2828: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1360; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZX7fIPyhfoWe3x3Opdux5j7i3m2RdphT4AeEDcS2RAXUAWtK2DBaw8AfR8k2FUVZAm9Ws5mJqNOMoxNpsmaQ5RbcZyBoZXsBZgjTlC2b7CtrSbhQvjYiXqS08wOxm162V37r5+PQQAk7M0sgmj6tlc8skIQ/u8JRh77+TBlDJpKh3xuaHZ7UM4ynVOySeqkeu7iZorG9EHRONrpiFB2hyuVHIV7Fp7+/3/40idjyvvx/nwCI8zj3NIQ+EAVQwDcw5Z48yL7b7vqUvuCTj1G7NPO9rsS8EupnfJVqA6uwVKZBMw87eC1ULFAo1Nk1+YwTMF1YHtd2cVUEuUnarx+HdWi5FB+DXHlbgHxJ4XhRbR83VVz8p1k59g2dMQscH6Dpp5PGiC1Inkwomq1pvOT3xdONlrV8JwV5CkGpDuL5ze99fxcspvN3LGXYjv2/SU7YM9yl8n8ZnQmbdColLRtpjBRYUs/qJnYeFhYYXOQppsik1ljV/yJThCbz2gGMOhsqkR7pzn+9mF9MMznSY/mle8J7EzMj7wwaRBYGHfL6NktCwrq+rsCLPZgaxLsPdfJu/HQJGMotQ30GWj/J0BH4lzZ3hIyiZcDWP701VmvJDS2pmyW+iHzS+aztY/WVS7/JlNPTntE3fMimILHUQr4oe/D5hsq1CZU2k5wNbNZKGvYqQWTbmwfEvKI/+XB8VuKp6GIR+3Q/FCZdSldfMZGri7mJn4Nt+37+2BKPYRdfxeAEI4jf0wKlS00F7xkG1L4/8VVWgs0vmRSo6z/7KyihVA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB4545.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(136003)(346002)(39850400004)(396003)(366004)(316002)(54906003)(38350700002)(1076003)(6506007)(478600001)(26005)(4326008)(36756003)(6486002)(66556008)(966005)(8676002)(110136005)(6512007)(66946007)(5660300002)(86362001)(44832011)(16526019)(83380400001)(2616005)(52116002)(956004)(186003)(38100700002)(8936002)(2906002)(6666004)(66476007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: En0SRwkApf2/8Ajy8aWSB9FoJqBvvmBjAjekrhlMBhtcU1/uDqyk1JDEJwbXvGR+OHyaoFWhB/kJWe/+EvNl7FNa3okHsoq/iCxAJSOd09VB4KtAa3crUF1NIcB3T4q2BKE2Fmh3qV5mwlyODQZOvAQC+ooBYPU9GXwcqbBNbm3Qy8+1GhzUxsKEeI7X882PxJuFPwc4Of4lyzvgMglbyzjgfM/BrGADAyZOrgGbrTbOq0vzO9R7xY2tpcUNqh9XIjQgwXmbmZq1OUWxVc8qDoam1pl4+4fWjmolp/4XRF88xW3AyOwmdhmSCCRcoJjY0YSsKrKZg6ILzag63hBkIL2eVTQHzDwyeL0zYchSmfmfeZ1OSKLF/QCqO6uF6kAlphHGVNXFbdeUerhaPtVCv2aYrLfkMJC6HVIIpAw9NcCuzpV928ARAVHjeVar40gjvYMM6DfifCdkkwA8V9l2g0Qltf6rg8cO9nm4X/VJkr/8rbUOGDJ1y5EvodcnB1WeGR+VezCXiDA9JP0K4tBH0XB81QnHG27msG51I0tGvxYT/cRF8cvUw38/fZWE8mx+H6lMMFA9AkjZXky3W2j6M5jRc36peVvylbq+xLa9MymGocEliXAI6MofLaiYUyob7hn/Z9mZpitLsHY5cs9XEqKO7fjtTinhOkqXBWlzHKXEsLqjfow1kV/1E30ueEjoa53Lmzt+d9kFqN3MlzTBvMX6/2NDIh/CGmKIGLE5ecWh3I6y7rHVjbLYZa1mVKHskJmPwcUq6rUUxGJ4TmZYsBHnAHpRgNlc6UvGqgOkQfcshYVe0lMAxJcxXe4GUDopauNUhxPQ4RVeYbnQ+INPpVUbtV8c2LJEq+CneeuA0Y2EEkZ+gv3QGJsqYTAoTZg8oDOrz6ZVM1mKVHpx66sc20CaqCKy+Cpf5vS9bajyOUQTDwHHgdtLwRfqrfHr90zIe09pxrT1HDDohyPLx+jf9C5TonQsrluvk2YnLzmolQUPTvETlMONrCCnPE7y6qW7RjLNnz2iucMWeYH1Cw6VHyqAQ4E9lKrjOyOcWTI4CRlb06NRAMzQ1T+xwbyeFOfeQKuSLySPGVhs9ljlW/V+SFCnh0y7z91aAyNJtWkh/+/EEojAsQuuOYwy4QG3lsaBROSbLByNWzzv6m/WZ2MGxdq5dsUxHS3IZ5DzHZN2nVTO7s5S4ktKj0uXpsy8nNjXyYrGyKsgwI1FPx6NkbY4UURSHE1XAF0yilphsFnfU4OEkFBzzh+cAtM7UsYZIfGnr6qVgU8up64w75IBE6WZ9RgMA1PxYqpOKdHrGmdzzOgsZ7YjX9oamBalOq3E1+C4 X-OriginatorOrg: windriver.com X-MS-Exchange-CrossTenant-Network-Message-Id: ad40ccea-d7e8-4d6e-8903-08d92a3732c5 X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4545.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jun 2021 04:38:01.3846 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ddb2873-a1ad-4a18-ae4e-4644631433be X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: dmFo/wsjFr3Y7Qn9Y5W7HH2UTP1EsQaIVS+zTftvU0RbfdWrs0Ll8BYep5VwW/RE0GT7SHEg81Hyd9niZvUjOjduclhaLOHlnCq4mkyrUYE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB2828 X-Proofpoint-GUID: f443S2OBtOb0CfpCoTg33Sb6v9SswIdz X-Proofpoint-ORIG-GUID: f443S2OBtOb0CfpCoTg33Sb6v9SswIdz X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-06-08_01:2021-06-04,2021-06-08 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 phishscore=0 spamscore=0 mlxlogscore=999 priorityscore=1501 adultscore=0 lowpriorityscore=0 impostorscore=0 suspectscore=0 malwarescore=0 clxscore=1015 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2106080030 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: Peter Zijlstra commit 50caf9c14b1498c90cf808dbba2ca29bd32ccba4 upstream. Now that we have set_affinity_pending::stop_pending to indicate if a stopper is in progress, and we have the guarantee that if that stopper exists, it will (eventually) complete our @pending we can simplify the refcount scheme by no longer counting the stopper thread. Fixes: 6d337eab041d ("sched: Fix migrate_disable() vs set_cpus_allowed_ptr()") Cc: stable@kernel.org Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Reviewed-by: Valentin Schneider Link: https://lkml.kernel.org/r/20210224131355.724130207@infradead.org Signed-off-by: Paul Gortmaker --- kernel/sched/core.c | 32 ++++++++++++++++++++------------ 1 file changed, 20 insertions(+), 12 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 20588a59300d..35d8b80d7cb8 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1898,6 +1898,10 @@ struct migration_arg { struct set_affinity_pending *pending; }; +/* + * @refs: number of wait_for_completion() + * @stop_pending: is @stop_work in use + */ struct set_affinity_pending { refcount_t refs; unsigned int stop_pending; @@ -2033,10 +2037,6 @@ static int migration_cpu_stop(void *data) if (complete) complete_all(&pending->done); - /* For pending->{arg,stop_work} */ - if (pending && refcount_dec_and_test(&pending->refs)) - wake_up_var(&pending->refs); - return 0; } @@ -2235,12 +2235,16 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag push_task = get_task_struct(p); } + /* + * If there are pending waiters, but no pending stop_work, + * then complete now. + */ pending = p->migration_pending; - if (pending) { - refcount_inc(&pending->refs); + if (pending && !pending->stop_pending) { p->migration_pending = NULL; complete = true; } + task_rq_unlock(rq, p, rf); if (push_task) { @@ -2249,7 +2253,7 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag } if (complete) - goto do_complete; + complete_all(&pending->done); return 0; } @@ -2300,9 +2304,9 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag if (!stop_pending) pending->stop_pending = true; - refcount_inc(&pending->refs); /* pending->{arg,stop_work} */ if (flags & SCA_MIGRATE_ENABLE) p->migration_flags &= ~MDF_PUSH; + task_rq_unlock(rq, p, rf); if (!stop_pending) { @@ -2318,12 +2322,13 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag if (task_on_rq_queued(p)) rq = move_queued_task(rq, rf, p, dest_cpu); - p->migration_pending = NULL; - complete = true; + if (!pending->stop_pending) { + p->migration_pending = NULL; + complete = true; + } } task_rq_unlock(rq, p, rf); -do_complete: if (complete) complete_all(&pending->done); } @@ -2331,7 +2336,7 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag wait_for_completion(&pending->done); if (refcount_dec_and_test(&pending->refs)) - wake_up_var(&pending->refs); + wake_up_var(&pending->refs); /* No UaF, just an address */ /* * Block the original owner of &pending until all subsequent callers @@ -2339,6 +2344,9 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag */ wait_var_event(&my_pending.refs, !refcount_read(&my_pending.refs)); + /* ARGH */ + WARN_ON_ONCE(my_pending.stop_pending); + return 0; } From patchwork Tue Jun 8 04:37:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Gortmaker X-Patchwork-Id: 456272 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA9DBC4743D for ; Tue, 8 Jun 2021 04:38:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 85DB26124B for ; Tue, 8 Jun 2021 04:38:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230160AbhFHEkl (ORCPT ); Tue, 8 Jun 2021 00:40:41 -0400 Received: from mx0a-0064b401.pphosted.com ([205.220.166.238]:11566 "EHLO mx0a-0064b401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229535AbhFHEkf (ORCPT ); Tue, 8 Jun 2021 00:40:35 -0400 Received: from pps.filterd (m0250810.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1584c15J014935; Mon, 7 Jun 2021 21:38:04 -0700 Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2177.outbound.protection.outlook.com [104.47.59.177]) by mx0a-0064b401.pphosted.com with ESMTP id 391j7b8m7x-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Jun 2021 21:38:04 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MGsPGapgz+KqlN7/2U7yjPkV4oBQNdnIpSujco/3oWQbt/PO889j61NkqBhoo49OjcUfZFTSxnpswWL2X1FV9Jei2QDVhagh3fqKrl1Y1Zx9PTWd5Gi639tNxk2xKEZLu5y3lE8OSpGV4UU93daRDUT1XRY95mfYgPIkG87BCmZ8FOd0mXB5BDf1xMVhjFwE1SK5erUtEXCOZHo98513qyb71KgjcpypGj9CLe3vGKn5ICZX3iBMRBO09i4uUI+dyLph4N7MaMWTbLk0Z4Ah8/hXTyOEjKzGf1exLEhc3L+Pbb12do2F8YHPFuer9ty+h1E3Dd+DHlTcWuUwabMDcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rlcaOOylX1RuHOEcFjVnFwYJhJISwxivRum+qTLX0gk=; b=FlocoN7OEoPecwoZrNXMLD6DxoMNrT+4LcRNRUcSfN9VMb+sRQAtQtb2d2MEjPhJZJsAljCTP0Hd8WKj1fyH74rZ679xnwU5dCt+e2tetVYmXba1Zvdqgn48JLUp7fh/jMxalRedVldoo4eRxe4nJEyLqw1KrG18FTTTSbdrpSyKLUs1MbMBJe0QyEiHngkxIsHu/75he/9xsSoSXDwp97YWBxX/sVO3NE/mNBjmYvd/GMxHrX9J0k8aE9z2imjgPcsWa+XEKCBpb6f0/BVLNSHbfKoGpBe8GeamapQp0NnawTUhJzPzWCTXt7ur7xh/SM4+bDpxaDt1Q2efROKfKA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=windriver.com; dmarc=pass action=none header.from=windriver.com; dkim=pass header.d=windriver.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriversystems.onmicrosoft.com; s=selector2-windriversystems-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rlcaOOylX1RuHOEcFjVnFwYJhJISwxivRum+qTLX0gk=; b=Y8pwy+B4sswfKa/akiwZ/bAjW2vUDyRpJ7rh7Qq7OC8xucvD62WbjXTw8eadQqfDw2FldyxssOY2Oj0V8cQjN+mYQE5pr9dR5J1eDHK5s0tTOYODtSotvtcPxxYEukKsz+0ukYftfGbJ07HM93YiXm0E2XUQ0x7jJksYPfWD5e4= Authentication-Results: goodmis.org; dkim=none (message not signed) header.d=none; goodmis.org; dmarc=none action=none header.from=windriver.com; Received: from DM6PR11MB4545.namprd11.prod.outlook.com (2603:10b6:5:2ae::14) by DM6PR11MB2828.namprd11.prod.outlook.com (2603:10b6:5:c6::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.25; Tue, 8 Jun 2021 04:38:02 +0000 Received: from DM6PR11MB4545.namprd11.prod.outlook.com ([fe80::1caa:f0c2:b584:4aea]) by DM6PR11MB4545.namprd11.prod.outlook.com ([fe80::1caa:f0c2:b584:4aea%3]) with mapi id 15.20.4195.030; Tue, 8 Jun 2021 04:38:02 +0000 From: Paul Gortmaker To: Steven Rostedt , linux-rt-users Cc: Peter Zijlstra , Valentin Schneider Subject: [PATCH 7/7] sched: Don't defer CPU pick to migration_cpu_stop() Date: Tue, 8 Jun 2021 00:37:36 -0400 Message-Id: <20210608043736.1102914-8-paul.gortmaker@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210608043736.1102914-1-paul.gortmaker@windriver.com> References: <20210608043736.1102914-1-paul.gortmaker@windriver.com> X-Originating-IP: [128.224.252.2] X-ClientProxiedBy: YT2PR01CA0006.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::11) To DM6PR11MB4545.namprd11.prod.outlook.com (2603:10b6:5:2ae::14) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from yow-cube1.wrs.com (128.224.252.2) by YT2PR01CA0006.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.27 via Frontend Transport; Tue, 8 Jun 2021 04:38:01 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a8a4add3-2433-4294-a65b-08d92a37331d X-MS-TrafficTypeDiagnostic: DM6PR11MB2828: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3173; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: nTVioMqKvtpEIrfmvZghOO5vKV9Tqik/c5wPvq/nJxgH3rTbpykC/UmL1/6atsMdw8ipnQ2iGWPqvobKVaseig8oGWmMC6q06hksOoreWoBt/WEtaJYcCXNXhpZEje8wQke3euF0apxiyHjYUaEiZHpzwYZ+tHRzJZ7v+uD4ZMiMcEWJ40MLfKgClKb7gzZGbNSbW3HRMwBMhHRSfBKT3MW/cJMZO/Bb4Ry0BLRDGeGfnPJUj9jhT5CyDAI11MIxUXjWsAFIL2JpSKKP0qAJg4WxCznlA0+r2KASxPUE38SMxeFXYEblIIH18Da+576pjaXFYl1Z0caQyigSdXq9Agp6B8PEMxyEcZPl0oBXWAVsfRZH+JxZQt18QsmLXk6F8eYWcmvScudAGXTKWHsssEYJzcbEfSHGy3YkEJuPJc7yBVo9vgcTarokKAK1d7YFR5B2hSpLKvG3udFLpg9ChUJi8ydJZ+2C+CFiPAFZ1cGDeAOXEauCfiP79p+O5nRBEPGBwj03o18svQ1y5A5McbVMyBCk/t7iJY2zhQPKlVCRNvgbJj97YtyVi5sxhS8h50PCx85+iJFjSpYFdNqDtLm3jFyEjUDBFrh8tgS/uJIn5uDy3MG0GZWCTergGBiZpVKX3yPRSXOPynkuiiDuok1Cm+lXF2ZQKSLeRPxY8ZvgAOj06fthaAHYPz0crnRRznDFuDIOdCNEUm/vH1eZ6p5GZ66FCWZGkvfPs5Yoh96n/c3S4+dgkG360VqPCwhaEgE27RHtH+Lt3mzXuqCWxzAwS4W/jg6XVrwjyuQ7RQo= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB4545.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(136003)(346002)(39850400004)(396003)(366004)(316002)(54906003)(38350700002)(1076003)(6506007)(478600001)(26005)(4326008)(36756003)(6486002)(66556008)(966005)(8676002)(110136005)(6512007)(66946007)(5660300002)(86362001)(44832011)(16526019)(83380400001)(2616005)(52116002)(956004)(186003)(38100700002)(8936002)(2906002)(6666004)(66476007)(21314003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Hph+zRNCUdnk+h5PAn0PRSpw0WbGj+xOuNiCaL3ayAsg437MCVYKG5hQSSyBc0P3NI2DgaGfvRhkNwi4HEl/G0JF3mbxS2wOwLH3/Qawa2aaBGctoorhVKsMIyz+XfeUUg66885PGn7VJ/oys/vyVWv1Yo3TwEhLopI8CCuSOivpI1JxNrvHRK8wPQOw4vLLRWt/1UDIbibcgeqPeB3anWbtkHa9U+oLVpm+tE0VQqNU9irCgk6ng6GEWNKlQwAYQo+Gpa+2EzoExcoUHZP5GrUKsVkquS3GhJLExAaad5aBJymCiYZFnWfAn69K1yJl/ekM0cG46f1y3vAg+i1Vl+qhPBflxuVdFxNM+GJJXvYcSJ/aViME9s+NrDsWZnwZdsxMSWo0BQu4Bum+r01v3ZzRFMpcT88eyzts/BJ7teyHW6xr05fB4cSggk0C+l04tqS4mK0SHAG8LH8ddTuWqfjG0ATBWuo2zJe78iaHYufKRgCDCnUAHSJfJ9/Vbt/Xx4fKB55sC3Wg+Q/5m6AiRq7OKWFM9X6BtJYMlMP9jFP2ku0uZYbosZfMXynaOT0aqZdlJr2xZy3KTIQ8roaqwQrhy3155buWVR+91P2WyyszE1usrhLzGg9A45mJ9u1GujJUHSrialqUVZ1e6Ve9IW0Sa2cfaUncOBdF7mZ8xdOG9hnnzM7W/lezDQcd+dVtWiCevjLWNCBgJZiuGsK61Vq6JmN+Z1qS4O6E0Vlp4wOuvFLsxIyCdQ1JProQKq/lRXxDBaePCk+YrbIYAN6/XC0cgPbVQ2wkGjwJ6D9AVESjdgELPJcNX5dqhlGrk/cp7LhVUgcNR8EdjWho8aYNm8P4cg1RdrU7qHZ1IY7H/bltq5GBkzG0aj4yZlL1yrAGyqjj8GYrSkbupJ3HfRyRf4eRVhYZsPM0Yl2bSBszdwT5kMQtoE+HD2qkm3VtEhmis2RTejZJ32lmQpykCmMfkpiVIsRxcck5I6ipdVtLzUVnZgVf1Llft9GMPpWGVmVuJv6khJ0X9bgWN+nhMHcKAg0kGA7Bnz89sGwL+9ZMD21WHdZ8aYDvZ6ydM+Pj74qLdnZsgxDA3bZXct04ynpe/j46EnOmUu351Ce1mGKG711wJJFVickF7e2zW6Y723FT/UC0bO3xgU6MMEi36IFPr4HGlc2/tWAZwY2qLQ+kzjwGNH6WhJLonJHJJEWDGGnL9P3xGp5ft6u8VwA1zo+l9Wx5deQsNik4Z2gi8pdxDEp+jmoTZLd+nIvu03lIne3jrbwxPEBt9N3XNV3Vh8zKTzS8ZhWIK+3GonYpGxaOe1AtST2s5zNtZAhByTzw6YSc X-OriginatorOrg: windriver.com X-MS-Exchange-CrossTenant-Network-Message-Id: a8a4add3-2433-4294-a65b-08d92a37331d X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4545.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jun 2021 04:38:01.9690 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ddb2873-a1ad-4a18-ae4e-4644631433be X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: yxhLOTSZKn9ZCm93Ra9SZVfSJ1b/OnAj9owEGMiXuxrsRtfCbAh8vs6k2xIxsqspGLh39ejGtavuKWBESU8T8Z6934BZf9l9OSooinOIgKY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB2828 X-Proofpoint-GUID: sHJfLBXgafWSWSL4i1WFEA---F0KtsXt X-Proofpoint-ORIG-GUID: sHJfLBXgafWSWSL4i1WFEA---F0KtsXt X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-06-08_01:2021-06-04,2021-06-08 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 phishscore=0 spamscore=0 mlxlogscore=999 priorityscore=1501 adultscore=0 lowpriorityscore=0 impostorscore=0 suspectscore=0 malwarescore=0 clxscore=1015 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2106080030 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: Valentin Schneider commit 475ea6c60279e9f2ddf7e4cf2648cd8ae0608361 upstream. Will reported that the 'XXX __migrate_task() can fail' in migration_cpu_stop() can happen, and it *is* sort of a big deal. Looking at it some more, one will note there is a glaring hole in the deferred CPU selection: (w/ CONFIG_CPUSET=n, so that the affinity mask passed via taskset doesn't get AND'd with cpu_online_mask) $ taskset -pc 0-2 $PID # offline CPUs 3-4 $ taskset -pc 3-5 $PID `\ $PID may stay on 0-2 due to the cpumask_any_distribute() picking an offline CPU and __migrate_task() refusing to do anything due to cpu_is_allowed(). set_cpus_allowed_ptr() goes to some length to pick a dest_cpu that matches the right constraints vs affinity and the online/active state of the CPUs. Reuse that instead of discarding it in the affine_move_task() case. Fixes: 6d337eab041d ("sched: Fix migrate_disable() vs set_cpus_allowed_ptr()") Reported-by: Will Deacon Signed-off-by: Valentin Schneider Signed-off-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20210526205751.842360-2-valentin.schneider@arm.com Signed-off-by: Paul Gortmaker --- kernel/sched/core.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 35d8b80d7cb8..b1e87f304ade 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1942,7 +1942,6 @@ static int migration_cpu_stop(void *data) struct migration_arg *arg = data; struct set_affinity_pending *pending = arg->pending; struct task_struct *p = arg->task; - int dest_cpu = arg->dest_cpu; struct rq *rq = this_rq(); bool complete = false; struct rq_flags rf; @@ -1975,19 +1974,15 @@ static int migration_cpu_stop(void *data) if (p->migration_pending == pending) p->migration_pending = NULL; complete = true; - } - if (dest_cpu < 0) { if (cpumask_test_cpu(task_cpu(p), &p->cpus_mask)) goto out; - - dest_cpu = cpumask_any_distribute(&p->cpus_mask); } if (task_on_rq_queued(p)) - rq = __migrate_task(rq, &rf, p, dest_cpu); + rq = __migrate_task(rq, &rf, p, arg->dest_cpu); else - p->wake_cpu = dest_cpu; + p->wake_cpu = arg->dest_cpu; /* * XXX __migrate_task() can fail, at which point we might end @@ -2266,7 +2261,7 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag init_completion(&my_pending.done); my_pending.arg = (struct migration_arg) { .task = p, - .dest_cpu = -1, /* any */ + .dest_cpu = dest_cpu, .pending = &my_pending, }; @@ -2274,6 +2269,15 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag } else { pending = p->migration_pending; refcount_inc(&pending->refs); + /* + * Affinity has changed, but we've already installed a + * pending. migration_cpu_stop() *must* see this, else + * we risk a completion of the pending despite having a + * task on a disallowed CPU. + * + * Serialized by p->pi_lock, so this is safe. + */ + pending->arg.dest_cpu = dest_cpu; } } pending = p->migration_pending;