From patchwork Mon Nov 23 17:58:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 330978 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D720C63798 for ; Mon, 23 Nov 2020 17:59:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3023420758 for ; Mon, 23 Nov 2020 17:59:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="c2IbUq2W"; dkim=pass (1024-bit key) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.b="MYTLPTPv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390541AbgKWR65 (ORCPT ); Mon, 23 Nov 2020 12:58:57 -0500 Received: from mx0a-0016f401.pphosted.com ([67.231.148.174]:16822 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726817AbgKWR64 (ORCPT ); Mon, 23 Nov 2020 12:58:56 -0500 Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 0ANHtYXr032333; Mon, 23 Nov 2020 09:58:09 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0220; bh=+5zfWOoo9C8mJq2CA6DN/odbo41v8z2mssdpSRAZiuk=; b=c2IbUq2WCM2FVrMQ71tRDjNaAcePiuU/t6rgwoJUqCHk/tZiYouBDsrlG48Sp9NOG5+D kRfWfZWud9qY68ilR/kTvBRcVZwsaMupn4ZtttDeLNx2CT7yaMFBeI7c/WcGmm8XySgF uycBfyXp1p/JmAtFQEvmWvI58XBoypK0YPzqmzwjNLa0Cem6zh6b4oowG9HsYx4Flkc6 p4dZxRzxfbYFqrniYIlMvtHe9kbJM68fVkYxdcHFXfkKFYQYzM007PPqhZvO0A5x5I+V zOiigwft+nM0ak2S/DQUVFe0gPQQx6XAjuA/7m6fFGIjZ1BqSTWndxi0q0nQQO1DRaAk zw== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0a-0016f401.pphosted.com with ESMTP id 34y14u6u1m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 23 Nov 2020 09:58:09 -0800 Received: from SC-EXCH02.marvell.com (10.93.176.82) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 23 Nov 2020 09:58:07 -0800 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.172) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Mon, 23 Nov 2020 09:58:08 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kakNQczypO3n5WuKKrocEGsTUIcvN6v68w3+NYc2kmRZxQMmVzDpllvqf56khMNOHYjwcPZ1Me3zmzS7J1yjJG3jkK5VmUI0RM67G+hpxXfkoXB4OmbvFNC/iQKO++GdaKy8ADI8phcs/AcfU7xhHAiPlbdgN/h/EydSczL/f2a7GnTrLupRY1UWYJVzuvasKW8fJFSUda7h2iGS4OqmABBIHAshovJSAKW+SpABeKc/r+PRgLKRG8EDZazMj/NN3PvedYke5dbmc3uy+QTW+ZKnlP3epDLxbxvsWKcFFxR01Kp6bvXw3HHcsynFVIAXGPAvpBqLTbea5yTsLAQPEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+5zfWOoo9C8mJq2CA6DN/odbo41v8z2mssdpSRAZiuk=; b=W/stWnds9YsnhaEvRzARLe/nRZsKrR9+N/WjcDoBdDxkuoy37xVnB/+GrsqFQEVhGUvm0S2O07XJpTiYVV1lXYQZ0fbXEPHCrkLQeIeywzaxqC3zkiWdhAZtwwQpthpJY/ic2rCiUeKPCzF4cs8QFmG5D8Ul1aoDhyfeOYzVjRxms2shmEld2YzbpW8+sO73GS4C4h4lxCpaLvTwV2zCCkcEp2Loj8hnqS5jeOGumAmsPyDymV/Y8XtKYQ1gD3drbiQObOOUonwXYCoigQjXIOcNwGlV75d16rH5VXmHFR/oJSApwIoG1lBmtwkrPC0uJIQ2Ydo2ynUs8uKJVlPWkg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+5zfWOoo9C8mJq2CA6DN/odbo41v8z2mssdpSRAZiuk=; b=MYTLPTPvPhcHwdLaBkO9jiXcv5pzQDZqajOqWHyIpNLofmIFyHUXdY2dxHtWV++nSDbmn+DRCdAw/I3rlVXOF47OdsWsU+RKZoeKsDlORo4rvZvZqZzd9Mx486DPQ+ZiWFcOTbjJE+nghW0uQt/Gz4dDNvmpvvgOSLDsp6zBvyE= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MW3PR18MB3530.namprd18.prod.outlook.com (2603:10b6:303:55::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3589.22; Mon, 23 Nov 2020 17:58:06 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::e17f:37fb:4eb3:c703]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::e17f:37fb:4eb3:c703%4]) with mapi id 15.20.3564.039; Mon, 23 Nov 2020 17:58:06 +0000 From: Alex Belits To: "nitesh@redhat.com" , "frederic@kernel.org" CC: Prasun Kapoor , "linux-api@vger.kernel.org" , "davem@davemloft.net" , "trix@redhat.com" , "mingo@kernel.org" , "catalin.marinas@arm.com" , "rostedt@goodmis.org" , "linux-kernel@vger.kernel.org" , "peterx@redhat.com" , "tglx@linutronix.de" , "linux-arch@vger.kernel.org" , "mtosatti@redhat.com" , "will@kernel.org" , "peterz@infradead.org" , "leon@sidebranch.com" , "linux-arm-kernel@lists.infradead.org" , "pauld@redhat.com" , "netdev@vger.kernel.org" Subject: [PATCH v5 6/9] task_isolation: arch/arm64: enable task isolation functionality Thread-Topic: [PATCH v5 6/9] task_isolation: arch/arm64: enable task isolation functionality Thread-Index: AQHWwcIyUgmIaBCwmkq+zZ19oo0twQ== Date: Mon, 23 Nov 2020 17:58:06 +0000 Message-ID: <91496c0cf8d24717a2641fc4d02063f3f10dc733.camel@marvell.com> References: <8d887e59ca713726f4fcb25a316e1e932b02823e.camel@marvell.com> In-Reply-To: <8d887e59ca713726f4fcb25a316e1e932b02823e.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: redhat.com; dkim=none (message not signed) header.d=none; redhat.com; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: badb8142-3f0b-4ab7-4572-08d88fd954eb x-ms-traffictypediagnostic: MW3PR18MB3530: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: y7cpnyqMXnf/qIeNvze+DmXHo+E++Mwy4et/kpZmEc7KqSuoAbtMtnemN6shMt0awQJelgKO86j2pWjYTGQnoel0U6VUfwN45NDxCk6vMxrs78VVVpldHGAYBWp/LS4aQf0GHLxhB4TF/qlXatqaXVBujn9ZDYt7vH+vQlZ7E/h8KhBT+tdk2fboFuv9Ber5gI9Ocesof1DckxO46Jk4E8a17di8eeEQOklm2/gBCyTkRKDRmHLic3193oNjBfdeUkZColF56WBYingjMi/Y08cxx+UdPNjdfcxplbVxx4LfDN1f3pG3P6bG0nu+hTU/ x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MW2PR18MB2267.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(346002)(376002)(39860400002)(136003)(396003)(2906002)(2616005)(71200400001)(5660300002)(6512007)(83380400001)(110136005)(26005)(54906003)(6486002)(186003)(6506007)(36756003)(66946007)(8676002)(316002)(478600001)(8936002)(86362001)(7416002)(66476007)(66556008)(64756008)(66446008)(91956017)(76116006)(4326008); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: ndd38rUCP6CTRl+l53zcuDt4JXCMljHr99x1wbZo8l6GkMPxSmcqkYmJqrqeRGYZy8HMJtWsB/BNsr6ydi0mUNZh8CsMJzLEmzcSyMHQJYuuoswtligrKTfCxtqXEHO4vPK5qJIE1voLGF4FkbdL4OF01iX0ykRaEZUSKlQhcAHDd67ZBAa1/Q6pU3pHBWAzJ3JkaNwzBUxNoB0RXAbDJu9y1uMxu04SfjK2W94p5IhPGgxPtncwkQgE1jb5gikZD74t6+yJTAeHs70JsmbRRuTYZjmUTdqAvt5Vt7KqIfQ9082KC/s6iwdsbw11WO2qFWhBNa3Ge6Mbd5TCCZkUm9lKajR5C4m2dA1l/XhtOwwsMHx3CGUlTPAjok3IUAhOK/SGjxO6dn1zL9IfL/RvfpbCezNOPZtmeBRsfIAgTAvQ9pZm/5dnx/Bi5Oh4VaR36bQSLceXtU9k5/tQRVZh9CB/GqMyVk26uIibdtXId0YBOaqjArEqHA5H19tCUZjJXbCOw76LqJ8dSn4KMRXzzI2XUQluJZO5flpHz/WNwB93aZNPTXA3MwFQwW3ZD6aii3W+SuLnXQIwRTvXXGn9UZAS/n/z7UsfkK5tJk0+lHU2WvUw6YzYW5odgjukmMceAuuc8oosrFTjgFvRA8fGPRJoWMd4JptZj1DJD+Qf34tzvAHnZnJ+sRWeqILSQjFDD2NstpRTZkKliLCDU9SRle6dDBDz40RoOkk9DiEtrkZMqqqhpKS9iqh53GjIMQWcEG+89BmdAhoimrwZdt9IVWByeElNAYt3VZRydTLTtchrCNIirmnzgkZwXPERe68nh/vdc4OiSf8QC2axalR1hJbBWbIBM43s1cs+R40yUnfW55ghW/iSZa8Ewku69X68mSDK2xGbJxeutxWFTzZxBQ== Content-ID: MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: badb8142-3f0b-4ab7-4572-08d88fd954eb X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Nov 2020 17:58:06.6176 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: lgCYBa3/FtYA4udOMnenVmYAaoGIDmO9d1541TkqzFnTqqVdNnrFta1BpM5Y6Ebna0Oui74RYW3bWrHo+zUHLA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR18MB3530 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-11-23_17:2020-11-23,2020-11-23 signatures=0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org In do_notify_resume(), call task_isolation_before_pending_work_check() first, to report isolation breaking, then after handling all pending work, call task_isolation_start() for TIF_TASK_ISOLATION tasks. Add _TIF_TASK_ISOLATION to _TIF_WORK_MASK, and _TIF_SYSCALL_WORK, define local NOTIFY_RESUME_LOOP_FLAGS to check in the loop, since we don't clear _TIF_TASK_ISOLATION in the loop. Early kernel entry code calls task_isolation_kernel_enter(). In particular: Vectors: el1_sync -> el1_sync_handler() -> task_isolation_kernel_enter() el1_irq -> asm_nmi_enter(), handle_arch_irq() el1_error -> do_serror() el0_sync -> el0_sync_handler() el0_irq -> handle_arch_irq() el0_error -> do_serror() el0_sync_compat -> el0_sync_compat_handler() el0_irq_compat -> handle_arch_irq() el0_error_compat -> do_serror() SDEI entry: __sdei_asm_handler -> __sdei_handler() -> nmi_enter() Functions called from there: asm_nmi_enter() -> nmi_enter() -> task_isolation_kernel_enter() asm_nmi_exit() -> nmi_exit() -> task_isolation_kernel_return() Handlers: do_serror() -> nmi_enter() -> task_isolation_kernel_enter() or task_isolation_kernel_enter() el1_sync_handler() -> task_isolation_kernel_enter() el0_sync_handler() -> task_isolation_kernel_enter() el0_sync_compat_handler() -> task_isolation_kernel_enter() handle_arch_irq() is irqchip-specific, most call handle_domain_irq() There is a separate patch for irqchips that do not follow this rule. handle_domain_irq() -> task_isolation_kernel_enter() do_handle_IPI() -> task_isolation_kernel_enter() (may be redundant) nmi_enter() -> task_isolation_kernel_enter() Signed-off-by: Chris Metcalf [abelits@marvell.com: simplified to match kernel 5.10] Signed-off-by: Alex Belits --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/barrier.h | 1 + arch/arm64/include/asm/thread_info.h | 7 +++++-- arch/arm64/kernel/entry-common.c | 7 +++++++ arch/arm64/kernel/ptrace.c | 10 ++++++++++ arch/arm64/kernel/signal.c | 13 ++++++++++++- arch/arm64/kernel/smp.c | 3 +++ 7 files changed, 39 insertions(+), 3 deletions(-) -- 2.20.1 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 1515f6f153a0..fc958d8d8945 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -141,6 +141,7 @@ config ARM64 select HAVE_ARCH_PREL32_RELOCATIONS select HAVE_ARCH_SECCOMP_FILTER select HAVE_ARCH_STACKLEAK + select HAVE_ARCH_TASK_ISOLATION select HAVE_ARCH_THREAD_STRUCT_WHITELIST select HAVE_ARCH_TRACEHOOK select HAVE_ARCH_TRANSPARENT_HUGEPAGE diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h index c3009b0e5239..ad5a6dd380cf 100644 --- a/arch/arm64/include/asm/barrier.h +++ b/arch/arm64/include/asm/barrier.h @@ -49,6 +49,7 @@ #define dma_rmb() dmb(oshld) #define dma_wmb() dmb(oshst) +#define instr_sync() isb() /* * Generate a mask for array_index__nospec() that is ~0UL when 0 <= idx < sz * and 0 otherwise. diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index 1fbab854a51b..3321c69c46fe 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -68,6 +68,7 @@ void arch_release_task_struct(struct task_struct *tsk); #define TIF_UPROBE 4 /* uprobe breakpoint or singlestep */ #define TIF_FSCHECK 5 /* Check FS is USER_DS on return */ #define TIF_MTE_ASYNC_FAULT 6 /* MTE Asynchronous Tag Check Fault */ +#define TIF_TASK_ISOLATION 7 /* task isolation enabled for task */ #define TIF_SYSCALL_TRACE 8 /* syscall trace active */ #define TIF_SYSCALL_AUDIT 9 /* syscall auditing */ #define TIF_SYSCALL_TRACEPOINT 10 /* syscall tracepoint for ftrace */ @@ -87,6 +88,7 @@ void arch_release_task_struct(struct task_struct *tsk); #define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED) #define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME) #define _TIF_FOREIGN_FPSTATE (1 << TIF_FOREIGN_FPSTATE) +#define _TIF_TASK_ISOLATION (1 << TIF_TASK_ISOLATION) #define _TIF_SYSCALL_TRACE (1 << TIF_SYSCALL_TRACE) #define _TIF_SYSCALL_AUDIT (1 << TIF_SYSCALL_AUDIT) #define _TIF_SYSCALL_TRACEPOINT (1 << TIF_SYSCALL_TRACEPOINT) @@ -101,11 +103,12 @@ void arch_release_task_struct(struct task_struct *tsk); #define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE | \ - _TIF_UPROBE | _TIF_FSCHECK | _TIF_MTE_ASYNC_FAULT) + _TIF_UPROBE | _TIF_FSCHECK | \ + _TIF_MTE_ASYNC_FAULT | _TIF_TASK_ISOLATION) #define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ - _TIF_SYSCALL_EMU) + _TIF_SYSCALL_EMU | _TIF_TASK_ISOLATION) #ifdef CONFIG_SHADOW_CALL_STACK #define INIT_SCS \ diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 43d4c329775f..8152760de683 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include @@ -77,6 +78,8 @@ asmlinkage void notrace el1_sync_handler(struct pt_regs *regs) { unsigned long esr = read_sysreg(esr_el1); + task_isolation_kernel_enter(); + switch (ESR_ELx_EC(esr)) { case ESR_ELx_EC_DABT_CUR: case ESR_ELx_EC_IABT_CUR: @@ -249,6 +252,8 @@ asmlinkage void notrace el0_sync_handler(struct pt_regs *regs) { unsigned long esr = read_sysreg(esr_el1); + task_isolation_kernel_enter(); + switch (ESR_ELx_EC(esr)) { case ESR_ELx_EC_SVC64: el0_svc(regs); @@ -321,6 +326,8 @@ asmlinkage void notrace el0_sync_compat_handler(struct pt_regs *regs) { unsigned long esr = read_sysreg(esr_el1); + task_isolation_kernel_enter(); + switch (ESR_ELx_EC(esr)) { case ESR_ELx_EC_SVC32: el0_svc_compat(regs); diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index f49b349e16a3..2941f2b16796 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include @@ -1803,6 +1804,15 @@ int syscall_trace_enter(struct pt_regs *regs) return NO_SYSCALL; } + /* + * In task isolation mode, we may prevent the syscall from + * running, and if so we also deliver a signal to the process. + */ + if (test_thread_flag(TIF_TASK_ISOLATION)) { + if (task_isolation_syscall(regs->syscallno) == -1) + return NO_SYSCALL; + } + /* Do the secure computing after ptrace; failures should be fast. */ if (secure_computing() == -1) return NO_SYSCALL; diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index a8184cad8890..e3a82b75e39d 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include @@ -911,6 +912,11 @@ static void do_signal(struct pt_regs *regs) restore_saved_sigmask(); } +#define NOTIFY_RESUME_LOOP_FLAGS \ + (_TIF_NEED_RESCHED | _TIF_SIGPENDING | _TIF_NOTIFY_RESUME | \ + _TIF_FOREIGN_FPSTATE | _TIF_UPROBE | _TIF_FSCHECK | \ + _TIF_MTE_ASYNC_FAULT) + asmlinkage void do_notify_resume(struct pt_regs *regs, unsigned long thread_flags) { @@ -921,6 +927,8 @@ asmlinkage void do_notify_resume(struct pt_regs *regs, */ trace_hardirqs_off(); + task_isolation_before_pending_work_check(); + do { /* Check valid user FS if needed */ addr_limit_user_check(); @@ -956,7 +964,10 @@ asmlinkage void do_notify_resume(struct pt_regs *regs, local_daif_mask(); thread_flags = READ_ONCE(current_thread_info()->flags); - } while (thread_flags & _TIF_WORK_MASK); + } while (thread_flags & NOTIFY_RESUME_LOOP_FLAGS); + + if (thread_flags & _TIF_TASK_ISOLATION) + task_isolation_start(); } unsigned long __ro_after_init signal_minsigstksz; diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 18e9727d3f64..4401eac4710c 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include @@ -890,6 +891,8 @@ static void do_handle_IPI(int ipinr) { unsigned int cpu = smp_processor_id(); + task_isolation_kernel_enter(); + if ((unsigned)ipinr < NR_IPI) trace_ipi_entry_rcuidle(ipi_types[ipinr]);