From patchwork Wed Nov 22 09:42:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746299 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 619532BAE2; Wed, 22 Nov 2023 09:43:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="q2GNnt1V" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 141ADC433C7; Wed, 22 Nov 2023 09:42:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646179; bh=LBKDiUG7NBwWsYEeStbCS35c5iJdRmlcOY1N9hBogGM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=q2GNnt1Vg0QJQRjesae4sFYhYpcYrw8xuFx8kU7s+OytKR4wzd1yCHi8+32jxqUC1 AWXRxXmwzXvoYtmicILUtYBc+WxAtCvvhpW99BVlem8s8EkD4oOaigl455Hy4ZYMx3 iZ58lbARbqthj3Fz/zCsx8LDPev0F8etIRFBxbkiJ/7XbkI11XtDhT+FZCdf/Y/fS8 JPsY05zr1HGkfkKAUW1l44mtjfkf1MS4IEFL6TYiUmj3frTdbz5owGw53mqr56wpEY ia86O0YKKUQGxM+0JZoc3SjjbemxCBARirVY0+Wq88KNiqBT29fwGRPlkuuiA9heEW AQSiIbLnkjP/Q== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:11 +0000 Subject: [PATCH v7 01/39] arm64/mm: Restructure arch_validate_flags() for extensibility Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-1-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=1303; i=broonie@kernel.org; h=from:subject:message-id; bh=LBKDiUG7NBwWsYEeStbCS35c5iJdRmlcOY1N9hBogGM=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXcz3ufb4BdT3G27Wye1QtqLYalH9gsoNYB6OX 7EyWk64VmOJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3M9wAKCRAk1otyXVSH 0HxEB/9DTjmmFbNFvHu6r1ymZm8lbbHpqTlnIGPujl+4mFYJWSjhEaVe181wljq4nhtxPsm0u3a dtx7lvzR0Vo0RuGiNEprMGOGHjwlvyFifKfYoG4Hx4DMRzjmHYt9ADFh8JeBcEB7b6VU+8Q54LP choSLtIxGB9bEyLaaFCHrXHn4bsHJebYUBC4hxEk6shMvhPPnKS5YH4sDQVR8xrEqqwA/0e0Nzc 5tg52h+rCta72nz4wUxtrmsCeWCqAeEWxTWayPeXLN4EyGQuAeIcLmLfXs+iQZGDJHimnFgJB+Q IZrJy/B0h5hSoxkO3lXClEw8RP6mm5v2VaonL/9R49GDeD3t X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Currently arch_validate_flags() is written in a very non-extensible fashion, returning immediately if MTE is not supported and writing the MTE check as a direct return. Since we will want to add more checks for GCS refactor the existing code to be more extensible, no functional change intended. Signed-off-by: Mark Brown --- arch/arm64/include/asm/mman.h | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/mman.h b/arch/arm64/include/asm/mman.h index 5966ee4a6154..c21849ffdd88 100644 --- a/arch/arm64/include/asm/mman.h +++ b/arch/arm64/include/asm/mman.h @@ -52,11 +52,17 @@ static inline bool arch_validate_prot(unsigned long prot, static inline bool arch_validate_flags(unsigned long vm_flags) { - if (!system_supports_mte()) - return true; + if (system_supports_mte()) { + /* + * only allow VM_MTE if VM_MTE_ALLOWED has been set + * previously + */ + if ((vm_flags & VM_MTE) && !(vm_flags & VM_MTE_ALLOWED)) + return false; + } + + return true; - /* only allow VM_MTE if VM_MTE_ALLOWED has been set previously */ - return !(vm_flags & VM_MTE) || (vm_flags & VM_MTE_ALLOWED); } #define arch_validate_flags(vm_flags) arch_validate_flags(vm_flags) From patchwork Wed Nov 22 09:42:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746298 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 26CE61E524; Wed, 22 Nov 2023 09:43:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VS4V1mmE" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BB115C433C7; Wed, 22 Nov 2023 09:43:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646195; bh=EkyJ2sjNqguwXqxmxIIfJqgpolmeU80tDQ1mr2D5TfQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=VS4V1mmE0ZeLoutxYILjpbDNn60e5Tw0OcoSbTasgmO/VejZ//vSk5A4xO+WKFwqr Vt+kO1vC6TXvQBTVbFHHuAp1E9yemr98c7HwHxCb8pBTuq2ba5qEzAzV28dH5jUvlU zMV7HFvQUN1V1bnbNkCq15iyICNQRqCvaWVMvZx4cPo7LnC/W6QUANmM/9t8FXhwrZ eQ+p2PRpxpv2DgKFpKGxSozrGbwpYTwkR7GsL9szoTE6zP6Poc7ZZozJj+u3Frk+Ot uAG2FJIIdF6HfISncjYll7FQ2wapnt8Bc+ndxxZBDEI5t7GlqrUHlXf3Jwmo1mtA3k w60pzdRod63Lw== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:13 +0000 Subject: [PATCH v7 03/39] mman: Add map_shadow_stack() flags Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-3-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=1759; i=broonie@kernel.org; h=from:subject:message-id; bh=EkyJ2sjNqguwXqxmxIIfJqgpolmeU80tDQ1mr2D5TfQ=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXcz5ctjongA13l+3s8rDr6lCcYzCiV/EZ+w01 Sjq0p1+agmJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3M+QAKCRAk1otyXVSH 0BmeB/0fdILWv3xnmoh3iIUnb+EGSP88bn3NvTdnqSR6EOCHL4XrJPUgwQteJL4EkrLBwM7IXDO phEWSI0/nmWusNr3v9UW6kZ1NmYF+253MWc4v54TjgzG9k4mafDzYYfcZzuGeMZK6JCCAWkGLr0 wv9DrCEa/2jyyNb5n2y+K8nYdNHuJuRrS+1Upl0ZSZ1w9M1JxGUsJgLvnPmoIDgcox7Wg6uxeHf BzbRTK0693utYKeBvr3gTPU/c80PO0X47jTSfpB+WRA2FLQW9FCrDBwc3Z6WFvN2+z7guWL/86q nFREV+64zb5OuzopYwcn7fjb3cADnfSnvSBqlAmHmHfk5T6+ X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB In preparation for adding arm64 GCS support make the map_shadow_stack() SHADOW_STACK_SET_TOKEN flag generic and add _SET_MARKER. The existing flag indicats that a token usable for stack switch should be added to the top of the newly mapped GCS region while the new flag indicates that a top of stack marker suitable for use by unwinders should be added above that. For arm64 the top of stack marker is all bits 0. Signed-off-by: Mark Brown --- arch/x86/include/uapi/asm/mman.h | 3 --- include/uapi/asm-generic/mman.h | 4 ++++ 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/uapi/asm/mman.h b/arch/x86/include/uapi/asm/mman.h index 46cdc941f958..ac1e6277212b 100644 --- a/arch/x86/include/uapi/asm/mman.h +++ b/arch/x86/include/uapi/asm/mman.h @@ -5,9 +5,6 @@ #define MAP_32BIT 0x40 /* only give out 32bit addresses */ #define MAP_ABOVE4G 0x80 /* only map above 4GB */ -/* Flags for map_shadow_stack(2) */ -#define SHADOW_STACK_SET_TOKEN (1ULL << 0) /* Set up a restore token in the shadow stack */ - #include #endif /* _ASM_X86_MMAN_H */ diff --git a/include/uapi/asm-generic/mman.h b/include/uapi/asm-generic/mman.h index 57e8195d0b53..d6a282687af5 100644 --- a/include/uapi/asm-generic/mman.h +++ b/include/uapi/asm-generic/mman.h @@ -19,4 +19,8 @@ #define MCL_FUTURE 2 /* lock all future mappings */ #define MCL_ONFAULT 4 /* lock all pages that are faulted in */ +#define SHADOW_STACK_SET_TOKEN (1ULL << 0) /* Set up a restore token in the shadow stack */ +#define SHADOW_STACK_SET_MARKER (1ULL << 1) /* Set up a top of stack merker in the shadow stack */ + + #endif /* __ASM_GENERIC_MMAN_H */ From patchwork Wed Nov 22 09:42:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746297 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25C071E524; Wed, 22 Nov 2023 09:43:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Qv4wreSI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 37A19C433C9; Wed, 22 Nov 2023 09:43:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646210; bh=L9ZRGf4mNC+FkwLFIfy9nFVoaeaPEXiAYS+7gHUi1aM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Qv4wreSIHv5Ype3rrIep2FDZ7oa0JFgTAurBBMARGex0o4XvAcaDvVDagnPxYpBs8 S0dZwlyIztWNnmtN29WH5IFf3Z4N5THAbBxDrk3ea8UARZshQREWPxgVR7Wv+igHTp M4Dd41SuepuOCJKUMQZRg3SP3Dlid7y4kz5nzFsXWlzTe5qHw1NvB7Cq67vDWXIc/1 ziE5HMgNniJ1C0aps5k2QVT1nyg5FKCSD9+DzweQF5qsd9Cm8/uxx+vVc85FFY1mC8 Dpg8R0WYdDuXT31mv3nxtJ11Fmfx3OguZUhTcNmu3wv5jvR0QBPbwLkT8ZPKRp1EKV 8SiewU3eO5sPw== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:15 +0000 Subject: [PATCH v7 05/39] arm64/gcs: Document the ABI for Guarded Control Stacks Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-5-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=10660; i=broonie@kernel.org; h=from:subject:message-id; bh=L9ZRGf4mNC+FkwLFIfy9nFVoaeaPEXiAYS+7gHUi1aM=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXcz6rEW3Yum0dqw0RdOxkewLl5v2bk8+GVDh6 TDhRy8sFkeJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3M+gAKCRAk1otyXVSH 0A3WB/9GAbpWImDGZqMaksFSfDV0vvm48cRi9UQP9h3li2xxngiotbFbNoY/F2rH7KBYTWmqEn/ hVUSdxoYyrkr2+FOtj2FGeqZ6i1NzhUhfn2FB8aQRHueO6Hmres3zdyKTON/0ZBnlxGFdX5QMMH S0b7vDZL1VNwR+2bfJJGv1PBvTDiGih/85wN9kG5J3YtMzVxlDlzdzsFNFwtgPm3CHIqo5Gkqm1 zHdfoXStP+WXM2PsCNcej+/PeTsPUigGoMSuG40U+vtMCtd75DrLgwrL5Kaz6fjvtaOJkvM7aar x6OErRhAEPoY63b68qVZC7SoZUTHnzKJSsj4+3yTty/wK7dv X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Add some documentation of the userspace ABI for Guarded Control Stacks. Signed-off-by: Mark Brown --- Documentation/arch/arm64/gcs.rst | 233 +++++++++++++++++++++++++++++++++++++ Documentation/arch/arm64/index.rst | 1 + 2 files changed, 234 insertions(+) diff --git a/Documentation/arch/arm64/gcs.rst b/Documentation/arch/arm64/gcs.rst new file mode 100644 index 000000000000..c45c0326836a --- /dev/null +++ b/Documentation/arch/arm64/gcs.rst @@ -0,0 +1,233 @@ +=============================================== +Guarded Control Stack support for AArch64 Linux +=============================================== + +This document outlines briefly the interface provided to userspace by Linux in +order to support use of the ARM Guarded Control Stack (GCS) feature. + +This is an outline of the most important features and issues only and not +intended to be exhaustive. + + + +1. General +----------- + +* GCS is an architecture feature intended to provide greater protection + against return oriented programming (ROP) attacks and to simplify the + implementation of features that need to collect stack traces such as + profiling. + +* When GCS is enabled a separate guarded control stack is maintained by the + PE which is writeable only through specific GCS operations. This + stores the call stack only, when a procedure call instruction is + performed the current PC is pushed onto the GCS and on RET the + address in the LR is verified against that on the top of the GCS. + +* When active current GCS pointer is stored in the system register + GCSPR_EL0. This is readable by userspace but can only be updated + via specific GCS instructions. + +* The architecture provides instructions for switching between guarded + control stacks with checks to ensure that the new stack is a valid + target for switching. + +* The functionality of GCS is similar to that provided by the x86 Shadow + Stack feature, due to sharing of userspace interfaces the ABI refers to + shadow stacks rather than GCS. + +* Support for GCS is reported to userspace via HWCAP2_GCS in the aux vector + AT_HWCAP2 entry. + +* GCS is enabled per thread. While there is support for disabling GCS + at runtime this should be done with great care. + +* GCS memory access faults are reported as normal memory access faults. + +* GCS specific errors (those reported with EC 0x2d) will be reported as + SIGSEGV with a si_code of SEGV_CPERR (control protection error). + +* GCS is supported only for AArch64. + +* On systems where GCS is supported GCSPR_EL0 is always readable by EL0 + regardless of the GCS configuration for the thread. + +* The architecture supports enabling GCS without verifying that return values + in LR match those in the GCS, the LR will be ignored. This is not supported + by Linux. + +* EL0 GCS entries with bit 63 set are reserved for use, one such use is defined + below for signals and should be ignored when parsing the stack if not + understood. + + +2. Enabling and disabling Guarded Control Stacks +------------------------------------------------- + +* GCS is enabled and disabled for a thread via the PR_SET_SHADOW_STACK_STATUS + prctl(), this takes a single flags argument specifying which GCS features + should be used. + +* When set PR_SHADOW_STACK_ENABLE flag allocates a Guarded Control Stack + and enables GCS for the thread, enabling the functionality controlled by + GCSCRE0_EL1.{nTR, RVCHKEN, PCRSEL}. + +* When set the PR_SHADOW_STACK_PUSH flag enables the functionality controlled + by GCSCRE0_EL1.PUSHMEn, allowing explicit GCS pushes. + +* When set the PR_SHADOW_STACK_WRITE flag enables the functionality controlled + by GCSCRE0_EL1.STREn, allowing explicit stores to the Guarded Control Stack. + +* Any unknown flags will cause PR_SET_SHADOW_STACK_STATUS to return -EINVAL. + +* PR_LOCK_SHADOW_STACK_STATUS is passed a bitmask of features with the same + values as used for PR_SET_SHADOW_STACK_STATUS. Any future changes to the + status of the specified GCS mode bits will be rejected. + +* PR_LOCK_SHADOW_STACK_STATUS allows any bit to be locked, this allows + userspace to prevent changes to any future features. + +* There is no support for a process to remove a lock that has been set for + it. + +* PR_SET_SHADOW_STACK_STATUS and PR_LOCK_SHADOW_STACK_STATUS affect only the + thread that called them, any other running threads will be unaffected. + +* New threads inherit the GCS configuration of the thread that created them. + +* GCS is disabled on exec(). + +* The current GCS configuration for a thread may be read with the + PR_GET_SHADOW_STACK_STATUS prctl(), this returns the same flags that + are passed to PR_SET_SHADOW_STACK_STATUS. + +* If GCS is disabled for a thread after having previously been enabled then + the stack will remain allocated for the lifetime of the thread. At present + any attempt to reenable GCS for the thread will be rejected, this may be + revisited in future. + +* It should be noted that since enabling GCS will result in GCS becoming + active immediately it is not normally possible to return from the function + that invoked the prctl() that enabled GCS. It is expected that the normal + usage will be that GCS is enabled very early in execution of a program. + + + +3. Allocation of Guarded Control Stacks +---------------------------------------- + +* When GCS is enabled for a thread a new Guarded Control Stack will be + allocated for it of size RLIMIT_STACK or 4 gigabytes, whichever is + smaller. + +* When a new thread is created by a thread which has GCS enabled then a + new Guarded Control Stack will be allocated for the new thread with + half the size of the standard stack. + +* When a stack is allocated by enabling GCS or during thread creation then + the top 8 bytes of the stack will be initialised to 0 and GCSPR_EL0 will + be set to point to the address of this 0 value, this can be used to + detect the top of the stack. + +* Additional Guarded Control Stacks can be allocated using the + map_shadow_stack() system call. + +* Stacks allocated using map_shadow_stack() can optionally have an end of + stack marker and cap placed at the top of the stack. If the flag + SHADOW_STACK_SET_TOKEN is specified a cap will be placed on the stack, + if SHADOW_STACK_SET_MARKER is not specified the cap will be the top 8 + bytes of the stack and if it is specified then the cap will be the next + 8 bytes. While specifying just SHADOW_STACK_SET_MARKER by itself is + valid since the marker is all bits 0 it has no observable effect. + +* Stacks allocated using map_shadow_stack() must have a size which is a + multiple of 8 bytes larger than 8 bytes and must be 8 bytes aligned. + +* An address can be specified to map_shadow_stack(), if one is provided then + it must be aligned to a page boundary. + +* When a thread is freed the Guarded Control Stack initially allocated for + that thread will be freed. Note carefully that if the stack has been + switched this may not be the stack currently in use by the thread. + + +4. Signal handling +-------------------- + +* A new signal frame record gcs_context encodes the current GCS mode and + pointer for the interrupted context on signal delivery. This will always + be present on systems that support GCS. + +* The record contains a flag field which reports the current GCS configuration + for the interrupted context as PR_GET_SHADOW_STACK_STATUS would. + +* The signal handler is run with the same GCS configuration as the interrupted + context. + +* When GCS is enabled for the interrupted thread a signal handling specific + GCS cap token will be written to the GCS, this is an architectural GCS cap + token with bit 63 set and the token type (bits 0..11) all clear. The + GCSPR_EL0 reported in the signal frame will point to this cap token. + +* The signal handler will use the same GCS as the interrupted context. + +* When GCS is enabled on signal entry a frame with the address of the signal + return handler will be pushed onto the GCS, allowing return from the signal + handler via RET as normal. This will not be reported in the gcs_context in + the signal frame. + + +5. Signal return +----------------- + +When returning from a signal handler: + +* If there is a gcs_context record in the signal frame then the GCS flags + and GCSPR_EL0 will be restored from that context prior to further + validation. + +* If there is no gcs_context record in the signal frame then the GCS + configuration will be unchanged. + +* If GCS is enabled on return from a signal handler then GCSPR_EL0 must + point to a valid GCS signal cap record, this will be popped from the + GCS prior to signal return. + +* If the GCS configuration is locked when returning from a signal then any + attempt to change the GCS configuration will be treated as an error. This + is true even if GCS was not enabled prior to signal entry. + +* GCS may be disabled via signal return but any attempt to enable GCS via + signal return will be rejected. + + +6. ptrace extensions +--------------------- + +* A new regset NT_ARM_GCS is defined for use with PTRACE_GETREGSET and + PTRACE_SETREGSET. + +* Due to the complexity surrounding allocation and deallocation of stacks and + lack of practical application it is not possible to enable GCS via ptrace. + GCS may be disabled via the ptrace interface. + +* Other GCS modes may be configured via ptrace. + +* Configuration via ptrace ignores locking of GCS mode bits. + + +7. ELF coredump extensions +--------------------------- + +* NT_ARM_GCS notes will be added to each coredump for each thread of the + dumped process. The contents will be equivalent to the data that would + have been read if a PTRACE_GETREGSET of the corresponding type were + executed for each thread when the coredump was generated. + + + +8. /proc extensions +-------------------- + +* Guarded Control Stack pages will include "ss" in their VmFlags in + /proc//smaps. diff --git a/Documentation/arch/arm64/index.rst b/Documentation/arch/arm64/index.rst index d08e924204bf..dcf3ee3eb8c0 100644 --- a/Documentation/arch/arm64/index.rst +++ b/Documentation/arch/arm64/index.rst @@ -14,6 +14,7 @@ ARM64 Architecture booting cpu-feature-registers elf_hwcaps + gcs hugetlbpage kdump legacy_instructions From patchwork Wed Nov 22 09:42:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746296 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B07D016407; Wed, 22 Nov 2023 09:43:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qs5YB6BE" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 77F1AC433C9; Wed, 22 Nov 2023 09:43:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646225; bh=AdenxNKaKLfO1zzZHzslCIJebdi1YPs1wuXZd9U1dO4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=qs5YB6BECnMAmghk+5k7QLjQ4iTf/O6w9uwKHheCTbxYsU4dCielXXbKpJWvVJbOO eHvstQrw3PY/tc2Itmd0DGJ7a9XLXgveUYEreP0NdC2bZwRKN8Csxmok8dA98yK6jJ pc/VhN++Iddv1n65+uxrwhQVBNX4wTnb4RntXV6p1LG9HjaGajgFeuYEg7d8W2vnS6 DsVLe11iXrOgM+nfF8hv0a4E3fBTkDuQgmYeEb5x+merErzYBI1UBOYtqum9DhkAEu iz1H6w7wU93+McmzZK2pTguofmFSA0eW8JcDbFbW+BkqVIf9s/MphpA8RcRNk9/QCg +DKBkAjfQe8QQ== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:17 +0000 Subject: [PATCH v7 07/39] arm64/sysreg: Add definitions for architected GCS caps Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-7-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=1363; i=broonie@kernel.org; h=from:subject:message-id; bh=AdenxNKaKLfO1zzZHzslCIJebdi1YPs1wuXZd9U1dO4=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXcz8bmHxgHWCFRqVN8uz2M3iReNk3BZizcael fYUhMZ5DoWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3M/AAKCRAk1otyXVSH 0F7RB/4ql9JuD5+zDZkKa3IqDccJ/whQ1MzEyDazvd5D9iSOO2uFVWKf/mgBe42Co9F7iJXvawY fnM4HLJ+2I6Jpps5E2t0pRb5MPSbFPYrmiDpnpwyH+gCNdD/ISBWGGKi19hQLMzotC2usKxe7n/ ZPPgZlodquDSXAvzd6yBK/QAT+KqARuuUYjttCxdlJOeU0BHNEbhz8/owbin6JfrC1GTlcGGFai aASFR2cqBA+/sxYNdN9NjVEQv2zxSBdFJ+zVZ1qt29sVNQq4+UVEYWYK5MXwadGBn1rZoKRfQA/ I2UNt33vEzbJko8U/aAkwKFHIOuyMBeaY7xivvSZlmH0QBVz X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB The architecture defines a format for guarded control stack caps, used to mark the top of an unused GCS in order to limit the potential for exploitation via stack switching. Add definitions associated with these. Signed-off-by: Mark Brown --- arch/arm64/include/asm/sysreg.h | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 5e65f51c10d2..75a49b41fa93 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -1039,6 +1039,26 @@ #define PIRx_ELx_PERM(idx, perm) ((perm) << ((idx) * 4)) +/* + * Definitions for Guarded Control Stack + */ + +#define GCS_CAP_ADDR_MASK GENMASK(63, 12) +#define GCS_CAP_ADDR_SHIFT 12 +#define GCS_CAP_ADDR_WIDTH 52 +#define GCS_CAP_ADDR(x) FIELD_GET(GCS_CAP_ADDR_MASK, x) + +#define GCS_CAP_TOKEN_MASK GENMASK(11, 0) +#define GCS_CAP_TOKEN_SHIFT 0 +#define GCS_CAP_TOKEN_WIDTH 12 +#define GCS_CAP_TOKEN(x) FIELD_GET(GCS_CAP_TOKEN_MASK, x) + +#define GCS_CAP_VALID_TOKEN 0x1 +#define GCS_CAP_IN_PROGRESS_TOKEN 0x5 + +#define GCS_CAP(x) ((((unsigned long)x) & GCS_CAP_ADDR_MASK) | \ + GCS_CAP_VALID_TOKEN) + #define ARM64_FEATURE_FIELD_BITS 4 /* Defined for compatibility only, do not add new users. */ From patchwork Wed Nov 22 09:42:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746295 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACAFD16407; Wed, 22 Nov 2023 09:44:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="skgEBYvx" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 96852C433C8; Wed, 22 Nov 2023 09:43:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646240; bh=vd1vSvagODzOx6kbpI7IvwSkS4eBsjmFRyLwRdBrVrA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=skgEBYvx/fAYU83PbKMGhZ+jFRWnelXbJMhtlbDO94lZzbLAQu2e6sHPZRgWHd+sx gtvmAgwjSpvlSEQjQzBmlxxsXWS82g6bxtvPaFkQ8pvd5rEGx1NUa6mmsajYrS2HXJ 9SUa7yYSQ4By+Yl/Qgm1N6yHb1EuBPq31sojdUVFxdN/L6m2Rc5/En1jfZfcN6fVd0 1Hue8SThE2M1637XnM8B3AjO9hFNOP+uQE1W0maFQM8JCIjJ7WlFtq4sCyHcLk/Da8 60C8qJmM22SVHPuFPEY+HS4Qp3f7l5E4nEP4L2Ac7hl6Qa7XDz7Xjm83GZP/ILzYcJ 5IDYaXeS3YwZg== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:19 +0000 Subject: [PATCH v7 09/39] arm64/gcs: Provide put_user_gcs() Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-9-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=1037; i=broonie@kernel.org; h=from:subject:message-id; bh=vd1vSvagODzOx6kbpI7IvwSkS4eBsjmFRyLwRdBrVrA=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXcz9ntfDGYRfW3xISuNKutZJXpSjiIRaViJ0k n4sUnukccCJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3M/QAKCRAk1otyXVSH 0N3bB/44FbabVeFeFQXmXvNQGbQr73EvxPiLmtFhVunzGXVexn0MqHJIPQxDC2ugEGWeKqK+7nz mHcpRXkSvzsDxa0SVk2G9hz7TrXNC6LMK87Ap6mUxpUx/sl71tnM08rKVQzi+iw4Ye5ZPtSiDWW L5diQOrA4wDNpimxkSurVohp7umfjFe/leUGxJGrezbj+MEWgxqb/endM8PgCOg4sdzQBljGFdX WMXW9ZlcwjjEOdBjomF3WAbeNIdpvDJQHFW9t9p9cLOLrxovGepwQ/qOzb7pt0bnZZ/ely3LOX/ 7IrP/ur/Ss2aXoZkMeS2okOUMjg3vSIb9e5S9aaS+RcHJC2B X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB In order for EL1 to write to an EL0 GCS it must use the GCSSTTR instruction rather than a normal STTR. Provide a put_user_gcs() which does this. Signed-off-by: Mark Brown --- arch/arm64/include/asm/uaccess.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 22e10e79f56a..e118c3d772c8 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -445,6 +445,24 @@ static inline int gcssttr(unsigned long __user *addr, unsigned long val) return err; } +static inline void put_user_gcs(unsigned long val, unsigned long __user *addr, + int *err) +{ + int ret; + + if (!access_ok((char __user *)addr, sizeof(u64))) { + *err = -EFAULT; + return; + } + + uaccess_ttbr0_enable(); + ret = gcssttr(addr, val); + if (ret != 0) + *err = ret; + uaccess_ttbr0_disable(); +} + + #endif /* CONFIG_ARM64_GCS */ #endif /* __ASM_UACCESS_H */ From patchwork Wed Nov 22 09:42:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746294 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B055D219FF; Wed, 22 Nov 2023 09:44:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="h210R7jL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9F7A2C433CA; Wed, 22 Nov 2023 09:44:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646255; bh=gCkTJXN9hgWCfubwLDLAmQpOXrRFEWNkqe7LcsfqPUI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=h210R7jLnqkzHFNW2i+bNC/uW8728czTflygATeqPHl26mTG4iNTO/3gkMXGYPXFf PH8vpoZOA7k8ZL9aFvlCjOOCziT6vy26FbZL1HE665xm4hvAUJvhE2zp6t9hzCfeqo t3IJNIHcA7tQBbVdWXAS2/RmKl+4V3XS5f8pVjvm5p17xaH9rKDG8KJPCnyHe4rKv0 HxFWNWYXflOIzbRkGZ3oNzHRXJweodM+ntU8HkYYfwHN9YnMhlU/AiZ+ctbxSLTi3K 3WwpCNuPtS29zMJyc0EWpf9s7RymWtpI9ltp4+horOAkwOTgBysp6m7F9DquRI7H+/ 0znHAiLzjXVdg== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:21 +0000 Subject: [PATCH v7 11/39] arm64/mm: Allocate PIE slots for EL0 guarded control stack Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-11-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=2936; i=broonie@kernel.org; h=from:subject:message-id; bh=gCkTJXN9hgWCfubwLDLAmQpOXrRFEWNkqe7LcsfqPUI=; b=owGbwMvMwMWocq27KDak/QLjabUkhtTYM/8lHrpkfRDKsNrcv+HM4ftX0yZd2fNToL57XXlv5 UOvmIi3nYzGLAyMXAyyYoosa59lrEoPl9g6/9H8VzCDWJlApjBwcQrARDYpsP937Bf6G8WaH8wS YTJz+hT7PkttgeLI1xVXNCfMl/p82eTBruI2duXO/qP+f9YFX1rxK9B2h3NLW5GFu63oJoVFOc7 nV3j7lmhOOmrZMmGVgMOGJ4cce5+8qrfkvBvQJLSi70Sdr71Os2Jxf7PPrIgIzU8e9Wx3HhrOE3 ysuznx3ozqD5f5a+2631l8OJr1ce1l7+Soja23YtudX1UYLvRvdFzOZWPi92LpBmcVmZ5psxr/6 r3u15w538Em9ti7wptlYtEt+X7dAtEnsv4v5Hn4b9vTpHNKzlr3+ULlnBjm/GI4y7naUo9PZJX0 9sqNSfN2StnU8k4oPOfcdSVw4iMT1+awP0b8OSEmRlssAQ== X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Pages used for guarded control stacks need to be described to the hardware using the Permission Indirection Extension, GCS is not supported without PIE. In order to support copy on write for guarded stacks we allocate two values, one for active GCSs and one for GCS pages marked as read only prior to copy. Since the actual effect is defined using PIE the specific bit pattern used does not matter to the hardware but we choose two values which differ only in PTE_WRITE in order to help share code with non-PIE cases. Signed-off-by: Mark Brown --- arch/arm64/include/asm/pgtable-prot.h | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h index e9624f6326dd..297a645e8a0f 100644 --- a/arch/arm64/include/asm/pgtable-prot.h +++ b/arch/arm64/include/asm/pgtable-prot.h @@ -127,15 +127,23 @@ extern bool arm64_use_ng_mappings; /* 6: PTE_PXN | PTE_WRITE */ /* 7: PAGE_SHARED_EXEC PTE_PXN | PTE_WRITE | PTE_USER */ /* 8: PAGE_KERNEL_ROX PTE_UXN */ -/* 9: PTE_UXN | PTE_USER */ +/* 9: PAGE_GCS_RO PTE_UXN | PTE_USER */ /* a: PAGE_KERNEL_EXEC PTE_UXN | PTE_WRITE */ -/* b: PTE_UXN | PTE_WRITE | PTE_USER */ +/* b: PAGE_GCS PTE_UXN | PTE_WRITE | PTE_USER */ /* c: PAGE_KERNEL_RO PTE_UXN | PTE_PXN */ /* d: PAGE_READONLY PTE_UXN | PTE_PXN | PTE_USER */ /* e: PAGE_KERNEL PTE_UXN | PTE_PXN | PTE_WRITE */ /* f: PAGE_SHARED PTE_UXN | PTE_PXN | PTE_WRITE | PTE_USER */ +#define _PAGE_GCS (_PAGE_DEFAULT | PTE_NG | PTE_UXN | PTE_WRITE | PTE_USER) +#define _PAGE_GCS_RO (_PAGE_DEFAULT | PTE_NG | PTE_UXN | PTE_USER) + +#define PAGE_GCS __pgprot(_PAGE_GCS) +#define PAGE_GCS_RO __pgprot(_PAGE_GCS_RO) + #define PIE_E0 ( \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS), PIE_GCS) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS_RO), PIE_R) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_EXECONLY), PIE_X_O) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_READONLY_EXEC), PIE_RX) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED_EXEC), PIE_RWX) | \ @@ -143,6 +151,8 @@ extern bool arm64_use_ng_mappings; PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED), PIE_RW)) #define PIE_E1 ( \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS), PIE_NONE_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS_RO), PIE_NONE_O) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_EXECONLY), PIE_NONE_O) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_READONLY_EXEC), PIE_R) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED_EXEC), PIE_RW) | \ From patchwork Wed Nov 22 09:42:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746293 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 61CEF16407; Wed, 22 Nov 2023 09:44:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rTp3IgyI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 50C67C433C7; Wed, 22 Nov 2023 09:44:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646270; bh=qEFrCNf3UEgHGW1T9wtS9jRJz/KS9KwPB3lxI2GEsTI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=rTp3IgyIvQfGD4I+L4TFOAbCJv9NbsiNeIDD4RtpVFyB4VfP3fsoai+tC4z4rivNB 8csRCTYL8ZxeIFyOEnSQA4auqXLMjPB0TV4q9PD4M8mgIYeKJidyttb6RB1TqgN3n/ 7KFCMpd6Y97LNcgEUgDa3TuB2NG38oyO3WYbYyT4ZlzJStWJg4OpYtI1zx1GNbFLcQ EqvGVx0LQC1NgAEZKJhwgqD7BpUehJeQAuBogQ81mPWkWennLwbFNmzi6M9MU9ePt7 EfPhNaYflYB6bWpv0JVKbnZGesMDPr2M3n8dHIwjEE+6mu+EB7/vi4iVvbD9/uCSfp qfBU5xs5TJplw== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:23 +0000 Subject: [PATCH v7 13/39] arm64/mm: Map pages for guarded control stack Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-13-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=1837; i=broonie@kernel.org; h=from:subject:message-id; bh=qEFrCNf3UEgHGW1T9wtS9jRJz/KS9KwPB3lxI2GEsTI=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXc0A24D56zutL8bLQvd52fo0wC43SRULlWAh1 +rU/4PE4q6JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3NAAAKCRAk1otyXVSH 0HxeB/wMjuD6MVRsjymmm5q0PFo3mPpIDfrsn5bfPM93xyIkhexzp3o8MKMLZJR6AyWc5SU9ANA 9VNb9/Gq2SqulwM+T75NERQoIaX5QN0rnERoUsbZEovn2OVcl1rKgSR9V3pSg6Qe7bTHQI1RXba 52pV6lnw7K6QWG67Rs68SOCjwuPipie41cnu7hHYrKAVPUJgJ3xAaTF3zvDCHGppy2hPtemSDue xvPJt2OEUG/I7wwXpBQXAIdu64425sWrMATEdrAKaBbTDiRIX3QHYitwt4URVvOPC092qkbtgUq ocdLzNoIZ+MRZVauVM6J+z8pX5L1QoZViXQ6EXKLKo+5igsQ X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Map pages flagged as being part of a GCS as such rather than using the full set of generic VM flags. This is done using a conditional rather than extending the size of protection_map since that would make for a very sparse array. Signed-off-by: Mark Brown --- arch/arm64/include/asm/mman.h | 9 +++++++++ arch/arm64/mm/mmap.c | 13 ++++++++++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/mman.h b/arch/arm64/include/asm/mman.h index c21849ffdd88..6d3fe6433a62 100644 --- a/arch/arm64/include/asm/mman.h +++ b/arch/arm64/include/asm/mman.h @@ -61,6 +61,15 @@ static inline bool arch_validate_flags(unsigned long vm_flags) return false; } + if (system_supports_gcs() && (vm_flags & VM_SHADOW_STACK)) { + /* + * An executable GCS isn't a good idea, and the mm + * core can't cope with a shared GCS. + */ + if (vm_flags & (VM_EXEC | VM_ARM64_BTI | VM_SHARED)) + return false; + } + return true; } diff --git a/arch/arm64/mm/mmap.c b/arch/arm64/mm/mmap.c index 645fe60d000f..605d4e6edc1d 100644 --- a/arch/arm64/mm/mmap.c +++ b/arch/arm64/mm/mmap.c @@ -79,9 +79,20 @@ arch_initcall(adjust_protection_map); pgprot_t vm_get_page_prot(unsigned long vm_flags) { - pteval_t prot = pgprot_val(protection_map[vm_flags & + pteval_t prot; + + /* If this is a GCS then only interpret VM_WRITE. */ + if (system_supports_gcs() && (vm_flags & VM_SHADOW_STACK)) { + if (vm_flags & VM_WRITE) + prot = _PAGE_GCS; + else + prot = _PAGE_GCS_RO; + } else { + prot = pgprot_val(protection_map[vm_flags & (VM_READ|VM_WRITE|VM_EXEC|VM_SHARED)]); + } + /* VM_ARM64_BTI on a GCS is rejected in arch_valdiate_flags() */ if (vm_flags & VM_ARM64_BTI) prot |= PTE_GP; From patchwork Wed Nov 22 09:42:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746292 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC09B16407; Wed, 22 Nov 2023 09:44:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SyPO5tjn" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 70B4AC433C7; Wed, 22 Nov 2023 09:44:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646285; bh=eAZyVctWUIW822C6a4XspzvFhF8YmdJ2KmfW+KTTZkg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=SyPO5tjneJ1lWQ6vMvro8FogjGi2iuc2b+pcmTXuO8VKhRqYlvw00yJlvvbHsdh84 TjGzB+y7+/Y3Q920f0JAUO7oIDuDC/YZstqwF38A6ixh909kP+lzJHULCdJGX+fIzu q2JxSOEaKpH+uJ3EXxGW9/9dWeXbeAXUDeWofNO1W6RlsAf2V1MwnpzjgSB/ogU4X+ iQxSL+7QlIiwyGcC1yZ1M5CuYrxppxx40R+51v4wstppfqJqFFJl8enKcPtvvp49Ck 5eg7e8+cZV57vDaFzG3Pv8smXqbDeUH6q2VouWbCf5uw9sS0ZDl4rykCqfkbJEE4hX ADwfS0/JxHqSg== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:25 +0000 Subject: [PATCH v7 15/39] arm64/gcs: Allow GCS usage at EL0 and EL1 Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-15-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=2246; i=broonie@kernel.org; h=from:subject:message-id; bh=eAZyVctWUIW822C6a4XspzvFhF8YmdJ2KmfW+KTTZkg=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXc0CgBXsZyH7UmX+GJUnolHom10FZwrrBFtz2 fm0irI1LESJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3NAgAKCRAk1otyXVSH 0DSKB/0YQg4ct+cd7LzPnbzosVtkcpo9NOAazn/sUsJ2DtI354warKGrRY6PxciWlxzrMDgiIus HiJme7H4zGCw0IDbOmwwTfwCBkzLrvskL6mtcVElBOwwj0p22FxA2z06bnEyl+hOGCVOvm6HJGr S2L6/PMBzs/qmDYtEl04cqXyrEjy7B073aCQ39bj4OWb8e9pd1x45oG1v1BG9nEsklqU/iYgKZl +W/7rPeizJfM/yQ1ZEYuaJ73f8isMI/FMN+vA1i0JVuceYiVSaL7GypOFOYgDGwoP/Xmt6S2aUk PoVEJ298MhJe8H7+mrvEZOIqa4+3U3qcVTURWXFcrBZzuVbh X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB There is a control HCRX_EL2.GCSEn which must be set to allow GCS features to take effect at lower ELs and also fine grained traps for GCS usage at EL0 and EL1. Configure all these to allow GCS usage by EL0 and EL1. Signed-off-by: Mark Brown --- arch/arm64/include/asm/el2_setup.h | 17 +++++++++++++++++ arch/arm64/include/asm/kvm_arm.h | 4 ++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h index b7afaa026842..17672563e333 100644 --- a/arch/arm64/include/asm/el2_setup.h +++ b/arch/arm64/include/asm/el2_setup.h @@ -27,6 +27,14 @@ ubfx x0, x0, #ID_AA64MMFR1_EL1_HCX_SHIFT, #4 cbz x0, .Lskip_hcrx_\@ mov_q x0, HCRX_HOST_FLAGS + + /* Enable GCS if supported */ + mrs_s x1, SYS_ID_AA64PFR1_EL1 + ubfx x1, x1, #ID_AA64PFR1_EL1_GCS_SHIFT, #4 + cbz x1, .Lset_hcrx_\@ + orr x0, x0, #HCRX_EL2_GCSEn + +.Lset_hcrx_\@: msr_s SYS_HCRX_EL2, x0 .Lskip_hcrx_\@: .endm @@ -190,6 +198,15 @@ orr x0, x0, #HFGxTR_EL2_nPIR_EL1 orr x0, x0, #HFGxTR_EL2_nPIRE0_EL1 + /* GCS depends on PIE so we don't check it if PIE is absent */ + mrs_s x1, SYS_ID_AA64PFR1_EL1 + ubfx x1, x1, #ID_AA64PFR1_EL1_GCS_SHIFT, #4 + cbz x1, .Lset_fgt_\@ + + /* Disable traps of access to GCS registers at EL0 and EL1 */ + orr x0, x0, #HFGxTR_EL2_nGCS_EL1_MASK + orr x0, x0, #HFGxTR_EL2_nGCS_EL0_MASK + .Lset_fgt_\@: msr_s SYS_HFGRTR_EL2, x0 msr_s SYS_HFGWTR_EL2, x0 diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index b85f46a73e21..e4b6a29879b7 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -103,9 +103,9 @@ #define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H) #define HCRX_GUEST_FLAGS \ - (HCRX_EL2_SMPME | HCRX_EL2_TCR2En | \ + (HCRX_EL2_SMPME | HCRX_EL2_TCR2En | HCRX_EL2_GCSEn |\ (cpus_have_final_cap(ARM64_HAS_MOPS) ? (HCRX_EL2_MSCEn | HCRX_EL2_MCE2) : 0)) -#define HCRX_HOST_FLAGS (HCRX_EL2_MSCEn | HCRX_EL2_TCR2En) +#define HCRX_HOST_FLAGS (HCRX_EL2_MSCEn | HCRX_EL2_TCR2En | HCRX_EL2_GCSEn) /* TCR_EL2 Registers bits */ #define TCR_EL2_RES1 ((1U << 31) | (1 << 23)) From patchwork Wed Nov 22 09:42:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746291 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B65154F9A; Wed, 22 Nov 2023 09:45:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hzQJt+Qj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 11890C433C9; Wed, 22 Nov 2023 09:44:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646299; bh=aU3xh1nd0j/0DloWdWCWjw7vV0cVDgM+Blqu3VdYPtQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=hzQJt+QjotGyzdW5aJXId+R762os1ez45wKIR4uvXUTtbTXvvCKCEaxjLQyE5+11O K/3A3ODSi956zsBNcLEf8f9eLMdYm4TfOH9+z7tMTrIVtOdP9B4+I+wuRvMERobQ0h JUCP4jebtXbiUqppMsA8G+rhZvXVKolcxtB4XUuTV/t0oWGtgR4VblnkO4cK75mYwx 9qI5OOQOQdgWtzvzwMesWgtsn9CBkG0LqBT1uOptR+3dcOUP2In7IV76hwF2BkIhlS 3YE7/eFIr9cmqTmfXYxf0PZPbAKrclY8V1Irgn/qaq2Sr3mVtzvKuhhfqIj/o2sm81 2gp9m8bABLqRQ== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:27 +0000 Subject: [PATCH v7 17/39] arm64/hwcap: Add hwcap for GCS Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-17-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=2995; i=broonie@kernel.org; h=from:subject:message-id; bh=aU3xh1nd0j/0DloWdWCWjw7vV0cVDgM+Blqu3VdYPtQ=; b=owGbwMvMwMWocq27KDak/QLjabUkhtTYs8yBxYdLZ52e+l5Oe09GrZNLak6ww3nOF9bCn5eUz FSfv86zk9GYhYGRi0FWTJFl7bOMVenhElvnP5r/CmYQKxPIFAYuTgGYiBcP+z8LRdmId5Olin8b 5BuLuhe2H8zc9ODU6aXpE8zOPVi9ocN05a+2aUysdeZfqnTPtFgnF7PO8Qh455/rUsuYdEXCoWZ e5IkWn8z3tRp6xy/cEp5xxCrjXWG54XKjD0tzVhQL8U0tviwncOXjLeGMSaYXTx7ZMyv5ZSe/z9 KE6susi0KynUof7X5r/u7qHfdQVpVtXS8614fxdCQcYuox5Nz0pPapaL5lp2gH9/Zn+zrrWY6+C l5S0Cl4egfbyxm98Wl8aS1hDzNipDcaelg8K3/9QGndFfevh306TfLYk3wcJffkljH++LlJwvkP m6kTi1VGgv/ioufK5joTf4rHJIo9vjD7mFjODZHj5Qq/AA== X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Provide a hwcap to enable userspace to detect support for GCS. Signed-off-by: Mark Brown --- Documentation/arch/arm64/elf_hwcaps.rst | 3 +++ arch/arm64/include/asm/hwcap.h | 1 + arch/arm64/include/uapi/asm/hwcap.h | 1 + arch/arm64/kernel/cpufeature.c | 3 +++ arch/arm64/kernel/cpuinfo.c | 1 + 5 files changed, 9 insertions(+) diff --git a/Documentation/arch/arm64/elf_hwcaps.rst b/Documentation/arch/arm64/elf_hwcaps.rst index ced7b335e2e0..86d4ace9c75c 100644 --- a/Documentation/arch/arm64/elf_hwcaps.rst +++ b/Documentation/arch/arm64/elf_hwcaps.rst @@ -317,6 +317,9 @@ HWCAP2_LRCPC3 HWCAP2_LSE128 Functionality implied by ID_AA64ISAR0_EL1.Atomic == 0b0011. +HWCAP2_GCS + Functionality implied by ID_AA64PFR1_EL1.GCS == 0b1 + 4. Unused AT_HWCAP bits ----------------------- diff --git a/arch/arm64/include/asm/hwcap.h b/arch/arm64/include/asm/hwcap.h index cd71e09ea14d..e01e6b72a839 100644 --- a/arch/arm64/include/asm/hwcap.h +++ b/arch/arm64/include/asm/hwcap.h @@ -142,6 +142,7 @@ #define KERNEL_HWCAP_SVE_B16B16 __khwcap2_feature(SVE_B16B16) #define KERNEL_HWCAP_LRCPC3 __khwcap2_feature(LRCPC3) #define KERNEL_HWCAP_LSE128 __khwcap2_feature(LSE128) +#define KERNEL_HWCAP_GCS __khwcap2_feature(GCS) /* * This yields a mask that user programs can use to figure out what diff --git a/arch/arm64/include/uapi/asm/hwcap.h b/arch/arm64/include/uapi/asm/hwcap.h index 5023599fa278..996b5b5d4c4e 100644 --- a/arch/arm64/include/uapi/asm/hwcap.h +++ b/arch/arm64/include/uapi/asm/hwcap.h @@ -107,5 +107,6 @@ #define HWCAP2_SVE_B16B16 (1UL << 45) #define HWCAP2_LRCPC3 (1UL << 46) #define HWCAP2_LSE128 (1UL << 47) +#define HWCAP2_GCS (1UL << 48) #endif /* _UAPI__ASM_HWCAP_H */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 5477451cefa1..dab80391543f 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -2859,6 +2859,9 @@ static const struct arm64_cpu_capabilities arm64_elf_hwcaps[] = { HWCAP_CAP(ID_AA64ZFR0_EL1, I8MM, IMP, CAP_HWCAP, KERNEL_HWCAP_SVEI8MM), HWCAP_CAP(ID_AA64ZFR0_EL1, F32MM, IMP, CAP_HWCAP, KERNEL_HWCAP_SVEF32MM), HWCAP_CAP(ID_AA64ZFR0_EL1, F64MM, IMP, CAP_HWCAP, KERNEL_HWCAP_SVEF64MM), +#endif +#ifdef CONFIG_ARM64_GCS + HWCAP_CAP(ID_AA64PFR1_EL1, GCS, IMP, CAP_HWCAP, KERNEL_HWCAP_GCS), #endif HWCAP_CAP(ID_AA64PFR1_EL1, SSBS, SSBS2, CAP_HWCAP, KERNEL_HWCAP_SSBS), #ifdef CONFIG_ARM64_BTI diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c index a257da7b56fe..d557ee301464 100644 --- a/arch/arm64/kernel/cpuinfo.c +++ b/arch/arm64/kernel/cpuinfo.c @@ -130,6 +130,7 @@ static const char *const hwcap_str[] = { [KERNEL_HWCAP_SVE_B16B16] = "sveb16b16", [KERNEL_HWCAP_LRCPC3] = "lrcpc3", [KERNEL_HWCAP_LSE128] = "lse128", + [KERNEL_HWCAP_GCS] = "gcs", }; #ifdef CONFIG_COMPAT From patchwork Wed Nov 22 09:42:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746290 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42BA916407; Wed, 22 Nov 2023 09:45:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="l/tPHDZN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BEE1DC433C8; Wed, 22 Nov 2023 09:45:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646314; bh=BzV/RIjxnxMn9VbcnDWXuF4bpk5mXr34eQJiBtwP4NE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=l/tPHDZNixS3bQk2n+SQhubpzTfcvaboniqmwUh/PTbCtGaJTfzsMFW7xRRZgI7QS iDE7cilI1DbJeAIwmDbOwO1mCCxOaIuTdR27BQhkBBc96Q7dT+6aLYUnAWSz4LUfqS wca9ympps1BSiJrViPZBxwqOIYjvs+Cu2pjPu8UHjcTdLxe209jcyhOb8S+92tP93r azGWWAvwK+SVC3z+7QS1CM2oNASn21FY+PMbJs3GDyS8RPf8e/bhbklg09GkGiJIcg hdpXy/Qq3ZPD9ODrfB7CvUWAmiZpuaUPoVHyapkCxoEFHvcrqWe74qnkqE/a3ZRzPR Yb5KCzU8Uh2Xg== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:29 +0000 Subject: [PATCH v7 19/39] arm64/mm: Handle GCS data aborts Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-19-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=5688; i=broonie@kernel.org; h=from:subject:message-id; bh=BzV/RIjxnxMn9VbcnDWXuF4bpk5mXr34eQJiBtwP4NE=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXc0FZrStz9Xr8Prvr2S0JFwovux2nNPRmOouZ p8eOCwtpimJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3NBQAKCRAk1otyXVSH 0NmMB/9iOfF48aW2DH8GfqhVA2m6OttDsfi8udYamecnJPcdti7OK5IEBs6O/7ccnIxI6+Dt8H3 Dm4IDlD9qdSD04W5mGrRffn6p3plw6PmIuzC7gd4iqjAG+6QDVAnQ2vRWkISWgXJ8VojIpu5cJg jETt5f7+meyZpX7Dz6KDygBxHgMpOs4Ua9UmN/yeDU6ZMIpCMGqOljcXee/8Xg5yn2itZwWOs0b MM2K35WGayANmXha1Z1vIME/dVrDqVqaF81BflwSn+dVaBejDxkNnVaECEDo4Y6qvCWsBs1OrYL OSJsoHrfhXZPAVAeHEiCu4/SYLssCMlwH/7uyX4yQ17OTMQx X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB All GCS operations at EL0 must happen on a page which is marked as having UnprivGCS access, including read operations. If a GCS operation attempts to access a page without this then it will generate a data abort with the GCS bit set in ESR_EL1.ISS2. EL0 may validly generate such faults, for example due to copy on write which will cause the GCS data to be stored in a read only page with no GCS permissions until the actual copy happens. Since UnprivGCS allows both reads and writes to the GCS (though only through GCS operations) we need to ensure that the memory management subsystem handles GCS accesses as writes at all times. Do this by adding FAULT_FLAG_WRITE to any GCS page faults, adding handling to ensure that invalid cases are identfied as such early so the memory management core does not think they will succeed. The core cannot distinguish between VMAs which are generally writeable and VMAs which are only writeable through GCS operations. EL1 may validly write to EL0 GCS for management purposes (eg, while initialising with cap tokens). We also report any GCS faults in VMAs not marked as part of a GCS as access violations, causing a fault to be delivered to userspace if it attempts to do GCS operations outside a GCS. Signed-off-by: Mark Brown --- arch/arm64/mm/fault.c | 79 +++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 71 insertions(+), 8 deletions(-) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 460d799e1296..28de0807b4a1 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -494,13 +494,30 @@ static void do_bad_area(unsigned long far, unsigned long esr, } } +/* + * Note: not valid for EL1 DC IVAC, but we never use that such that it + * should fault. EL0 cannot issue DC IVAC (undef). + */ +static bool is_write_abort(unsigned long esr) +{ + return (esr & ESR_ELx_WNR) && !(esr & ESR_ELx_CM); +} + +static bool is_gcs_fault(unsigned long esr) +{ + if (!esr_is_data_abort(esr)) + return false; + + return ESR_ELx_ISS2(esr) & ESR_ELx_GCS; +} + #define VM_FAULT_BADMAP ((__force vm_fault_t)0x010000) #define VM_FAULT_BADACCESS ((__force vm_fault_t)0x020000) static vm_fault_t __do_page_fault(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned int mm_flags, unsigned long vm_flags, - struct pt_regs *regs) + unsigned long esr, struct pt_regs *regs) { /* * Ok, we have a good vm_area for this memory access, so we can handle @@ -510,6 +527,26 @@ static vm_fault_t __do_page_fault(struct mm_struct *mm, */ if (!(vma->vm_flags & vm_flags)) return VM_FAULT_BADACCESS; + + if (vma->vm_flags & VM_SHADOW_STACK) { + /* + * Writes to a GCS must either be generated by a GCS + * operation or be from EL1. + */ + if (is_write_abort(esr) && + !(is_gcs_fault(esr) || is_el1_data_abort(esr))) + return VM_FAULT_BADACCESS; + } else { + /* + * GCS faults should never happen for pages that are + * not part of a GCS and the operation being attempted + * can never succeed. + */ + if (is_gcs_fault(esr)) + return VM_FAULT_BADACCESS; + } + + return handle_mm_fault(vma, addr, mm_flags, regs); } @@ -518,13 +555,18 @@ static bool is_el0_instruction_abort(unsigned long esr) return ESR_ELx_EC(esr) == ESR_ELx_EC_IABT_LOW; } -/* - * Note: not valid for EL1 DC IVAC, but we never use that such that it - * should fault. EL0 cannot issue DC IVAC (undef). - */ -static bool is_write_abort(unsigned long esr) +static bool is_invalid_el0_gcs_access(struct vm_area_struct *vma, u64 esr) { - return (esr & ESR_ELx_WNR) && !(esr & ESR_ELx_CM); + if (!system_supports_gcs()) + return false; + if (likely(!(vma->vm_flags & VM_SHADOW_STACK))) { + if (is_gcs_fault(esr)) + return true; + return false; + } + if (is_gcs_fault(esr)) + return false; + return is_write_abort(esr); } static int __kprobes do_page_fault(unsigned long far, unsigned long esr, @@ -573,6 +615,13 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, /* If EPAN is absent then exec implies read */ if (!alternative_has_cap_unlikely(ARM64_HAS_EPAN)) vm_flags |= VM_EXEC; + /* + * Upgrade read faults to write faults, GCS reads must + * occur on a page marked as GCS so we need to trigger + * copy on write always. + */ + if (is_gcs_fault(esr)) + mm_flags |= FAULT_FLAG_WRITE; } if (is_ttbr0_addr(addr) && is_el1_permission_fault(addr, esr, regs)) { @@ -594,6 +643,20 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, if (!vma) goto lock_mmap; + /* + * We get legitimate write faults for GCS pages from GCS + * operations, even when the initial operation was a read, as + * a result of upgrading GCS accesses to writes for CoW but + * GCS acceses outside of a GCS must fail. Specifically check + * for this since the mm core isn't able to distinguish + * invalid GCS access from valid ones and will try to resolve + * the fault. + */ + if (is_invalid_el0_gcs_access(vma, esr)) { + vma_end_read(vma); + goto lock_mmap; + } + if (!(vma->vm_flags & vm_flags)) { vma_end_read(vma); goto lock_mmap; @@ -623,7 +686,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, goto done; } - fault = __do_page_fault(mm, vma, addr, mm_flags, vm_flags, regs); + fault = __do_page_fault(mm, vma, addr, mm_flags, vm_flags, esr, regs); /* Quick path to respond to signals */ if (fault_signal_pending(fault, regs)) { From patchwork Wed Nov 22 09:42:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746289 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E25B45579B; Wed, 22 Nov 2023 09:45:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="J/t21EOt" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8BBB3C433C7; Wed, 22 Nov 2023 09:45:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646329; bh=p7vhwNE+CJYbXpRAkTVE4uNTVeLjV9vl6fV2mHCV5l8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=J/t21EOtX9LkHO6S2pMhuc938Wc3YdgkYmn0ooGJB+tFn7kn2xaS1fJa9v/tWKi43 ZTDDKJiFXArP+8mEXvmx2sBLcyO6KjFUwilZJ9QSpTmZBHex9DHtV7uGhhjFfTOdHA BFtt+5Y6vqCpuXIYKBUJbZqMwDI8e9aBheS5w/j6wc1DsyC3Z2ID5HtMpFHCoHTcJo rWLQurPTEuIvboKnpuVymwk7QeZNeC8YHWb609HHt82+32TM1oD3/3qoKdGeGwAAaa xy2l0tOR5Nb7hVNhSMufPGx6/Ar5BoHrwMGH9nsGTb2eLlGyS1xBLrlgKhaomAQEKx tYJH4Vzj89TFg== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:31 +0000 Subject: [PATCH v7 21/39] arm64/gcs: Allocate a new GCS for threads with GCS enabled Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-21-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=5979; i=broonie@kernel.org; h=from:subject:message-id; bh=p7vhwNE+CJYbXpRAkTVE4uNTVeLjV9vl6fV2mHCV5l8=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXc0GtL0+c11OyZaFaNeVlqO8pwcT9Q3SQtGvl DiVL+0NZkSJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3NBgAKCRAk1otyXVSH 0I5ZB/9ZoJk5A/i0hsPYxTWiFmHCmO/wYkYfWs9TeOH7TaQylRIdkTSuSPmzgHhApbE5QeBZyaU uS997l8TNA33lJJ2js6ybFa7dhO+PFXo/WWqrdncps5Mt7P8IC0Zd/Z4jyBPVQumNeLPc2oJD5D uW83f704kQiGeBYcxmPTNph0Iy2NoFCK2tjIHAQ/zeTosrqitio0MFnrAvbvWNl5t/K3F67Tp6B Bf7jfpqYvIL/b83Z9wbP3/iID7f1dllpQQH5JL/Ig4/pVRu10RPXKm8ia5gAOIBI4b2tXMldcdb 2y5Sqo6qdVr9SvMj0mxKxslcvS7VqlDB4ZvxXeja7O8Ko7EQ X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB When a new thread is created by a thread with GCS enabled the GCS needs to be specified along with the regular stack. clone3() has been extended to support this case, allowing userspace to explicitly request the size for the GCS to be created, but plain clone() is not extensible and existing clone3() users will not specify a size. For compatibility with these cases and also x86 (which did not initially implement clone3() support for shadow stacks) if no GCS is specified we will allocate one thread so when a thread is created which has GCS enabled allocate one for it. We follow the extensively discussed x86 implementation and allocate min(RLIMIT_STACK, 4G). Since the GCS only stores the call stack and not any variables this should be more than sufficient for most applications. GCSs allocated via this mechanism then it will be freed when the thread exits. Signed-off-by: Mark Brown --- arch/arm64/include/asm/gcs.h | 9 ++++++ arch/arm64/kernel/process.c | 25 +++++++++++++++ arch/arm64/mm/gcs.c | 76 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 110 insertions(+) diff --git a/arch/arm64/include/asm/gcs.h b/arch/arm64/include/asm/gcs.h index 04594ef59dad..c1f274fdb9c0 100644 --- a/arch/arm64/include/asm/gcs.h +++ b/arch/arm64/include/asm/gcs.h @@ -8,6 +8,8 @@ #include #include +struct kernel_clone_args; + static inline void gcsb_dsync(void) { asm volatile(".inst 0xd503227f" : : : "memory"); @@ -58,6 +60,8 @@ static inline bool task_gcs_el0_enabled(struct task_struct *task) void gcs_set_el0_mode(struct task_struct *task); void gcs_free(struct task_struct *task); void gcs_preserve_current_state(void); +unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args); #else @@ -69,6 +73,11 @@ static inline bool task_gcs_el0_enabled(struct task_struct *task) static inline void gcs_set_el0_mode(struct task_struct *task) { } static inline void gcs_free(struct task_struct *task) { } static inline void gcs_preserve_current_state(void) { } +static inline unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args) +{ + return -ENOTSUPP; +} #endif diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index fd80b43c2969..588d2bc4b329 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -285,9 +285,29 @@ static void flush_gcs(void) write_sysreg_s(0, SYS_GCSPR_EL0); } +static int copy_thread_gcs(struct task_struct *p, + const struct kernel_clone_args *args) +{ + unsigned long gcs; + + gcs = gcs_alloc_thread_stack(p, args); + if (IS_ERR_VALUE(gcs)) + return PTR_ERR((void *)gcs); + + /* Ensure the current state of the GCS is seen by CoW */ + gcsb_dsync(); + + return 0; +} + #else static void flush_gcs(void) { } +static int copy_thread_gcs(struct task_struct *p, + const struct kernel_clone_args *args) +{ + return 0; +} #endif @@ -369,6 +389,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) unsigned long stack_start = args->stack; unsigned long tls = args->tls; struct pt_regs *childregs = task_pt_regs(p); + int ret; memset(&p->thread.cpu_context, 0, sizeof(struct cpu_context)); @@ -410,6 +431,10 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) p->thread.uw.tp_value = tls; p->thread.tpidr2_el0 = 0; } + + ret = copy_thread_gcs(p, args); + if (ret != 0) + return ret; } else { /* * A kthread has no context to ERET to, so ensure any buggy diff --git a/arch/arm64/mm/gcs.c b/arch/arm64/mm/gcs.c index b0a67efc522b..0d4881173b5f 100644 --- a/arch/arm64/mm/gcs.c +++ b/arch/arm64/mm/gcs.c @@ -8,6 +8,82 @@ #include #include +static unsigned long alloc_gcs(unsigned long addr, unsigned long size, + unsigned long token_offset, bool set_res_tok) +{ + int flags = MAP_ANONYMOUS | MAP_PRIVATE; + struct mm_struct *mm = current->mm; + unsigned long mapped_addr, unused; + + if (addr) + flags |= MAP_FIXED_NOREPLACE; + + mmap_write_lock(mm); + mapped_addr = do_mmap(NULL, addr, size, PROT_READ | PROT_WRITE, flags, + VM_SHADOW_STACK, 0, &unused, NULL); + mmap_write_unlock(mm); + + return mapped_addr; +} + +static unsigned long gcs_size(unsigned long size) +{ + if (size) + return PAGE_ALIGN(size); + + /* Allocate RLIMIT_STACK/2 with limits of PAGE_SIZE..2G */ + size = PAGE_ALIGN(min_t(unsigned long long, + rlimit(RLIMIT_STACK) / 2, SZ_2G)); + return max(PAGE_SIZE, size); +} + +unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args) +{ + unsigned long addr, size; + + /* If the user specified a GCS use it. */ + if (args->shadow_stack_size) { + if (!system_supports_gcs()) + return (unsigned long)ERR_PTR(-EINVAL); + + size = args->shadow_stack_size; + + if (size < 8) + return (unsigned long)ERR_PTR(-EINVAL); + + } else { + /* + * Otherwise fall back to legacy clone() support and + * implicitly allocate a GCS if we need a new one. + */ + + if (!system_supports_gcs()) + return 0; + + if (!task_gcs_el0_enabled(tsk)) + return 0; + + if ((args->flags & (CLONE_VFORK | CLONE_VM)) != CLONE_VM) { + tsk->thread.gcspr_el0 = read_sysreg_s(SYS_GCSPR_EL0); + return 0; + } + + size = args->stack_size; + } + + size = gcs_size(size); + addr = alloc_gcs(0, size, 0, 0); + if (IS_ERR_VALUE(addr)) + return addr; + + tsk->thread.gcs_base = addr; + tsk->thread.gcs_size = size; + tsk->thread.gcspr_el0 = addr + size - sizeof(u64); + + return addr; +} + /* * Apply the GCS mode configured for the specified task to the * hardware. From patchwork Wed Nov 22 09:42:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746288 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6E9B555791; Wed, 22 Nov 2023 09:45:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="K/G00O1W" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5FF98C43395; Wed, 22 Nov 2023 09:45:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646345; bh=NWVJpdky41RR5z+76XsSxPAX8JBO81FQgE/78Pibz4I=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=K/G00O1WsBVKAEIul9z7M9wwqUCctNvCWPIaRc94+Gvlyf4ztcyShJLjmynNGk+0B 7hq8h2uawUA22gE15dEZ51ZTQ925UaN/8M9vgcTDnOwWQsuUaff39e7A4k0MYaYQei kbeJ26pPSy+RavFC5jiQzP68OxdewcWNTDUx+97/iR+Wod7dBZrpJhL+zFSllgaOU4 tTAwtGqFpeCW/XVRsAzFRAm4U0v25Q+Vt+EYrKl876crUoM8lp3JjUPcemuqRNmigp osHPuhMKVym0baNytxZlIId5GP3XNo+tBA2/HwMdSZQ13zyKoep+bol7KRP3sZl1Y6 5c5nQ4gyCyVog== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:33 +0000 Subject: [PATCH v7 23/39] arm64/mm: Implement map_shadow_stack() Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-23-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=3103; i=broonie@kernel.org; h=from:subject:message-id; bh=NWVJpdky41RR5z+76XsSxPAX8JBO81FQgE/78Pibz4I=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXc0Ip6PqHo/mfAIYxGDeH30hQhlkTVEtnfk4p U/mjwHwk4uJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3NCAAKCRAk1otyXVSH 0MvzB/9fzWnM1qf2rhce2NWjEOKbFvvKGljLcOVUTHACXD9yQBR9F7owkcySZ9VXjzQbfvkKGXP 9KBhY9vBnEONt/apYyW01/mhfN+uI9680+VgwQRrwPP2yyMoTRFej/ZISyD4X7zqhOohmgBMTls we4TRIcF7x/2ZIRovz3jyjHZW4De+AalqERXlrJXgUEBHNiMQyi97JE+muLMn0kqMhzS0KM0fqo EUqWgO/phoFoU4p6c8gieIGuKOJ8elT3uUiZAeB+kK2rCc7bZbL9koFjnhS4mYcFlRDToQhQpFv GD6HdRG8hNRI9z4hGkH/MB88HcPqkexmwToLVqmqilFiuS8b X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB As discussed extensively in the changelog for the addition of this syscall on x86 ("x86/shstk: Introduce map_shadow_stack syscall") the existing mmap() and madvise() syscalls do not map entirely well onto the security requirements for guarded control stacks since they lead to windows where memory is allocated but not yet protected or stacks which are not properly and safely initialised. Instead a new syscall map_shadow_stack() has been defined which allocates and initialises a shadow stack page. Implement this for arm64. Two flags are provided, allowing applications to request that the stack be initialised with a valid cap token at the top of the stack and optionally also an end of stack marker above that. We support requesting an end of stack marker alone but since this is a NULL pointer it is indistinguishable from not initialising anything by itself. Since the x86 code has not yet been rebased to v6.5-rc1 this includes the architecture neutral parts of Rick Edgecmbe's "x86/shstk: Introduce map_shadow_stack syscall". Signed-off-by: Mark Brown --- arch/arm64/mm/gcs.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/arch/arm64/mm/gcs.c b/arch/arm64/mm/gcs.c index 315bd698de86..02f8f6046c10 100644 --- a/arch/arm64/mm/gcs.c +++ b/arch/arm64/mm/gcs.c @@ -84,6 +84,67 @@ unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, return addr; } +SYSCALL_DEFINE3(map_shadow_stack, unsigned long, addr, unsigned long, size, unsigned int, flags) +{ + unsigned long alloc_size; + unsigned long __user *cap_ptr; + unsigned long cap_val; + int ret = 0; + int cap_offset; + + if (!system_supports_gcs()) + return -EOPNOTSUPP; + + if (flags & ~(SHADOW_STACK_SET_TOKEN | SHADOW_STACK_SET_MARKER)) + return -EINVAL; + + if (addr && (addr % PAGE_SIZE)) + return -EINVAL; + + if (size == 8 || size % 8) + return -EINVAL; + + /* + * An overflow would result in attempting to write the restore token + * to the wrong location. Not catastrophic, but just return the right + * error code and block it. + */ + alloc_size = PAGE_ALIGN(size); + if (alloc_size < size) + return -EOVERFLOW; + + addr = alloc_gcs(addr, alloc_size, 0, false); + if (IS_ERR_VALUE(addr)) + return addr; + + /* + * Put a cap token at the end of the allocated region so it + * can be switched to. + */ + if (flags & SHADOW_STACK_SET_TOKEN) { + /* Leave an extra empty frame as a top of stack marker? */ + if (flags & SHADOW_STACK_SET_MARKER) + cap_offset = 2; + else + cap_offset = 1; + + cap_ptr = (unsigned long __user *)(addr + size - + (cap_offset * sizeof(unsigned long))); + cap_val = GCS_CAP(cap_ptr); + + put_user_gcs(cap_val, cap_ptr, &ret); + if (ret != 0) { + vm_munmap(addr, size); + return -EFAULT; + } + + /* Ensure the new cap is viaible for GCS */ + gcsb_dsync(); + } + + return addr; +} + /* * Apply the GCS mode configured for the specified task to the * hardware. From patchwork Wed Nov 22 09:42:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746287 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 84B721DA53; Wed, 22 Nov 2023 09:46:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fKnWI82w" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0329AC433C9; Wed, 22 Nov 2023 09:45:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646365; bh=OmIISmzme0R4+ZZhAdBzRBNSULPhQIu1MUcvbR0SztM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=fKnWI82whn0XtZdXkPreiufY8bmPTbg/EhFaJf6C45E2ClZ6Xv2Zhoc7zg0gpoeKF brXoGKtIf3FI21RXrD8eJdIK6zfz+RvJtBibK7z+vO32SdMZw0j02YzIsdu8hFM69t k5vymDvbbKl+So6XOiIp2P1QrhzMC8diEO4SkAXIaJT+8b/RFjhLN8r5ZCcg8ozxox s3ru+GLB8ogDtroFfmq7CVBeUd55pS39+ibUDo4S0B5qsBQA+/XTrb4PzFS9Fcq58F qp9aZp7TCaLSSPxRfKqFXEemhscbubzUQLAS4H3J2bluTM/10+9X4yKtBNgwLSoSrv DEQ1osdYiB55g== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:35 +0000 Subject: [PATCH v7 25/39] arm64/signal: Expose GCS state in signal frames Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-25-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=5977; i=broonie@kernel.org; h=from:subject:message-id; bh=OmIISmzme0R4+ZZhAdBzRBNSULPhQIu1MUcvbR0SztM=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXc0JHIYxL2UrB1tx7MSl6aRe7ZhUp/W9tRbtj gsL6qbyLg6JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3NCQAKCRAk1otyXVSH 0ItlB/97EBZweJSTZu8YX5Q8kOJfn4Y40tfmiBjCNZTMEzx/rxeR5DJ2uOALUItLzFzYwbr7xXt wlujHSBFfUEgz0xA9ogYtQECmdVjCE/1BTflw4B4J9zHnKw44v3FiJJJ4dLxTsH7E2+7s250vqt YKDkAz/HyNgSwVgfXy1zwlwovNqebe6+TENHaZPYUSn5eb5Yfm/ZX1+3AZQ35aDuvaAvvw5w0H9 EOdFTsc5+LnecbmGHXGtOqbJCISKXhAQDlvu6GklYlCoex0A0rt9J1XZJz3zxYcZvNiJUG45SgD WDFtGSLC6tHpbS+rC3B+RRoHZwnU+qXse8UzLqUFOyrB50aY X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Add a context for the GCS state and include it in the signal context when running on a system that supports GCS. We reuse the same flags that the prctl() uses to specify which GCS features are enabled and also provide the current GCS pointer. We do not support enabling GCS via signal return, there is a conflict between specifying GCSPR_EL0 and allocation of a new GCS and this is not an ancticipated use case. We also enforce GCS configuration locking on signal return. Signed-off-by: Mark Brown --- arch/arm64/include/uapi/asm/sigcontext.h | 9 +++ arch/arm64/kernel/signal.c | 107 +++++++++++++++++++++++++++++++ 2 files changed, 116 insertions(+) diff --git a/arch/arm64/include/uapi/asm/sigcontext.h b/arch/arm64/include/uapi/asm/sigcontext.h index f23c1dc3f002..7b66d245f2d2 100644 --- a/arch/arm64/include/uapi/asm/sigcontext.h +++ b/arch/arm64/include/uapi/asm/sigcontext.h @@ -168,6 +168,15 @@ struct zt_context { __u16 __reserved[3]; }; +#define GCS_MAGIC 0x47435300 + +struct gcs_context { + struct _aarch64_ctx head; + __u64 gcspr; + __u64 features_enabled; + __u64 reserved; +}; + #endif /* !__ASSEMBLY__ */ #include diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index b5dff91274fb..7ff775495f3e 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -88,6 +88,7 @@ struct rt_sigframe_user_layout { unsigned long fpsimd_offset; unsigned long esr_offset; + unsigned long gcs_offset; unsigned long sve_offset; unsigned long tpidr2_offset; unsigned long za_offset; @@ -214,6 +215,8 @@ struct user_ctxs { u32 za_size; struct zt_context __user *zt; u32 zt_size; + struct gcs_context __user *gcs; + u32 gcs_size; }; static int preserve_fpsimd_context(struct fpsimd_context __user *ctx) @@ -606,6 +609,82 @@ extern int restore_zt_context(struct user_ctxs *user); #endif /* ! CONFIG_ARM64_SME */ +#ifdef CONFIG_ARM64_GCS + +static int preserve_gcs_context(struct gcs_context __user *ctx) +{ + int err = 0; + u64 gcspr; + + /* + * We will add a cap token to the frame, include it in the + * GCSPR_EL0 we report to support stack switching via + * sigreturn. + */ + gcs_preserve_current_state(); + gcspr = current->thread.gcspr_el0; + if (task_gcs_el0_enabled(current)) + gcspr -= 8; + + __put_user_error(GCS_MAGIC, &ctx->head.magic, err); + __put_user_error(sizeof(*ctx), &ctx->head.size, err); + __put_user_error(gcspr, &ctx->gcspr, err); + __put_user_error(current->thread.gcs_el0_mode, + &ctx->features_enabled, err); + + return err; +} + +static int restore_gcs_context(struct user_ctxs *user) +{ + u64 gcspr, enabled; + int err = 0; + + if (user->gcs_size != sizeof(*user->gcs)) + return -EINVAL; + + __get_user_error(gcspr, &user->gcs->gcspr, err); + __get_user_error(enabled, &user->gcs->features_enabled, err); + if (err) + return err; + + /* Don't allow unknown modes */ + if (enabled & ~PR_SHADOW_STACK_SUPPORTED_STATUS_MASK) + return -EINVAL; + + err = gcs_check_locked(current, enabled); + if (err != 0) + return err; + + /* Don't allow enabling */ + if (!task_gcs_el0_enabled(current) && + (enabled & PR_SHADOW_STACK_ENABLE)) + return -EINVAL; + + /* If we are disabling disable everything */ + if (!(enabled & PR_SHADOW_STACK_ENABLE)) + enabled = 0; + + current->thread.gcs_el0_mode = enabled; + + /* + * We let userspace set GCSPR_EL0 to anything here, we will + * validate later in gcs_restore_signal(). + */ + current->thread.gcspr_el0 = gcspr; + write_sysreg_s(current->thread.gcspr_el0, SYS_GCSPR_EL0); + + return 0; +} + +#else /* ! CONFIG_ARM64_GCS */ + +/* Turn any non-optimised out attempts to use these into a link error: */ +extern int preserve_gcs_context(void __user *ctx); +extern int restore_gcs_context(struct user_ctxs *user); + +#endif /* ! CONFIG_ARM64_GCS */ + static int parse_user_sigframe(struct user_ctxs *user, struct rt_sigframe __user *sf) { @@ -622,6 +701,7 @@ static int parse_user_sigframe(struct user_ctxs *user, user->tpidr2 = NULL; user->za = NULL; user->zt = NULL; + user->gcs = NULL; if (!IS_ALIGNED((unsigned long)base, 16)) goto invalid; @@ -716,6 +796,17 @@ static int parse_user_sigframe(struct user_ctxs *user, user->zt_size = size; break; + case GCS_MAGIC: + if (!system_supports_gcs()) + goto invalid; + + if (user->gcs) + goto invalid; + + user->gcs = (struct gcs_context __user *)head; + user->gcs_size = size; + break; + case EXTRA_MAGIC: if (have_extra_context) goto invalid; @@ -835,6 +926,9 @@ static int restore_sigframe(struct pt_regs *regs, err = restore_fpsimd_context(&user); } + if (err == 0 && system_supports_gcs() && user.gcs) + err = restore_gcs_context(&user); + if (err == 0 && system_supports_tpidr2() && user.tpidr2) err = restore_tpidr2_context(&user); @@ -949,6 +1043,13 @@ static int setup_sigframe_layout(struct rt_sigframe_user_layout *user, return err; } + if (system_supports_gcs()) { + err = sigframe_alloc(user, &user->gcs_offset, + sizeof(struct gcs_context)); + if (err) + return err; + } + if (system_supports_sve() || system_supports_sme()) { unsigned int vq = 0; @@ -1042,6 +1143,12 @@ static int setup_sigframe(struct rt_sigframe_user_layout *user, __put_user_error(current->thread.fault_code, &esr_ctx->esr, err); } + if (system_supports_gcs() && err == 0 && user->gcs_offset) { + struct gcs_context __user *gcs_ctx = + apply_user_offset(user, user->gcs_offset); + err |= preserve_gcs_context(gcs_ctx); + } + /* Scalable Vector Extension state (including streaming), if present */ if ((system_supports_sve() || system_supports_sme()) && err == 0 && user->sve_offset) { From patchwork Wed Nov 22 09:42:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746286 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 903BE55791; Wed, 22 Nov 2023 09:46:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZNt/mhOl" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 866E8C43395; Wed, 22 Nov 2023 09:46:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646383; bh=0y6nS2SVdzl2qxV5TcDCnpKi8YQvhWLC/gR5WsuQggw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ZNt/mhOldQGyAGrFdcd7zCoYi9c4Cpbfl611N1nCbvre1FYvH7Zl1qxYMHlLUXK/R oTr4pGgCl3ZqkdfLg9vxV823aJQE0eCLlAhRLaW1gT0pmdwjk38uP6NXqfphyDxggd Nce0uiHdxAcfp2N1Y9laenaA7HV7sOUFODNyqrBMy7XcrOzKnRNQTKmHrYboi3WFf4 QOpE3cQlVvo2KE6NieOOuSEronhIGHfX5fucm1ZO0NVH0NxleNUmMK8+wdtaJLpRNh bN4Wz6vhiC+V3XP1enBFhlP/PZm5aE6IiSJnXGkBC62BqC2W7W+tQNNElkgCPc4NEY Pt1AESbQWlpmQ== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:37 +0000 Subject: [PATCH v7 27/39] arm64: Add Kconfig for Guarded Control Stack (GCS) Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-27-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=1372; i=broonie@kernel.org; h=from:subject:message-id; bh=0y6nS2SVdzl2qxV5TcDCnpKi8YQvhWLC/gR5WsuQggw=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXc0LjTRQMrZZBOJgPLB48tbG4uiml7CZIDEV5 uQD798eCNqJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3NCwAKCRAk1otyXVSH 0Iq3B/9HlE8pfPFk2nuyqu1U5wC2TDnfo30tlZoqa72rjxWoZMeOZIt03d7ifiDGT60BAhV7DR/ X2q4ijl0gxsqAdoQTECIT09I17i0jhkg4sOrFnUM9S88Dc3B+OIQORdu8k6AozeCurJMkcd/ICY snWt5N0Nh2ox8T7U6IwsCVSrOZfctm8ylWDVAVHxsy53/DBhuI0FtT+rlbYGseRcYj1zRLPXpf+ BVttx7Ntz2AelFYWVTuV1+GEKIwFIHMxKR/QSrU8uS9CzItwTUAOZKN30cLz7YqjsJMsYAnSVKN Bd8Eqt+f6s9NrIoUWZmOF8FfRH4sNLQwPl0XXWRCxkdsmJNx X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Provide a Kconfig option allowing the user to select if GCS support is built into the kernel. Signed-off-by: Mark Brown --- arch/arm64/Kconfig | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 7b071a00425d..05cffc0f5786 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2078,6 +2078,26 @@ config ARM64_EPAN if the cpu does not implement the feature. endmenu # "ARMv8.7 architectural features" +menu "v9.4 architectural features" + +config ARM64_GCS + bool "Enable support for Guarded Control Stack (GCS)" + default y + select ARCH_HAS_USER_SHADOW_STACK + select ARCH_USES_HIGH_VMA_FLAGS + help + Guarded Control Stack (GCS) provides support for a separate + stack with restricted access which contains only return + addresses. This can be used to harden against some attacks + by comparing return address used by the program with what is + stored in the GCS, and may also be used to efficiently obtain + the call stack for applications such as profiling. + + The feature is detected at runtime, and will remain disabled + if the system does not implement the feature. + +endmenu # "v9.4 architectural features" + config ARM64_SVE bool "ARM Scalable Vector Extension support" default y From patchwork Wed Nov 22 09:42:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746285 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED0761DA53; Wed, 22 Nov 2023 09:46:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="p3JwdAwl" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 495C6C433C8; Wed, 22 Nov 2023 09:46:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646401; bh=eI3VK56kyhvq7JOnBJXgl6j+EIFmz1Cd350CxZrMcNA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=p3JwdAwl1IwDl1cRu279YeAAcZ/jDDNBFvD6epk9Cj3NAtBjknB4UKuxychR75RtB FhjbV32JXVmGGIo7RvOYvZI21X1Va108tpxqaoXhAlxRto69QxeD4jrYLKfxzRH+xS fS4SsKyIPwP7gr6HZsK/zCT+FGosjhk9bH6qPcKXRN5/Gg8IcmfmTi+fnplKR0d3NB d4XtyugCFtLBNbW2QMhR75JadHd3ITyQBaQ4BS9ZMhBgxqaRG6GWK5XglKVU3qBEyH 51/2dg7wqDRYv0qDxXDHavJlTBhDl1MK0iQvFuQaTnqc0cUJPWw7BCj4c2ii5isa1m 5uI1oYMi+XrMQ== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:39 +0000 Subject: [PATCH v7 29/39] kselftest/arm64: Add GCS as a detected feature in the signal tests Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-29-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=1828; i=broonie@kernel.org; h=from:subject:message-id; bh=eI3VK56kyhvq7JOnBJXgl6j+EIFmz1Cd350CxZrMcNA=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXc0MpoaH+B7so44Op51dKDPnZ+HR/liFSB4lO rxQeP/wL7iJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3NDAAKCRAk1otyXVSH 0OPrB/0bYwxvcTBgjII2I1FWsRdgrDk0RAdSe1MhLGxQngmZhESoEneo2Hq5C+22QiDRyqPOBWG NBhs+DcJn2N8QakBKktW1j2hZa/f2Fq4KvqQXWzEzaozqttPg3+lRVUc/+5XW2a+SMylEt1YziP D2jfrKJj4dayCnvjh7KLRR9I6EXK8+UH3eNDRgBoO3xugNgtV822vuRQkc43QiV5uAvpfJ3I5ZM ddSrXmzHk2QU0QffGhU8k1hfn4I2IqmpsZW99HiDapofjAKTjk9jUeKd4kbu3X+PH/DOibREYO3 VHJNr+Al5/4t9DoAy8UzphVB0ymspKLRJlwLJbX6p/eOOAfu X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB In preparation for testing GCS related signal handling add it as a feature we check for in the signal handling support code. Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/signal/test_signals.h | 2 ++ tools/testing/selftests/arm64/signal/test_signals_utils.c | 3 +++ 2 files changed, 5 insertions(+) diff --git a/tools/testing/selftests/arm64/signal/test_signals.h b/tools/testing/selftests/arm64/signal/test_signals.h index 1e6273d81575..7ada43688c02 100644 --- a/tools/testing/selftests/arm64/signal/test_signals.h +++ b/tools/testing/selftests/arm64/signal/test_signals.h @@ -35,6 +35,7 @@ enum { FSME_BIT, FSME_FA64_BIT, FSME2_BIT, + FGCS_BIT, FMAX_END }; @@ -43,6 +44,7 @@ enum { #define FEAT_SME (1UL << FSME_BIT) #define FEAT_SME_FA64 (1UL << FSME_FA64_BIT) #define FEAT_SME2 (1UL << FSME2_BIT) +#define FEAT_GCS (1UL << FGCS_BIT) /* * A descriptor used to describe and configure a test case. diff --git a/tools/testing/selftests/arm64/signal/test_signals_utils.c b/tools/testing/selftests/arm64/signal/test_signals_utils.c index 0dc948db3a4a..89ef95c1af0e 100644 --- a/tools/testing/selftests/arm64/signal/test_signals_utils.c +++ b/tools/testing/selftests/arm64/signal/test_signals_utils.c @@ -30,6 +30,7 @@ static char const *const feats_names[FMAX_END] = { " SME ", " FA64 ", " SME2 ", + " GCS ", }; #define MAX_FEATS_SZ 128 @@ -329,6 +330,8 @@ int test_init(struct tdescr *td) td->feats_supported |= FEAT_SME_FA64; if (getauxval(AT_HWCAP2) & HWCAP2_SME2) td->feats_supported |= FEAT_SME2; + if (getauxval(AT_HWCAP2) & HWCAP2_GCS) + td->feats_supported |= FEAT_GCS; if (feats_ok(td)) { if (td->feats_required & td->feats_supported) fprintf(stderr, From patchwork Wed Nov 22 09:42:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746284 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 312801DA53; Wed, 22 Nov 2023 09:47:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="syOMsnFW" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6C568C4167D; Wed, 22 Nov 2023 09:46:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646421; bh=qxIqlAUMc2bBTJ1O51B6Tb6fnmxFBkD4x3sOYOI7QNE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=syOMsnFWzOG+xmHUkIsaKs6NTpc6AaYK4nG1MDHbqXgrSkOqjN7up45LrEl5p7Kj8 Ym73t13eE56T+NFw9qfvMT6cKIN/58qjCmJqzZxAJ1kGvbKFLul5nd9dC6SQUz1jrA EXxSfZQgZcLsl/twjdlg0vE6CSiRLXxi/NK/28aJYlwlXlwyWmtSXRBJT/xwFsHoxv cnjdJbcrEHQku+7vy1h9M37JgKh9Q68/onZlKJEqYCHDVnv0PkCs/RRlNmSkgV+RF4 oOhzEu7XmVtkSX4RB82dUstRQ0AKsXe8scyUjoFXpHdqZZGidv6d7sY3xjsf23P0T0 9QQcGPMdtp0aw== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:41 +0000 Subject: [PATCH v7 31/39] kselftest/arm64: Allow signals tests to specify an expected si_code Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-31-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=2627; i=broonie@kernel.org; h=from:subject:message-id; bh=qxIqlAUMc2bBTJ1O51B6Tb6fnmxFBkD4x3sOYOI7QNE=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXc0OiKQHWj96U6Iv+vrEb7owlLDF7A+UeVAJW s3BN30QZ7mJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3NDgAKCRAk1otyXVSH 0Cj/B/4v4VwoMeU53pDf1fNeG//YL1WSnuk6c3A7i2dDESTIut5NK5DCoQTU6va5PSrNx6bHcHQ U209ci3F/Zr+o/E5MK0xtlUtuT0eeijo8MCPj04SF2s9C7Dh86AMZ5nTAvggAIsFDYLgQgOmNM+ XlQhib+aoBQStnNJmu+VdxLsNvBqIxdMk0E5RP7S8lMT9FNHmPXIsAfuYt4IpvN/kglnDyFWP2g AlBAMocbBXbx6N9D+rWLO3KOz7hiQ9AVkz/OUwp5hFzfb6hy96kQW529H3py/8jX/rxRH/DfOlw srt4yM6xjmvQ1TSj6Nvsge/pgI2bILXYvrzKkwYSC4BlkQsu X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Currently we ignore si_code unless the expected signal is a SIGSEGV, in which case we enforce it being SEGV_ACCERR. Allow test cases to specify exactly which si_code should be generated so we can validate this, and test for other segfault codes. Signed-off-by: Mark Brown --- .../testing/selftests/arm64/signal/test_signals.h | 4 +++ .../selftests/arm64/signal/test_signals_utils.c | 29 ++++++++++++++-------- 2 files changed, 23 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/arm64/signal/test_signals.h b/tools/testing/selftests/arm64/signal/test_signals.h index 7ada43688c02..ee75a2c25ce7 100644 --- a/tools/testing/selftests/arm64/signal/test_signals.h +++ b/tools/testing/selftests/arm64/signal/test_signals.h @@ -71,6 +71,10 @@ struct tdescr { * Zero when no signal is expected on success */ int sig_ok; + /* + * expected si_code for sig_ok, or 0 to not check + */ + int sig_ok_code; /* signum expected on unsupported CPU features. */ int sig_unsupp; /* a timeout in second for test completion */ diff --git a/tools/testing/selftests/arm64/signal/test_signals_utils.c b/tools/testing/selftests/arm64/signal/test_signals_utils.c index 89ef95c1af0e..63deca32b0df 100644 --- a/tools/testing/selftests/arm64/signal/test_signals_utils.c +++ b/tools/testing/selftests/arm64/signal/test_signals_utils.c @@ -143,16 +143,25 @@ static bool handle_signal_ok(struct tdescr *td, "current->token ZEROED...test is probably broken!\n"); abort(); } - /* - * Trying to narrow down the SEGV to the ones generated by Kernel itself - * via arm64_notify_segfault(). This is a best-effort check anyway, and - * the si_code check may need to change if this aspect of the kernel - * ABI changes. - */ - if (td->sig_ok == SIGSEGV && si->si_code != SEGV_ACCERR) { - fprintf(stdout, - "si_code != SEGV_ACCERR...test is probably broken!\n"); - abort(); + if (td->sig_ok_code) { + if (si->si_code != td->sig_ok_code) { + fprintf(stdout, "si_code is %d not %d\n", + si->si_code, td->sig_ok_code); + abort(); + } + } else { + /* + * Trying to narrow down the SEGV to the ones + * generated by Kernel itself via + * arm64_notify_segfault(). This is a best-effort + * check anyway, and the si_code check may need to + * change if this aspect of the kernel ABI changes. + */ + if (td->sig_ok == SIGSEGV && si->si_code != SEGV_ACCERR) { + fprintf(stdout, + "si_code != SEGV_ACCERR...test is probably broken!\n"); + abort(); + } } td->pass = 1; /* From patchwork Wed Nov 22 09:42:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746283 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8CF731DA53; Wed, 22 Nov 2023 09:47:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mrQEru6U" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EF758C433C9; Wed, 22 Nov 2023 09:47:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646438; bh=fd8/cfgrFfzpt2lwLCcC40w5n1WmV3tDKMJNBUAz6S0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=mrQEru6UqcQG3kmh3zGDnmQlXjA0eIKg8zenwv1l/XR3Dg7szS339bGIZh9BFm1Yk P1AIfRE2XzOMbX/PuCKYsMmKkpi2972srVgCpJYYBG3wITIgz9Zyw55f9UNoj77FhE FzIlVixUqKmjSUPDH60z5vYW+0+kVgeP+7yW8Ul3Y4XOxJjc0HX38hmrh61CxuN9To ZUirmXZ7+WbL1G8x1dT/R0iZK00g1pWAkMjvt82ZhuLi5OmwZ6AK6JYpTsrEGrCZuv j4FgEzgFyZxCnExeVsUpmq0HzZcim0g8I5vplJmnHGwMsRA/3ohMGUGCyLiM9KN3Si TYzFXlBNsH5cQ== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:43 +0000 Subject: [PATCH v7 33/39] kselftest/arm64: Add very basic GCS test program Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-33-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=14970; i=broonie@kernel.org; h=from:subject:message-id; bh=fd8/cfgrFfzpt2lwLCcC40w5n1WmV3tDKMJNBUAz6S0=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXc0PQh7pjrRYnlrLrpuindC5JLbPPt4Kx5awS WgIPTusXEuJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3NDwAKCRAk1otyXVSH 0JY1B/96J7GYvZ9NL4qhsuoGQz6SomkCN+gDSk3QczolnszCDJVS+6leV5AOmBbCVzF+XJCMa9X XhHVTECyEGkrduwrTmdUcl+MsqxBPFKzuO44CkzP4dNWWUEWJNACeA9hYFM8WWi+8/2M36TX+zI y2bw/NJhTabfRyceyD9rZ7Tw1+7U1ipIJTeqU5P/BZxkWpCkMeWAPiXyQwA5UcBkINsbpQXX8lK 6K0P8pa9VNQlO4nUe6SFHNauimmxzM+v7U/+JdLFYghzg6kAm9fJbBOmZ2s0YsInb1RPrPOi/Uf +eb+bOFAbNltdxaTf3G6NqzvvTuI5vUBE0TrbZZgECFs3ICa X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB This test program just covers the basic GCS ABI, covering aspects of the ABI as standalone features without attempting to integrate things. Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/Makefile | 2 +- tools/testing/selftests/arm64/gcs/.gitignore | 1 + tools/testing/selftests/arm64/gcs/Makefile | 18 ++ tools/testing/selftests/arm64/gcs/basic-gcs.c | 428 ++++++++++++++++++++++++++ tools/testing/selftests/arm64/gcs/gcs-util.h | 90 ++++++ 5 files changed, 538 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/arm64/Makefile b/tools/testing/selftests/arm64/Makefile index 28b93cab8c0d..22029e60eff3 100644 --- a/tools/testing/selftests/arm64/Makefile +++ b/tools/testing/selftests/arm64/Makefile @@ -4,7 +4,7 @@ ARCH ?= $(shell uname -m 2>/dev/null || echo not) ifneq (,$(filter $(ARCH),aarch64 arm64)) -ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi +ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi gcs else ARM64_SUBTARGETS := endif diff --git a/tools/testing/selftests/arm64/gcs/.gitignore b/tools/testing/selftests/arm64/gcs/.gitignore new file mode 100644 index 000000000000..0e5e695ecba5 --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/.gitignore @@ -0,0 +1 @@ +basic-gcs diff --git a/tools/testing/selftests/arm64/gcs/Makefile b/tools/testing/selftests/arm64/gcs/Makefile new file mode 100644 index 000000000000..61a30f483429 --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/Makefile @@ -0,0 +1,18 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2023 ARM Limited +# +# In order to avoid interaction with the toolchain and dynamic linker the +# portions of these tests that interact with the GCS are implemented using +# nolibc. +# + +TEST_GEN_PROGS := basic-gcs + +include ../../lib.mk + +$(OUTPUT)/basic-gcs: basic-gcs.c + $(CC) -g -fno-asynchronous-unwind-tables -fno-ident -s -Os -nostdlib \ + -static -include ../../../../include/nolibc/nolibc.h \ + -I../../../../../usr/include \ + -std=gnu99 -I../.. -g \ + -ffreestanding -Wall $^ -o $@ -lgcc diff --git a/tools/testing/selftests/arm64/gcs/basic-gcs.c b/tools/testing/selftests/arm64/gcs/basic-gcs.c new file mode 100644 index 000000000000..3ef957e0065f --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/basic-gcs.c @@ -0,0 +1,428 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2023 ARM Limited. + */ + +#include +#include + +#include + +#include +#include +#include + +#include "kselftest.h" +#include "gcs-util.h" + +/* nolibc doesn't have sysconf(), just hard code the maximum */ +static size_t page_size = 65536; + +static __attribute__((noinline)) void valid_gcs_function(void) +{ + /* Do something the compiler can't optimise out */ + my_syscall1(__NR_prctl, PR_SVE_GET_VL); +} + +static inline int gcs_set_status(unsigned long mode) +{ + bool enabling = mode & PR_SHADOW_STACK_ENABLE; + int ret; + unsigned long new_mode; + + /* + * The prctl takes 1 argument but we need to ensure that the + * other 3 values passed in registers to the syscall are zero + * since the kernel validates them. + */ + ret = my_syscall5(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, mode, + 0, 0, 0); + + if (ret == 0) { + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, + &new_mode, 0, 0, 0); + if (ret == 0) { + if (new_mode != mode) { + ksft_print_msg("Mode set to %x not %x\n", + new_mode, mode); + ret = -EINVAL; + } + } else { + ksft_print_msg("Failed to validate mode: %d\n", ret); + } + + if (enabling != chkfeat_gcs()) { + ksft_print_msg("%senabled by prctl but %senabled in CHKFEAT\n", + enabling ? "" : "not ", + chkfeat_gcs() ? "" : "not "); + ret = -EINVAL; + } + } + + return ret; +} + +/* Try to read the status */ +static bool read_status(void) +{ + unsigned long state; + int ret; + + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, + &state, 0, 0, 0); + if (ret != 0) { + ksft_print_msg("Failed to read state: %d\n", ret); + return false; + } + + return state & PR_SHADOW_STACK_ENABLE; +} + +/* Just a straight enable */ +static bool base_enable(void) +{ + int ret; + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE); + if (ret) { + ksft_print_msg("PR_SHADOW_STACK_ENABLE failed %d\n", ret); + return false; + } + + return true; +} + +/* Check we can read GCSPR_EL0 when GCS is enabled */ +static bool read_gcspr_el0(void) +{ + unsigned long *gcspr_el0; + + ksft_print_msg("GET GCSPR\n"); + gcspr_el0 = get_gcspr(); + ksft_print_msg("GCSPR_EL0 is %p\n", gcspr_el0); + + return true; +} + +/* Also allow writes to stack */ +static bool enable_writeable(void) +{ + int ret; + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_WRITE); + if (ret) { + ksft_print_msg("PR_SHADOW_STACK_ENABLE writeable failed: %d\n", ret); + return false; + } + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE); + if (ret) { + ksft_print_msg("failed to restore plain enable %d\n", ret); + return false; + } + + return true; +} + +/* Also allow writes to stack */ +static bool enable_push_pop(void) +{ + int ret; + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_PUSH); + if (ret) { + ksft_print_msg("PR_SHADOW_STACK_ENABLE with push failed: %d\n", + ret); + return false; + } + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE); + if (ret) { + ksft_print_msg("failed to restore plain enable %d\n", ret); + return false; + } + + return true; +} + +/* Enable GCS and allow everything */ +static bool enable_all(void) +{ + int ret; + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_PUSH | + PR_SHADOW_STACK_WRITE); + if (ret) { + ksft_print_msg("PR_SHADOW_STACK_ENABLE with everything failed: %d\n", + ret); + return false; + } + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE); + if (ret) { + ksft_print_msg("failed to restore plain enable %d\n", ret); + return false; + } + + return true; +} + +static bool enable_invalid(void) +{ + int ret = gcs_set_status(ULONG_MAX); + if (ret == 0) { + ksft_print_msg("GCS_SET_STATUS %lx succeeded\n", ULONG_MAX); + return false; + } + + return true; +} + +/* Map a GCS */ +static bool map_guarded_stack(void) +{ + int ret; + uint64_t *buf; + uint64_t expected_cap; + int elem; + bool pass = true; + + buf = (void *)my_syscall3(__NR_map_shadow_stack, 0, page_size, + SHADOW_STACK_SET_MARKER | + SHADOW_STACK_SET_TOKEN); + if (buf == MAP_FAILED) { + ksft_print_msg("Failed to map %d byte GCS: %d\n", + page_size, errno); + return false; + } + ksft_print_msg("Mapped GCS at %p-%p\n", buf, + (uint64_t)buf + page_size); + + /* The top of the newly allocated region should be 0 */ + elem = (page_size / sizeof(uint64_t)) - 1; + if (buf[elem]) { + ksft_print_msg("Last entry is 0x%lx not 0x0\n", buf[elem]); + pass = false; + } + + /* Then a valid cap token */ + elem--; + expected_cap = ((uint64_t)buf + page_size - 16); + expected_cap &= GCS_CAP_ADDR_MASK; + expected_cap |= GCS_CAP_VALID_TOKEN; + if (buf[elem] != expected_cap) { + ksft_print_msg("Cap entry is 0x%lx not 0x%lx\n", + buf[elem], expected_cap); + pass = false; + } + ksft_print_msg("cap token is 0x%lx\n", buf[elem]); + + /* The rest should be zeros */ + for (elem = 0; elem < page_size / sizeof(uint64_t) - 2; elem++) { + if (!buf[elem]) + continue; + ksft_print_msg("GCS slot %d is 0x%lx not 0x0\n", + elem, buf[elem]); + pass = false; + } + + ret = munmap(buf, page_size); + if (ret != 0) { + ksft_print_msg("Failed to unmap %d byte GCS: %d\n", + page_size, errno); + pass = false; + } + + return pass; +} + +/* A fork()ed process can run */ +static bool test_fork(void) +{ + unsigned long child_mode; + int ret, status; + pid_t pid; + bool pass = true; + + pid = fork(); + if (pid == -1) { + ksft_print_msg("fork() failed: %d\n", errno); + pass = false; + goto out; + } + if (pid == 0) { + /* In child, make sure we can call a function, read + * the GCS pointer and status and then exit */ + valid_gcs_function(); + get_gcspr(); + + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, + &child_mode, 0, 0, 0); + if (ret == 0 && !(child_mode & PR_SHADOW_STACK_ENABLE)) { + ksft_print_msg("GCS not enabled in child\n"); + ret = -EINVAL; + } + + exit(ret); + } + + /* + * In parent, check we can still do function calls then block + * for the child. + */ + valid_gcs_function(); + + ksft_print_msg("Waiting for child %d\n", pid); + + ret = waitpid(pid, &status, 0); + if (ret == -1) { + ksft_print_msg("Failed to wait for child: %d\n", + errno); + return false; + } + + if (!WIFEXITED(status)) { + ksft_print_msg("Child exited due to signal %d\n", + WTERMSIG(status)); + pass = false; + } else { + if (WEXITSTATUS(status)) { + ksft_print_msg("Child exited with status %d\n", + WEXITSTATUS(status)); + pass = false; + } + } + +out: + + return pass; +} + +/* Check that we can explicitly specify a GCS via clone3() */ +static bool test_clone3(void) +{ + struct clone_args args; + unsigned long child_mode; + pid_t pid = -1; + int status, ret; + bool pass; + + memset(&args, 0, sizeof(args)); + args.flags = CLONE_VM; + args.shadow_stack_size = page_size; + + pid = my_syscall2(__NR_clone3, &args, sizeof(args)); + if (pid < 0) { + ksft_print_msg("clone3() failed: %d\n", errno); + pass = false; + goto out; + } + + /* In child? */ + if (pid == 0) { + /* Do we have GCS enabled? */ + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, + &child_mode, 0, 0, 0); + if (ret != 0) { + ksft_print_msg("PR_GET_SHADOW_STACK_STATUS failed: %d\n", + ret); + exit(EXIT_FAILURE); + } + + if (!(child_mode & PR_SHADOW_STACK_ENABLE)) { + ksft_print_msg("GCS not enabled in child\n"); + exit(EXIT_FAILURE); + } + + ksft_print_msg("GCS enabled in child\n"); + + /* We've probably already called a function but make sure */ + valid_gcs_function(); + + exit(EXIT_SUCCESS); + } + + if (waitpid(-1, &status, __WALL) < 0) { + ksft_print_msg("waitpid() failed %d\n", errno); + pass = false; + goto out; + } + if (WIFEXITED(status)) { + if (WEXITSTATUS(status) == EXIT_SUCCESS) { + pass = true; + } else { + ksft_print_msg("Child returned status %d\n", + WEXITSTATUS(status)); + pass = false; + } + } else if (WIFSIGNALED(status)) { + ksft_print_msg("Child exited due to signal %d\n", + WTERMSIG(status)); + pass = false; + } else { + ksft_print_msg("Child exited uncleanly\n"); + pass = false; + } + +out: + return pass; +} + +typedef bool (*gcs_test)(void); + +static struct { + char *name; + gcs_test test; + bool needs_enable; +} tests[] = { + { "read_status", read_status }, + { "base_enable", base_enable, true }, + { "read_gcspr_el0", read_gcspr_el0 }, + { "enable_writeable", enable_writeable, true }, + { "enable_push_pop", enable_push_pop, true }, + { "enable_all", enable_all, true }, + { "enable_invalid", enable_invalid, true }, + { "map_guarded_stack", map_guarded_stack }, + { "fork", test_fork }, + { "clone3", test_clone3 }, +}; + +int main(void) +{ + int i, ret; + unsigned long gcs_mode; + + ksft_print_header(); + + /* + * We don't have getauxval() with nolibc so treat a failure to + * read GCS state as a lack of support and skip. + */ + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, + &gcs_mode, 0, 0, 0); + if (ret != 0) + ksft_exit_skip("Failed to read GCS state: %d\n", ret); + + if (!(gcs_mode & PR_SHADOW_STACK_ENABLE)) { + gcs_mode = PR_SHADOW_STACK_ENABLE; + ret = my_syscall5(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + gcs_mode, 0, 0, 0); + if (ret != 0) + ksft_exit_fail_msg("Failed to enable GCS: %d\n", ret); + } + + ksft_set_plan(ARRAY_SIZE(tests)); + + for (i = 0; i < ARRAY_SIZE(tests); i++) { + ksft_test_result((*tests[i].test)(), "%s\n", tests[i].name); + } + + /* One last test: disable GCS, we can do this one time */ + my_syscall5(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, 0, 0, 0, 0); + if (ret != 0) + ksft_print_msg("Failed to disable GCS: %d\n", ret); + + ksft_finished(); + + return 0; +} diff --git a/tools/testing/selftests/arm64/gcs/gcs-util.h b/tools/testing/selftests/arm64/gcs/gcs-util.h new file mode 100644 index 000000000000..b37801c95604 --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/gcs-util.h @@ -0,0 +1,90 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2023 ARM Limited. + */ + +#ifndef GCS_UTIL_H +#define GCS_UTIL_H + +#include + +#ifndef __NR_map_shadow_stack +#define __NR_map_shadow_stack 453 +#endif + +#ifndef __NR_prctl +#define __NR_prctl 167 +#endif + +/* Shadow Stack/Guarded Control Stack interface */ +#define PR_GET_SHADOW_STACK_STATUS 71 +#define PR_SET_SHADOW_STACK_STATUS 72 +#define PR_LOCK_SHADOW_STACK_STATUS 73 + +# define PR_SHADOW_STACK_ENABLE (1UL << 0) +# define PR_SHADOW_STACK_WRITE (1UL << 1) +# define PR_SHADOW_STACK_PUSH (1UL << 2) + +#define PR_SHADOW_STACK_ALL_MODES \ + PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_WRITE | PR_SHADOW_STACK_PUSH + +#define SHADOW_STACK_SET_TOKEN (1ULL << 0) /* Set up a restore token in the shadow stack */ +#define SHADOW_STACK_SET_MARKER (1ULL << 1) /* Set up a top of stack merker in the shadow stack */ + +#define GCS_CAP_ADDR_MASK (0xfffffffffffff000UL) +#define GCS_CAP_TOKEN_MASK (0x0000000000000fffUL) +#define GCS_CAP_VALID_TOKEN 1 +#define GCS_CAP_IN_PROGRESS_TOKEN 5 + +#define GCS_CAP(x) (((unsigned long)(x) & GCS_CAP_ADDR_MASK) | \ + GCS_CAP_VALID_TOKEN) + +static inline unsigned long *get_gcspr(void) +{ + unsigned long *gcspr; + + asm volatile( + "mrs %0, S3_3_C2_C5_1" + : "=r" (gcspr) + : + : "cc"); + + return gcspr; +} + +static inline void __attribute__((always_inline)) gcsss1(unsigned long *Xt) +{ + asm volatile ( + "sys #3, C7, C7, #2, %0\n" + : + : "rZ" (Xt) + : "memory"); +} + +static inline unsigned long __attribute__((always_inline)) *gcsss2(void) +{ + unsigned long *Xt; + + asm volatile( + "SYSL %0, #3, C7, C7, #3\n" + : "=r" (Xt) + : + : "memory"); + + return Xt; +} + +static inline bool chkfeat_gcs(void) +{ + register long val __asm__ ("x16") = 1; + + /* CHKFEAT x16 */ + asm volatile( + "hint #0x28\n" + : "=r" (val) + : "r" (val)); + + return val != 1; +} + +#endif From patchwork Wed Nov 22 09:42:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746282 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF3EA56458; Wed, 22 Nov 2023 09:47:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LB7SgNo6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D4F21C433CA; Wed, 22 Nov 2023 09:47:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646455; bh=IY5iS/oB2kmvzWci1tpDsay+5IRKl4+rZruMuksVAkE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=LB7SgNo6RWsd/lBKNFyJagX+3r7PWPvlIJ5G3FMo2sllE01HAa9PKVh2iTUQC1Rb0 yJO3uvkYJBQLOOKiPyugzUzRJOTqPRpAXQEHDVNvJz+dmwM9loA2xCVU0eBxB3Adhk 4eSvxd0/5uB+sqB75puTFV1XqKmTd3U0Gw31hE0HN6Hm27frLA/QcTSwmZEi+tgcDJ CrMhDqL4E66FQF+J95ojydeB9TeDY71OMglxa5YAcKrxIFSz3hAI+zVA8FGHv8xGjH Y5hgS/ewIyrkErkAXzLwEgeiwm9G3lgz4rDYZhyOrYs6lKfIfkWGMCC+hbucH9LeSw 3D7zj3rB2jY2Q== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:45 +0000 Subject: [PATCH v7 35/39] kselftest/arm64: Add test coverage for GCS mode locking Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-35-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=7316; i=broonie@kernel.org; h=from:subject:message-id; bh=IY5iS/oB2kmvzWci1tpDsay+5IRKl4+rZruMuksVAkE=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXc0R4LQ2MwbVplKl/x3FSskzznneCR0XUHsAY ynLlIjNK3GJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3NEQAKCRAk1otyXVSH 0E0WB/9tkEX9bs6tnZMZkzAOlUq8oTeqBNNURyadraQK2/RhxR4Sy+dX4QUAjn/b45ZksrLYaWY bcOelN06l97omQdTH6MJ4L/6kUTEafS/Ev4AUAQNjnyWSNkqgunuuq9dwz004R3UXCTWQYNkYQX HG2Ivmx378wXBBzq9LcKWfObX4bk9rAmTOp+RwzKVQwavq9+YIZ7u7TrjibaNUyb4vYYWoDDpmT smMy1dPOM/qExaIIV8j/HbHFNDKkfhLqt8Wsw0PR8c/smnGUz3BHs8VKklqSDgxE0CGIkKN6mKP pyaZ1dWc8QfLKzLZZMEUhb5Xt4ZvlwpIIHbYlm1gvInEP27j X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Verify that we can lock individual GCS mode bits, that other modes aren't affected and as a side effect also that every combination of modes can be enabled. Normally the inability to reenable GCS after disabling it would be an issue with testing but fortunately the kselftest_harness runs each test within a fork()ed child. This can be inconvenient for some kinds of testing but here it means that each test is in a separate thread and therefore won't be affected by other tests in the suite. Once we get toolchains with support for enabling GCS by default we will need to take care to not do that in the build system but there are no such toolchains yet so it is not yet an issue. Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/gcs/.gitignore | 1 + tools/testing/selftests/arm64/gcs/Makefile | 2 +- tools/testing/selftests/arm64/gcs/gcs-locking.c | 200 ++++++++++++++++++++++++ 3 files changed, 202 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/arm64/gcs/.gitignore b/tools/testing/selftests/arm64/gcs/.gitignore index 5810c4a163d4..0c86f53f68ad 100644 --- a/tools/testing/selftests/arm64/gcs/.gitignore +++ b/tools/testing/selftests/arm64/gcs/.gitignore @@ -1,2 +1,3 @@ basic-gcs libc-gcs +gcs-locking diff --git a/tools/testing/selftests/arm64/gcs/Makefile b/tools/testing/selftests/arm64/gcs/Makefile index a8fdf21e9a47..2173d6275956 100644 --- a/tools/testing/selftests/arm64/gcs/Makefile +++ b/tools/testing/selftests/arm64/gcs/Makefile @@ -6,7 +6,7 @@ # nolibc. # -TEST_GEN_PROGS := basic-gcs libc-gcs +TEST_GEN_PROGS := basic-gcs libc-gcs gcs-locking LDLIBS+=-lpthread diff --git a/tools/testing/selftests/arm64/gcs/gcs-locking.c b/tools/testing/selftests/arm64/gcs/gcs-locking.c new file mode 100644 index 000000000000..f6a73254317e --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/gcs-locking.c @@ -0,0 +1,200 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2023 ARM Limited. + * + * Tests for GCS mode locking. These tests rely on both having GCS + * unconfigured on entry and on the kselftest harness running each + * test in a fork()ed process which will have it's own mode. + */ + +#include + +#include +#include + +#include + +#include "kselftest_harness.h" + +#include "gcs-util.h" + +#define my_syscall2(num, arg1, arg2) \ +({ \ + register long _num __asm__ ("x8") = (num); \ + register long _arg1 __asm__ ("x0") = (long)(arg1); \ + register long _arg2 __asm__ ("x1") = (long)(arg2); \ + register long _arg3 __asm__ ("x2") = 0; \ + register long _arg4 __asm__ ("x3") = 0; \ + register long _arg5 __asm__ ("x4") = 0; \ + \ + __asm__ volatile ( \ + "svc #0\n" \ + : "=r"(_arg1) \ + : "r"(_arg1), "r"(_arg2), \ + "r"(_arg3), "r"(_arg4), \ + "r"(_arg5), "r"(_num) \ + : "memory", "cc" \ + ); \ + _arg1; \ +}) + +/* No mode bits are rejected for locking */ +TEST(lock_all_modes) +{ + int ret; + + ret = prctl(PR_LOCK_SHADOW_STACK_STATUS, ULONG_MAX, 0, 0, 0); + ASSERT_EQ(ret, 0); +} + +FIXTURE(valid_modes) +{ +}; + +FIXTURE_VARIANT(valid_modes) +{ + unsigned long mode; +}; + +FIXTURE_VARIANT_ADD(valid_modes, enable) +{ + .mode = PR_SHADOW_STACK_ENABLE, +}; + +FIXTURE_VARIANT_ADD(valid_modes, enable_write) +{ + .mode = PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_WRITE, +}; + +FIXTURE_VARIANT_ADD(valid_modes, enable_push) +{ + .mode = PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_PUSH, +}; + +FIXTURE_VARIANT_ADD(valid_modes, enable_write_push) +{ + .mode = PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_WRITE | + PR_SHADOW_STACK_PUSH, +}; + +FIXTURE_SETUP(valid_modes) +{ +} + +FIXTURE_TEARDOWN(valid_modes) +{ +} + +/* We can set the mode at all */ +TEST_F(valid_modes, set) +{ + int ret; + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + variant->mode); + ASSERT_EQ(ret, 0); + + _exit(0); +} + +/* Enabling, locking then disabling is rejected */ +TEST_F(valid_modes, enable_lock_disable) +{ + unsigned long mode; + int ret; + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + variant->mode); + ASSERT_EQ(ret, 0); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + ASSERT_EQ(mode, variant->mode); + + ret = prctl(PR_LOCK_SHADOW_STACK_STATUS, variant->mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, 0); + ASSERT_EQ(ret, -EBUSY); + + _exit(0); +} + +/* Locking then enabling is rejected */ +TEST_F(valid_modes, lock_enable) +{ + unsigned long mode; + int ret; + + ret = prctl(PR_LOCK_SHADOW_STACK_STATUS, variant->mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + variant->mode); + ASSERT_EQ(ret, -EBUSY); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + ASSERT_EQ(mode, 0); + + _exit(0); +} + +/* Locking then changing other modes is fine */ +TEST_F(valid_modes, lock_enable_disable_others) +{ + unsigned long mode; + int ret; + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + variant->mode); + ASSERT_EQ(ret, 0); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + ASSERT_EQ(mode, variant->mode); + + ret = prctl(PR_LOCK_SHADOW_STACK_STATUS, variant->mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + PR_SHADOW_STACK_ALL_MODES); + ASSERT_EQ(ret, 0); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + ASSERT_EQ(mode, PR_SHADOW_STACK_ALL_MODES); + + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + variant->mode); + ASSERT_EQ(ret, 0); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + ASSERT_EQ(mode, variant->mode); + + _exit(0); +} + +int main(int argc, char **argv) +{ + unsigned long mode; + int ret; + + if (!(getauxval(AT_HWCAP2) & HWCAP2_GCS)) + ksft_exit_skip("SKIP GCS not supported\n"); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + if (ret) { + ksft_print_msg("Failed to read GCS state: %d\n", ret); + return EXIT_FAILURE; + } + + if (mode & PR_SHADOW_STACK_ENABLE) { + ksft_print_msg("GCS was enabled, test unsupported\n"); + return KSFT_SKIP; + } + + return test_harness_run(argc, argv); +} From patchwork Wed Nov 22 09:42:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746281 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A35031DA53; Wed, 22 Nov 2023 09:47:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VTfCt5Jk" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 716F5C433CA; Wed, 22 Nov 2023 09:47:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646474; bh=8h4PDzJe8RjVew6BhwmHNeyc214Uhnuy6Y86+OPZqvw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=VTfCt5JkV+3ZknfE1kWEnGBNKOi8vM0WJjx8C5Ewthj/W0w1tvry9bX2fqoGle0I7 4ZTaO8VN6BDOWCbo9MVsJtfmsoGNKgS/8cMkh6vR5z/fGrOL2196d87UyeP7EbnNoB m+ebPkB2gwqSLmQgvLYcmAOnX6iBy8CDZtTR9RCsYLUQU58ubP8RTlY2o3/fN96Tgn T4ESs43VPLeBK2CN8yLzYxWst4SAMJXt5cUHEZTU7iOtDlr35gsMZYGRyZjcqb/DVu 9XoZ/Yq3bhNKe9pAEGWSHhNZCnO1m1kkEqQLs4V0N1i8IYcP3iwZF1RGuaYPs+fWDW InHKZxfsOX0vA== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:47 +0000 Subject: [PATCH v7 37/39] kselftest/arm64: Add a GCS stress test Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-37-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=21150; i=broonie@kernel.org; h=from:subject:message-id; bh=8h4PDzJe8RjVew6BhwmHNeyc214Uhnuy6Y86+OPZqvw=; b=owGbwMvMwMWocq27KDak/QLjabUkhtTYs0LqxTN5/XV0rvl/bS9tEHyWuOHYYe1ydcvSxd909 GI3mLl3MhqzMDByMciKKbKsfZaxKj1cYuv8R/NfwQxiZQKZwsDFKQATkXjM/j9HPzZAlNetqPa4 SaVjUDHzPK00zksRX3aYvjHX9zBc0yWes2Czh8v1SXzTeNIYnC95KMpfVY67KTCtPqyO/1Jlkc2 nuQfNkrhqrVZNn3A293wez2q5kOL1W8SONX/T9nWL36rSvaHM5ecfz9+PGEWmfCo+sNdSZ9rJ3y uK9fYv7xL2Ff2Umvjy2a1I64Dzrlpx6mqTp/gJFe5PXTJ3OUfFVOeotht3dN+/ZFWZ/T/rz6SJ3 o8MbSS50yemh219Ga+SH3DUl+nYc5cu4XsJcYJGr5h+nebpMU1kVYj+FRmqp6Kd6LtW6rs/94Kp OaxsDT9WTNZ6MdN6qtoG5QLWiHau3rakHBa5P9uSajMA X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Add a stress test which runs one more process than we have CPUs spinning through a very recursive function with frequent syscalls immediately prior to return and signals being injected every 100ms. The goal is to flag up any scheduling related issues, for example failure to ensure that barriers are inserted when moving a GCS using task to another CPU. The test runs for a configurable amount of time, defaulting to 10 seconds. Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/gcs/.gitignore | 2 + tools/testing/selftests/arm64/gcs/Makefile | 6 +- tools/testing/selftests/arm64/gcs/asm-offsets.h | 0 .../selftests/arm64/gcs/gcs-stress-thread.S | 311 ++++++++++++ tools/testing/selftests/arm64/gcs/gcs-stress.c | 532 +++++++++++++++++++++ 5 files changed, 850 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/arm64/gcs/.gitignore b/tools/testing/selftests/arm64/gcs/.gitignore index 0c86f53f68ad..1e8d1f6b27f2 100644 --- a/tools/testing/selftests/arm64/gcs/.gitignore +++ b/tools/testing/selftests/arm64/gcs/.gitignore @@ -1,3 +1,5 @@ basic-gcs libc-gcs gcs-locking +gcs-stress +gcs-stress-thread diff --git a/tools/testing/selftests/arm64/gcs/Makefile b/tools/testing/selftests/arm64/gcs/Makefile index 2173d6275956..d8b06ca51e22 100644 --- a/tools/testing/selftests/arm64/gcs/Makefile +++ b/tools/testing/selftests/arm64/gcs/Makefile @@ -6,7 +6,8 @@ # nolibc. # -TEST_GEN_PROGS := basic-gcs libc-gcs gcs-locking +TEST_GEN_PROGS := basic-gcs libc-gcs gcs-locking gcs-stress +TEST_GEN_PROGS_EXTENDED := gcs-stress-thread LDLIBS+=-lpthread @@ -18,3 +19,6 @@ $(OUTPUT)/basic-gcs: basic-gcs.c -I../../../../../usr/include \ -std=gnu99 -I../.. -g \ -ffreestanding -Wall $^ -o $@ -lgcc + +$(OUTPUT)/gcs-stress-thread: gcs-stress-thread.S + $(CC) -nostdlib $^ -o $@ diff --git a/tools/testing/selftests/arm64/gcs/asm-offsets.h b/tools/testing/selftests/arm64/gcs/asm-offsets.h new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/tools/testing/selftests/arm64/gcs/gcs-stress-thread.S b/tools/testing/selftests/arm64/gcs/gcs-stress-thread.S new file mode 100644 index 000000000000..4fe8695333e5 --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/gcs-stress-thread.S @@ -0,0 +1,311 @@ +// Program that loops for ever doing lots of recursions and system calls, +// intended to be used as part of a stress test for GCS context switching. +// +// Copyright 2015-2023 Arm Ltd + +#include + +#define sa_sz 32 +#define sa_flags 8 +#define sa_handler 0 +#define sa_mask_sz 8 + +#define si_code 8 + +#define SIGINT 2 +#define SIGABRT 6 +#define SIGUSR1 10 +#define SIGSEGV 11 +#define SIGUSR2 12 +#define SIGTERM 15 +#define SEGV_CPERR 10 + +#define SA_NODEFER 1073741824 +#define SA_SIGINFO 4 +#define ucontext_regs 184 + +#define PR_SET_SHADOW_STACK_STATUS 72 +# define PR_SHADOW_STACK_ENABLE (1UL << 0) + +#define GCSPR_EL0 S3_3_C2_C5_1 + +.macro function name + .macro endfunction + .type \name, @function + .purgem endfunction + .endm +\name: +.endm + +// Print a single character x0 to stdout +// Clobbers x0-x2,x8 +function putc + str x0, [sp, #-16]! + + mov x0, #1 // STDOUT_FILENO + mov x1, sp + mov x2, #1 + mov x8, #__NR_write + svc #0 + + add sp, sp, #16 + ret +endfunction +.globl putc + +// Print a NUL-terminated string starting at address x0 to stdout +// Clobbers x0-x3,x8 +function puts + mov x1, x0 + + mov x2, #0 +0: ldrb w3, [x0], #1 + cbz w3, 1f + add x2, x2, #1 + b 0b + +1: mov w0, #1 // STDOUT_FILENO + mov x8, #__NR_write + svc #0 + + ret +endfunction +.globl puts + +// Utility macro to print a literal string +// Clobbers x0-x4,x8 +.macro puts string + .pushsection .rodata.str1.1, "aMS", @progbits, 1 +.L__puts_literal\@: .string "\string" + .popsection + + ldr x0, =.L__puts_literal\@ + bl puts +.endm + +// Print an unsigned decimal number x0 to stdout +// Clobbers x0-x4,x8 +function putdec + mov x1, sp + str x30, [sp, #-32]! // Result can't be > 20 digits + + mov x2, #0 + strb w2, [x1, #-1]! // Write the NUL terminator + + mov x2, #10 +0: udiv x3, x0, x2 // div-mod loop to generate the digits + msub x0, x3, x2, x0 + add w0, w0, #'0' + strb w0, [x1, #-1]! + mov x0, x3 + cbnz x3, 0b + + ldrb w0, [x1] + cbnz w0, 1f + mov w0, #'0' // Print "0" for 0, not "" + strb w0, [x1, #-1]! + +1: mov x0, x1 + bl puts + + ldr x30, [sp], #32 + ret +endfunction +.globl putdec + +// Print an unsigned decimal number x0 to stdout, followed by a newline +// Clobbers x0-x5,x8 +function putdecn + mov x5, x30 + + bl putdec + mov x0, #'\n' + bl putc + + ret x5 +endfunction +.globl putdecn + +// Fill x1 bytes starting at x0 with 0. +// Clobbers x1, x2. +function memclr + mov w2, #0 +endfunction +.globl memclr + // fall through to memfill + +// Trivial memory fill: fill x1 bytes starting at address x0 with byte w2 +// Clobbers x1 +function memfill + cmp x1, #0 + b.eq 1f + +0: strb w2, [x0], #1 + subs x1, x1, #1 + b.ne 0b + +1: ret +endfunction +.globl memfill + +// w0: signal number +// x1: sa_action +// w2: sa_flags +// Clobbers x0-x6,x8 +function setsignal + str x30, [sp, #-((sa_sz + 15) / 16 * 16 + 16)]! + + mov w4, w0 + mov x5, x1 + mov w6, w2 + + add x0, sp, #16 + mov x1, #sa_sz + bl memclr + + mov w0, w4 + add x1, sp, #16 + str w6, [x1, #sa_flags] + str x5, [x1, #sa_handler] + mov x2, #0 + mov x3, #sa_mask_sz + mov x8, #__NR_rt_sigaction + svc #0 + + cbz w0, 1f + + puts "sigaction failure\n" + b abort + +1: ldr x30, [sp], #((sa_sz + 15) / 16 * 16 + 16) + ret +endfunction + + +function tickle_handler + // Perhaps collect GCSPR_EL0 here in future? + ret +endfunction + +function terminate_handler + mov w21, w0 + mov x20, x2 + + puts "Terminated by signal " + mov w0, w21 + bl putdec + puts ", no error\n" + + mov x0, #0 + mov x8, #__NR_exit + svc #0 +endfunction + +function segv_handler + // stash the siginfo_t * + mov x20, x1 + + // Disable GCS, we don't want additional faults logging things + mov x0, PR_SET_SHADOW_STACK_STATUS + mov x1, xzr + mov x2, xzr + mov x3, xzr + mov x4, xzr + mov x5, xzr + mov x8, #__NR_prctl + svc #0 + + puts "Got SIGSEGV code " + + ldr x21, [x20, #si_code] + mov x0, x21 + bl putdec + + // GCS faults should have si_code SEGV_CPERR + cmp x21, #SEGV_CPERR + bne 1f + + puts " (GCS violation)" +1: + mov x0, '\n' + bl putc + b abort +endfunction + +// Recurse x20 times +.macro recurse id +function recurse\id + stp x29, x30, [sp, #-16]! + mov x29, sp + + cmp x20, 0 + beq 1f + sub x20, x20, 1 + bl recurse\id + +1: + ldp x29, x30, [sp], #16 + + // Do a syscall immediately prior to returning to try to provoke + // scheduling and migration at a point where coherency issues + // might trigger. + mov x8, #__NR_getpid + svc #0 + + ret +endfunction +.endmacro + +// Generate and use two copies so we're changing the GCS contents +recurse 1 +recurse 2 + +.globl _start +function _start + // Run with GCS + mov x0, PR_SET_SHADOW_STACK_STATUS + mov x1, PR_SHADOW_STACK_ENABLE + mov x2, xzr + mov x3, xzr + mov x4, xzr + mov x5, xzr + mov x8, #__NR_prctl + svc #0 + cbz x0, 1f + puts "Failed to enable GCS\n" + b abort +1: + + mov w0, #SIGTERM + adr x1, terminate_handler + mov w2, #SA_SIGINFO + bl setsignal + + mov w0, #SIGUSR1 + adr x1, tickle_handler + mov w2, #SA_SIGINFO + orr w2, w2, #SA_NODEFER + bl setsignal + + mov w0, #SIGSEGV + adr x1, segv_handler + mov w2, #SA_SIGINFO + orr w2, w2, #SA_NODEFER + bl setsignal + + puts "Running\n" + +loop: + // Small recursion depth so we're frequently flipping between + // the two recursors and changing what's on the stack + mov x20, #5 + bl recurse1 + mov x20, #5 + bl recurse2 + b loop +endfunction + +abort: + mov x0, #255 + mov x8, #__NR_exit + svc #0 diff --git a/tools/testing/selftests/arm64/gcs/gcs-stress.c b/tools/testing/selftests/arm64/gcs/gcs-stress.c new file mode 100644 index 000000000000..23fd8ec37bdc --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/gcs-stress.c @@ -0,0 +1,532 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2022-3 ARM Limited. + */ + +#define _GNU_SOURCE +#define _POSIX_C_SOURCE 199309L + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../../kselftest.h" + +struct child_data { + char *name, *output; + pid_t pid; + int stdout; + bool output_seen; + bool exited; + int exit_status; + int exit_signal; +}; + +static int epoll_fd; +static struct child_data *children; +static struct epoll_event *evs; +static int tests; +static int num_children; +static bool terminate; + +static int startup_pipe[2]; + +static int num_processors(void) +{ + long nproc = sysconf(_SC_NPROCESSORS_CONF); + if (nproc < 0) { + perror("Unable to read number of processors\n"); + exit(EXIT_FAILURE); + } + + return nproc; +} + +static void start_thread(struct child_data *child) +{ + int ret, pipefd[2], i; + struct epoll_event ev; + + ret = pipe(pipefd); + if (ret != 0) + ksft_exit_fail_msg("Failed to create stdout pipe: %s (%d)\n", + strerror(errno), errno); + + child->pid = fork(); + if (child->pid == -1) + ksft_exit_fail_msg("fork() failed: %s (%d)\n", + strerror(errno), errno); + + if (!child->pid) { + /* + * In child, replace stdout with the pipe, errors to + * stderr from here as kselftest prints to stdout. + */ + ret = dup2(pipefd[1], 1); + if (ret == -1) { + fprintf(stderr, "dup2() %d\n", errno); + exit(EXIT_FAILURE); + } + + /* + * Duplicate the read side of the startup pipe to + * FD 3 so we can close everything else. + */ + ret = dup2(startup_pipe[0], 3); + if (ret == -1) { + fprintf(stderr, "dup2() %d\n", errno); + exit(EXIT_FAILURE); + } + + /* + * Very dumb mechanism to clean open FDs other than + * stdio. We don't want O_CLOEXEC for the pipes... + */ + for (i = 4; i < 8192; i++) + close(i); + + /* + * Read from the startup pipe, there should be no data + * and we should block until it is closed. We just + * carry on on error since this isn't super critical. + */ + ret = read(3, &i, sizeof(i)); + if (ret < 0) + fprintf(stderr, "read(startp pipe) failed: %s (%d)\n", + strerror(errno), errno); + if (ret > 0) + fprintf(stderr, "%d bytes of data on startup pipe\n", + ret); + close(3); + + ret = execl("gcs-stress-thread", "gcs-stress-thread", NULL); + fprintf(stderr, "execl(gcs-stress-thread) failed: %d (%s)\n", + errno, strerror(errno)); + + exit(EXIT_FAILURE); + } else { + /* + * In parent, remember the child and close our copy of the + * write side of stdout. + */ + close(pipefd[1]); + child->stdout = pipefd[0]; + child->output = NULL; + child->exited = false; + child->output_seen = false; + + ev.events = EPOLLIN | EPOLLHUP; + ev.data.ptr = child; + + ret = asprintf(&child->name, "Thread-%d", child->pid); + if (ret == -1) + ksft_exit_fail_msg("asprintf() failed\n"); + + ret = epoll_ctl(epoll_fd, EPOLL_CTL_ADD, child->stdout, &ev); + if (ret < 0) { + ksft_exit_fail_msg("%s EPOLL_CTL_ADD failed: %s (%d)\n", + child->name, strerror(errno), errno); + } + } + + ksft_print_msg("Started %s\n", child->name); + num_children++; +} + +static bool child_output_read(struct child_data *child) +{ + char read_data[1024]; + char work[1024]; + int ret, len, cur_work, cur_read; + + ret = read(child->stdout, read_data, sizeof(read_data)); + if (ret < 0) { + if (errno == EINTR) + return true; + + ksft_print_msg("%s: read() failed: %s (%d)\n", + child->name, strerror(errno), + errno); + return false; + } + len = ret; + + child->output_seen = true; + + /* Pick up any partial read */ + if (child->output) { + strncpy(work, child->output, sizeof(work) - 1); + cur_work = strnlen(work, sizeof(work)); + free(child->output); + child->output = NULL; + } else { + cur_work = 0; + } + + cur_read = 0; + while (cur_read < len) { + work[cur_work] = read_data[cur_read++]; + + if (work[cur_work] == '\n') { + work[cur_work] = '\0'; + ksft_print_msg("%s: %s\n", child->name, work); + cur_work = 0; + } else { + cur_work++; + } + } + + if (cur_work) { + work[cur_work] = '\0'; + ret = asprintf(&child->output, "%s", work); + if (ret == -1) + ksft_exit_fail_msg("Out of memory\n"); + } + + return false; +} + +static void child_output(struct child_data *child, uint32_t events, + bool flush) +{ + bool read_more; + + if (events & EPOLLIN) { + do { + read_more = child_output_read(child); + } while (read_more); + } + + if (events & EPOLLHUP) { + close(child->stdout); + child->stdout = -1; + flush = true; + } + + if (flush && child->output) { + ksft_print_msg("%s: %s\n", child->name, child->output); + free(child->output); + child->output = NULL; + } +} + +static void child_tickle(struct child_data *child) +{ + if (child->output_seen && !child->exited) + kill(child->pid, SIGUSR1); +} + +static void child_stop(struct child_data *child) +{ + if (!child->exited) + kill(child->pid, SIGTERM); +} + +static void child_cleanup(struct child_data *child) +{ + pid_t ret; + int status; + bool fail = false; + + if (!child->exited) { + do { + ret = waitpid(child->pid, &status, 0); + if (ret == -1 && errno == EINTR) + continue; + + if (ret == -1) { + ksft_print_msg("waitpid(%d) failed: %s (%d)\n", + child->pid, strerror(errno), + errno); + fail = true; + break; + } + + if (WIFEXITED(status)) { + child->exit_status = WEXITSTATUS(status); + child->exited = true; + } + + if (WIFSIGNALED(status)) { + child->exit_signal = WTERMSIG(status); + ksft_print_msg("%s: Exited due to signal %d\n", + child->name); + fail = true; + child->exited = true; + } + } while (!child->exited); + } + + if (!child->output_seen) { + ksft_print_msg("%s no output seen\n", child->name); + fail = true; + } + + if (child->exit_status != 0) { + ksft_print_msg("%s exited with error code %d\n", + child->name, child->exit_status); + fail = true; + } + + ksft_test_result(!fail, "%s\n", child->name); +} + +static void handle_child_signal(int sig, siginfo_t *info, void *context) +{ + int i; + bool found = false; + + for (i = 0; i < num_children; i++) { + if (children[i].pid == info->si_pid) { + children[i].exited = true; + children[i].exit_status = info->si_status; + found = true; + break; + } + } + + if (!found) + ksft_print_msg("SIGCHLD for unknown PID %d with status %d\n", + info->si_pid, info->si_status); +} + +static void handle_exit_signal(int sig, siginfo_t *info, void *context) +{ + int i; + + /* If we're already exiting then don't signal again */ + if (terminate) + return; + + ksft_print_msg("Got signal, exiting...\n"); + + terminate = true; + + /* + * This should be redundant, the main loop should clean up + * after us, but for safety stop everything we can here. + */ + for (i = 0; i < num_children; i++) + child_stop(&children[i]); +} + +/* Handle any pending output without blocking */ +static void drain_output(bool flush) +{ + int ret = 1; + int i; + + while (ret > 0) { + ret = epoll_wait(epoll_fd, evs, tests, 0); + if (ret < 0) { + if (errno == EINTR) + continue; + ksft_print_msg("epoll_wait() failed: %s (%d)\n", + strerror(errno), errno); + } + + for (i = 0; i < ret; i++) + child_output(evs[i].data.ptr, evs[i].events, flush); + } +} + +static const struct option options[] = { + { "timeout", required_argument, NULL, 't' }, + { } +}; + +int main(int argc, char **argv) +{ + int seen_children; + bool all_children_started = false; + int gcs_threads; + int timeout = 10; + int ret, cpus, i, c; + struct sigaction sa; + + while ((c = getopt_long(argc, argv, "t:", options, NULL)) != -1) { + switch (c) { + case 't': + ret = sscanf(optarg, "%d", &timeout); + if (ret != 1) + ksft_exit_fail_msg("Failed to parse timeout %s\n", + optarg); + break; + default: + ksft_exit_fail_msg("Unknown argument\n"); + } + } + + cpus = num_processors(); + tests = 0; + + if (getauxval(AT_HWCAP2) & HWCAP2_GCS) { + /* One extra thread, trying to trigger migrations */ + gcs_threads = cpus + 1; + tests += gcs_threads; + } else { + gcs_threads = 0; + } + + ksft_print_header(); + ksft_set_plan(tests); + + ksft_print_msg("%d CPUs, %d GCS threads\n", + cpus, gcs_threads); + + if (!tests) + ksft_exit_skip("No tests scheduled\n"); + + if (timeout > 0) + ksft_print_msg("Will run for %ds\n", timeout); + else + ksft_print_msg("Will run until terminated\n"); + + children = calloc(sizeof(*children), tests); + if (!children) + ksft_exit_fail_msg("Unable to allocate child data\n"); + + ret = epoll_create1(EPOLL_CLOEXEC); + if (ret < 0) + ksft_exit_fail_msg("epoll_create1() failed: %s (%d)\n", + strerror(errno), ret); + epoll_fd = ret; + + /* Create a pipe which children will block on before execing */ + ret = pipe(startup_pipe); + if (ret != 0) + ksft_exit_fail_msg("Failed to create startup pipe: %s (%d)\n", + strerror(errno), errno); + + /* Get signal handers ready before we start any children */ + memset(&sa, 0, sizeof(sa)); + sa.sa_sigaction = handle_exit_signal; + sa.sa_flags = SA_RESTART | SA_SIGINFO; + sigemptyset(&sa.sa_mask); + ret = sigaction(SIGINT, &sa, NULL); + if (ret < 0) + ksft_print_msg("Failed to install SIGINT handler: %s (%d)\n", + strerror(errno), errno); + ret = sigaction(SIGTERM, &sa, NULL); + if (ret < 0) + ksft_print_msg("Failed to install SIGTERM handler: %s (%d)\n", + strerror(errno), errno); + sa.sa_sigaction = handle_child_signal; + ret = sigaction(SIGCHLD, &sa, NULL); + if (ret < 0) + ksft_print_msg("Failed to install SIGCHLD handler: %s (%d)\n", + strerror(errno), errno); + + evs = calloc(tests, sizeof(*evs)); + if (!evs) + ksft_exit_fail_msg("Failed to allocated %d epoll events\n", + tests); + + for (i = 0; i < gcs_threads; i++) + start_thread(&children[i]); + + /* + * All children started, close the startup pipe and let them + * run. + */ + close(startup_pipe[0]); + close(startup_pipe[1]); + + timeout *= 10; + for (;;) { + /* Did we get a signal asking us to exit? */ + if (terminate) + break; + + /* + * Timeout is counted in 100ms with no output, the + * tests print during startup then are silent when + * running so this should ensure they all ran enough + * to install the signal handler, this is especially + * useful in emulation where we will both be slow and + * likely to have a large set of VLs. + */ + ret = epoll_wait(epoll_fd, evs, tests, 100); + if (ret < 0) { + if (errno == EINTR) + continue; + ksft_exit_fail_msg("epoll_wait() failed: %s (%d)\n", + strerror(errno), errno); + } + + /* Output? */ + if (ret > 0) { + for (i = 0; i < ret; i++) { + child_output(evs[i].data.ptr, evs[i].events, + false); + } + continue; + } + + /* Otherwise epoll_wait() timed out */ + + /* + * If the child processes have not produced output they + * aren't actually running the tests yet. + */ + if (!all_children_started) { + seen_children = 0; + + for (i = 0; i < num_children; i++) + if (children[i].output_seen || + children[i].exited) + seen_children++; + + if (seen_children != num_children) { + ksft_print_msg("Waiting for %d children\n", + num_children - seen_children); + continue; + } + + all_children_started = true; + } + + ksft_print_msg("Sending signals, timeout remaining: %d00ms\n", + timeout); + + for (i = 0; i < num_children; i++) + child_tickle(&children[i]); + + /* Negative timeout means run indefinitely */ + if (timeout < 0) + continue; + if (--timeout == 0) + break; + } + + ksft_print_msg("Finishing up...\n"); + terminate = true; + + for (i = 0; i < tests; i++) + child_stop(&children[i]); + + drain_output(false); + + for (i = 0; i < tests; i++) + child_cleanup(&children[i]); + + drain_output(true); + + ksft_print_cnts(); + + return 0; +} From patchwork Wed Nov 22 09:42:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 746280 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 52B3056476; Wed, 22 Nov 2023 09:48:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="uZVDPcKF" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EB795C433C9; Wed, 22 Nov 2023 09:48:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700646492; bh=+DcIGbRVvw60QaNepPSRFBnETXstp9aOb5bVveLOaFg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=uZVDPcKFka+wlMJXjfzajobqpM6EaIscRoqp8YtTGuiZLIBclLh/18h+beyzfCCak O611N3cFEGc9ooTue7jvmAjHduLzlohXVD9CMIjCWn79H/TlnFbuXsNc8sX94PgzpD CziDJmtJS4Lxr7Q90BN45O6K7RZDp60OhWbeVYOizHCAv+pOxqIKTxJVflyNZ+J6XH /WFEwQWOydbHD+eyzieR8s/u37khmup0ORNwiLjGPrd5KdyWHwd+12K9tNSjVXsVMU h4yc9Yk3hd0sUoKwmILtXr0+4rKmOlfPSNZJELuR9deSXlQR1au1DbSTFSMH3s0Dw3 UjGBYMzpNCEaQ== From: Mark Brown Date: Wed, 22 Nov 2023 09:42:49 +0000 Subject: [PATCH v7 39/39] kselftest/clone3: Enable GCS in the clone3 selftests Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20231122-arm64-gcs-v7-39-201c483bd775@kernel.org> References: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> In-Reply-To: <20231122-arm64-gcs-v7-0-201c483bd775@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=2416; i=broonie@kernel.org; h=from:subject:message-id; bh=+DcIGbRVvw60QaNepPSRFBnETXstp9aOb5bVveLOaFg=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlXc0Uf5JbJdKOG4lKmXzlRMK5DRWagk6L/dGG+ 23SAtCf39uJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZV3NFAAKCRAk1otyXVSH 0HKwB/44qmjC2meQurJ74cEJ5LyDpr1HY6Y1uWSWnrnkSyRi8ifcJAyyiipIeWWT2m2p19Mpf+V 5qrOSTOZdbGPeoAgIKaULvFGqHP/ksUE8h92lW7xsOkWWWvpcMBPOCkmhlsIYyaWpVOSMobVII8 PBDQf7mV3cJuvY8YfCMIkBz3Ul6mwtxdu9vNby4YUtXDRU9U3cGAcL6XtW3KntS2xI0+aDTcs7i YX48MJ+hv3H+CYavScUNHVuNSx8y1Q3eta1i1xV3wGLOaAAxBPAvtoCuwkfsLCOAy6jNqbmbBPU 2P7pVrY78UsvDTA8mvMcJrJSnASVS4HY2NdgEMkZbCxKMCbn X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB In order to allow testing without full userspace support for shadow stacks the clone3() selftests open code enabling them at runtime, hook up arm64 support. Signed-off-by: Mark Brown --- tools/testing/selftests/clone3/clone3.c | 37 +++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/tools/testing/selftests/clone3/clone3.c b/tools/testing/selftests/clone3/clone3.c index dbe52582573c..6114a33d6ec2 100644 --- a/tools/testing/selftests/clone3/clone3.c +++ b/tools/testing/selftests/clone3/clone3.c @@ -432,6 +432,43 @@ static inline void enable_shadow_stack(void) #endif +#ifdef __aarc64__ +#define PR_SET_SHADOW_STACK_STATUS 72 +# define PR_SHADOW_STACK_ENABLE (1UL << 0) + +#define my_syscall2(num, arg1, arg2) \ +({ \ + register long _num __asm__ ("x8") = (num); \ + register long _arg1 __asm__ ("x0") = (long)(arg1); \ + register long _arg2 __asm__ ("x1") = (long)(arg2); \ + register long _arg3 __asm__ ("x2") = 0; \ + register long _arg4 __asm__ ("x3") = 0; \ + register long _arg5 __asm__ ("x4") = 0; \ + \ + __asm__ volatile ( \ + "svc #0\n" \ + : "=r"(_arg1) \ + : "r"(_arg1), "r"(_arg2), \ + "r"(_arg3), "r"(_arg4), \ + "r"(_arg5), "r"(_num) \ + : "memory", "cc" \ + ); \ + _arg1; \ +}) + +#define ENABLE_SHADOW_STACK +static inline void enable_shadow_stack(void) +{ + int ret; + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + PR_SHADOW_STACK_ENABLE); + if (ret == 0) + shadow_stack_enabled = true; +} + +#endif + #ifndef ENABLE_SHADOW_STACK static void enable_shadow_stack(void) {