From patchwork Tue Jan 25 00:10:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 536687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9687BC433FE for ; Tue, 25 Jan 2022 03:37:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1325348AbiAYDgt (ORCPT ); Mon, 24 Jan 2022 22:36:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37560 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S3421399AbiAYC0v (ORCPT ); Mon, 24 Jan 2022 21:26:51 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 42473C08B4E1 for ; Mon, 24 Jan 2022 16:15:47 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D50FD61354 for ; Tue, 25 Jan 2022 00:15:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D8CEAC340E8; Tue, 25 Jan 2022 00:15:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1643069746; bh=HMQHyHN36InZca7SOj753D+ZgRaa/ByOBuanPgEEWxc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vJGCBqlutgdfxuViC7p4F4O25Tkjm7Vi7zxGijMXEB/0c8Lh6YUqivxZA8+nq9zgT 8YZUkVrQ0ejONkPfZIWTuLSjZL+N29hdodWzKSYn6ZNN9Ves6Kv8+v+96JVYzOKgzD pO5sXbg92IrfYVwBWxBBX9rJ+pHkUbs/iDj84ePTooytv3pxi7DwOHQiYWgaINknFU AgIkZTFyKDhLkfU+hOoR4Lp5nllO67PxpepxswyNQ+lmr8O2gB0dD1j1SaQqXl13lN OUHt4oPXUQH1lyenSWfi2w6WUcaQHit99WKKH/1jMd4OJmBkDYrI9Wu/HtOhEQ3oN7 lcCS36TiH30og== From: Mark Brown To: Catalin Marinas , Will Deacon , Marc Zyngier , Shuah Khan , Shuah Khan Cc: Alan Hayward , Luis Machado , Salil Akerkar , Basant Kumar Dwivedi , Szabolcs Nagy , James Morse , Alexandru Elisei , Suzuki K Poulose , linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, kvmarm@lists.cs.columbia.edu, Mark Brown Subject: [PATCH v8 06/38] arm64/sme: Manually encode SME instructions Date: Tue, 25 Jan 2022 00:10:42 +0000 Message-Id: <20220125001114.193425-7-broonie@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220125001114.193425-1-broonie@kernel.org> References: <20220125001114.193425-1-broonie@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3671; h=from:subject; bh=HMQHyHN36InZca7SOj753D+ZgRaa/ByOBuanPgEEWxc=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBh70AGU4t0P2TtzTYSd2s2Hen3CRw27ngQIyNfDOJC 4/nuJxWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCYe9ABgAKCRAk1otyXVSH0K7vB/ 0U4TNTZJacfpzOTFso5P+blU86sahrN3FDGigBiaXbFcvcDYJN3dHAobCgATuBPyUFaj7XVsjJdaiS fSzm0KTH+EfMLoyNvlAkA7IjByqL7Eeu1B/937WMlglqIyYvc1hI1HaDssR/Jv++mabQT7mPLTb18F OApA4ZRtN7ijspB8uxTk4AF77L92wa2dRNJiC0tICD8MIq7N3ckM9Mxn/xkFPqDr/1i4yfL7PkkwCB xRmfs3Jr1eNqrl28au6PdQqeF1RgrpmVXLCv3erlGr5OuiUay4pQxuQHyDpSsC/UMNezz1Olb21waQ 1B780W2qdO2emAnkvzYTkFMmBnEm7r X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org As with SVE rather than impose ambitious toolchain requirements for SME we manually encode the few instructions which we require in order to perform the work the kernel needs to do. The instructions used to save and restore context are provided as assembler macros while those for entering and leaving streaming mode are done in asm volatile blocks since they are expected to be used from C. We could do the SMSTART and SMSTOP operations with read/modify/write cycles on SVCR but using the aliases provided for individual field accesses should be slightly faster. These instructions are aliases for MSR but since our minimum toolchain requirements are old enough to mean that we can't use the sX_X_cX_cX_X form and they always use xzr rather than taking a value like write_sysreg_s() wants we just use .inst. Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 25 +++++++++++++ arch/arm64/include/asm/fpsimdmacros.h | 53 +++++++++++++++++++++++++++ 2 files changed, 78 insertions(+) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index cb24385e3632..c90f7f99a768 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -249,6 +249,31 @@ static inline void sve_setup(void) { } #endif /* ! CONFIG_ARM64_SVE */ +#ifdef CONFIG_ARM64_SME + +static inline void sme_smstart_sm(void) +{ + asm volatile(".inst 0xd503437f"); +} + +static inline void sme_smstop_sm(void) +{ + asm volatile(".inst 0xd503427f"); +} + +static inline void sme_smstop(void) +{ + asm volatile(".inst 0xd503467f"); +} + +#else + +static inline void sme_smstart_sm(void) { } +static inline void sme_smstop_sm(void) { } +static inline void sme_smstop(void) { } + +#endif /* ! CONFIG_ARM64_SME */ + /* For use by EFI runtime services calls only */ extern void __efi_fpsimd_begin(void); extern void __efi_fpsimd_end(void); diff --git a/arch/arm64/include/asm/fpsimdmacros.h b/arch/arm64/include/asm/fpsimdmacros.h index 2509d7dde55a..11c426ddd62c 100644 --- a/arch/arm64/include/asm/fpsimdmacros.h +++ b/arch/arm64/include/asm/fpsimdmacros.h @@ -93,6 +93,12 @@ .endif .endm +.macro _sme_check_wv v + .if (\v) < 12 || (\v) > 15 + .error "Bad vector select register \v." + .endif +.endm + /* SVE instruction encodings for non-SVE-capable assemblers */ /* (pre binutils 2.28, all kernel capable clang versions support SVE) */ @@ -174,6 +180,53 @@ | (\np) .endm +/* SME instruction encodings for non-SME-capable assemblers */ + +/* RDSVL X\nx, #\imm */ +.macro _sme_rdsvl nx, imm + _check_general_reg \nx + _check_num (\imm), -0x20, 0x1f + .inst 0x04bf5800 \ + | (\nx) \ + | (((\imm) & 0x3f) << 5) +.endm + +/* + * STR (vector from ZA array): + * STR ZA[\nw, #\offset], [X\nxbase, #\offset, MUL VL] + */ +.macro _sme_str_zav nw, nxbase, offset=0 + _sme_check_wv \nw + _check_general_reg \nxbase + _check_num (\offset), -0x100, 0xff + .inst 0xe1200000 \ + | (((\nw) & 3) << 13) \ + | ((\nxbase) << 5) \ + | ((\offset) & 7) +.endm + +/* + * LDR (vector to ZA array): + * LDR ZA[\nw, #\offset], [X\nxbase, #\offset, MUL VL] + */ +.macro _sme_ldr_zav nw, nxbase, offset=0 + _sme_check_wv \nw + _check_general_reg \nxbase + _check_num (\offset), -0x100, 0xff + .inst 0xe1000000 \ + | (((\nw) & 3) << 13) \ + | ((\nxbase) << 5) \ + | ((\offset) & 7) +.endm + +/* + * Zero the entire ZA array + * ZERO ZA + */ +.macro zero_za + .inst 0xc00800ff +.endm + .macro __for from:req, to:req .if (\from) == (\to) _for__body %\from