From patchwork Tue Feb 27 16:37:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jia Jie Ho X-Patchwork-Id: 776349 Received: from CHN02-BJS-obe.outbound.protection.partner.outlook.cn (mail-bjschn02on2095.outbound.protection.partner.outlook.cn [139.219.17.95]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78D2B208A3; Tue, 27 Feb 2024 16:38:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=139.219.17.95 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709051898; cv=fail; b=WB2NINGvVwxQC4+xqZYWn2V2HVZkB9opMeeIOyyKHZdzj5E6aDDW1dXarpZAHl8GlnMfHlN6Gc4TfeqjorUP274Nr3JWfbKbhAhAP6/BOBRRiHccMw9i5koAG4m+n54YSajv6iw+84hml3jhdqRTjHUfVTzDOLruZTfGMQ8m7yc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709051898; c=relaxed/simple; bh=wfVs5+YZnoQc8CcxqlenIofRy9YTq5OUy9r4NsL1wS0=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=gkmCecv++o9I0Xe/5WIelKNKXP2WG7kjOo82zIBga62sz4P7IStGWcZJl31vhykH2JhBS9n767Ex2JsIo7IdfzrNkqhJ/HAILUXdX8ngDmkhuZnKagi1bslJcL1alPQW1D2AaeQJ8rKF3vA4ed3mKrXMa6NETR9FzrUCQdvkfmQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=starfivetech.com; spf=pass smtp.mailfrom=starfivetech.com; arc=fail smtp.client-ip=139.219.17.95 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=starfivetech.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=starfivetech.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kYNjhkqe88LsiEoGYWHeycnEZ0FlOs3kE6/CS+EXz4Yv6t6lhU22JcyOQwvhk8H2BowBRIwHgnpPpRbxHO7X6vmBQ2lWZiR2mtmwbWVsXzeTQHG3w/qWvdeyFHvwPbnSPjfzjC3arb/uHxNsWZTmdzPTevxR0zD+le1hXV554aA3hIbNTpqoIvOwiePYn+FPLCdYNILbAT5cX100B+WaTuhUwbYZhbmQuAcz3vLxhKLOAzFejKjlsingMqWomFWh3+twcDX2zXzwyfoPZuAWprVXU0pHke9UCR27dRDByhNrSqhr9g0AhP0C9hptD8woKy1rbBDa8SMOqyEtIl2IHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gvkjuJQgBaQMbXcQsb5bU6na+3fU18CU/GW7PffI/30=; b=YmtRGkm1IlpHXhYwrn1G9UWBlPaFCbUiySsXux56mtkb4kiXsnMeKdk72dcD/JC/gzMcFtt+0jJpwvgKQdtvqyOHDf4BzDukcrEB3Pn2M11KA+GDIe1UIyJAACG3sqHCDILs3bKgV9ZjpIj+CsFRf/jXWXRbXd0LibzFqJ8FBt3GiG1WyphevYZJM48xf6r/6M+y7+jTOaDMp+drKpkHNjP1Fi1aLu6Vp/idEzC4aHVu+Tcefj4qnr5xhlnHZnB4q8IdrKBwImFCxkVuFu69ZnkP+nz9s5Y1ivZGcmZaKdmW3BJU2hxkCeNTc2nsGppgFYjCUOb0K/U5WvZ/sJe5AA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=starfivetech.com; dmarc=pass action=none header.from=starfivetech.com; dkim=pass header.d=starfivetech.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=starfivetech.com; Received: from SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn (2406:e500:c311:26::16) by SHXPR01MB0685.CHNPR01.prod.partner.outlook.cn (2406:e500:c311:27::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.42; Tue, 27 Feb 2024 16:38:08 +0000 Received: from SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn ([fe80::9ffd:1956:b45a:4a5d]) by SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn ([fe80::9ffd:1956:b45a:4a5d%6]) with mapi id 15.20.7249.041; Tue, 27 Feb 2024 16:38:08 +0000 From: Jia Jie Ho To: Herbert Xu , "David S . Miller" , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Eugeniy Paltsev , Vinod Koul , linux-crypto@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, dmaengine@vger.kernel.org Subject: [PATCH v3 1/6] dt-bindings: crypto: starfive: Add jh8100 support Date: Wed, 28 Feb 2024 00:37:53 +0800 Message-Id: <20240227163758.198133-2-jiajie.ho@starfivetech.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240227163758.198133-1-jiajie.ho@starfivetech.com> References: <20240227163758.198133-1-jiajie.ho@starfivetech.com> X-ClientProxiedBy: NT0PR01CA0014.CHNPR01.prod.partner.outlook.cn (2406:e500:c510::16) To SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn (2406:e500:c311:26::16) Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SHXPR01MB0670:EE_|SHXPR01MB0685:EE_ X-MS-Office365-Filtering-Correlation-Id: 846691b7-858d-415d-22c9-08dc37b27a98 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NflUOejmdlGd0aPEjSgCH9n9XtaGaPWSeED4g/8XRaRSXGEOOFzeAfYO1OxRhqK0NBWmY5mlYqLOeuDHHXAAELMncEDYf2ze0pQyulAYlyN+1Ad0pOsYrbvmt3Uda8PW/ujL7FWE8b8meygN9hwpp1RkN39msoUqybGG7LPSnCEpeA4vUP/HEGxoi+DGDtcocuC5723wmmHf/CyiOGQ/PlwOJ3Mvyvl8d9R65oXZPs4BYFky7oSnYkQHGwUHB4cXxOik/y9sUfbFJ6knmrc0s5rs/ODhqMg/a6mHZBqmkIL/yvsM1oryT5vvNAThJqbRK29G48ROKy936UTJrRSuc01/v+9ylD/NemTT4PEGdl6jTUmD2mGq5cU4hmlx9q3TDFA0Lnk0sCatME9QgUw/d8fFKLQgPa1fI6WDrDq1DYBURpAVhdOvp+jwSpeMLoDQlkF8459Pl7wITUQGEafVDLjE9tyWV+v0bI7gmcn9ST/xMlS/dRpDQu+0y7FHyJLRVgNpp9nP75ri21iTurpxA+SzNaajoBYQR78X6oq4ACRLBzOtDqqYFJajEpwlDfu1uJJZEzYAz9bC4bO7Z9riHTBzOdJ02N60yOUodfBvzK7N0LUjHl+XLul9wStefRijNvTD568MPuyRZtTy4jQomQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn; PTR:; CAT:NONE; SFS:(13230031)(38350700005)(921011); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: EH2CV9SEOmj4/30CuCNEEkGB1+0zKllgLj0hJchCFWtc2BViJw5fLLmGYzJAxqOQ3nwl2lhCN69pXaiX0dozt5K8UQVveHL9UshdklWSOE9zJq6ALp2/lNWk/Zjp0CVF36g0S/j37IwUoBiYu6L2t3Nw4EM3FsBf2krbpqsLcCR+44qekIaCSP+YbSZRUj+G2/rVyDVVwKTqjDbR8UAA+FodswuPwSjXAHeGwZc8WFdalOBplYgTDap1FB2GHmsAWv9V5v/W8CU0m20hw5Cn9lyvxMoEQQ7eV9mFxFG2BZQJcMCXQ1Hd6GlO+xmnlV5IwhfaJ/1x/CRfYnPTE5ViuH9kMov/tTP3kJYRSRZhDacnKeV2Tp39I1OsOI1n6GHRErAmVG5uRyIlaMbqXj0XMNqcOZgq5xBn87s/YJknWdgnAD0Z8JOCvLsaKnEtfa4QHZz+4aFdFZoP6oexWdYdWmiXYajYWa0yf8yrisUtuja8z1L8OPqLoCkvzCkd0YEHZWTd00rhj9EvDKebUWPB6qgCh4wUQeMHuKqfTWH7+Y2n2Qg7i7tJboZ5sd+MtOc1HTUGLlSxYjOLku63dau/FtJqaiu6UV2uPjHBtdIAeAx95VtpAgOLvI5wRESPVNx281FwH0qac0ZsG4B8gKeYNpJqkAchOxgD6WHqHlmdm2MXoooomUh7gk5OFv/XHhNTHpmdGlcijC/BpeDeCrq9oQyZAjW8lSN7dszJXSyA0WB9nPlKJRWKrTCOvv+O5XNZI1wfMx5A5mwYHj/uz42vSOuH1I3grptioqYHsirA42Ody4fq9xMdirspP9FUedM0rgsMeMJMvm0yDmSR1vuY3/IzLuxibg63MfGfsv8THmXcVtZx+w/FLzL7HSW8PbmEq7OGilOoEDtATI7U5LRW0fFOplwaashssiKd9PTl4aWk4jo33ZorfqZrP9zyfU9bXtZ7dVZh1bh5fMDi7p45zaHg8uDC1+sD/hLov5u/qjIxxQiqdFzM+CCblQ/WWj7k+1GP3Ws86onOlaiksIk8lnu2k7KzvUl7+R/rEPTaWoCovDxaUQvx0Y9G9yjyOBXi5oOuF0LuwqcV+yTfAL+SpZ2i8f0NpEVIz2/aKgK4HCZCeOLxvHzqe78lClV7CzO/vT5j8/AqLcbwebDBZidxNTvjQtbranSC1qZUn7bqmCG+JPDtW3+5glsJNVWMCXdQK6BwLf0iQepU0XCfFYdd7g4qI32Ebcu5xWTgeJICJvVbRfvoZVed8UUYlGgYKKR7mqom9tfNiBUsIug9Jx8jD+OvuAzNFc+RSFAOCvfPAToVUFFp0PihBssieFvIlRVKOUHK/26ikGbW282NPhEjcNldsJz0shZ7ryg5c3rIRlNh479HS67AIcfLNkMfeZXqLP86ExrROj/jm1sdoR3tCoF7Gg81zL/RLdwTNyvoiKP+ybMwv7+1i4RDeVBwl/FJUSYHGqKUd/UkbpK6yMMh7kcTJB8StHov3EQPWKhrUNjDohLMSx8TTImxNVa8p8aVovkrxhRlAYIIX5c29quLu8f/kIQaCwn9A5DgNR5qqpcezp+fGtulNrG/AiZkTPwt4fdcCcd4NRrwIH7uyqlCTg== X-OriginatorOrg: starfivetech.com X-MS-Exchange-CrossTenant-Network-Message-Id: 846691b7-858d-415d-22c9-08dc37b27a98 X-MS-Exchange-CrossTenant-AuthSource: SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2024 16:38:08.1053 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 06fe3fa3-1221-43d3-861b-5a4ee687a85c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ycJ8qfjnjfOOVT4YkV+Pukbkr901ikG81JvRujFb8MDX++zvT652wJzFCGvsyuorZWtNJTIAACfJdp/3WlIeAwpDMiHnJjW3Tc1y+HD/9fk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SHXPR01MB0685 Add compatible string and additional interrupt for StarFive JH8100 crypto engine. Signed-off-by: Jia Jie Ho --- .../crypto/starfive,jh7110-crypto.yaml | 30 +++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/Documentation/devicetree/bindings/crypto/starfive,jh7110-crypto.yaml b/Documentation/devicetree/bindings/crypto/starfive,jh7110-crypto.yaml index 71a2876bd6e4..d44d77908966 100644 --- a/Documentation/devicetree/bindings/crypto/starfive,jh7110-crypto.yaml +++ b/Documentation/devicetree/bindings/crypto/starfive,jh7110-crypto.yaml @@ -12,7 +12,9 @@ maintainers: properties: compatible: - const: starfive,jh7110-crypto + enum: + - starfive,jh8100-crypto + - starfive,jh7110-crypto reg: maxItems: 1 @@ -28,7 +30,10 @@ properties: - const: ahb interrupts: - maxItems: 1 + minItems: 1 + items: + - description: SHA2 module irq + - description: SM3 module irq resets: maxItems: 1 @@ -54,6 +59,27 @@ required: additionalProperties: false +allOf: + - if: + properties: + compatible: + const: starfive,jh7110-crypto + + then: + properties: + interrupts: + maxItems: 1 + + - if: + properties: + compatible: + const: starfive,jh8100-crypto + + then: + properties: + interrupts: + maxItems: 2 + examples: - | crypto: crypto@16000000 { From patchwork Tue Feb 27 16:37:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jia Jie Ho X-Patchwork-Id: 776348 Received: from CHN02-BJS-obe.outbound.protection.partner.outlook.cn (mail-bjschn02on2095.outbound.protection.partner.outlook.cn [139.219.17.95]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 904A54CB30; Tue, 27 Feb 2024 16:38:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=139.219.17.95 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709051904; cv=fail; b=VsNYNBRcreJ02gIcbHspfJJg5NnNmA7/x3xGmoSs4DSTvYTasbgFdbt8pad/TcvBiceTu79AcA9wBcZJ0BBTVVjgiz8n+3VjY9eES1eYduwEjNiZ4DQQL1pdcK8xR+HDFR1MWTAOjmHitR7B++Ob8HId6I1xsERZWMuTOCvgxiU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709051904; c=relaxed/simple; bh=NZ/M4xuBeEhFXE0Iq3QzB1Hs47D4Mwww8Y53G+GTsaI=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=TJACAiJjGU45i6hFOe/0JssB+ZyaduARe7hcIsQM6p8LftcZ6Cb1HnFr42kxjk9S6crBm6eSuNQkmCKHjUBMrJR+jxsKYYoa1ffB7gdNU6fdF3csjE61hlCcF6aRP/eGq7VNmB321iUi63gRn+eEN748qNrZzORwSdeSTPCpOho= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=starfivetech.com; spf=pass smtp.mailfrom=starfivetech.com; arc=fail smtp.client-ip=139.219.17.95 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=starfivetech.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=starfivetech.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Q/fU4+JjN5Zoh7CrEJvPLsQkIIEI30a3wtYLlmTLRb4Rzl5zmh9p6W8DpOfTrCr9imOJZ/fVM/Wt0CGLYDKMjONpXqeE9ayu3v7FvURxxEl1ONPaax/jNnO5yWLK/tjtPYzMVvpfRPDVolOnN0HDzFrjyK/YF7FPT0tvD3B10SP1RygCg/FuNsnvdMPIBdI/MHKL1zE2TFQ5V38g18M5oNaQ8y2byB4Ytn1xj5TYKy3qIsVpmObr708r3gtvu1tjrTbh7YLBMHp9jKpn4dgQp8SOZM87d/YlJ4xxQmQFqbJQ7eGwT18Rmv+dUtpd9/Pgon2nDMthoJo5ZGHSVNqJ9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Z/IF5+vxDHjNEIUSsIbLH5WSbf5uqgixkJs2tf/d9dw=; b=lbK9MXZvHqaj2/UkOxxHjM5kYr6faRevoR3qGPqw1vlh6Tst6thtE7brx6+Y5ocvMxPuxXKpO8hTmkt1+ZoxHDm75svEoeAvvfl385Q+KoE0I2QJLUUy/Uyd9DvSmUWcsiK3er4ZAhmWlOPP6Je+Bj1i+64w69myA5b1g0F9q3bd/JsCdVZvPDz2HLMLsc2l0dSpgy3d7aAhRNWob5PU+xKPOouxEv72sTl6qm3Oi6B8TcgeQ3LJtgvUu6+o1Ln2Iur5zNEcdKohw9hs3PDSYtLt06BAwCzlIdeBEh8eQOCQOUqhrH+Bxj3Lp87RME+AvW4IiYwjP+QxEhotpOndTw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=starfivetech.com; dmarc=pass action=none header.from=starfivetech.com; dkim=pass header.d=starfivetech.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=starfivetech.com; Received: from SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn (2406:e500:c311:26::16) by SHXPR01MB0685.CHNPR01.prod.partner.outlook.cn (2406:e500:c311:27::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.42; Tue, 27 Feb 2024 16:38:11 +0000 Received: from SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn ([fe80::9ffd:1956:b45a:4a5d]) by SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn ([fe80::9ffd:1956:b45a:4a5d%6]) with mapi id 15.20.7249.041; Tue, 27 Feb 2024 16:38:11 +0000 From: Jia Jie Ho To: Herbert Xu , "David S . Miller" , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Eugeniy Paltsev , Vinod Koul , linux-crypto@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, dmaengine@vger.kernel.org Subject: [PATCH v3 3/6] crypto: starfive: Use dma for aes requests Date: Wed, 28 Feb 2024 00:37:55 +0800 Message-Id: <20240227163758.198133-4-jiajie.ho@starfivetech.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240227163758.198133-1-jiajie.ho@starfivetech.com> References: <20240227163758.198133-1-jiajie.ho@starfivetech.com> X-ClientProxiedBy: NT0PR01CA0014.CHNPR01.prod.partner.outlook.cn (2406:e500:c510::16) To SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn (2406:e500:c311:26::16) Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SHXPR01MB0670:EE_|SHXPR01MB0685:EE_ X-MS-Office365-Filtering-Correlation-Id: c2004ad5-a24a-4dd6-070b-08dc37b27c4a X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lpyIrzI0Hcts98Nwk644qaNUGzqbFXgl4FocgBL7ZU52ZyiwkovIf809ONHies2BT9T7b2lxTNBiXzC9K9Gb+SsnnLeJgaDjgRX97WbjzU1Q1fXWNd7A3PaycL3WN3myPVAEVSUjji8qurvql04BR0Q4Z7qZ6P8E0olasxsoE0CvHOLQRAlXSSND+qb4RZC5hwpgvskx1IhQ6buzPWPnktG51Y+vOq87320EhkPpOnv8tY7QMCMaF0+DxbB8Yk6FmqqN2/mlGp8YP1ZswDhQCyXFw/V6ha+p2FdTnoxybGmDASsOwDOsiic3y7nh+YLerBsgB8oEfZpTEUnqvpC8XzJ/nCwuquOfDQ/XAF4LL04APTdsj8Dvxl/doW2oPBgj8ykwENaI3eyhec+cp5BT9y7N9YnhFPGJdUBQC103o9noFxC5nJ+9sE1pxS46b7j8AhrLG7hWWHCXc08LFWza5mItF92Vq868PWUg8xXiegltrXbCfqe4fhLhm5QO51Wpm5g4FjCycQNhNTVyJvqD/mYjGyDMJcBA7SKp3UJt36hKWJDHg7BSpXsT4XiuOyF/fIoELFV/PZUu0wME1BmFrvB6yPNk1j+B4ddvZGPWOmwY6Igg2MQpSLWUH4suEUVVx9HzupI71qz4PW3b8cEs7Q== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn; PTR:; CAT:NONE; SFS:(13230031)(38350700005)(921011); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: k1ewSa2CXx2BXRYALyZs7qGspFTp53Kf8F8ehOhJfOeTlKPX8UaV+VYIaMjCIw4yAJW8yOA4Xb6/j9utlVcuuOWMSl406fqduUwf4Fux5jIMaPyPOsdO1kpMVvJ9ro7z2NXSW/HAuDFxd9ZEXH6SnFX0Nwx0V+Xsgg3rBUTBtTZWYFxqqPSRGCTxTG9MZMherTjaPh5Io1xD2y97Rc0n8qDy7fu/SwCit7jY6GCJPA9t67C803veczfH9QpLgLlJS1wOebPX9UpbxhOAY3CuwFUnfaUp7JXQRJFvIzC3O6O68Y5gebMzE2Ax5bQ3hxqrm9KB9xt7KHpcFwqLQ7iWqXjwrFh4eXgj1EJmpbhwHnT2r0BCOa2Sv9qTjZJZ4sSPXKhlLUNic0eX+7chq6OebUQKeZ/aPAtwTZ8rz7I7yJ36cfma77PIqsw0zjrRKH8aDJ9noaUxgwG+2pNq6J8NbHPrl82hmve/jadmWQArL3bvn+sCnMwNdyjVrpjP2bI7mJJjSOpRPxnU7J1hc2OKCMhtFIyWrOjvlk7QC6gs5uEKxW7jyO1MJzlxnyMd+ZiO8/a9iB3dwHJwP8s1UkUe4rZjh7SB7JU2HaGA3IWjaGaxPKe+RyT4udWogE8GUzxtmj9UZfZQyYkqW4mcLhP4Zg1CFv33NQo59sXxP2RRRHW1hqDsyz22XgubI7wYQSaxKjZc671OJvJ0oFYHeF6waiVgDo9vpmbf3oV7ds2VnLF7ozZ6CNR3yQa3eQABoytcbZl2CEWF+ybrE25+3DL7TFC7x+RUlIEXVi+4SDpYSSTo9KVUw2rkv14Wcrs3Vh2FZhNACz75q5dqHEelg1RQXJ83gCvPt2gK3uMOiv5mVmc+nrYyyVKqqYV1wLPGmfHuylk6Nn6m4puFCb8jNOfli2JtqVQW68GQ9Pyc/d5qMN+ko2CxsZxe/aYH6YT7/I8wcHibYpP+7hDlpjFkPGnBr65QWw1ewCYYVjmaPOg2NsEMMaTmk+8FHb6/mMDzJTDhLhiak2xv5i4mr8R4jFGpZTRNoLBXzXNNESuWQ+ATU+DwGoqFbX4VUhoAlR2kETp6PmKmWyRq+pMCyz+4kCbxh6aO9gaQK3FHm+GnyDegGJNbkbUIKbzZH85E16j+wRIIiCIyXEYa3BmgslTjvEsD3PMUxX/4ZbbbsLBrgJTjw09hwxDC77XQTlDWHan1s7NURp8YWWj0Phafn9D3V0i8B8t6EoJAHVqS2/4zBPtymyMeA9EFvgB94kIXyyk5ay7JyKjpVABD4EDv7Wh5q7zFxTADyraBcQi/wd8uincqMmEPLh4HMeiPHM51hDLLHAtptagvvnMKviBp+kPyReArwkV5oNfgclA0IYjdQUnbdJWLkmlM4fsiNjIAkopyU700Uaj3VslVgJWYQtMqWxA7WdWEFQlZ0ivXi55++MoqPxNrsI5/5IlJGxCPBmvY/1iRoBkW9bxHMYTa65UR34MbGLtOkM2BiKj3l67F2AvttLXXxVeKRH3BpQxl6OV+839L4WVPZxiyNVuWqxWAYzbkeKDRTN/YcxGRZUlkWHJPr99soWAdvP1bKiFz1QDklkzlcQlu2OjOf93e/OHvfkfecQ== X-OriginatorOrg: starfivetech.com X-MS-Exchange-CrossTenant-Network-Message-Id: c2004ad5-a24a-4dd6-070b-08dc37b27c4a X-MS-Exchange-CrossTenant-AuthSource: SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2024 16:38:11.0665 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 06fe3fa3-1221-43d3-861b-5a4ee687a85c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mI3zZ3fg4oNKyA9d3SrrC7GUupEeD8pPjuKiESwKkq1akqLZLzM+OK8HPpHaq2j0tdMdXIPm66nmPgl5UX3AQbp6RD362Inei5VNNKoo3QI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SHXPR01MB0685 Convert AES module to use dma for data transfers to reduce cpu load and compatible with future variants. Signed-off-by: Jia Jie Ho --- drivers/crypto/starfive/Kconfig | 4 + drivers/crypto/starfive/jh7110-aes.c | 589 +++++++++++++++++--------- drivers/crypto/starfive/jh7110-cryp.c | 24 -- drivers/crypto/starfive/jh7110-cryp.h | 6 +- 4 files changed, 395 insertions(+), 228 deletions(-) diff --git a/drivers/crypto/starfive/Kconfig b/drivers/crypto/starfive/Kconfig index cb59357b58b2..0fe389e9f932 100644 --- a/drivers/crypto/starfive/Kconfig +++ b/drivers/crypto/starfive/Kconfig @@ -14,6 +14,10 @@ config CRYPTO_DEV_JH7110 select CRYPTO_RSA select CRYPTO_AES select CRYPTO_CCM + select CRYPTO_GCM + select CRYPTO_ECB + select CRYPTO_CBC + select CRYPTO_CTR help Support for StarFive JH7110 crypto hardware acceleration engine. This module provides acceleration for public key algo, diff --git a/drivers/crypto/starfive/jh7110-aes.c b/drivers/crypto/starfive/jh7110-aes.c index 1ac15cc4ef3c..72b7d46150d5 100644 --- a/drivers/crypto/starfive/jh7110-aes.c +++ b/drivers/crypto/starfive/jh7110-aes.c @@ -78,7 +78,7 @@ static inline int is_gcm(struct starfive_cryp_dev *cryp) return (cryp->flags & FLG_MODE_MASK) == STARFIVE_AES_MODE_GCM; } -static inline int is_encrypt(struct starfive_cryp_dev *cryp) +static inline bool is_encrypt(struct starfive_cryp_dev *cryp) { return cryp->flags & FLG_ENCRYPT; } @@ -103,16 +103,6 @@ static void starfive_aes_aead_hw_start(struct starfive_cryp_ctx *ctx, u32 hw_mod } } -static inline void starfive_aes_set_ivlen(struct starfive_cryp_ctx *ctx) -{ - struct starfive_cryp_dev *cryp = ctx->cryp; - - if (is_gcm(cryp)) - writel(GCM_AES_IV_SIZE, cryp->base + STARFIVE_AES_IVLEN); - else - writel(AES_BLOCK_SIZE, cryp->base + STARFIVE_AES_IVLEN); -} - static inline void starfive_aes_set_alen(struct starfive_cryp_ctx *ctx) { struct starfive_cryp_dev *cryp = ctx->cryp; @@ -261,7 +251,6 @@ static int starfive_aes_hw_init(struct starfive_cryp_ctx *ctx) rctx->csr.aes.mode = hw_mode; rctx->csr.aes.cmode = !is_encrypt(cryp); - rctx->csr.aes.ie = 1; rctx->csr.aes.stmode = STARFIVE_AES_MODE_XFB_1; if (cryp->side_chan) { @@ -279,7 +268,7 @@ static int starfive_aes_hw_init(struct starfive_cryp_ctx *ctx) case STARFIVE_AES_MODE_GCM: starfive_aes_set_alen(ctx); starfive_aes_set_mlen(ctx); - starfive_aes_set_ivlen(ctx); + writel(GCM_AES_IV_SIZE, cryp->base + STARFIVE_AES_IVLEN); starfive_aes_aead_hw_start(ctx, hw_mode); starfive_aes_write_iv(ctx, (void *)cryp->req.areq->iv); break; @@ -300,28 +289,30 @@ static int starfive_aes_hw_init(struct starfive_cryp_ctx *ctx) return cryp->err; } -static int starfive_aes_read_authtag(struct starfive_cryp_dev *cryp) +static int starfive_aes_read_authtag(struct starfive_cryp_ctx *ctx) { - int i, start_addr; + struct starfive_cryp_dev *cryp = ctx->cryp; + struct starfive_cryp_request_ctx *rctx = ctx->rctx; + int i; if (starfive_aes_wait_busy(cryp)) return dev_err_probe(cryp->dev, -ETIMEDOUT, "Timeout waiting for tag generation."); - start_addr = STARFIVE_AES_NONCE0; - - if (is_gcm(cryp)) - for (i = 0; i < AES_BLOCK_32; i++, start_addr += 4) - cryp->tag_out[i] = readl(cryp->base + start_addr); - else + if ((cryp->flags & FLG_MODE_MASK) == STARFIVE_AES_MODE_GCM) { + cryp->tag_out[0] = readl(cryp->base + STARFIVE_AES_NONCE0); + cryp->tag_out[1] = readl(cryp->base + STARFIVE_AES_NONCE1); + cryp->tag_out[2] = readl(cryp->base + STARFIVE_AES_NONCE2); + cryp->tag_out[3] = readl(cryp->base + STARFIVE_AES_NONCE3); + } else { for (i = 0; i < AES_BLOCK_32; i++) cryp->tag_out[i] = readl(cryp->base + STARFIVE_AES_AESDIO0R); + } if (is_encrypt(cryp)) { - scatterwalk_copychunks(cryp->tag_out, &cryp->out_walk, cryp->authsize, 1); + scatterwalk_map_and_copy(cryp->tag_out, rctx->out_sg, + cryp->total_in, cryp->authsize, 1); } else { - scatterwalk_copychunks(cryp->tag_in, &cryp->in_walk, cryp->authsize, 0); - if (crypto_memneq(cryp->tag_in, cryp->tag_out, cryp->authsize)) return dev_err_probe(cryp->dev, -EBADMSG, "Failed tag verification\n"); } @@ -329,23 +320,18 @@ static int starfive_aes_read_authtag(struct starfive_cryp_dev *cryp) return 0; } -static void starfive_aes_finish_req(struct starfive_cryp_dev *cryp) +static void starfive_aes_finish_req(struct starfive_cryp_ctx *ctx) { - union starfive_aes_csr csr; + struct starfive_cryp_dev *cryp = ctx->cryp; int err = cryp->err; if (!err && cryp->authsize) - err = starfive_aes_read_authtag(cryp); + err = starfive_aes_read_authtag(ctx); if (!err && ((cryp->flags & FLG_MODE_MASK) == STARFIVE_AES_MODE_CBC || (cryp->flags & FLG_MODE_MASK) == STARFIVE_AES_MODE_CTR)) starfive_aes_get_iv(cryp, (void *)cryp->req.sreq->iv); - /* reset irq flags*/ - csr.v = 0; - csr.aesrst = 1; - writel(csr.v, cryp->base + STARFIVE_AES_CSR); - if (cryp->authsize) crypto_finalize_aead_request(cryp->engine, cryp->req.areq, err); else @@ -353,39 +339,6 @@ static void starfive_aes_finish_req(struct starfive_cryp_dev *cryp) err); } -void starfive_aes_done_task(unsigned long param) -{ - struct starfive_cryp_dev *cryp = (struct starfive_cryp_dev *)param; - u32 block[AES_BLOCK_32]; - u32 stat; - int i; - - for (i = 0; i < AES_BLOCK_32; i++) - block[i] = readl(cryp->base + STARFIVE_AES_AESDIO0R); - - scatterwalk_copychunks(block, &cryp->out_walk, min_t(size_t, AES_BLOCK_SIZE, - cryp->total_out), 1); - - cryp->total_out -= min_t(size_t, AES_BLOCK_SIZE, cryp->total_out); - - if (!cryp->total_out) { - starfive_aes_finish_req(cryp); - return; - } - - memset(block, 0, AES_BLOCK_SIZE); - scatterwalk_copychunks(block, &cryp->in_walk, min_t(size_t, AES_BLOCK_SIZE, - cryp->total_in), 0); - cryp->total_in -= min_t(size_t, AES_BLOCK_SIZE, cryp->total_in); - - for (i = 0; i < AES_BLOCK_32; i++) - writel(block[i], cryp->base + STARFIVE_AES_AESDIO0R); - - stat = readl(cryp->base + STARFIVE_IE_MASK_OFFSET); - stat &= ~STARFIVE_IE_MASK_AES_DONE; - writel(stat, cryp->base + STARFIVE_IE_MASK_OFFSET); -} - static int starfive_aes_gcm_write_adata(struct starfive_cryp_ctx *ctx) { struct starfive_cryp_dev *cryp = ctx->cryp; @@ -451,60 +404,165 @@ static int starfive_aes_ccm_write_adata(struct starfive_cryp_ctx *ctx) return 0; } -static int starfive_aes_prepare_req(struct skcipher_request *req, - struct aead_request *areq) +static void starfive_aes_dma_done(void *param) { - struct starfive_cryp_ctx *ctx; - struct starfive_cryp_request_ctx *rctx; - struct starfive_cryp_dev *cryp; + struct starfive_cryp_dev *cryp = param; - if (!req && !areq) - return -EINVAL; + complete(&cryp->dma_done); +} - ctx = req ? crypto_skcipher_ctx(crypto_skcipher_reqtfm(req)) : - crypto_aead_ctx(crypto_aead_reqtfm(areq)); +static void starfive_aes_dma_init(struct starfive_cryp_dev *cryp) +{ + cryp->cfg_in.direction = DMA_MEM_TO_DEV; + cryp->cfg_in.src_addr_width = DMA_SLAVE_BUSWIDTH_16_BYTES; + cryp->cfg_in.dst_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES; + cryp->cfg_in.src_maxburst = cryp->dma_maxburst; + cryp->cfg_in.dst_maxburst = cryp->dma_maxburst; + cryp->cfg_in.dst_addr = cryp->phys_base + STARFIVE_ALG_FIFO_OFFSET; - cryp = ctx->cryp; - rctx = req ? skcipher_request_ctx(req) : aead_request_ctx(areq); + dmaengine_slave_config(cryp->tx, &cryp->cfg_in); - if (req) { - cryp->req.sreq = req; - cryp->total_in = req->cryptlen; - cryp->total_out = req->cryptlen; - cryp->assoclen = 0; - cryp->authsize = 0; - } else { - cryp->req.areq = areq; - cryp->assoclen = areq->assoclen; - cryp->authsize = crypto_aead_authsize(crypto_aead_reqtfm(areq)); - if (is_encrypt(cryp)) { - cryp->total_in = areq->cryptlen; - cryp->total_out = areq->cryptlen; - } else { - cryp->total_in = areq->cryptlen - cryp->authsize; - cryp->total_out = cryp->total_in; - } - } + cryp->cfg_out.direction = DMA_DEV_TO_MEM; + cryp->cfg_out.src_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES; + cryp->cfg_out.dst_addr_width = DMA_SLAVE_BUSWIDTH_16_BYTES; + cryp->cfg_out.src_maxburst = 4; + cryp->cfg_out.dst_maxburst = 4; + cryp->cfg_out.src_addr = cryp->phys_base + STARFIVE_ALG_FIFO_OFFSET; - rctx->in_sg = req ? req->src : areq->src; - scatterwalk_start(&cryp->in_walk, rctx->in_sg); + dmaengine_slave_config(cryp->rx, &cryp->cfg_out); - rctx->out_sg = req ? req->dst : areq->dst; - scatterwalk_start(&cryp->out_walk, rctx->out_sg); + init_completion(&cryp->dma_done); +} - if (cryp->assoclen) { - rctx->adata = kzalloc(cryp->assoclen + AES_BLOCK_SIZE, GFP_KERNEL); - if (!rctx->adata) - return dev_err_probe(cryp->dev, -ENOMEM, - "Failed to alloc memory for adata"); +static int starfive_aes_dma_xfer(struct starfive_cryp_dev *cryp, + struct scatterlist *src, + struct scatterlist *dst, + int len) +{ + struct dma_async_tx_descriptor *in_desc, *out_desc; + union starfive_alg_cr alg_cr; + int ret = 0, in_save, out_save; + + alg_cr.v = 0; + alg_cr.start = 1; + alg_cr.aes_dma_en = 1; + writel(alg_cr.v, cryp->base + STARFIVE_ALG_CR_OFFSET); - scatterwalk_copychunks(rctx->adata, &cryp->in_walk, cryp->assoclen, 0); - scatterwalk_copychunks(NULL, &cryp->out_walk, cryp->assoclen, 2); + in_save = sg_dma_len(src); + out_save = sg_dma_len(dst); + + writel(ALIGN(len, AES_BLOCK_SIZE), cryp->base + STARFIVE_DMA_IN_LEN_OFFSET); + writel(ALIGN(len, AES_BLOCK_SIZE), cryp->base + STARFIVE_DMA_OUT_LEN_OFFSET); + + sg_dma_len(src) = ALIGN(len, AES_BLOCK_SIZE); + sg_dma_len(dst) = ALIGN(len, AES_BLOCK_SIZE); + + out_desc = dmaengine_prep_slave_sg(cryp->rx, dst, 1, DMA_DEV_TO_MEM, + DMA_PREP_INTERRUPT | DMA_CTRL_ACK); + if (!out_desc) { + ret = -EINVAL; + goto dma_err; } - ctx->rctx = rctx; + out_desc->callback = starfive_aes_dma_done; + out_desc->callback_param = cryp; + + reinit_completion(&cryp->dma_done); + dmaengine_submit(out_desc); + dma_async_issue_pending(cryp->rx); + + in_desc = dmaengine_prep_slave_sg(cryp->tx, src, 1, DMA_MEM_TO_DEV, + DMA_PREP_INTERRUPT | DMA_CTRL_ACK); + if (!in_desc) { + ret = -EINVAL; + goto dma_err; + } + + dmaengine_submit(in_desc); + dma_async_issue_pending(cryp->tx); + + if (!wait_for_completion_timeout(&cryp->dma_done, + msecs_to_jiffies(1000))) + ret = -ETIMEDOUT; - return starfive_aes_hw_init(ctx); +dma_err: + sg_dma_len(src) = in_save; + sg_dma_len(dst) = out_save; + + alg_cr.v = 0; + alg_cr.clear = 1; + writel(alg_cr.v, cryp->base + STARFIVE_ALG_CR_OFFSET); + + return ret; +} + +static int starfive_aes_map_sg(struct starfive_cryp_dev *cryp, + struct scatterlist *src, + struct scatterlist *dst) +{ + struct scatterlist *stsg, *dtsg; + struct scatterlist _src[2], _dst[2]; + unsigned int remain = cryp->total_in; + unsigned int len, src_nents, dst_nents; + int ret; + + if (src == dst) { + for (stsg = src, dtsg = dst; remain > 0; + stsg = sg_next(stsg), dtsg = sg_next(dtsg)) { + src_nents = dma_map_sg(cryp->dev, stsg, 1, DMA_BIDIRECTIONAL); + if (src_nents == 0) + return dev_err_probe(cryp->dev, -ENOMEM, + "dma_map_sg error\n"); + + dst_nents = src_nents; + len = min(sg_dma_len(stsg), remain); + + ret = starfive_aes_dma_xfer(cryp, stsg, dtsg, len); + dma_unmap_sg(cryp->dev, stsg, 1, DMA_BIDIRECTIONAL); + if (ret) + return ret; + + remain -= len; + } + } else { + for (stsg = src, dtsg = dst;;) { + src_nents = dma_map_sg(cryp->dev, stsg, 1, DMA_TO_DEVICE); + if (src_nents == 0) + return dev_err_probe(cryp->dev, -ENOMEM, + "dma_map_sg src error\n"); + + dst_nents = dma_map_sg(cryp->dev, dtsg, 1, DMA_FROM_DEVICE); + if (dst_nents == 0) + return dev_err_probe(cryp->dev, -ENOMEM, + "dma_map_sg dst error\n"); + + len = min(sg_dma_len(stsg), sg_dma_len(dtsg)); + len = min(len, remain); + + ret = starfive_aes_dma_xfer(cryp, stsg, dtsg, len); + dma_unmap_sg(cryp->dev, stsg, 1, DMA_TO_DEVICE); + dma_unmap_sg(cryp->dev, dtsg, 1, DMA_FROM_DEVICE); + if (ret) + return ret; + + remain -= len; + if (remain == 0) + break; + + if (sg_dma_len(stsg) - len) { + stsg = scatterwalk_ffwd(_src, stsg, len); + dtsg = sg_next(dtsg); + } else if (sg_dma_len(dtsg) - len) { + dtsg = scatterwalk_ffwd(_dst, dtsg, len); + stsg = sg_next(stsg); + } else { + stsg = sg_next(stsg); + dtsg = sg_next(dtsg); + } + } + } + + return 0; } static int starfive_aes_do_one_req(struct crypto_engine *engine, void *areq) @@ -513,35 +571,38 @@ static int starfive_aes_do_one_req(struct crypto_engine *engine, void *areq) container_of(areq, struct skcipher_request, base); struct starfive_cryp_ctx *ctx = crypto_skcipher_ctx(crypto_skcipher_reqtfm(req)); + struct starfive_cryp_request_ctx *rctx = skcipher_request_ctx(req); struct starfive_cryp_dev *cryp = ctx->cryp; - u32 block[AES_BLOCK_32]; - u32 stat; - int err; - int i; + int ret; - err = starfive_aes_prepare_req(req, NULL); - if (err) - return err; + cryp->req.sreq = req; + cryp->total_in = req->cryptlen; + cryp->total_out = req->cryptlen; + cryp->assoclen = 0; + cryp->authsize = 0; - /* - * Write first plain/ciphertext block to start the module - * then let irq tasklet handle the rest of the data blocks. - */ - scatterwalk_copychunks(block, &cryp->in_walk, min_t(size_t, AES_BLOCK_SIZE, - cryp->total_in), 0); - cryp->total_in -= min_t(size_t, AES_BLOCK_SIZE, cryp->total_in); + rctx->in_sg = req->src; + rctx->out_sg = req->dst; + + ctx->rctx = rctx; + + ret = starfive_aes_hw_init(ctx); + if (ret) + return ret; - for (i = 0; i < AES_BLOCK_32; i++) - writel(block[i], cryp->base + STARFIVE_AES_AESDIO0R); + starfive_aes_dma_init(cryp); - stat = readl(cryp->base + STARFIVE_IE_MASK_OFFSET); - stat &= ~STARFIVE_IE_MASK_AES_DONE; - writel(stat, cryp->base + STARFIVE_IE_MASK_OFFSET); + ret = starfive_aes_map_sg(cryp, rctx->in_sg, rctx->out_sg); + if (ret) + return ret; + + starfive_aes_finish_req(ctx); return 0; } -static int starfive_aes_init_tfm(struct crypto_skcipher *tfm) +static int starfive_aes_init_tfm(struct crypto_skcipher *tfm, + const char *alg_name) { struct starfive_cryp_ctx *ctx = crypto_skcipher_ctx(tfm); @@ -549,12 +610,26 @@ static int starfive_aes_init_tfm(struct crypto_skcipher *tfm) if (!ctx->cryp) return -ENODEV; + ctx->skcipher_fbk = crypto_alloc_skcipher(alg_name, 0, + CRYPTO_ALG_NEED_FALLBACK); + if (IS_ERR(ctx->skcipher_fbk)) + return dev_err_probe(ctx->cryp->dev, PTR_ERR(ctx->skcipher_fbk), + "%s() failed to allocate fallback for %s\n", + __func__, alg_name); + crypto_skcipher_set_reqsize(tfm, sizeof(struct starfive_cryp_request_ctx) + - sizeof(struct skcipher_request)); + crypto_skcipher_reqsize(ctx->skcipher_fbk)); return 0; } +static void starfive_aes_exit_tfm(struct crypto_skcipher *tfm) +{ + struct starfive_cryp_ctx *ctx = crypto_skcipher_ctx(tfm); + + crypto_free_skcipher(ctx->skcipher_fbk); +} + static int starfive_aes_aead_do_one_req(struct crypto_engine *engine, void *areq) { struct aead_request *req = @@ -562,79 +637,99 @@ static int starfive_aes_aead_do_one_req(struct crypto_engine *engine, void *areq struct starfive_cryp_ctx *ctx = crypto_aead_ctx(crypto_aead_reqtfm(req)); struct starfive_cryp_dev *cryp = ctx->cryp; - struct starfive_cryp_request_ctx *rctx; - u32 block[AES_BLOCK_32]; - u32 stat; - int err; - int i; + struct starfive_cryp_request_ctx *rctx = aead_request_ctx(req); + struct scatterlist _src[2], _dst[2]; + int ret; + + cryp->req.areq = req; + cryp->assoclen = req->assoclen; + cryp->authsize = crypto_aead_authsize(crypto_aead_reqtfm(req)); + + rctx->in_sg = scatterwalk_ffwd(_src, req->src, cryp->assoclen); + if (req->src == req->dst) + rctx->out_sg = rctx->in_sg; + else + rctx->out_sg = scatterwalk_ffwd(_dst, req->dst, cryp->assoclen); - err = starfive_aes_prepare_req(NULL, req); - if (err) - return err; + if (is_encrypt(cryp)) { + cryp->total_in = req->cryptlen; + cryp->total_out = req->cryptlen; + } else { + cryp->total_in = req->cryptlen - cryp->authsize; + cryp->total_out = cryp->total_in; + scatterwalk_map_and_copy(cryp->tag_in, req->src, + cryp->total_in + cryp->assoclen, + cryp->authsize, 0); + } - rctx = ctx->rctx; + if (cryp->assoclen) { + rctx->adata = kzalloc(cryp->assoclen + AES_BLOCK_SIZE, GFP_KERNEL); + if (!rctx->adata) + return dev_err_probe(cryp->dev, -ENOMEM, + "Failed to alloc memory for adata"); + + if (sg_copy_to_buffer(req->src, sg_nents_for_len(req->src, cryp->assoclen), + rctx->adata, cryp->assoclen) != cryp->assoclen) + return -EINVAL; + } + + if (cryp->total_in) + sg_zero_buffer(rctx->in_sg, sg_nents(rctx->in_sg), + sg_dma_len(rctx->in_sg) - cryp->total_in, + cryp->total_in); + + ctx->rctx = rctx; + + ret = starfive_aes_hw_init(ctx); + if (ret) + return ret; if (!cryp->assoclen) goto write_text; if ((cryp->flags & FLG_MODE_MASK) == STARFIVE_AES_MODE_CCM) - cryp->err = starfive_aes_ccm_write_adata(ctx); + ret = starfive_aes_ccm_write_adata(ctx); else - cryp->err = starfive_aes_gcm_write_adata(ctx); + ret = starfive_aes_gcm_write_adata(ctx); kfree(rctx->adata); - if (cryp->err) - return cryp->err; + if (ret) + return ret; write_text: if (!cryp->total_in) goto finish_req; - /* - * Write first plain/ciphertext block to start the module - * then let irq tasklet handle the rest of the data blocks. - */ - scatterwalk_copychunks(block, &cryp->in_walk, min_t(size_t, AES_BLOCK_SIZE, - cryp->total_in), 0); - cryp->total_in -= min_t(size_t, AES_BLOCK_SIZE, cryp->total_in); - - for (i = 0; i < AES_BLOCK_32; i++) - writel(block[i], cryp->base + STARFIVE_AES_AESDIO0R); + starfive_aes_dma_init(cryp); - stat = readl(cryp->base + STARFIVE_IE_MASK_OFFSET); - stat &= ~STARFIVE_IE_MASK_AES_DONE; - writel(stat, cryp->base + STARFIVE_IE_MASK_OFFSET); - - return 0; + ret = starfive_aes_map_sg(cryp, rctx->in_sg, rctx->out_sg); + if (ret) + return ret; finish_req: - starfive_aes_finish_req(cryp); + starfive_aes_finish_req(ctx); return 0; } -static int starfive_aes_aead_init_tfm(struct crypto_aead *tfm) +static int starfive_aes_aead_init_tfm(struct crypto_aead *tfm, + const char *alg_name) { struct starfive_cryp_ctx *ctx = crypto_aead_ctx(tfm); - struct starfive_cryp_dev *cryp = ctx->cryp; - struct crypto_tfm *aead = crypto_aead_tfm(tfm); - struct crypto_alg *alg = aead->__crt_alg; ctx->cryp = starfive_cryp_find_dev(ctx); if (!ctx->cryp) return -ENODEV; - if (alg->cra_flags & CRYPTO_ALG_NEED_FALLBACK) { - ctx->aead_fbk = crypto_alloc_aead(alg->cra_name, 0, - CRYPTO_ALG_NEED_FALLBACK); - if (IS_ERR(ctx->aead_fbk)) - return dev_err_probe(cryp->dev, PTR_ERR(ctx->aead_fbk), - "%s() failed to allocate fallback for %s\n", - __func__, alg->cra_name); - } + ctx->aead_fbk = crypto_alloc_aead(alg_name, 0, + CRYPTO_ALG_NEED_FALLBACK); + if (IS_ERR(ctx->aead_fbk)) + return dev_err_probe(ctx->cryp->dev, PTR_ERR(ctx->aead_fbk), + "%s() failed to allocate fallback for %s\n", + __func__, alg_name); - crypto_aead_set_reqsize(tfm, sizeof(struct starfive_cryp_ctx) + - sizeof(struct aead_request)); + crypto_aead_set_reqsize(tfm, sizeof(struct starfive_cryp_request_ctx) + + crypto_aead_reqsize(ctx->aead_fbk)); return 0; } @@ -646,6 +741,44 @@ static void starfive_aes_aead_exit_tfm(struct crypto_aead *tfm) crypto_free_aead(ctx->aead_fbk); } +static bool starfive_aes_check_unaligned(struct starfive_cryp_dev *cryp, + struct scatterlist *src, + struct scatterlist *dst) +{ + struct scatterlist *tsg; + int i; + + for_each_sg(src, tsg, sg_nents(src), i) + if (!IS_ALIGNED(tsg->length, AES_BLOCK_SIZE) && + !sg_is_last(tsg)) + return true; + + if (src != dst) + for_each_sg(dst, tsg, sg_nents(dst), i) + if (!IS_ALIGNED(tsg->length, AES_BLOCK_SIZE) && + !sg_is_last(tsg)) + return true; + + return false; +} + +static int starfive_aes_do_fallback(struct skcipher_request *req, bool enc) +{ + struct starfive_cryp_ctx *ctx = + crypto_skcipher_ctx(crypto_skcipher_reqtfm(req)); + struct skcipher_request *subreq = skcipher_request_ctx(req); + + skcipher_request_set_tfm(subreq, ctx->skcipher_fbk); + skcipher_request_set_callback(subreq, req->base.flags, + req->base.complete, + req->base.data); + skcipher_request_set_crypt(subreq, req->src, req->dst, + req->cryptlen, req->iv); + + return enc ? crypto_skcipher_encrypt(subreq) : + crypto_skcipher_decrypt(subreq); +} + static int starfive_aes_crypt(struct skcipher_request *req, unsigned long flags) { struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req); @@ -660,32 +793,54 @@ static int starfive_aes_crypt(struct skcipher_request *req, unsigned long flags) if (req->cryptlen & blocksize_align) return -EINVAL; + if (starfive_aes_check_unaligned(cryp, req->src, req->dst)) + return starfive_aes_do_fallback(req, is_encrypt(cryp)); + return crypto_transfer_skcipher_request_to_engine(cryp->engine, req); } +static int starfive_aes_aead_do_fallback(struct aead_request *req, bool enc) +{ + struct starfive_cryp_ctx *ctx = + crypto_aead_ctx(crypto_aead_reqtfm(req)); + struct aead_request *subreq = aead_request_ctx(req); + + aead_request_set_tfm(subreq, ctx->aead_fbk); + aead_request_set_callback(subreq, req->base.flags, + req->base.complete, + req->base.data); + aead_request_set_crypt(subreq, req->src, req->dst, + req->cryptlen, req->iv); + aead_request_set_ad(subreq, req->assoclen); + + return enc ? crypto_aead_encrypt(subreq) : + crypto_aead_decrypt(subreq); +} + static int starfive_aes_aead_crypt(struct aead_request *req, unsigned long flags) { struct starfive_cryp_ctx *ctx = crypto_aead_ctx(crypto_aead_reqtfm(req)); struct starfive_cryp_dev *cryp = ctx->cryp; + struct scatterlist *src, *dst, _src[2], _dst[2]; cryp->flags = flags; - /* - * HW engine could not perform CCM tag verification on - * non-blocksize aligned text, use fallback algo instead + /* aes-ccm does not support tag verification for non-aligned text, + * use fallback for ccm decryption instead. */ - if (ctx->aead_fbk && !is_encrypt(cryp)) { - struct aead_request *subreq = aead_request_ctx(req); + if (((cryp->flags & FLG_MODE_MASK) == STARFIVE_AES_MODE_CCM) && + !is_encrypt(cryp)) + return starfive_aes_aead_do_fallback(req, 0); - aead_request_set_tfm(subreq, ctx->aead_fbk); - aead_request_set_callback(subreq, req->base.flags, - req->base.complete, req->base.data); - aead_request_set_crypt(subreq, req->src, - req->dst, req->cryptlen, req->iv); - aead_request_set_ad(subreq, req->assoclen); + src = scatterwalk_ffwd(_src, req->src, req->assoclen); - return crypto_aead_decrypt(subreq); - } + if (req->src == req->dst) + dst = src; + else + dst = scatterwalk_ffwd(_dst, req->dst, req->assoclen); + + if (starfive_aes_check_unaligned(cryp, src, dst)) + return starfive_aes_aead_do_fallback(req, is_encrypt(cryp)); return crypto_transfer_aead_request_to_engine(cryp->engine, req); } @@ -706,7 +861,7 @@ static int starfive_aes_setkey(struct crypto_skcipher *tfm, const u8 *key, memcpy(ctx->key, key, keylen); ctx->keylen = keylen; - return 0; + return crypto_skcipher_setkey(ctx->skcipher_fbk, key, keylen); } static int starfive_aes_aead_setkey(struct crypto_aead *tfm, const u8 *key, @@ -725,16 +880,20 @@ static int starfive_aes_aead_setkey(struct crypto_aead *tfm, const u8 *key, memcpy(ctx->key, key, keylen); ctx->keylen = keylen; - if (ctx->aead_fbk) - return crypto_aead_setkey(ctx->aead_fbk, key, keylen); - - return 0; + return crypto_aead_setkey(ctx->aead_fbk, key, keylen); } static int starfive_aes_gcm_setauthsize(struct crypto_aead *tfm, unsigned int authsize) { - return crypto_gcm_check_authsize(authsize); + struct starfive_cryp_ctx *ctx = crypto_aead_ctx(tfm); + int ret; + + ret = crypto_gcm_check_authsize(authsize); + if (ret) + return ret; + + return crypto_aead_setauthsize(ctx->aead_fbk, authsize); } static int starfive_aes_ccm_setauthsize(struct crypto_aead *tfm, @@ -820,9 +979,35 @@ static int starfive_aes_ccm_decrypt(struct aead_request *req) return starfive_aes_aead_crypt(req, STARFIVE_AES_MODE_CCM); } +static int starfive_aes_ecb_init_tfm(struct crypto_skcipher *tfm) +{ + return starfive_aes_init_tfm(tfm, "ecb(aes-generic)"); +} + +static int starfive_aes_cbc_init_tfm(struct crypto_skcipher *tfm) +{ + return starfive_aes_init_tfm(tfm, "cbc(aes-generic)"); +} + +static int starfive_aes_ctr_init_tfm(struct crypto_skcipher *tfm) +{ + return starfive_aes_init_tfm(tfm, "ctr(aes-generic)"); +} + +static int starfive_aes_ccm_init_tfm(struct crypto_aead *tfm) +{ + return starfive_aes_aead_init_tfm(tfm, "ccm_base(ctr(aes-generic),cbcmac(aes-generic))"); +} + +static int starfive_aes_gcm_init_tfm(struct crypto_aead *tfm) +{ + return starfive_aes_aead_init_tfm(tfm, "gcm_base(ctr(aes-generic),ghash-generic)"); +} + static struct skcipher_engine_alg skcipher_algs[] = { { - .base.init = starfive_aes_init_tfm, + .base.init = starfive_aes_ecb_init_tfm, + .base.exit = starfive_aes_exit_tfm, .base.setkey = starfive_aes_setkey, .base.encrypt = starfive_aes_ecb_encrypt, .base.decrypt = starfive_aes_ecb_decrypt, @@ -832,7 +1017,8 @@ static struct skcipher_engine_alg skcipher_algs[] = { .cra_name = "ecb(aes)", .cra_driver_name = "starfive-ecb-aes", .cra_priority = 200, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags = CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct starfive_cryp_ctx), .cra_alignmask = 0xf, @@ -842,7 +1028,8 @@ static struct skcipher_engine_alg skcipher_algs[] = { .do_one_request = starfive_aes_do_one_req, }, }, { - .base.init = starfive_aes_init_tfm, + .base.init = starfive_aes_cbc_init_tfm, + .base.exit = starfive_aes_exit_tfm, .base.setkey = starfive_aes_setkey, .base.encrypt = starfive_aes_cbc_encrypt, .base.decrypt = starfive_aes_cbc_decrypt, @@ -853,7 +1040,8 @@ static struct skcipher_engine_alg skcipher_algs[] = { .cra_name = "cbc(aes)", .cra_driver_name = "starfive-cbc-aes", .cra_priority = 200, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags = CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct starfive_cryp_ctx), .cra_alignmask = 0xf, @@ -863,7 +1051,8 @@ static struct skcipher_engine_alg skcipher_algs[] = { .do_one_request = starfive_aes_do_one_req, }, }, { - .base.init = starfive_aes_init_tfm, + .base.init = starfive_aes_ctr_init_tfm, + .base.exit = starfive_aes_exit_tfm, .base.setkey = starfive_aes_setkey, .base.encrypt = starfive_aes_ctr_encrypt, .base.decrypt = starfive_aes_ctr_decrypt, @@ -874,7 +1063,8 @@ static struct skcipher_engine_alg skcipher_algs[] = { .cra_name = "ctr(aes)", .cra_driver_name = "starfive-ctr-aes", .cra_priority = 200, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags = CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct starfive_cryp_ctx), .cra_alignmask = 0xf, @@ -892,7 +1082,7 @@ static struct aead_engine_alg aead_algs[] = { .base.setauthsize = starfive_aes_gcm_setauthsize, .base.encrypt = starfive_aes_gcm_encrypt, .base.decrypt = starfive_aes_gcm_decrypt, - .base.init = starfive_aes_aead_init_tfm, + .base.init = starfive_aes_gcm_init_tfm, .base.exit = starfive_aes_aead_exit_tfm, .base.ivsize = GCM_AES_IV_SIZE, .base.maxauthsize = AES_BLOCK_SIZE, @@ -900,7 +1090,8 @@ static struct aead_engine_alg aead_algs[] = { .cra_name = "gcm(aes)", .cra_driver_name = "starfive-gcm-aes", .cra_priority = 200, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags = CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct starfive_cryp_ctx), .cra_alignmask = 0xf, @@ -914,7 +1105,7 @@ static struct aead_engine_alg aead_algs[] = { .base.setauthsize = starfive_aes_ccm_setauthsize, .base.encrypt = starfive_aes_ccm_encrypt, .base.decrypt = starfive_aes_ccm_decrypt, - .base.init = starfive_aes_aead_init_tfm, + .base.init = starfive_aes_ccm_init_tfm, .base.exit = starfive_aes_aead_exit_tfm, .base.ivsize = AES_BLOCK_SIZE, .base.maxauthsize = AES_BLOCK_SIZE, diff --git a/drivers/crypto/starfive/jh7110-cryp.c b/drivers/crypto/starfive/jh7110-cryp.c index 2685f5483639..cc4139a88a0c 100644 --- a/drivers/crypto/starfive/jh7110-cryp.c +++ b/drivers/crypto/starfive/jh7110-cryp.c @@ -89,28 +89,10 @@ static void starfive_dma_cleanup(struct starfive_cryp_dev *cryp) dma_release_channel(cryp->rx); } -static irqreturn_t starfive_cryp_irq(int irq, void *priv) -{ - u32 status; - u32 mask; - struct starfive_cryp_dev *cryp = (struct starfive_cryp_dev *)priv; - - mask = readl(cryp->base + STARFIVE_IE_MASK_OFFSET); - status = readl(cryp->base + STARFIVE_IE_FLAG_OFFSET); - if (status & STARFIVE_IE_FLAG_AES_DONE) { - mask |= STARFIVE_IE_MASK_AES_DONE; - writel(mask, cryp->base + STARFIVE_IE_MASK_OFFSET); - tasklet_schedule(&cryp->aes_done); - } - - return IRQ_HANDLED; -} - static int starfive_cryp_probe(struct platform_device *pdev) { struct starfive_cryp_dev *cryp; struct resource *res; - int irq; int ret; cryp = devm_kzalloc(&pdev->dev, sizeof(*cryp), GFP_KERNEL); @@ -125,8 +107,6 @@ static int starfive_cryp_probe(struct platform_device *pdev) return dev_err_probe(&pdev->dev, PTR_ERR(cryp->base), "Error remapping memory for platform device\n"); - tasklet_init(&cryp->aes_done, starfive_aes_done_task, (unsigned long)cryp); - cryp->phys_base = res->start; cryp->dma_maxburst = 32; cryp->side_chan = side_chan; @@ -212,8 +192,6 @@ static int starfive_cryp_probe(struct platform_device *pdev) clk_disable_unprepare(cryp->ahb); reset_control_assert(cryp->rst); - tasklet_kill(&cryp->aes_done); - return ret; } @@ -225,8 +203,6 @@ static void starfive_cryp_remove(struct platform_device *pdev) starfive_hash_unregister_algs(); starfive_rsa_unregister_algs(); - tasklet_kill(&cryp->aes_done); - crypto_engine_stop(cryp->engine); crypto_engine_exit(cryp->engine); diff --git a/drivers/crypto/starfive/jh7110-cryp.h b/drivers/crypto/starfive/jh7110-cryp.h index 60cc269a0f28..6e523e45cd9f 100644 --- a/drivers/crypto/starfive/jh7110-cryp.h +++ b/drivers/crypto/starfive/jh7110-cryp.h @@ -168,6 +168,7 @@ struct starfive_cryp_ctx { struct crypto_akcipher *akcipher_fbk; struct crypto_ahash *ahash_fbk; struct crypto_aead *aead_fbk; + struct crypto_skcipher *skcipher_fbk; }; struct starfive_cryp_dev { @@ -185,10 +186,7 @@ struct starfive_cryp_dev { struct dma_chan *rx; struct dma_slave_config cfg_in; struct dma_slave_config cfg_out; - struct scatter_walk in_walk; - struct scatter_walk out_walk; struct crypto_engine *engine; - struct tasklet_struct aes_done; struct completion dma_done; size_t assoclen; size_t total_in; @@ -236,6 +234,4 @@ void starfive_rsa_unregister_algs(void); int starfive_aes_register_algs(void); void starfive_aes_unregister_algs(void); - -void starfive_aes_done_task(unsigned long param); #endif From patchwork Tue Feb 27 16:37:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jia Jie Ho X-Patchwork-Id: 776347 Received: from CHN02-BJS-obe.outbound.protection.partner.outlook.cn (mail-bjschn02on2095.outbound.protection.partner.outlook.cn [139.219.17.95]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDAEC4EB3B; Tue, 27 Feb 2024 16:38:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=139.219.17.95 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709051909; cv=fail; b=NlKGLXRkh0Kxn9Q9IHdknsVWQKALcSNGZ8dMNcEYiEXpldX9v/y5b5v8oL+TJLtm6q89zgtBglEU/rtmyrzzwnuk7V4GtuTkgRVURZEP5BvYvcpoJTEpGLPTAGJpWgK8Ny8M+RY9Ds0NxKK7WVs8FY7gt8aRnJDnUwe5vC5eODg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709051909; c=relaxed/simple; bh=a2kSj/N41fu3wtttz6tKElnDO/7+BZI876DcZw6HYNg=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=j3lkx2+cGmPSKiPX+GpNuc6aFUxKovzSj2sQ3W0xtfkVGTrA7w06AM4eh+gpSKtiCLXfvkFQtrqTAalc8Tu7bsiPj1pmrs2qK6lNEj5VF0VmjTp448zp52EFHFl7urfdmrWUgnUPfvZ01jIBDK/ceVJV+1CIuhlgY/e3i/kfOOw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=starfivetech.com; spf=pass smtp.mailfrom=starfivetech.com; arc=fail smtp.client-ip=139.219.17.95 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=starfivetech.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=starfivetech.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=O3he/M31EFG5rbnDg1DtvpVFq+tQxkIm24hYNYFdsVBbdtbww4ArCF0O5hqhRLYmqEaNSYEV0quwRWvdWb1sMgUqNPUJHkqUZ4ce4nqA/LbGGQUzY1XGK0YoZJmgA5B4/3HY7aFGiAG3cU92i+TrA17bREeT3fsgAkILGlNqJMdrfC2pGTDCdDQ5ngLnDgq7C4FRYztuqZ02Tpy9+3rcf7zj1uH4RstVN3rwbOZ0ocXMF+jNw6Chwc7JlpRpnTPBQFnxMsCD1u3utRaK9X873rgGzpDXEOL9c57QFaGuAt/skROW6Fvu0Z+A7GaVSn2u6Yvz6Yd25Dk7+hsqdZHkHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yDo+XQFzr6zgQmUdXaMtdgTpGc8h6PBylFQde99iOXQ=; b=KhRMPLR/E0Sj/Y4KtpFUbdRwRlFpc9hrqnU1OESIiM7czi9db0D4OJnaodJZ2i3axfvJsQEHXojE7ia/WasZaat/8DMeNVMsOdonxxnfXAaKxhRM9SKoygYon4wCftIzR7WoVePoB2rY1arFda3hTiR7eRD3pLalw8K8At0AMsPKhR+YKNL22bLD+XUIqyumi/hEszGRw0FFy9Xp+HzBW8K+qKDn9nmgRbMLHjdZiMjyeuaK1khzh6SPK4HTrb0x1dnWdj+za90ff+2lzzaLhauRhV3NZW36qtWxoZ9YOzhVn7R18qZhjMOE2jfzSHnJ2PS0tsm62C5dwPQxSPBBHw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=starfivetech.com; dmarc=pass action=none header.from=starfivetech.com; dkim=pass header.d=starfivetech.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=starfivetech.com; Received: from SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn (2406:e500:c311:26::16) by SHXPR01MB0685.CHNPR01.prod.partner.outlook.cn (2406:e500:c311:27::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.42; Tue, 27 Feb 2024 16:38:13 +0000 Received: from SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn ([fe80::9ffd:1956:b45a:4a5d]) by SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn ([fe80::9ffd:1956:b45a:4a5d%6]) with mapi id 15.20.7249.041; Tue, 27 Feb 2024 16:38:13 +0000 From: Jia Jie Ho To: Herbert Xu , "David S . Miller" , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Eugeniy Paltsev , Vinod Koul , linux-crypto@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, dmaengine@vger.kernel.org Subject: [PATCH v3 5/6] crypto: starfive: Add sm3 support for JH8100 Date: Wed, 28 Feb 2024 00:37:57 +0800 Message-Id: <20240227163758.198133-6-jiajie.ho@starfivetech.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240227163758.198133-1-jiajie.ho@starfivetech.com> References: <20240227163758.198133-1-jiajie.ho@starfivetech.com> X-ClientProxiedBy: NT0PR01CA0014.CHNPR01.prod.partner.outlook.cn (2406:e500:c510::16) To SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn (2406:e500:c311:26::16) Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SHXPR01MB0670:EE_|SHXPR01MB0685:EE_ X-MS-Office365-Filtering-Correlation-Id: 3a423db7-fe19-4e41-9b82-08dc37b27e0c X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: e8xiqWmS8Qs9yJSqgvEOA5BInNLCFTVA6wviGKV+7dsSEH0UaDoHQpNw1bBA10Xg/1XWqbvsH0I17w4epe1HYFhv5qMmUfQoKKHP67jvI+CYKgPuljecxZ9/oDWGy4OH9uCD3Ers5tB7x463kkgs30GOHmoFrmUmcQB/CdujevZfOFieNnChAVHl0NnrN1/b+Fae2GxbHSskkQwa1UEtKO3+fjXbAxxgLBKkgtBUOwQkFH8mOqSutYpuBWfAy2DSWoC++8cxPLFGjsm/Ao3h8ADFY/eL4bu6Up2U20+Azb0aKXkuugMlrIlmxLOe30xZqRxdUgzYSuh/jkMcRMiNJ6d07T3wq4H9IF/lUNG2+/0av+pxwcDL2FDKmviKzg5axeuaehjxce1xQgOhnST0Xqc76saKqp7ZnIrmml4NEMCKInqe+xJRgoN3X62a4R5p846kIAAsn86gJOzDjHxhj26Hnvi9xxOr9RexB6LtqkS/Sy4bK8PEQIcc6hN+5yPEcsE7r+PoHsU04FOYd2IXZLYjQtL5nKpP9iiC7T7gCxfVv1un1sB2GWKDLs1lc4RaSyesYhwOUJbgyfnQzMk2r9X3oeTeYJQJWu3JMfF39EcS7PFQrs+haUFMKJ/zZkRXXtr4MyFOShhvR7IR43vitA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn; PTR:; CAT:NONE; SFS:(13230031)(38350700005)(921011); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: +ZliB5Q0nVBM/qlFktDDAzJHCiEAva72CqjZ/PRuw9Mvb4JavcusDfQ2Qnvw6vjJCLZu6GStzdR6kCyJBsecxBFs+S+XLDk2a4IBI/TqwioY4AjivxMRW5TsVXZgTYAL79ZrXNXxlX3dtGN/toRasVpYmBeZCk7j8AhXg9ntsxuAfsOlw23i0zcPBhz0CumMmRW46GkwO13e0qlkSK1ua8lJ6feooGKyjqYxj4/BtYQKkkVvG1AfvVJ5MEruxuXi3zRZbiZ5fouJlF9yOFfCVSejTL4JsDgU1yC+F+A6yoDm/9iNjsMVF1ZH3Uov4n84SigHBVN50ZwSFYRjbsnUmEUtvEdJsZSeebuV2XMfEeM3iV4vZtuT6pmRRYJ9UjLOOk0sJMxeUVvg1j/Ke2zlrtVHCu3JXF+L3lzSKijNon5f9A9ltcBGFQOh8B7BxV6l4yAmsY4ytdg2Ey3RrYFChWApIQqvrsIMlEy8Watmw59YNjkJ77s6pVYJd/+Y974abZAh+Rs4ua7j2zSiXTMAJx/kheY9iFGBrzKgLoMxGwzwrSI6RjP7sihtW/ThgpSyJCkkK3yx2yzABkWufUL0pcneSxg4MYgGid/DfUhWe4UWfcFLkU9ZgQ5zaun9URqgaPyuAxrWzige5QrhB2nYgMOnCf50YJ0H6yW4SSunWdzYUKycFHMnWIx68ytl0wEusjyIBx8A6wPxARosHJw9EWSFX7bkdPgDYAejVKBXw2yPkmp78fAUEVmvGlZlEIgoNDEwewcwH0fiIvm7CvPJDx1qEgFF/HpkVeDWhBQ8y0LPRH7J4ogNsbIm7dSc6xzrCmmQDttehLyPBs853ezRZGmVAhSbcd5yCt5TO7weS9D5c/niGoblaDsX3PseMdVRY3arCH59YvopIDX3x5wrfbeyaZmaEzmHWr8sWYx/RYZKdOEs01koR4FdWjUBBxJFsR17iryxJPljOsqgSriFAzvXmlIFa72RpblueTEsq2xElBqiV2eGPKHt8pnPehmUAIPUHgY6Sia8KpRLscqF4PfaZPCXl31Oq9deZoBnIVp3Ga3XcdHvIbHispdwu3uHdVK0xgtfrk8kwJfUqDjtAeTuATHtuk0Pfkwds015v89lIuQ+qG3wUkG58mkRf4/OO+kJ0Hp61cUkZIWcM+QRk9t26umXAH0dbI7jkJx+WfHkbbuhSyAV+GXvCjZRvEaCSiB1rEsTPV77tBYc21C66UXNeFy2nztq4cvEBiPT2hdrqm8W/0i7iAFwgeLrbOnRfIJQIwlFF70KYqGqaL44I3aleteXhXYEDJlv5S/tePEnOU9YlHo38MCUuPRsN5Q+PW5MxqgYnxBkjkb0+OtHFU9wtNXJ0YR9q1b3GnNZZpl3mpYYrQlkiVtHrwE1ObIgcSZrV5zGypHRLUy0NpTNZXlGskGqr/i+MPP7kmqPZ8WqhbhlR25RCEy6IlEH01m9XKzcyieiK3axc0ZWq3oCdMpWV9N50xD/zGWPTbT9DuVKiyo6ukFmUDF7Xii85Z1jHMxmtH0dSDYntYBvRr9VBkFqOZlusKKFAEOQ0C8wDoNLGu3ZunrgMsTGMW1mbTKPAylAwEWIWhxLQIhxV9nGPQ== X-OriginatorOrg: starfivetech.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3a423db7-fe19-4e41-9b82-08dc37b27e0c X-MS-Exchange-CrossTenant-AuthSource: SHXPR01MB0670.CHNPR01.prod.partner.outlook.cn X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2024 16:38:13.9173 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 06fe3fa3-1221-43d3-861b-5a4ee687a85c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: C4khyjPI4fWIB0nzqk1KBHzrvmVeCdzJD3x89y6wukywuZhnubaH+fkQT/x5tmiUKMm+agwXJe1Tv7uUmAyS5RpeqQmS0/Q10nH8dSssxm0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SHXPR01MB0685 Add driver support for SM3 hash/HMAC for JH8100 SoC. JH8100 contains a separate SM algo engine and new dedicated dma that supports 64-bit address access. Signed-off-by: Jia Jie Ho --- drivers/crypto/starfive/Kconfig | 25 +- drivers/crypto/starfive/Makefile | 5 +- drivers/crypto/starfive/jh7110-aes.c | 3 + drivers/crypto/starfive/jh7110-cryp.c | 38 +- drivers/crypto/starfive/jh7110-cryp.h | 66 +++- drivers/crypto/starfive/jh7110-hash.c | 45 ++- drivers/crypto/starfive/jh8100-sm3.c | 535 ++++++++++++++++++++++++++ 7 files changed, 686 insertions(+), 31 deletions(-) create mode 100644 drivers/crypto/starfive/jh8100-sm3.c diff --git a/drivers/crypto/starfive/Kconfig b/drivers/crypto/starfive/Kconfig index 0fe389e9f932..e6bf02d0ed1f 100644 --- a/drivers/crypto/starfive/Kconfig +++ b/drivers/crypto/starfive/Kconfig @@ -5,7 +5,7 @@ config CRYPTO_DEV_JH7110 tristate "StarFive JH7110 cryptographic engine driver" depends on (SOC_STARFIVE && AMBA_PL08X) || COMPILE_TEST - depends on HAS_DMA + depends on HAS_DMA && !CRYPTO_DEV_JH8100 select CRYPTO_ENGINE select CRYPTO_HMAC select CRYPTO_SHA256 @@ -24,3 +24,26 @@ config CRYPTO_DEV_JH7110 skciphers, AEAD and hash functions. If you choose 'M' here, this module will be called jh7110-crypto. + +config CRYPTO_DEV_JH8100 + tristate "StarFive JH8100 cryptographic engine drivers" + depends on (SOC_STARFIVE && DW_AXI_DMAC) || COMPILE_TEST + depends on HAS_DMA + select CRYPTO_ENGINE + select CRYPTO_HMAC + select CRYPTO_SHA256 + select CRYPTO_SHA512 + select CRYPTO_SM3_GENERIC + select CRYPTO_RSA + select CRYPTO_AES + select CRYPTO_CCM + select CRYPTO_GCM + select CRYPTO_CBC + select CRYPTO_ECB + select CRYPTO_CTR + help + Support for StarFive JH8100 crypto hardware acceleration engine. + This module provides additional support for SM2 signature verification, + SM3 hash/hmac functions and SM4 skcipher. + + If you choose 'M' here, this module will be called jh8100-crypto. diff --git a/drivers/crypto/starfive/Makefile b/drivers/crypto/starfive/Makefile index 8c137afe58ad..867ce035af19 100644 --- a/drivers/crypto/starfive/Makefile +++ b/drivers/crypto/starfive/Makefile @@ -1,4 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_CRYPTO_DEV_JH7110) += jh7110-crypto.o -jh7110-crypto-objs := jh7110-cryp.o jh7110-hash.o jh7110-rsa.o jh7110-aes.o +jh7110-crypto-objs := jh7110-cryp.o jh7110-hash.o jh7110-rsa.o jh7110-aes.o jh8100-sm3.o + +obj-$(CONFIG_CRYPTO_DEV_JH8100) += jh8100-crypto.o +jh8100-crypto-objs := jh7110-cryp.o jh7110-hash.o jh7110-rsa.o jh7110-aes.o jh8100-sm3.o diff --git a/drivers/crypto/starfive/jh7110-aes.c b/drivers/crypto/starfive/jh7110-aes.c index 72b7d46150d5..3ee782b6c028 100644 --- a/drivers/crypto/starfive/jh7110-aes.c +++ b/drivers/crypto/starfive/jh7110-aes.c @@ -413,6 +413,9 @@ static void starfive_aes_dma_done(void *param) static void starfive_aes_dma_init(struct starfive_cryp_dev *cryp) { + memset(&cryp->cfg_in, 0, sizeof(struct dma_slave_config)); + memset(&cryp->cfg_out, 0, sizeof(struct dma_slave_config)); + cryp->cfg_in.direction = DMA_MEM_TO_DEV; cryp->cfg_in.src_addr_width = DMA_SLAVE_BUSWIDTH_16_BYTES; cryp->cfg_in.dst_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES; diff --git a/drivers/crypto/starfive/jh7110-cryp.c b/drivers/crypto/starfive/jh7110-cryp.c index cc4139a88a0c..19bbcaaec18d 100644 --- a/drivers/crypto/starfive/jh7110-cryp.c +++ b/drivers/crypto/starfive/jh7110-cryp.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -99,6 +100,8 @@ static int starfive_cryp_probe(struct platform_device *pdev) if (!cryp) return -ENOMEM; + cryp->type = (uintptr_t)of_device_get_match_data(&pdev->dev); + platform_set_drvdata(pdev, cryp); cryp->dev = &pdev->dev; @@ -126,16 +129,6 @@ static int starfive_cryp_probe(struct platform_device *pdev) return dev_err_probe(&pdev->dev, PTR_ERR(cryp->rst), "Error getting hardware reset line\n"); - irq = platform_get_irq(pdev, 0); - if (irq < 0) - return irq; - - ret = devm_request_irq(&pdev->dev, irq, starfive_cryp_irq, 0, pdev->name, - (void *)cryp); - if (ret) - return dev_err_probe(&pdev->dev, ret, - "Failed to register interrupt handler\n"); - clk_prepare_enable(cryp->hclk); clk_prepare_enable(cryp->ahb); reset_control_deassert(cryp->rst); @@ -163,7 +156,7 @@ static int starfive_cryp_probe(struct platform_device *pdev) if (ret) goto err_algs_aes; - ret = starfive_hash_register_algs(); + ret = starfive_hash_register_algs(cryp); if (ret) goto err_algs_hash; @@ -171,10 +164,18 @@ static int starfive_cryp_probe(struct platform_device *pdev) if (ret) goto err_algs_rsa; + if (cryp->type == STARFIVE_CRYPTO_JH8100) { + ret = starfive_sm3_register_algs(); + if (ret) + goto err_algs_sm3; + } + return 0; +err_algs_sm3: + starfive_rsa_unregister_algs(); err_algs_rsa: - starfive_hash_unregister_algs(); + starfive_hash_unregister_algs(cryp); err_algs_hash: starfive_aes_unregister_algs(); err_algs_aes: @@ -200,9 +201,12 @@ static void starfive_cryp_remove(struct platform_device *pdev) struct starfive_cryp_dev *cryp = platform_get_drvdata(pdev); starfive_aes_unregister_algs(); - starfive_hash_unregister_algs(); + starfive_hash_unregister_algs(cryp); starfive_rsa_unregister_algs(); + if (cryp->type == STARFIVE_CRYPTO_JH8100) + starfive_sm3_unregister_algs(); + crypto_engine_stop(cryp->engine); crypto_engine_exit(cryp->engine); @@ -218,7 +222,13 @@ static void starfive_cryp_remove(struct platform_device *pdev) } static const struct of_device_id starfive_dt_ids[] __maybe_unused = { - { .compatible = "starfive,jh7110-crypto", .data = NULL}, + { + .compatible = "starfive,jh7110-crypto", + .data = (const void *)STARFIVE_CRYPTO_JH7110, + }, { + .compatible = "starfive,jh8100-crypto", + .data = (const void *)STARFIVE_CRYPTO_JH8100, + }, {}, }; MODULE_DEVICE_TABLE(of, starfive_dt_ids); diff --git a/drivers/crypto/starfive/jh7110-cryp.h b/drivers/crypto/starfive/jh7110-cryp.h index 6e523e45cd9f..0e2bd03cc3bc 100644 --- a/drivers/crypto/starfive/jh7110-cryp.h +++ b/drivers/crypto/starfive/jh7110-cryp.h @@ -19,18 +19,34 @@ #define STARFIVE_DMA_IN_LEN_OFFSET 0x10 #define STARFIVE_DMA_OUT_LEN_OFFSET 0x14 +#define STARFIVE_SM_ALG_CR_OFFSET 0x4000 +#define STARFIVE_SM_IE_MASK_OFFSET (STARFIVE_SM_ALG_CR_OFFSET + 0x4) +#define STARFIVE_SM_IE_FLAG_OFFSET (STARFIVE_SM_ALG_CR_OFFSET + 0x8) +#define STARFIVE_SM_DMA_IN_LEN_OFFSET (STARFIVE_SM_ALG_CR_OFFSET + 0xc) +#define STARFIVE_SM_DMA_OUT_LEN_OFFSET (STARFIVE_SM_ALG_CR_OFFSET + 0x10) +#define STARFIVE_SM_ALG_FIFO_IN_OFFSET (STARFIVE_SM_ALG_CR_OFFSET + 0x20) +#define STARFIVE_SM_ALG_FIFO_OUT_OFFSET (STARFIVE_SM_ALG_CR_OFFSET + 0x28) + #define STARFIVE_IE_MASK_AES_DONE 0x1 #define STARFIVE_IE_MASK_HASH_DONE 0x4 #define STARFIVE_IE_MASK_PKA_DONE 0x8 #define STARFIVE_IE_FLAG_AES_DONE 0x1 #define STARFIVE_IE_FLAG_HASH_DONE 0x4 #define STARFIVE_IE_FLAG_PKA_DONE 0x8 +#define STARFIVE_SM_IE_MASK_SM3_DONE 0x2 +#define STARFIVE_SM_IE_FLAG_SM3_DONE 0x2 #define STARFIVE_MSG_BUFFER_SIZE SZ_16K #define MAX_KEY_SIZE SHA512_BLOCK_SIZE #define STARFIVE_AES_IV_LEN AES_BLOCK_SIZE #define STARFIVE_AES_CTR_LEN AES_BLOCK_SIZE +enum starfive_crypto_type { + STARFIVE_CRYPTO_UNKNOWN = 0, + STARFIVE_CRYPTO_JH7110, + STARFIVE_CRYPTO_JH8100, +}; + union starfive_aes_csr { u32 v; struct { @@ -68,6 +84,20 @@ union starfive_aes_csr { }; }; +union starfive_sm_alg_cr { + u32 v; + struct { + u32 start :1; + u32 sm4_dma_en :1; + u32 sm3_dma_en :1; + u32 rsvd_0 :1; + u32 alg_done :1; + u32 rsvd_1 :3; + u32 clear :1; + u32 rsvd_2 :23; + }; +}; + union starfive_hash_csr { u32 v; struct { @@ -132,6 +162,32 @@ union starfive_pka_casr { }; }; +union starfive_sm3_csr { + u32 v; + struct { + u32 start :1; + u32 reset :1; + u32 ie :1; + u32 firstb :1; +#define STARFIVE_SM3_MODE 0x0 + u32 mode :3; + u32 rsvd_0 :1; + u32 final :1; + u32 rsvd_1 :2; +#define STARFIVE_SM3_HMAC_FLAGS 0x800 + u32 hmac :1; + u32 rsvd_2 :1; +#define STARFIVE_SM3_KEY_DONE BIT(13) + u32 key_done :1; + u32 key_flag :1; + u32 hmac_done :1; +#define STARFIVE_SM3_BUSY BIT(16) + u32 busy :1; + u32 hashdone :1; + u32 rsvd_3 :14; + }; +}; + struct starfive_rsa_key { u8 *n; u8 *e; @@ -177,7 +233,7 @@ struct starfive_cryp_dev { struct clk *hclk; struct clk *ahb; struct reset_control *rst; - + enum starfive_crypto_type type; void __iomem *base; phys_addr_t phys_base; @@ -210,6 +266,7 @@ struct starfive_cryp_request_ctx { union starfive_hash_csr hash; union starfive_pka_cacr pka; union starfive_aes_csr aes; + union starfive_sm3_csr sm3; } csr; struct scatterlist *in_sg; @@ -226,12 +283,15 @@ struct starfive_cryp_request_ctx { struct starfive_cryp_dev *starfive_cryp_find_dev(struct starfive_cryp_ctx *ctx); -int starfive_hash_register_algs(void); -void starfive_hash_unregister_algs(void); +int starfive_hash_register_algs(struct starfive_cryp_dev *cryp); +void starfive_hash_unregister_algs(struct starfive_cryp_dev *cryp); int starfive_rsa_register_algs(void); void starfive_rsa_unregister_algs(void); int starfive_aes_register_algs(void); void starfive_aes_unregister_algs(void); + +int starfive_sm3_register_algs(void); +void starfive_sm3_unregister_algs(void); #endif diff --git a/drivers/crypto/starfive/jh7110-hash.c b/drivers/crypto/starfive/jh7110-hash.c index 4e82f05a7df7..bd01d3ad520b 100644 --- a/drivers/crypto/starfive/jh7110-hash.c +++ b/drivers/crypto/starfive/jh7110-hash.c @@ -103,6 +103,8 @@ static void starfive_hash_dma_callback(void *param) static void starfive_hash_dma_init(struct starfive_cryp_dev *cryp) { + memset(&cryp->cfg_in, 0, sizeof(struct dma_slave_config)); + cryp->cfg_in.src_addr_width = DMA_SLAVE_BUSWIDTH_16_BYTES; cryp->cfg_in.dst_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES; cryp->cfg_in.src_maxburst = cryp->dma_maxburst; @@ -505,12 +507,6 @@ static int starfive_sha512_init_tfm(struct crypto_ahash *hash) STARFIVE_HASH_SHA512, 0); } -static int starfive_sm3_init_tfm(struct crypto_ahash *hash) -{ - return starfive_hash_init_tfm(hash, "sm3-generic", - STARFIVE_HASH_SM3, 0); -} - static int starfive_hmac_sha224_init_tfm(struct crypto_ahash *hash) { return starfive_hash_init_tfm(hash, "hmac(sha224-generic)", @@ -535,13 +531,19 @@ static int starfive_hmac_sha512_init_tfm(struct crypto_ahash *hash) STARFIVE_HASH_SHA512, 1); } +static int starfive_sm3_init_tfm(struct crypto_ahash *hash) +{ + return starfive_hash_init_tfm(hash, "sm3-generic", + STARFIVE_HASH_SM3, 0); +} + static int starfive_hmac_sm3_init_tfm(struct crypto_ahash *hash) { return starfive_hash_init_tfm(hash, "hmac(sm3-generic)", STARFIVE_HASH_SM3, 1); } -static struct ahash_engine_alg algs_sha2_sm3[] = { +static struct ahash_engine_alg algs_sha2[] = { { .base.init = starfive_hash_init, .base.update = starfive_hash_update, @@ -770,7 +772,11 @@ static struct ahash_engine_alg algs_sha2_sm3[] = { .op = { .do_one_request = starfive_hash_one_request, }, -}, { +}, +}; + +static struct ahash_engine_alg algs_sm3[] = { +{ .base.init = starfive_hash_init, .base.update = starfive_hash_update, .base.final = starfive_hash_final, @@ -830,12 +836,27 @@ static struct ahash_engine_alg algs_sha2_sm3[] = { }, }; -int starfive_hash_register_algs(void) +int starfive_hash_register_algs(struct starfive_cryp_dev *cryp) { - return crypto_engine_register_ahashes(algs_sha2_sm3, ARRAY_SIZE(algs_sha2_sm3)); + int ret; + + ret = crypto_engine_register_ahashes(algs_sha2, ARRAY_SIZE(algs_sha2)); + if (ret) + return ret; + + if (cryp->type == STARFIVE_CRYPTO_JH7110) { + ret = crypto_engine_register_ahashes(algs_sm3, ARRAY_SIZE(algs_sm3)); + if (ret) + crypto_engine_unregister_ahashes(algs_sha2, ARRAY_SIZE(algs_sha2)); + } + + return ret; } -void starfive_hash_unregister_algs(void) +void starfive_hash_unregister_algs(struct starfive_cryp_dev *cryp) { - crypto_engine_unregister_ahashes(algs_sha2_sm3, ARRAY_SIZE(algs_sha2_sm3)); + crypto_engine_unregister_ahashes(algs_sha2, ARRAY_SIZE(algs_sha2)); + + if (cryp->type == STARFIVE_CRYPTO_JH7110) + crypto_engine_unregister_ahashes(algs_sm3, ARRAY_SIZE(algs_sm3)); } diff --git a/drivers/crypto/starfive/jh8100-sm3.c b/drivers/crypto/starfive/jh8100-sm3.c new file mode 100644 index 000000000000..3ffc8882c241 --- /dev/null +++ b/drivers/crypto/starfive/jh8100-sm3.c @@ -0,0 +1,535 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * SM3 Hash function and HMAC support for StarFive driver + * + * Copyright (c) 2022 - 2023 StarFive Technology + * + */ + +#include +#include +#include +#include +#include "jh7110-cryp.h" +#include +#include +#include +#include + +#define STARFIVE_SM3_REGS_OFFSET 0x4200 +#define STARFIVE_SM3_CSR (STARFIVE_SM3_REGS_OFFSET + 0x0) +#define STARFIVE_SM3_WDR (STARFIVE_SM3_REGS_OFFSET + 0x4) +#define STARFIVE_SM3_RDR (STARFIVE_SM3_REGS_OFFSET + 0x8) +#define STARFIVE_SM3_WSR (STARFIVE_SM3_REGS_OFFSET + 0xC) +#define STARFIVE_SM3_WLEN3 (STARFIVE_SM3_REGS_OFFSET + 0x10) +#define STARFIVE_SM3_WLEN2 (STARFIVE_SM3_REGS_OFFSET + 0x14) +#define STARFIVE_SM3_WLEN1 (STARFIVE_SM3_REGS_OFFSET + 0x18) +#define STARFIVE_SM3_WLEN0 (STARFIVE_SM3_REGS_OFFSET + 0x1C) +#define STARFIVE_SM3_WKR (STARFIVE_SM3_REGS_OFFSET + 0x20) +#define STARFIVE_SM3_WKLEN (STARFIVE_SM3_REGS_OFFSET + 0x24) + +#define STARFIVE_SM3_BUFLEN SHA512_BLOCK_SIZE +#define STARFIVE_SM3_RESET 0x2 + +static inline int starfive_sm3_wait_busy(struct starfive_cryp_ctx *ctx) +{ + struct starfive_cryp_dev *cryp = ctx->cryp; + u32 status; + + return readl_relaxed_poll_timeout(cryp->base + STARFIVE_SM3_CSR, status, + !(status & STARFIVE_SM3_BUSY), 10, 100000); +} + +static inline int starfive_sm3_wait_key_done(struct starfive_cryp_ctx *ctx) +{ + struct starfive_cryp_dev *cryp = ctx->cryp; + u32 status; + + return readl_relaxed_poll_timeout(cryp->base + STARFIVE_SM3_CSR, status, + (status & STARFIVE_SM3_KEY_DONE), 10, 100000); +} + +static int starfive_sm3_hmac_key(struct starfive_cryp_ctx *ctx) +{ + struct starfive_cryp_request_ctx *rctx = ctx->rctx; + struct starfive_cryp_dev *cryp = ctx->cryp; + int klen = ctx->keylen, loop; + unsigned int *key = (unsigned int *)ctx->key; + unsigned char *cl; + + writel(ctx->keylen, cryp->base + STARFIVE_SM3_WKLEN); + + rctx->csr.sm3.hmac = 1; + rctx->csr.sm3.key_flag = 1; + + writel(rctx->csr.sm3.v, cryp->base + STARFIVE_SM3_CSR); + + for (loop = 0; loop < klen / sizeof(unsigned int); loop++, key++) + writel(*key, cryp->base + STARFIVE_SM3_WKR); + + if (klen & 0x3) { + cl = (unsigned char *)key; + for (loop = 0; loop < (klen & 0x3); loop++, cl++) + writeb(*cl, cryp->base + STARFIVE_SM3_WKR); + } + + if (starfive_sm3_wait_key_done(ctx)) + return dev_err_probe(cryp->dev, -ETIMEDOUT, + "starfive_sm3_wait_key_done error\n"); + + return 0; +} + +static void starfive_sm3_start(struct starfive_cryp_dev *cryp) +{ + union starfive_sm3_csr csr; + + csr.v = readl(cryp->base + STARFIVE_SM3_CSR); + csr.firstb = 0; + csr.final = 1; + writel(csr.v, cryp->base + STARFIVE_SM3_CSR); +} + +static void starfive_sm3_dma_callback(void *param) +{ + struct starfive_cryp_dev *cryp = param; + + complete(&cryp->dma_done); +} + +static void starfive_sm3_dma_init(struct starfive_cryp_dev *cryp) +{ + struct dw_axi_peripheral_config periph_conf = {}; + + memset(&cryp->cfg_in, 0, sizeof(struct dma_slave_config)); + periph_conf.quirks = DWAXIDMAC_STARFIVE_SM_ALGO; + + cryp->cfg_in.direction = DMA_MEM_TO_DEV; + cryp->cfg_in.src_addr_width = DMA_SLAVE_BUSWIDTH_8_BYTES; + cryp->cfg_in.dst_addr_width = DMA_SLAVE_BUSWIDTH_8_BYTES; + cryp->cfg_in.src_maxburst = cryp->dma_maxburst; + cryp->cfg_in.dst_maxburst = cryp->dma_maxburst; + cryp->cfg_in.dst_addr = cryp->phys_base + STARFIVE_SM_ALG_FIFO_IN_OFFSET; + cryp->cfg_in.peripheral_config = &periph_conf; + cryp->cfg_in.peripheral_size = sizeof(struct dw_axi_peripheral_config); + + dmaengine_slave_config(cryp->tx, &cryp->cfg_in); + + init_completion(&cryp->dma_done); +} + +static int starfive_sm3_dma_xfer(struct starfive_cryp_dev *cryp, + struct scatterlist *sg) +{ + struct dma_async_tx_descriptor *in_desc; + union starfive_sm_alg_cr alg_cr; + int ret = 0; + + alg_cr.v = 0; + alg_cr.start = 1; + alg_cr.sm3_dma_en = 1; + writel(alg_cr.v, cryp->base + STARFIVE_SM_ALG_CR_OFFSET); + + writel(sg_dma_len(sg), cryp->base + STARFIVE_SM_DMA_IN_LEN_OFFSET); + sg_dma_len(sg) = ALIGN(sg_dma_len(sg), sizeof(u32)); + + in_desc = dmaengine_prep_slave_sg(cryp->tx, sg, 1, DMA_MEM_TO_DEV, + DMA_PREP_INTERRUPT | DMA_CTRL_ACK); + if (!in_desc) { + ret = -EINVAL; + goto end; + } + + reinit_completion(&cryp->dma_done); + in_desc->callback = starfive_sm3_dma_callback; + in_desc->callback_param = cryp; + + dmaengine_submit(in_desc); + dma_async_issue_pending(cryp->tx); + + if (!wait_for_completion_timeout(&cryp->dma_done, + msecs_to_jiffies(1000))) + ret = -ETIMEDOUT; + +end: + alg_cr.v = 0; + alg_cr.clear = 1; + writel(alg_cr.v, cryp->base + STARFIVE_SM_ALG_CR_OFFSET); + + return ret; +} + +static int starfive_sm3_copy_hash(struct ahash_request *req) +{ + struct starfive_cryp_request_ctx *rctx = ahash_request_ctx(req); + struct starfive_cryp_ctx *ctx = crypto_ahash_ctx(crypto_ahash_reqtfm(req)); + int count, *data; + int mlen; + + if (!req->result) + return 0; + + mlen = rctx->digsize / sizeof(u32); + data = (u32 *)req->result; + + for (count = 0; count < mlen; count++) + data[count] = readl(ctx->cryp->base + STARFIVE_SM3_RDR); + + return 0; +} + +static void starfive_sm3_done_task(struct starfive_cryp_dev *cryp) +{ + int err; + + err = starfive_sm3_copy_hash(cryp->req.hreq); + + crypto_finalize_hash_request(cryp->engine, cryp->req.hreq, err); +} + +static int starfive_sm3_one_request(struct crypto_engine *engine, void *areq) +{ + struct ahash_request *req = + container_of(areq, struct ahash_request, base); + struct starfive_cryp_ctx *ctx = + crypto_ahash_ctx(crypto_ahash_reqtfm(req)); + struct starfive_cryp_dev *cryp = ctx->cryp; + struct starfive_cryp_request_ctx *rctx = ctx->rctx; + struct scatterlist *tsg; + int ret, src_nents, i; + + rctx->csr.sm3.v = 0; + rctx->csr.sm3.reset = 1; + + writel(rctx->csr.sm3.v, cryp->base + STARFIVE_SM3_CSR); + + if (starfive_sm3_wait_busy(ctx)) + return dev_err_probe(cryp->dev, -ETIMEDOUT, + "Error resetting engine.\n"); + + rctx->csr.sm3.v = 0; + rctx->csr.sm3.mode = ctx->hash_mode; + + if (ctx->is_hmac) { + ret = starfive_sm3_hmac_key(ctx); + if (ret) + return ret; + } else { + rctx->csr.sm3.start = 1; + rctx->csr.sm3.firstb = 1; + writel(rctx->csr.sm3.v, cryp->base + STARFIVE_SM3_CSR); + } + + /* No input message, get digest and end. */ + if (!rctx->total) + goto hash_start; + + starfive_sm3_dma_init(cryp); + + for_each_sg(rctx->in_sg, tsg, rctx->in_sg_len, i) { + src_nents = dma_map_sg(cryp->dev, tsg, 1, DMA_TO_DEVICE); + if (src_nents == 0) + return dev_err_probe(cryp->dev, -ENOMEM, + "dma_map_sg error\n"); + + ret = starfive_sm3_dma_xfer(cryp, tsg); + dma_unmap_sg(cryp->dev, tsg, 1, DMA_TO_DEVICE); + if (ret) + return ret; + } + +hash_start: + starfive_sm3_start(cryp); + + if (starfive_sm3_wait_busy(ctx)) + return dev_err_probe(cryp->dev, -ETIMEDOUT, "Error generating digest.\n"); + + starfive_sm3_done_task(cryp); + + return 0; +} + +static void starfive_sm3_set_ahash(struct ahash_request *req, + struct starfive_cryp_ctx *ctx, + struct starfive_cryp_request_ctx *rctx) +{ + ahash_request_set_tfm(&rctx->ahash_fbk_req, ctx->ahash_fbk); + ahash_request_set_callback(&rctx->ahash_fbk_req, + req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP, + req->base.complete, req->base.data); + ahash_request_set_crypt(&rctx->ahash_fbk_req, req->src, + req->result, req->nbytes); +} + +static int starfive_sm3_init(struct ahash_request *req) +{ + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct starfive_cryp_request_ctx *rctx = ahash_request_ctx(req); + struct starfive_cryp_ctx *ctx = crypto_ahash_ctx(tfm); + + starfive_sm3_set_ahash(req, ctx, rctx); + + return crypto_ahash_init(&rctx->ahash_fbk_req); +} + +static int starfive_sm3_update(struct ahash_request *req) +{ + struct starfive_cryp_request_ctx *rctx = ahash_request_ctx(req); + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct starfive_cryp_ctx *ctx = crypto_ahash_ctx(tfm); + + starfive_sm3_set_ahash(req, ctx, rctx); + + return crypto_ahash_update(&rctx->ahash_fbk_req); +} + +static int starfive_sm3_final(struct ahash_request *req) +{ + struct starfive_cryp_request_ctx *rctx = ahash_request_ctx(req); + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct starfive_cryp_ctx *ctx = crypto_ahash_ctx(tfm); + + starfive_sm3_set_ahash(req, ctx, rctx); + + return crypto_ahash_final(&rctx->ahash_fbk_req); +} + +static int starfive_sm3_finup(struct ahash_request *req) +{ + struct starfive_cryp_request_ctx *rctx = ahash_request_ctx(req); + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct starfive_cryp_ctx *ctx = crypto_ahash_ctx(tfm); + + starfive_sm3_set_ahash(req, ctx, rctx); + + return crypto_ahash_finup(&rctx->ahash_fbk_req); +} + +static int starfive_sm3_digest(struct ahash_request *req) +{ + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct starfive_cryp_ctx *ctx = crypto_ahash_ctx(tfm); + struct starfive_cryp_request_ctx *rctx = ahash_request_ctx(req); + struct starfive_cryp_dev *cryp = ctx->cryp; + + memset(rctx, 0, sizeof(struct starfive_cryp_request_ctx)); + + cryp->req.hreq = req; + rctx->total = req->nbytes; + rctx->in_sg = req->src; + rctx->blksize = crypto_tfm_alg_blocksize(crypto_ahash_tfm(tfm)); + rctx->digsize = crypto_ahash_digestsize(tfm); + rctx->in_sg_len = sg_nents_for_len(rctx->in_sg, rctx->total); + ctx->rctx = rctx; + + return crypto_transfer_hash_request_to_engine(cryp->engine, req); +} + +static int starfive_sm3_export(struct ahash_request *req, void *out) +{ + struct starfive_cryp_request_ctx *rctx = ahash_request_ctx(req); + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct starfive_cryp_ctx *ctx = crypto_ahash_ctx(tfm); + + ahash_request_set_tfm(&rctx->ahash_fbk_req, ctx->ahash_fbk); + ahash_request_set_callback(&rctx->ahash_fbk_req, + req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP, + req->base.complete, req->base.data); + + return crypto_ahash_export(&rctx->ahash_fbk_req, out); +} + +static int starfive_sm3_import(struct ahash_request *req, const void *in) +{ + struct starfive_cryp_request_ctx *rctx = ahash_request_ctx(req); + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct starfive_cryp_ctx *ctx = crypto_ahash_ctx(tfm); + + ahash_request_set_tfm(&rctx->ahash_fbk_req, ctx->ahash_fbk); + ahash_request_set_callback(&rctx->ahash_fbk_req, + req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP, + req->base.complete, req->base.data); + + return crypto_ahash_import(&rctx->ahash_fbk_req, in); +} + +static int starfive_sm3_init_algo(struct crypto_ahash *hash, + const char *alg_name, + bool is_hmac) +{ + struct starfive_cryp_ctx *ctx = crypto_ahash_ctx(hash); + + ctx->cryp = starfive_cryp_find_dev(ctx); + if (!ctx->cryp) + return -ENODEV; + + ctx->ahash_fbk = crypto_alloc_ahash(alg_name, 0, + CRYPTO_ALG_NEED_FALLBACK); + + if (IS_ERR(ctx->ahash_fbk)) + return dev_err_probe(ctx->cryp->dev, PTR_ERR(ctx->ahash_fbk), + "starfive-sm3: Could not load fallback driver.\n"); + + crypto_ahash_set_statesize(hash, crypto_ahash_statesize(ctx->ahash_fbk)); + crypto_ahash_set_reqsize(hash, sizeof(struct starfive_cryp_request_ctx) + + crypto_ahash_reqsize(ctx->ahash_fbk)); + + ctx->keylen = 0; + ctx->hash_mode = STARFIVE_SM3_MODE; + ctx->is_hmac = is_hmac; + + return 0; +} + +static void starfive_sm3_exit_tfm(struct crypto_ahash *hash) +{ + struct starfive_cryp_ctx *ctx = crypto_ahash_ctx(hash); + + crypto_free_ahash(ctx->ahash_fbk); +} + +static int starfive_sm3_long_setkey(struct starfive_cryp_ctx *ctx, + const u8 *key, unsigned int keylen) +{ + struct crypto_wait wait; + struct ahash_request *req; + struct scatterlist sg; + struct crypto_ahash *ahash_tfm; + struct starfive_cryp_dev *cryp = ctx->cryp; + u8 *buf; + int ret; + + ahash_tfm = crypto_alloc_ahash("sm3-starfive", 0, 0); + if (IS_ERR(ahash_tfm)) + return PTR_ERR(ahash_tfm); + + req = ahash_request_alloc(ahash_tfm, GFP_KERNEL); + if (!req) { + ret = -ENOMEM; + goto err_free_ahash; + } + + crypto_init_wait(&wait); + ahash_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG, + crypto_req_done, &wait); + crypto_ahash_clear_flags(ahash_tfm, ~0); + + buf = devm_kzalloc(cryp->dev, keylen + STARFIVE_SM3_BUFLEN, GFP_KERNEL); + if (!buf) { + ret = -ENOMEM; + goto err_free_req; + } + + memcpy(buf, key, keylen); + sg_init_one(&sg, buf, keylen); + ahash_request_set_crypt(req, &sg, ctx->key, keylen); + + ret = crypto_wait_req(crypto_ahash_digest(req), &wait); + +err_free_req: + ahash_request_free(req); +err_free_ahash: + crypto_free_ahash(ahash_tfm); + return ret; +} + +static int starfive_sm3_setkey(struct crypto_ahash *hash, + const u8 *key, unsigned int keylen) +{ + struct starfive_cryp_ctx *ctx = crypto_ahash_ctx(hash); + unsigned int digestsize = crypto_ahash_digestsize(hash); + unsigned int blocksize = crypto_ahash_blocksize(hash); + + crypto_ahash_setkey(ctx->ahash_fbk, key, keylen); + + if (keylen <= blocksize) { + memcpy(ctx->key, key, keylen); + ctx->keylen = keylen; + return 0; + } + + ctx->keylen = digestsize; + + return starfive_sm3_long_setkey(ctx, key, keylen); +} + +static int starfive_sm3_init_tfm(struct crypto_ahash *hash) +{ + return starfive_sm3_init_algo(hash, "sm3-generic", 0); +} + +static int starfive_hmac_sm3_init_tfm(struct crypto_ahash *hash) +{ + return starfive_sm3_init_algo(hash, "hmac(sm3-generic)", 1); +} + +static struct ahash_engine_alg algs_sm3[] = { +{ + .base.init = starfive_sm3_init, + .base.update = starfive_sm3_update, + .base.final = starfive_sm3_final, + .base.finup = starfive_sm3_finup, + .base.digest = starfive_sm3_digest, + .base.export = starfive_sm3_export, + .base.import = starfive_sm3_import, + .base.init_tfm = starfive_sm3_init_tfm, + .base.exit_tfm = starfive_sm3_exit_tfm, + .base.halg = { + .digestsize = SM3_DIGEST_SIZE, + .statesize = sizeof(struct sm3_state), + .base = { + .cra_name = "sm3", + .cra_driver_name = "sm3-starfive", + .cra_priority = 200, + .cra_flags = CRYPTO_ALG_ASYNC | + CRYPTO_ALG_TYPE_AHASH | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize = SM3_BLOCK_SIZE, + .cra_ctxsize = sizeof(struct starfive_cryp_ctx), + .cra_module = THIS_MODULE, + } + }, + .op = { + .do_one_request = starfive_sm3_one_request, + }, +}, { + .base.init = starfive_sm3_init, + .base.update = starfive_sm3_update, + .base.final = starfive_sm3_final, + .base.finup = starfive_sm3_finup, + .base.digest = starfive_sm3_digest, + .base.export = starfive_sm3_export, + .base.import = starfive_sm3_import, + .base.init_tfm = starfive_hmac_sm3_init_tfm, + .base.exit_tfm = starfive_sm3_exit_tfm, + .base.setkey = starfive_sm3_setkey, + .base.halg = { + .digestsize = SM3_DIGEST_SIZE, + .statesize = sizeof(struct sm3_state), + .base = { + .cra_name = "hmac(sm3)", + .cra_driver_name = "sm3-hmac-starfive", + .cra_priority = 200, + .cra_flags = CRYPTO_ALG_ASYNC | + CRYPTO_ALG_TYPE_AHASH | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize = SM3_BLOCK_SIZE, + .cra_ctxsize = sizeof(struct starfive_cryp_ctx), + .cra_module = THIS_MODULE, + } + }, + .op = { + .do_one_request = starfive_sm3_one_request, + }, +}, +}; + +int starfive_sm3_register_algs(void) +{ + return crypto_engine_register_ahashes(algs_sm3, ARRAY_SIZE(algs_sm3)); +} + +void starfive_sm3_unregister_algs(void) +{ + crypto_engine_unregister_ahashes(algs_sm3, ARRAY_SIZE(algs_sm3)); +}