From patchwork Fri Oct 5 08:13:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 148154 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp151235lji; Fri, 5 Oct 2018 01:14:03 -0700 (PDT) X-Google-Smtp-Source: ACcGV63R3Xf62S85JjXMKvvOOfUmVtucBbg6KwCNM4ZmNWd8m81Kb2tAgVKBTVq7A/Te2fxYz7uf X-Received: by 2002:a17:902:b213:: with SMTP id t19-v6mr10586788plr.51.1538727243158; Fri, 05 Oct 2018 01:14:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538727243; cv=none; d=google.com; s=arc-20160816; b=u5LNLFxrpHHjjVqOxdBX2o+htSGLQYucI6jT/DdFGvz5L/R1monv0GTznkWECD4ly0 ff2fvmPs8nQ1UeYhSuJvto3IbS/T8kJP54geQESfmUJVJmzfCdDXtUL+qBE318CkTa0D uWO2UZTCLq88zL+Hus3vbVPm0skdKAoa2Fq06cz3TbzB49aoCCME9vsz9VdrRPLezxwk dX7qNt+O1bBFPv4S4toe021dRwiti2kBgiIcAIUlysH95L5rn6SQeb2CwjDLdIlHvwth xiX6II8GsZ6dhgeUj4NkYpJv3Z4WoHO4LgChBBf36stYhH6al1Ab/o60kfVDXBw87esH Z8OA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=A+1FdRMKX5kWT68MST48ZYdoIobJ5B3kYMhv71JKOAg=; b=bI/Y1BaMDdXUGi8izs00Syk/DQmcRijPgw+/FGfCOtekx6Bc1VvhWRDT9DFHRsdnGl Iv6BLzSD4I0pBkLddCR6g22NFKtxYDw17lWUXQFt+oI+zldPOLbdsJgsOkB6Kjw/SVNO Ce+Q4mKusbw60hA7Fb6X3kyTNY3MBsc2K9Uc92xCnR3hsFqM00Aop03d9WgXZJ5I9JBe 1ROcTna1LLCNp0GGnWsy5euT+6vob/96JKBe/h9hon8tr1J01oePv92fAIq2ngW6uI5i SQxuyheeExtvpjrYcCRXSfuMj1fpb0MdXu6/T7QJOo7XRVODRUADXyBUBd0I5qCm831u YSdw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PJcLwtVX; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n2-v6si7341775plk.255.2018.10.05.01.14.02; Fri, 05 Oct 2018 01:14:03 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PJcLwtVX; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728614AbeJEPLg (ORCPT + 32 others); Fri, 5 Oct 2018 11:11:36 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:35454 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728581AbeJEPLa (ORCPT ); Fri, 5 Oct 2018 11:11:30 -0400 Received: by mail-wm1-f68.google.com with SMTP id e187-v6so1036453wmf.0 for ; Fri, 05 Oct 2018 01:13:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=A+1FdRMKX5kWT68MST48ZYdoIobJ5B3kYMhv71JKOAg=; b=PJcLwtVXRuyJPKWbbwJBGaYD/0+A6pH/bRN1v8Ky+MsOeDl1X8nZVqwwcKBgUCWjct PMh/eQPIxbpQRmrMSUG0NVWRyNWL60QRHPjdWY8s3j9dnprSTAVQeym8nwAdgE0Kfn39 qt/i7giQasS5aiIXOxSLGpmt7KZ1soLYUoyuA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=A+1FdRMKX5kWT68MST48ZYdoIobJ5B3kYMhv71JKOAg=; b=tluxwEiIPDSJloz+1X+IG6iOxivHJCcj7VFN3GI3k9ynxYazam65OuSDmD1tqpjFrD 5gebJftGlpeApmHmueXgA6YLtF9JNym2F31lBl4viCwjQLERmTRkb4VkDUKFewIEQ3Fs mTc1qRayiXCbguKoSCzDyAcWHc/9Fd+xCFKx2x9/S/t9vspnweixVlNdTGvSsdybj3xL M+37WceBLO/GCstMm9bUfmcJzYm+aPF0LmTCkrTAIwITLdJeGm90QPkYKvswJ6mRsaXr wcEKQZAenow33QvJfepZN0fV3NEFyqGRYFK8lxabirZYh6EbfmDWPrEQUZzr7rJFL7NE MqkQ== X-Gm-Message-State: ABuFfogglaQiDw746rUjizPb1/G/yZaL05+jdkyh9NM9AXzw0riiQoJ5 baR9nVD+s/pEQSAARAXfnWfsHrNSEVE= X-Received: by 2002:a1c:9e8f:: with SMTP id h137-v6mr380822wme.142.1538727232789; Fri, 05 Oct 2018 01:13:52 -0700 (PDT) Received: from localhost.localdomain ([2a01:cb1d:112:6f00:697e:67d9:a05d:22c7]) by smtp.gmail.com with ESMTPSA id t4-v6sm6565620wrb.45.2018.10.05.01.13.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Oct 2018 01:13:52 -0700 (PDT) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , "Jason A . Donenfeld" , Eric Biggers , Samuel Neves , Andy Lutomirski , Arnd Bergmann , Herbert Xu , "David S. Miller" , Catalin Marinas , Will Deacon , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Thomas Gleixner , Ingo Molnar , Kees Cook , "Martin K. Petersen" , Greg Kroah-Hartman , Andrew Morton , Richard Weinberger , Peter Zijlstra , linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH 9/9] crypto: crc-t10dif/x86 - move PMULL based code into core library Date: Fri, 5 Oct 2018 10:13:33 +0200 Message-Id: <20181005081333.15018-10-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181005081333.15018-1-ard.biesheuvel@linaro.org> References: <20181005081333.15018-1-ard.biesheuvel@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Move the PMULL based routines out of the crypto API into the core CRC-T10DIF library. Signed-off-by: Ard Biesheuvel --- arch/x86/crypto/crct10dif-pclmul_glue.c | 98 +++----------------- 1 file changed, 13 insertions(+), 85 deletions(-) -- 2.11.0 diff --git a/arch/x86/crypto/crct10dif-pclmul_glue.c b/arch/x86/crypto/crct10dif-pclmul_glue.c index cd4df9322501..47f0017ad7f3 100644 --- a/arch/x86/crypto/crct10dif-pclmul_glue.c +++ b/arch/x86/crypto/crct10dif-pclmul_glue.c @@ -25,7 +25,7 @@ #include #include #include -#include +#include #include #include #include @@ -36,90 +36,17 @@ asmlinkage __u16 crc_t10dif_pcl(__u16 crc, const unsigned char *buf, size_t len); -struct chksum_desc_ctx { - __u16 crc; -}; - -/* - * Steps through buffer one byte at at time, calculates reflected - * crc using table. - */ - -static int chksum_init(struct shash_desc *desc) -{ - struct chksum_desc_ctx *ctx = shash_desc_ctx(desc); - - ctx->crc = 0; - - return 0; -} - -static int chksum_update(struct shash_desc *desc, const u8 *data, - unsigned int length) -{ - struct chksum_desc_ctx *ctx = shash_desc_ctx(desc); - - if (irq_fpu_usable()) { - kernel_fpu_begin(); - ctx->crc = crc_t10dif_pcl(ctx->crc, data, length); - kernel_fpu_end(); - } else - ctx->crc = crc_t10dif_generic(ctx->crc, data, length); - return 0; -} - -static int chksum_final(struct shash_desc *desc, u8 *out) -{ - struct chksum_desc_ctx *ctx = shash_desc_ctx(desc); - - *(__u16 *)out = ctx->crc; - return 0; -} - -static int __chksum_finup(__u16 *crcp, const u8 *data, unsigned int len, - u8 *out) +static __u16 crct10dif_update_x86(__u16 crc, const u8 *data, unsigned int length) { if (irq_fpu_usable()) { kernel_fpu_begin(); - *(__u16 *)out = crc_t10dif_pcl(*crcp, data, len); + crc = crc_t10dif_pcl(crc, data, length); kernel_fpu_end(); - } else - *(__u16 *)out = crc_t10dif_generic(*crcp, data, len); - return 0; -} - -static int chksum_finup(struct shash_desc *desc, const u8 *data, - unsigned int len, u8 *out) -{ - struct chksum_desc_ctx *ctx = shash_desc_ctx(desc); - - return __chksum_finup(&ctx->crc, data, len, out); -} - -static int chksum_digest(struct shash_desc *desc, const u8 *data, - unsigned int length, u8 *out) -{ - struct chksum_desc_ctx *ctx = shash_desc_ctx(desc); - - return __chksum_finup(&ctx->crc, data, length, out); -} - -static struct shash_alg alg = { - .digestsize = CRC_T10DIF_DIGEST_SIZE, - .init = chksum_init, - .update = chksum_update, - .final = chksum_final, - .finup = chksum_finup, - .digest = chksum_digest, - .descsize = sizeof(struct chksum_desc_ctx), - .base = { - .cra_name = "crct10dif", - .cra_driver_name = "crct10dif-pclmul", - .cra_priority = 200, - .cra_blocksize = CRC_T10DIF_BLOCK_SIZE, - .cra_module = THIS_MODULE, + } else { + crc = crc_t10dif_generic(crc, data, length); } -}; + return crc; +} static const struct x86_cpu_id crct10dif_cpu_id[] = { X86_FEATURE_MATCH(X86_FEATURE_PCLMULQDQ), @@ -132,12 +59,16 @@ static int __init crct10dif_intel_mod_init(void) if (!x86_match_cpu(crct10dif_cpu_id)) return -ENODEV; - return crypto_register_shash(&alg); + SET_FFP(crc_t10dif_update_arch, crct10dif_update_x86); + + return 0; } static void __exit crct10dif_intel_mod_fini(void) { - crypto_unregister_shash(&alg); + RESET_FFP(crc_t10dif_update_arch); + /* wait for all users to switch back to the old version */ + synchronize_rcu(); } module_init(crct10dif_intel_mod_init); @@ -146,6 +77,3 @@ module_exit(crct10dif_intel_mod_fini); MODULE_AUTHOR("Tim Chen "); MODULE_DESCRIPTION("T10 DIF CRC calculation accelerated with PCLMULQDQ."); MODULE_LICENSE("GPL"); - -MODULE_ALIAS_CRYPTO("crct10dif"); -MODULE_ALIAS_CRYPTO("crct10dif-pclmul");