From patchwork Mon Sep 24 12:48:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 147348 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp680813lji; Mon, 24 Sep 2018 05:48:58 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZOx8NQJroKIBmTVXvL3MWGQvQEcNi83GvOGDKj2cimlTO/u4hjoxl+bhvwuaFAPg94g0ls X-Received: by 2002:a62:22c7:: with SMTP id p68-v6mr8677453pfj.53.1537793338340; Mon, 24 Sep 2018 05:48:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537793338; cv=none; d=google.com; s=arc-20160816; b=nHXnX+HDUOO+QqXKxbv6YtIuLgEk1yV0eDO4NbLl1bL1kjqZ8zxfwfBZKVRKklBzqd aPJI9d62C6OMYG5DUuf5s4d4WQE4o6Qwjt8hQ4rGvI1vNbT+p8aMpLoA2NBfTwhDee01 +mU8Ht49XsmZlubJrLHmR+xXXiDburHeYp53IVbBX45iRtF9T/27wBy9g+VFAVm5zpfF cdG1a6FEwFrcAK/jQ9jaCLrzP++8j/cVpNmzZKXRQGN+UceWT9U4GtiI+zh14CZF9UuF 0pVG+ldgOzkcDC81DK4wJ3JDhdDHn4YlACwMPCi47GitAuiT4NhKTLh4TYsJq3iGQgrz 0vgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :dkim-signature; bh=ws7/oVGU6bXVT0asbF2DAhftbnP7Idgr2lyqqsOT0qY=; b=tnWkGd9XicVFsYx4JAEH0440F+AF4Z2m8ioIMMa9xnipDSUbwoYm5nYb6WO6tCDwu6 zddX4eDgXeJKccVwyOeqgR6uwa28UnSmVsFO0bC6DF/Qw+kMz8MkcfFqFf1JMaQIl+Zr CuauO1TNJYyXlNQAQysrT76e5smn1QjYmSZXsnPbt2TH0PQx+TH/b2ir49wZlGCKT32t hyoDwPpYv3V2cqBRdfzcotuPbOyUKQtcV82KKq5LMKBcQU2SMVQj/hRIxkVNQIs9s428 PPFpV+Msl0bPp7JQV2Riwg3cJbbTKvVrvvTdyE6KlqLguYHVUJ54Ppq2KRL7udMC9h/N okOQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iP13l8IT; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w3-v6si34099909plz.333.2018.09.24.05.48.58; Mon, 24 Sep 2018 05:48:58 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iP13l8IT; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729233AbeIXSu4 (ORCPT + 2 others); Mon, 24 Sep 2018 14:50:56 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:41708 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727518AbeIXSu4 (ORCPT ); Mon, 24 Sep 2018 14:50:56 -0400 Received: by mail-wr1-f68.google.com with SMTP id j15-v6so15462192wrt.8 for ; Mon, 24 Sep 2018 05:48:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=ws7/oVGU6bXVT0asbF2DAhftbnP7Idgr2lyqqsOT0qY=; b=iP13l8ITnLqeI3qkhAUz4OEC7cq0XQsMb2hBrribLC8H29S0roxE8kLItESprnicyn UyRug+V26EACt4Pk2u7XuiBNwJ4gXhr6Hy94ChtbWkFOT9jCGXuURb+YHpV4X0AtCtht B2W1paPJVM5YwNUMIbnAlO8S3eca2qZv+tRM8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=ws7/oVGU6bXVT0asbF2DAhftbnP7Idgr2lyqqsOT0qY=; b=XBYiJbhDke3Jpna/1r1mNEfg53aH7rMaF4wWTEWRbRMmN7ImAxlJEw1ZvGgpXjDu68 aQpI1Nt7bN+yIy4IHf/FOUSEtoDDDnFNd2KZcpRCvROOCs/NEKqOkK+Sv87ck+mJR+Zz +5Mypd8FaegTLh36LdO4VwQhQNlNq+5LYpjOt7wmYU06r19aPxtRPyEgShCL71U+NnRt rtDmAlyaVsvcVgxThalLgDjmW8AxjbclDQGe7wxVoMPCSLVwiKz4TXludqUE5u3w3YNy ng29+bqtvKmbe1QNkKuBo5ODZkIj401LaOlQQsueuRb+XPTFNjSFx/JmB5mFbSox/UYk 3S9Q== X-Gm-Message-State: ABuFfogTs8RCwF1P2OwwYG810YUH8OjI6swWnxYVwFRES98qTGPgB/bd tMumkuBB3C/6g6Vy/x2aJJ8s55hn39w= X-Received: by 2002:adf:ba12:: with SMTP id o18-v6mr8268466wrg.249.1537793334654; Mon, 24 Sep 2018 05:48:54 -0700 (PDT) Received: from rev02.home ([2a01:cb1d:112:6f00:b0c7:f621:ad3c:e6fe]) by smtp.gmail.com with ESMTPSA id a11-v6sm32747037wrr.81.2018.09.24.05.48.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Sep 2018 05:48:53 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, megha.dey@linux.intel.com, keescook@chromium.org, Ard Biesheuvel Subject: [RFC PATCH] crypto: x86/aes-ni - remove special handling of AES in PCBC mode Date: Mon, 24 Sep 2018 14:48:16 +0200 Message-Id: <20180924124816.18372-1-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.18.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org For historical reasons, the AES-NI based implementation of the PCBC chaining mode uses a special FPU chaining mode wrapper template to amortize the FPU start/stop overhead over multiple blocks. When this FPU wrapper was introduced, it supported widely used chaining modes such as XTS and CTR (as well as LRW), but currently, PCBC is the only remaining user. Since there are no known users of pcbc(aes) in the kernel, let's remove this special driver, and rely on the generic pcbc driver to encapsulate the AES-NI core cipher. Signed-off-by: Ard Biesheuvel --- arch/x86/crypto/Makefile | 2 +- arch/x86/crypto/aesni-intel_glue.c | 32 --- arch/x86/crypto/fpu.c | 207 -------------------- crypto/Kconfig | 2 +- 4 files changed, 2 insertions(+), 241 deletions(-) -- 2.18.0 diff --git a/arch/x86/crypto/Makefile b/arch/x86/crypto/Makefile index a450ad573dcb..42d22005764c 100644 --- a/arch/x86/crypto/Makefile +++ b/arch/x86/crypto/Makefile @@ -106,7 +106,7 @@ ifeq ($(avx2_supported),yes) morus1280-avx2-y := morus1280-avx2-asm.o morus1280-avx2-glue.o endif -aesni-intel-y := aesni-intel_asm.o aesni-intel_glue.o fpu.o +aesni-intel-y := aesni-intel_asm.o aesni-intel_glue.o aesni-intel-$(CONFIG_64BIT) += aesni-intel_avx-x86_64.o aes_ctrby8_avx-x86_64.o ghash-clmulni-intel-y := ghash-clmulni-intel_asm.o ghash-clmulni-intel_glue.o sha1-ssse3-y := sha1_ssse3_asm.o sha1_ssse3_glue.o diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c index acbe7e8336d8..d90770c43b40 100644 --- a/arch/x86/crypto/aesni-intel_glue.c +++ b/arch/x86/crypto/aesni-intel_glue.c @@ -1253,22 +1253,6 @@ static struct skcipher_alg aesni_skciphers[] = { static struct simd_skcipher_alg *aesni_simd_skciphers[ARRAY_SIZE(aesni_skciphers)]; -static struct { - const char *algname; - const char *drvname; - const char *basename; - struct simd_skcipher_alg *simd; -} aesni_simd_skciphers2[] = { -#if (defined(MODULE) && IS_ENABLED(CONFIG_CRYPTO_PCBC)) || \ - IS_BUILTIN(CONFIG_CRYPTO_PCBC) - { - .algname = "pcbc(aes)", - .drvname = "pcbc-aes-aesni", - .basename = "fpu(pcbc(__aes-aesni))", - }, -#endif -}; - #ifdef CONFIG_X86_64 static int generic_gcmaes_set_key(struct crypto_aead *aead, const u8 *key, unsigned int key_len) @@ -1422,10 +1406,6 @@ static void aesni_free_simds(void) for (i = 0; i < ARRAY_SIZE(aesni_simd_skciphers) && aesni_simd_skciphers[i]; i++) simd_skcipher_free(aesni_simd_skciphers[i]); - - for (i = 0; i < ARRAY_SIZE(aesni_simd_skciphers2); i++) - if (aesni_simd_skciphers2[i].simd) - simd_skcipher_free(aesni_simd_skciphers2[i].simd); } static int __init aesni_init(void) @@ -1499,18 +1479,6 @@ static int __init aesni_init(void) aesni_simd_skciphers[i] = simd; } - for (i = 0; i < ARRAY_SIZE(aesni_simd_skciphers2); i++) { - algname = aesni_simd_skciphers2[i].algname; - drvname = aesni_simd_skciphers2[i].drvname; - basename = aesni_simd_skciphers2[i].basename; - simd = simd_skcipher_create_compat(algname, drvname, basename); - err = PTR_ERR(simd); - if (IS_ERR(simd)) - continue; - - aesni_simd_skciphers2[i].simd = simd; - } - return 0; unregister_simds: diff --git a/arch/x86/crypto/fpu.c b/arch/x86/crypto/fpu.c deleted file mode 100644 index 406680476c52..000000000000 --- a/arch/x86/crypto/fpu.c +++ /dev/null @@ -1,207 +0,0 @@ -/* - * FPU: Wrapper for blkcipher touching fpu - * - * Copyright (c) Intel Corp. - * Author: Huang Ying - * - * This program is free software; you can redistribute it and/or modify it - * under the terms of the GNU General Public License as published by the Free - * Software Foundation; either version 2 of the License, or (at your option) - * any later version. - * - */ - -#include -#include -#include -#include -#include -#include -#include - -struct crypto_fpu_ctx { - struct crypto_skcipher *child; -}; - -static int crypto_fpu_setkey(struct crypto_skcipher *parent, const u8 *key, - unsigned int keylen) -{ - struct crypto_fpu_ctx *ctx = crypto_skcipher_ctx(parent); - struct crypto_skcipher *child = ctx->child; - int err; - - crypto_skcipher_clear_flags(child, CRYPTO_TFM_REQ_MASK); - crypto_skcipher_set_flags(child, crypto_skcipher_get_flags(parent) & - CRYPTO_TFM_REQ_MASK); - err = crypto_skcipher_setkey(child, key, keylen); - crypto_skcipher_set_flags(parent, crypto_skcipher_get_flags(child) & - CRYPTO_TFM_RES_MASK); - return err; -} - -static int crypto_fpu_encrypt(struct skcipher_request *req) -{ - struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req); - struct crypto_fpu_ctx *ctx = crypto_skcipher_ctx(tfm); - struct crypto_skcipher *child = ctx->child; - SKCIPHER_REQUEST_ON_STACK(subreq, child); - int err; - - skcipher_request_set_tfm(subreq, child); - skcipher_request_set_callback(subreq, 0, NULL, NULL); - skcipher_request_set_crypt(subreq, req->src, req->dst, req->cryptlen, - req->iv); - - kernel_fpu_begin(); - err = crypto_skcipher_encrypt(subreq); - kernel_fpu_end(); - - skcipher_request_zero(subreq); - return err; -} - -static int crypto_fpu_decrypt(struct skcipher_request *req) -{ - struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req); - struct crypto_fpu_ctx *ctx = crypto_skcipher_ctx(tfm); - struct crypto_skcipher *child = ctx->child; - SKCIPHER_REQUEST_ON_STACK(subreq, child); - int err; - - skcipher_request_set_tfm(subreq, child); - skcipher_request_set_callback(subreq, 0, NULL, NULL); - skcipher_request_set_crypt(subreq, req->src, req->dst, req->cryptlen, - req->iv); - - kernel_fpu_begin(); - err = crypto_skcipher_decrypt(subreq); - kernel_fpu_end(); - - skcipher_request_zero(subreq); - return err; -} - -static int crypto_fpu_init_tfm(struct crypto_skcipher *tfm) -{ - struct skcipher_instance *inst = skcipher_alg_instance(tfm); - struct crypto_fpu_ctx *ctx = crypto_skcipher_ctx(tfm); - struct crypto_skcipher_spawn *spawn; - struct crypto_skcipher *cipher; - - spawn = skcipher_instance_ctx(inst); - cipher = crypto_spawn_skcipher(spawn); - if (IS_ERR(cipher)) - return PTR_ERR(cipher); - - ctx->child = cipher; - - return 0; -} - -static void crypto_fpu_exit_tfm(struct crypto_skcipher *tfm) -{ - struct crypto_fpu_ctx *ctx = crypto_skcipher_ctx(tfm); - - crypto_free_skcipher(ctx->child); -} - -static void crypto_fpu_free(struct skcipher_instance *inst) -{ - crypto_drop_skcipher(skcipher_instance_ctx(inst)); - kfree(inst); -} - -static int crypto_fpu_create(struct crypto_template *tmpl, struct rtattr **tb) -{ - struct crypto_skcipher_spawn *spawn; - struct skcipher_instance *inst; - struct crypto_attr_type *algt; - struct skcipher_alg *alg; - const char *cipher_name; - int err; - - algt = crypto_get_attr_type(tb); - if (IS_ERR(algt)) - return PTR_ERR(algt); - - if ((algt->type ^ (CRYPTO_ALG_INTERNAL | CRYPTO_ALG_TYPE_SKCIPHER)) & - algt->mask) - return -EINVAL; - - if (!(algt->mask & CRYPTO_ALG_INTERNAL)) - return -EINVAL; - - cipher_name = crypto_attr_alg_name(tb[1]); - if (IS_ERR(cipher_name)) - return PTR_ERR(cipher_name); - - inst = kzalloc(sizeof(*inst) + sizeof(*spawn), GFP_KERNEL); - if (!inst) - return -ENOMEM; - - spawn = skcipher_instance_ctx(inst); - - crypto_set_skcipher_spawn(spawn, skcipher_crypto_instance(inst)); - err = crypto_grab_skcipher(spawn, cipher_name, CRYPTO_ALG_INTERNAL, - CRYPTO_ALG_INTERNAL | CRYPTO_ALG_ASYNC); - if (err) - goto out_free_inst; - - alg = crypto_skcipher_spawn_alg(spawn); - - err = crypto_inst_setname(skcipher_crypto_instance(inst), "fpu", - &alg->base); - if (err) - goto out_drop_skcipher; - - inst->alg.base.cra_flags = CRYPTO_ALG_INTERNAL; - inst->alg.base.cra_priority = alg->base.cra_priority; - inst->alg.base.cra_blocksize = alg->base.cra_blocksize; - inst->alg.base.cra_alignmask = alg->base.cra_alignmask; - - inst->alg.ivsize = crypto_skcipher_alg_ivsize(alg); - inst->alg.min_keysize = crypto_skcipher_alg_min_keysize(alg); - inst->alg.max_keysize = crypto_skcipher_alg_max_keysize(alg); - - inst->alg.base.cra_ctxsize = sizeof(struct crypto_fpu_ctx); - - inst->alg.init = crypto_fpu_init_tfm; - inst->alg.exit = crypto_fpu_exit_tfm; - - inst->alg.setkey = crypto_fpu_setkey; - inst->alg.encrypt = crypto_fpu_encrypt; - inst->alg.decrypt = crypto_fpu_decrypt; - - inst->free = crypto_fpu_free; - - err = skcipher_register_instance(tmpl, inst); - if (err) - goto out_drop_skcipher; - -out: - return err; - -out_drop_skcipher: - crypto_drop_skcipher(spawn); -out_free_inst: - kfree(inst); - goto out; -} - -static struct crypto_template crypto_fpu_tmpl = { - .name = "fpu", - .create = crypto_fpu_create, - .module = THIS_MODULE, -}; - -int __init crypto_fpu_init(void) -{ - return crypto_register_template(&crypto_fpu_tmpl); -} - -void crypto_fpu_exit(void) -{ - crypto_unregister_template(&crypto_fpu_tmpl); -} - -MODULE_ALIAS_CRYPTO("fpu"); diff --git a/crypto/Kconfig b/crypto/Kconfig index f3e40ac56d93..a9c17ca77bb8 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -1133,7 +1133,7 @@ config CRYPTO_AES_NI_INTEL In addition to AES cipher algorithm support, the acceleration for some popular block cipher mode is supported too, including - ECB, CBC, LRW, PCBC, XTS. The 64 bit version has additional + ECB, CBC, LRW, XTS. The 64 bit version has additional acceleration for CTR. config CRYPTO_AES_SPARC64