From patchwork Sun Sep 29 17:38:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 174676 Delivered-To: patch@linaro.org Received: by 2002:a92:7e96:0:0:0:0:0 with SMTP id q22csp6171594ill; Sun, 29 Sep 2019 10:39:04 -0700 (PDT) X-Google-Smtp-Source: APXvYqy1b+0lXWKCyN7cs07eFTq7q57lBXVpptH+WNaJRrT9Edb59jSr7oVtZs60ucLVS8/2Ab25 X-Received: by 2002:a50:cf0d:: with SMTP id c13mr13547881edk.125.1569778744733; Sun, 29 Sep 2019 10:39:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1569778744; cv=none; d=google.com; s=arc-20160816; b=L+HHpRjnpzpeHWTMDwhUICtiu3wF7FXmLDriYNRXwERStQzhd4qWC0JjCybKCgC872 C1YPQ2szhkZ7vGIOJM7LsVyvQKCbU54Ix1d2Qpr6hjy9ySLWdUFyR+jD0UggkocvLv2d AE6zw3DdVv6fMVha403d7UhN8GgT400q/vWZclonBw9PuZWZ9Iql7mfJMAMI9XEbZhbm k2Sme1GFLOxMG1rBEcDNKCEvdlhAOqoOFOFJJFdQa6zP0Z0RzcVdCXh14kFctpO+UFgO C24PzpMF7g8df0r0g9Wac+OxU7gWa3yzhmGG1yaHJCerp7jDk0zKQmQWl+Fj5uN3CkYK reIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=EiwPtV0QFOq6zlLEHTm5HZSlYrdGW9J+wv2xPz8xyFY=; b=SYLnSzw5MoY/fuzKoaaIb9utTIxvBzFbksNrrgbqX+hjktZal/XYnScdJSSPjWozfa qcXC1jb2Cjs3Q4oPrnODv5wKFuvXY0vOirLVIrj5vvRuosRR2KCxSWX22USa/FFfhA1M 5B63F2JJYMaXU09h5qfeSkdTCVEILFhQYyrIqNcWydRNqWvlii32YDnHAtXX1Q2TFw0w VqQU1z+DaCl+eBiuvRrtC7X6qdYK6EzFwcDgEThbqOt0PE+hpfGpTtmi3rcadrnvum2t SJT544lFC2AtCUbwtyAGKk57Q4ZjT+dKEwap+8BNMhDqphX+OlST52XY1AjD0mCjU8ma IX2A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=C5im0cXQ; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a54si5497520edc.333.2019.09.29.10.39.04; Sun, 29 Sep 2019 10:39:04 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=C5im0cXQ; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730219AbfI2RjD (ORCPT + 3 others); Sun, 29 Sep 2019 13:39:03 -0400 Received: from mail-wm1-f65.google.com ([209.85.128.65]:35915 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729742AbfI2RjC (ORCPT ); Sun, 29 Sep 2019 13:39:02 -0400 Received: by mail-wm1-f65.google.com with SMTP id m18so10215381wmc.1 for ; Sun, 29 Sep 2019 10:38:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=EiwPtV0QFOq6zlLEHTm5HZSlYrdGW9J+wv2xPz8xyFY=; b=C5im0cXQNv3wGsmiCLsIcsFTaYYE97Ji1QEDLT98kH4bBwMlnuT0RmjtWkmdCyTFAi Jk1bPl29M1SEIu4iSZmeZl9Cufa8woVy6eFdZyngAXnJIwG7dlRHWzKOfpBOKQHjid38 qT4VdUp3zcBV/GlPcJAx0VgNAiprMHf3JzC1VMjXh+NDpyisW8D+saVmRZoBOGtV7use jlq9DwT1tLbnsPg6Bg4JAbr6pbmZlc+BvH36Mvx2YY+WrViQZqgbrk4y7qkRrOBLOnGz MIjzLs/4A83Jw6lrQVdIhbXaEW4X1TFhxRwqjZv/85fEOBPFa2i7P8tlxpVxQCaQmD9v zFYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=EiwPtV0QFOq6zlLEHTm5HZSlYrdGW9J+wv2xPz8xyFY=; b=nJQTzurRO2DtjaksPA/1tzz+/WZzN4/PJu+E905S4SmzmZvnopm4gver11TnJ3U8VO YIfNEok6ACndFTfkGrDGEktrhuKKY96hdGWArdbUIa4d2u/OsRq5J/dsqn6PoDkC3w1W +bMzsaLEwqEMYmL5vlAgcj8LDfnOE6WG1EmAiwU4QO4z1JEBDuhSrj5O3N9itjvyzY8d 0YjV7DkHPbnRFYPIkjgrLDzVpjflFNg6QK4HMwsiEZtS1dIad5gHKWex7HVSPfidh5l5 +xYhtQhfcydsXgUHRMApxj6F5lJnj44CLYcW/MwkUkWthlARJQmvqKQHlp2wNGdl2oEw PeNg== X-Gm-Message-State: APjAAAWXebtbmKg5gKMslWMh02uNyKJ4r21dbyeMVIlFb7BIuynlEjV8 bknLAiZd0Y2fyzlyy+PHmqPo8a9R8UMPsjzU X-Received: by 2002:a7b:c932:: with SMTP id h18mr11214396wml.86.1569778738649; Sun, 29 Sep 2019 10:38:58 -0700 (PDT) Received: from e123331-lin.nice.arm.com (bar06-5-82-246-156-241.fbx.proxad.net. [82.246.156.241]) by smtp.gmail.com with ESMTPSA id q192sm17339779wme.23.2019.09.29.10.38.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Sep 2019 10:38:57 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, Ard Biesheuvel , Herbert Xu , David Miller , Greg KH , Linus Torvalds , "Jason A . Donenfeld" , Samuel Neves , Dan Carpenter , Arnd Bergmann , Eric Biggers , Andy Lutomirski , Will Deacon , Marc Zyngier , Catalin Marinas , Martin Willi Subject: [RFC PATCH 01/20] crypto: chacha - move existing library code into lib/crypto Date: Sun, 29 Sep 2019 19:38:31 +0200 Message-Id: <20190929173850.26055-2-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190929173850.26055-1-ard.biesheuvel@linaro.org> References: <20190929173850.26055-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Move the existing shared ChaCha code into lib/crypto, and at the same time, split the support header into a public version, and an internal version that is only intended for consumption by crypto implementations. Also, refactor the generic implementation so it only gets exposed as the chacha_crypt() library function if the architecture does not override it with its own implementation, potentially falling back to the generic routine if needed. And while at it, tidy up lib/crypto/Makefile a bit so we are ready for some new arrivals. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/chacha-neon-glue.c | 2 +- arch/arm64/crypto/chacha-neon-glue.c | 2 +- arch/x86/crypto/chacha_glue.c | 2 +- crypto/Kconfig | 8 ++++ crypto/chacha_generic.c | 44 ++---------------- include/crypto/chacha.h | 49 +++++++++++++------- include/crypto/internal/chacha.h | 25 ++++++++++ lib/Makefile | 3 +- lib/crypto/Makefile | 19 ++++---- lib/{ => crypto}/chacha.c | 37 +++++++++++++-- 10 files changed, 118 insertions(+), 73 deletions(-) -- 2.17.1 diff --git a/arch/arm/crypto/chacha-neon-glue.c b/arch/arm/crypto/chacha-neon-glue.c index a8e9b534c8da..26576772f18b 100644 --- a/arch/arm/crypto/chacha-neon-glue.c +++ b/arch/arm/crypto/chacha-neon-glue.c @@ -20,7 +20,7 @@ */ #include -#include +#include #include #include #include diff --git a/arch/arm64/crypto/chacha-neon-glue.c b/arch/arm64/crypto/chacha-neon-glue.c index 1495d2b18518..d4cc61bfe79d 100644 --- a/arch/arm64/crypto/chacha-neon-glue.c +++ b/arch/arm64/crypto/chacha-neon-glue.c @@ -20,7 +20,7 @@ */ #include -#include +#include #include #include #include diff --git a/arch/x86/crypto/chacha_glue.c b/arch/x86/crypto/chacha_glue.c index 388f95a4ec24..bc62daa8dafd 100644 --- a/arch/x86/crypto/chacha_glue.c +++ b/arch/x86/crypto/chacha_glue.c @@ -7,7 +7,7 @@ */ #include -#include +#include #include #include #include diff --git a/crypto/Kconfig b/crypto/Kconfig index 9ea2b22aff90..5826381aca3a 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -1374,6 +1374,14 @@ config CRYPTO_SALSA20 The Salsa20 stream cipher algorithm is designed by Daniel J. Bernstein . See +config CRYPTO_ARCH_HAVE_LIB_CHACHA + tristate + +config CRYPTO_LIB_CHACHA + tristate + default y + depends on CRYPTO_ARCH_HAVE_LIB_CHACHA || !CRYPTO_ARCH_HAVE_LIB_CHACHA + config CRYPTO_CHACHA20 tristate "ChaCha stream cipher algorithms" select CRYPTO_BLKCIPHER diff --git a/crypto/chacha_generic.c b/crypto/chacha_generic.c index 085d8d219987..15a244e2f410 100644 --- a/crypto/chacha_generic.c +++ b/crypto/chacha_generic.c @@ -8,29 +8,10 @@ #include #include -#include +#include #include #include -static void chacha_docrypt(u32 *state, u8 *dst, const u8 *src, - unsigned int bytes, int nrounds) -{ - /* aligned to potentially speed up crypto_xor() */ - u8 stream[CHACHA_BLOCK_SIZE] __aligned(sizeof(long)); - - while (bytes >= CHACHA_BLOCK_SIZE) { - chacha_block(state, stream, nrounds); - crypto_xor_cpy(dst, src, stream, CHACHA_BLOCK_SIZE); - bytes -= CHACHA_BLOCK_SIZE; - dst += CHACHA_BLOCK_SIZE; - src += CHACHA_BLOCK_SIZE; - } - if (bytes) { - chacha_block(state, stream, nrounds); - crypto_xor_cpy(dst, src, stream, bytes); - } -} - static int chacha_stream_xor(struct skcipher_request *req, const struct chacha_ctx *ctx, const u8 *iv) { @@ -48,8 +29,8 @@ static int chacha_stream_xor(struct skcipher_request *req, if (nbytes < walk.total) nbytes = round_down(nbytes, CHACHA_BLOCK_SIZE); - chacha_docrypt(state, walk.dst.virt.addr, walk.src.virt.addr, - nbytes, ctx->nrounds); + chacha_crypt_generic(state, walk.dst.virt.addr, + walk.src.virt.addr, nbytes, ctx->nrounds); err = skcipher_walk_done(&walk, walk.nbytes - nbytes); } @@ -58,22 +39,7 @@ static int chacha_stream_xor(struct skcipher_request *req, void crypto_chacha_init(u32 *state, const struct chacha_ctx *ctx, const u8 *iv) { - state[0] = 0x61707865; /* "expa" */ - state[1] = 0x3320646e; /* "nd 3" */ - state[2] = 0x79622d32; /* "2-by" */ - state[3] = 0x6b206574; /* "te k" */ - state[4] = ctx->key[0]; - state[5] = ctx->key[1]; - state[6] = ctx->key[2]; - state[7] = ctx->key[3]; - state[8] = ctx->key[4]; - state[9] = ctx->key[5]; - state[10] = ctx->key[6]; - state[11] = ctx->key[7]; - state[12] = get_unaligned_le32(iv + 0); - state[13] = get_unaligned_le32(iv + 4); - state[14] = get_unaligned_le32(iv + 8); - state[15] = get_unaligned_le32(iv + 12); + chacha_init_generic(state, ctx->key, iv); } EXPORT_SYMBOL_GPL(crypto_chacha_init); @@ -126,7 +92,7 @@ int crypto_xchacha_crypt(struct skcipher_request *req) /* Compute the subkey given the original key and first 128 nonce bits */ crypto_chacha_init(state, ctx, req->iv); - hchacha_block(state, subctx.key, ctx->nrounds); + hchacha_block_generic(state, subctx.key, ctx->nrounds); subctx.nrounds = ctx->nrounds; /* Build the real IV */ diff --git a/include/crypto/chacha.h b/include/crypto/chacha.h index d1e723c6a37d..c29d8f7d69ed 100644 --- a/include/crypto/chacha.h +++ b/include/crypto/chacha.h @@ -15,9 +15,8 @@ #ifndef _CRYPTO_CHACHA_H #define _CRYPTO_CHACHA_H -#include +#include #include -#include /* 32-bit stream position, then 96-bit nonce (RFC7539 convention) */ #define CHACHA_IV_SIZE 16 @@ -29,26 +28,42 @@ /* 192-bit nonce, then 64-bit stream position */ #define XCHACHA_IV_SIZE 32 -struct chacha_ctx { - u32 key[8]; - int nrounds; -}; - -void chacha_block(u32 *state, u8 *stream, int nrounds); +void chacha_block_generic(u32 *state, u8 *stream, int nrounds); static inline void chacha20_block(u32 *state, u8 *stream) { - chacha_block(state, stream, 20); + chacha_block_generic(state, stream, 20); +} +void hchacha_block_generic(const u32 *in, u32 *out, int nrounds); + +static inline void chacha_init_generic(u32 *state, const u32 *key, const u8 *iv) +{ + state[0] = 0x61707865; /* "expa" */ + state[1] = 0x3320646e; /* "nd 3" */ + state[2] = 0x79622d32; /* "2-by" */ + state[3] = 0x6b206574; /* "te k" */ + state[4] = key[0]; + state[5] = key[1]; + state[6] = key[2]; + state[7] = key[3]; + state[8] = key[4]; + state[9] = key[5]; + state[10] = key[6]; + state[11] = key[7]; + state[12] = get_unaligned_le32(iv + 0); + state[13] = get_unaligned_le32(iv + 4); + state[14] = get_unaligned_le32(iv + 8); + state[15] = get_unaligned_le32(iv + 12); } -void hchacha_block(const u32 *in, u32 *out, int nrounds); -void crypto_chacha_init(u32 *state, const struct chacha_ctx *ctx, const u8 *iv); +static inline void chacha_init(u32 *state, const u32 *key, const u8 *iv) +{ + chacha_init_generic(state, key, iv); +} -int crypto_chacha20_setkey(struct crypto_skcipher *tfm, const u8 *key, - unsigned int keysize); -int crypto_chacha12_setkey(struct crypto_skcipher *tfm, const u8 *key, - unsigned int keysize); +void chacha_crypt(u32 *state, u8 *dst, const u8 *src, unsigned int bytes, + int nrounds); -int crypto_chacha_crypt(struct skcipher_request *req); -int crypto_xchacha_crypt(struct skcipher_request *req); +void chacha_crypt_generic(u32 *state, u8 *dst, const u8 *src, + unsigned int bytes, int nrounds); #endif /* _CRYPTO_CHACHA_H */ diff --git a/include/crypto/internal/chacha.h b/include/crypto/internal/chacha.h new file mode 100644 index 000000000000..f7ffe0f3fa47 --- /dev/null +++ b/include/crypto/internal/chacha.h @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _CRYPTO_INTERNAL_CHACHA_H +#define _CRYPTO_INTERNAL_CHACHA_H + +#include +#include +#include + +struct chacha_ctx { + u32 key[8]; + int nrounds; +}; + +void crypto_chacha_init(u32 *state, const struct chacha_ctx *ctx, const u8 *iv); + +int crypto_chacha20_setkey(struct crypto_skcipher *tfm, const u8 *key, + unsigned int keysize); +int crypto_chacha12_setkey(struct crypto_skcipher *tfm, const u8 *key, + unsigned int keysize); + +int crypto_chacha_crypt(struct skcipher_request *req); +int crypto_xchacha_crypt(struct skcipher_request *req); + +#endif /* _CRYPTO_CHACHA_H */ diff --git a/lib/Makefile b/lib/Makefile index 29c02a924973..1436c9608fdb 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -30,8 +30,7 @@ endif lib-y := ctype.o string.o vsprintf.o cmdline.o \ rbtree.o radix-tree.o timerqueue.o xarray.o \ - idr.o extable.o \ - sha1.o chacha.o irq_regs.o argv_split.o \ + idr.o extable.o sha1.o irq_regs.o argv_split.o \ flex_proportions.o ratelimit.o show_mem.o \ is_single_threaded.o plist.o decompress.o kobject_uevent.o \ earlycpio.o seq_buf.o siphash.o dec_and_lock.o \ diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile index cbe0b6a6450d..24dad058f2ae 100644 --- a/lib/crypto/Makefile +++ b/lib/crypto/Makefile @@ -1,13 +1,16 @@ # SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_CRYPTO_LIB_AES) += libaes.o -libaes-y := aes.o +# chacha is used by the /dev/random driver which is always builtin +obj-y += chacha.o -obj-$(CONFIG_CRYPTO_LIB_ARC4) += libarc4.o -libarc4-y := arc4.o +obj-$(CONFIG_CRYPTO_LIB_AES) += libaes.o +libaes-y := aes.o -obj-$(CONFIG_CRYPTO_LIB_DES) += libdes.o -libdes-y := des.o +obj-$(CONFIG_CRYPTO_LIB_ARC4) += libarc4.o +libarc4-y := arc4.o -obj-$(CONFIG_CRYPTO_LIB_SHA256) += libsha256.o -libsha256-y := sha256.o +obj-$(CONFIG_CRYPTO_LIB_DES) += libdes.o +libdes-y := des.o + +obj-$(CONFIG_CRYPTO_LIB_SHA256) += libsha256.o +libsha256-y := sha256.o diff --git a/lib/chacha.c b/lib/crypto/chacha.c similarity index 76% rename from lib/chacha.c rename to lib/crypto/chacha.c index c7c9826564d3..d4b7c391d934 100644 --- a/lib/chacha.c +++ b/lib/crypto/chacha.c @@ -5,11 +5,14 @@ * Copyright (C) 2015 Martin Willi */ +#include #include #include #include +#include #include #include +#include // for crypto_xor_cpy #include static void chacha_permute(u32 *x, int nrounds) @@ -72,7 +75,7 @@ static void chacha_permute(u32 *x, int nrounds) * The caller has already converted the endianness of the input. This function * also handles incrementing the block counter in the input matrix. */ -void chacha_block(u32 *state, u8 *stream, int nrounds) +void chacha_block_generic(u32 *state, u8 *stream, int nrounds) { u32 x[16]; int i; @@ -86,7 +89,7 @@ void chacha_block(u32 *state, u8 *stream, int nrounds) state[12]++; } -EXPORT_SYMBOL(chacha_block); +EXPORT_SYMBOL(chacha_block_generic); /** * hchacha_block - abbreviated ChaCha core, for XChaCha @@ -99,7 +102,7 @@ EXPORT_SYMBOL(chacha_block); * skips the final addition of the initial state, and outputs only certain words * of the state. It should not be used for streaming directly. */ -void hchacha_block(const u32 *in, u32 *out, int nrounds) +void hchacha_block_generic(const u32 *in, u32 *out, int nrounds) { u32 x[16]; @@ -110,4 +113,30 @@ void hchacha_block(const u32 *in, u32 *out, int nrounds) memcpy(&out[0], &x[0], 16); memcpy(&out[4], &x[12], 16); } -EXPORT_SYMBOL(hchacha_block); +EXPORT_SYMBOL(hchacha_block_generic); + +void chacha_crypt_generic(u32 *state, u8 *dst, const u8 *src, + unsigned int bytes, int nrounds) +{ + /* aligned to potentially speed up crypto_xor() */ + u8 stream[CHACHA_BLOCK_SIZE] __aligned(sizeof(long)); + + while (bytes >= CHACHA_BLOCK_SIZE) { + chacha_block_generic(state, stream, nrounds); + crypto_xor_cpy(dst, src, stream, CHACHA_BLOCK_SIZE); + bytes -= CHACHA_BLOCK_SIZE; + dst += CHACHA_BLOCK_SIZE; + src += CHACHA_BLOCK_SIZE; + } + if (bytes) { + chacha_block_generic(state, stream, nrounds); + crypto_xor_cpy(dst, src, stream, bytes); + } +} +EXPORT_SYMBOL(chacha_crypt_generic); + +#ifndef CONFIG_CRYPTO_ARCH_HAVE_LIB_CHACHA +extern void chacha_crypt(u32 *, u8 *, const u8 *, unsigned int, int) + __alias(chacha_crypt_generic); +EXPORT_SYMBOL(chacha_crypt); +#endif