From patchwork Thu Jun 27 10:26:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167898 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148358ilk; Thu, 27 Jun 2019 03:27:54 -0700 (PDT) X-Google-Smtp-Source: APXvYqzONAWdk58rgI12Jz7poDv1iNccTIk4w0g/HYTwdoBCPea5AoejuChZppfJ14qzYmGy8ew6 X-Received: by 2002:a63:2a83:: with SMTP id q125mr3075756pgq.102.1561631274355; Thu, 27 Jun 2019 03:27:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631274; cv=none; d=google.com; s=arc-20160816; b=W4DkTiViauN7bS/HHYJ9c1j/DfSlIJJKAZskCbW1Z9BtIZanku8Oh5vSQM0t60HD99 rRziVVHx1x9Tzp31bJ6Il/YBks//i1MGKwA1uAW2Yt+cNFDeY/dmT36WK7TMGeEvRXlH ekrdUqyPojhSWRkpVXnvToGXq31nAYxk4R3Olt74TSKBvTUcIZkoHsKCTc0a8F1o8A55 9e421CvxXkqFScQ9cVbs8eGE7thCEo0ZndwQDnxtPUmDlUQ2Kig/ow1mv0hsKTJVkGHT lguk4ggvacycsUSt/h3wDnBOF/zNIwCc6O0onbzK5s5E4He/6JADNAsb4G44kcLeyBKm l3mA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=BVjKB83CGDhuLXJgz703gQuRW2ORbNsUnSLW3NNqkVc=; b=myMv70e+k3hmXnJlZ8/ExcRBHekw82SeczWaVt/1xE/495dEBQQPEEwzqLwoyLO+fa iVfExKNIdtMc1RSS+F0NGnxi3YWMZGOtLCSvvDEJ4aIjZARFea6K4iJa/bmrqDMIN/Hh IzEREd6bbrR8eugVG1wa+X1aK1xSi/3Tf/hG/Gh/pf6tmuAy032LpQ4xtZR+8XuBnPvv pPQZCl5iEefPyT3UZtkeLP/afHo6LpEuuqmOKzF+oqExRG2m2Dxfcyhx4DM/R5V41TBQ AsxAi4mM0AdBHNyBoj99i8r3rq4I+w6jLt0Aaz7EVfFYLZHSjonjWHvaQkpK4LkYO0HT 6KrQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=pFRNGVBV; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.27.54; Thu, 27 Jun 2019 03:27:54 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=pFRNGVBV; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726370AbfF0K1x (ORCPT + 3 others); Thu, 27 Jun 2019 06:27:53 -0400 Received: from mail-wm1-f66.google.com ([209.85.128.66]:38051 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726657AbfF0K1x (ORCPT ); Thu, 27 Jun 2019 06:27:53 -0400 Received: by mail-wm1-f66.google.com with SMTP id s15so5103705wmj.3 for ; Thu, 27 Jun 2019 03:27:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BVjKB83CGDhuLXJgz703gQuRW2ORbNsUnSLW3NNqkVc=; b=pFRNGVBV903LkrX6Ej7fElal8/dsVoWDCuZybsQAlqT1+nIllH9Vc7gEtNmwwCpctg scw9hyKtMmF21FYPvrznEo/dLZrf3kCTfjrK9QYizKP6jJ/Y7SWmWUcEncfWsyg+nlA7 pzhRZacgY6+FMLHITkkEnhjOdpPdPcMlWQkhqgGIau2npN12wjlHLKtDSAonx1ytKPSk TZB/RBwHVML2sx5ulymyVcgK/UbXVZ4GJ7FbSnaG1B6PcV7CS2/EyvlKvMJgGnN9znlA fv5KnztOyXVaZPid25gPwoqNuI1u7/L4Fh5Id4BVtSZ7Y8tgv6jCUDr3y2+ZBNi+XOWS G4MA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BVjKB83CGDhuLXJgz703gQuRW2ORbNsUnSLW3NNqkVc=; b=Mca/JI58KrRY9qDw/wDSvn8BsTsL9LKuuQbwo72TK8efIpwea4XL8uahNQ0nu8F+cF ipng6LY5chF4l6LXyr7GjhvLjW5HobiL/HYxPyAw3xPzAqVTPzXMpGA7C6Zhoie95nDG 8GVyA9oj+B/zRu+tBs/VSLA46VJQk05yuV0PjSqUhnJyXKC54cnG5sXYcXomkFDsA8xP 2JSF4f15JB835vbLIRxwIzH2bUFBHiEOakZEQyqMaE97k5I6VXUtJtIWY5trvRzwfwKM V3aTW0J50CF0dsFxNpomRMMw2s+HNmcX6CUiqHAimpbSDK2IIewE50JTBK2fcty8rsq1 +Izg== X-Gm-Message-State: APjAAAW1WZfFmkqN2pMQqY5CPjcX2Vt49QU7z0WGsV894Y2fQbNJVTxg V1ZxgGwVWSfJqgDizA5I7GO2jki9aFY= X-Received: by 2002:a7b:cc16:: with SMTP id f22mr2553680wmh.115.1561631270702; Thu, 27 Jun 2019 03:27:50 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.27.48 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:27:49 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 01/32] crypto: arm/aes-ce - cosmetic/whitespace cleanup Date: Thu, 27 Jun 2019 12:26:16 +0200 Message-Id: <20190627102647.2992-2-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Rearrange the aes_algs[] array for legibility. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/aes-ce-glue.c | 116 ++++++++++---------- 1 file changed, 56 insertions(+), 60 deletions(-) -- 2.20.1 diff --git a/arch/arm/crypto/aes-ce-glue.c b/arch/arm/crypto/aes-ce-glue.c index 5affb8482379..04ba66903674 100644 --- a/arch/arm/crypto/aes-ce-glue.c +++ b/arch/arm/crypto/aes-ce-glue.c @@ -337,69 +337,65 @@ static int xts_decrypt(struct skcipher_request *req) } static struct skcipher_alg aes_algs[] = { { - .base = { - .cra_name = "__ecb(aes)", - .cra_driver_name = "__ecb-aes-ce", - .cra_priority = 300, - .cra_flags = CRYPTO_ALG_INTERNAL, - .cra_blocksize = AES_BLOCK_SIZE, - .cra_ctxsize = sizeof(struct crypto_aes_ctx), - .cra_module = THIS_MODULE, - }, - .min_keysize = AES_MIN_KEY_SIZE, - .max_keysize = AES_MAX_KEY_SIZE, - .setkey = ce_aes_setkey, - .encrypt = ecb_encrypt, - .decrypt = ecb_decrypt, + .base.cra_name = "__ecb(aes)", + .base.cra_driver_name = "__ecb-aes-ce", + .base.cra_priority = 300, + .base.cra_flags = CRYPTO_ALG_INTERNAL, + .base.cra_blocksize = AES_BLOCK_SIZE, + .base.cra_ctxsize = sizeof(struct crypto_aes_ctx), + .base.cra_module = THIS_MODULE, + + .min_keysize = AES_MIN_KEY_SIZE, + .max_keysize = AES_MAX_KEY_SIZE, + .setkey = ce_aes_setkey, + .encrypt = ecb_encrypt, + .decrypt = ecb_decrypt, }, { - .base = { - .cra_name = "__cbc(aes)", - .cra_driver_name = "__cbc-aes-ce", - .cra_priority = 300, - .cra_flags = CRYPTO_ALG_INTERNAL, - .cra_blocksize = AES_BLOCK_SIZE, - .cra_ctxsize = sizeof(struct crypto_aes_ctx), - .cra_module = THIS_MODULE, - }, - .min_keysize = AES_MIN_KEY_SIZE, - .max_keysize = AES_MAX_KEY_SIZE, - .ivsize = AES_BLOCK_SIZE, - .setkey = ce_aes_setkey, - .encrypt = cbc_encrypt, - .decrypt = cbc_decrypt, + .base.cra_name = "__cbc(aes)", + .base.cra_driver_name = "__cbc-aes-ce", + .base.cra_priority = 300, + .base.cra_flags = CRYPTO_ALG_INTERNAL, + .base.cra_blocksize = AES_BLOCK_SIZE, + .base.cra_ctxsize = sizeof(struct crypto_aes_ctx), + .base.cra_module = THIS_MODULE, + + .min_keysize = AES_MIN_KEY_SIZE, + .max_keysize = AES_MAX_KEY_SIZE, + .ivsize = AES_BLOCK_SIZE, + .setkey = ce_aes_setkey, + .encrypt = cbc_encrypt, + .decrypt = cbc_decrypt, }, { - .base = { - .cra_name = "__ctr(aes)", - .cra_driver_name = "__ctr-aes-ce", - .cra_priority = 300, - .cra_flags = CRYPTO_ALG_INTERNAL, - .cra_blocksize = 1, - .cra_ctxsize = sizeof(struct crypto_aes_ctx), - .cra_module = THIS_MODULE, - }, - .min_keysize = AES_MIN_KEY_SIZE, - .max_keysize = AES_MAX_KEY_SIZE, - .ivsize = AES_BLOCK_SIZE, - .chunksize = AES_BLOCK_SIZE, - .setkey = ce_aes_setkey, - .encrypt = ctr_encrypt, - .decrypt = ctr_encrypt, + .base.cra_name = "__ctr(aes)", + .base.cra_driver_name = "__ctr-aes-ce", + .base.cra_priority = 300, + .base.cra_flags = CRYPTO_ALG_INTERNAL, + .base.cra_blocksize = 1, + .base.cra_ctxsize = sizeof(struct crypto_aes_ctx), + .base.cra_module = THIS_MODULE, + + .min_keysize = AES_MIN_KEY_SIZE, + .max_keysize = AES_MAX_KEY_SIZE, + .ivsize = AES_BLOCK_SIZE, + .chunksize = AES_BLOCK_SIZE, + .setkey = ce_aes_setkey, + .encrypt = ctr_encrypt, + .decrypt = ctr_encrypt, }, { - .base = { - .cra_name = "__xts(aes)", - .cra_driver_name = "__xts-aes-ce", - .cra_priority = 300, - .cra_flags = CRYPTO_ALG_INTERNAL, - .cra_blocksize = AES_BLOCK_SIZE, - .cra_ctxsize = sizeof(struct crypto_aes_xts_ctx), - .cra_module = THIS_MODULE, - }, - .min_keysize = 2 * AES_MIN_KEY_SIZE, - .max_keysize = 2 * AES_MAX_KEY_SIZE, - .ivsize = AES_BLOCK_SIZE, - .setkey = xts_set_key, - .encrypt = xts_encrypt, - .decrypt = xts_decrypt, + .base.cra_name = "__xts(aes)", + .base.cra_driver_name = "__xts-aes-ce", + .base.cra_priority = 300, + .base.cra_flags = CRYPTO_ALG_INTERNAL, + .base.cra_blocksize = AES_BLOCK_SIZE, + .base.cra_ctxsize = sizeof(struct crypto_aes_xts_ctx), + .base.cra_module = THIS_MODULE, + + .min_keysize = 2 * AES_MIN_KEY_SIZE, + .max_keysize = 2 * AES_MAX_KEY_SIZE, + .ivsize = AES_BLOCK_SIZE, + .setkey = xts_set_key, + .encrypt = xts_encrypt, + .decrypt = xts_decrypt, } }; static struct simd_skcipher_alg *aes_simd_algs[ARRAY_SIZE(aes_algs)]; From patchwork Thu Jun 27 10:26:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167899 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148374ilk; Thu, 27 Jun 2019 03:27:55 -0700 (PDT) X-Google-Smtp-Source: APXvYqwKVZKPyDGnPxbEeIhhaIqW3GUvOxwqF668Ao0a4SCv6IkZTEYCkqRnsigWv8o7pxb9LQ5o X-Received: by 2002:a65:498f:: with SMTP id r15mr3186198pgs.37.1561631274936; Thu, 27 Jun 2019 03:27:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631274; cv=none; d=google.com; s=arc-20160816; b=b8fpUA1KKVswlZVeOwuuw4j/gK0Ko8nmcxE+Z62oQ/k+q616csvXvv25LnoU/bFAQO dmV/b3rgPtFtVsWBpsRBqWVsdeZKwS4fXI6swLPKgEbsfLjU50/yN7zADYyH/UN1TMah 9cBgdO/Fz4IZThu+KcFk1jy0LVpfozeOHtz9u3nqDzbMPEBMt4Hy5B6qwLPG5Xmy5KrB a7O3jj3rBlWWydpXLO3DBbvprxBzuBBzvxmApjZMQa+5ud7oFLyGyRAbov2uG3jPTe2c gxA1376NIZf60LGsLtnEeXQJSrafN1sftPS1wIbOz5eaWRdn/ovgGCrsRZ+DzYqxGJme s3jQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9jtH53Jk7KStQcdKvZdfqjCIjywTrKC4erKe/oL3K6E=; b=H6h+cjaxbYko+wnis1Phnoj/ICazI5HwgPVtr6+7pbxLE7RT6oC3XKq+xKkJ+x6I0W iPqGttMoaLUgwl7zvKK7lgjPSNstKFxykemOuRcVOEAWitPT/bxUctyqKhtWq3E2KRtG G9xJ447q+HSV8FLQwu+Cyieo72tcZRSwLnTyqsy5ANThB+As2lCIU6yv8mhzYbxK0PlU g1G8Cv0Le/N5WH4zZy8FU5/NAWdI7U/7NsYNjDiOO/AGrO+WxQIAJJczl2wykXSGhj3+ IIyFJaBvdEs5EU6qrN0TcGWMp5hgfgl5qG0o6PDxtp/uHIcCVxWp9S6/niwdsU4bU19a fGCw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=czGwJyXJ; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.27.54; Thu, 27 Jun 2019 03:27:54 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=czGwJyXJ; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726663AbfF0K1y (ORCPT + 3 others); Thu, 27 Jun 2019 06:27:54 -0400 Received: from mail-wr1-f67.google.com ([209.85.221.67]:38267 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726659AbfF0K1y (ORCPT ); Thu, 27 Jun 2019 06:27:54 -0400 Received: by mail-wr1-f67.google.com with SMTP id d18so1921486wrs.5 for ; Thu, 27 Jun 2019 03:27:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9jtH53Jk7KStQcdKvZdfqjCIjywTrKC4erKe/oL3K6E=; b=czGwJyXJ+zNVPQyK6xTMltzIoJ0kfvneThDNU1tLZ+bXDWbKeo0ksXTFAIedwNjgF+ h9h6Lhd9GwGzieKWif/HPYN9bTlF9ZeZ9dNMThONufUH1XKy5LMkE53FhDtMmY6/BRky EHGrmRKxnH+LbssJ0phJeeqANidGiQQXjUF4wL05n6nlpTxlbAVGTLT2dYuup5c1LZ8X K+fXRGJaE/STIMSQzI5E5ZbWngjzkhuDtBRzMliyJiKl0Kh/D9BZ4v9jAZw9ig050LLQ 6kkG2IDS3yCyeCoh6iqxeD6F2YE+rHcZxb6v3Eb/aTu8Q7dqL62h0rp1D1TcRvhQh1Xu ffUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9jtH53Jk7KStQcdKvZdfqjCIjywTrKC4erKe/oL3K6E=; b=GKoQiOP+KaO0u3WYnY5NYg0sGT9WQGs6zH61E5owwJ662kUrO6mqkJypmohiacWBcV VenzRm8YYOHue0bxpy8GJm8Zv6Gf0/hCIRRRm8nvk44sENUoVNwMHZpeyWRdMXz8bF3O WV7ZTfac1M+usJMlaE7nRRUzx+Cg8ptnLDN86OZeSK76BhH/X+AiPa8eSRU2bI0illEN MCOMFza8nOWh88pq82vhgwegMPO/idYEUdHrYyUyquFS1Zkdyq97Z0h5p1CkZwWz6S0W pztAEDkX199pgI9hBZrKHpqnZxwKDLao9HssnXkAlADOQBUXVHmRgNAMngVLLEFkkvOn EdzA== X-Gm-Message-State: APjAAAWYyXHUBlCMWeri75UfeLSmuk/NAjrMY2NB6rKg+DPDkK88LLQk xYmXnBpo2d5J+3sm6Bki0ADt/vA5O+M= X-Received: by 2002:a05:6000:11c2:: with SMTP id i2mr2723505wrx.199.1561631272143; Thu, 27 Jun 2019 03:27:52 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.27.50 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:27:51 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 02/32] crypto: aes - rename local routines to prevent future clashes Date: Thu, 27 Jun 2019 12:26:17 +0200 Message-Id: <20190627102647.2992-3-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Rename some local AES encrypt/decrypt routines so they don't clash with the names we are about to introduce for the routines exposed by the generic AES library. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/aes-cipher-glue.c | 8 ++++---- arch/arm64/crypto/aes-cipher-glue.c | 8 ++++---- arch/x86/crypto/aesni-intel_glue.c | 8 ++++---- crypto/aes_generic.c | 8 ++++---- drivers/crypto/padlock-aes.c | 8 ++++---- 5 files changed, 20 insertions(+), 20 deletions(-) -- 2.20.1 diff --git a/arch/arm/crypto/aes-cipher-glue.c b/arch/arm/crypto/aes-cipher-glue.c index c222f6e072ad..f6c07867b8ff 100644 --- a/arch/arm/crypto/aes-cipher-glue.c +++ b/arch/arm/crypto/aes-cipher-glue.c @@ -19,7 +19,7 @@ EXPORT_SYMBOL(__aes_arm_encrypt); asmlinkage void __aes_arm_decrypt(u32 *rk, int rounds, const u8 *in, u8 *out); EXPORT_SYMBOL(__aes_arm_decrypt); -static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void aes_arm_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); int rounds = 6 + ctx->key_length / 4; @@ -27,7 +27,7 @@ static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) __aes_arm_encrypt(ctx->key_enc, rounds, in, out); } -static void aes_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void aes_arm_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); int rounds = 6 + ctx->key_length / 4; @@ -47,8 +47,8 @@ static struct crypto_alg aes_alg = { .cra_cipher.cia_min_keysize = AES_MIN_KEY_SIZE, .cra_cipher.cia_max_keysize = AES_MAX_KEY_SIZE, .cra_cipher.cia_setkey = crypto_aes_set_key, - .cra_cipher.cia_encrypt = aes_encrypt, - .cra_cipher.cia_decrypt = aes_decrypt, + .cra_cipher.cia_encrypt = aes_arm_encrypt, + .cra_cipher.cia_decrypt = aes_arm_decrypt, #ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS .cra_alignmask = 3, diff --git a/arch/arm64/crypto/aes-cipher-glue.c b/arch/arm64/crypto/aes-cipher-glue.c index 7288e7cbebff..0e90b06ebcec 100644 --- a/arch/arm64/crypto/aes-cipher-glue.c +++ b/arch/arm64/crypto/aes-cipher-glue.c @@ -18,7 +18,7 @@ EXPORT_SYMBOL(__aes_arm64_encrypt); asmlinkage void __aes_arm64_decrypt(u32 *rk, u8 *out, const u8 *in, int rounds); EXPORT_SYMBOL(__aes_arm64_decrypt); -static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void aes_arm64_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); int rounds = 6 + ctx->key_length / 4; @@ -26,7 +26,7 @@ static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) __aes_arm64_encrypt(ctx->key_enc, out, in, rounds); } -static void aes_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void aes_arm64_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); int rounds = 6 + ctx->key_length / 4; @@ -46,8 +46,8 @@ static struct crypto_alg aes_alg = { .cra_cipher.cia_min_keysize = AES_MIN_KEY_SIZE, .cra_cipher.cia_max_keysize = AES_MAX_KEY_SIZE, .cra_cipher.cia_setkey = crypto_aes_set_key, - .cra_cipher.cia_encrypt = aes_encrypt, - .cra_cipher.cia_decrypt = aes_decrypt + .cra_cipher.cia_encrypt = aes_arm64_encrypt, + .cra_cipher.cia_decrypt = aes_arm64_decrypt }; static int __init aes_init(void) diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c index c95bd397dc07..836d50bd096f 100644 --- a/arch/x86/crypto/aesni-intel_glue.c +++ b/arch/x86/crypto/aesni-intel_glue.c @@ -349,7 +349,7 @@ static int aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, return aes_set_key_common(tfm, crypto_tfm_ctx(tfm), in_key, key_len); } -static void aes_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) +static void aesni_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) { struct crypto_aes_ctx *ctx = aes_ctx(crypto_tfm_ctx(tfm)); @@ -362,7 +362,7 @@ static void aes_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) } } -static void aes_decrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) +static void aesni_decrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) { struct crypto_aes_ctx *ctx = aes_ctx(crypto_tfm_ctx(tfm)); @@ -923,8 +923,8 @@ static struct crypto_alg aesni_cipher_alg = { .cia_min_keysize = AES_MIN_KEY_SIZE, .cia_max_keysize = AES_MAX_KEY_SIZE, .cia_setkey = aes_set_key, - .cia_encrypt = aes_encrypt, - .cia_decrypt = aes_decrypt + .cia_encrypt = aesni_encrypt, + .cia_decrypt = aesni_decrypt } } }; diff --git a/crypto/aes_generic.c b/crypto/aes_generic.c index f217568917e4..3aa4a715c216 100644 --- a/crypto/aes_generic.c +++ b/crypto/aes_generic.c @@ -1332,7 +1332,7 @@ EXPORT_SYMBOL_GPL(crypto_aes_set_key); f_rl(bo, bi, 3, k); \ } while (0) -static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void crypto_aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { const struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); u32 b0[4], b1[4]; @@ -1402,7 +1402,7 @@ static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) i_rl(bo, bi, 3, k); \ } while (0) -static void aes_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void crypto_aes_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { const struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); u32 b0[4], b1[4]; @@ -1454,8 +1454,8 @@ static struct crypto_alg aes_alg = { .cia_min_keysize = AES_MIN_KEY_SIZE, .cia_max_keysize = AES_MAX_KEY_SIZE, .cia_setkey = crypto_aes_set_key, - .cia_encrypt = aes_encrypt, - .cia_decrypt = aes_decrypt + .cia_encrypt = crypto_aes_encrypt, + .cia_decrypt = crypto_aes_decrypt } } }; diff --git a/drivers/crypto/padlock-aes.c b/drivers/crypto/padlock-aes.c index 09d823d36d3a..854539512c35 100644 --- a/drivers/crypto/padlock-aes.c +++ b/drivers/crypto/padlock-aes.c @@ -299,7 +299,7 @@ static inline u8 *padlock_xcrypt_cbc(const u8 *input, u8 *output, void *key, return iv; } -static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void padlock_aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { struct aes_ctx *ctx = aes_ctx(tfm); @@ -308,7 +308,7 @@ static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) padlock_store_cword(&ctx->cword.encrypt); } -static void aes_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void padlock_aes_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { struct aes_ctx *ctx = aes_ctx(tfm); @@ -331,8 +331,8 @@ static struct crypto_alg aes_alg = { .cia_min_keysize = AES_MIN_KEY_SIZE, .cia_max_keysize = AES_MAX_KEY_SIZE, .cia_setkey = aes_set_key, - .cia_encrypt = aes_encrypt, - .cia_decrypt = aes_decrypt, + .cia_encrypt = padlock_aes_encrypt, + .cia_decrypt = padlock_aes_decrypt, } } }; From patchwork Thu Jun 27 10:26:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167900 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148405ilk; Thu, 27 Jun 2019 03:27:56 -0700 (PDT) X-Google-Smtp-Source: APXvYqwhcQNeb5RSiXbHAu4LBQDOUUHXU7ubONbzIxoKKevqDiqschRT+1iy9vYdX7gGSRppsBBC X-Received: by 2002:a17:902:8649:: with SMTP id y9mr3697854plt.289.1561631276889; Thu, 27 Jun 2019 03:27:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631276; cv=none; d=google.com; s=arc-20160816; b=MqMYHJ1GRLeVQH9T9dH4UYmD878dXY+Zqfmna1hL/gEN6+kdmXGhovTpfkQ1Q3shky BLuT9JNVQjUpto8ruO6+CiW5Y0zpXkRRMyiEikoAbVRYX/opjhSdwkwYgulVcYnP2cTf 4yi05cQas6S8+ELyNcjb7IAOAOX87d/a3VPfog15dM2ukuQOgMNsVYuuS2Jed5sM8UO9 DZ4jFtxoDSy/OVqPUaHqGWHiVHZKQXexKyQ0hjQuv2lvCIinNRXw5oZffEo1JqT74gTN c7HqK3pWs3oJyraPck50D5DbiScvDkAJKZpexPZMk2F/UH/eD+wCHAKl2TNSt6bb1Usg cpog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=OVlu3R/XDhYHCSEro6NMIXQLAFRHmhGDxGC17BFWD8A=; b=Hhk5Ev4G8o2Y3U2P0udpiyQuS1geATR1aEyzNclcORgf/HY7kQLSzYhOQtCB3wnrcM Gs6TGyqfY+d9XzoMgFGaKHGp4oAbNte6aQr95Gr1QMfL/pNNfW0ctjQsSLrDt1wFYrBu xGj55GUc/8kjBQ+606iJTkd/xSivJ8r6/+2M7BbkG/7zQvWQkJtk5U4VN0NMrKcDbXff SvN/QAnvLY4BrtU4eic7xrw8qcFnoiU4QNJVPhaU8q4nS0Cz6VUivTlzfScLLu9ff9jL cIiWBuTlfvv+J55hjnDMhUtRRaIrBOpAuua5oEWrXqa/71n58/9PMMYBL8kQvBC80Ja9 gJCw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=UJ27Ao12; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.27.56; Thu, 27 Jun 2019 03:27:56 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=UJ27Ao12; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726660AbfF0K14 (ORCPT + 3 others); Thu, 27 Jun 2019 06:27:56 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:34849 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726657AbfF0K14 (ORCPT ); Thu, 27 Jun 2019 06:27:56 -0400 Received: by mail-wr1-f68.google.com with SMTP id f15so1937981wrp.2 for ; Thu, 27 Jun 2019 03:27:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OVlu3R/XDhYHCSEro6NMIXQLAFRHmhGDxGC17BFWD8A=; b=UJ27Ao12RucOZXSw7oZqhFXuwtj3+u7hH2ANLe2rfHQwRMH/1ldc0qpLpuS4eP9iya zBtTzoESXoo1f52xoW7FCSaJv2m77g6Hk7g3nuR13HKWQmI2D++4OxHewIuI4zGP5vjv akUUab84w915ev/cn/DxbVI/Pu1z93Qlu6xDF2+HREYKDghT5TkGKTNd1sKjoiF+Jqld wt/qpE3LXtQ+FTFiyn3kK8qGb7TSwRowCwRd9hBgMAwz/kXeOZU8yJjVyyOOsXESu0dr MRopa9SEmtJ9evKMw3yQ5LLNH2l2imot8rhSFikNlm0qS+DzwIbnEUx2OsiwHUqC1Ndh JVnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OVlu3R/XDhYHCSEro6NMIXQLAFRHmhGDxGC17BFWD8A=; b=JjfDvGdXo74HEdPI59FCWE09FGz5hfy6OheMU8bEPnR5KRZjxS5X3so7BwYdAhEJk7 ln4L7mFeyEEzopzoBZ1fVGqx36Q7Z9bQpdngxAIOoc5J7J+77AodQQzH/JDXQ41pAvEo p2Wy49fbuwvkj5BNC6dmNoBI5nV3HhfhY7D4rxXa8dpRVm5pGod/LefHfI6NfJwU/yg2 2tAmSUUT5xWMgrpaEcopQPghJiX0VTVGWYmXW9aU46p/1COBD3CQhE+E6KYl2K905rAj JeG4cL8oGEScuXpk2QavR/1M4vrR69+EGWaV/S1r8vLPfZcl8LlpfNChXreGDzho+84Y 3xlQ== X-Gm-Message-State: APjAAAUG9K2q6531sv8TEp70RWySB/a2QMJaT5rhol+tlNINPwdrcQB0 spwf9fNYa2c2pUGdU9uTspKulSGB/ao= X-Received: by 2002:adf:c654:: with SMTP id u20mr2782163wrg.271.1561631273149; Thu, 27 Jun 2019 03:27:53 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.27.52 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:27:52 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 03/32] crypto: aes/fixed-time - align key schedule with other implementations Date: Thu, 27 Jun 2019 12:26:18 +0200 Message-Id: <20190627102647.2992-4-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The fixed time AES code mangles the key schedule so that xoring the first round key with values at fixed offsets across the Sbox produces the correct value. This primes the D-cache with the entire Sbox before any data dependent lookups are done, making it more difficult to infer key bits from timing variances when the plaintext is known. The downside of this approach is that it renders the key schedule incompatible with other implementations of AES in the kernel, which makes it cumbersome to use this implementation as a fallback for SIMD based AES in contexts where this is not allowed. So let's tweak the fixed Sbox indexes so that they add up to zero under the xor operation. While at it, increase the granularity to 16 bytes so we cover the entire Sbox even on systems with 16 byte cachelines. Signed-off-by: Ard Biesheuvel --- crypto/aes_ti.c | 52 ++++++++------------ 1 file changed, 21 insertions(+), 31 deletions(-) -- 2.20.1 diff --git a/crypto/aes_ti.c b/crypto/aes_ti.c index 1ff9785b30f5..fd70dc322634 100644 --- a/crypto/aes_ti.c +++ b/crypto/aes_ti.c @@ -237,30 +237,8 @@ static int aesti_set_key(struct crypto_tfm *tfm, const u8 *in_key, unsigned int key_len) { struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); - int err; - err = aesti_expand_key(ctx, in_key, key_len); - if (err) - return err; - - /* - * In order to force the compiler to emit data independent Sbox lookups - * at the start of each block, xor the first round key with values at - * fixed indexes in the Sbox. This will need to be repeated each time - * the key is used, which will pull the entire Sbox into the D-cache - * before any data dependent Sbox lookups are performed. - */ - ctx->key_enc[0] ^= __aesti_sbox[ 0] ^ __aesti_sbox[128]; - ctx->key_enc[1] ^= __aesti_sbox[32] ^ __aesti_sbox[160]; - ctx->key_enc[2] ^= __aesti_sbox[64] ^ __aesti_sbox[192]; - ctx->key_enc[3] ^= __aesti_sbox[96] ^ __aesti_sbox[224]; - - ctx->key_dec[0] ^= __aesti_inv_sbox[ 0] ^ __aesti_inv_sbox[128]; - ctx->key_dec[1] ^= __aesti_inv_sbox[32] ^ __aesti_inv_sbox[160]; - ctx->key_dec[2] ^= __aesti_inv_sbox[64] ^ __aesti_inv_sbox[192]; - ctx->key_dec[3] ^= __aesti_inv_sbox[96] ^ __aesti_inv_sbox[224]; - - return 0; + return aesti_expand_key(ctx, in_key, key_len); } static void aesti_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) @@ -283,10 +261,16 @@ static void aesti_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) */ local_irq_save(flags); - st0[0] ^= __aesti_sbox[ 0] ^ __aesti_sbox[128]; - st0[1] ^= __aesti_sbox[32] ^ __aesti_sbox[160]; - st0[2] ^= __aesti_sbox[64] ^ __aesti_sbox[192]; - st0[3] ^= __aesti_sbox[96] ^ __aesti_sbox[224]; + /* + * Force the compiler to emit data independent Sbox references, + * by xoring the input with Sbox values that are known to add up + * to zero. This pulls the entire Sbox into the D-cache before any + * data dependent lookups are done. + */ + st0[0] ^= __aesti_sbox[ 0] ^ __aesti_sbox[ 64] ^ __aesti_sbox[134] ^ __aesti_sbox[195]; + st0[1] ^= __aesti_sbox[16] ^ __aesti_sbox[ 82] ^ __aesti_sbox[158] ^ __aesti_sbox[221]; + st0[2] ^= __aesti_sbox[32] ^ __aesti_sbox[ 96] ^ __aesti_sbox[160] ^ __aesti_sbox[234]; + st0[3] ^= __aesti_sbox[48] ^ __aesti_sbox[112] ^ __aesti_sbox[186] ^ __aesti_sbox[241]; for (round = 0;; round += 2, rkp += 8) { st1[0] = mix_columns(subshift(st0, 0)) ^ rkp[0]; @@ -331,10 +315,16 @@ static void aesti_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) */ local_irq_save(flags); - st0[0] ^= __aesti_inv_sbox[ 0] ^ __aesti_inv_sbox[128]; - st0[1] ^= __aesti_inv_sbox[32] ^ __aesti_inv_sbox[160]; - st0[2] ^= __aesti_inv_sbox[64] ^ __aesti_inv_sbox[192]; - st0[3] ^= __aesti_inv_sbox[96] ^ __aesti_inv_sbox[224]; + /* + * Force the compiler to emit data independent Sbox references, + * by xoring the input with Sbox values that are known to add up + * to zero. This pulls the entire Sbox into the D-cache before any + * data dependent lookups are done. + */ + st0[0] ^= __aesti_inv_sbox[ 0] ^ __aesti_inv_sbox[ 64] ^ __aesti_inv_sbox[129] ^ __aesti_inv_sbox[200]; + st0[1] ^= __aesti_inv_sbox[16] ^ __aesti_inv_sbox[ 83] ^ __aesti_inv_sbox[150] ^ __aesti_inv_sbox[212]; + st0[2] ^= __aesti_inv_sbox[32] ^ __aesti_inv_sbox[ 96] ^ __aesti_inv_sbox[160] ^ __aesti_inv_sbox[236]; + st0[3] ^= __aesti_inv_sbox[48] ^ __aesti_inv_sbox[112] ^ __aesti_inv_sbox[187] ^ __aesti_inv_sbox[247]; for (round = 0;; round += 2, rkp += 8) { st1[0] = inv_mix_columns(inv_subshift(st0, 0)) ^ rkp[0]; From patchwork Thu Jun 27 10:26:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167902 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148469ilk; Thu, 27 Jun 2019 03:28:00 -0700 (PDT) X-Google-Smtp-Source: APXvYqyIDzsCrAS0tUnmXPKULD01C8yv2z0QBthWwjwvBXjnQWuwAdlR+FOxhLAWOIdg5QflwKfP X-Received: by 2002:a63:5158:: with SMTP id r24mr3046149pgl.79.1561631279872; Thu, 27 Jun 2019 03:27:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631279; cv=none; d=google.com; s=arc-20160816; b=G3xslRk/seFGRH7rKsSszK6DbX1VA0jRnRtQwvPVFUj7WYR2JVUGbmt6+R6fzYzjZi Uf71HCMlitW5izYF/a9dz9RZO4+QnaxIt4N+RlG7uCXMEayCuvYJ+SsWnaMALKiSiDE+ YFG/yTwwj2qNvY4V4r6LrW+uVSMFa3mmHS5UkEbhEcgCRGtDHuzWPaxNIm8AhXs9NTTs YxL/GrZZRm5AyLRGL2E0PUup0XMLA92H9miXXMNHhr6eIX/CyOXL+bN6UWUOoJVpEinL O2ey2DScXOix+Vh7TYfBjBVXnzgbY/Sriy9NbeIVa6Bqqbmg9Ivx+tywgHA/3bJeGgGJ 4kVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=JYfyuXtkWyqjgUR6IexmK+iDcDL5/HP/psv9afBMZ2s=; b=IxQd3yzUTEWQiL9h6UoR7xEwqzhESQ3u7viKYYWyvrXwLFJeVec2cumgMWaVbwmzjN 0pdK6zq3c/BhVJewSpNRtkwYe7AZPis4e5b/7LopBdg0O0Sx1oxoOmsEgd2I7QOjoeEA jAvQ3/JuvLXjR4/a13DMKqz/PCi4lEmupNgEdE6NvxyFyQtFbSvknTbt0DaLhnxBkvYj Qlijm5L6oRHCI5dqVMfiW7kj8IiZMntsBxr4lp8JXgnSpuTDyzWO/4fjKNDIaJVJXoqp hrDLJ8iFZR9n/bMJ4zrDrmUG7mra6kv84geDSQG8wml7lP/LTdfx686RPiGsA/9mEFGy XNoA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rNl2+eFQ; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.27.59; Thu, 27 Jun 2019 03:27:59 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rNl2+eFQ; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726657AbfF0K17 (ORCPT + 3 others); Thu, 27 Jun 2019 06:27:59 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:42864 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726659AbfF0K16 (ORCPT ); Thu, 27 Jun 2019 06:27:58 -0400 Received: by mail-wr1-f68.google.com with SMTP id x17so1906689wrl.9 for ; Thu, 27 Jun 2019 03:27:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JYfyuXtkWyqjgUR6IexmK+iDcDL5/HP/psv9afBMZ2s=; b=rNl2+eFQjBju+QmDCensdOowWubE0/reIt2FWtNLy0+zlTyARcan1bd52vmwU2P0a5 23LUwYqd11eDXZ9RmAKdvMxzHRsq3s+11iN8nu5qbMyL7T5rzwclSSJf/k7J5VNqFaci WS31RiXxUchxmH00q3FZVpY3CY2ceQa8Z6qc+RW+4LPjyipVFlIaNWYMmaHD7c8GWDDg 0H2SmXga1lCKWTF2Pzz8EUeu7IR5arIok/7Qb3IDZVNi2LKrzC6AE7XEWJ/DDHLc+T25 u1lbb8mqz7BTp88lAN+MeMm8JvxemuqX0xGfw1SZchFk3e0gKlYgRujiGntRLITGgl+R FxIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JYfyuXtkWyqjgUR6IexmK+iDcDL5/HP/psv9afBMZ2s=; b=WYl0KQjeNTuenk3POtqffmFuLN/DtDqoOSNWNy4FGqSykbzoUSLnck+90iClA6YKvB 5lpXnNBOILlmDC3y+kh4ASeHNFer1iHJ3pU29p4uSMqSsnM0ua/GgKcMikUNb6dFO8cn g+SfnE1UgdJ/if7zwkdhgDMb5w3+sYHwPfWozSC3tDqqt7PEb92k2ZQOuUm/oUbvRwBc +xpRznSoi90XMEVp3hMzCyVsYKxH9EpTCzR7UZ4YdFkHYCE8ZlmkuvtXJbxekf4gDhDW pYKtmURM7Az2ajevRE1PufR9bAtDckx1TUDzu//c3Z9jcrpbUdaG9ZeqH13wlt8Bcpxm SEOg== X-Gm-Message-State: APjAAAUobErQ91dIEZ+TF+1S+7/h9H5hO8GR43ybgm4RVbLZS417mBXE CxcMCfnP68KB51qJTU4H1ZJBogKd2Gc= X-Received: by 2002:a05:6000:1c9:: with SMTP id t9mr2792693wrx.187.1561631274382; Thu, 27 Jun 2019 03:27:54 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.27.53 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:27:53 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 04/32] crypto: aes - create AES library based on the fixed time AES code Date: Thu, 27 Jun 2019 12:26:19 +0200 Message-Id: <20190627102647.2992-5-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Take the existing small footprint and mostly time invariant C code and turn it into a AES library that can be used for non-performance critical, casual use of AES, and as a fallback for, e.g., SIMD code that needs a secondary path that can be taken in contexts where the SIMD unit is off limits (e.g., in hard interrupts taken from kernel context) Signed-off-by: Ard Biesheuvel --- crypto/Kconfig | 4 + crypto/aes_ti.c | 307 +---------------- include/crypto/aes.h | 34 ++ lib/crypto/Makefile | 3 + lib/crypto/aes.c | 350 ++++++++++++++++++++ 5 files changed, 395 insertions(+), 303 deletions(-) -- 2.20.1 diff --git a/crypto/Kconfig b/crypto/Kconfig index e801450bcb1c..091ebbbc9655 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -1066,6 +1066,9 @@ config CRYPTO_GHASH_CLMUL_NI_INTEL comment "Ciphers" +config CRYPTO_LIB_AES + tristate + config CRYPTO_AES tristate "AES cipher algorithms" select CRYPTO_ALGAPI @@ -1089,6 +1092,7 @@ config CRYPTO_AES config CRYPTO_AES_TI tristate "Fixed time AES cipher" select CRYPTO_ALGAPI + select CRYPTO_LIB_AES help This is a generic implementation of AES that attempts to eliminate data dependent latencies as much as possible without affecting diff --git a/crypto/aes_ti.c b/crypto/aes_ti.c index fd70dc322634..339915db9aeb 100644 --- a/crypto/aes_ti.c +++ b/crypto/aes_ti.c @@ -1,259 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0 /* * Scalar fixed time AES core transform * * Copyright (C) 2017 Linaro Ltd - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. */ #include #include #include -#include - -/* - * Emit the sbox as volatile const to prevent the compiler from doing - * constant folding on sbox references involving fixed indexes. - */ -static volatile const u8 __cacheline_aligned __aesti_sbox[] = { - 0x63, 0x7c, 0x77, 0x7b, 0xf2, 0x6b, 0x6f, 0xc5, - 0x30, 0x01, 0x67, 0x2b, 0xfe, 0xd7, 0xab, 0x76, - 0xca, 0x82, 0xc9, 0x7d, 0xfa, 0x59, 0x47, 0xf0, - 0xad, 0xd4, 0xa2, 0xaf, 0x9c, 0xa4, 0x72, 0xc0, - 0xb7, 0xfd, 0x93, 0x26, 0x36, 0x3f, 0xf7, 0xcc, - 0x34, 0xa5, 0xe5, 0xf1, 0x71, 0xd8, 0x31, 0x15, - 0x04, 0xc7, 0x23, 0xc3, 0x18, 0x96, 0x05, 0x9a, - 0x07, 0x12, 0x80, 0xe2, 0xeb, 0x27, 0xb2, 0x75, - 0x09, 0x83, 0x2c, 0x1a, 0x1b, 0x6e, 0x5a, 0xa0, - 0x52, 0x3b, 0xd6, 0xb3, 0x29, 0xe3, 0x2f, 0x84, - 0x53, 0xd1, 0x00, 0xed, 0x20, 0xfc, 0xb1, 0x5b, - 0x6a, 0xcb, 0xbe, 0x39, 0x4a, 0x4c, 0x58, 0xcf, - 0xd0, 0xef, 0xaa, 0xfb, 0x43, 0x4d, 0x33, 0x85, - 0x45, 0xf9, 0x02, 0x7f, 0x50, 0x3c, 0x9f, 0xa8, - 0x51, 0xa3, 0x40, 0x8f, 0x92, 0x9d, 0x38, 0xf5, - 0xbc, 0xb6, 0xda, 0x21, 0x10, 0xff, 0xf3, 0xd2, - 0xcd, 0x0c, 0x13, 0xec, 0x5f, 0x97, 0x44, 0x17, - 0xc4, 0xa7, 0x7e, 0x3d, 0x64, 0x5d, 0x19, 0x73, - 0x60, 0x81, 0x4f, 0xdc, 0x22, 0x2a, 0x90, 0x88, - 0x46, 0xee, 0xb8, 0x14, 0xde, 0x5e, 0x0b, 0xdb, - 0xe0, 0x32, 0x3a, 0x0a, 0x49, 0x06, 0x24, 0x5c, - 0xc2, 0xd3, 0xac, 0x62, 0x91, 0x95, 0xe4, 0x79, - 0xe7, 0xc8, 0x37, 0x6d, 0x8d, 0xd5, 0x4e, 0xa9, - 0x6c, 0x56, 0xf4, 0xea, 0x65, 0x7a, 0xae, 0x08, - 0xba, 0x78, 0x25, 0x2e, 0x1c, 0xa6, 0xb4, 0xc6, - 0xe8, 0xdd, 0x74, 0x1f, 0x4b, 0xbd, 0x8b, 0x8a, - 0x70, 0x3e, 0xb5, 0x66, 0x48, 0x03, 0xf6, 0x0e, - 0x61, 0x35, 0x57, 0xb9, 0x86, 0xc1, 0x1d, 0x9e, - 0xe1, 0xf8, 0x98, 0x11, 0x69, 0xd9, 0x8e, 0x94, - 0x9b, 0x1e, 0x87, 0xe9, 0xce, 0x55, 0x28, 0xdf, - 0x8c, 0xa1, 0x89, 0x0d, 0xbf, 0xe6, 0x42, 0x68, - 0x41, 0x99, 0x2d, 0x0f, 0xb0, 0x54, 0xbb, 0x16, -}; - -static volatile const u8 __cacheline_aligned __aesti_inv_sbox[] = { - 0x52, 0x09, 0x6a, 0xd5, 0x30, 0x36, 0xa5, 0x38, - 0xbf, 0x40, 0xa3, 0x9e, 0x81, 0xf3, 0xd7, 0xfb, - 0x7c, 0xe3, 0x39, 0x82, 0x9b, 0x2f, 0xff, 0x87, - 0x34, 0x8e, 0x43, 0x44, 0xc4, 0xde, 0xe9, 0xcb, - 0x54, 0x7b, 0x94, 0x32, 0xa6, 0xc2, 0x23, 0x3d, - 0xee, 0x4c, 0x95, 0x0b, 0x42, 0xfa, 0xc3, 0x4e, - 0x08, 0x2e, 0xa1, 0x66, 0x28, 0xd9, 0x24, 0xb2, - 0x76, 0x5b, 0xa2, 0x49, 0x6d, 0x8b, 0xd1, 0x25, - 0x72, 0xf8, 0xf6, 0x64, 0x86, 0x68, 0x98, 0x16, - 0xd4, 0xa4, 0x5c, 0xcc, 0x5d, 0x65, 0xb6, 0x92, - 0x6c, 0x70, 0x48, 0x50, 0xfd, 0xed, 0xb9, 0xda, - 0x5e, 0x15, 0x46, 0x57, 0xa7, 0x8d, 0x9d, 0x84, - 0x90, 0xd8, 0xab, 0x00, 0x8c, 0xbc, 0xd3, 0x0a, - 0xf7, 0xe4, 0x58, 0x05, 0xb8, 0xb3, 0x45, 0x06, - 0xd0, 0x2c, 0x1e, 0x8f, 0xca, 0x3f, 0x0f, 0x02, - 0xc1, 0xaf, 0xbd, 0x03, 0x01, 0x13, 0x8a, 0x6b, - 0x3a, 0x91, 0x11, 0x41, 0x4f, 0x67, 0xdc, 0xea, - 0x97, 0xf2, 0xcf, 0xce, 0xf0, 0xb4, 0xe6, 0x73, - 0x96, 0xac, 0x74, 0x22, 0xe7, 0xad, 0x35, 0x85, - 0xe2, 0xf9, 0x37, 0xe8, 0x1c, 0x75, 0xdf, 0x6e, - 0x47, 0xf1, 0x1a, 0x71, 0x1d, 0x29, 0xc5, 0x89, - 0x6f, 0xb7, 0x62, 0x0e, 0xaa, 0x18, 0xbe, 0x1b, - 0xfc, 0x56, 0x3e, 0x4b, 0xc6, 0xd2, 0x79, 0x20, - 0x9a, 0xdb, 0xc0, 0xfe, 0x78, 0xcd, 0x5a, 0xf4, - 0x1f, 0xdd, 0xa8, 0x33, 0x88, 0x07, 0xc7, 0x31, - 0xb1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xec, 0x5f, - 0x60, 0x51, 0x7f, 0xa9, 0x19, 0xb5, 0x4a, 0x0d, - 0x2d, 0xe5, 0x7a, 0x9f, 0x93, 0xc9, 0x9c, 0xef, - 0xa0, 0xe0, 0x3b, 0x4d, 0xae, 0x2a, 0xf5, 0xb0, - 0xc8, 0xeb, 0xbb, 0x3c, 0x83, 0x53, 0x99, 0x61, - 0x17, 0x2b, 0x04, 0x7e, 0xba, 0x77, 0xd6, 0x26, - 0xe1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0c, 0x7d, -}; - -static u32 mul_by_x(u32 w) -{ - u32 x = w & 0x7f7f7f7f; - u32 y = w & 0x80808080; - - /* multiply by polynomial 'x' (0b10) in GF(2^8) */ - return (x << 1) ^ (y >> 7) * 0x1b; -} - -static u32 mul_by_x2(u32 w) -{ - u32 x = w & 0x3f3f3f3f; - u32 y = w & 0x80808080; - u32 z = w & 0x40404040; - - /* multiply by polynomial 'x^2' (0b100) in GF(2^8) */ - return (x << 2) ^ (y >> 7) * 0x36 ^ (z >> 6) * 0x1b; -} - -static u32 mix_columns(u32 x) -{ - /* - * Perform the following matrix multiplication in GF(2^8) - * - * | 0x2 0x3 0x1 0x1 | | x[0] | - * | 0x1 0x2 0x3 0x1 | | x[1] | - * | 0x1 0x1 0x2 0x3 | x | x[2] | - * | 0x3 0x1 0x1 0x2 | | x[3] | - */ - u32 y = mul_by_x(x) ^ ror32(x, 16); - - return y ^ ror32(x ^ y, 8); -} - -static u32 inv_mix_columns(u32 x) -{ - /* - * Perform the following matrix multiplication in GF(2^8) - * - * | 0xe 0xb 0xd 0x9 | | x[0] | - * | 0x9 0xe 0xb 0xd | | x[1] | - * | 0xd 0x9 0xe 0xb | x | x[2] | - * | 0xb 0xd 0x9 0xe | | x[3] | - * - * which can conveniently be reduced to - * - * | 0x2 0x3 0x1 0x1 | | 0x5 0x0 0x4 0x0 | | x[0] | - * | 0x1 0x2 0x3 0x1 | | 0x0 0x5 0x0 0x4 | | x[1] | - * | 0x1 0x1 0x2 0x3 | x | 0x4 0x0 0x5 0x0 | x | x[2] | - * | 0x3 0x1 0x1 0x2 | | 0x0 0x4 0x0 0x5 | | x[3] | - */ - u32 y = mul_by_x2(x); - - return mix_columns(x ^ y ^ ror32(y, 16)); -} - -static __always_inline u32 subshift(u32 in[], int pos) -{ - return (__aesti_sbox[in[pos] & 0xff]) ^ - (__aesti_sbox[(in[(pos + 1) % 4] >> 8) & 0xff] << 8) ^ - (__aesti_sbox[(in[(pos + 2) % 4] >> 16) & 0xff] << 16) ^ - (__aesti_sbox[(in[(pos + 3) % 4] >> 24) & 0xff] << 24); -} - -static __always_inline u32 inv_subshift(u32 in[], int pos) -{ - return (__aesti_inv_sbox[in[pos] & 0xff]) ^ - (__aesti_inv_sbox[(in[(pos + 3) % 4] >> 8) & 0xff] << 8) ^ - (__aesti_inv_sbox[(in[(pos + 2) % 4] >> 16) & 0xff] << 16) ^ - (__aesti_inv_sbox[(in[(pos + 1) % 4] >> 24) & 0xff] << 24); -} - -static u32 subw(u32 in) -{ - return (__aesti_sbox[in & 0xff]) ^ - (__aesti_sbox[(in >> 8) & 0xff] << 8) ^ - (__aesti_sbox[(in >> 16) & 0xff] << 16) ^ - (__aesti_sbox[(in >> 24) & 0xff] << 24); -} - -static int aesti_expand_key(struct crypto_aes_ctx *ctx, const u8 *in_key, - unsigned int key_len) -{ - u32 kwords = key_len / sizeof(u32); - u32 rc, i, j; - - if (key_len != AES_KEYSIZE_128 && - key_len != AES_KEYSIZE_192 && - key_len != AES_KEYSIZE_256) - return -EINVAL; - - ctx->key_length = key_len; - - for (i = 0; i < kwords; i++) - ctx->key_enc[i] = get_unaligned_le32(in_key + i * sizeof(u32)); - - for (i = 0, rc = 1; i < 10; i++, rc = mul_by_x(rc)) { - u32 *rki = ctx->key_enc + (i * kwords); - u32 *rko = rki + kwords; - - rko[0] = ror32(subw(rki[kwords - 1]), 8) ^ rc ^ rki[0]; - rko[1] = rko[0] ^ rki[1]; - rko[2] = rko[1] ^ rki[2]; - rko[3] = rko[2] ^ rki[3]; - - if (key_len == 24) { - if (i >= 7) - break; - rko[4] = rko[3] ^ rki[4]; - rko[5] = rko[4] ^ rki[5]; - } else if (key_len == 32) { - if (i >= 6) - break; - rko[4] = subw(rko[3]) ^ rki[4]; - rko[5] = rko[4] ^ rki[5]; - rko[6] = rko[5] ^ rki[6]; - rko[7] = rko[6] ^ rki[7]; - } - } - - /* - * Generate the decryption keys for the Equivalent Inverse Cipher. - * This involves reversing the order of the round keys, and applying - * the Inverse Mix Columns transformation to all but the first and - * the last one. - */ - ctx->key_dec[0] = ctx->key_enc[key_len + 24]; - ctx->key_dec[1] = ctx->key_enc[key_len + 25]; - ctx->key_dec[2] = ctx->key_enc[key_len + 26]; - ctx->key_dec[3] = ctx->key_enc[key_len + 27]; - - for (i = 4, j = key_len + 20; j > 0; i += 4, j -= 4) { - ctx->key_dec[i] = inv_mix_columns(ctx->key_enc[j]); - ctx->key_dec[i + 1] = inv_mix_columns(ctx->key_enc[j + 1]); - ctx->key_dec[i + 2] = inv_mix_columns(ctx->key_enc[j + 2]); - ctx->key_dec[i + 3] = inv_mix_columns(ctx->key_enc[j + 3]); - } - ctx->key_dec[i] = ctx->key_enc[0]; - ctx->key_dec[i + 1] = ctx->key_enc[1]; - ctx->key_dec[i + 2] = ctx->key_enc[2]; - ctx->key_dec[i + 3] = ctx->key_enc[3]; - - return 0; -} static int aesti_set_key(struct crypto_tfm *tfm, const u8 *in_key, unsigned int key_len) { struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); - return aesti_expand_key(ctx, in_key, key_len); + return aes_expandkey(ctx, in_key, key_len); } static void aesti_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { const struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); - const u32 *rkp = ctx->key_enc + 4; - int rounds = 6 + ctx->key_length / 4; - u32 st0[4], st1[4]; unsigned long flags; - int round; - - st0[0] = ctx->key_enc[0] ^ get_unaligned_le32(in); - st0[1] = ctx->key_enc[1] ^ get_unaligned_le32(in + 4); - st0[2] = ctx->key_enc[2] ^ get_unaligned_le32(in + 8); - st0[3] = ctx->key_enc[3] ^ get_unaligned_le32(in + 12); /* * Temporarily disable interrupts to avoid races where cachelines are @@ -261,36 +29,7 @@ static void aesti_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) */ local_irq_save(flags); - /* - * Force the compiler to emit data independent Sbox references, - * by xoring the input with Sbox values that are known to add up - * to zero. This pulls the entire Sbox into the D-cache before any - * data dependent lookups are done. - */ - st0[0] ^= __aesti_sbox[ 0] ^ __aesti_sbox[ 64] ^ __aesti_sbox[134] ^ __aesti_sbox[195]; - st0[1] ^= __aesti_sbox[16] ^ __aesti_sbox[ 82] ^ __aesti_sbox[158] ^ __aesti_sbox[221]; - st0[2] ^= __aesti_sbox[32] ^ __aesti_sbox[ 96] ^ __aesti_sbox[160] ^ __aesti_sbox[234]; - st0[3] ^= __aesti_sbox[48] ^ __aesti_sbox[112] ^ __aesti_sbox[186] ^ __aesti_sbox[241]; - - for (round = 0;; round += 2, rkp += 8) { - st1[0] = mix_columns(subshift(st0, 0)) ^ rkp[0]; - st1[1] = mix_columns(subshift(st0, 1)) ^ rkp[1]; - st1[2] = mix_columns(subshift(st0, 2)) ^ rkp[2]; - st1[3] = mix_columns(subshift(st0, 3)) ^ rkp[3]; - - if (round == rounds - 2) - break; - - st0[0] = mix_columns(subshift(st1, 0)) ^ rkp[4]; - st0[1] = mix_columns(subshift(st1, 1)) ^ rkp[5]; - st0[2] = mix_columns(subshift(st1, 2)) ^ rkp[6]; - st0[3] = mix_columns(subshift(st1, 3)) ^ rkp[7]; - } - - put_unaligned_le32(subshift(st1, 0) ^ rkp[4], out); - put_unaligned_le32(subshift(st1, 1) ^ rkp[5], out + 4); - put_unaligned_le32(subshift(st1, 2) ^ rkp[6], out + 8); - put_unaligned_le32(subshift(st1, 3) ^ rkp[7], out + 12); + aes_encrypt(ctx, out, in); local_irq_restore(flags); } @@ -298,16 +37,7 @@ static void aesti_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) static void aesti_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { const struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); - const u32 *rkp = ctx->key_dec + 4; - int rounds = 6 + ctx->key_length / 4; - u32 st0[4], st1[4]; unsigned long flags; - int round; - - st0[0] = ctx->key_dec[0] ^ get_unaligned_le32(in); - st0[1] = ctx->key_dec[1] ^ get_unaligned_le32(in + 4); - st0[2] = ctx->key_dec[2] ^ get_unaligned_le32(in + 8); - st0[3] = ctx->key_dec[3] ^ get_unaligned_le32(in + 12); /* * Temporarily disable interrupts to avoid races where cachelines are @@ -315,36 +45,7 @@ static void aesti_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) */ local_irq_save(flags); - /* - * Force the compiler to emit data independent Sbox references, - * by xoring the input with Sbox values that are known to add up - * to zero. This pulls the entire Sbox into the D-cache before any - * data dependent lookups are done. - */ - st0[0] ^= __aesti_inv_sbox[ 0] ^ __aesti_inv_sbox[ 64] ^ __aesti_inv_sbox[129] ^ __aesti_inv_sbox[200]; - st0[1] ^= __aesti_inv_sbox[16] ^ __aesti_inv_sbox[ 83] ^ __aesti_inv_sbox[150] ^ __aesti_inv_sbox[212]; - st0[2] ^= __aesti_inv_sbox[32] ^ __aesti_inv_sbox[ 96] ^ __aesti_inv_sbox[160] ^ __aesti_inv_sbox[236]; - st0[3] ^= __aesti_inv_sbox[48] ^ __aesti_inv_sbox[112] ^ __aesti_inv_sbox[187] ^ __aesti_inv_sbox[247]; - - for (round = 0;; round += 2, rkp += 8) { - st1[0] = inv_mix_columns(inv_subshift(st0, 0)) ^ rkp[0]; - st1[1] = inv_mix_columns(inv_subshift(st0, 1)) ^ rkp[1]; - st1[2] = inv_mix_columns(inv_subshift(st0, 2)) ^ rkp[2]; - st1[3] = inv_mix_columns(inv_subshift(st0, 3)) ^ rkp[3]; - - if (round == rounds - 2) - break; - - st0[0] = inv_mix_columns(inv_subshift(st1, 0)) ^ rkp[4]; - st0[1] = inv_mix_columns(inv_subshift(st1, 1)) ^ rkp[5]; - st0[2] = inv_mix_columns(inv_subshift(st1, 2)) ^ rkp[6]; - st0[3] = inv_mix_columns(inv_subshift(st1, 3)) ^ rkp[7]; - } - - put_unaligned_le32(inv_subshift(st1, 0) ^ rkp[4], out); - put_unaligned_le32(inv_subshift(st1, 1) ^ rkp[5], out + 4); - put_unaligned_le32(inv_subshift(st1, 2) ^ rkp[6], out + 8); - put_unaligned_le32(inv_subshift(st1, 3) ^ rkp[7], out + 12); + aes_decrypt(ctx, out, in); local_irq_restore(flags); } diff --git a/include/crypto/aes.h b/include/crypto/aes.h index 0fdb542c70cd..d0067fca0cd0 100644 --- a/include/crypto/aes.h +++ b/include/crypto/aes.h @@ -37,4 +37,38 @@ int crypto_aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, unsigned int key_len); int crypto_aes_expand_key(struct crypto_aes_ctx *ctx, const u8 *in_key, unsigned int key_len); + +/** + * aes_expandkey - Expands the AES key as described in FIPS-197 + * @ctx: The location where the computed key will be stored. + * @in_key: The supplied key. + * @key_len: The length of the supplied key. + * + * Returns 0 on success. The function fails only if an invalid key size (or + * pointer) is supplied. + * The expanded key size is 240 bytes (max of 14 rounds with a unique 16 bytes + * key schedule plus a 16 bytes key which is used before the first round). + * The decryption key is prepared for the "Equivalent Inverse Cipher" as + * described in FIPS-197. The first slot (16 bytes) of each key (enc or dec) is + * for the initial combination, the second slot for the first round and so on. + */ +int aes_expandkey(struct crypto_aes_ctx *ctx, const u8 *in_key, + unsigned int key_len); + +/** + * aes_encrypt - Encrypt a single AES block + * @ctx: Context struct containing the key schedule + * @out: Buffer to store the ciphertext + * @in: Buffer containing the plaintext + */ +void aes_encrypt(const struct crypto_aes_ctx *ctx, u8 *out, const u8 *in); + +/** + * aes_decrypt - Decrypt a single AES block + * @ctx: Context struct containing the key schedule + * @out: Buffer to store the plaintext + * @in: Buffer containing the ciphertext + */ +void aes_decrypt(const struct crypto_aes_ctx *ctx, u8 *out, const u8 *in); + #endif diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile index 88195c34932d..42a91c62d96d 100644 --- a/lib/crypto/Makefile +++ b/lib/crypto/Makefile @@ -1,4 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_CRYPTO_LIB_AES) += libaes.o +libaes-y := aes.o + obj-$(CONFIG_CRYPTO_LIB_ARC4) += libarc4.o libarc4-y := arc4.o diff --git a/lib/crypto/aes.c b/lib/crypto/aes.c new file mode 100644 index 000000000000..9928b23e0a8a --- /dev/null +++ b/lib/crypto/aes.c @@ -0,0 +1,350 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2017-2019 Linaro Ltd + */ + +#include +#include +#include +#include + +/* + * Emit the sbox as volatile const to prevent the compiler from doing + * constant folding on sbox references involving fixed indexes. + */ +static volatile const u8 __cacheline_aligned aes_sbox[] = { + 0x63, 0x7c, 0x77, 0x7b, 0xf2, 0x6b, 0x6f, 0xc5, + 0x30, 0x01, 0x67, 0x2b, 0xfe, 0xd7, 0xab, 0x76, + 0xca, 0x82, 0xc9, 0x7d, 0xfa, 0x59, 0x47, 0xf0, + 0xad, 0xd4, 0xa2, 0xaf, 0x9c, 0xa4, 0x72, 0xc0, + 0xb7, 0xfd, 0x93, 0x26, 0x36, 0x3f, 0xf7, 0xcc, + 0x34, 0xa5, 0xe5, 0xf1, 0x71, 0xd8, 0x31, 0x15, + 0x04, 0xc7, 0x23, 0xc3, 0x18, 0x96, 0x05, 0x9a, + 0x07, 0x12, 0x80, 0xe2, 0xeb, 0x27, 0xb2, 0x75, + 0x09, 0x83, 0x2c, 0x1a, 0x1b, 0x6e, 0x5a, 0xa0, + 0x52, 0x3b, 0xd6, 0xb3, 0x29, 0xe3, 0x2f, 0x84, + 0x53, 0xd1, 0x00, 0xed, 0x20, 0xfc, 0xb1, 0x5b, + 0x6a, 0xcb, 0xbe, 0x39, 0x4a, 0x4c, 0x58, 0xcf, + 0xd0, 0xef, 0xaa, 0xfb, 0x43, 0x4d, 0x33, 0x85, + 0x45, 0xf9, 0x02, 0x7f, 0x50, 0x3c, 0x9f, 0xa8, + 0x51, 0xa3, 0x40, 0x8f, 0x92, 0x9d, 0x38, 0xf5, + 0xbc, 0xb6, 0xda, 0x21, 0x10, 0xff, 0xf3, 0xd2, + 0xcd, 0x0c, 0x13, 0xec, 0x5f, 0x97, 0x44, 0x17, + 0xc4, 0xa7, 0x7e, 0x3d, 0x64, 0x5d, 0x19, 0x73, + 0x60, 0x81, 0x4f, 0xdc, 0x22, 0x2a, 0x90, 0x88, + 0x46, 0xee, 0xb8, 0x14, 0xde, 0x5e, 0x0b, 0xdb, + 0xe0, 0x32, 0x3a, 0x0a, 0x49, 0x06, 0x24, 0x5c, + 0xc2, 0xd3, 0xac, 0x62, 0x91, 0x95, 0xe4, 0x79, + 0xe7, 0xc8, 0x37, 0x6d, 0x8d, 0xd5, 0x4e, 0xa9, + 0x6c, 0x56, 0xf4, 0xea, 0x65, 0x7a, 0xae, 0x08, + 0xba, 0x78, 0x25, 0x2e, 0x1c, 0xa6, 0xb4, 0xc6, + 0xe8, 0xdd, 0x74, 0x1f, 0x4b, 0xbd, 0x8b, 0x8a, + 0x70, 0x3e, 0xb5, 0x66, 0x48, 0x03, 0xf6, 0x0e, + 0x61, 0x35, 0x57, 0xb9, 0x86, 0xc1, 0x1d, 0x9e, + 0xe1, 0xf8, 0x98, 0x11, 0x69, 0xd9, 0x8e, 0x94, + 0x9b, 0x1e, 0x87, 0xe9, 0xce, 0x55, 0x28, 0xdf, + 0x8c, 0xa1, 0x89, 0x0d, 0xbf, 0xe6, 0x42, 0x68, + 0x41, 0x99, 0x2d, 0x0f, 0xb0, 0x54, 0xbb, 0x16, +}; + +static volatile const u8 __cacheline_aligned aes_inv_sbox[] = { + 0x52, 0x09, 0x6a, 0xd5, 0x30, 0x36, 0xa5, 0x38, + 0xbf, 0x40, 0xa3, 0x9e, 0x81, 0xf3, 0xd7, 0xfb, + 0x7c, 0xe3, 0x39, 0x82, 0x9b, 0x2f, 0xff, 0x87, + 0x34, 0x8e, 0x43, 0x44, 0xc4, 0xde, 0xe9, 0xcb, + 0x54, 0x7b, 0x94, 0x32, 0xa6, 0xc2, 0x23, 0x3d, + 0xee, 0x4c, 0x95, 0x0b, 0x42, 0xfa, 0xc3, 0x4e, + 0x08, 0x2e, 0xa1, 0x66, 0x28, 0xd9, 0x24, 0xb2, + 0x76, 0x5b, 0xa2, 0x49, 0x6d, 0x8b, 0xd1, 0x25, + 0x72, 0xf8, 0xf6, 0x64, 0x86, 0x68, 0x98, 0x16, + 0xd4, 0xa4, 0x5c, 0xcc, 0x5d, 0x65, 0xb6, 0x92, + 0x6c, 0x70, 0x48, 0x50, 0xfd, 0xed, 0xb9, 0xda, + 0x5e, 0x15, 0x46, 0x57, 0xa7, 0x8d, 0x9d, 0x84, + 0x90, 0xd8, 0xab, 0x00, 0x8c, 0xbc, 0xd3, 0x0a, + 0xf7, 0xe4, 0x58, 0x05, 0xb8, 0xb3, 0x45, 0x06, + 0xd0, 0x2c, 0x1e, 0x8f, 0xca, 0x3f, 0x0f, 0x02, + 0xc1, 0xaf, 0xbd, 0x03, 0x01, 0x13, 0x8a, 0x6b, + 0x3a, 0x91, 0x11, 0x41, 0x4f, 0x67, 0xdc, 0xea, + 0x97, 0xf2, 0xcf, 0xce, 0xf0, 0xb4, 0xe6, 0x73, + 0x96, 0xac, 0x74, 0x22, 0xe7, 0xad, 0x35, 0x85, + 0xe2, 0xf9, 0x37, 0xe8, 0x1c, 0x75, 0xdf, 0x6e, + 0x47, 0xf1, 0x1a, 0x71, 0x1d, 0x29, 0xc5, 0x89, + 0x6f, 0xb7, 0x62, 0x0e, 0xaa, 0x18, 0xbe, 0x1b, + 0xfc, 0x56, 0x3e, 0x4b, 0xc6, 0xd2, 0x79, 0x20, + 0x9a, 0xdb, 0xc0, 0xfe, 0x78, 0xcd, 0x5a, 0xf4, + 0x1f, 0xdd, 0xa8, 0x33, 0x88, 0x07, 0xc7, 0x31, + 0xb1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xec, 0x5f, + 0x60, 0x51, 0x7f, 0xa9, 0x19, 0xb5, 0x4a, 0x0d, + 0x2d, 0xe5, 0x7a, 0x9f, 0x93, 0xc9, 0x9c, 0xef, + 0xa0, 0xe0, 0x3b, 0x4d, 0xae, 0x2a, 0xf5, 0xb0, + 0xc8, 0xeb, 0xbb, 0x3c, 0x83, 0x53, 0x99, 0x61, + 0x17, 0x2b, 0x04, 0x7e, 0xba, 0x77, 0xd6, 0x26, + 0xe1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0c, 0x7d, +}; + +static u32 mul_by_x(u32 w) +{ + u32 x = w & 0x7f7f7f7f; + u32 y = w & 0x80808080; + + /* multiply by polynomial 'x' (0b10) in GF(2^8) */ + return (x << 1) ^ (y >> 7) * 0x1b; +} + +static u32 mul_by_x2(u32 w) +{ + u32 x = w & 0x3f3f3f3f; + u32 y = w & 0x80808080; + u32 z = w & 0x40404040; + + /* multiply by polynomial 'x^2' (0b100) in GF(2^8) */ + return (x << 2) ^ (y >> 7) * 0x36 ^ (z >> 6) * 0x1b; +} + +static u32 mix_columns(u32 x) +{ + /* + * Perform the following matrix multiplication in GF(2^8) + * + * | 0x2 0x3 0x1 0x1 | | x[0] | + * | 0x1 0x2 0x3 0x1 | | x[1] | + * | 0x1 0x1 0x2 0x3 | x | x[2] | + * | 0x3 0x1 0x1 0x2 | | x[3] | + */ + u32 y = mul_by_x(x) ^ ror32(x, 16); + + return y ^ ror32(x ^ y, 8); +} + +static u32 inv_mix_columns(u32 x) +{ + /* + * Perform the following matrix multiplication in GF(2^8) + * + * | 0xe 0xb 0xd 0x9 | | x[0] | + * | 0x9 0xe 0xb 0xd | | x[1] | + * | 0xd 0x9 0xe 0xb | x | x[2] | + * | 0xb 0xd 0x9 0xe | | x[3] | + * + * which can conveniently be reduced to + * + * | 0x2 0x3 0x1 0x1 | | 0x5 0x0 0x4 0x0 | | x[0] | + * | 0x1 0x2 0x3 0x1 | | 0x0 0x5 0x0 0x4 | | x[1] | + * | 0x1 0x1 0x2 0x3 | x | 0x4 0x0 0x5 0x0 | x | x[2] | + * | 0x3 0x1 0x1 0x2 | | 0x0 0x4 0x0 0x5 | | x[3] | + */ + u32 y = mul_by_x2(x); + + return mix_columns(x ^ y ^ ror32(y, 16)); +} + +static __always_inline u32 subshift(u32 in[], int pos) +{ + return (aes_sbox[in[pos] & 0xff]) ^ + (aes_sbox[(in[(pos + 1) % 4] >> 8) & 0xff] << 8) ^ + (aes_sbox[(in[(pos + 2) % 4] >> 16) & 0xff] << 16) ^ + (aes_sbox[(in[(pos + 3) % 4] >> 24) & 0xff] << 24); +} + +static __always_inline u32 inv_subshift(u32 in[], int pos) +{ + return (aes_inv_sbox[in[pos] & 0xff]) ^ + (aes_inv_sbox[(in[(pos + 3) % 4] >> 8) & 0xff] << 8) ^ + (aes_inv_sbox[(in[(pos + 2) % 4] >> 16) & 0xff] << 16) ^ + (aes_inv_sbox[(in[(pos + 1) % 4] >> 24) & 0xff] << 24); +} + +static u32 subw(u32 in) +{ + return (aes_sbox[in & 0xff]) ^ + (aes_sbox[(in >> 8) & 0xff] << 8) ^ + (aes_sbox[(in >> 16) & 0xff] << 16) ^ + (aes_sbox[(in >> 24) & 0xff] << 24); +} + +/** + * aes_expandkey - Expands the AES key as described in FIPS-197 + * @ctx: The location where the computed key will be stored. + * @in_key: The supplied key. + * @key_len: The length of the supplied key. + * + * Returns 0 on success. The function fails only if an invalid key size (or + * pointer) is supplied. + * The expanded key size is 240 bytes (max of 14 rounds with a unique 16 bytes + * key schedule plus a 16 bytes key which is used before the first round). + * The decryption key is prepared for the "Equivalent Inverse Cipher" as + * described in FIPS-197. The first slot (16 bytes) of each key (enc or dec) is + * for the initial combination, the second slot for the first round and so on. + */ +int aes_expandkey(struct crypto_aes_ctx *ctx, const u8 *in_key, + unsigned int key_len) +{ + u32 kwords = key_len / sizeof(u32); + u32 rc, i, j; + + if (key_len != AES_KEYSIZE_128 && + key_len != AES_KEYSIZE_192 && + key_len != AES_KEYSIZE_256) + return -EINVAL; + + ctx->key_length = key_len; + + for (i = 0; i < kwords; i++) + ctx->key_enc[i] = get_unaligned_le32(in_key + i * sizeof(u32)); + + for (i = 0, rc = 1; i < 10; i++, rc = mul_by_x(rc)) { + u32 *rki = ctx->key_enc + (i * kwords); + u32 *rko = rki + kwords; + + rko[0] = ror32(subw(rki[kwords - 1]), 8) ^ rc ^ rki[0]; + rko[1] = rko[0] ^ rki[1]; + rko[2] = rko[1] ^ rki[2]; + rko[3] = rko[2] ^ rki[3]; + + if (key_len == AES_KEYSIZE_192) { + if (i >= 7) + break; + rko[4] = rko[3] ^ rki[4]; + rko[5] = rko[4] ^ rki[5]; + } else if (key_len == AES_KEYSIZE_256) { + if (i >= 6) + break; + rko[4] = subw(rko[3]) ^ rki[4]; + rko[5] = rko[4] ^ rki[5]; + rko[6] = rko[5] ^ rki[6]; + rko[7] = rko[6] ^ rki[7]; + } + } + + /* + * Generate the decryption keys for the Equivalent Inverse Cipher. + * This involves reversing the order of the round keys, and applying + * the Inverse Mix Columns transformation to all but the first and + * the last one. + */ + ctx->key_dec[0] = ctx->key_enc[key_len + 24]; + ctx->key_dec[1] = ctx->key_enc[key_len + 25]; + ctx->key_dec[2] = ctx->key_enc[key_len + 26]; + ctx->key_dec[3] = ctx->key_enc[key_len + 27]; + + for (i = 4, j = key_len + 20; j > 0; i += 4, j -= 4) { + ctx->key_dec[i] = inv_mix_columns(ctx->key_enc[j]); + ctx->key_dec[i + 1] = inv_mix_columns(ctx->key_enc[j + 1]); + ctx->key_dec[i + 2] = inv_mix_columns(ctx->key_enc[j + 2]); + ctx->key_dec[i + 3] = inv_mix_columns(ctx->key_enc[j + 3]); + } + + ctx->key_dec[i] = ctx->key_enc[0]; + ctx->key_dec[i + 1] = ctx->key_enc[1]; + ctx->key_dec[i + 2] = ctx->key_enc[2]; + ctx->key_dec[i + 3] = ctx->key_enc[3]; + + return 0; +} +EXPORT_SYMBOL(aes_expandkey); + +/** + * aes_encrypt - Encrypt a single AES block + * @ctx: Context struct containing the key schedule + * @out: Buffer to store the ciphertext + * @in: Buffer containing the plaintext + */ +void aes_encrypt(const struct crypto_aes_ctx *ctx, u8 *out, const u8 *in) +{ + const u32 *rkp = ctx->key_enc + 4; + int rounds = 6 + ctx->key_length / 4; + u32 st0[4], st1[4]; + int round; + + st0[0] = ctx->key_enc[0] ^ get_unaligned_le32(in); + st0[1] = ctx->key_enc[1] ^ get_unaligned_le32(in + 4); + st0[2] = ctx->key_enc[2] ^ get_unaligned_le32(in + 8); + st0[3] = ctx->key_enc[3] ^ get_unaligned_le32(in + 12); + + /* + * Force the compiler to emit data independent Sbox references, + * by xoring the input with Sbox values that are known to add up + * to zero. This pulls the entire Sbox into the D-cache before any + * data dependent lookups are done. + */ + st0[0] ^= aes_sbox[ 0] ^ aes_sbox[ 64] ^ aes_sbox[134] ^ aes_sbox[195]; + st0[1] ^= aes_sbox[16] ^ aes_sbox[ 82] ^ aes_sbox[158] ^ aes_sbox[221]; + st0[2] ^= aes_sbox[32] ^ aes_sbox[ 96] ^ aes_sbox[160] ^ aes_sbox[234]; + st0[3] ^= aes_sbox[48] ^ aes_sbox[112] ^ aes_sbox[186] ^ aes_sbox[241]; + + for (round = 0;; round += 2, rkp += 8) { + st1[0] = mix_columns(subshift(st0, 0)) ^ rkp[0]; + st1[1] = mix_columns(subshift(st0, 1)) ^ rkp[1]; + st1[2] = mix_columns(subshift(st0, 2)) ^ rkp[2]; + st1[3] = mix_columns(subshift(st0, 3)) ^ rkp[3]; + + if (round == rounds - 2) + break; + + st0[0] = mix_columns(subshift(st1, 0)) ^ rkp[4]; + st0[1] = mix_columns(subshift(st1, 1)) ^ rkp[5]; + st0[2] = mix_columns(subshift(st1, 2)) ^ rkp[6]; + st0[3] = mix_columns(subshift(st1, 3)) ^ rkp[7]; + } + + put_unaligned_le32(subshift(st1, 0) ^ rkp[4], out); + put_unaligned_le32(subshift(st1, 1) ^ rkp[5], out + 4); + put_unaligned_le32(subshift(st1, 2) ^ rkp[6], out + 8); + put_unaligned_le32(subshift(st1, 3) ^ rkp[7], out + 12); +} +EXPORT_SYMBOL(aes_encrypt); + +/** + * aes_decrypt - Decrypt a single AES block + * @ctx: Context struct containing the key schedule + * @out: Buffer to store the plaintext + * @in: Buffer containing the ciphertext + */ +void aes_decrypt(const struct crypto_aes_ctx *ctx, u8 *out, const u8 *in) +{ + const u32 *rkp = ctx->key_dec + 4; + int rounds = 6 + ctx->key_length / 4; + u32 st0[4], st1[4]; + int round; + + st0[0] = ctx->key_dec[0] ^ get_unaligned_le32(in); + st0[1] = ctx->key_dec[1] ^ get_unaligned_le32(in + 4); + st0[2] = ctx->key_dec[2] ^ get_unaligned_le32(in + 8); + st0[3] = ctx->key_dec[3] ^ get_unaligned_le32(in + 12); + + /* + * Force the compiler to emit data independent Sbox references, + * by xoring the input with Sbox values that are known to add up + * to zero. This pulls the entire Sbox into the D-cache before any + * data dependent lookups are done. + */ + st0[0] ^= aes_inv_sbox[ 0] ^ aes_inv_sbox[ 64] ^ aes_inv_sbox[129] ^ aes_inv_sbox[200]; + st0[1] ^= aes_inv_sbox[16] ^ aes_inv_sbox[ 83] ^ aes_inv_sbox[150] ^ aes_inv_sbox[212]; + st0[2] ^= aes_inv_sbox[32] ^ aes_inv_sbox[ 96] ^ aes_inv_sbox[160] ^ aes_inv_sbox[236]; + st0[3] ^= aes_inv_sbox[48] ^ aes_inv_sbox[112] ^ aes_inv_sbox[187] ^ aes_inv_sbox[247]; + + for (round = 0;; round += 2, rkp += 8) { + st1[0] = inv_mix_columns(inv_subshift(st0, 0)) ^ rkp[0]; + st1[1] = inv_mix_columns(inv_subshift(st0, 1)) ^ rkp[1]; + st1[2] = inv_mix_columns(inv_subshift(st0, 2)) ^ rkp[2]; + st1[3] = inv_mix_columns(inv_subshift(st0, 3)) ^ rkp[3]; + + if (round == rounds - 2) + break; + + st0[0] = inv_mix_columns(inv_subshift(st1, 0)) ^ rkp[4]; + st0[1] = inv_mix_columns(inv_subshift(st1, 1)) ^ rkp[5]; + st0[2] = inv_mix_columns(inv_subshift(st1, 2)) ^ rkp[6]; + st0[3] = inv_mix_columns(inv_subshift(st1, 3)) ^ rkp[7]; + } + + put_unaligned_le32(inv_subshift(st1, 0) ^ rkp[4], out); + put_unaligned_le32(inv_subshift(st1, 1) ^ rkp[5], out + 4); + put_unaligned_le32(inv_subshift(st1, 2) ^ rkp[6], out + 8); + put_unaligned_le32(inv_subshift(st1, 3) ^ rkp[7], out + 12); +} +EXPORT_SYMBOL(aes_decrypt); + +MODULE_DESCRIPTION("Generic AES library"); +MODULE_AUTHOR("Ard Biesheuvel "); +MODULE_LICENSE("GPL v2"); From patchwork Thu Jun 27 10:26:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167901 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148461ilk; Thu, 27 Jun 2019 03:27:59 -0700 (PDT) X-Google-Smtp-Source: APXvYqy0zpSa/0mLPq4s2jhDkHxkkrmYlEW/sUZlV3esPZdxhdr+0v0xoo8dsURYQtfrLsmOfyEF X-Received: by 2002:a17:902:2983:: with SMTP id h3mr3826396plb.45.1561631279596; Thu, 27 Jun 2019 03:27:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631279; cv=none; d=google.com; s=arc-20160816; b=tqs7onBXZtZWT63XuF0e4o8b+zoktLv40BLa5UqfiKIBVz3cikbOyW1LEp0gtYsRDx BUrDD3m3kmyMZYHb6jJcOTiG6wMKPgFkmztUsotANuCJz+Qh31piayDUD3WXvJn/8St7 koxiy5pmLKp97XYOelvx9yaFXOTemubVe9KieVvJTlsrxhCziR1/4DHCxWVOXNW3oeRN l6+alSqTsxC+MQH3xx8oUHBEJQ2fUG+IDCf0jJ43Skz/Quq9iGwut4vtuY4mkzVr+xju 4pIw0RAaRsKeooAnlblqN7Kp3tx3q+VPIsSkPdM/fJ32qsaNGf++y6QuZX7uKhH7qPwU SO6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=5Oqvcot1yZKpSWtboJ7OjflLCxxMos1RB3yPJW9XMLo=; b=OWEUFUYOAfw1iYahjfNCgHQLQ8W3Q8DiPBFQG8eylgO3omF/KbBuOteTED5G79wFn6 6LWVOvSqQ/tPkw387YdTyDuFvlf10WuKN6P2KXCA8njytoGKMmkSNfPNEAHDuaPTKko2 rs3N1lgiSTXAhtbL2oQ4Lau9oKDD47KmQi9Mqpf8g9bXuBjluqlqS7SM1CKUb7rjhI9b 3SOCt7jyZAOK7f3eYCoAjNwWEHbBR3OQYyRLKLYeeaTgAx+kUj9wVi11s7+g5LLnZ71k VD+PjA7XK4n9q4pql+hgQU2A13mfwaABFuJ/+pIbQq+WsPgXhWbNd+l4Hu4jC7fFw5cG iwLw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=IV0UzQeU; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.27.59; Thu, 27 Jun 2019 03:27:59 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=IV0UzQeU; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726666AbfF0K16 (ORCPT + 3 others); Thu, 27 Jun 2019 06:27:58 -0400 Received: from mail-wm1-f67.google.com ([209.85.128.67]:38992 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726664AbfF0K16 (ORCPT ); Thu, 27 Jun 2019 06:27:58 -0400 Received: by mail-wm1-f67.google.com with SMTP id z23so5116606wma.4 for ; Thu, 27 Jun 2019 03:27:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5Oqvcot1yZKpSWtboJ7OjflLCxxMos1RB3yPJW9XMLo=; b=IV0UzQeU5nxRPBwgUzmlex0+nuCJuhK1tG6xuTVNNDrs5l8F5NP2aal5QT9p8LmBRl 31agMe1aPpyGI+ScZ6c7H06SRbRp+Z03pmXGCZZ5ZZImO7hYL+ffNgoXMrL7SM1Y37fd HW5NaYoNOp+sq5nmLdpCFYveBbhB9iXI9VpT82bM1/ZdljTclos8ZpYrkvCU75bEJtrV 3AZofas0KtNQEu0tcl0EE0JCWdA3vCrisj02vKy1Tvjw4jdHyfNMVAmpgIbOym3IejTn QJULX9rmoJ1cdcQML8+p6ihG19GjxkWaNCybX787p7bJcpthAE/RgCVfoO2wdAucwg4N 85/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5Oqvcot1yZKpSWtboJ7OjflLCxxMos1RB3yPJW9XMLo=; b=srGG581k74g+rULPzrbf+3KufCGBYyc2Dx/J0yR5csB8/IYdv7zkxr3DMQoU3pz3M6 iXLUchteDDsZthBMxtFGwpinb631zW9h8Dpw80x0zl8j8qbRJxQ6bBzGQ8nfBLQDzBox 8SiSbD3jLYAXn1LjmRkmM+ArS4m+Brrt01S8Dz5tYZXSKm53L20/Ar11q8Lrd51Lo+10 DBZZpSfG/xH2ebC4UmvhWXv0sUhKOJxi7ZhCWBKZ6sXPcro86xOcp2VHWR492/HqTkuM yPGbKxOcmFh9cZQuNoh3cu2wsrnvVi3EaFh4KNp34JykqzT1ZyzIaLv/a/3n9KG0pg+D +98A== X-Gm-Message-State: APjAAAWRM9gjYX+FMyLJ+PXSg1qzEtsNFs6baKgBoQarT/cT1u1GhSZl N+DYC078MOiyLFgDgq/41bpDtYde6sE= X-Received: by 2002:a7b:c8c3:: with SMTP id f3mr2884629wml.124.1561631275846; Thu, 27 Jun 2019 03:27:55 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.27.54 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:27:54 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 05/32] crypto: x86/aes-ni - switch to generic for fallback and key routines Date: Thu, 27 Jun 2019 12:26:20 +0200 Message-Id: <20190627102647.2992-6-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The AES-NI code contains fallbacks for invocations that occur from a context where the SIMD unit is unavailable, which really only occurs when running in softirq context that was entered from a hard IRQ that was taken while running kernel code that was already using the FPU. That means performance is not really a consideration, and we can just use the new library code for this use case, which has a smaller footprint and is believed to be time invariant. This will allow us to drop the non-SIMD asm routines in a subsequent patch. Signed-off-by: Ard Biesheuvel --- arch/x86/crypto/aesni-intel_glue.c | 15 +++++++-------- arch/x86/include/asm/crypto/aes.h | 12 ------------ crypto/Kconfig | 3 +-- 3 files changed, 8 insertions(+), 22 deletions(-) -- 2.20.1 diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c index 836d50bd096f..42873c1f6bb4 100644 --- a/arch/x86/crypto/aesni-intel_glue.c +++ b/arch/x86/crypto/aesni-intel_glue.c @@ -30,7 +30,6 @@ #include #include #include -#include #include #include #include @@ -333,7 +332,7 @@ static int aes_set_key_common(struct crypto_tfm *tfm, void *raw_ctx, } if (!crypto_simd_usable()) - err = crypto_aes_expand_key(ctx, in_key, key_len); + err = aes_expandkey(ctx, in_key, key_len); else { kernel_fpu_begin(); err = aesni_set_key(ctx, in_key, key_len); @@ -353,9 +352,9 @@ static void aesni_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) { struct crypto_aes_ctx *ctx = aes_ctx(crypto_tfm_ctx(tfm)); - if (!crypto_simd_usable()) - crypto_aes_encrypt_x86(ctx, dst, src); - else { + if (!crypto_simd_usable()) { + aes_encrypt(ctx, dst, src); + } else { kernel_fpu_begin(); aesni_enc(ctx, dst, src); kernel_fpu_end(); @@ -366,9 +365,9 @@ static void aesni_decrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) { struct crypto_aes_ctx *ctx = aes_ctx(crypto_tfm_ctx(tfm)); - if (!crypto_simd_usable()) - crypto_aes_decrypt_x86(ctx, dst, src); - else { + if (!crypto_simd_usable()) { + aes_decrypt(ctx, dst, src); + } else { kernel_fpu_begin(); aesni_dec(ctx, dst, src); kernel_fpu_end(); diff --git a/arch/x86/include/asm/crypto/aes.h b/arch/x86/include/asm/crypto/aes.h deleted file mode 100644 index c508521dd190..000000000000 --- a/arch/x86/include/asm/crypto/aes.h +++ /dev/null @@ -1,12 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef ASM_X86_AES_H -#define ASM_X86_AES_H - -#include -#include - -void crypto_aes_encrypt_x86(struct crypto_aes_ctx *ctx, u8 *dst, - const u8 *src); -void crypto_aes_decrypt_x86(struct crypto_aes_ctx *ctx, u8 *dst, - const u8 *src); -#endif diff --git a/crypto/Kconfig b/crypto/Kconfig index 091ebbbc9655..20af58068e6b 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -1156,8 +1156,7 @@ config CRYPTO_AES_NI_INTEL tristate "AES cipher algorithms (AES-NI)" depends on X86 select CRYPTO_AEAD - select CRYPTO_AES_X86_64 if 64BIT - select CRYPTO_AES_586 if !64BIT + select CRYPTO_LIB_AES select CRYPTO_ALGAPI select CRYPTO_BLKCIPHER select CRYPTO_GLUE_HELPER_X86 if 64BIT From patchwork Thu Jun 27 10:26:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167903 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148491ilk; Thu, 27 Jun 2019 03:28:01 -0700 (PDT) X-Google-Smtp-Source: APXvYqxWTCLl7ztMBXiX4U3t+oNnQq5Cst5K39xR/dB6EpXWw78JUSlesva5zA42qIDRsO7fkF5M X-Received: by 2002:a17:902:4c88:: with SMTP id b8mr3912322ple.29.1561631281819; Thu, 27 Jun 2019 03:28:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631281; cv=none; d=google.com; s=arc-20160816; b=R/VlZd2uFo2nN2QmhMXimMvYU6X5sEaKxRt6RWarfIbtaYZNKG3wFMshCoeH3LqQlw C3kaa1le6pz98xP+QnhR72tJN2m+ut3Z5gOHWVR77yPdoPU/SzxqKRDc/Ak8HBA7BNkK znNE+6BdhuCM01sVAexSPQuXKMvVapJUlRGwvC1Z7efTqvBxeN+JXMp8teDYmkZlocas ut0n2JemlB6cWxiEvZTwvK8xPic5JwosZscUxuhpeTQoSC7j2FV4qpwmPzrIJNVQQy7s S+pRRNoEm7ENgg7KrzIT8lraIvft556GPXkwzB9dC0P0jG50rCtexKA4+Nw6TWP84MK7 MHJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Rk3H4Hh084k052p7bt4Y0A038t9wYhRs8MD3q3JWnkU=; b=Dcf8ZoF6B1MXa4+WthqKcIKwDTRRG+WGylT+fGKSc6XuRllG7g2lmCYmHlknNll30G Rw6Ub6wek1W2K87ddcbCoQinxlSSyMvYtkTitAQvc9wu1wtB9aKI+/C+DQ/mbY7rLxDN oZGhhcKqIzeEhn8DOm7W54pK5tcxDI3J94SlEi54fEAmUadS8Nm4e90gPYtSQVj7gYnS X3Y+AwCXQAHj0I8Lp6ibAUxNgxpyu2MfuyOVe1hg45Quu5GLI8lr8V+/1skaeHjivvp5 jK8QxuqexFbi2Ew09jvp0AzQYXm/Pb/ggpL4kZN2GCVpZFkX9qm9DZuZZJbunh0JbFxF 6ZMw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="nTCJIHt/"; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.01; Thu, 27 Jun 2019 03:28:01 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="nTCJIHt/"; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726401AbfF0K2B (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:01 -0400 Received: from mail-wm1-f65.google.com ([209.85.128.65]:34246 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726605AbfF0K2A (ORCPT ); Thu, 27 Jun 2019 06:28:00 -0400 Received: by mail-wm1-f65.google.com with SMTP id w9so6715218wmd.1 for ; Thu, 27 Jun 2019 03:27:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Rk3H4Hh084k052p7bt4Y0A038t9wYhRs8MD3q3JWnkU=; b=nTCJIHt/TxG9oNyEeTW3PxQPHEjOsiwO7gOlk/8YEuZwDfilHS0kF4BOM07itElgAR jBy2BXe8ngkoTtomfftE9Eu5/GXvXqYKpzxEpOtlvuqTBQC+kLOjTVWX0qH74RqmERTP cQWUe4y0Z8ZaqtsBvD2NMLQAJrAhlXN6tI3x75JHsjo6qLKluPpS4HC28tVCKBdzaZP+ bJBjtM2yVW++QtQl+L/QfjQiRudp1CTbCTCUeDBhc/Chc209th6G/Jxg3D9rQ6jDrZ+f w4OcEbYqSiTfju1+hd+d42YkrhsVDzBYci2IZEnY6V7hmPj06k+SXuwz5iTqZ3E2UyVa 3/pQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Rk3H4Hh084k052p7bt4Y0A038t9wYhRs8MD3q3JWnkU=; b=NzZyI3EQGJM1epU2RZPaEaAzKPIxHQhIJZqEfYsBdKcpNUCD1P0YkvxV7jzI0RSzuH ERu1+0B7EiywqC5K98tHLu8ho7qD4f6w/Zs75Za5/52+LCqK6/VfZ3RvgihyLMS7Ax8D OmInDjYbWsu5knh+xUVdr8NYW0I1DZoImUsXrGcNlowDF+VPDT572XNvMBIx7gtg98qJ i3JkUeUf1Vhu0RlyHpAmH8cXjVchUwn0wHlCw37YiVxvasbsJeRQxoMAxiai2U8kgYrm dl1dxtoR4XcgOtbCnRIkTvnDhfFwTHMXHeLT+gOQufE0c9ipZlOxMXuAfXdE/mR6LO+5 Is4w== X-Gm-Message-State: APjAAAUKEL7XaTUtppmXnohuEZ/8vdeasTg2GA4SwvIgMfLdMexSK25x WNizU/85RG1hokUYyhWanrTBbYXo8Uc= X-Received: by 2002:a1c:7a01:: with SMTP id v1mr2845568wmc.10.1561631277078; Thu, 27 Jun 2019 03:27:57 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.27.55 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:27:56 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 06/32] crypto: x86/aes - drop scalar assembler implementations Date: Thu, 27 Jun 2019 12:26:21 +0200 Message-Id: <20190627102647.2992-7-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The AES assembler code for x86 isn't actually faster than code generated by the compiler from aes_generic.c, and considering the disproportionate maintenance burden of assembler code on x86, it is better just to drop it entirely. Modern x86 systems will use AES-NI anyway, and given that the modules being removed have a dependency on aes_generic already, we can remove them without running the risk of regressions. Signed-off-by: Ard Biesheuvel --- arch/x86/crypto/Makefile | 4 - arch/x86/crypto/aes-i586-asm_32.S | 362 -------------------- arch/x86/crypto/aes-x86_64-asm_64.S | 185 ---------- arch/x86/crypto/aes_glue.c | 70 ---- crypto/Kconfig | 44 --- 5 files changed, 665 deletions(-) -- 2.20.1 diff --git a/arch/x86/crypto/Makefile b/arch/x86/crypto/Makefile index 45734e1cf967..b96a14e67ab0 100644 --- a/arch/x86/crypto/Makefile +++ b/arch/x86/crypto/Makefile @@ -14,11 +14,9 @@ sha256_ni_supported :=$(call as-instr,sha256msg1 %xmm0$(comma)%xmm1,yes,no) obj-$(CONFIG_CRYPTO_GLUE_HELPER_X86) += glue_helper.o -obj-$(CONFIG_CRYPTO_AES_586) += aes-i586.o obj-$(CONFIG_CRYPTO_TWOFISH_586) += twofish-i586.o obj-$(CONFIG_CRYPTO_SERPENT_SSE2_586) += serpent-sse2-i586.o -obj-$(CONFIG_CRYPTO_AES_X86_64) += aes-x86_64.o obj-$(CONFIG_CRYPTO_DES3_EDE_X86_64) += des3_ede-x86_64.o obj-$(CONFIG_CRYPTO_CAMELLIA_X86_64) += camellia-x86_64.o obj-$(CONFIG_CRYPTO_BLOWFISH_X86_64) += blowfish-x86_64.o @@ -68,11 +66,9 @@ ifeq ($(avx2_supported),yes) obj-$(CONFIG_CRYPTO_MORUS1280_AVX2) += morus1280-avx2.o endif -aes-i586-y := aes-i586-asm_32.o aes_glue.o twofish-i586-y := twofish-i586-asm_32.o twofish_glue.o serpent-sse2-i586-y := serpent-sse2-i586-asm_32.o serpent_sse2_glue.o -aes-x86_64-y := aes-x86_64-asm_64.o aes_glue.o des3_ede-x86_64-y := des3_ede-asm_64.o des3_ede_glue.o camellia-x86_64-y := camellia-x86_64-asm_64.o camellia_glue.o blowfish-x86_64-y := blowfish-x86_64-asm_64.o blowfish_glue.o diff --git a/arch/x86/crypto/aes-i586-asm_32.S b/arch/x86/crypto/aes-i586-asm_32.S deleted file mode 100644 index 2849dbc59e11..000000000000 --- a/arch/x86/crypto/aes-i586-asm_32.S +++ /dev/null @@ -1,362 +0,0 @@ -// ------------------------------------------------------------------------- -// Copyright (c) 2001, Dr Brian Gladman < >, Worcester, UK. -// All rights reserved. -// -// LICENSE TERMS -// -// The free distribution and use of this software in both source and binary -// form is allowed (with or without changes) provided that: -// -// 1. distributions of this source code include the above copyright -// notice, this list of conditions and the following disclaimer// -// -// 2. distributions in binary form include the above copyright -// notice, this list of conditions and the following disclaimer -// in the documentation and/or other associated materials// -// -// 3. the copyright holder's name is not used to endorse products -// built using this software without specific written permission. -// -// -// ALTERNATIVELY, provided that this notice is retained in full, this product -// may be distributed under the terms of the GNU General Public License (GPL), -// in which case the provisions of the GPL apply INSTEAD OF those given above. -// -// Copyright (c) 2004 Linus Torvalds -// Copyright (c) 2004 Red Hat, Inc., James Morris - -// DISCLAIMER -// -// This software is provided 'as is' with no explicit or implied warranties -// in respect of its properties including, but not limited to, correctness -// and fitness for purpose. -// ------------------------------------------------------------------------- -// Issue Date: 29/07/2002 - -.file "aes-i586-asm.S" -.text - -#include -#include - -#define tlen 1024 // length of each of 4 'xor' arrays (256 32-bit words) - -/* offsets to parameters with one register pushed onto stack */ -#define ctx 8 -#define out_blk 12 -#define in_blk 16 - -/* offsets in crypto_aes_ctx structure */ -#define klen (480) -#define ekey (0) -#define dkey (240) - -// register mapping for encrypt and decrypt subroutines - -#define r0 eax -#define r1 ebx -#define r2 ecx -#define r3 edx -#define r4 esi -#define r5 edi - -#define eaxl al -#define eaxh ah -#define ebxl bl -#define ebxh bh -#define ecxl cl -#define ecxh ch -#define edxl dl -#define edxh dh - -#define _h(reg) reg##h -#define h(reg) _h(reg) - -#define _l(reg) reg##l -#define l(reg) _l(reg) - -// This macro takes a 32-bit word representing a column and uses -// each of its four bytes to index into four tables of 256 32-bit -// words to obtain values that are then xored into the appropriate -// output registers r0, r1, r4 or r5. - -// Parameters: -// table table base address -// %1 out_state[0] -// %2 out_state[1] -// %3 out_state[2] -// %4 out_state[3] -// idx input register for the round (destroyed) -// tmp scratch register for the round -// sched key schedule - -#define do_col(table, a1,a2,a3,a4, idx, tmp) \ - movzx %l(idx),%tmp; \ - xor table(,%tmp,4),%a1; \ - movzx %h(idx),%tmp; \ - shr $16,%idx; \ - xor table+tlen(,%tmp,4),%a2; \ - movzx %l(idx),%tmp; \ - movzx %h(idx),%idx; \ - xor table+2*tlen(,%tmp,4),%a3; \ - xor table+3*tlen(,%idx,4),%a4; - -// initialise output registers from the key schedule -// NB1: original value of a3 is in idx on exit -// NB2: original values of a1,a2,a4 aren't used -#define do_fcol(table, a1,a2,a3,a4, idx, tmp, sched) \ - mov 0 sched,%a1; \ - movzx %l(idx),%tmp; \ - mov 12 sched,%a2; \ - xor table(,%tmp,4),%a1; \ - mov 4 sched,%a4; \ - movzx %h(idx),%tmp; \ - shr $16,%idx; \ - xor table+tlen(,%tmp,4),%a2; \ - movzx %l(idx),%tmp; \ - movzx %h(idx),%idx; \ - xor table+3*tlen(,%idx,4),%a4; \ - mov %a3,%idx; \ - mov 8 sched,%a3; \ - xor table+2*tlen(,%tmp,4),%a3; - -// initialise output registers from the key schedule -// NB1: original value of a3 is in idx on exit -// NB2: original values of a1,a2,a4 aren't used -#define do_icol(table, a1,a2,a3,a4, idx, tmp, sched) \ - mov 0 sched,%a1; \ - movzx %l(idx),%tmp; \ - mov 4 sched,%a2; \ - xor table(,%tmp,4),%a1; \ - mov 12 sched,%a4; \ - movzx %h(idx),%tmp; \ - shr $16,%idx; \ - xor table+tlen(,%tmp,4),%a2; \ - movzx %l(idx),%tmp; \ - movzx %h(idx),%idx; \ - xor table+3*tlen(,%idx,4),%a4; \ - mov %a3,%idx; \ - mov 8 sched,%a3; \ - xor table+2*tlen(,%tmp,4),%a3; - - -// original Gladman had conditional saves to MMX regs. -#define save(a1, a2) \ - mov %a2,4*a1(%esp) - -#define restore(a1, a2) \ - mov 4*a2(%esp),%a1 - -// These macros perform a forward encryption cycle. They are entered with -// the first previous round column values in r0,r1,r4,r5 and -// exit with the final values in the same registers, using stack -// for temporary storage. - -// round column values -// on entry: r0,r1,r4,r5 -// on exit: r2,r1,r4,r5 -#define fwd_rnd1(arg, table) \ - save (0,r1); \ - save (1,r5); \ - \ - /* compute new column values */ \ - do_fcol(table, r2,r5,r4,r1, r0,r3, arg); /* idx=r0 */ \ - do_col (table, r4,r1,r2,r5, r0,r3); /* idx=r4 */ \ - restore(r0,0); \ - do_col (table, r1,r2,r5,r4, r0,r3); /* idx=r1 */ \ - restore(r0,1); \ - do_col (table, r5,r4,r1,r2, r0,r3); /* idx=r5 */ - -// round column values -// on entry: r2,r1,r4,r5 -// on exit: r0,r1,r4,r5 -#define fwd_rnd2(arg, table) \ - save (0,r1); \ - save (1,r5); \ - \ - /* compute new column values */ \ - do_fcol(table, r0,r5,r4,r1, r2,r3, arg); /* idx=r2 */ \ - do_col (table, r4,r1,r0,r5, r2,r3); /* idx=r4 */ \ - restore(r2,0); \ - do_col (table, r1,r0,r5,r4, r2,r3); /* idx=r1 */ \ - restore(r2,1); \ - do_col (table, r5,r4,r1,r0, r2,r3); /* idx=r5 */ - -// These macros performs an inverse encryption cycle. They are entered with -// the first previous round column values in r0,r1,r4,r5 and -// exit with the final values in the same registers, using stack -// for temporary storage - -// round column values -// on entry: r0,r1,r4,r5 -// on exit: r2,r1,r4,r5 -#define inv_rnd1(arg, table) \ - save (0,r1); \ - save (1,r5); \ - \ - /* compute new column values */ \ - do_icol(table, r2,r1,r4,r5, r0,r3, arg); /* idx=r0 */ \ - do_col (table, r4,r5,r2,r1, r0,r3); /* idx=r4 */ \ - restore(r0,0); \ - do_col (table, r1,r4,r5,r2, r0,r3); /* idx=r1 */ \ - restore(r0,1); \ - do_col (table, r5,r2,r1,r4, r0,r3); /* idx=r5 */ - -// round column values -// on entry: r2,r1,r4,r5 -// on exit: r0,r1,r4,r5 -#define inv_rnd2(arg, table) \ - save (0,r1); \ - save (1,r5); \ - \ - /* compute new column values */ \ - do_icol(table, r0,r1,r4,r5, r2,r3, arg); /* idx=r2 */ \ - do_col (table, r4,r5,r0,r1, r2,r3); /* idx=r4 */ \ - restore(r2,0); \ - do_col (table, r1,r4,r5,r0, r2,r3); /* idx=r1 */ \ - restore(r2,1); \ - do_col (table, r5,r0,r1,r4, r2,r3); /* idx=r5 */ - -// AES (Rijndael) Encryption Subroutine -/* void aes_enc_blk(struct crypto_aes_ctx *ctx, u8 *out_blk, const u8 *in_blk) */ - -.extern crypto_ft_tab -.extern crypto_fl_tab - -ENTRY(aes_enc_blk) - push %ebp - mov ctx(%esp),%ebp - -// CAUTION: the order and the values used in these assigns -// rely on the register mappings - -1: push %ebx - mov in_blk+4(%esp),%r2 - push %esi - mov klen(%ebp),%r3 // key size - push %edi -#if ekey != 0 - lea ekey(%ebp),%ebp // key pointer -#endif - -// input four columns and xor in first round key - - mov (%r2),%r0 - mov 4(%r2),%r1 - mov 8(%r2),%r4 - mov 12(%r2),%r5 - xor (%ebp),%r0 - xor 4(%ebp),%r1 - xor 8(%ebp),%r4 - xor 12(%ebp),%r5 - - sub $8,%esp // space for register saves on stack - add $16,%ebp // increment to next round key - cmp $24,%r3 - jb 4f // 10 rounds for 128-bit key - lea 32(%ebp),%ebp - je 3f // 12 rounds for 192-bit key - lea 32(%ebp),%ebp - -2: fwd_rnd1( -64(%ebp), crypto_ft_tab) // 14 rounds for 256-bit key - fwd_rnd2( -48(%ebp), crypto_ft_tab) -3: fwd_rnd1( -32(%ebp), crypto_ft_tab) // 12 rounds for 192-bit key - fwd_rnd2( -16(%ebp), crypto_ft_tab) -4: fwd_rnd1( (%ebp), crypto_ft_tab) // 10 rounds for 128-bit key - fwd_rnd2( +16(%ebp), crypto_ft_tab) - fwd_rnd1( +32(%ebp), crypto_ft_tab) - fwd_rnd2( +48(%ebp), crypto_ft_tab) - fwd_rnd1( +64(%ebp), crypto_ft_tab) - fwd_rnd2( +80(%ebp), crypto_ft_tab) - fwd_rnd1( +96(%ebp), crypto_ft_tab) - fwd_rnd2(+112(%ebp), crypto_ft_tab) - fwd_rnd1(+128(%ebp), crypto_ft_tab) - fwd_rnd2(+144(%ebp), crypto_fl_tab) // last round uses a different table - -// move final values to the output array. CAUTION: the -// order of these assigns rely on the register mappings - - add $8,%esp - mov out_blk+12(%esp),%ebp - mov %r5,12(%ebp) - pop %edi - mov %r4,8(%ebp) - pop %esi - mov %r1,4(%ebp) - pop %ebx - mov %r0,(%ebp) - pop %ebp - ret -ENDPROC(aes_enc_blk) - -// AES (Rijndael) Decryption Subroutine -/* void aes_dec_blk(struct crypto_aes_ctx *ctx, u8 *out_blk, const u8 *in_blk) */ - -.extern crypto_it_tab -.extern crypto_il_tab - -ENTRY(aes_dec_blk) - push %ebp - mov ctx(%esp),%ebp - -// CAUTION: the order and the values used in these assigns -// rely on the register mappings - -1: push %ebx - mov in_blk+4(%esp),%r2 - push %esi - mov klen(%ebp),%r3 // key size - push %edi -#if dkey != 0 - lea dkey(%ebp),%ebp // key pointer -#endif - -// input four columns and xor in first round key - - mov (%r2),%r0 - mov 4(%r2),%r1 - mov 8(%r2),%r4 - mov 12(%r2),%r5 - xor (%ebp),%r0 - xor 4(%ebp),%r1 - xor 8(%ebp),%r4 - xor 12(%ebp),%r5 - - sub $8,%esp // space for register saves on stack - add $16,%ebp // increment to next round key - cmp $24,%r3 - jb 4f // 10 rounds for 128-bit key - lea 32(%ebp),%ebp - je 3f // 12 rounds for 192-bit key - lea 32(%ebp),%ebp - -2: inv_rnd1( -64(%ebp), crypto_it_tab) // 14 rounds for 256-bit key - inv_rnd2( -48(%ebp), crypto_it_tab) -3: inv_rnd1( -32(%ebp), crypto_it_tab) // 12 rounds for 192-bit key - inv_rnd2( -16(%ebp), crypto_it_tab) -4: inv_rnd1( (%ebp), crypto_it_tab) // 10 rounds for 128-bit key - inv_rnd2( +16(%ebp), crypto_it_tab) - inv_rnd1( +32(%ebp), crypto_it_tab) - inv_rnd2( +48(%ebp), crypto_it_tab) - inv_rnd1( +64(%ebp), crypto_it_tab) - inv_rnd2( +80(%ebp), crypto_it_tab) - inv_rnd1( +96(%ebp), crypto_it_tab) - inv_rnd2(+112(%ebp), crypto_it_tab) - inv_rnd1(+128(%ebp), crypto_it_tab) - inv_rnd2(+144(%ebp), crypto_il_tab) // last round uses a different table - -// move final values to the output array. CAUTION: the -// order of these assigns rely on the register mappings - - add $8,%esp - mov out_blk+12(%esp),%ebp - mov %r5,12(%ebp) - pop %edi - mov %r4,8(%ebp) - pop %esi - mov %r1,4(%ebp) - pop %ebx - mov %r0,(%ebp) - pop %ebp - ret -ENDPROC(aes_dec_blk) diff --git a/arch/x86/crypto/aes-x86_64-asm_64.S b/arch/x86/crypto/aes-x86_64-asm_64.S deleted file mode 100644 index 8739cf7795de..000000000000 --- a/arch/x86/crypto/aes-x86_64-asm_64.S +++ /dev/null @@ -1,185 +0,0 @@ -/* AES (Rijndael) implementation (FIPS PUB 197) for x86_64 - * - * Copyright (C) 2005 Andreas Steinmetz, - * - * License: - * This code can be distributed under the terms of the GNU General Public - * License (GPL) Version 2 provided that the above header down to and - * including this sentence is retained in full. - */ - -.extern crypto_ft_tab -.extern crypto_it_tab -.extern crypto_fl_tab -.extern crypto_il_tab - -.text - -#include -#include - -#define R1 %rax -#define R1E %eax -#define R1X %ax -#define R1H %ah -#define R1L %al -#define R2 %rbx -#define R2E %ebx -#define R2X %bx -#define R2H %bh -#define R2L %bl -#define R3 %rcx -#define R3E %ecx -#define R3X %cx -#define R3H %ch -#define R3L %cl -#define R4 %rdx -#define R4E %edx -#define R4X %dx -#define R4H %dh -#define R4L %dl -#define R5 %rsi -#define R5E %esi -#define R6 %rdi -#define R6E %edi -#define R7 %r9 /* don't use %rbp; it breaks stack traces */ -#define R7E %r9d -#define R8 %r8 -#define R10 %r10 -#define R11 %r11 - -#define prologue(FUNC,KEY,B128,B192,r1,r2,r5,r6,r7,r8,r9,r10,r11) \ - ENTRY(FUNC); \ - movq r1,r2; \ - leaq KEY+48(r8),r9; \ - movq r10,r11; \ - movl (r7),r5 ## E; \ - movl 4(r7),r1 ## E; \ - movl 8(r7),r6 ## E; \ - movl 12(r7),r7 ## E; \ - movl 480(r8),r10 ## E; \ - xorl -48(r9),r5 ## E; \ - xorl -44(r9),r1 ## E; \ - xorl -40(r9),r6 ## E; \ - xorl -36(r9),r7 ## E; \ - cmpl $24,r10 ## E; \ - jb B128; \ - leaq 32(r9),r9; \ - je B192; \ - leaq 32(r9),r9; - -#define epilogue(FUNC,r1,r2,r5,r6,r7,r8,r9) \ - movq r1,r2; \ - movl r5 ## E,(r9); \ - movl r6 ## E,4(r9); \ - movl r7 ## E,8(r9); \ - movl r8 ## E,12(r9); \ - ret; \ - ENDPROC(FUNC); - -#define round(TAB,OFFSET,r1,r2,r3,r4,r5,r6,r7,r8,ra,rb,rc,rd) \ - movzbl r2 ## H,r5 ## E; \ - movzbl r2 ## L,r6 ## E; \ - movl TAB+1024(,r5,4),r5 ## E;\ - movw r4 ## X,r2 ## X; \ - movl TAB(,r6,4),r6 ## E; \ - roll $16,r2 ## E; \ - shrl $16,r4 ## E; \ - movzbl r4 ## L,r7 ## E; \ - movzbl r4 ## H,r4 ## E; \ - xorl OFFSET(r8),ra ## E; \ - xorl OFFSET+4(r8),rb ## E; \ - xorl TAB+3072(,r4,4),r5 ## E;\ - xorl TAB+2048(,r7,4),r6 ## E;\ - movzbl r1 ## L,r7 ## E; \ - movzbl r1 ## H,r4 ## E; \ - movl TAB+1024(,r4,4),r4 ## E;\ - movw r3 ## X,r1 ## X; \ - roll $16,r1 ## E; \ - shrl $16,r3 ## E; \ - xorl TAB(,r7,4),r5 ## E; \ - movzbl r3 ## L,r7 ## E; \ - movzbl r3 ## H,r3 ## E; \ - xorl TAB+3072(,r3,4),r4 ## E;\ - xorl TAB+2048(,r7,4),r5 ## E;\ - movzbl r1 ## L,r7 ## E; \ - movzbl r1 ## H,r3 ## E; \ - shrl $16,r1 ## E; \ - xorl TAB+3072(,r3,4),r6 ## E;\ - movl TAB+2048(,r7,4),r3 ## E;\ - movzbl r1 ## L,r7 ## E; \ - movzbl r1 ## H,r1 ## E; \ - xorl TAB+1024(,r1,4),r6 ## E;\ - xorl TAB(,r7,4),r3 ## E; \ - movzbl r2 ## H,r1 ## E; \ - movzbl r2 ## L,r7 ## E; \ - shrl $16,r2 ## E; \ - xorl TAB+3072(,r1,4),r3 ## E;\ - xorl TAB+2048(,r7,4),r4 ## E;\ - movzbl r2 ## H,r1 ## E; \ - movzbl r2 ## L,r2 ## E; \ - xorl OFFSET+8(r8),rc ## E; \ - xorl OFFSET+12(r8),rd ## E; \ - xorl TAB+1024(,r1,4),r3 ## E;\ - xorl TAB(,r2,4),r4 ## E; - -#define move_regs(r1,r2,r3,r4) \ - movl r3 ## E,r1 ## E; \ - movl r4 ## E,r2 ## E; - -#define entry(FUNC,KEY,B128,B192) \ - prologue(FUNC,KEY,B128,B192,R2,R8,R1,R3,R4,R6,R10,R5,R11) - -#define return(FUNC) epilogue(FUNC,R8,R2,R5,R6,R3,R4,R11) - -#define encrypt_round(TAB,OFFSET) \ - round(TAB,OFFSET,R1,R2,R3,R4,R5,R6,R7,R10,R5,R6,R3,R4) \ - move_regs(R1,R2,R5,R6) - -#define encrypt_final(TAB,OFFSET) \ - round(TAB,OFFSET,R1,R2,R3,R4,R5,R6,R7,R10,R5,R6,R3,R4) - -#define decrypt_round(TAB,OFFSET) \ - round(TAB,OFFSET,R2,R1,R4,R3,R6,R5,R7,R10,R5,R6,R3,R4) \ - move_regs(R1,R2,R5,R6) - -#define decrypt_final(TAB,OFFSET) \ - round(TAB,OFFSET,R2,R1,R4,R3,R6,R5,R7,R10,R5,R6,R3,R4) - -/* void aes_enc_blk(stuct crypto_tfm *tfm, u8 *out, const u8 *in) */ - - entry(aes_enc_blk,0,.Le128,.Le192) - encrypt_round(crypto_ft_tab,-96) - encrypt_round(crypto_ft_tab,-80) -.Le192: encrypt_round(crypto_ft_tab,-64) - encrypt_round(crypto_ft_tab,-48) -.Le128: encrypt_round(crypto_ft_tab,-32) - encrypt_round(crypto_ft_tab,-16) - encrypt_round(crypto_ft_tab, 0) - encrypt_round(crypto_ft_tab, 16) - encrypt_round(crypto_ft_tab, 32) - encrypt_round(crypto_ft_tab, 48) - encrypt_round(crypto_ft_tab, 64) - encrypt_round(crypto_ft_tab, 80) - encrypt_round(crypto_ft_tab, 96) - encrypt_final(crypto_fl_tab,112) - return(aes_enc_blk) - -/* void aes_dec_blk(struct crypto_tfm *tfm, u8 *out, const u8 *in) */ - - entry(aes_dec_blk,240,.Ld128,.Ld192) - decrypt_round(crypto_it_tab,-96) - decrypt_round(crypto_it_tab,-80) -.Ld192: decrypt_round(crypto_it_tab,-64) - decrypt_round(crypto_it_tab,-48) -.Ld128: decrypt_round(crypto_it_tab,-32) - decrypt_round(crypto_it_tab,-16) - decrypt_round(crypto_it_tab, 0) - decrypt_round(crypto_it_tab, 16) - decrypt_round(crypto_it_tab, 32) - decrypt_round(crypto_it_tab, 48) - decrypt_round(crypto_it_tab, 64) - decrypt_round(crypto_it_tab, 80) - decrypt_round(crypto_it_tab, 96) - decrypt_final(crypto_il_tab,112) - return(aes_dec_blk) diff --git a/arch/x86/crypto/aes_glue.c b/arch/x86/crypto/aes_glue.c deleted file mode 100644 index e26984f7ab8d..000000000000 --- a/arch/x86/crypto/aes_glue.c +++ /dev/null @@ -1,70 +0,0 @@ -/* - * Glue Code for the asm optimized version of the AES Cipher Algorithm - * - */ - -#include -#include -#include - -asmlinkage void aes_enc_blk(struct crypto_aes_ctx *ctx, u8 *out, const u8 *in); -asmlinkage void aes_dec_blk(struct crypto_aes_ctx *ctx, u8 *out, const u8 *in); - -void crypto_aes_encrypt_x86(struct crypto_aes_ctx *ctx, u8 *dst, const u8 *src) -{ - aes_enc_blk(ctx, dst, src); -} -EXPORT_SYMBOL_GPL(crypto_aes_encrypt_x86); - -void crypto_aes_decrypt_x86(struct crypto_aes_ctx *ctx, u8 *dst, const u8 *src) -{ - aes_dec_blk(ctx, dst, src); -} -EXPORT_SYMBOL_GPL(crypto_aes_decrypt_x86); - -static void aes_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) -{ - aes_enc_blk(crypto_tfm_ctx(tfm), dst, src); -} - -static void aes_decrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) -{ - aes_dec_blk(crypto_tfm_ctx(tfm), dst, src); -} - -static struct crypto_alg aes_alg = { - .cra_name = "aes", - .cra_driver_name = "aes-asm", - .cra_priority = 200, - .cra_flags = CRYPTO_ALG_TYPE_CIPHER, - .cra_blocksize = AES_BLOCK_SIZE, - .cra_ctxsize = sizeof(struct crypto_aes_ctx), - .cra_module = THIS_MODULE, - .cra_u = { - .cipher = { - .cia_min_keysize = AES_MIN_KEY_SIZE, - .cia_max_keysize = AES_MAX_KEY_SIZE, - .cia_setkey = crypto_aes_set_key, - .cia_encrypt = aes_encrypt, - .cia_decrypt = aes_decrypt - } - } -}; - -static int __init aes_init(void) -{ - return crypto_register_alg(&aes_alg); -} - -static void __exit aes_fini(void) -{ - crypto_unregister_alg(&aes_alg); -} - -module_init(aes_init); -module_exit(aes_fini); - -MODULE_DESCRIPTION("Rijndael (AES) Cipher Algorithm, asm optimized"); -MODULE_LICENSE("GPL"); -MODULE_ALIAS_CRYPTO("aes"); -MODULE_ALIAS_CRYPTO("aes-asm"); diff --git a/crypto/Kconfig b/crypto/Kconfig index 20af58068e6b..df6f0be66574 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -1108,50 +1108,6 @@ config CRYPTO_AES_TI block. Interrupts are also disabled to avoid races where cachelines are evicted when the CPU is interrupted to do something else. -config CRYPTO_AES_586 - tristate "AES cipher algorithms (i586)" - depends on (X86 || UML_X86) && !64BIT - select CRYPTO_ALGAPI - select CRYPTO_AES - help - AES cipher algorithms (FIPS-197). AES uses the Rijndael - algorithm. - - Rijndael appears to be consistently a very good performer in - both hardware and software across a wide range of computing - environments regardless of its use in feedback or non-feedback - modes. Its key setup time is excellent, and its key agility is - good. Rijndael's very low memory requirements make it very well - suited for restricted-space environments, in which it also - demonstrates excellent performance. Rijndael's operations are - among the easiest to defend against power and timing attacks. - - The AES specifies three key sizes: 128, 192 and 256 bits - - See for more information. - -config CRYPTO_AES_X86_64 - tristate "AES cipher algorithms (x86_64)" - depends on (X86 || UML_X86) && 64BIT - select CRYPTO_ALGAPI - select CRYPTO_AES - help - AES cipher algorithms (FIPS-197). AES uses the Rijndael - algorithm. - - Rijndael appears to be consistently a very good performer in - both hardware and software across a wide range of computing - environments regardless of its use in feedback or non-feedback - modes. Its key setup time is excellent, and its key agility is - good. Rijndael's very low memory requirements make it very well - suited for restricted-space environments, in which it also - demonstrates excellent performance. Rijndael's operations are - among the easiest to defend against power and timing attacks. - - The AES specifies three key sizes: 128, 192 and 256 bits - - See for more information. - config CRYPTO_AES_NI_INTEL tristate "AES cipher algorithms (AES-NI)" depends on X86 From patchwork Thu Jun 27 10:26:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167904 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148520ilk; Thu, 27 Jun 2019 03:28:02 -0700 (PDT) X-Google-Smtp-Source: APXvYqyFGDKKGzBPR1J0UxgObqOlItLTNdq6C0HBsittg1lC0DGgFOl4p0S+hFS6GZF4UohVsFF6 X-Received: by 2002:a65:498f:: with SMTP id r15mr3186741pgs.37.1561631282822; Thu, 27 Jun 2019 03:28:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631282; cv=none; d=google.com; s=arc-20160816; b=BALH03pEcMER8S4l4hVpt7K7oLIZR+yR8LDs0AhPMV32FYQed4VdWdxQKL3ODB9gH/ q2YPnbwY8ZgxY85HKz9DlwOi1JgZ+pZYJ1vl3+QWPIo4lqBq6J4pLsP+3/Jp8KA0C321 l5Zv8iPTNMHoU7SmeMuoPLUJxH8aIAAjjbYEOhjmILPKZ2vAo0tA4wB//n+AukogNOV2 nBy9mrPN8zL2XpYGTkN1y2/cbSCsH0/sR5e+2+rVDwCvjP1+KEKo54clcTNFAETc/uUx xZ9sBxrta46aCY196z+7CpuHzf6xfP+JIcD2OiEI2saBSMAAGgqFLU1i+4NGD5zuiHP1 g1kw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QMunKwfP+ecJeJsjLgotGX9cYxNfDTWVUzGiozUxkpA=; b=fsPfeOjnH8GTs+urpqwbKJEizTMU5BVa4GUkr/PCrm7mxOr4O/63eiMlqdvdWP7XEk NSR1/rMqrbeuONuQ/hEKkoudJm3Y8vSA34LCEL9NSnN8vePtwjPQAws8pXYk6K5UogD1 NwvBS6RfLEC8fwKMbvkQEKwRTWBx4mAxNivPXYPljVLyJLs9U0smKuhhxD5uufdrL2+F DdgC2z4kAnoir7D9YSQ7c8fpLZgdrryNn42k8E+2Vv7Lj+fl9vNexBisPTJ+o4ZgE4D2 CTOQa5pnOoYHe8MxZrOBYOkIvk/bg0B/POVXn3EclLzNdFfQK5jsDpOoCkpQ2OwdiYlP aiLw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uLHuagjD; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.02; Thu, 27 Jun 2019 03:28:02 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uLHuagjD; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726605AbfF0K2B (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:01 -0400 Received: from mail-wr1-f66.google.com ([209.85.221.66]:39891 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726659AbfF0K2A (ORCPT ); Thu, 27 Jun 2019 06:28:00 -0400 Received: by mail-wr1-f66.google.com with SMTP id x4so1917535wrt.6 for ; Thu, 27 Jun 2019 03:27:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QMunKwfP+ecJeJsjLgotGX9cYxNfDTWVUzGiozUxkpA=; b=uLHuagjDQQCYBBnoMEamxGqV4L4MCtbHGa83HxXpNaJVcUgZEpyvA/9DfVcmy95RKl onsUCiWAVXeD/O9pB7EWYxkwoja8E5JbzKdmkpea44TZ9GjvS8U3bD559XIAFXnnhGJ8 JQwckJYgoMFq4K1uxkE4RvYShHHMDglwCvpqUGiCKeKn+F0zBRIybLLcjXdr3tFcxgp4 pGI3N/truUTGhQP+3PnOcNc4xy7FdRywzMQifzuSeNSNTDlauRO64F2fvQkoYhGwGPgD oAx4/THjaLTeJ176idpaP2bo5DdA1xYiehkLPfONRMDBEFz4GXJn6v6RtgkHnUfNs2fF rFaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QMunKwfP+ecJeJsjLgotGX9cYxNfDTWVUzGiozUxkpA=; b=HQTsUxUEp1x0tax1AnPjQv9a+zRExKWrMtRSbgOO71iRvlhoGTk6gQB/bYAwf+oFia t0pOb5JM6CvLUww1uhZYpGFUs9L7pv3BAZXU+zhJrxRJNkvz9vI+X7Zdf/29igxuKtaH Nv4bHsoKStHxLNw8JfoX20qVDt17kz5CWsd0My6gMTrAZ5QeN9T2wLKMut+0KW3nsYOa 17AexzlXDsXrSZLNKVcmetBmSiLRUQ+K5rlnOxdtH8IWTia7MHLCc1pefxkQyVQ6muoM J0b8lFQJ3YmCxYLIWzHXWK67opN5XnNuCPk2OFQmEpZhnMt4ump54KVh9J08avNiWLVa 5sfw== X-Gm-Message-State: APjAAAVC+YqCr9XFq3UcYV7FWbSraSelS3WVy2U+HEGw4qa/UbKzSMTa zoUk5wcXljHFVgpAOl9qqfzpvxm4sd0= X-Received: by 2002:a05:6000:11c2:: with SMTP id i2mr2723893wrx.199.1561631278266; Thu, 27 Jun 2019 03:27:58 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.27.57 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:27:57 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 07/32] crypto: padlock/aes - switch to library version of key expansion routine Date: Thu, 27 Jun 2019 12:26:22 +0200 Message-Id: <20190627102647.2992-8-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Switch to the new AES library that also provides an implementation of the AES key expansion routine. This removes the dependency on the generic AES cipher, allowing it to be omitted entirely in the future. Signed-off-by: Ard Biesheuvel --- drivers/crypto/Kconfig | 2 +- drivers/crypto/padlock-aes.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- 2.20.1 diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig index 67af688d7d84..3fca5f7e38f0 100644 --- a/drivers/crypto/Kconfig +++ b/drivers/crypto/Kconfig @@ -26,7 +26,7 @@ config CRYPTO_DEV_PADLOCK_AES tristate "PadLock driver for AES algorithm" depends on CRYPTO_DEV_PADLOCK select CRYPTO_BLKCIPHER - select CRYPTO_AES + select CRYPTO_LIB_AES help Use VIA PadLock for AES algorithm. diff --git a/drivers/crypto/padlock-aes.c b/drivers/crypto/padlock-aes.c index 854539512c35..af90138eddb7 100644 --- a/drivers/crypto/padlock-aes.c +++ b/drivers/crypto/padlock-aes.c @@ -144,7 +144,7 @@ static int aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, ctx->cword.encrypt.keygen = 1; ctx->cword.decrypt.keygen = 1; - if (crypto_aes_expand_key(&gen_aes, in_key, key_len)) { + if (aes_expandkey(&gen_aes, in_key, key_len)) { *flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; return -EINVAL; } From patchwork Thu Jun 27 10:26:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167905 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148536ilk; Thu, 27 Jun 2019 03:28:03 -0700 (PDT) X-Google-Smtp-Source: APXvYqzzDVGC/kWlyEB8Xa6yCiWK7ol7VJGTAo/P7B/4Gpx1MfSo05ruVwyyQLfNjG4fQzcea0d/ X-Received: by 2002:a65:63cd:: with SMTP id n13mr3055471pgv.153.1561631283522; Thu, 27 Jun 2019 03:28:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631283; cv=none; d=google.com; s=arc-20160816; b=BE2oYlxdm7PB35BjXpQYwLeqlUgqX4m3o2IH+au65taiecG/6HXH/968XeUX7oInYY L/+qqmMeYzL+qcWLpUJiMDBJsTZAi5FFU+YMuFaEpEnurZBacC0n27KWRBQ149dL9tlA 5OSD4RJsXjSfK08zTW6+f6hF4woHmiv1jQGbFnvXsaxyOmnnuTfF9M46T5hLF841MsbE xm9bXsMUdDnsfxtBwT0zxAzD7YF6t6ahav1W/erlxnCzMnPoafX1QYlkGqAbTvOKg1eZ 1V8Mjqnl/H8LgjgzBSL/Zqe69feY0Rch9F2P0fdbeVtmKg+X9oLEwlLy8WAoCMbwpCo6 eyhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=lQLA1Z6u9tD3If/MHOVikbtefg1FLPAfk8jJsqm/d/c=; b=BWw1O4jr3HBshWRVoR5juGdE0jKn7VhiJRjyN0Av+LY3PYp4G4+3P3fNy9NGX8tJmm 4CgJc1UklOrVDLDxclkBxQgnZbMQOtVCJqUZTBlj1/yb5Pd9dOwHTYC8CP/qX9k/NYL3 xEd7zH/7ydA0UeNQzSACbF2gjyjbfn74wKFMjTQt4ZgbiUAfxIjMcwCJYk1OgZLfi73Y KDu7RCLk5Z+dEqGWfge0kMmnAvVwd67gD1sg+ttA2e1ni7ywRtJi1iDRTskVzAGZRyMh rrs+0R3TGtfEihlT9JG8Xl1U6ug31J0CnbPAJfLsm21pIGVrYNDoRKkuMGkY5WDDygwh PdyA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lxLzt3NF; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.03; Thu, 27 Jun 2019 03:28:03 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lxLzt3NF; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726668AbfF0K2C (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:02 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:46507 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726664AbfF0K2C (ORCPT ); Thu, 27 Jun 2019 06:28:02 -0400 Received: by mail-wr1-f65.google.com with SMTP id n4so1887287wrw.13 for ; Thu, 27 Jun 2019 03:28:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lQLA1Z6u9tD3If/MHOVikbtefg1FLPAfk8jJsqm/d/c=; b=lxLzt3NFRfs6f4epK37Ltp6P42+8nW+jSteVOJmx+PVMzF8CiB0qq1PuFWKxbSE/Gz FWaVyiOebaq0oEA4LaB4G04SPBXm7uaOzewDGxwEcfb5DOYkbmA76+qfjMBBThUVl10C 4sJi0XWZECubmCmdvalGxqbAGptUz21vuN6EUEzPrvTsndZrvkSDbMwriQnirl2C5G12 U6ZnRSmLg9Ai8o4r4NW6vilB9Qs0JckXVQwq1Iv2XNrjo6i+wF8vLV4R8S+EFVA8AmFe 5Wam4ah9cVv0haFjoR6rlLgeB4uJPhSbZ7JGhj4sBvLAOI08ZnGFrniADZMgFuttDpan jpIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lQLA1Z6u9tD3If/MHOVikbtefg1FLPAfk8jJsqm/d/c=; b=JFb/JtTmAx3EHuYSjJIM/OUmhOo1tadJETOL5eLX1c/x6gEUixMuJY0jjDxyuBWudJ 7QvbqjWtfb0z3g9y6IpyD7uE/k4yo8iEJw4n8iAVdRPlepd/rnWJFe10Ul+1OBZpFBRJ SOYvv++xwqdB5CLHnDqxa3AJHYJvCcW95ZZ1sW7mMJBzkIMSaIKhBntFmCLEtUIjRHns 7sgiVoa3ADKad6neaw43guMF03Ebv8gnlP85BiHT9b5Yh/CvoJlDxac2rXEcE30GrM/C I0hgDgQqAA/NhhGU6uG1FKHsPCMGgFzg+IT+D+JLkP0DL8y1Dwgr3QmeH1ZqumTe6jE0 s3zw== X-Gm-Message-State: APjAAAW7hvoXpC/v31t2GjxH7FfLde/0L4W1dcvBXGi6GbWZKLT66a8X pnAjIimkEG7Uw3qwiFgbdbmWs+e4Hbc= X-Received: by 2002:a5d:5745:: with SMTP id q5mr2798134wrw.75.1561631279558; Thu, 27 Jun 2019 03:27:59 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.27.58 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:27:58 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 08/32] crypto: cesa/aes - switch to library version of key expansion routine Date: Thu, 27 Jun 2019 12:26:23 +0200 Message-Id: <20190627102647.2992-9-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Switch to the new AES library that also provides an implementation of the AES key expansion routine. This removes the dependency on the generic AES cipher, allowing it to be omitted entirely in the future. Signed-off-by: Ard Biesheuvel --- drivers/crypto/Kconfig | 2 +- drivers/crypto/marvell/cipher.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- 2.20.1 diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig index 3fca5f7e38f0..fdccadc94819 100644 --- a/drivers/crypto/Kconfig +++ b/drivers/crypto/Kconfig @@ -213,7 +213,7 @@ config CRYPTO_CRC32_S390 config CRYPTO_DEV_MARVELL_CESA tristate "Marvell's Cryptographic Engine driver" depends on PLAT_ORION || ARCH_MVEBU - select CRYPTO_AES + select CRYPTO_LIB_AES select CRYPTO_DES select CRYPTO_BLKCIPHER select CRYPTO_HASH diff --git a/drivers/crypto/marvell/cipher.c b/drivers/crypto/marvell/cipher.c index 2fd936b19c6d..debe7d9f00ae 100644 --- a/drivers/crypto/marvell/cipher.c +++ b/drivers/crypto/marvell/cipher.c @@ -257,7 +257,7 @@ static int mv_cesa_aes_setkey(struct crypto_skcipher *cipher, const u8 *key, int ret; int i; - ret = crypto_aes_expand_key(&ctx->aes, key, len); + ret = aes_expandkey(&ctx->aes, key, len); if (ret) { crypto_skcipher_set_flags(cipher, CRYPTO_TFM_RES_BAD_KEY_LEN); return ret; From patchwork Thu Jun 27 10:26:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167906 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148550ilk; Thu, 27 Jun 2019 03:28:04 -0700 (PDT) X-Google-Smtp-Source: APXvYqwqIEzo30gJZ0sDuFbD1w6idikRbe3vqARwOBhBD3Zb/yiMt+Y9DgVwXi6wNjcIk6OxmHIV X-Received: by 2002:a17:90a:601:: with SMTP id j1mr5088125pjj.96.1561631283910; Thu, 27 Jun 2019 03:28:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631283; cv=none; d=google.com; s=arc-20160816; b=hB6WyOlRru5BRwv2iiZRWd6zV4MyIvkjg7+85nY3p5tS/yE2r0cyki884ufcECf1p6 6mj6hXxHSUGucXQSk9XzY3WDHaynC7qjJa+HwuOOPmtME6iDQhDfdpjQGosdUFHJ6c8E DNARTqu8JRyVKqS9aISta1bLZwerI77EH58BxSwNxQjjgDM51erla7xeZM8bvo5N8pNk 5a3vxP0wdUH22u0zZuihNiFH2EXMML9bduqNk+pHRtUBDVimXvmh8ByOlzhwjJ2VyP8E 5djnNCZhQHYzgZWr7C8fwGJfDBfzgV1d2T2kI7k6yk8XN2xSQJJOnRTuGnio6knwI6vr MlzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=TEkyEzYA9z24dLcVZzAphhOs6uMPB3TRScCj5mulxhM=; b=GQ0txNXmzLt1JMfHgWTuRn79fJ3W4TWbe7FGhtvu+NotDnBRL86h8RkCm5VWVTgftY r7k18qcE0GdsY7K9Nc3C+UQzBDCHCBjkYjDCF+Ghe3Yr3s4qkiHi/tfU7SfeNmM0M/CL e3JRelx5TAWeMNO6z/hBucqnW/UUb4ly3d2xZb1p1g5VXqi5mRkSkzPqXjABbVQRXf4g mKG0d/4EE1h0GzLpomo72UNvpNatIayRzCOBnQBPZUBsp23TdmwHyQp1+KFF/BmgJQew lMMvSQ4PpJTlYUg/dlJ2bPNnfU/yYqEXTPChHoF6+T4TSEy2Rnj71BACpnMQKyeml8Bl aAOA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=F0Qzyeux; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.03; Thu, 27 Jun 2019 03:28:03 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=F0Qzyeux; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726664AbfF0K2D (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:03 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:44949 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726659AbfF0K2C (ORCPT ); Thu, 27 Jun 2019 06:28:02 -0400 Received: by mail-wr1-f68.google.com with SMTP id r16so49154wrl.11 for ; Thu, 27 Jun 2019 03:28:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TEkyEzYA9z24dLcVZzAphhOs6uMPB3TRScCj5mulxhM=; b=F0Qzyeux+dCKngRehSdIDngCe+wHs5fNOoDW3q126dSrSAGOF270GYPZwYhrga9cBy mx/RvqAXRoHBNEc2bz2kuKB7eQjog1PHAukgcxKr8QoiFDwnQQpQbZG6tqMu9Zw/+pKy vE7Fi4EanxaBgLFcLkIVyDgktm+w7S81STdQ1yGvoqdZ72SirCamZPeUFd7q0ejBpSCK HsGEDn4cc7KQRlnaJIpQvxO2q+Ha77KaTfpLbqVgfFPOrNPSfZTCMhKawDCw8p9yEjap NzvOGCriAYL0nYiCaz1zoCkjbDwb9uUq1G8VP7I+6Iv0oNc39g57S4JrEBv24/KaNYmc V2wQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TEkyEzYA9z24dLcVZzAphhOs6uMPB3TRScCj5mulxhM=; b=GToyu3lokOoe9WccP+uLJ7Fr2vQn0JBjsUoD0skqTI3uT0/w7KEt4W7tYcqFOaLv+2 BVFY5hekwuBMWSlFxwJAulrf0kskKd9MegMBw1wiVvrmt+itBSQUpKY05agwjJaByvVc p1Oacs3anenRUke9//+ORTyh6j/GnbSHSad+cw3Y2DIleEOVcd6u+CE1Mb0kuemNU7xu 70orcMmJtUh8bpKk8z15GiBJQuw6U2WioYs3ySAe1ZJRG+bFJIJ/EI1HsQtcLot1ihZr dZADcqy7P4QFMvU63hVvu09ID6d+o+PLfPmms1S7J/+a2HoKXWIlsZQuKjY9L60j5lbO B1AQ== X-Gm-Message-State: APjAAAXF5FZvdqpWg/NnS81g3YVpBQRFq3NtFsI54bV+gwQIcMhciNOt OilNJVwma2FDqo+J+Oe0ZNwZe1p0NGg= X-Received: by 2002:a5d:63c9:: with SMTP id c9mr2802409wrw.81.1561631280581; Thu, 27 Jun 2019 03:28:00 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.27.59 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:27:59 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 09/32] crypto: safexcel/aes - switch to library version of key expansion routine Date: Thu, 27 Jun 2019 12:26:24 +0200 Message-Id: <20190627102647.2992-10-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Switch to the new AES library that also provides an implementation of the AES key expansion routine. This removes the dependency on the generic AES cipher, allowing it to be omitted entirely in the future. Signed-off-by: Ard Biesheuvel --- drivers/crypto/Kconfig | 2 +- drivers/crypto/inside-secure/safexcel_cipher.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- 2.20.1 diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig index fdccadc94819..b30b84089d11 100644 --- a/drivers/crypto/Kconfig +++ b/drivers/crypto/Kconfig @@ -718,7 +718,7 @@ config CRYPTO_DEV_SAFEXCEL tristate "Inside Secure's SafeXcel cryptographic engine driver" depends on OF depends on (ARM64 && ARCH_MVEBU) || (COMPILE_TEST && 64BIT) - select CRYPTO_AES + select CRYPTO_LIB_AES select CRYPTO_AUTHENC select CRYPTO_BLKCIPHER select CRYPTO_DES diff --git a/drivers/crypto/inside-secure/safexcel_cipher.c b/drivers/crypto/inside-secure/safexcel_cipher.c index 8cdbdbe35681..19ec086dce4f 100644 --- a/drivers/crypto/inside-secure/safexcel_cipher.c +++ b/drivers/crypto/inside-secure/safexcel_cipher.c @@ -178,7 +178,7 @@ static int safexcel_skcipher_aes_setkey(struct crypto_skcipher *ctfm, struct crypto_aes_ctx aes; int ret, i; - ret = crypto_aes_expand_key(&aes, key, len); + ret = aes_expandkey(&aes, key, len); if (ret) { crypto_skcipher_set_flags(ctfm, CRYPTO_TFM_RES_BAD_KEY_LEN); return ret; From patchwork Thu Jun 27 10:26:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167907 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148599ilk; Thu, 27 Jun 2019 03:28:06 -0700 (PDT) X-Google-Smtp-Source: APXvYqyNscsqy1vJomV1rx2EJQqfvtifSF57c66vMycdDJPA7H9nkkMNz5Kx+mOBPLNqhLDysGf4 X-Received: by 2002:a17:902:aa88:: with SMTP id d8mr3636051plr.274.1561631286347; Thu, 27 Jun 2019 03:28:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631286; cv=none; d=google.com; s=arc-20160816; b=Vp7rRayuB+C5KpuFRfIwEE0ZC5duZT1IJoxaEBWSnws2ntgZnnQ/XQmW7XdS/Wu5gv OuXKJJqyekBGz8VziEhf8KY5K7Ozrl361f27Wu6YgnE7PxG/gicG5Uvl8oesnjgG7gRV /KZdXAux4M0JaC0I6/n7cJzjqEuZwtrcxe6pYy4fp/nGlwtA0MuDsKlr1YBAJjLoRDfz o8M2M3rDUC4sExRCuafrIVCx/PSoteUVMu8StcibHlpBXIWpk7I8YKWpib4P3wuDl5q7 6QvLW6CSB4ifRlACkEhVIcV9f4ugpyf9TxxMy7OH3YtlofN3y+BEGn3qMlRcNIPAg9QE n7Zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=8rzDm2pxsxDof2+7Zzrqa2tkF2eXq+Yv+cQAP5RFetY=; b=afJ5lWmQqIKvaJn4jgAvPw9AC+zoDR53SlDKD1j8En5KN10A33eHJtVZLu2qkqdlFD 6w5JBsRGyi1zsDGxv8u/HJowASwbw3O0l0YtAlVdiGK8DcmnSAQKV3LPePEx1J3ARVu1 7zsbciBEoP0gOHviJwBgVEn5h6lggHSSqiVGd3bzhVzwHRW6VUUPWyDlIf72GIZlpO9K gXPdVqjrASRItHZdbeITEX7NdaQsLyUm0CoJQv0w5KiR8rk7u+uLRzxFiqKUfkRID4t2 3BKOnJlgayVUX0eqVYPLyW3MGivA+ZMxftCuN8sPTRyHm9xyHMptSpm8SNDrWUH13Chh RLBw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=aadRIWnO; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.05; Thu, 27 Jun 2019 03:28:06 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=aadRIWnO; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726671AbfF0K2F (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:05 -0400 Received: from mail-wr1-f66.google.com ([209.85.221.66]:38295 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726667AbfF0K2F (ORCPT ); Thu, 27 Jun 2019 06:28:05 -0400 Received: by mail-wr1-f66.google.com with SMTP id d18so1922067wrs.5 for ; Thu, 27 Jun 2019 03:28:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8rzDm2pxsxDof2+7Zzrqa2tkF2eXq+Yv+cQAP5RFetY=; b=aadRIWnOfwvrsxYEaKgKoJKubsySZRTcezcL0Y5U6MgwQ8x2L7EKaloZTkFnCEJISb ryZVhyElEmd3GAyC9mbLtSYcGrcZ/ZQ2HT707XvmK90Kavku7AUD3VaceUKZpqGAQZLz uSB7DbCwjFMCStgeUJMc28aDfwi7nOoQD9FxQv5oeXoeZZXjJVgLrdWOkqts38Vfvy8y Wh61goGzeQZ5tvWC/q+Kxga9BknXruGdBwme6VrIEpew6jVhEV/xoYK9Gd6/Zz0Sjv1r geS72GRGS0ffT6nGJd5QB8OTVauND389Lwy+3CAQYseS4+7hOJOGbDVLwJjw4xKsv0Hr 5O1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8rzDm2pxsxDof2+7Zzrqa2tkF2eXq+Yv+cQAP5RFetY=; b=jQM57InEVu9Xxq5Tl9e8BYNvTyjwBgf00bLLnRzTOdVIu0SqxOckCrJ63F8Dpf3uib pJ7FW6YqvcR4spVS2djQ4aao8FhUDST089m3AU7rztMe+zyZz3JPjwXfnYsmBtXlQUmr z809PbYIP61KsyISDz5bNNCpLHAXN7uyRMPEyL8Z29fZv2sEDd8hXfc/KzLcc/KAgSUc i+cdj9Nr1V1RZc9Pxe/t4iTT7K3963KpwYuAPW6M7Hzm9Y+2TXF82vNrH0kuQrFczPZz LEuvJqlHM0XVR4bmxlPE+w8F5TxItuFw5YLz3LwQB048lhE+ohTRa8t2YXi7jmlDTpHJ 0Ppw== X-Gm-Message-State: APjAAAXi6MUbYVXjz0engSpSQ8F3EzVtmhwE5F0z2+eUu1bLtFtbBsYf TB+2NiI8wdsNOzTccZTPWQX42ddhXno= X-Received: by 2002:a5d:400f:: with SMTP id n15mr2736785wrp.312.1561631281703; Thu, 27 Jun 2019 03:28:01 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.00 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:00 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 10/32] crypto: arm64/ghash - switch to AES library Date: Thu, 27 Jun 2019 12:26:25 +0200 Message-Id: <20190627102647.2992-11-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The GHASH code uses the generic AES key expansion routines, and calls directly into the scalar table based AES cipher for arm64 from the fallback path, and since this implementation is known to be non-time invariant, doing so from a time invariant SIMD cipher is a bit nasty. So let's switch to the AES library - this makes the code more robust, and drops the dependency on the generic AES cipher, allowing us to omit it entirely in the future. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/Kconfig | 3 +- arch/arm64/crypto/ghash-ce-glue.c | 30 +++++++------------- 2 files changed, 11 insertions(+), 22 deletions(-) -- 2.20.1 diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig index d9a523ecdd83..1762055e7093 100644 --- a/arch/arm64/crypto/Kconfig +++ b/arch/arm64/crypto/Kconfig @@ -58,8 +58,7 @@ config CRYPTO_GHASH_ARM64_CE depends on KERNEL_MODE_NEON select CRYPTO_HASH select CRYPTO_GF128MUL - select CRYPTO_AES - select CRYPTO_AES_ARM64 + select CRYPTO_LIB_AES config CRYPTO_CRCT10DIF_ARM64_CE tristate "CRCT10DIF digest algorithm using PMULL instructions" diff --git a/arch/arm64/crypto/ghash-ce-glue.c b/arch/arm64/crypto/ghash-ce-glue.c index b39ed99b06fb..90496765d22f 100644 --- a/arch/arm64/crypto/ghash-ce-glue.c +++ b/arch/arm64/crypto/ghash-ce-glue.c @@ -73,8 +73,6 @@ asmlinkage void pmull_gcm_decrypt(int blocks, u64 dg[], u8 dst[], asmlinkage void pmull_gcm_encrypt_block(u8 dst[], u8 const src[], u32 const rk[], int rounds); -asmlinkage void __aes_arm64_encrypt(u32 *rk, u8 *out, const u8 *in, int rounds); - static int ghash_init(struct shash_desc *desc) { struct ghash_desc_ctx *ctx = shash_desc_ctx(desc); @@ -312,14 +310,13 @@ static int gcm_setkey(struct crypto_aead *tfm, const u8 *inkey, u8 key[GHASH_BLOCK_SIZE]; int ret; - ret = crypto_aes_expand_key(&ctx->aes_key, inkey, keylen); + ret = aes_expandkey(&ctx->aes_key, inkey, keylen); if (ret) { tfm->base.crt_flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; return -EINVAL; } - __aes_arm64_encrypt(ctx->aes_key.key_enc, key, (u8[AES_BLOCK_SIZE]){}, - num_rounds(&ctx->aes_key)); + aes_encrypt(&ctx->aes_key, key, (u8[AES_BLOCK_SIZE]){}); return __ghash_setkey(&ctx->ghash_key, key, sizeof(be128)); } @@ -470,7 +467,7 @@ static int gcm_encrypt(struct aead_request *req) rk = ctx->aes_key.key_enc; } while (walk.nbytes >= 2 * AES_BLOCK_SIZE); } else { - __aes_arm64_encrypt(ctx->aes_key.key_enc, tag, iv, nrounds); + aes_encrypt(&ctx->aes_key, tag, iv); put_unaligned_be32(2, iv + GCM_IV_SIZE); while (walk.nbytes >= (2 * AES_BLOCK_SIZE)) { @@ -481,8 +478,7 @@ static int gcm_encrypt(struct aead_request *req) int remaining = blocks; do { - __aes_arm64_encrypt(ctx->aes_key.key_enc, - ks, iv, nrounds); + aes_encrypt(&ctx->aes_key, ks, iv); crypto_xor_cpy(dst, src, ks, AES_BLOCK_SIZE); crypto_inc(iv, AES_BLOCK_SIZE); @@ -498,13 +494,10 @@ static int gcm_encrypt(struct aead_request *req) walk.nbytes % (2 * AES_BLOCK_SIZE)); } if (walk.nbytes) { - __aes_arm64_encrypt(ctx->aes_key.key_enc, ks, iv, - nrounds); + aes_encrypt(&ctx->aes_key, ks, iv); if (walk.nbytes > AES_BLOCK_SIZE) { crypto_inc(iv, AES_BLOCK_SIZE); - __aes_arm64_encrypt(ctx->aes_key.key_enc, - ks + AES_BLOCK_SIZE, iv, - nrounds); + aes_encrypt(&ctx->aes_key, ks + AES_BLOCK_SIZE, iv); } } } @@ -608,7 +601,7 @@ static int gcm_decrypt(struct aead_request *req) rk = ctx->aes_key.key_enc; } while (walk.nbytes >= 2 * AES_BLOCK_SIZE); } else { - __aes_arm64_encrypt(ctx->aes_key.key_enc, tag, iv, nrounds); + aes_encrypt(&ctx->aes_key, tag, iv); put_unaligned_be32(2, iv + GCM_IV_SIZE); while (walk.nbytes >= (2 * AES_BLOCK_SIZE)) { @@ -621,8 +614,7 @@ static int gcm_decrypt(struct aead_request *req) pmull_ghash_update_p64); do { - __aes_arm64_encrypt(ctx->aes_key.key_enc, - buf, iv, nrounds); + aes_encrypt(&ctx->aes_key, buf, iv); crypto_xor_cpy(dst, src, buf, AES_BLOCK_SIZE); crypto_inc(iv, AES_BLOCK_SIZE); @@ -640,11 +632,9 @@ static int gcm_decrypt(struct aead_request *req) memcpy(iv2, iv, AES_BLOCK_SIZE); crypto_inc(iv2, AES_BLOCK_SIZE); - __aes_arm64_encrypt(ctx->aes_key.key_enc, iv2, - iv2, nrounds); + aes_encrypt(&ctx->aes_key, iv2, iv2); } - __aes_arm64_encrypt(ctx->aes_key.key_enc, iv, iv, - nrounds); + aes_encrypt(&ctx->aes_key, iv, iv); } } From patchwork Thu Jun 27 10:26:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167911 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148604ilk; Thu, 27 Jun 2019 03:28:06 -0700 (PDT) X-Google-Smtp-Source: APXvYqxT3GH3+2eJ5pMW2xQrqEya2zyrH68MFJQBg7L9H/jF04DTG9nmcWoSnYYlCmxz1kRpjkf3 X-Received: by 2002:a63:4553:: with SMTP id u19mr3068030pgk.420.1561631286663; Thu, 27 Jun 2019 03:28:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631286; cv=none; d=google.com; s=arc-20160816; b=lygZiUzaDHKq2bLrwPgyPKb67w9EGCPK8STHLmMFeJiXOAW8E4Ubu950/a67DIPm5v OJWt0bDDx8Lr/d5mh3yP6Zi9VpcsQlyobbZgAFUjlXm4b82lGk0kmoh6nldsCK8iICDz uyPYjNHhqOhukr1ebdv54fXc9klx9l7SIYctn6H7mmf77YEdidT8XwLYshCswu677420 WvU0Zd35mW2iCO0BDui/3LR84svTtWO9sGIojC2rBd4pg3jTThkp16bgzTczoybjFR6E as6wfTAY6p4mUf0lY8O9n000mmm5zQv/ME+05PiyCJ1teWihJ+5WUmFQ4gPY1HaWwb+j PRgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QXrlKaueW49CmOlW9q/RYTZvkffZ+6o0SuGgZq9akzA=; b=oTqoPgJImPfai2uBt+qSbzeKm6oHHaoRpdKWaQvAyKIsQAas5Y1Av82wPsyVCLg/bO TQDfKSNWTo54FK45vABAaOH5adQ0JZ3QWGucfB3wmGI+ZQppyRPzXjhz6S0WKK7LJMjA cMwpT5BT+sJMTOJ2CvAVTyoPo9wGqzeMtn+BcfbrBkQ/PiBRASwQJNwmN0JxLR4KWbZy EH2v4hxMVFB8eLZknMtEmZDeZfDwpas9fLmo6Zw5sSDJOKkbGh9Sjw2R4D9mL8zvVD4P 3JZrcrz0xDpdEMHoXkm35ypfejNBY8T+5XOBhKYtpJTGrYGLEgnfgH/byaChJQEbjVoX e6qw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rt2zKLIe; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.06; Thu, 27 Jun 2019 03:28:06 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rt2zKLIe; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726667AbfF0K2F (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:05 -0400 Received: from mail-wm1-f66.google.com ([209.85.128.66]:53447 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726659AbfF0K2F (ORCPT ); Thu, 27 Jun 2019 06:28:05 -0400 Received: by mail-wm1-f66.google.com with SMTP id x15so5188438wmj.3 for ; Thu, 27 Jun 2019 03:28:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QXrlKaueW49CmOlW9q/RYTZvkffZ+6o0SuGgZq9akzA=; b=rt2zKLIeJAWLh19X7pdXenP7Gy8ReHps31E4Hs4n1DWNIWrdM6ikJIrIhpu1fnqtST WIRomgcPzc8oMvVNT8QlR8oFTABtuwK07c2ccUgdb7CRV2hwdu7mvbKaiZwkDZO4oMyC Kti+5s1ewVvu9Hd1ACstAa5XllS7M3nI+ACteInFugod+Fzh7ffdUuFVMIZC4C/lcOe6 MKGW/+sFGzzoKq3CSCnzfxQ/xDRsb8MwcevXYMiaYSNKqrHALo8AxzxsSsRBxnpZSIjS QiyWogHs8xCFLAddJnXjFlUxx6/Ht2wy0L8ZGDKBQUl9BScVG7Smp8kblsPZfxlQg7iy le8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QXrlKaueW49CmOlW9q/RYTZvkffZ+6o0SuGgZq9akzA=; b=tzipoxJrrDrjWitgup/F1+udenxb8Yd1iNQ2PuOl9fpvpuBsLqiFKgjqHuBZ1jISch t8hojygHcPudPHM/Y6UOTbsEfAulVkgdjz8M19vwPdgUmxaPVC8S0kiQpt3TLYG6YMM6 0gXypECLzxyM7lj+hKtP0khd91EpO+UpNoqJDy8dRqzDkwXQhxcKZebO+SYyDIVaqEiA vG3E0r5wacMOiEkL3ajb51I0IfqBPITS0jg8LX3FSPHI7xlGcKLKoLnDx6daf6Yug2LJ FV32v/ciTJitEjZ5wPamuZa15Q8ZUSd11e80uyNlh2q9m0w08mXXo/CcYiEmEaZZWSr8 Cocw== X-Gm-Message-State: APjAAAU4wzMyoanPyz2D2az9F1yugK2oKWhhO1Wyn2S3W6T1nbXxpUnu X5bVJ0BwPrU7vdh+pZKcfSAXDOPF9Cc= X-Received: by 2002:a05:600c:c4:: with SMTP id u4mr2783801wmm.96.1561631282832; Thu, 27 Jun 2019 03:28:02 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.01 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:02 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 11/32] crypto: arm/aes-neonbs - switch to library version of key expansion routine Date: Thu, 27 Jun 2019 12:26:26 +0200 Message-Id: <20190627102647.2992-12-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Switch to the new AES library that also provides an implementation of the AES key expansion routine. This removes the dependency on the generic AES cipher, allowing it to be omitted entirely in the future. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/Kconfig | 2 +- arch/arm/crypto/aes-neonbs-glue.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) -- 2.20.1 diff --git a/arch/arm/crypto/Kconfig b/arch/arm/crypto/Kconfig index a95322b59799..b24df84a1d7a 100644 --- a/arch/arm/crypto/Kconfig +++ b/arch/arm/crypto/Kconfig @@ -82,8 +82,8 @@ config CRYPTO_AES_ARM_BS tristate "Bit sliced AES using NEON instructions" depends on KERNEL_MODE_NEON select CRYPTO_BLKCIPHER + select CRYPTO_LIB_AES select CRYPTO_SIMD - select CRYPTO_AES help Use a faster and more secure NEON based implementation of AES in CBC, CTR and XTS modes diff --git a/arch/arm/crypto/aes-neonbs-glue.c b/arch/arm/crypto/aes-neonbs-glue.c index 617c2c99ebfb..f43c9365b6a9 100644 --- a/arch/arm/crypto/aes-neonbs-glue.c +++ b/arch/arm/crypto/aes-neonbs-glue.c @@ -64,7 +64,7 @@ static int aesbs_setkey(struct crypto_skcipher *tfm, const u8 *in_key, struct crypto_aes_ctx rk; int err; - err = crypto_aes_expand_key(&rk, in_key, key_len); + err = aes_expandkey(&rk, in_key, key_len); if (err) return err; @@ -123,7 +123,7 @@ static int aesbs_cbc_setkey(struct crypto_skcipher *tfm, const u8 *in_key, struct crypto_aes_ctx rk; int err; - err = crypto_aes_expand_key(&rk, in_key, key_len); + err = aes_expandkey(&rk, in_key, key_len); if (err) return err; From patchwork Thu Jun 27 10:26:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167908 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148618ilk; Thu, 27 Jun 2019 03:28:07 -0700 (PDT) X-Google-Smtp-Source: APXvYqxTjn4xsw+IGFmRCMfJt1Vwtx+C8StHmGFi/0erz0vuXzK257m7zVL5WwNG0H81Ba7L8tbO X-Received: by 2002:a17:90a:5806:: with SMTP id h6mr5042667pji.126.1561631287290; Thu, 27 Jun 2019 03:28:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631287; cv=none; d=google.com; s=arc-20160816; b=t0turvgvuTgi2zX1i6ybAv++3IfGnrPBpW7nwzPGoidVoWZ11dS8VubA155REy6EC4 JbYhE3FN7Pj/sUNcbCcja+b7G4aiLwDG4AgpecH0ltYyqAM9P27Htu+gD+27Dha3f2hS ps6ybs/64JY6URvMqR4KKxn4GwMIQJiNoK2DQuaX0nmrQwaATWl+hP8Z34sFGxgGhUMS QMBRkhAyRTqF1n2SVgpXFlt792euWCQXb8Al9Ifn6sw2bcnEmzfi0d7QFpt+QDsj92VY 3Y/brfkiiuvEe6RmcTXx/sDT2feohPpgQf+Gu4ktYBD5YLSkV1n1cqhd+fX2aOhy3C7L zBKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Pgzq94PpKRgLR2IqqS+QLXIksHPk3/jPtTuBsqGb9fM=; b=MC0FP//x3VYmJO1PUIed0RUI4NJM0stYY+KRVv+BMcaFTPja9Hm+TH+S6Zl7A2Mg/m tXUYYdqp1BUOhGEXC5qpo4br8w8fAd9L8JAxtvRT7d70YVJWeBCr4jRfeyIdkJWgNyrE KgpnmgUyCDshTSB1HdXE4RglcqNJfz9R+NuE9aLdabRjbZo+lTqUmd0e3EWzuP+QbA5w jw6pJWiAo67z+Uo5JS0Do52xvWmZeQ/olnXI7ZGSmRcniSZjI639W9xoGp8dO7dBch4j V0NdykPpZCl1jCrC3I+ZKxdTCGN6tqhzHkNRQW9j2V7rD5/vaWjzTxUadX+14nWLDYD0 hCWw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=zqKRnQks; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.07; Thu, 27 Jun 2019 03:28:07 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=zqKRnQks; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726672AbfF0K2G (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:06 -0400 Received: from mail-wr1-f67.google.com ([209.85.221.67]:36036 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726669AbfF0K2G (ORCPT ); Thu, 27 Jun 2019 06:28:06 -0400 Received: by mail-wr1-f67.google.com with SMTP id n4so1934403wrs.3 for ; Thu, 27 Jun 2019 03:28:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Pgzq94PpKRgLR2IqqS+QLXIksHPk3/jPtTuBsqGb9fM=; b=zqKRnQks+zzywDk154HrQyQHGVKltKxj2se24VndSlzuLaEsncsM1YqSEitfnJBLw5 D+TPzsz/Ykp9Mu3ZCbSlgoiMe+NM3QRWGuuNmtf+Iwd+gLmNYbgksZQ9efJXh4WOiOis U8JBYx2/h2kRo/SaQ3fw5gUaVFVBv8YJUGuAiy6FHYghrlr3IbW130TwZZUizgEbmnS1 F7khnmPmbA73vzK27R+FJ+GtlZZk2D8X3uql7A2jC2Hei21vpducb9f4jnp5YLaGLce6 8wEj+7G1fT9wEcSip3wCEIHHIjGzvKNG24P1RYy6eV25ZHVM1ifJAcqUVMdDpwdCUoW9 VEQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Pgzq94PpKRgLR2IqqS+QLXIksHPk3/jPtTuBsqGb9fM=; b=Z9VIs4mvfgOhuSyZ+5d263IGXwQHB2MKGrDFNaHx1mQvH4Y+C0vyfGllRs3Tss9xf6 ICwFN00N0xkVRK3ZoXy59e8X+latUECayRHxYeGaPDJXlJwVdGB+L/LQWyhsY5F4eLik ZJfDWY+I8vUg4D/yLthoOluGuA5hZJ4k+v7VtSDhmhQASDEsBzJx9ZLj2HQyYe72tM+9 7x60BmVcPXtksnG9buhu5vTN9E7Ze67oRdAiK7jw/ESxL9+IxLhuCjWzfwE7YYGIKEuq 11W5xJ5Jh7jvE0ggp/Gk/pG19siNyzus1mQXrgOqJQmZ3JfW1++lnLBHZvPLiBK9grR0 2qwQ== X-Gm-Message-State: APjAAAU/VeFHDjKEQ1xKfgJKrfGZqVBQxKA3gdG2Ilzw71lh+EubQAXN cwLVVvpI4TbIQBLBQoA3qvxvirV+Qj4= X-Received: by 2002:adf:e806:: with SMTP id o6mr483716wrm.269.1561631283771; Thu, 27 Jun 2019 03:28:03 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.02 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:03 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 12/32] crypto: arm64/aes-ccm - switch to AES library Date: Thu, 27 Jun 2019 12:26:27 +0200 Message-Id: <20190627102647.2992-13-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The CCM code calls directly into the scalar table based AES cipher for arm64 from the fallback path, and since this implementation is known to be non-time invariant, doing so from a time invariant SIMD cipher is a bit nasty. So let's switch to the AES library - this makes the code more robust, and drops the dependency on the generic AES cipher, allowing us to omit it entirely in the future. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/Kconfig | 2 +- arch/arm64/crypto/aes-ce-ccm-glue.c | 18 ++++++------------ 2 files changed, 7 insertions(+), 13 deletions(-) -- 2.20.1 diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig index 1762055e7093..c6032bfb44fb 100644 --- a/arch/arm64/crypto/Kconfig +++ b/arch/arm64/crypto/Kconfig @@ -80,8 +80,8 @@ config CRYPTO_AES_ARM64_CE_CCM depends on ARM64 && KERNEL_MODE_NEON select CRYPTO_ALGAPI select CRYPTO_AES_ARM64_CE - select CRYPTO_AES_ARM64 select CRYPTO_AEAD + select CRYPTO_LIB_AES config CRYPTO_AES_ARM64_CE_BLK tristate "AES in ECB/CBC/CTR/XTS modes using ARMv8 Crypto Extensions" diff --git a/arch/arm64/crypto/aes-ce-ccm-glue.c b/arch/arm64/crypto/aes-ce-ccm-glue.c index cb89c80800b5..b9b7cf4b5a8f 100644 --- a/arch/arm64/crypto/aes-ce-ccm-glue.c +++ b/arch/arm64/crypto/aes-ce-ccm-glue.c @@ -46,8 +46,6 @@ asmlinkage void ce_aes_ccm_decrypt(u8 out[], u8 const in[], u32 cbytes, asmlinkage void ce_aes_ccm_final(u8 mac[], u8 const ctr[], u32 const rk[], u32 rounds); -asmlinkage void __aes_arm64_encrypt(u32 *rk, u8 *out, const u8 *in, int rounds); - static int ccm_setkey(struct crypto_aead *tfm, const u8 *in_key, unsigned int key_len) { @@ -127,8 +125,7 @@ static void ccm_update_mac(struct crypto_aes_ctx *key, u8 mac[], u8 const in[], } while (abytes >= AES_BLOCK_SIZE) { - __aes_arm64_encrypt(key->key_enc, mac, mac, - num_rounds(key)); + aes_encrypt(key, mac, mac); crypto_xor(mac, in, AES_BLOCK_SIZE); in += AES_BLOCK_SIZE; @@ -136,8 +133,7 @@ static void ccm_update_mac(struct crypto_aes_ctx *key, u8 mac[], u8 const in[], } if (abytes > 0) { - __aes_arm64_encrypt(key->key_enc, mac, mac, - num_rounds(key)); + aes_encrypt(key, mac, mac); crypto_xor(mac, in, abytes); *macp = abytes; } @@ -209,10 +205,8 @@ static int ccm_crypt_fallback(struct skcipher_walk *walk, u8 mac[], u8 iv0[], bsize = nbytes; crypto_inc(walk->iv, AES_BLOCK_SIZE); - __aes_arm64_encrypt(ctx->key_enc, buf, walk->iv, - num_rounds(ctx)); - __aes_arm64_encrypt(ctx->key_enc, mac, mac, - num_rounds(ctx)); + aes_encrypt(ctx, buf, walk->iv); + aes_encrypt(ctx, mac, mac); if (enc) crypto_xor(mac, src, bsize); crypto_xor_cpy(dst, src, buf, bsize); @@ -227,8 +221,8 @@ static int ccm_crypt_fallback(struct skcipher_walk *walk, u8 mac[], u8 iv0[], } if (!err) { - __aes_arm64_encrypt(ctx->key_enc, buf, iv0, num_rounds(ctx)); - __aes_arm64_encrypt(ctx->key_enc, mac, mac, num_rounds(ctx)); + aes_encrypt(ctx, buf, iv0); + aes_encrypt(ctx, mac, mac); crypto_xor(mac, buf, AES_BLOCK_SIZE); } return err; From patchwork Thu Jun 27 10:26:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167909 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148643ilk; Thu, 27 Jun 2019 03:28:08 -0700 (PDT) X-Google-Smtp-Source: APXvYqzAsl/rFF/NIbL6fYgKtteU7HA/+nsygxtxyyb9a9QOBIYcNUYjS8hIloGfS4tYc/4ZFc7A X-Received: by 2002:a17:902:788e:: with SMTP id q14mr3826027pll.234.1561631288100; Thu, 27 Jun 2019 03:28:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631288; cv=none; d=google.com; s=arc-20160816; b=klJroGN6Mw8Ge0C1iytkgR/UyQBius26W/h/yoElJlfS3bGsrsg8MWvWUB0ZGjr7f6 ZEqkD2xIIbFye7p0aNCnN3QQyhGDuwX2SWG/7YMHpaE34Eeg/uPKVwYR2/HESqOKNAWo XYCtUU3pzrYKj5H6cCzmuYsAKAF+kNyhpcUp9GfCSpkHqZqAGgK1BNjVNN/VW2Cx7b6R wIJKmg/ZtLEs2ZyfaaYg860Yh9QFcxxlAUWKugonVS9AbzUIKPat8+GK3/pN7Al3Y8cT keIN8J+sTKW98oSjN3jPu9ugozFRIZbOX47XF273/IN6Z34ht1LSiuXZkeMdHG4C9iLE VAZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0+abpCskFH64uUUzWz6MrJWQeWRPxd08F5mJmqOJSaY=; b=xG7pocw7VToK+jjHRG1wvErHNVjQe+RqEsjP21XgV/xZdkXKyImwBCktdD8sqEM2+M wDEVujzO8UI4jD60dQFnacJGn4/Yy/GGAGbC/fSzN0n9JC9PODpmHvirho4hJWLxVUBm xK1mL5zg+0kCNvxGv96VDGjUJNo2MlGo8EX9MUeOEppodpBl/yX3d8di7rTeIY5nwb6v N9h6FGdIsYMG4SxpjLpI31dLY4CFX0vWLoT+nvKiEODsiYIbqHyjvNNLSr2OXEOka26v O4qhmPFPgvGAv/prfkXd5t5gQdPNMdtsj3QdPr7bAICJ18TAMioJ/ZQIkHF36icnt5Df UL7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=viSwP1Jp; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.07; Thu, 27 Jun 2019 03:28:08 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=viSwP1Jp; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726673AbfF0K2H (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:07 -0400 Received: from mail-wm1-f66.google.com ([209.85.128.66]:54556 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726659AbfF0K2H (ORCPT ); Thu, 27 Jun 2019 06:28:07 -0400 Received: by mail-wm1-f66.google.com with SMTP id g135so5180115wme.4 for ; Thu, 27 Jun 2019 03:28:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0+abpCskFH64uUUzWz6MrJWQeWRPxd08F5mJmqOJSaY=; b=viSwP1JpAfPU32uHnw8rlxaTAk71HLe27U7d0CvdMB42udDUtWCIH+YutMCEZueel6 qWQNzI5bwZj2h5AdUeyLgE6RQvg6+3oHq+QvzNbz8lguMVEPy4MO7tecZKbGEwag5OKT HiHjZg0zSAJ7lYKY2AyvRbrKSnKb7WL02IF4lxRgrLkXxgv1Vp2K4z9hEAtt2MbsHJLA DjDbbqYpw451jWHd8pVCtKBhwTQ37ggNSnWbaNg4vmbabHBGLqBoDgiDtPNrgNo9FPD6 89vKsVVIyTWtKv/s/FUTy0VidzNUlLL9vIPb5xbFR9xkCaiIWwlj/Zho+6O/PqiiGmBQ uvxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0+abpCskFH64uUUzWz6MrJWQeWRPxd08F5mJmqOJSaY=; b=OGaMHhzNgF8SnXRIjHJOoRgzAoVSVO1jyYgCDIjQu2HVN+oejkIQs43yf8hNha9dP0 SD485wembMlw49RlGD8hll+eesjYbFqbSeTCMTMDNSQmWrrw2+5rlkcTDn80w9LhMUUF 8s/TwfseB960gDzw/3DogFcV0roO6aXxlyXAlZYyJNxeA63jYYGJlrzpx66zuP0TIKHc YIlmXi6Mf8ubM/ZiY6MNU5hAW31BpTRPUEqrGdu5PoKXC1Rw1VDwF7C3eXBO4ZNO5eyZ MdQ3oCM9bSLGtu/LxsWutgaS4E9zNCX44hvGmwrkj+ZuWTbPzD0wfcnjgSedV3V77haX s4bA== X-Gm-Message-State: APjAAAXBQ+WgcyhSvB7LH7dZJmJ9f0CpdzSddHhAkL+ltJ9W9EKZWbTi yh2jV2YNghnF/c81y8ggEfcMmD4gF8s= X-Received: by 2002:a1c:f415:: with SMTP id z21mr2891110wma.34.1561631284769; Thu, 27 Jun 2019 03:28:04 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.03 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:04 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 13/32] crypto: arm64/aes-neonbs - switch to library version of key expansion routine Date: Thu, 27 Jun 2019 12:26:28 +0200 Message-Id: <20190627102647.2992-14-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Switch to the new AES library that also provides an implementation of the AES key expansion routine. This removes the dependency on the generic AES cipher, allowing it to be omitted entirely in the future. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/Kconfig | 1 + arch/arm64/crypto/aes-neonbs-glue.c | 8 ++++---- 2 files changed, 5 insertions(+), 4 deletions(-) -- 2.20.1 diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig index c6032bfb44fb..17bf5dc10aad 100644 --- a/arch/arm64/crypto/Kconfig +++ b/arch/arm64/crypto/Kconfig @@ -116,6 +116,7 @@ config CRYPTO_AES_ARM64_BS select CRYPTO_BLKCIPHER select CRYPTO_AES_ARM64_NEON_BLK select CRYPTO_AES_ARM64 + select CRYPTO_LIB_AES select CRYPTO_SIMD endif diff --git a/arch/arm64/crypto/aes-neonbs-glue.c b/arch/arm64/crypto/aes-neonbs-glue.c index 02b65d9eb947..cb8d90f795a0 100644 --- a/arch/arm64/crypto/aes-neonbs-glue.c +++ b/arch/arm64/crypto/aes-neonbs-glue.c @@ -77,7 +77,7 @@ static int aesbs_setkey(struct crypto_skcipher *tfm, const u8 *in_key, struct crypto_aes_ctx rk; int err; - err = crypto_aes_expand_key(&rk, in_key, key_len); + err = aes_expandkey(&rk, in_key, key_len); if (err) return err; @@ -136,7 +136,7 @@ static int aesbs_cbc_setkey(struct crypto_skcipher *tfm, const u8 *in_key, struct crypto_aes_ctx rk; int err; - err = crypto_aes_expand_key(&rk, in_key, key_len); + err = aes_expandkey(&rk, in_key, key_len); if (err) return err; @@ -208,7 +208,7 @@ static int aesbs_ctr_setkey_sync(struct crypto_skcipher *tfm, const u8 *in_key, struct aesbs_ctr_ctx *ctx = crypto_skcipher_ctx(tfm); int err; - err = crypto_aes_expand_key(&ctx->fallback, in_key, key_len); + err = aes_expandkey(&ctx->fallback, in_key, key_len); if (err) return err; @@ -274,7 +274,7 @@ static int aesbs_xts_setkey(struct crypto_skcipher *tfm, const u8 *in_key, return err; key_len /= 2; - err = crypto_aes_expand_key(&rk, in_key + key_len, key_len); + err = aes_expandkey(&rk, in_key + key_len, key_len); if (err) return err; From patchwork Thu Jun 27 10:26:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167910 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148686ilk; Thu, 27 Jun 2019 03:28:10 -0700 (PDT) X-Google-Smtp-Source: APXvYqy5n/E46K1DY5SJS2vW6IgaACiVvIDys7BO2xSKtbFqgZ637UShoKBLYfkqzBqsqJGBk0lp X-Received: by 2002:a17:90a:21ac:: with SMTP id q41mr5217724pjc.31.1561631290159; Thu, 27 Jun 2019 03:28:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631290; cv=none; d=google.com; s=arc-20160816; b=YISXrOeXo4g3UReUanMrhIpK5VgMCHVFIneX5Q6/3SebaA5ZOqDFkqiTG314wILLRn pwQk54UwFu2u4AOiZ3L7NjS2p3uInq3XIgwYDCKoseG6WtHDstf2Bp7X4SK7aEE+DVNM CDTainJuX0opozWWymsBjtWlF0atHv53PtL0iUDSjFmafh8byMLRmrkcIoSWeN+vXGCi Vv1YxssyknVbBELHmxEtJAHhh0rcnbqJITIi9zDaPophAD/CEtLjCv7NOst7DsCwM8BE 94GK6F3GSy61cvLS7tPEy91x8T4driXwsNqezuVYo6sTk3KHePX2iZRJs2xkWMAF/Px5 g0Bg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=MzqBn3zXqdqT5rykfpJWuWfbyyxJXTIyDmCZ4k6FneI=; b=Rl6YNrWOSwDbcnqynUr5oqpZbv//UnSvgxp11Tfc34c2V350kqa3MBxFCyogBA4Kvr ZSSZBp+ujqaxcBAHm0lqplAqekXb/X69HPWUTEbt3J3JmdS/HL8/+bnSUO8/ZOuavyxM dlktoQ53tjLdLtqLTbTp80UuVzARDeDXYS6PA0CmMEVJRzkemWdk7URpK0E6HsKDFVgi 7qF3v+RStInN1i66V8oxVJ/BgwEWKL3yYh0u5h+gnWsiUCwQJDNJ0IXjzI4aooWaaUdT y/8rrgKSnmb8JoLvttEjzBKjv/PjB9aOqNhp2xVBimi3lowEt+BtlTb/AIz5gDim0SGa xTfA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lHyUdSKe; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.09; Thu, 27 Jun 2019 03:28:10 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lHyUdSKe; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726675AbfF0K2J (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:09 -0400 Received: from mail-wm1-f67.google.com ([209.85.128.67]:52168 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726669AbfF0K2H (ORCPT ); Thu, 27 Jun 2019 06:28:07 -0400 Received: by mail-wm1-f67.google.com with SMTP id 207so5186408wma.1 for ; Thu, 27 Jun 2019 03:28:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MzqBn3zXqdqT5rykfpJWuWfbyyxJXTIyDmCZ4k6FneI=; b=lHyUdSKe1Lk4pZMIN7FFIlIwZlX62h6Y1D9LbZs1h2DKVMvn94II8Uq4bnkgX+YZ0v B55cG8ghG/UxT5BEEOodD0MeIoi1USJWDmgUUMqBZ+C52oXiwjlMXncuUtesY2pVgXvn bPLE2EKwzfnOvvuUl75A4NfbEXnCPfsUqUc4hbjznDd4LmIRqQtYAyHOHLfAtPw27JJj 3czkG3oW+9SyzYDAu77W+SIsSJovDedQKYNu+J+d+nw9g8XHrhhAH2+HMCGaIlOeEVcB CefOdjmQvXMViw2+eveNANXaGDLE1QkiTJFR87NCiuW5CKF5N/wFHCJQItcW5o17Fs43 29YQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MzqBn3zXqdqT5rykfpJWuWfbyyxJXTIyDmCZ4k6FneI=; b=O13YD+0maXkRKUfTHB24Qc5JLNTsk4Epmbfce19iwld4rhHH7f5QM04Vi1So090rKo 7FF/dZU0peOoLHInSKeAc9ls+TrzbeH3w3n/GImyVLZXhnBlnXEFVOQ9vMCI+g7y5/+3 XwUM308WUibUmYP4hG2M91111/qYhY1FiBemwr9wBGDTSclei1plinCkuL/X7PdVelDH gXzrlCCh7eqihOrbgkQn7+g7fDZ95KSUGW2M2kNSflO34Q4p93HPJZcv9dNQek6gJ11t 6ZHqgWBrBt7HktE+MviOC4v7Zbp7+BaIRaNj0243OCNtPszKwjunFXPFXAERZk4hs7WO F6vg== X-Gm-Message-State: APjAAAUDD9j7u3vlxtSLOppzSqzO6VUi9BSgZaTvwUj7+P6coE9/Wzja J2s83eRouZeXvUMS/kqqBmDtrSZC0k4= X-Received: by 2002:a1c:1bc1:: with SMTP id b184mr2926031wmb.42.1561631285978; Thu, 27 Jun 2019 03:28:05 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.04 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:05 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 14/32] crypto: arm64/aes-ce - switch to library version of key expansion routine Date: Thu, 27 Jun 2019 12:26:29 +0200 Message-Id: <20190627102647.2992-15-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Switch to the new AES library that also provides an implementation of the AES key expansion routine. This removes the dependency on the generic AES cipher, allowing it to be omitted entirely in the future. While at it, remove some references to the table based arm64 version of AES and replace them with AES library calls as well. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/Kconfig | 2 +- arch/arm64/crypto/aes-glue.c | 17 ++++++++++------- 2 files changed, 11 insertions(+), 8 deletions(-) -- 2.20.1 diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig index 17bf5dc10aad..66dea518221c 100644 --- a/arch/arm64/crypto/Kconfig +++ b/arch/arm64/crypto/Kconfig @@ -96,7 +96,7 @@ config CRYPTO_AES_ARM64_NEON_BLK depends on KERNEL_MODE_NEON select CRYPTO_BLKCIPHER select CRYPTO_AES_ARM64 - select CRYPTO_AES + select CRYPTO_LIB_AES select CRYPTO_SIMD config CRYPTO_CHACHA20_NEON diff --git a/arch/arm64/crypto/aes-glue.c b/arch/arm64/crypto/aes-glue.c index f0ceb545bd1e..3c80345d914f 100644 --- a/arch/arm64/crypto/aes-glue.c +++ b/arch/arm64/crypto/aes-glue.c @@ -26,7 +26,6 @@ #ifdef USE_V8_CRYPTO_EXTENSIONS #define MODE "ce" #define PRIO 300 -#define aes_setkey ce_aes_setkey #define aes_expandkey ce_aes_expandkey #define aes_ecb_encrypt ce_aes_ecb_encrypt #define aes_ecb_decrypt ce_aes_ecb_decrypt @@ -42,8 +41,6 @@ MODULE_DESCRIPTION("AES-ECB/CBC/CTR/XTS using ARMv8 Crypto Extensions"); #else #define MODE "neon" #define PRIO 200 -#define aes_setkey crypto_aes_set_key -#define aes_expandkey crypto_aes_expand_key #define aes_ecb_encrypt neon_aes_ecb_encrypt #define aes_ecb_decrypt neon_aes_ecb_decrypt #define aes_cbc_encrypt neon_aes_cbc_encrypt @@ -121,7 +118,14 @@ struct mac_desc_ctx { static int skcipher_aes_setkey(struct crypto_skcipher *tfm, const u8 *in_key, unsigned int key_len) { - return aes_setkey(crypto_skcipher_tfm(tfm), in_key, key_len); + struct crypto_aes_ctx *ctx = crypto_skcipher_ctx(tfm); + int ret; + + ret = aes_expandkey(ctx, in_key, key_len); + if (ret) + crypto_skcipher_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN); + + return ret; } static int xts_set_key(struct crypto_skcipher *tfm, const u8 *in_key, @@ -649,15 +653,14 @@ static void mac_do_update(struct crypto_aes_ctx *ctx, u8 const in[], int blocks, kernel_neon_end(); } else { if (enc_before) - __aes_arm64_encrypt(ctx->key_enc, dg, dg, rounds); + aes_encrypt(ctx, dg, dg); while (blocks--) { crypto_xor(dg, in, AES_BLOCK_SIZE); in += AES_BLOCK_SIZE; if (blocks || enc_after) - __aes_arm64_encrypt(ctx->key_enc, dg, dg, - rounds); + aes_encrypt(ctx, dg, dg); } } } From patchwork Thu Jun 27 10:26:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167912 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148723ilk; Thu, 27 Jun 2019 03:28:12 -0700 (PDT) X-Google-Smtp-Source: APXvYqx2yC8eJv8doWnJGlsg37HIU3szf2qfl83TEcz7qktd+KlZo0WuheYifw12E+h21RJP7ssu X-Received: by 2002:a17:902:848c:: with SMTP id c12mr3707673plo.17.1561631292235; Thu, 27 Jun 2019 03:28:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631292; cv=none; d=google.com; s=arc-20160816; b=QDtd2eeTRemcLbbmZDnKBIzYGLLtnIv+Zc/Hbz/6tO7xoWP2ZVAahpJgaixjPED+NJ B5JGz9NFqRCEsAQyUMucZLWfdSkrOVLBAw/kDJx7jf0znMAk3wkSLAJdqyl2vRNWLsBP whSpT6mb9a31JQFCDhRvOzSCnZCwJyyn+flidWvarpx4ziSO2gk1jNKSUvpbB+T+nmVd dm8UsIA14/nRNxqH/aVY2qePrZy0gxUP5XLkyjrvYMyCRl2YTYs+1ca27zsy3lp7STu3 PRYipHYy80nGSgrftrrz7jNBXKJnlacsLZaprWNs0f1gjEwqN3tsdSJmGEttglXqSfuI ibhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=m8xE/gQHhxeiNIomdxtjEbSnrgED29gmEg1crTL/DPU=; b=vbYdnx6QZdiF+D0E5jXQDQFAVvYrbOKfANucY7cjBK3LGf9IYJFQy7O4ZIBdt6kWwz NjXYqkgxZlYrsFfs/9kBgcvMqnZQ2CC4iWvuKDpup3TS/WGzUTUYcSkngIchuSYHy0Ph dmGqoKXt8QZ8vhb6SkIzw+vGIoB6NAU2qLA4W4Yyn2npYdShyI0k4D8wmT0E3MLOWYLo CC5rskvJVIEoYH7GjN0nq1DRArPZsfPV+UWQE5wXr3fOLDMpkf8HVCT6Iac6VvVRzCbj 6LUxAhgM7r+CtpTimwQkJk/JxrkjJPufGPFPZ6q/rNeDqTGjixGVfWjI8jjI6+OrZXnj avrg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OejxQjIB; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.12; Thu, 27 Jun 2019 03:28:12 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OejxQjIB; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726679AbfF0K2L (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:11 -0400 Received: from mail-wm1-f67.google.com ([209.85.128.67]:37182 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726659AbfF0K2J (ORCPT ); Thu, 27 Jun 2019 06:28:09 -0400 Received: by mail-wm1-f67.google.com with SMTP id f17so5110682wme.2 for ; Thu, 27 Jun 2019 03:28:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=m8xE/gQHhxeiNIomdxtjEbSnrgED29gmEg1crTL/DPU=; b=OejxQjIB89BU767t4fgihO4UHbWl0rBGHTZQd3ZnLY881ZcQJ0ntfukXV4fHgYcCuy ndpnYg/d0svy9LXIllcwqjm+6s7pgrFo43EZ0HyL9AQZ+snX9awFlASOiS57xNl0w8wF 0OSdh8WvgqrNF/68j4LZQk8OI/rTKJ2DBvxdT5OPYSleBls80ueI0kDI0Ttw7VB0H7BA 64r73PyWeq3YCGCYatu9gz7WgmVxXTLCkRlbRVSZJyjPAG17CJkbZITq1+qm6pEp+eDX qonMTEDOjWPEqFixD/kkk4FWL9dLAgIu7Wefb4nFehxGSB08DzBV38oiJlqCIdo1lm/G S4BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=m8xE/gQHhxeiNIomdxtjEbSnrgED29gmEg1crTL/DPU=; b=qDTsTLGcVf4K53BIj1a/fk56Cz4RN6J75tn+NfBaGiWeT/uaaMkT/7RTTCJEG4Zyjd 4P/U93tvswy7wzgY35SY7m0iaEw7/v+RQg1D0mwWpLIbDEMvKrohyaYuMrK/g93Qec7e egMvvt6aaImrK2S5PwYGtpZApLMUyTZDJImETjgnUkqdCVtjtXKDW7uoM1zSXqhI2+eY jsC/p0hxuNkKVMpHt1YDshb19VD+Hx+7igH1hG9G5hdxhMWiNWzEeWryuI/ofr8glbGG xlPrdjjSmimzs0JSDvIrxjRZIFUU2kzMRHsOnGrJjEh+bi2BRvG0N+Huf+iPa2fwoNRb f8pg== X-Gm-Message-State: APjAAAUExuM2K4j2vvOv3idN817GtIzCP0spWv+gl1celM8LHLIaa0Lx ZvCI1406JPO5hGor1HV1+tuPQdx1Mts= X-Received: by 2002:a1c:2dd2:: with SMTP id t201mr2804977wmt.109.1561631286922; Thu, 27 Jun 2019 03:28:06 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.06 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:06 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 15/32] crypto: generic/aes - drop key expansion routine in favor of library version Date: Thu, 27 Jun 2019 12:26:30 +0200 Message-Id: <20190627102647.2992-16-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Drop aes-generic's version of crypto_aes_expand_key(), and switch to the key expansion routine provided by the AES library. AES key expansion is not performance critical, and it is better to have a single version shared by all AES implementations. Signed-off-by: Ard Biesheuvel --- crypto/Kconfig | 1 + crypto/aes_generic.c | 153 +------------------- include/crypto/aes.h | 2 - 3 files changed, 3 insertions(+), 153 deletions(-) -- 2.20.1 diff --git a/crypto/Kconfig b/crypto/Kconfig index df6f0be66574..80ea118600ab 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -1072,6 +1072,7 @@ config CRYPTO_LIB_AES config CRYPTO_AES tristate "AES cipher algorithms" select CRYPTO_ALGAPI + select CRYPTO_LIB_AES help AES cipher algorithms (FIPS-197). AES uses the Rijndael algorithm. diff --git a/crypto/aes_generic.c b/crypto/aes_generic.c index 3aa4a715c216..426deb437f19 100644 --- a/crypto/aes_generic.c +++ b/crypto/aes_generic.c @@ -1125,155 +1125,6 @@ EXPORT_SYMBOL_GPL(crypto_fl_tab); EXPORT_SYMBOL_GPL(crypto_it_tab); EXPORT_SYMBOL_GPL(crypto_il_tab); -/* initialise the key schedule from the user supplied key */ - -#define star_x(x) (((x) & 0x7f7f7f7f) << 1) ^ ((((x) & 0x80808080) >> 7) * 0x1b) - -#define imix_col(y, x) do { \ - u = star_x(x); \ - v = star_x(u); \ - w = star_x(v); \ - t = w ^ (x); \ - (y) = u ^ v ^ w; \ - (y) ^= ror32(u ^ t, 8) ^ \ - ror32(v ^ t, 16) ^ \ - ror32(t, 24); \ -} while (0) - -#define ls_box(x) \ - crypto_fl_tab[0][byte(x, 0)] ^ \ - crypto_fl_tab[1][byte(x, 1)] ^ \ - crypto_fl_tab[2][byte(x, 2)] ^ \ - crypto_fl_tab[3][byte(x, 3)] - -#define loop4(i) do { \ - t = ror32(t, 8); \ - t = ls_box(t) ^ rco_tab[i]; \ - t ^= ctx->key_enc[4 * i]; \ - ctx->key_enc[4 * i + 4] = t; \ - t ^= ctx->key_enc[4 * i + 1]; \ - ctx->key_enc[4 * i + 5] = t; \ - t ^= ctx->key_enc[4 * i + 2]; \ - ctx->key_enc[4 * i + 6] = t; \ - t ^= ctx->key_enc[4 * i + 3]; \ - ctx->key_enc[4 * i + 7] = t; \ -} while (0) - -#define loop6(i) do { \ - t = ror32(t, 8); \ - t = ls_box(t) ^ rco_tab[i]; \ - t ^= ctx->key_enc[6 * i]; \ - ctx->key_enc[6 * i + 6] = t; \ - t ^= ctx->key_enc[6 * i + 1]; \ - ctx->key_enc[6 * i + 7] = t; \ - t ^= ctx->key_enc[6 * i + 2]; \ - ctx->key_enc[6 * i + 8] = t; \ - t ^= ctx->key_enc[6 * i + 3]; \ - ctx->key_enc[6 * i + 9] = t; \ - t ^= ctx->key_enc[6 * i + 4]; \ - ctx->key_enc[6 * i + 10] = t; \ - t ^= ctx->key_enc[6 * i + 5]; \ - ctx->key_enc[6 * i + 11] = t; \ -} while (0) - -#define loop8tophalf(i) do { \ - t = ror32(t, 8); \ - t = ls_box(t) ^ rco_tab[i]; \ - t ^= ctx->key_enc[8 * i]; \ - ctx->key_enc[8 * i + 8] = t; \ - t ^= ctx->key_enc[8 * i + 1]; \ - ctx->key_enc[8 * i + 9] = t; \ - t ^= ctx->key_enc[8 * i + 2]; \ - ctx->key_enc[8 * i + 10] = t; \ - t ^= ctx->key_enc[8 * i + 3]; \ - ctx->key_enc[8 * i + 11] = t; \ -} while (0) - -#define loop8(i) do { \ - loop8tophalf(i); \ - t = ctx->key_enc[8 * i + 4] ^ ls_box(t); \ - ctx->key_enc[8 * i + 12] = t; \ - t ^= ctx->key_enc[8 * i + 5]; \ - ctx->key_enc[8 * i + 13] = t; \ - t ^= ctx->key_enc[8 * i + 6]; \ - ctx->key_enc[8 * i + 14] = t; \ - t ^= ctx->key_enc[8 * i + 7]; \ - ctx->key_enc[8 * i + 15] = t; \ -} while (0) - -/** - * crypto_aes_expand_key - Expands the AES key as described in FIPS-197 - * @ctx: The location where the computed key will be stored. - * @in_key: The supplied key. - * @key_len: The length of the supplied key. - * - * Returns 0 on success. The function fails only if an invalid key size (or - * pointer) is supplied. - * The expanded key size is 240 bytes (max of 14 rounds with a unique 16 bytes - * key schedule plus a 16 bytes key which is used before the first round). - * The decryption key is prepared for the "Equivalent Inverse Cipher" as - * described in FIPS-197. The first slot (16 bytes) of each key (enc or dec) is - * for the initial combination, the second slot for the first round and so on. - */ -int crypto_aes_expand_key(struct crypto_aes_ctx *ctx, const u8 *in_key, - unsigned int key_len) -{ - u32 i, t, u, v, w, j; - - if (key_len != AES_KEYSIZE_128 && key_len != AES_KEYSIZE_192 && - key_len != AES_KEYSIZE_256) - return -EINVAL; - - ctx->key_length = key_len; - - ctx->key_enc[0] = get_unaligned_le32(in_key); - ctx->key_enc[1] = get_unaligned_le32(in_key + 4); - ctx->key_enc[2] = get_unaligned_le32(in_key + 8); - ctx->key_enc[3] = get_unaligned_le32(in_key + 12); - - ctx->key_dec[key_len + 24] = ctx->key_enc[0]; - ctx->key_dec[key_len + 25] = ctx->key_enc[1]; - ctx->key_dec[key_len + 26] = ctx->key_enc[2]; - ctx->key_dec[key_len + 27] = ctx->key_enc[3]; - - switch (key_len) { - case AES_KEYSIZE_128: - t = ctx->key_enc[3]; - for (i = 0; i < 10; ++i) - loop4(i); - break; - - case AES_KEYSIZE_192: - ctx->key_enc[4] = get_unaligned_le32(in_key + 16); - t = ctx->key_enc[5] = get_unaligned_le32(in_key + 20); - for (i = 0; i < 8; ++i) - loop6(i); - break; - - case AES_KEYSIZE_256: - ctx->key_enc[4] = get_unaligned_le32(in_key + 16); - ctx->key_enc[5] = get_unaligned_le32(in_key + 20); - ctx->key_enc[6] = get_unaligned_le32(in_key + 24); - t = ctx->key_enc[7] = get_unaligned_le32(in_key + 28); - for (i = 0; i < 6; ++i) - loop8(i); - loop8tophalf(i); - break; - } - - ctx->key_dec[0] = ctx->key_enc[key_len + 24]; - ctx->key_dec[1] = ctx->key_enc[key_len + 25]; - ctx->key_dec[2] = ctx->key_enc[key_len + 26]; - ctx->key_dec[3] = ctx->key_enc[key_len + 27]; - - for (i = 4; i < key_len + 24; ++i) { - j = key_len + 24 - (i & ~3) + (i & 3); - imix_col(ctx->key_dec[j], ctx->key_enc[i]); - } - return 0; -} -EXPORT_SYMBOL_GPL(crypto_aes_expand_key); - /** * crypto_aes_set_key - Set the AES key. * @tfm: The %crypto_tfm that is used in the context. @@ -1281,7 +1132,7 @@ EXPORT_SYMBOL_GPL(crypto_aes_expand_key); * @key_len: The size of the key. * * Returns 0 on success, on failure the %CRYPTO_TFM_RES_BAD_KEY_LEN flag in tfm - * is set. The function uses crypto_aes_expand_key() to expand the key. + * is set. The function uses aes_expand_key() to expand the key. * &crypto_aes_ctx _must_ be the private data embedded in @tfm which is * retrieved with crypto_tfm_ctx(). */ @@ -1292,7 +1143,7 @@ int crypto_aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, u32 *flags = &tfm->crt_flags; int ret; - ret = crypto_aes_expand_key(ctx, in_key, key_len); + ret = aes_expandkey(ctx, in_key, key_len); if (!ret) return 0; diff --git a/include/crypto/aes.h b/include/crypto/aes.h index d0067fca0cd0..0a64a977f9b3 100644 --- a/include/crypto/aes.h +++ b/include/crypto/aes.h @@ -35,8 +35,6 @@ extern const u32 crypto_il_tab[4][256] ____cacheline_aligned; int crypto_aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, unsigned int key_len); -int crypto_aes_expand_key(struct crypto_aes_ctx *ctx, const u8 *in_key, - unsigned int key_len); /** * aes_expandkey - Expands the AES key as described in FIPS-197 From patchwork Thu Jun 27 10:26:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167914 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148730ilk; Thu, 27 Jun 2019 03:28:12 -0700 (PDT) X-Google-Smtp-Source: APXvYqzaozLubP841x0tb7t9i7vBs5dY1lOFM4rsY+I8XwRLqRiHV5hb97RUwCLwd4Qt+od1OQ40 X-Received: by 2002:a17:902:788e:: with SMTP id q14mr3826418pll.234.1561631292519; Thu, 27 Jun 2019 03:28:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631292; cv=none; d=google.com; s=arc-20160816; b=Vtube7U0u/6Liot3V4+etfv/nu743HDcrc1dvJwmMR4i3XpjBAZXhmPzLNcg7SU9EA jimNb07glOyaKB8HXkcfDfTUBkvsK7YTQNlFIofoiZLR/xIp3YRoa0O9SKaUvtXhPUcD eeRbGJ5l2eK1ESKGZ/gC4veUm6GIHaddeFQDuTb939iAHnq1Wjp0DTgbZsyzjXVMKb7U xQ/z4hAH6VCvMjr6vDfAMbDFN6lMBfg5YLh8RjSpyiuZ069/25ILZtO8Ii6Pf7NcFGK+ UzF1+fnZyxrYNL8EutkyEdy6VCypHcQHPaUqFiCxrOQfWzHzB+mac1jx7U4zZ1ItboOM UvYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=u6Q45pPiPnuFNJpKPysKttSzbQMmfbBttYB0PrnTC+o=; b=mXeC++GBHUGOBYbVLs/kCRZSzKkZQqxU+g6Xfxt4ays2472GSyEAkUuXC+2FmTlcnN 2jTglCC0lAVyMGpOwoqO7uPDRwkPCjeZJbPrPo2kGr7qNQBBKpMAUMGzXkQUqpIm/4Ou UU5O+H1sdgxXi9yZjAwDX7QxHKU2CHDyp5OHLb0PSVkGGYT+zCtpnAzh7EF3toNY9CPy pSlZXXJIkRuIqBCQDGS+xQB1MTYuWjeLjyh60pXVn5kAxeCScXKXApzbC5omjxv9DI/e T6lGRiWoSHFSTdXNaL48oknmDs9I6zHL3f0tOeeKQ0F8xhBQoEuK6Va288PmJ/MkBEcU bKOQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=yMCUEiJ1; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.12; Thu, 27 Jun 2019 03:28:12 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=yMCUEiJ1; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726659AbfF0K2L (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:11 -0400 Received: from mail-wm1-f66.google.com ([209.85.128.66]:40814 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726674AbfF0K2K (ORCPT ); Thu, 27 Jun 2019 06:28:10 -0400 Received: by mail-wm1-f66.google.com with SMTP id v19so5106135wmj.5 for ; Thu, 27 Jun 2019 03:28:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=u6Q45pPiPnuFNJpKPysKttSzbQMmfbBttYB0PrnTC+o=; b=yMCUEiJ1wmfKpip7QF0/jBbHIedZntLzBTadFMegGfUEfCiOukmCkANW6wl5JX5rnZ +EB1PLnwabwL8NvVy1tBiGhk0qlyeLMLpGp/1heEFvlMA+TJoF4d6I0UNYgMS/MBGrKM wKa1B85dJI9lxm2mK/GLdr9pDSRkWFsWYBzNQrUAFKz4Fhv4xOPI4yP5lxPV2VjrIFYa 0YgD6d9oBFnB3MBbEZkDhSqEqQ/lsePZd8SydDG1xLdrE3RIoi7w1jF1/UQM8ec3j9dH KkNiD43dGdQ0pb9wxUK1wixPV9JUrj08x1n6xrHW4Jd3Q0TKTMygGgdyXC2ApFwnRIk5 JDAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=u6Q45pPiPnuFNJpKPysKttSzbQMmfbBttYB0PrnTC+o=; b=FfBzfiJ4UawB6Mf9cY/cUKdlC1uQbrrPrqvXXpw+TbefV7zuBp0uFldMlxzC6sy5Ck 1Qb9pVa+Pdju9KcjQuzEQcIqHixPgWKOiAdIspnDQAGrMdnt4djVsFi8Dp/QGRi/60Rq /qSbDvCxGstGnIthc09lbtfPe34PDlNCIpYInoq6/EBD8yxXIrGtTcyxaq3HWzD/erTr iF+/kR7tV6RVdaSBa66aTo+s8McX5Y5tdBfoCRHhC6OubIDvHOxriMTd+mB5C0Jp98H0 DcnWhmcVowzdGisDpV08KQebIV6qbBClDIe7hbM2n0wIaKgv3oXMyO/GdtxCaSt/MaQs zwfw== X-Gm-Message-State: APjAAAU1aom8ByObHh2xhzCVy5qEOhT22DgJLzg0jzDhqQ1mJ+o+t4Ba TvJXRfZs5GdMl7bMf5q75w1FjE96AgU= X-Received: by 2002:a7b:cc97:: with SMTP id p23mr2833802wma.120.1561631288174; Thu, 27 Jun 2019 03:28:08 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.06 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:07 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 16/32] crypto: ctr - add helper for performing a CTR encryption walk Date: Thu, 27 Jun 2019 12:26:31 +0200 Message-Id: <20190627102647.2992-17-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Add a static inline helper modeled after crypto_cbc_encrypt_walk() that can be reused for SIMD algorithms that need to implement a non-SIMD fallback for performing CTR encryption. Signed-off-by: Ard Biesheuvel --- include/crypto/ctr.h | 50 ++++++++++++++++++++ 1 file changed, 50 insertions(+) -- 2.20.1 diff --git a/include/crypto/ctr.h b/include/crypto/ctr.h index 4180fc080e3b..d64017fae41c 100644 --- a/include/crypto/ctr.h +++ b/include/crypto/ctr.h @@ -13,8 +13,58 @@ #ifndef _CRYPTO_CTR_H #define _CRYPTO_CTR_H +#include +#include +#include +#include + #define CTR_RFC3686_NONCE_SIZE 4 #define CTR_RFC3686_IV_SIZE 8 #define CTR_RFC3686_BLOCK_SIZE 16 +static inline int crypto_ctr_encrypt_walk(struct skcipher_request *req, + void (*fn)(struct crypto_skcipher *, + const u8 *, u8 *)) +{ + struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req); + int blocksize = crypto_skcipher_chunksize(tfm); + u8 buf[MAX_CIPHER_BLOCKSIZE]; + struct skcipher_walk walk; + int err; + + /* avoid integer division due to variable blocksize parameter */ + if (WARN_ON_ONCE(!is_power_of_2(blocksize))) + return -EINVAL; + + err = skcipher_walk_virt(&walk, req, false); + + while (walk.nbytes > 0) { + u8 *dst = walk.dst.virt.addr; + u8 *src = walk.src.virt.addr; + int nbytes = walk.nbytes; + int tail = 0; + + if (nbytes < walk.total) { + tail = walk.nbytes & (blocksize - 1); + nbytes -= tail; + } + + do { + int bsize = min(nbytes, blocksize); + + fn(tfm, walk.iv, buf); + + crypto_xor_cpy(dst, src, buf, bsize); + crypto_inc(walk.iv, blocksize); + + dst += bsize; + src += bsize; + nbytes -= bsize; + } while (nbytes > 0); + + err = skcipher_walk_done(&walk, tail); + } + return err; +} + #endif /* _CRYPTO_CTR_H */ From patchwork Thu Jun 27 10:26:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167913 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148738ilk; Thu, 27 Jun 2019 03:28:13 -0700 (PDT) X-Google-Smtp-Source: APXvYqz/WmdvZ+Q5yua9TEOWfw2TLgKmgKMt0mIcWT9Y1FPRc2KKBGeo3fhZG/O0Ttz05JrMMo8v X-Received: by 2002:a17:90a:b78b:: with SMTP id m11mr5289925pjr.106.1561631293019; Thu, 27 Jun 2019 03:28:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631293; cv=none; d=google.com; s=arc-20160816; b=j5otkgzxmwNiua78jckLLp4uelTuaKjxy8Oxs1Bd27VFrTOU4qPxxUqBXMBbnWntJy 52N8Tyvi3TQ+t/r98kQOLOSh+99fo9C6Oj2VdJfBYAT9hwqVcgc+NjuDoj4wpUcHnZNS AhlYRQocP4jsrfo7LfFwSsS/sfJbPzMhaDAk5HdwrzoGBIUFpD41Wi96DODh9CBdYyO+ pdGUd+HFSjzfetJzNi/oRa1ik33/ZsSc5PclOzjNGL/WatWXjkpqAgZV/eE3kv+TfB6Z HmV2Du9e4Zbp5057Ce1trGASwHn0zUTlPfQZ8OZ9Ijf45zP1w6lH46SklSFUIvaB/FOB XgNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=oSCqTZSbnLGprwAW+e+bdrFNNEkfE8SXrdCsMDCAbr0=; b=pPcZ0nTu8EsV7VMPmqSXwVMvC14EXwV7qoIUqJd2jExibVZFL7FK3QNsoBM1AwFqYL eENXH8AFFXuKkUGjcQdQyYqhD3gOjpXl0vq6xyDtovkX4GI/hstgTwTVLRdDlaVU+xBR ywmeTIQWNkx/6PGZQ4XEuLBA+pWZsfeDJGzmKomxw96kDXj4LhNz8g5nK3f7IrVJoifz v1S5/sOq6H2ar5DsrYzSMp7Rq02BTiqRdhY16UxOrqLKxSf+QqM9cL2JVz1qoXMxNWRx rgIsAo8/USj9rwdNl9YeSA/7r+PruZ4nAoCqRH+ipmhZrVUPKNhpRNPqc3XVVmoeBYWY +xfw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uH6LAflM; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.12; Thu, 27 Jun 2019 03:28:13 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uH6LAflM; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726674AbfF0K2M (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:12 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:39914 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726669AbfF0K2L (ORCPT ); Thu, 27 Jun 2019 06:28:11 -0400 Received: by mail-wr1-f65.google.com with SMTP id x4so1918162wrt.6 for ; Thu, 27 Jun 2019 03:28:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oSCqTZSbnLGprwAW+e+bdrFNNEkfE8SXrdCsMDCAbr0=; b=uH6LAflM9kUATgqIET7X+ah0VppC4JxoWVyXuragBeImnlFhlhm1lyzlP3g0wNpIwI pqwWmC+csqfdw9N3Lmz7ADIgq+rsq9Q9t4WV2iyOEWwdDzmniJW7Qa2iNS5nWIOGVlmJ NNYRhl/4o48r/K51/GbDBSqzlls3KCXzOP/XYb8MQgQrl0MevxaWCYjYLQ6KoWLocoZV BKZVUZCIPCRTtbNbEAQ8+uFN872ozvESM2y75LmttTke93BSqajtZmDdpP3ZaccbR+Bf ZeswzqjquZXJaVRNKCX0PevVVgHXqaZUqvKNttyRa8fvkLG8OeIFE6Th8ThHMd1Lwf7b bdfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oSCqTZSbnLGprwAW+e+bdrFNNEkfE8SXrdCsMDCAbr0=; b=Cp4qxCcupH2Q47fxzPvaYSOUY9cd3DKOYO1WAxNRPgcU+ucGvc+5xwaf+J/2ZaJCCM vWVZS/ZBiAOGKmNUQqQWt3zLyesBn8JPL54EL+Cave+OwIaAw6hlD8BtAuvbEZJSw3zJ lGn9xZLhc5ZBNkcfybCF50l7ka6wPKf4CGaKqcqF36y34GWWONwtCPueH7hJVGGWzm6V UUksOMcAc6yOoDf5oWM4vfK4WSN16kdzUsgwSK1a1m1qCMvb+H1rF3LBlH0Ndst6usWH p7UtstQx093KmuRAi11TvUXBX6VbWGvZKVIHp5wwmNOmP/sLCllf49G8Y+oSaaoj009N xsXA== X-Gm-Message-State: APjAAAXpJ5WJE9/yXIXz4Q9mFnB4t1X3tfNXEyXIdM60eP+i/tR53aEY 029I5FomraIfg8StK4lO0AOKe0K1iq0= X-Received: by 2002:a05:6000:11c2:: with SMTP id i2mr2724806wrx.199.1561631289230; Thu, 27 Jun 2019 03:28:09 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.08 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:08 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 17/32] crypto: aes - move sync ctr(aes) to AES library and generic helper Date: Thu, 27 Jun 2019 12:26:32 +0200 Message-Id: <20190627102647.2992-18-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org In preparation of duplicating the sync ctr(aes) functionality to modules under arch/arm, move the helper function from a inline .h file to the AES library, which is already depended upon by the drivers that use this fallback. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/aes-ctr-fallback.h | 53 -------------------- arch/arm64/crypto/aes-glue.c | 22 ++++++-- arch/arm64/crypto/aes-neonbs-glue.c | 21 ++++++-- 3 files changed, 33 insertions(+), 63 deletions(-) -- 2.20.1 diff --git a/arch/arm64/crypto/aes-ctr-fallback.h b/arch/arm64/crypto/aes-ctr-fallback.h deleted file mode 100644 index c9285717b6b5..000000000000 --- a/arch/arm64/crypto/aes-ctr-fallback.h +++ /dev/null @@ -1,53 +0,0 @@ -/* - * Fallback for sync aes(ctr) in contexts where kernel mode NEON - * is not allowed - * - * Copyright (C) 2017 Linaro Ltd - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - */ - -#include -#include - -asmlinkage void __aes_arm64_encrypt(u32 *rk, u8 *out, const u8 *in, int rounds); - -static inline int aes_ctr_encrypt_fallback(struct crypto_aes_ctx *ctx, - struct skcipher_request *req) -{ - struct skcipher_walk walk; - u8 buf[AES_BLOCK_SIZE]; - int err; - - err = skcipher_walk_virt(&walk, req, true); - - while (walk.nbytes > 0) { - u8 *dst = walk.dst.virt.addr; - u8 *src = walk.src.virt.addr; - int nbytes = walk.nbytes; - int tail = 0; - - if (nbytes < walk.total) { - nbytes = round_down(nbytes, AES_BLOCK_SIZE); - tail = walk.nbytes % AES_BLOCK_SIZE; - } - - do { - int bsize = min(nbytes, AES_BLOCK_SIZE); - - __aes_arm64_encrypt(ctx->key_enc, buf, walk.iv, - 6 + ctx->key_length / 4); - crypto_xor_cpy(dst, src, buf, bsize); - crypto_inc(walk.iv, AES_BLOCK_SIZE); - - dst += AES_BLOCK_SIZE; - src += AES_BLOCK_SIZE; - nbytes -= AES_BLOCK_SIZE; - } while (nbytes > 0); - - err = skcipher_walk_done(&walk, tail); - } - return err; -} diff --git a/arch/arm64/crypto/aes-glue.c b/arch/arm64/crypto/aes-glue.c index 3c80345d914f..6dc90557282d 100644 --- a/arch/arm64/crypto/aes-glue.c +++ b/arch/arm64/crypto/aes-glue.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -21,7 +22,6 @@ #include #include "aes-ce-setkey.h" -#include "aes-ctr-fallback.h" #ifdef USE_V8_CRYPTO_EXTENSIONS #define MODE "ce" @@ -404,13 +404,25 @@ static int ctr_encrypt(struct skcipher_request *req) return err; } -static int ctr_encrypt_sync(struct skcipher_request *req) +static void ctr_encrypt_one(struct crypto_skcipher *tfm, const u8 *src, u8 *dst) { - struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req); - struct crypto_aes_ctx *ctx = crypto_skcipher_ctx(tfm); + const struct crypto_aes_ctx *ctx = crypto_skcipher_ctx(tfm); + unsigned long flags; + + /* + * Temporarily disable interrupts to avoid races where + * evicted when the CPU is interrupted to do something + * else. + */ + local_irq_save(flags); + aes_encrypt(ctx, dst, src); + local_irq_restore(flags); +} +static int ctr_encrypt_sync(struct skcipher_request *req) +{ if (!crypto_simd_usable()) - return aes_ctr_encrypt_fallback(ctx, req); + return crypto_ctr_encrypt_walk(req, ctr_encrypt_one); return ctr_encrypt(req); } diff --git a/arch/arm64/crypto/aes-neonbs-glue.c b/arch/arm64/crypto/aes-neonbs-glue.c index cb8d90f795a0..933ce70a2504 100644 --- a/arch/arm64/crypto/aes-neonbs-glue.c +++ b/arch/arm64/crypto/aes-neonbs-glue.c @@ -11,13 +11,12 @@ #include #include #include +#include #include #include #include #include -#include "aes-ctr-fallback.h" - MODULE_AUTHOR("Ard Biesheuvel "); MODULE_LICENSE("GPL v2"); @@ -283,13 +282,25 @@ static int aesbs_xts_setkey(struct crypto_skcipher *tfm, const u8 *in_key, return aesbs_setkey(tfm, in_key, key_len); } -static int ctr_encrypt_sync(struct skcipher_request *req) +static void ctr_encrypt_one(struct crypto_skcipher *tfm, const u8 *src, u8 *dst) { - struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req); struct aesbs_ctr_ctx *ctx = crypto_skcipher_ctx(tfm); + unsigned long flags; + + /* + * Temporarily disable interrupts to avoid races where + * evicted when the CPU is interrupted to do something + * else. + */ + local_irq_save(flags); + aes_encrypt(&ctx->fallback, dst, src); + local_irq_restore(flags); +} +static int ctr_encrypt_sync(struct skcipher_request *req) +{ if (!crypto_simd_usable()) - return aes_ctr_encrypt_fallback(&ctx->fallback, req); + return crypto_ctr_encrypt_walk(req, ctr_encrypt_one); return ctr_encrypt(req); } From patchwork Thu Jun 27 10:26:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167915 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148749ilk; Thu, 27 Jun 2019 03:28:13 -0700 (PDT) X-Google-Smtp-Source: APXvYqzIUMjnhDHdHX+Zb2Y4XmHMaCyQioOy3rFUw+TW+whK9E4FGgl1a4/Uw7Fno1EzB22cyA5+ X-Received: by 2002:a17:90a:cb97:: with SMTP id a23mr5061261pju.67.1561631293570; Thu, 27 Jun 2019 03:28:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631293; cv=none; d=google.com; s=arc-20160816; b=LffjJhsmncX7z4aDrjLthV2gnXNMRRnhMDXskuKMapIrvGC5bUkLu2lfasheS2A8ML VKNdvD/iezd8e5x/n0S35Q2BcrBHblm4JjgF4PWDKGvvfYVuegiam2iu6CJxMdmf73tb Q2nFviY3qnCcniMerqbQwJ/TlTqc/3KkvFd9ktBNBBlXcESEV6MweGARqoung7//twrh VD7ytszG6DIO63lRfYL38B7bWngScXyYl0jFZY4Av+DF1Qwrzjaug/GsAqlABgQl38yI sbvmSZfyVA6bULuzl7fvOiZR7HWt7SgzC7EbOfUex1oP/XUjFQNCk/6kaRXzOwcKiV9f uCAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Lm7oGps4teOEwcLPMuq0r39VG4DYBydnuDjdvE8qRsQ=; b=ABCE9199eVwTGTBmCw1DxKF8KiBP1+xgKKSBdDYqsZ1OpK144ztZc+b6feRJ4+thlC RfxvttyyeCD/+uIz7Zs3iE/EEGaGbIr9c7kjdCZGOU7mDy6QMkk+mYNYj2syqvtUuYfO qpkworrF/a8SrLVRXMeOdCTrgDphwVw5s5KhupHd0hOitzML9eZ1HAzKPSJ93SpLSxB4 lCD7VlbWYOrIAxV3vxuOPs72e1YFFlRb6Q99lrnomknV1WU7nGZIjXSJSFjRNuYZnb3m 9ddkYmI7aQgXl2w8UN7igBiv4e7w02T0v38EcHg13p7N7mMtXaIdyIWdXrZvIUl6DdX3 zQRA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TtoZmtib; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.13; Thu, 27 Jun 2019 03:28:13 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TtoZmtib; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726681AbfF0K2M (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:12 -0400 Received: from mail-wm1-f67.google.com ([209.85.128.67]:37188 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726678AbfF0K2M (ORCPT ); Thu, 27 Jun 2019 06:28:12 -0400 Received: by mail-wm1-f67.google.com with SMTP id f17so5110878wme.2 for ; Thu, 27 Jun 2019 03:28:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Lm7oGps4teOEwcLPMuq0r39VG4DYBydnuDjdvE8qRsQ=; b=TtoZmtibts70TH1a3wjS/gKu5pk5IvIOaZcpHpJay0/wt1KzHU/5mkmjSDI7esgDVf TkqpbMLLzVhD7G2+9q2D8ZeapC8IvfQW5XJgrp0QcCr3qxJrEoOHLek3aaFBywbstMPV oEkOe7BJM/UhqPDkm2coXdE288VWc9tr4UvSnMY3KCkaLtHCIakOUGuMP5BZdcijs1st vy/KVQVpwx2KmvjG+cGxf4DlQ7YuAllVxREc8oQCFPpzNTMdzcdLpmxn2ydd1r2Nnj7h ZHTmpMKGTcm2KWsi+iJmYMpQ2DDGh58+KWR33/sVAoggG3aD89wAYFJ659zuHdFdUm9O H5HQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Lm7oGps4teOEwcLPMuq0r39VG4DYBydnuDjdvE8qRsQ=; b=pRGSe1KQ8lROgLKIidumtfHfneAciTIs4YXFCd7uphsNY3sfRmt1mgDHpnJHvznYNz yC52DmYNLawJoxVR68sAZ8d4a1hvsc6OrQQdxlAkzShHLdHKb3NJvk0NGRVYqIWO4B9Q My/Kb+Ls/hA59S0kxsH5TKIhSdvD8/NwhpSIRT7KSzjIKDXuCMszJsgMH/cpZsHETx3g iOhN3gJwyvB/ayRpd0ZlpVNeIeZOOcE8nFbyGbKlj49Px+NHhkPKkga37h1hOJfMQnzd ia5u0AdGyA+cFjDu4CWJpOlX04mfP59eEUgVnghKakEDFsAe/P32weQSezDO/qS4HQl2 fGHA== X-Gm-Message-State: APjAAAWwiC+hy1tUnaW5/DCJUs93xZaNX3bWcZDo1G2qBLUnpha/CAy3 y8ZRqWdBdbChCf1ZpL34344engnViTs= X-Received: by 2002:a1c:9ecd:: with SMTP id h196mr2822857wme.98.1561631290333; Thu, 27 Jun 2019 03:28:10 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.09 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:09 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 18/32] crypto: arm64/aes-ce-cipher - use AES library as fallback Date: Thu, 27 Jun 2019 12:26:33 +0200 Message-Id: <20190627102647.2992-19-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Instead of calling into the table based scalar AES code in situations where the SIMD unit may not be used, use the generic AES code, which is more appropriate since it is less likely to be susceptible to timing attacks. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/Kconfig | 2 +- arch/arm64/crypto/aes-ce-glue.c | 7 ++----- arch/arm64/crypto/aes-cipher-glue.c | 3 --- 3 files changed, 3 insertions(+), 9 deletions(-) -- 2.20.1 diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig index 66dea518221c..4922c4451e7c 100644 --- a/arch/arm64/crypto/Kconfig +++ b/arch/arm64/crypto/Kconfig @@ -73,7 +73,7 @@ config CRYPTO_AES_ARM64_CE tristate "AES core cipher using ARMv8 Crypto Extensions" depends on ARM64 && KERNEL_MODE_NEON select CRYPTO_ALGAPI - select CRYPTO_AES_ARM64 + select CRYPTO_LIB_AES config CRYPTO_AES_ARM64_CE_CCM tristate "AES in CCM mode using ARMv8 Crypto Extensions" diff --git a/arch/arm64/crypto/aes-ce-glue.c b/arch/arm64/crypto/aes-ce-glue.c index 3213843fcb46..6890e003b8f1 100644 --- a/arch/arm64/crypto/aes-ce-glue.c +++ b/arch/arm64/crypto/aes-ce-glue.c @@ -23,9 +23,6 @@ MODULE_DESCRIPTION("Synchronous AES cipher using ARMv8 Crypto Extensions"); MODULE_AUTHOR("Ard Biesheuvel "); MODULE_LICENSE("GPL v2"); -asmlinkage void __aes_arm64_encrypt(u32 *rk, u8 *out, const u8 *in, int rounds); -asmlinkage void __aes_arm64_decrypt(u32 *rk, u8 *out, const u8 *in, int rounds); - struct aes_block { u8 b[AES_BLOCK_SIZE]; }; @@ -54,7 +51,7 @@ static void aes_cipher_encrypt(struct crypto_tfm *tfm, u8 dst[], u8 const src[]) struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); if (!crypto_simd_usable()) { - __aes_arm64_encrypt(ctx->key_enc, dst, src, num_rounds(ctx)); + aes_encrypt(ctx, dst, src); return; } @@ -68,7 +65,7 @@ static void aes_cipher_decrypt(struct crypto_tfm *tfm, u8 dst[], u8 const src[]) struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); if (!crypto_simd_usable()) { - __aes_arm64_decrypt(ctx->key_dec, dst, src, num_rounds(ctx)); + aes_decrypt(ctx, dst, src); return; } diff --git a/arch/arm64/crypto/aes-cipher-glue.c b/arch/arm64/crypto/aes-cipher-glue.c index 0e90b06ebcec..bf32cc6489e1 100644 --- a/arch/arm64/crypto/aes-cipher-glue.c +++ b/arch/arm64/crypto/aes-cipher-glue.c @@ -13,10 +13,7 @@ #include asmlinkage void __aes_arm64_encrypt(u32 *rk, u8 *out, const u8 *in, int rounds); -EXPORT_SYMBOL(__aes_arm64_encrypt); - asmlinkage void __aes_arm64_decrypt(u32 *rk, u8 *out, const u8 *in, int rounds); -EXPORT_SYMBOL(__aes_arm64_decrypt); static void aes_arm64_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { From patchwork Thu Jun 27 10:26:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167916 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148777ilk; Thu, 27 Jun 2019 03:28:15 -0700 (PDT) X-Google-Smtp-Source: APXvYqyQoMmKdS/lZVz/9vXlAteor7gPcfHmWBzi3Jm8rKxG5XK1EvKeZ92ihx8eQOLRb4LGi6hb X-Received: by 2002:a17:90a:384d:: with SMTP id l13mr5248160pjf.86.1561631295354; Thu, 27 Jun 2019 03:28:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631295; cv=none; d=google.com; s=arc-20160816; b=Q2pMi+hfRTRlu/ehm5Kuvp7nHj83KZN7zbRDiQsc2VJJiDueGNmIUC5KcV1pI/pV0m EOsA4tRBLY06mfk+FS6QxclrWq71lGSLy0pLAVDX1u3lZV9GgMODdLwnuh00htwv1n1/ DPkzy10YuChXeQ6YrfwAjPHHFlYK621Zat7oF16u3RJ9ptfq9QlbHchy/ZwsKbk3IHmj 6J+QJLji6ylGXbO8dKIz3raSO/fl6nNWYiM7oUHg3EzlT4ZrgNGgzVlPzQHiGsg4SOMz gVexvP8kMCFsqMkdNQcHceCXrE6JMbCgOLiLqoipKd2bvpsznh5PRzISdiJodR+AlDaO ANvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=JgRwk7rV4YDW0Cr7MyntD6TN2vlr6LPUNeCX8j451cs=; b=E4D+dDiTOIXDL0HnUfh20fAlMCsSMzJlkWYJHJpid+15rkDugrBuAckr1+5N01LMty 3YHLAjDLd15AciwRftoGgs725F0MDTfgxVwtDgPlc4kPc1Y6Y4EgjJXoWRYZRsYqQKSV zZMBEDSDA0RITr3wH11PaJfgotQyO+VzL6WjO41AFO03sYH7mlyUGWyJIu/cAVQsX/vv hnllwOpBTtGcy8u5Wz5B+QvVq7MRYeEEuqzavZ3UhmGMiuifkl1dbOKqvitSiC+FIBTl yoKqbeAn9ECrlteNKxRQfMSJ2PzULYZJymmmN1HaU0Y+evG6XRlf951eWtRprZaCDlmf dVVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="LbL/3Rpq"; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.15; Thu, 27 Jun 2019 03:28:15 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="LbL/3Rpq"; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726382AbfF0K2O (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:14 -0400 Received: from mail-wm1-f52.google.com ([209.85.128.52]:52003 "EHLO mail-wm1-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726669AbfF0K2N (ORCPT ); Thu, 27 Jun 2019 06:28:13 -0400 Received: by mail-wm1-f52.google.com with SMTP id 207so5186655wma.1 for ; Thu, 27 Jun 2019 03:28:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JgRwk7rV4YDW0Cr7MyntD6TN2vlr6LPUNeCX8j451cs=; b=LbL/3RpqaXV6IJH+KwFIsRaEzDlymHrfnWS+TLt48c+YJKKUiudhNceRorjI8YOG2V hus8CmnRWKtWW+Df7e9OUaheziY8BdwIdeau9Gqt9SXAqcsRKh7PSrwekMPEslPX4+hl d0ntvwoWTKZKIutRBwhOeSmIBH3APnZYid5Yyiv5tzM7/USKPk9fx3BJhHi9aglDKkaS 4tAh7th4amPKhUE+0xMZlaZXPhpPj6gyUOqdVVBtf7bz7fE3Y6uK9FGy7CVk7hSH77tj EXWxY6oXjibKEX6CLiPf4wqT60pE0o1VKYKgGelkdI9Mhm+rtK1+FDqOOeKiH19L158Y FA1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JgRwk7rV4YDW0Cr7MyntD6TN2vlr6LPUNeCX8j451cs=; b=SLxgPkID2LhXMfghS5gIlTnBDA6PQpAHvD2Liz34407a+RT1dpNZB5zadYpGhTvgdj ghwVoOXdZAdYfJMnSX9+zi0hfFHt6uZT/HSq/nyircnjJiwQ++8qFQuAQIxKq4mY5dTM XDO2bYuevUhxOsnINrgJVzxMdq60m5S8XDEXFnCKna9gDzswl1naSslkBgzKp6E5gpOC DA0u1b3zcnI9Rk+n3blYYkvE5b1iI6I5Bnb6dJR8kJqJQLNliLo0r8FSR/yqvLKwFU6r xezZN9cHmsWZLRKxSoyWXbRuGuZ5LoQOgpyhuXuYQ1e1ZiDU3oDK2xt6oqJz5xIrLBy8 KxIQ== X-Gm-Message-State: APjAAAU0XGl3pxJPoFRzlCsDNSxwGnBOxFrZSiE0oqQqQkUrfD8pdrw8 76pgGxEaMxFncvjNM7oZVaV+cP7Nhwk= X-Received: by 2002:a1c:7a01:: with SMTP id v1mr2846599wmc.10.1561631291353; Thu, 27 Jun 2019 03:28:11 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.10 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:10 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 19/32] crypto: aes/arm - use native endiannes for key schedule Date: Thu, 27 Jun 2019 12:26:34 +0200 Message-Id: <20190627102647.2992-20-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Align ARM's hw instruction based AES implementation with other versions that keep the key schedule in native endianness. This will allow us to merge the various implementations going forward. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/aes-ce-core.S | 20 ++++++++++---------- arch/arm/crypto/aes-ce-glue.c | 9 +++------ 2 files changed, 13 insertions(+), 16 deletions(-) -- 2.20.1 diff --git a/arch/arm/crypto/aes-ce-core.S b/arch/arm/crypto/aes-ce-core.S index bc53bcaa772e..3692b8735ef7 100644 --- a/arch/arm/crypto/aes-ce-core.S +++ b/arch/arm/crypto/aes-ce-core.S @@ -91,19 +91,19 @@ .macro do_block, dround, fround cmp r3, #12 @ which key size? - vld1.8 {q10-q11}, [ip]! + vld1.32 {q10-q11}, [ip]! \dround q8, q9 - vld1.8 {q12-q13}, [ip]! + vld1.32 {q12-q13}, [ip]! \dround q10, q11 - vld1.8 {q10-q11}, [ip]! + vld1.32 {q10-q11}, [ip]! \dround q12, q13 - vld1.8 {q12-q13}, [ip]! + vld1.32 {q12-q13}, [ip]! \dround q10, q11 blo 0f @ AES-128: 10 rounds - vld1.8 {q10-q11}, [ip]! + vld1.32 {q10-q11}, [ip]! \dround q12, q13 beq 1f @ AES-192: 12 rounds - vld1.8 {q12-q13}, [ip] + vld1.32 {q12-q13}, [ip] \dround q10, q11 0: \fround q12, q13, q14 bx lr @@ -152,8 +152,8 @@ ENDPROC(aes_decrypt_3x) .macro prepare_key, rk, rounds add ip, \rk, \rounds, lsl #4 - vld1.8 {q8-q9}, [\rk] @ load first 2 round keys - vld1.8 {q14}, [ip] @ load last round key + vld1.32 {q8-q9}, [\rk] @ load first 2 round keys + vld1.32 {q14}, [ip] @ load last round key .endm /* @@ -508,8 +508,8 @@ ENDPROC(ce_aes_sub) * operation on round key *src */ ENTRY(ce_aes_invert) - vld1.8 {q0}, [r1] + vld1.32 {q0}, [r1] aesimc.8 q0, q0 - vst1.8 {q0}, [r0] + vst1.32 {q0}, [r0] bx lr ENDPROC(ce_aes_invert) diff --git a/arch/arm/crypto/aes-ce-glue.c b/arch/arm/crypto/aes-ce-glue.c index 04ba66903674..e6da3e30018b 100644 --- a/arch/arm/crypto/aes-ce-glue.c +++ b/arch/arm/crypto/aes-ce-glue.c @@ -10,6 +10,7 @@ #include #include +#include #include #include #include @@ -80,21 +81,17 @@ static int ce_aes_expandkey(struct crypto_aes_ctx *ctx, const u8 *in_key, key_len != AES_KEYSIZE_256) return -EINVAL; - memcpy(ctx->key_enc, in_key, key_len); ctx->key_length = key_len; + for (i = 0; i < kwords; i++) + ctx->key_enc[i] = get_unaligned_le32(in_key + i * sizeof(u32)); kernel_neon_begin(); for (i = 0; i < sizeof(rcon); i++) { u32 *rki = ctx->key_enc + (i * kwords); u32 *rko = rki + kwords; -#ifndef CONFIG_CPU_BIG_ENDIAN rko[0] = ror32(ce_aes_sub(rki[kwords - 1]), 8); rko[0] = rko[0] ^ rki[0] ^ rcon[i]; -#else - rko[0] = rol32(ce_aes_sub(rki[kwords - 1]), 8); - rko[0] = rko[0] ^ rki[0] ^ (rcon[i] << 24); -#endif rko[1] = rko[0] ^ rki[1]; rko[2] = rko[1] ^ rki[2]; rko[3] = rko[2] ^ rki[3]; From patchwork Thu Jun 27 10:26:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167919 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148785ilk; Thu, 27 Jun 2019 03:28:15 -0700 (PDT) X-Google-Smtp-Source: APXvYqzgwFZARm0/46LFI55nr66pSsm1fgMzcHxF1J6X0xPAeIecRgsy56MFTl0pC/H6jkT+pwzy X-Received: by 2002:a17:902:8205:: with SMTP id x5mr3815813pln.279.1561631295639; Thu, 27 Jun 2019 03:28:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631295; cv=none; d=google.com; s=arc-20160816; b=c4FiYhDQ/p0i0sNWc0wKbgLCC85eV47uTPKUHH1jypMf795jLxqs2AEj6sxM2hDIeQ B9AYontFV0J9PoiY3WXR5P7mLXVwpu6lIc2syXCQQotYQ3GMXGY5l9LGHN97dpVAo5QJ 7eod+1ZICqSDNNsKzxuZV4LPi1SUjsMSrld/yOOZIjf1fMghYJDAttCm78Bz/TJHYfjk AXSWdSufvXUKO1TOAG7oI5CctDozPD0leh7kBaMqwUcFAXzdJRErdoAsta4JpZrj3g+f lyhI9Ggdmc8ivh8oux/+5mblXVMbOR9xigDJieoNjQkkH7rUx1uNamBDoFzGlafMG+sQ h4Ew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=i5YAml7KVGLr/1+CAScarRaj3NSVjbEGTREd9Bd1Ltw=; b=qnuH2evO7YCGEyEHfxmBZb+Jp15VrRCCCm2sYdrtXCDk3FrP/AwpszI8nWL+rVTSUL rRb0COj5wwW//Cp1geS6IFSH6WMkt73YJSLpQxgPlANfTxTE6t0BJ+0cd2Oyb5T5wfGY cGd25azuL1PJuKekvlXZIqOGdwgiUq2BI59cMqzauCIQE8lK/E/6BIZUh8+R4UB15zG2 qB7y9Hmw9645rcm6Nv3sFDMc6yL+zB/OX2Ch67EL6A9bZyB3iyBLc9mzk0m141IW2iT7 Jw9E1Z4wfsHj/pVAZVZem4TFD3CCXIWfQmy2xjziLzCHFFuv8H1Dj616/AjEn9rTTfDr SgSQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tankqLnd; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.15; Thu, 27 Jun 2019 03:28:15 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tankqLnd; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726669AbfF0K2O (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:14 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:44978 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726678AbfF0K2O (ORCPT ); Thu, 27 Jun 2019 06:28:14 -0400 Received: by mail-wr1-f65.google.com with SMTP id r16so49818wrl.11 for ; Thu, 27 Jun 2019 03:28:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=i5YAml7KVGLr/1+CAScarRaj3NSVjbEGTREd9Bd1Ltw=; b=tankqLndRYXGNmAVTZk/Pky4kvmRMq1n2DYXeBLsNcqN8AqdSHFpUqyTNDAsj1gmJ3 wWsscDBDce7G71pfIaDJW2vj6uuiWnJY2MIR+1Bepvhl3DW0DrTSl0bbMYipl6osGYp+ LTJCsAh2bRq+d8e4ZiXJMVqPAWn5EL1ELFMIEUVwCjfbhwBKXwfuFHbE7V7FUhEJyk0s vXzDL4pqqTOPyBpk4+1R/MkGDgHJ+3fPXQhhpeNyUmQ3J4ikBEEEcJNMDjgUGcsexVkM /KBbPTyzrel5SZaIo6rGgvSNbNI2hvOvvbRFzH9NoSCfenQFbucSdGO1BFOK7sa2BMAK wHRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=i5YAml7KVGLr/1+CAScarRaj3NSVjbEGTREd9Bd1Ltw=; b=JeHCI9Q+hslh41z5JQJmwPxqpVf4P1qovPLrRuEQoX6/Nr2xj7LEucagOvgRHcqijB fxuV9u5tOcWpGHmcCVse1DNyj1XSsBztfUmunlnqAG2tqy5HN+30fBx1DRi9UwP0cynW fP3e24I+gfnF5G2dYiUo3jBIyvirdlpPG7DOQheXYwDUlya9wsAaa0nBRiv+armn2EKJ TSBKFA5P+bRSALgCxWYGsChDw9YKT1OeoBCSNUQnq86sugwl445wXytmHY9uHwllFekL QY8TCfp7ZqC4i8xDQ1XMRmgXoP2GiFV0QF6sud5WV3wFn/pbV69y9f3WWdTZG3LRhQkl Zmjw== X-Gm-Message-State: APjAAAXJ673Gz9iBOnqS0O5TzcSU1KKY5kZJX5EaBmTQS/HZHrSMWsyn sWG2dK667J+8lkenxqxTFUwgRZDkYnc= X-Received: by 2002:adf:a143:: with SMTP id r3mr95191wrr.236.1561631292517; Thu, 27 Jun 2019 03:28:12 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.11 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:11 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 20/32] crypto: arm/aes-ce - provide a synchronous version of ctr(aes) Date: Thu, 27 Jun 2019 12:26:35 +0200 Message-Id: <20190627102647.2992-21-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org AES in CTR mode is used by modes such as GCM and CCM, which are often used in contexts where only synchronous ciphers are permitted. So provide a synchronous version of ctr(aes) based on the existing code. This requires a non-SIMD fallback to deal with invocations occurring from a context where SIMD instructions may not be used. We have a helper for this now in the AES library, so wire that up. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/aes-ce-glue.c | 43 ++++++++++++++++++++ 1 file changed, 43 insertions(+) -- 2.20.1 diff --git a/arch/arm/crypto/aes-ce-glue.c b/arch/arm/crypto/aes-ce-glue.c index e6da3e30018b..c3a78c5a5c35 100644 --- a/arch/arm/crypto/aes-ce-glue.c +++ b/arch/arm/crypto/aes-ce-glue.c @@ -10,8 +10,10 @@ #include #include +#include #include #include +#include #include #include #include @@ -289,6 +291,29 @@ static int ctr_encrypt(struct skcipher_request *req) return err; } +static void ctr_encrypt_one(struct crypto_skcipher *tfm, const u8 *src, u8 *dst) +{ + struct crypto_aes_ctx *ctx = crypto_skcipher_ctx(tfm); + unsigned long flags; + + /* + * Temporarily disable interrupts to avoid races where + * evicted when the CPU is interrupted to do something + * else. + */ + local_irq_save(flags); + aes_encrypt(ctx, dst, src); + local_irq_restore(flags); +} + +static int ctr_encrypt_sync(struct skcipher_request *req) +{ + if (!crypto_simd_usable()) + return crypto_ctr_encrypt_walk(req, ctr_encrypt_one); + + return ctr_encrypt(req); +} + static int xts_encrypt(struct skcipher_request *req) { struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req); @@ -378,6 +403,21 @@ static struct skcipher_alg aes_algs[] = { { .setkey = ce_aes_setkey, .encrypt = ctr_encrypt, .decrypt = ctr_encrypt, +}, { + .base.cra_name = "ctr(aes)", + .base.cra_driver_name = "ctr-aes-ce-sync", + .base.cra_priority = 300 - 1, + .base.cra_blocksize = 1, + .base.cra_ctxsize = sizeof(struct crypto_aes_ctx), + .base.cra_module = THIS_MODULE, + + .min_keysize = AES_MIN_KEY_SIZE, + .max_keysize = AES_MAX_KEY_SIZE, + .ivsize = AES_BLOCK_SIZE, + .chunksize = AES_BLOCK_SIZE, + .setkey = ce_aes_setkey, + .encrypt = ctr_encrypt_sync, + .decrypt = ctr_encrypt_sync, }, { .base.cra_name = "__xts(aes)", .base.cra_driver_name = "__xts-aes-ce", @@ -421,6 +461,9 @@ static int __init aes_init(void) return err; for (i = 0; i < ARRAY_SIZE(aes_algs); i++) { + if (!(aes_algs[i].base.cra_flags & CRYPTO_ALG_INTERNAL)) + continue; + algname = aes_algs[i].base.cra_name + 2; drvname = aes_algs[i].base.cra_driver_name + 2; basename = aes_algs[i].base.cra_driver_name; From patchwork Thu Jun 27 10:26:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167917 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148817ilk; Thu, 27 Jun 2019 03:28:16 -0700 (PDT) X-Google-Smtp-Source: APXvYqyFPqXpdFnVyy/nDhyze8fUYa47chij+ROwgHXiKa7EntWZh7V7qLaV3rWPZaD0XS/qkT9p X-Received: by 2002:a63:6cc3:: with SMTP id h186mr3065559pgc.292.1561631296862; Thu, 27 Jun 2019 03:28:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631296; cv=none; d=google.com; s=arc-20160816; b=zP70WuFeSx7EsXVXfczTYs2Df9m+hYNbr/KosxFRMP/Jc4n/G2dsr384NgfvBekGlF eEyTeAKj6/WTNbSeL3fX7j5sdy7DpboAjCPbCmJ7nX4rJmHzJ3Y4/7NgV6Su4o7Ea57E sElWtEP77xrUB4yOWq+4BS8n4LAk4PpDUsFSkQS4hkRs2l8seQl/KLugbHh9B6N3Jj31 RhUL0Qma5k1u3vFej6711+mb5b7tXoslougVhWmwqSW9XRXTVXT3zjVMhblYsGPZk2ja uCl4lmXkvtRue/XKPr8bSYKHDet9mo1p0bmbihB6JZgdqv4Y0rqvGG/Z9f3jvbQAxV3x tIBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=vL6L5hDUn9mhzlJsgchovHgAG3ROBqwVQK1Apo8D4Bw=; b=hJ07v2xG5qP78boPld2fYTQSq72RThuQrvW6ePwRy1cBSEbJNeYl38CofzDZdw6Jt3 SBgwbzgGXgISmPlPfMVgyz3sdLOFOjkHT4iuMq6gMTHOY47lk642+O1zjD5wdcRtP3S4 c6IldA2doSKUC5drVGJDbhhjaLBBf2vvYBa2PhQKDr6nIt6Qm0q5ppXdpTJzF+TQXuMt ykbA9M9ziDA2F/7+ym35x48htTbyPGk6UhU3S8wxny+75Ing3fQi8J1CIneLTRpfaI5B jkGKOaPtLsrctEJP8ihTzjjsintBhJtBxqdA3SVBz4j3Ouav8gc7DleJFlMOdEuBddC/ 1g2A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rjbJKmZa; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.16; Thu, 27 Jun 2019 03:28:16 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rjbJKmZa; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726687AbfF0K2Q (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:16 -0400 Received: from mail-wr1-f67.google.com ([209.85.221.67]:41688 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726682AbfF0K2P (ORCPT ); Thu, 27 Jun 2019 06:28:15 -0400 Received: by mail-wr1-f67.google.com with SMTP id c2so1903284wrm.8 for ; Thu, 27 Jun 2019 03:28:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vL6L5hDUn9mhzlJsgchovHgAG3ROBqwVQK1Apo8D4Bw=; b=rjbJKmZaLfFvG68L2HeJd6+nxvTiSFwE2g5H7gZ6eXamIPreK2PdaFtkeBtxL+JLpD lp0oa3N8G966wEQvOo2DduyWYINH+Nnwzis5OH9LnLV2DzfWv6xOVFajOJqTbbTbztKS 3YOYwdwqCR0S0RsurTF9cMKsgxCQVS6N8RiknP8V6MnVjEmanMMcpLoF6dkgBY7iz7On dLMhA9fdgF+Y66G8/7/zt5/MLbzQ7l8IHr3L2NzWbU5Vy9k0nsROBosGBXm1iFq0JbF1 ouzVBDX+LvCKXGiYEfHgNG4Zc3RGUzYA8ZW0LQLLsTfnn7ZwpJZ/BSFndKstWkiWxx7J C1Rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vL6L5hDUn9mhzlJsgchovHgAG3ROBqwVQK1Apo8D4Bw=; b=tuUD7jAEiYeCM8L/quJH8IlUxHzopr9+zEboG1ywzPqT8kKlF/NQC5LyOv54uxQbjF ay+BVPR//2VqXvefEb8vC1nOXYwE3ciIMHviVFGvl4WCsm/6xsYSSDv+d1zIL1VdWLlC iH47HxvIqULMZSJCXkS50Gd2WvdGpkvdc56cZUpNoO29i8NlMUXNJswzCFUyjiUWmcTt Qkj7k3DN6SpXkFuH3lFp7gUfopR7W63VtjqVV38oHCm9UMdIxw9+kzLdtA8YkFw474wS HbIBE0Lj/QSgRBrrGhsUl7dJ2QH7P9D2n6IqJdeH9sBESYfcoWBKO3uu9SAiI5Qi1RwM oQpA== X-Gm-Message-State: APjAAAVVGGSL/I7P8FUjc8LGKx+hnGn9toP1IWCyNx84/vVZRIAMYT4C H+DDiS2Yq+vlN1H9HC10jLvRRYGaXOU= X-Received: by 2002:a5d:400f:: with SMTP id n15mr2737951wrp.312.1561631293493; Thu, 27 Jun 2019 03:28:13 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.12 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:12 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 21/32] crypto: arm/aes-neonbs - provide a synchronous version of ctr(aes) Date: Thu, 27 Jun 2019 12:26:36 +0200 Message-Id: <20190627102647.2992-22-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org AES in CTR mode is used by modes such as GCM and CCM, which are often used in contexts where only synchronous ciphers are permitted. So provide a synchronous version of ctr(aes) based on the existing code. This requires a non-SIMD fallback to deal with invocations occurring from a context where SIMD instructions may not be used. We have a helper for this now in the AES library, so wire that up. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/aes-neonbs-glue.c | 65 ++++++++++++++++++++ 1 file changed, 65 insertions(+) -- 2.20.1 diff --git a/arch/arm/crypto/aes-neonbs-glue.c b/arch/arm/crypto/aes-neonbs-glue.c index f43c9365b6a9..2f1aa199926c 100644 --- a/arch/arm/crypto/aes-neonbs-glue.c +++ b/arch/arm/crypto/aes-neonbs-glue.c @@ -9,8 +9,10 @@ */ #include +#include #include #include +#include #include #include #include @@ -57,6 +59,11 @@ struct aesbs_xts_ctx { struct crypto_cipher *tweak_tfm; }; +struct aesbs_ctr_ctx { + struct aesbs_ctx key; /* must be first member */ + struct crypto_aes_ctx fallback; +}; + static int aesbs_setkey(struct crypto_skcipher *tfm, const u8 *in_key, unsigned int key_len) { @@ -192,6 +199,25 @@ static void cbc_exit(struct crypto_tfm *tfm) crypto_free_cipher(ctx->enc_tfm); } +static int aesbs_ctr_setkey_sync(struct crypto_skcipher *tfm, const u8 *in_key, + unsigned int key_len) +{ + struct aesbs_ctr_ctx *ctx = crypto_skcipher_ctx(tfm); + int err; + + err = aes_expandkey(&ctx->fallback, in_key, key_len); + if (err) + return err; + + ctx->key.rounds = 6 + key_len / 4; + + kernel_neon_begin(); + aesbs_convert_key(ctx->key.rk, ctx->fallback.key_enc, ctx->key.rounds); + kernel_neon_end(); + + return 0; +} + static int ctr_encrypt(struct skcipher_request *req) { struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req); @@ -234,6 +260,29 @@ static int ctr_encrypt(struct skcipher_request *req) return err; } +static void ctr_encrypt_one(struct crypto_skcipher *tfm, const u8 *src, u8 *dst) +{ + struct aesbs_ctr_ctx *ctx = crypto_skcipher_ctx(tfm); + unsigned long flags; + + /* + * Temporarily disable interrupts to avoid races where + * evicted when the CPU is interrupted to do something + * else. + */ + local_irq_save(flags); + aes_encrypt(&ctx->fallback, dst, src); + local_irq_restore(flags); +} + +static int ctr_encrypt_sync(struct skcipher_request *req) +{ + if (!crypto_simd_usable()) + return crypto_ctr_encrypt_walk(req, ctr_encrypt_one); + + return ctr_encrypt(req); +} + static int aesbs_xts_setkey(struct crypto_skcipher *tfm, const u8 *in_key, unsigned int key_len) { @@ -361,6 +410,22 @@ static struct skcipher_alg aes_algs[] = { { .setkey = aesbs_setkey, .encrypt = ctr_encrypt, .decrypt = ctr_encrypt, +}, { + .base.cra_name = "ctr(aes)", + .base.cra_driver_name = "ctr-aes-neonbs-sync", + .base.cra_priority = 250 - 1, + .base.cra_blocksize = 1, + .base.cra_ctxsize = sizeof(struct aesbs_ctr_ctx), + .base.cra_module = THIS_MODULE, + + .min_keysize = AES_MIN_KEY_SIZE, + .max_keysize = AES_MAX_KEY_SIZE, + .chunksize = AES_BLOCK_SIZE, + .walksize = 8 * AES_BLOCK_SIZE, + .ivsize = AES_BLOCK_SIZE, + .setkey = aesbs_ctr_setkey_sync, + .encrypt = ctr_encrypt_sync, + .decrypt = ctr_encrypt_sync, }, { .base.cra_name = "__xts(aes)", .base.cra_driver_name = "__xts-aes-neonbs", From patchwork Thu Jun 27 10:26:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167918 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148852ilk; Thu, 27 Jun 2019 03:28:18 -0700 (PDT) X-Google-Smtp-Source: APXvYqwaJQBXco+aGItWai6osXdPPWrfgx/y2B1LoBmze0A6mrYoFn7HEZMoSyxKJNMshO/1GCa8 X-Received: by 2002:a63:e506:: with SMTP id r6mr3117543pgh.324.1561631298787; Thu, 27 Jun 2019 03:28:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631298; cv=none; d=google.com; s=arc-20160816; b=TTnjSBqZ+Iu4g1HtyBl7sOciCQjG3DCRYfJ9VNFGj8zuVnDa9i8RxzjaNOITKlmnBM JcU9mBiMxZfPPAgq4Aqsf4MGasKgKkGWuuduX7MaH/YSjB/bnKwdGc0y9dneLxUU8Oju itSSWd8T1i67pXe3+qAO8VaY8F6BGnQSb84Cd7ndnJiAwp4HmO0KPQq+Gbyikt9n0BGv 4jmcBGBziytrdk+VqpCmYq/cJJGayeAR/t7HuSFnt/XrRE11aq0BDmG6Ir/RKZ0sindh eMmHElCpRaEPZ2bEUmWS+Qj5IUpLe+rbtM9HRVTfpsn7hQ4+oH0DqOs5NYiNUKfs78Bw BPBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0vLqup8kXDSW4TBicl2CoUwy3k2oxTX/PfRrsOfcNio=; b=tVt/+mMIFpHqkMuwC1vcohxaHCvHdsP3vaMIaCydHOibXvQjqOUTlx1o4wa1XcZd0S ycrrS3cYm7uOJ6KTD0pM4K7fJI5KYq6vVhbRE2/4A8Qw0fc9vBLOFpCpZIeeacFZPbaE tRHA00GNdQA5AFFXReXiolcKXYC4aLx/8y9yuG2uYfz5YSsNKxS14Ost3qrQPq1Nj3rd uDGnSrBRU1VSCdUvG+gyNJzCuMUyqQFZpNd6OHqfgx+AyUBJeItjpMLVx3JPQnRIt2rF 64F5LK9lLoghXm6F4GLDxU4CSFjw5vNPEiwFCyb9RY5uMIIy+7zShehFcFAMaT8kKpv6 9cnQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="OZchAA/q"; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.18; Thu, 27 Jun 2019 03:28:18 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="OZchAA/q"; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726688AbfF0K2S (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:18 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:34912 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726678AbfF0K2R (ORCPT ); Thu, 27 Jun 2019 06:28:17 -0400 Received: by mail-wr1-f65.google.com with SMTP id f15so1939371wrp.2 for ; Thu, 27 Jun 2019 03:28:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0vLqup8kXDSW4TBicl2CoUwy3k2oxTX/PfRrsOfcNio=; b=OZchAA/q1hTUC9cas8hAf9R8PdSiHKHjlBK/GAVwkT1ghWkWPy6kxs7rAAnWBYSIGJ jn/B/MS1A06Xd5CkR/T4FMkY67CDidISj4l76xApXeQG48EjRIJC2b6SyCXLC5iQ9mI3 U6fwAoK+gYEmOrF6ZmmUUk5VQBXck9sMDzZGIgK0xx8TsVqf1k8bip/+/3vNSpMtTMlk VTeGhmWOyEnzkIKjOLkG1+tg/T5N/7EFs//a9bQ8BQY1uJBP9BGEWB+IN8gkKENR5hgo Wk9QB3ICGaNp6a7REV7rvWigp8WGxqoEM1t6wU8ZodjBQwfi0K5ab959j+4GkAwGFiFK b2VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0vLqup8kXDSW4TBicl2CoUwy3k2oxTX/PfRrsOfcNio=; b=PeFQgyN0JV62FqYcjl8dHeGOVPsMQeQUB0TP8V8boIQgYKA+UkOg3FOtnn2IYfBbw4 ptYfqKTuLsFB8QO3EbiZHrOQfPn46i1WyNwZ5XtBxsR2ttfKZM8USXVI+WOpMqc+ZSVe atx4ODGzqg9nWshk+Vrpt0tekoztVjMX7MWArlYvQwk1qPqKap47XUo1zISfXKnIGTED n6DCjPlBQGaQ7WYHxgE+UtlfdGBHd9P1D8kQtdEilE+FboEXW3tQ3DbffHSfjAyxhN1e 2Y+5pRdupWIXDTr69SDgYGJjEfS3uxGqvhqNEQaBXFRspIfebYa+3wrYZqFFTLT6Rb3H uLNw== X-Gm-Message-State: APjAAAWNIenADLMe5AECol0tQrfNdJaZUPHtjFhqBYBvEZz4Jao8+q9j wRjLJj4mlWnGAmZvA4i+6cb/37xIpxQ= X-Received: by 2002:adf:f683:: with SMTP id v3mr2826972wrp.258.1561631294654; Thu, 27 Jun 2019 03:28:14 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.13 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:14 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 22/32] crypto: arm/ghash - provide a synchronous version Date: Thu, 27 Jun 2019 12:26:37 +0200 Message-Id: <20190627102647.2992-23-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org GHASH is used by the GCM mode, which is often used in contexts where only synchronous ciphers are permitted. So provide a synchronous version of GHASH based on the existing code. This requires a non-SIMD fallback to deal with invocations occurring from a context where SIMD instructions may not be used. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/ghash-ce-glue.c | 78 +++++++++++++------- 1 file changed, 52 insertions(+), 26 deletions(-) -- 2.20.1 diff --git a/arch/arm/crypto/ghash-ce-glue.c b/arch/arm/crypto/ghash-ce-glue.c index 39d1ccec1aab..ebb237ca874b 100644 --- a/arch/arm/crypto/ghash-ce-glue.c +++ b/arch/arm/crypto/ghash-ce-glue.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -33,6 +34,8 @@ struct ghash_key { u64 h2[2]; u64 h3[2]; u64 h4[2]; + + be128 k; }; struct ghash_desc_ctx { @@ -65,6 +68,36 @@ static int ghash_init(struct shash_desc *desc) return 0; } +static void ghash_do_update(int blocks, u64 dg[], const char *src, + struct ghash_key *key, const char *head) +{ + if (likely(crypto_simd_usable())) { + kernel_neon_begin(); + pmull_ghash_update(blocks, dg, src, key, head); + kernel_neon_end(); + } else { + be128 dst = { cpu_to_be64(dg[1]), cpu_to_be64(dg[0]) }; + + do { + const u8 *in = src; + + if (head) { + in = head; + blocks++; + head = NULL; + } else { + src += GHASH_BLOCK_SIZE; + } + + crypto_xor((u8 *)&dst, in, GHASH_BLOCK_SIZE); + gf128mul_lle(&dst, &key->k); + } while (--blocks); + + dg[0] = be64_to_cpu(dst.b); + dg[1] = be64_to_cpu(dst.a); + } +} + static int ghash_update(struct shash_desc *desc, const u8 *src, unsigned int len) { @@ -88,10 +121,8 @@ static int ghash_update(struct shash_desc *desc, const u8 *src, blocks = len / GHASH_BLOCK_SIZE; len %= GHASH_BLOCK_SIZE; - kernel_neon_begin(); - pmull_ghash_update(blocks, ctx->digest, src, key, - partial ? ctx->buf : NULL); - kernel_neon_end(); + ghash_do_update(blocks, ctx->digest, src, key, + partial ? ctx->buf : NULL); src += blocks * GHASH_BLOCK_SIZE; partial = 0; } @@ -109,9 +140,7 @@ static int ghash_final(struct shash_desc *desc, u8 *dst) struct ghash_key *key = crypto_shash_ctx(desc->tfm); memset(ctx->buf + partial, 0, GHASH_BLOCK_SIZE - partial); - kernel_neon_begin(); - pmull_ghash_update(1, ctx->digest, ctx->buf, key, NULL); - kernel_neon_end(); + ghash_do_update(1, ctx->digest, ctx->buf, key, NULL); } put_unaligned_be64(ctx->digest[1], dst); put_unaligned_be64(ctx->digest[0], dst + 8); @@ -135,24 +164,25 @@ static int ghash_setkey(struct crypto_shash *tfm, const u8 *inkey, unsigned int keylen) { struct ghash_key *key = crypto_shash_ctx(tfm); - be128 h, k; + be128 h; if (keylen != GHASH_BLOCK_SIZE) { crypto_shash_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN); return -EINVAL; } - memcpy(&k, inkey, GHASH_BLOCK_SIZE); - ghash_reflect(key->h, &k); + /* needed for the fallback */ + memcpy(&key->k, inkey, GHASH_BLOCK_SIZE); + ghash_reflect(key->h, &key->k); - h = k; - gf128mul_lle(&h, &k); + h = key->k; + gf128mul_lle(&h, &key->k); ghash_reflect(key->h2, &h); - gf128mul_lle(&h, &k); + gf128mul_lle(&h, &key->k); ghash_reflect(key->h3, &h); - gf128mul_lle(&h, &k); + gf128mul_lle(&h, &key->k); ghash_reflect(key->h4, &h); return 0; @@ -165,15 +195,13 @@ static struct shash_alg ghash_alg = { .final = ghash_final, .setkey = ghash_setkey, .descsize = sizeof(struct ghash_desc_ctx), - .base = { - .cra_name = "__ghash", - .cra_driver_name = "__driver-ghash-ce", - .cra_priority = 0, - .cra_flags = CRYPTO_ALG_INTERNAL, - .cra_blocksize = GHASH_BLOCK_SIZE, - .cra_ctxsize = sizeof(struct ghash_key), - .cra_module = THIS_MODULE, - }, + + .base.cra_name = "ghash", + .base.cra_driver_name = "ghash-ce-sync", + .base.cra_priority = 300 - 1, + .base.cra_blocksize = GHASH_BLOCK_SIZE, + .base.cra_ctxsize = sizeof(struct ghash_key), + .base.cra_module = THIS_MODULE, }; static int ghash_async_init(struct ahash_request *req) @@ -288,9 +316,7 @@ static int ghash_async_init_tfm(struct crypto_tfm *tfm) struct cryptd_ahash *cryptd_tfm; struct ghash_async_ctx *ctx = crypto_tfm_ctx(tfm); - cryptd_tfm = cryptd_alloc_ahash("__driver-ghash-ce", - CRYPTO_ALG_INTERNAL, - CRYPTO_ALG_INTERNAL); + cryptd_tfm = cryptd_alloc_ahash("ghash-ce-sync", 0, 0); if (IS_ERR(cryptd_tfm)) return PTR_ERR(cryptd_tfm); ctx->cryptd_tfm = cryptd_tfm; From patchwork Thu Jun 27 10:26:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167921 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148913ilk; Thu, 27 Jun 2019 03:28:22 -0700 (PDT) X-Google-Smtp-Source: APXvYqyYuUgUjeZeyjRvEWs6P8xlNBkMCR1hXTLEF6zV20vrw28uzOCn9eCCyoH3jqivEsK5ZLNM X-Received: by 2002:a17:90a:e397:: with SMTP id b23mr5106810pjz.140.1561631302635; Thu, 27 Jun 2019 03:28:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631302; cv=none; d=google.com; s=arc-20160816; b=UlkKNknRISVhQHVuYdazUBp5YXIYepxwGYlICl+lSvgchf16ofEfNtfqBE0mSPjvex pnwLUZN4+TWOvCCaiy47o0W4DrIeIsXd19LKFCtHZy84Ngjz0dg2go67Io54B6Frxa4G /3g5EwIO6RnifMbIBWp5OviGIRntJzUVQ0rscGpdNmtsLedwT8A2Inloh5NltGAOU47r vjqrwsHXsg97FVFZPweHA7FG9/sTfWtLN/7ul08EsVx8+tlpVBpUkVLjOGr3Mt3P+XVi pzb9ryCZbrch1R2v+djpTjG/exjMkd63ARKZPQq8t6Q3YbAG/NN9JElE1CFIPlXquWF8 G+KA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Q96/29TdV0TBH+QbXhY2WMZ256VAP+kFBl0pIhaOEg0=; b=N/vz3/e6eBPrbzy6lKiL0rBtfIRey+5haQlBbWWj+rpsRJRXxjC4xmszhRT5kOOB+U USF/7TW+Ak7PDye0lTuxPq6WA7r3m+SpuRspfa7nKmDJA4Ab23hf6nqGd4L3iG+Mp8M4 xDUxWoT2W9mm8ICRSz2EJM5j0UlNxrZuXrBJDcV4PSLgMPgedk3l/94TssMpBjPO1OOW C298UDcgM7OZ1fRIxSTS1D8TY3gnGardRZ0MKQfdllMJSBsC23oUao9P5qGQeowCZ+Yi U4ctOPy1Sf3MUXgA+Z1FMqRkdFvfDvysrhlwb9pfOIlPbTrVwCriIlLWuP/Zw1toZqJ2 t+bQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fDpDUeTh; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.22; Thu, 27 Jun 2019 03:28:22 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fDpDUeTh; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726686AbfF0K2V (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:21 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:41693 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726462AbfF0K2S (ORCPT ); Thu, 27 Jun 2019 06:28:18 -0400 Received: by mail-wr1-f65.google.com with SMTP id c2so1903412wrm.8 for ; Thu, 27 Jun 2019 03:28:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Q96/29TdV0TBH+QbXhY2WMZ256VAP+kFBl0pIhaOEg0=; b=fDpDUeThFvNCzFhp9NQqgylZHBb5OgbMoozEiZqrhAhU+yXb7B3lZOU7dXAv1SirsS X8Ln6R4nWEHMcGoIZMZs4Hij5BQK9BZdzEXPdAkJMzdnzHit/bK4O+dm74jFXxL500Lv VjEaUqwiSu+nLJFr4nyHgu6/8VaeYOtQqAigrJAB+WhwmjI84LilxAv0y8O0yIDRrdc4 PZRbMACRDPOLHXT7hvzpXPBmjUaKAx1W/1D2n7kkwI4PPgUbBBnzvXDtm/NS++1gf8PS 5a3WVBEkUHL/ACzZNQgbcYsyIGi86Qz3grASlgz+gd0Nl4ei5u05aa3zkJz2wqcnztrK 5BxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Q96/29TdV0TBH+QbXhY2WMZ256VAP+kFBl0pIhaOEg0=; b=XL52EHYASNQ03gHO78TI8UYMb02zPMrGEgjbQRX+nJe791op4y8ZHXblUxXiaUrYmd odqeGHxgI6zUL9zDvUfbgZD/VxlHTN9a5piWCrVgcZ0BKCVSRPEesIB9kz0NugiPpp6J 7XgCKH7qgIrCzIeVfWLCHXXAxgKKCwh4idkn+azj3sslQ8eQeV84fCUnHKsPk8aWNj9c QmVz2E/yxVQ92hrA4P5kCTa7UxGb0B+1WOn+BldKlFqjuyEvJxeFtcbOFTVd8S6PdluX OkdMA1RwfOGcZ7MtyRMPQZWaNBIJV9TdyXx9wtc+JQSplksrg6dVM63tkmVffjF+bujT fXvQ== X-Gm-Message-State: APjAAAU1SOaFROjzC789J5IbzktYsJRWlXOgNsdWf1kPKmpLlgZljsJO ZhORCuAB83S4zkZ//T5nkEi8y3OWtog= X-Received: by 2002:adf:82e2:: with SMTP id 89mr2753495wrc.33.1561631295611; Thu, 27 Jun 2019 03:28:15 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.14 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:14 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 23/32] bluetooth: switch to AES library Date: Thu, 27 Jun 2019 12:26:38 +0200 Message-Id: <20190627102647.2992-24-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The bluetooth code uses a bare AES cipher for the encryption operations. Given that it carries out a set_key() operation right before every encryption operation, this is clearly not a hot path, and so the use of the cipher interface (which provides the best implementation available on the system) is not really required. In fact, when using a cipher like AES-NI or AES-CE, both the set_key() and the encrypt() operations involve en/disabling preemption as well as stacking and unstacking the SIMD context, and this is most certainly not worth it for encrypting 16 bytes of data. So let's switch to the new lightweight library interface instead. Signed-off-by: Ard Biesheuvel --- net/bluetooth/Kconfig | 3 +- net/bluetooth/smp.c | 103 ++++++-------------- 2 files changed, 33 insertions(+), 73 deletions(-) -- 2.20.1 diff --git a/net/bluetooth/Kconfig b/net/bluetooth/Kconfig index db82a40875e8..a9d83ec4ee33 100644 --- a/net/bluetooth/Kconfig +++ b/net/bluetooth/Kconfig @@ -9,7 +9,8 @@ menuconfig BT select CRC16 select CRYPTO select CRYPTO_BLKCIPHER - select CRYPTO_AES + select CRYPTO_LIB_AES + imply CRYPTO_AES select CRYPTO_CMAC select CRYPTO_ECB select CRYPTO_SHA256 diff --git a/net/bluetooth/smp.c b/net/bluetooth/smp.c index e68c715f8d37..b5045b57ead3 100644 --- a/net/bluetooth/smp.c +++ b/net/bluetooth/smp.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -88,7 +89,6 @@ struct smp_dev { u8 local_rand[16]; bool debug_key; - struct crypto_cipher *tfm_aes; struct crypto_shash *tfm_cmac; struct crypto_kpp *tfm_ecdh; }; @@ -127,7 +127,6 @@ struct smp_chan { u8 dhkey[32]; u8 mackey[16]; - struct crypto_cipher *tfm_aes; struct crypto_shash *tfm_cmac; struct crypto_kpp *tfm_ecdh; }; @@ -377,22 +376,18 @@ static int smp_h7(struct crypto_shash *tfm_cmac, const u8 w[16], * s1 and ah. */ -static int smp_e(struct crypto_cipher *tfm, const u8 *k, u8 *r) +static int smp_e(const u8 *k, u8 *r) { + struct crypto_aes_ctx ctx; uint8_t tmp[16], data[16]; int err; SMP_DBG("k %16phN r %16phN", k, r); - if (!tfm) { - BT_ERR("tfm %p", tfm); - return -EINVAL; - } - /* The most significant octet of key corresponds to k[0] */ swap_buf(k, tmp, 16); - err = crypto_cipher_setkey(tfm, tmp, 16); + err = aes_expandkey(&ctx, tmp, 16); if (err) { BT_ERR("cipher setkey failed: %d", err); return err; @@ -401,17 +396,18 @@ static int smp_e(struct crypto_cipher *tfm, const u8 *k, u8 *r) /* Most significant octet of plaintextData corresponds to data[0] */ swap_buf(r, data, 16); - crypto_cipher_encrypt_one(tfm, data, data); + aes_encrypt(&ctx, data, data); /* Most significant octet of encryptedData corresponds to data[0] */ swap_buf(data, r, 16); SMP_DBG("r %16phN", r); + memzero_explicit(&ctx, sizeof (ctx)); return err; } -static int smp_c1(struct crypto_cipher *tfm_aes, const u8 k[16], +static int smp_c1(const u8 k[16], const u8 r[16], const u8 preq[7], const u8 pres[7], u8 _iat, const bdaddr_t *ia, u8 _rat, const bdaddr_t *ra, u8 res[16]) { @@ -436,7 +432,7 @@ static int smp_c1(struct crypto_cipher *tfm_aes, const u8 k[16], u128_xor((u128 *) res, (u128 *) r, (u128 *) p1); /* res = e(k, res) */ - err = smp_e(tfm_aes, k, res); + err = smp_e(k, res); if (err) { BT_ERR("Encrypt data error"); return err; @@ -453,14 +449,14 @@ static int smp_c1(struct crypto_cipher *tfm_aes, const u8 k[16], u128_xor((u128 *) res, (u128 *) res, (u128 *) p2); /* res = e(k, res) */ - err = smp_e(tfm_aes, k, res); + err = smp_e(k, res); if (err) BT_ERR("Encrypt data error"); return err; } -static int smp_s1(struct crypto_cipher *tfm_aes, const u8 k[16], +static int smp_s1(const u8 k[16], const u8 r1[16], const u8 r2[16], u8 _r[16]) { int err; @@ -469,15 +465,14 @@ static int smp_s1(struct crypto_cipher *tfm_aes, const u8 k[16], memcpy(_r, r2, 8); memcpy(_r + 8, r1, 8); - err = smp_e(tfm_aes, k, _r); + err = smp_e(k, _r); if (err) BT_ERR("Encrypt data error"); return err; } -static int smp_ah(struct crypto_cipher *tfm, const u8 irk[16], - const u8 r[3], u8 res[3]) +static int smp_ah(const u8 irk[16], const u8 r[3], u8 res[3]) { u8 _res[16]; int err; @@ -486,7 +481,7 @@ static int smp_ah(struct crypto_cipher *tfm, const u8 irk[16], memcpy(_res, r, 3); memset(_res + 3, 0, 13); - err = smp_e(tfm, irk, _res); + err = smp_e(irk, _res); if (err) { BT_ERR("Encrypt error"); return err; @@ -518,7 +513,7 @@ bool smp_irk_matches(struct hci_dev *hdev, const u8 irk[16], BT_DBG("RPA %pMR IRK %*phN", bdaddr, 16, irk); - err = smp_ah(smp->tfm_aes, irk, &bdaddr->b[3], hash); + err = smp_ah(irk, &bdaddr->b[3], hash); if (err) return false; @@ -541,7 +536,7 @@ int smp_generate_rpa(struct hci_dev *hdev, const u8 irk[16], bdaddr_t *rpa) rpa->b[5] &= 0x3f; /* Clear two most significant bits */ rpa->b[5] |= 0x40; /* Set second most significant bit */ - err = smp_ah(smp->tfm_aes, irk, &rpa->b[3], rpa->b); + err = smp_ah(irk, &rpa->b[3], rpa->b); if (err < 0) return err; @@ -768,7 +763,6 @@ static void smp_chan_destroy(struct l2cap_conn *conn) kzfree(smp->slave_csrk); kzfree(smp->link_key); - crypto_free_cipher(smp->tfm_aes); crypto_free_shash(smp->tfm_cmac); crypto_free_kpp(smp->tfm_ecdh); @@ -957,7 +951,7 @@ static u8 smp_confirm(struct smp_chan *smp) BT_DBG("conn %p", conn); - ret = smp_c1(smp->tfm_aes, smp->tk, smp->prnd, smp->preq, smp->prsp, + ret = smp_c1(smp->tk, smp->prnd, smp->preq, smp->prsp, conn->hcon->init_addr_type, &conn->hcon->init_addr, conn->hcon->resp_addr_type, &conn->hcon->resp_addr, cp.confirm_val); @@ -983,12 +977,9 @@ static u8 smp_random(struct smp_chan *smp) u8 confirm[16]; int ret; - if (IS_ERR_OR_NULL(smp->tfm_aes)) - return SMP_UNSPECIFIED; - BT_DBG("conn %p %s", conn, conn->hcon->out ? "master" : "slave"); - ret = smp_c1(smp->tfm_aes, smp->tk, smp->rrnd, smp->preq, smp->prsp, + ret = smp_c1(smp->tk, smp->rrnd, smp->preq, smp->prsp, hcon->init_addr_type, &hcon->init_addr, hcon->resp_addr_type, &hcon->resp_addr, confirm); if (ret) @@ -1005,7 +996,7 @@ static u8 smp_random(struct smp_chan *smp) __le64 rand = 0; __le16 ediv = 0; - smp_s1(smp->tfm_aes, smp->tk, smp->rrnd, smp->prnd, stk); + smp_s1(smp->tk, smp->rrnd, smp->prnd, stk); if (test_and_set_bit(HCI_CONN_ENCRYPT_PEND, &hcon->flags)) return SMP_UNSPECIFIED; @@ -1021,7 +1012,7 @@ static u8 smp_random(struct smp_chan *smp) smp_send_cmd(conn, SMP_CMD_PAIRING_RANDOM, sizeof(smp->prnd), smp->prnd); - smp_s1(smp->tfm_aes, smp->tk, smp->prnd, smp->rrnd, stk); + smp_s1(smp->tk, smp->prnd, smp->rrnd, stk); if (hcon->pending_sec_level == BT_SECURITY_HIGH) auth = 1; @@ -1389,16 +1380,10 @@ static struct smp_chan *smp_chan_create(struct l2cap_conn *conn) if (!smp) return NULL; - smp->tfm_aes = crypto_alloc_cipher("aes", 0, 0); - if (IS_ERR(smp->tfm_aes)) { - BT_ERR("Unable to create AES crypto context"); - goto zfree_smp; - } - smp->tfm_cmac = crypto_alloc_shash("cmac(aes)", 0, 0); if (IS_ERR(smp->tfm_cmac)) { BT_ERR("Unable to create CMAC crypto context"); - goto free_cipher; + goto zfree_smp; } smp->tfm_ecdh = crypto_alloc_kpp("ecdh", CRYPTO_ALG_INTERNAL, 0); @@ -1420,8 +1405,6 @@ static struct smp_chan *smp_chan_create(struct l2cap_conn *conn) free_shash: crypto_free_shash(smp->tfm_cmac); -free_cipher: - crypto_free_cipher(smp->tfm_aes); zfree_smp: kzfree(smp); return NULL; @@ -3219,7 +3202,6 @@ static struct l2cap_chan *smp_add_cid(struct hci_dev *hdev, u16 cid) { struct l2cap_chan *chan; struct smp_dev *smp; - struct crypto_cipher *tfm_aes; struct crypto_shash *tfm_cmac; struct crypto_kpp *tfm_ecdh; @@ -3232,17 +3214,9 @@ static struct l2cap_chan *smp_add_cid(struct hci_dev *hdev, u16 cid) if (!smp) return ERR_PTR(-ENOMEM); - tfm_aes = crypto_alloc_cipher("aes", 0, 0); - if (IS_ERR(tfm_aes)) { - BT_ERR("Unable to create AES crypto context"); - kzfree(smp); - return ERR_CAST(tfm_aes); - } - tfm_cmac = crypto_alloc_shash("cmac(aes)", 0, 0); if (IS_ERR(tfm_cmac)) { BT_ERR("Unable to create CMAC crypto context"); - crypto_free_cipher(tfm_aes); kzfree(smp); return ERR_CAST(tfm_cmac); } @@ -3251,13 +3225,11 @@ static struct l2cap_chan *smp_add_cid(struct hci_dev *hdev, u16 cid) if (IS_ERR(tfm_ecdh)) { BT_ERR("Unable to create ECDH crypto context"); crypto_free_shash(tfm_cmac); - crypto_free_cipher(tfm_aes); kzfree(smp); return ERR_CAST(tfm_ecdh); } smp->local_oob = false; - smp->tfm_aes = tfm_aes; smp->tfm_cmac = tfm_cmac; smp->tfm_ecdh = tfm_ecdh; @@ -3265,7 +3237,6 @@ static struct l2cap_chan *smp_add_cid(struct hci_dev *hdev, u16 cid) chan = l2cap_chan_create(); if (!chan) { if (smp) { - crypto_free_cipher(smp->tfm_aes); crypto_free_shash(smp->tfm_cmac); crypto_free_kpp(smp->tfm_ecdh); kzfree(smp); @@ -3313,7 +3284,6 @@ static void smp_del_chan(struct l2cap_chan *chan) smp = chan->data; if (smp) { chan->data = NULL; - crypto_free_cipher(smp->tfm_aes); crypto_free_shash(smp->tfm_cmac); crypto_free_kpp(smp->tfm_ecdh); kzfree(smp); @@ -3569,7 +3539,7 @@ static int __init test_debug_key(struct crypto_kpp *tfm_ecdh) return 0; } -static int __init test_ah(struct crypto_cipher *tfm_aes) +static int __init test_ah(void) { const u8 irk[16] = { 0x9b, 0x7d, 0x39, 0x0a, 0xa6, 0x10, 0x10, 0x34, @@ -3579,7 +3549,7 @@ static int __init test_ah(struct crypto_cipher *tfm_aes) u8 res[3]; int err; - err = smp_ah(tfm_aes, irk, r, res); + err = smp_ah(irk, r, res); if (err) return err; @@ -3589,7 +3559,7 @@ static int __init test_ah(struct crypto_cipher *tfm_aes) return 0; } -static int __init test_c1(struct crypto_cipher *tfm_aes) +static int __init test_c1(void) { const u8 k[16] = { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, @@ -3609,7 +3579,7 @@ static int __init test_c1(struct crypto_cipher *tfm_aes) u8 res[16]; int err; - err = smp_c1(tfm_aes, k, r, preq, pres, _iat, &ia, _rat, &ra, res); + err = smp_c1(k, r, preq, pres, _iat, &ia, _rat, &ra, res); if (err) return err; @@ -3619,7 +3589,7 @@ static int __init test_c1(struct crypto_cipher *tfm_aes) return 0; } -static int __init test_s1(struct crypto_cipher *tfm_aes) +static int __init test_s1(void) { const u8 k[16] = { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, @@ -3634,7 +3604,7 @@ static int __init test_s1(struct crypto_cipher *tfm_aes) u8 res[16]; int err; - err = smp_s1(tfm_aes, k, r1, r2, res); + err = smp_s1(k, r1, r2, res); if (err) return err; @@ -3815,8 +3785,7 @@ static const struct file_operations test_smp_fops = { .llseek = default_llseek, }; -static int __init run_selftests(struct crypto_cipher *tfm_aes, - struct crypto_shash *tfm_cmac, +static int __init run_selftests(struct crypto_shash *tfm_cmac, struct crypto_kpp *tfm_ecdh) { ktime_t calltime, delta, rettime; @@ -3831,19 +3800,19 @@ static int __init run_selftests(struct crypto_cipher *tfm_aes, goto done; } - err = test_ah(tfm_aes); + err = test_ah(); if (err) { BT_ERR("smp_ah test failed"); goto done; } - err = test_c1(tfm_aes); + err = test_c1(); if (err) { BT_ERR("smp_c1 test failed"); goto done; } - err = test_s1(tfm_aes); + err = test_s1(); if (err) { BT_ERR("smp_s1 test failed"); goto done; @@ -3900,21 +3869,13 @@ static int __init run_selftests(struct crypto_cipher *tfm_aes, int __init bt_selftest_smp(void) { - struct crypto_cipher *tfm_aes; struct crypto_shash *tfm_cmac; struct crypto_kpp *tfm_ecdh; int err; - tfm_aes = crypto_alloc_cipher("aes", 0, 0); - if (IS_ERR(tfm_aes)) { - BT_ERR("Unable to create AES crypto context"); - return PTR_ERR(tfm_aes); - } - tfm_cmac = crypto_alloc_shash("cmac(aes)", 0, 0); if (IS_ERR(tfm_cmac)) { BT_ERR("Unable to create CMAC crypto context"); - crypto_free_cipher(tfm_aes); return PTR_ERR(tfm_cmac); } @@ -3922,14 +3883,12 @@ int __init bt_selftest_smp(void) if (IS_ERR(tfm_ecdh)) { BT_ERR("Unable to create ECDH crypto context"); crypto_free_shash(tfm_cmac); - crypto_free_cipher(tfm_aes); return PTR_ERR(tfm_ecdh); } - err = run_selftests(tfm_aes, tfm_cmac, tfm_ecdh); + err = run_selftests(tfm_cmac, tfm_ecdh); crypto_free_shash(tfm_cmac); - crypto_free_cipher(tfm_aes); crypto_free_kpp(tfm_ecdh); return err; From patchwork Thu Jun 27 10:26:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167922 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148923ilk; Thu, 27 Jun 2019 03:28:23 -0700 (PDT) X-Google-Smtp-Source: APXvYqy80FtLQFt84+xSKSIBOluLY3Vnk1f3MiOIwOeGNZDJjGN3YZNpfzfQyA5Xu0UUVSUZy283 X-Received: by 2002:a17:902:aa88:: with SMTP id d8mr3637421plr.274.1561631303367; Thu, 27 Jun 2019 03:28:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631303; cv=none; d=google.com; s=arc-20160816; b=TalZPbgDheQaLmwBYjQPHRuh3g3DAlYjPjLkmK+s4O/VUQNymaDxZB4CLyuCyp++25 jVY6byzx3+8tBYcdRGJDHXnRr/Q4DNQGe1oJo0wgWV2H8/neBNQQmzyPJRC3e/xLDpvQ 8TqL2G71TQd6/pbvCLXQogbDvQkT62pD7SEEdrjwMNpIT0MtBYMNaipz9d3ojUgtU57m OFkxHx5gxuITiyYy+dNUI/nAmzhkbuY1TRHimbklz6zflNO66mKlBkkkPlz+beVNVgrw a1jSW/AnYnf9jWZwbR1iQiSyUSlppuV3PVzdine9IaX9pRFwWXxEQrab79J5m80Ae/me dCcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=h9xdExBrqYFWsuLQCJksPQkQTr4z/marqcQjFd4StkE=; b=EiGH+ujNClG4afBJ56eTY+cdUXx+KmKO52O4BbIZZPJSVvPi7Ib7wzbmzp9eji+wY3 WV2HPgPPnnhulVai7v01JWnzaj1ojKllkZjNzRkMO+BIQxdcdqZee3JsEAHUZ8HtSS7k LCARQ2U9cbaZyufIPwpEsiM+7RFBRSZcEqavmfNEVvW+sQAkOpl0XtFwRElfTk2Xzgzc VG8rcZ/qAGHxMfEXETGLsLOE4emS0PXjT/IGk9l8ID1FwS8J+PSFxTiIJMgrH4dx+P54 28E9IBfPK93t66p7lm2TItqJGZ/TjoGm9fJxHXa5SQzogNl2q0OmkQ372oH9nUkDC886 6Ebw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bY+OLwCR; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.23; Thu, 27 Jun 2019 03:28:23 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bY+OLwCR; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726462AbfF0K2V (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:21 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:38343 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726682AbfF0K2S (ORCPT ); Thu, 27 Jun 2019 06:28:18 -0400 Received: by mail-wr1-f68.google.com with SMTP id d18so1923029wrs.5 for ; Thu, 27 Jun 2019 03:28:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=h9xdExBrqYFWsuLQCJksPQkQTr4z/marqcQjFd4StkE=; b=bY+OLwCRpBmnowhgehO5dqYTwvrIIXAtSeaPTWQypdaJtfTLWdzos7Shnt5ZWRSLT3 HqUEYEnmcXzXLOVxxOdef9ETkAF3VOxK0o697B90/JL4GyxGDQ86vPY4707hnY+TFH1D mx6cvKlM2dySCzI2ilc0FTXHPzkjnB1A/cjNoe/emKsOYcGDzhtPLDXm4jmLc89LRcvB hqh9cbVU7QEdJtC7998ZiFohjsFGLpa2rEr9sRYe/LhXiXwvKaKgK3CqpCEqSdFln3ht l3mQWZ+dfLWGkGNUuUlexCwMTx6S/QEclc1Sm/hMKlx9b4LatfP73dNpY2PkvA1m7qYN RVFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=h9xdExBrqYFWsuLQCJksPQkQTr4z/marqcQjFd4StkE=; b=EsWnWoNC+5VfjpZUUKEjePKV2y5/g1vYi1u0E4yfVDStkoObAX75YLq3VCW9LUvuSr 6A6QZrBnG5n44PswheGYTWMDivpsPKLCNBwYM5Z5+AEaRafwGidP5yiCQv4FX7/7Oz6j 9G1lxzTtIbcvrn1Firr9FxP7jAMwI0WXIlKRCfcfVEB0BNS32hPN/A54q/uRudOYf0XH znP7jXj+JjBpzk2RBVeKSr41zpgP7r7VK/dW1H2AgKtUMgCmAvReWx5pNcBtvtoaGiIY KaKhNR/fdk9BL0sXzEl3oUq8GTYcArBtWIdFxd9lLX6iM51EphD05xJQiNDPbFa6BnGk fmTg== X-Gm-Message-State: APjAAAWXp7gL/HEOdltPaGWTuhNpHO9ey3C+yKCpJ7xE8YHObGK5GTc8 kkP/gSFw9pZQm/kbes6E0ecwyzo1Slk= X-Received: by 2002:a5d:4647:: with SMTP id j7mr2804360wrs.334.1561631296694; Thu, 27 Jun 2019 03:28:16 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.15 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:16 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 24/32] crypto: amcc/aes - switch to AES library for GCM key derivation Date: Thu, 27 Jun 2019 12:26:39 +0200 Message-Id: <20190627102647.2992-25-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The AMCC code for GCM key derivation allocates a AES cipher to perform a single block encryption. So let's switch to the new and more lightweight AES library instead. Signed-off-by: Ard Biesheuvel --- drivers/crypto/Kconfig | 2 +- drivers/crypto/amcc/crypto4xx_alg.c | 24 +++++++------------- 2 files changed, 9 insertions(+), 17 deletions(-) -- 2.20.1 diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig index b30b84089d11..c7ac1e6d23d4 100644 --- a/drivers/crypto/Kconfig +++ b/drivers/crypto/Kconfig @@ -311,7 +311,7 @@ config CRYPTO_DEV_PPC4XX depends on PPC && 4xx select CRYPTO_HASH select CRYPTO_AEAD - select CRYPTO_AES + select CRYPTO_LIB_AES select CRYPTO_CCM select CRYPTO_CTR select CRYPTO_GCM diff --git a/drivers/crypto/amcc/crypto4xx_alg.c b/drivers/crypto/amcc/crypto4xx_alg.c index 26f86fd7532b..d3660703a36c 100644 --- a/drivers/crypto/amcc/crypto4xx_alg.c +++ b/drivers/crypto/amcc/crypto4xx_alg.c @@ -536,28 +536,20 @@ static int crypto4xx_aes_gcm_validate_keylen(unsigned int keylen) static int crypto4xx_compute_gcm_hash_key_sw(__le32 *hash_start, const u8 *key, unsigned int keylen) { - struct crypto_cipher *aes_tfm = NULL; + struct crypto_aes_ctx ctx; uint8_t src[16] = { 0 }; - int rc = 0; - - aes_tfm = crypto_alloc_cipher("aes", 0, CRYPTO_ALG_NEED_FALLBACK); - if (IS_ERR(aes_tfm)) { - rc = PTR_ERR(aes_tfm); - pr_warn("could not load aes cipher driver: %d\n", rc); - return rc; - } + int rc; - rc = crypto_cipher_setkey(aes_tfm, key, keylen); + rc = aes_expandkey(&ctx, key, keylen); if (rc) { - pr_err("setkey() failed: %d\n", rc); - goto out; + pr_err("aes_expandkey() failed: %d\n", rc); + return rc; } - crypto_cipher_encrypt_one(aes_tfm, src, src); + aes_encrypt(&ctx, src, src); crypto4xx_memcpy_to_le32(hash_start, src, 16); -out: - crypto_free_cipher(aes_tfm); - return rc; + memzero_explicit(&ctx, sizeof(ctx)); + return 0; } int crypto4xx_setkey_aes_gcm(struct crypto_aead *cipher, From patchwork Thu Jun 27 10:26:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167920 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148907ilk; Thu, 27 Jun 2019 03:28:22 -0700 (PDT) X-Google-Smtp-Source: APXvYqzEXlmUilUh6N5GIiVe2jjXYVr7BmZvSDRqh2ApFF5GsOjjHLcSBAkpJTWX7tKoGX9aUdMA X-Received: by 2002:a65:63cd:: with SMTP id n13mr3056793pgv.153.1561631302230; Thu, 27 Jun 2019 03:28:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631302; cv=none; d=google.com; s=arc-20160816; b=0hHWzQTqEIkMEQdYSKS9RyPg4mWl7wTCSJAVl/r7KPLfUZKGjv1j5uc/3BmXVJo9yL pSYDFt5F1aDzDloevDE5HfSNX8PF6o6ErZYqe3pMDQsV/A+yXGNNOyKLKmlgcAgCuvxv qhNxnbLVKNn0Dc68rKTiMl/yfUoYL5cla6ERyxBtDOwebgc6/JDfSIv4qpkv97fXilEd DalGgiCbduxNK+FOP3q+sqaEDQZHcgy5LLi7vPiXBvx3H6tRC/dCYiRWqQzZUAXn1EyW yBOoNl33A+FDF/Dy/GELpM1UIWmxUabU09cCW8vPyYCHCkZ+gbOA5qTJvJaFHoahRNDh c5aQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Fm4ramXN1vd7JQGliUbSFSJzkgKkgIE9Zzs9yUh/C1k=; b=VOaO878isiMBdhywo2ps17hLgLshYB7NIiJktWbJmYgAWOQgItSgJ5C24BVZwVV2OG YSajCIDTeu1kcPff/Own+VkXeW7Zj4y2BhAtMqnZH60ypvpcoMS1dYWtfL/Ocy7S92uC HkeYFAHXTnF0wJbymtSpMXrPHoZjbfsvT8Rohv1m5GKF+b/qIP6v4p3BtPTkEyDEBpKO BBdT0ekJv1PbokNTL3DFu6gmY8JRbhkUawOm+F18f7dfw5hASG7MGq9NhQVcCaAIMpXg 8R4nKQZi/DbB5AoZkcgS/9r4C4FlfOkwcuY6CQzF9DL3XN4kSjoFocfjuqfO71ludHuW rT+Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fFy98Taa; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.22; Thu, 27 Jun 2019 03:28:22 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fFy98Taa; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726523AbfF0K2V (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:21 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:33587 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726465AbfF0K2U (ORCPT ); Thu, 27 Jun 2019 06:28:20 -0400 Received: by mail-wr1-f65.google.com with SMTP id n9so1950597wru.0 for ; Thu, 27 Jun 2019 03:28:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Fm4ramXN1vd7JQGliUbSFSJzkgKkgIE9Zzs9yUh/C1k=; b=fFy98Taa+uZKo/+CTlhAZt+/oqgxSi4LEvHwozvTfwz0O52XhYxDVGCK9x7vTuxtya 3OSxoMTU+XoxQPloPMPjU+bI2a+eRaXSjcJljiz4fmdhIQtBNXAfab6ifcd5053CudR7 xSADxmsHtGXQQ6I4ek03F5YqGAQAIceFvKDqP903Jfm2DpG8zmQJnh97bP2hH/LdUDOJ JOKtztYKLXZ0uRg3jSpxxEIOjaXp3cOqVlGo7+B3dlGOP1kbpswp6qcInrgiN5ZjLQ4q eEsEStLmi6rxDudRziw1UJoqeoy4+IMvIXFOQGgJoYzrSTLGoD8AG8fSrSt+ZfvoILK8 llyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Fm4ramXN1vd7JQGliUbSFSJzkgKkgIE9Zzs9yUh/C1k=; b=erwf5oAhgkD6dCpasNihVntF3X+z6/wH2IOH1NZInNYsKWynukPSFKB1e5I6YWWMjo NlpqQ2wGJtqnxs88t11a77+3qb0V3CLWzEDNQGw306L/MI4o5uuLkVe7yvmsl6UlsLAd 8aedfGIJ8qNuF7w1pz43WRLApOhPY802n3GyI1TTlDYYlnn1OjaWEPd1wf9JmMTGUFNx WpbWyMTZFlBSuO5EykXZYw8rnHwDTU8YKtsDWry0vtlsSabrwJUPbiqgFM7Mh8qFHs2B t+xhVBi23LW338G9M+1xutRoQlu+4iCrt+v3wQEgNu0wSPsqRzc8g2RiBVTeedUtKbGS yEVQ== X-Gm-Message-State: APjAAAW8cJIZUhbvRK33FteaotevFM3Ltym4EXhAT1qV4MWQee7+2ISF EmxZJmaX+l07bJyFeYGwD/EmqJyhB/A= X-Received: by 2002:adf:c654:: with SMTP id u20mr2784397wrg.271.1561631297735; Thu, 27 Jun 2019 03:28:17 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.16 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:17 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 25/32] crypto: ccp - move to AES library for CMAC key derivation Date: Thu, 27 Jun 2019 12:26:40 +0200 Message-Id: <20190627102647.2992-26-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Use the AES library instead of the cipher interface to perform the single block of AES processing involved in updating the key of the cmac(aes) hash. Signed-off-by: Ard Biesheuvel --- drivers/crypto/ccp/Kconfig | 1 + drivers/crypto/ccp/ccp-crypto-aes-cmac.c | 25 ++++---------------- drivers/crypto/ccp/ccp-crypto.h | 3 --- 3 files changed, 5 insertions(+), 24 deletions(-) -- 2.20.1 diff --git a/drivers/crypto/ccp/Kconfig b/drivers/crypto/ccp/Kconfig index b9dfae47aefd..ee06d0fccdb5 100644 --- a/drivers/crypto/ccp/Kconfig +++ b/drivers/crypto/ccp/Kconfig @@ -29,6 +29,7 @@ config CRYPTO_DEV_CCP_CRYPTO select CRYPTO_BLKCIPHER select CRYPTO_AUTHENC select CRYPTO_RSA + select CRYPTO_LIB_AES help Support for using the cryptographic API with the AMD Cryptographic Coprocessor. This module supports offload of SHA and AES algorithms. diff --git a/drivers/crypto/ccp/ccp-crypto-aes-cmac.c b/drivers/crypto/ccp/ccp-crypto-aes-cmac.c index f6e252c1d6fb..c8f4b29bf044 100644 --- a/drivers/crypto/ccp/ccp-crypto-aes-cmac.c +++ b/drivers/crypto/ccp/ccp-crypto-aes-cmac.c @@ -264,6 +264,7 @@ static int ccp_aes_cmac_setkey(struct crypto_ahash *tfm, const u8 *key, ccp_crypto_ahash_alg(crypto_ahash_tfm(tfm)); u64 k0_hi, k0_lo, k1_hi, k1_lo, k2_hi, k2_lo; u64 rb_hi = 0x00, rb_lo = 0x87; + struct crypto_aes_ctx aes; __be64 *gk; int ret; @@ -287,14 +288,14 @@ static int ccp_aes_cmac_setkey(struct crypto_ahash *tfm, const u8 *key, ctx->u.aes.key_len = 0; /* Set the key for the AES cipher used to generate the keys */ - ret = crypto_cipher_setkey(ctx->u.aes.tfm_cipher, key, key_len); + ret = aes_expandkey(&aes, key, key_len); if (ret) return ret; /* Encrypt a block of zeroes - use key area in context */ memset(ctx->u.aes.key, 0, sizeof(ctx->u.aes.key)); - crypto_cipher_encrypt_one(ctx->u.aes.tfm_cipher, ctx->u.aes.key, - ctx->u.aes.key); + aes_encrypt(&aes, ctx->u.aes.key, ctx->u.aes.key); + memzero_explicit(&aes, sizeof(aes)); /* Generate K1 and K2 */ k0_hi = be64_to_cpu(*((__be64 *)ctx->u.aes.key)); @@ -339,32 +340,15 @@ static int ccp_aes_cmac_cra_init(struct crypto_tfm *tfm) { struct ccp_ctx *ctx = crypto_tfm_ctx(tfm); struct crypto_ahash *ahash = __crypto_ahash_cast(tfm); - struct crypto_cipher *cipher_tfm; ctx->complete = ccp_aes_cmac_complete; ctx->u.aes.key_len = 0; crypto_ahash_set_reqsize(ahash, sizeof(struct ccp_aes_cmac_req_ctx)); - cipher_tfm = crypto_alloc_cipher("aes", 0, CRYPTO_ALG_NEED_FALLBACK); - if (IS_ERR(cipher_tfm)) { - pr_warn("could not load aes cipher driver\n"); - return PTR_ERR(cipher_tfm); - } - ctx->u.aes.tfm_cipher = cipher_tfm; - return 0; } -static void ccp_aes_cmac_cra_exit(struct crypto_tfm *tfm) -{ - struct ccp_ctx *ctx = crypto_tfm_ctx(tfm); - - if (ctx->u.aes.tfm_cipher) - crypto_free_cipher(ctx->u.aes.tfm_cipher); - ctx->u.aes.tfm_cipher = NULL; -} - int ccp_register_aes_cmac_algs(struct list_head *head) { struct ccp_crypto_ahash_alg *ccp_alg; @@ -404,7 +388,6 @@ int ccp_register_aes_cmac_algs(struct list_head *head) base->cra_ctxsize = sizeof(struct ccp_ctx); base->cra_priority = CCP_CRA_PRIORITY; base->cra_init = ccp_aes_cmac_cra_init; - base->cra_exit = ccp_aes_cmac_cra_exit; base->cra_module = THIS_MODULE; ret = crypto_register_ahash(alg); diff --git a/drivers/crypto/ccp/ccp-crypto.h b/drivers/crypto/ccp/ccp-crypto.h index 28819e11db96..9100df77a7b3 100644 --- a/drivers/crypto/ccp/ccp-crypto.h +++ b/drivers/crypto/ccp/ccp-crypto.h @@ -90,9 +90,6 @@ struct ccp_aes_ctx { /* Fallback cipher for XTS with unsupported unit sizes */ struct crypto_sync_skcipher *tfm_skcipher; - /* Cipher used to generate CMAC K1/K2 keys */ - struct crypto_cipher *tfm_cipher; - enum ccp_engine engine; enum ccp_aes_type type; enum ccp_aes_mode mode; From patchwork Thu Jun 27 10:26:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167923 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148917ilk; Thu, 27 Jun 2019 03:28:23 -0700 (PDT) X-Google-Smtp-Source: APXvYqyZk9ixK2mN5O1o0pWhF2kwvd24gqHnRIbfs0grPpSCjziXTCadP20jhUIDQ4FXLPnGbi8M X-Received: by 2002:a17:902:7583:: with SMTP id j3mr3812330pll.196.1561631303080; Thu, 27 Jun 2019 03:28:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631303; cv=none; d=google.com; s=arc-20160816; b=IAHxJDHUdWqYby/CZtYKDexcCM9PLkKAp/omEN+GyAFv3ehVI+wjPRl6U44dzkVgMH 213JtKQ/QrZ+pDY4yE5KdlTpScXRQYs0x3WQCohIwoaxYnkN/vGSTRONAGzjv6+qbeb6 KXgM/MwrHO2GEdo61NWw/ERC8/SV5IkmyLa1GGm3VEJae4IDdSbSdJoq7KFisfITQEgl S8jSuIh3yLdR1zyAWX1Kohh2SpEIByst/Axm0kfegDbReeM15GRC+rXnuDQNXBJr39Y/ YjP9a9DwQHotfKv/xDr1wNeZOJRoUReGolrCAsAYxJraTa30gFO0X5/kOOvBusdpbKGC HKIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=SI+i3SEIksVLr/oeokNkhVP46cu/BheSlfNWTBwHojU=; b=Ny71u5XNQ5O5ezK4lSmX7WugnKvUjsdABT07Xhbhy0Pf+euUW/TZJvZfgxHpQm/cFS O+83muvr9dcPemDMLVaSC0Q5MGCXo55xDBJb5E5ZUsYGcIgPaGobjV4gheiNHs6Jt+b9 pLHwsKBIFkXJEl/IMbKceRzr+iPfPGrjTOvRwp7MNnzg8etFXnIAOShggtl+ck0SfEWf eSONvArCz814lf+4eSnvZsuImT47RmB55MKfiyBb1k6+qsPdldXBt0YevNi90sLXN24b QJBhU2fIji+8Y9bPNMHOS5Q3/B+Fg0yB5oenPZ5L1z7aDcBevZkX4wQupKnOQ8VRVN87 F5eQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YlrnsHIG; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.22; Thu, 27 Jun 2019 03:28:23 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YlrnsHIG; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726465AbfF0K2W (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:22 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:33862 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726623AbfF0K2V (ORCPT ); Thu, 27 Jun 2019 06:28:21 -0400 Received: by mail-wr1-f68.google.com with SMTP id k11so1941343wrl.1 for ; Thu, 27 Jun 2019 03:28:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SI+i3SEIksVLr/oeokNkhVP46cu/BheSlfNWTBwHojU=; b=YlrnsHIGinzZy0k+MYE7NtYe+nhsMUqfbjKt5N2RkCbkWb4iDdr98mmEpHvrTZpd/9 Tt9KNknJ1853cgPae5Aipruh7cClUth2tF1isvYt50KVV2xisZhz7M2/wMDYetWMagxS 4IBP46vwDnG6pZNKinGQ5FFjAJkYORLu0rHbAC+854Obb4F9pNOhXxdeRExMHu6DApUQ CnxxJJJcEANcdPoD4jI41zEh+sk3F0OYmKHWamKeR8I3EF/juq2mqDZ6rxpIlwylM/Cc KCo7NIK01uQdbT8jQrfzipd6GKHOLnD+W58/fSrWr/3WTj7ZsUI/nDOGuaDlzF3Wrbyi POZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SI+i3SEIksVLr/oeokNkhVP46cu/BheSlfNWTBwHojU=; b=p/PRfEwdhideLM/KwuNBtlKmNRdyfzZNDUp4YunpIT61CAT888S5Y/625+y/B6zzOA l8hbIJrBHb0XFhBB9v7vsBvxT9AlfAKCqqfVfWcyaTvzuBRIh9zexFPFwBTcTTc4wcD4 AnX2uCOYjCkfpZe7/JhfWzM7W9t24QN6sgmPRpPg6ByeMTbHjj3aHFO6s25pAynyMZXo KEmjzT/H4Vkii/2aW6AMmXZ94pzY6YTdfSpMUaxpBaCEh+DnV5hI0TItqFTiS0XCrbjw tcoWoIHIh8WW03nrpuL+wcXNgrJ1G/PVyz00f1SDGpRk8kfUiMGpCaWrG02Ro0VMUv3Y 0iDg== X-Gm-Message-State: APjAAAWHqOJfogOMZssNux5k6dWLv3NJp1RPep3buLn81/kWg2XLK4k/ LIuMVacGnYLg/laSrlAg6wPYBmfVq7Y= X-Received: by 2002:a5d:5745:: with SMTP id q5mr2800024wrw.75.1561631298633; Thu, 27 Jun 2019 03:28:18 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.17 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:18 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 26/32] crypto: chelsio/aes - replace AES cipher calls with library calls Date: Thu, 27 Jun 2019 12:26:41 +0200 Message-Id: <20190627102647.2992-27-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Replace a couple of occurrences where the "aes-generic" cipher is instantiated explicitly and only used for encryption of a single block. Use AES library calls instead. Signed-off-by: Ard Biesheuvel --- drivers/crypto/chelsio/Kconfig | 1 + drivers/crypto/chelsio/chcr_algo.c | 46 ++++++-------------- drivers/crypto/chelsio/chcr_crypto.h | 1 - drivers/crypto/chelsio/chcr_ipsec.c | 19 +++----- drivers/crypto/chelsio/chtls/chtls_hw.c | 20 +++------ 5 files changed, 26 insertions(+), 61 deletions(-) -- 2.20.1 diff --git a/drivers/crypto/chelsio/Kconfig b/drivers/crypto/chelsio/Kconfig index 930d82d991f2..36402ba63b50 100644 --- a/drivers/crypto/chelsio/Kconfig +++ b/drivers/crypto/chelsio/Kconfig @@ -1,6 +1,7 @@ config CRYPTO_DEV_CHELSIO tristate "Chelsio Crypto Co-processor Driver" depends on CHELSIO_T4 + select CRYPTO_LIB_AES select CRYPTO_SHA1 select CRYPTO_SHA256 select CRYPTO_SHA512 diff --git a/drivers/crypto/chelsio/chcr_algo.c b/drivers/crypto/chelsio/chcr_algo.c index 177f572b9589..38ee38b37ae6 100644 --- a/drivers/crypto/chelsio/chcr_algo.c +++ b/drivers/crypto/chelsio/chcr_algo.c @@ -1023,22 +1023,21 @@ static int chcr_update_tweak(struct ablkcipher_request *req, u8 *iv, struct crypto_ablkcipher *tfm = crypto_ablkcipher_reqtfm(req); struct ablk_ctx *ablkctx = ABLK_CTX(c_ctx(tfm)); struct chcr_blkcipher_req_ctx *reqctx = ablkcipher_request_ctx(req); - struct crypto_cipher *cipher; + struct crypto_aes_ctx aes; int ret, i; u8 *key; unsigned int keylen; int round = reqctx->last_req_len / AES_BLOCK_SIZE; int round8 = round / 8; - cipher = ablkctx->aes_generic; memcpy(iv, reqctx->iv, AES_BLOCK_SIZE); keylen = ablkctx->enckey_len / 2; key = ablkctx->key + keylen; - ret = crypto_cipher_setkey(cipher, key, keylen); + ret = aes_expandkey(&aes, key, keylen); if (ret) - goto out; - crypto_cipher_encrypt_one(cipher, iv, iv); + return ret; + aes_encrypt(&aes, iv, iv); for (i = 0; i < round8; i++) gf128mul_x8_ble((le128 *)iv, (le128 *)iv); @@ -1046,9 +1045,10 @@ static int chcr_update_tweak(struct ablkcipher_request *req, u8 *iv, gf128mul_x_ble((le128 *)iv, (le128 *)iv); if (!isfinal) - crypto_cipher_decrypt_one(cipher, iv, iv); -out: - return ret; + aes_decrypt(&aes, iv, iv); + + memzero_explicit(&aes, sizeof(aes)); + return 0; } static int chcr_update_cipher_iv(struct ablkcipher_request *req, @@ -1411,16 +1411,6 @@ static int chcr_cra_init(struct crypto_tfm *tfm) return PTR_ERR(ablkctx->sw_cipher); } - if (get_cryptoalg_subtype(tfm) == CRYPTO_ALG_SUB_TYPE_XTS) { - /* To update tweak*/ - ablkctx->aes_generic = crypto_alloc_cipher("aes-generic", 0, 0); - if (IS_ERR(ablkctx->aes_generic)) { - pr_err("failed to allocate aes cipher for tweak\n"); - return PTR_ERR(ablkctx->aes_generic); - } - } else - ablkctx->aes_generic = NULL; - tfm->crt_ablkcipher.reqsize = sizeof(struct chcr_blkcipher_req_ctx); return chcr_device_init(crypto_tfm_ctx(tfm)); } @@ -1451,8 +1441,6 @@ static void chcr_cra_exit(struct crypto_tfm *tfm) struct ablk_ctx *ablkctx = ABLK_CTX(ctx); crypto_free_sync_skcipher(ablkctx->sw_cipher); - if (ablkctx->aes_generic) - crypto_free_cipher(ablkctx->aes_generic); } static int get_alg_config(struct algo_param *params, @@ -3364,9 +3352,9 @@ static int chcr_gcm_setkey(struct crypto_aead *aead, const u8 *key, { struct chcr_aead_ctx *aeadctx = AEAD_CTX(a_ctx(aead)); struct chcr_gcm_ctx *gctx = GCM_CTX(aeadctx); - struct crypto_cipher *cipher; unsigned int ck_size; int ret = 0, key_ctx_size = 0; + struct crypto_aes_ctx aes; aeadctx->enckey_len = 0; crypto_aead_clear_flags(aeadctx->sw_cipher, CRYPTO_TFM_REQ_MASK); @@ -3409,23 +3397,15 @@ static int chcr_gcm_setkey(struct crypto_aead *aead, const u8 *key, /* Calculate the H = CIPH(K, 0 repeated 16 times). * It will go in key context */ - cipher = crypto_alloc_cipher("aes-generic", 0, 0); - if (IS_ERR(cipher)) { - aeadctx->enckey_len = 0; - ret = -ENOMEM; - goto out; - } - - ret = crypto_cipher_setkey(cipher, key, keylen); + ret = aes_expandkey(&aes, key, keylen); if (ret) { aeadctx->enckey_len = 0; - goto out1; + goto out; } memset(gctx->ghash_h, 0, AEAD_H_SIZE); - crypto_cipher_encrypt_one(cipher, gctx->ghash_h, gctx->ghash_h); + aes_encrypt(&aes, gctx->ghash_h, gctx->ghash_h); + memzero_explicit(&aes, sizeof(aes)); -out1: - crypto_free_cipher(cipher); out: return ret; } diff --git a/drivers/crypto/chelsio/chcr_crypto.h b/drivers/crypto/chelsio/chcr_crypto.h index 655606f2e4d0..993c97e70565 100644 --- a/drivers/crypto/chelsio/chcr_crypto.h +++ b/drivers/crypto/chelsio/chcr_crypto.h @@ -172,7 +172,6 @@ static inline struct chcr_context *h_ctx(struct crypto_ahash *tfm) struct ablk_ctx { struct crypto_sync_skcipher *sw_cipher; - struct crypto_cipher *aes_generic; __be32 key_ctx_hdr; unsigned int enckey_len; unsigned char ciph_mode; diff --git a/drivers/crypto/chelsio/chcr_ipsec.c b/drivers/crypto/chelsio/chcr_ipsec.c index f429aae72542..24355680f30a 100644 --- a/drivers/crypto/chelsio/chcr_ipsec.c +++ b/drivers/crypto/chelsio/chcr_ipsec.c @@ -132,11 +132,11 @@ static inline int chcr_ipsec_setauthsize(struct xfrm_state *x, static inline int chcr_ipsec_setkey(struct xfrm_state *x, struct ipsec_sa_entry *sa_entry) { - struct crypto_cipher *cipher; int keylen = (x->aead->alg_key_len + 7) / 8; unsigned char *key = x->aead->alg_key; int ck_size, key_ctx_size = 0; unsigned char ghash_h[AEAD_H_SIZE]; + struct crypto_aes_ctx aes; int ret = 0; if (keylen > 3) { @@ -170,26 +170,19 @@ static inline int chcr_ipsec_setkey(struct xfrm_state *x, /* Calculate the H = CIPH(K, 0 repeated 16 times). * It will go in key context */ - cipher = crypto_alloc_cipher("aes-generic", 0, 0); - if (IS_ERR(cipher)) { - sa_entry->enckey_len = 0; - ret = -ENOMEM; - goto out; - } - - ret = crypto_cipher_setkey(cipher, key, keylen); + ret = aes_expandkey(&aes, key, keylen); if (ret) { sa_entry->enckey_len = 0; - goto out1; + goto out; } memset(ghash_h, 0, AEAD_H_SIZE); - crypto_cipher_encrypt_one(cipher, ghash_h, ghash_h); + aes_encrypt(&aes, ghash_h, ghash_h); + memzero_explicit(&aes, sizeof(aes)); + memcpy(sa_entry->key + (DIV_ROUND_UP(sa_entry->enckey_len, 16) * 16), ghash_h, AEAD_H_SIZE); sa_entry->kctx_len = ((DIV_ROUND_UP(sa_entry->enckey_len, 16)) << 4) + AEAD_H_SIZE; -out1: - crypto_free_cipher(cipher); out: return ret; } diff --git a/drivers/crypto/chelsio/chtls/chtls_hw.c b/drivers/crypto/chelsio/chtls/chtls_hw.c index 490960755864..a6f0278f3597 100644 --- a/drivers/crypto/chelsio/chtls/chtls_hw.c +++ b/drivers/crypto/chelsio/chtls/chtls_hw.c @@ -216,8 +216,8 @@ static int chtls_key_info(struct chtls_sock *csk, unsigned char key[AES_KEYSIZE_128]; struct tls12_crypto_info_aes_gcm_128 *gcm_ctx; unsigned char ghash_h[AEAD_H_SIZE]; - struct crypto_cipher *cipher; int ck_size, key_ctx_size; + struct crypto_aes_ctx aes; int ret; gcm_ctx = (struct tls12_crypto_info_aes_gcm_128 *) @@ -237,18 +237,13 @@ static int chtls_key_info(struct chtls_sock *csk, /* Calculate the H = CIPH(K, 0 repeated 16 times). * It will go in key context */ - cipher = crypto_alloc_cipher("aes", 0, 0); - if (IS_ERR(cipher)) { - ret = -ENOMEM; - goto out; - } - - ret = crypto_cipher_setkey(cipher, key, keylen); + ret = aes_expandkey(&aes, key, keylen); if (ret) - goto out1; + return ret; memset(ghash_h, 0, AEAD_H_SIZE); - crypto_cipher_encrypt_one(cipher, ghash_h, ghash_h); + aes_encrypt(&aes, ghash_h, ghash_h); + memzero_explicit(&aes, sizeof(aes)); csk->tlshws.keylen = key_ctx_size; /* Copy the Key context */ @@ -272,10 +267,7 @@ static int chtls_key_info(struct chtls_sock *csk, /* erase key info from driver */ memset(gcm_ctx->key, 0, keylen); -out1: - crypto_free_cipher(cipher); -out: - return ret; + return 0; } static void chtls_set_scmd(struct chtls_sock *csk) From patchwork Thu Jun 27 10:26:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167925 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148935ilk; Thu, 27 Jun 2019 03:28:24 -0700 (PDT) X-Google-Smtp-Source: APXvYqw7RBn4JSpv5wb+cUt7mFu3+/VX1CIJbyAU6934HzBNsJi+TnfB+p6vBqbD4KfMAobNuS5Q X-Received: by 2002:a65:518d:: with SMTP id h13mr3088080pgq.22.1561631304319; Thu, 27 Jun 2019 03:28:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631304; cv=none; d=google.com; s=arc-20160816; b=uKdaL2VWZQGdq0NctR6fFsGgPly4sl9by1sdzRiU9T+oQzGo/38FRTuR2uSznkIV4P 9QGDJvUzAGpxbvkm45M87UBinx6h6rwyo7A5Kz6m5y3kkHuIBhh9CSnizb2mYzEV+jUO gCDO4i2XaJCkv3P9ROoji/EKETyfq6TvGlMMbffsf/9htKffnoACwOIbsw9HdVT9Ua+T g9BglQUkh7svyKIsTP9P3bFTnIlj3TNL4lrwWWm87EOxsxKXyU8wut7hcPWxW4QeprkT ZqXzks19mO3I9mPTrv2ITl1iKwFbo3POBBu65qXCvkp4mGrMIofIJCHNqo5xJlP503aD s26Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=38BYhMp+Qqctzgii2HzkxX6b3+ad6JE63d8OrKpU3XI=; b=Q34o3y6pSmdtg549J2rYy+P42jNbvYKOpWYgOhlDP0V9pHtlpOSQVNehoos/XeZEN9 eDtM52vZBwBNT8SKSqpE0oKSZpjCovSlW6CjKhsfIXB1L/Tiql5C0MS2udBPvwRuhhMW VgMEgPbbGdpIqYWodSGKqHuWj5eJpBZ82siF9vqgFgIo17SWyiJJzsw4bWQLDH47vXcZ om3vs2DlN+jkEz3R0AmPwcyyBEbeBgBpwv+QlpSXWjNGkqgBtBlCPLkS7UtGfUg6qMVT uwSo2eHtWKE01/aWb2aU1G5Wdv/avAdQs7eMrVzCmRDA246eNxFfy4hlFmyizapLwoaW 4PtQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=x9HU+92+; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.24; Thu, 27 Jun 2019 03:28:24 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=x9HU+92+; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726682AbfF0K2X (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:23 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:34278 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726678AbfF0K2W (ORCPT ); Thu, 27 Jun 2019 06:28:22 -0400 Received: by mail-wm1-f68.google.com with SMTP id w9so6715922wmd.1 for ; Thu, 27 Jun 2019 03:28:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=38BYhMp+Qqctzgii2HzkxX6b3+ad6JE63d8OrKpU3XI=; b=x9HU+92+YRSlODOqR3JQBAPBD4lpOxUVs9sF7rp4FOZuodLWSHl4MoDTQZ/h/9FeD8 repxIWE33NkIcBOIlDLWz0rg+MBcU5eUfQli1AvJ72r2BEu0tTmNB1bLmX24lSpQNMD0 I+7G36/VCOgfHHbg68wiKw8DQe2kJDb0U8kAWOEiSZWaQGvH0/l7UhXLSev2eZPvcgTS pdQ7BHz3QjhnL/fa8mrFHo5LGXo6Z+qIubK+JXYm2WAp/f7GdjxJKiJUtlwonIT/Kr0l wZ6qrj284z0uXe1hAxt7XOs74xyeZv94EDA++UMWNyQPhvAW24LMRAYU+sNX385NnxoQ 2M4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=38BYhMp+Qqctzgii2HzkxX6b3+ad6JE63d8OrKpU3XI=; b=o2BS3FKEDBuF7nkUMqUt97ZNrrLmd5fxhYYQ7KYFxTOeS78cWKjr3TqYb52NCTsb54 8TdyH8uZidtQri2iueX+eqN+Z235GBjdlLNuF3C8jsE3iKOI9wUqYG30AbTQ1UeamIve MtqzibrKQSLUakO8E153cJh4R60FDTB+MkQULN4dIneJhIW/ZkVpj3akegSATdhKOPdt ynKbc9D8dBZd80atyK/FJ5bnJTIB41KWd21HWLT0bs7iXobUw21YEoR5nSsby4OJMv31 qIlwG6n2aHRv2I+yoJJ3eUmb3QB4dzWFmhvGcalP5IHoz7c4tpDLoFgNbSQrLHQUAoaC Y7Rw== X-Gm-Message-State: APjAAAWF9VWcRIBR4em9aH28D3vxk6wwIP9S8yvpERcDwP14L28y75HU qMWbahXv2q/hAobKI9U4tNCpWtLGA5E= X-Received: by 2002:a7b:c751:: with SMTP id w17mr2854853wmk.127.1561631299702; Thu, 27 Jun 2019 03:28:19 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.18 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:18 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 27/32] crypto: aes/generic - unexport last-round AES tables Date: Thu, 27 Jun 2019 12:26:42 +0200 Message-Id: <20190627102647.2992-28-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The versions of the AES lookup tables that are only used during the last round are never used outside of the driver, so there is no need to export their symbols. Signed-off-by: Ard Biesheuvel --- crypto/aes_generic.c | 6 ++---- include/crypto/aes.h | 2 -- 2 files changed, 2 insertions(+), 6 deletions(-) -- 2.20.1 diff --git a/crypto/aes_generic.c b/crypto/aes_generic.c index 426deb437f19..71a5c190d360 100644 --- a/crypto/aes_generic.c +++ b/crypto/aes_generic.c @@ -328,7 +328,7 @@ __visible const u32 crypto_ft_tab[4][256] ____cacheline_aligned = { } }; -__visible const u32 crypto_fl_tab[4][256] ____cacheline_aligned = { +static const u32 crypto_fl_tab[4][256] ____cacheline_aligned = { { 0x00000063, 0x0000007c, 0x00000077, 0x0000007b, 0x000000f2, 0x0000006b, 0x0000006f, 0x000000c5, @@ -856,7 +856,7 @@ __visible const u32 crypto_it_tab[4][256] ____cacheline_aligned = { } }; -__visible const u32 crypto_il_tab[4][256] ____cacheline_aligned = { +static const u32 crypto_il_tab[4][256] ____cacheline_aligned = { { 0x00000052, 0x00000009, 0x0000006a, 0x000000d5, 0x00000030, 0x00000036, 0x000000a5, 0x00000038, @@ -1121,9 +1121,7 @@ __visible const u32 crypto_il_tab[4][256] ____cacheline_aligned = { }; EXPORT_SYMBOL_GPL(crypto_ft_tab); -EXPORT_SYMBOL_GPL(crypto_fl_tab); EXPORT_SYMBOL_GPL(crypto_it_tab); -EXPORT_SYMBOL_GPL(crypto_il_tab); /** * crypto_aes_set_key - Set the AES key. diff --git a/include/crypto/aes.h b/include/crypto/aes.h index 0a64a977f9b3..df8426fd8051 100644 --- a/include/crypto/aes.h +++ b/include/crypto/aes.h @@ -29,9 +29,7 @@ struct crypto_aes_ctx { }; extern const u32 crypto_ft_tab[4][256] ____cacheline_aligned; -extern const u32 crypto_fl_tab[4][256] ____cacheline_aligned; extern const u32 crypto_it_tab[4][256] ____cacheline_aligned; -extern const u32 crypto_il_tab[4][256] ____cacheline_aligned; int crypto_aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, unsigned int key_len); From patchwork Thu Jun 27 10:26:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167924 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148941ilk; Thu, 27 Jun 2019 03:28:24 -0700 (PDT) X-Google-Smtp-Source: APXvYqz55mX4nvbuaSDr9MQac3oV+I07QF655UwwxsjMXt2ER2eTbzLhO1NkOxey/nD4UCPpEdf7 X-Received: by 2002:a17:902:9896:: with SMTP id s22mr3668327plp.4.1561631304668; Thu, 27 Jun 2019 03:28:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631304; cv=none; d=google.com; s=arc-20160816; b=RwpW2FowgoPXzAXYkl5fxIHS0YetNEed8yr6lQ8/mndCbU5fnVjnK9KkUQ3TamSnYn OKKnM6Tug1nkmt6DdK1kvAbfpJBrOkNmPWiaG1MRNQOLprf/cLKNWN1xKIQcD58G+h/P i2hYnbgneb34Up7J101MuZj09A20EZUWv4/GKaJia+guQOShciI2LjFbtPeiDiOdRuBP 0wNWsZhPvNXAclaToEKzIFpR9a4L9QHKIqJL0ZXzdId1juXB4MCGCVWOF1/nQ4EC+0Cy X6d1MThpuB/WLJMzjD+Y5FsaXzRaFU1Lhm4mYvJPc6IPeyFzEcMAfRmQTuvy9K0tRPWY 9hsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=MjklBC4QXLLYOVvVYsNk23Xh/eNadKBF3H0s2LOsbpU=; b=Uzoss4RXDtVLSsAaVjv3rEdxC8Jik7IyF0e2JrXvRFj8/2aWtTxIKoPFCUccERMeHS xukY+HMh21/vgHY5ClqGZ8CAYPYvon07laagpO+kYIYk1I2GqI+2XlWaigL/HIYE5YyE 60tbLcuyPHxIVKRdJq9wnRneA+N4p9vESSHtcgVLV6/jiyS6ylB1F00u1foTCszMR0qi +8EjT8Iw3ml+RAaodbvCR7JTRdn20Sgw97gHFTB9xlAyAf53ylHgrEwksogdP2zcs658 0K/iTxTHKQFQHhcOCXDjdC0FGo6fwzsPO1yhO2tZdacrdvQPA9Jr8R03gxRQFXiDJmvy OoXA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="Za/MTEXK"; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.24; Thu, 27 Jun 2019 03:28:24 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="Za/MTEXK"; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726187AbfF0K2X (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:23 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:33600 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726690AbfF0K2W (ORCPT ); Thu, 27 Jun 2019 06:28:22 -0400 Received: by mail-wr1-f65.google.com with SMTP id n9so1950786wru.0 for ; Thu, 27 Jun 2019 03:28:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MjklBC4QXLLYOVvVYsNk23Xh/eNadKBF3H0s2LOsbpU=; b=Za/MTEXKeBIrGQve8XmJXITdWVnr5g9xRMV7x2h+pxLHKCenXG82yFlL+Qv+YPEd5G tbYH8cVKuLDtrbVr1dpGMy7R5taeoGMxUG+jM6kH27v0yLUj8V1pX32ICu6KYu8qNHES Xn5OO5Y0zyy6SJ6IZZvWK14sjV6iDaL/gKhCbuNtJxb4h71FyPfMYbX8Yd6tanXijzT8 3Awfk6nMnXwjfZgrR1VwDEVlRtWrfY8azFwoWm0Bg8KxXpdAEp/nslRrNFGPneh3q/Qd 9pvs48TDIvgbtAnVAoTIL9I/ofMY6DjDCFf6VhhBmCaRL+UYlIy/smOFq4R0B3EQNXFF LZaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MjklBC4QXLLYOVvVYsNk23Xh/eNadKBF3H0s2LOsbpU=; b=Cffn1P8YYsy72Z8b1fYOjpKsmNxrh1G37zK9au6gcaCTXqkYoSH2DEl/IhyDDojZFE S/Dyq5n/G8dbKTzuIOSUi3ZM+oFWgaSX2pe9+5YICknW0tUjYPT5Pi9nzUW3reHjl7tQ gsuIYAGu98TU/slpwfqFdMc9AGiRaBpWDfTs2VtDez9/8om7qiftuWKuaE3sAI+urdUX R+WEqPH0T+3GPvWZC5UirbEmc6LrAWwP5yqYfzeiu5bShl6My8qtR4zAD+ee9wvLN1Mt 3qFhe4vN4+AokxCKWUI3ROZfuqsPYcC0/wFMCLxQ9M+bhdXUQWPUfirTP+clm0fQo4oi 1sdw== X-Gm-Message-State: APjAAAXSWv17mFHMoox3VjvDEeAOgZa8LPUvd4PLjUFL273KNY/hUJsz etfvjiQUeYPrtfmcSvvBbBiIvD2Wxt0= X-Received: by 2002:adf:fb84:: with SMTP id a4mr2816484wrr.41.1561631300630; Thu, 27 Jun 2019 03:28:20 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.19 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:20 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 28/32] crypto: lib/aes - export sbox and inverse sbox Date: Thu, 27 Jun 2019 12:26:43 +0200 Message-Id: <20190627102647.2992-29-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org There are a few copies of the AES S-boxes floating around, so export the ones from the AES library so that we can reuse them in other modules. Signed-off-by: Ard Biesheuvel --- include/crypto/aes.h | 3 +++ lib/crypto/aes.c | 6 ++++++ 2 files changed, 9 insertions(+) -- 2.20.1 diff --git a/include/crypto/aes.h b/include/crypto/aes.h index df8426fd8051..8e0f4cf948e5 100644 --- a/include/crypto/aes.h +++ b/include/crypto/aes.h @@ -67,4 +67,7 @@ void aes_encrypt(const struct crypto_aes_ctx *ctx, u8 *out, const u8 *in); */ void aes_decrypt(const struct crypto_aes_ctx *ctx, u8 *out, const u8 *in); +extern const u8 crypto_aes_sbox[]; +extern const u8 crypto_aes_inv_sbox[]; + #endif diff --git a/lib/crypto/aes.c b/lib/crypto/aes.c index 9928b23e0a8a..467f0c35a0e0 100644 --- a/lib/crypto/aes.c +++ b/lib/crypto/aes.c @@ -82,6 +82,12 @@ static volatile const u8 __cacheline_aligned aes_inv_sbox[] = { 0xe1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0c, 0x7d, }; +extern const u8 crypto_aes_sbox[] __alias(aes_sbox); +extern const u8 crypto_aes_inv_sbox[] __alias(aes_inv_sbox); + +EXPORT_SYMBOL(crypto_aes_sbox); +EXPORT_SYMBOL(crypto_aes_inv_sbox); + static u32 mul_by_x(u32 w) { u32 x = w & 0x7f7f7f7f; From patchwork Thu Jun 27 10:26:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167927 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148974ilk; Thu, 27 Jun 2019 03:28:26 -0700 (PDT) X-Google-Smtp-Source: APXvYqwJ/Ei9qUeCJls0xa5m5T8lwvRIjsmmuPgtl7NNvU10faJEvnUC3mGgYA1raKkwZzMbqjUp X-Received: by 2002:a63:545c:: with SMTP id e28mr3215495pgm.374.1561631306202; Thu, 27 Jun 2019 03:28:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631306; cv=none; d=google.com; s=arc-20160816; b=Pr0lk66Rw+LneJ1OrK5hzaRVvwcOmJY6MLxyTTQwgzrdBtSdBZQ7IMPHh6wjrNiKgR NVYYAEo52qj2b1Y2S7K9oxljrUHSY0/d7ARym+5FRM9uC1/PH0FQrX5HhIBOCLW22RpO TYxMo4+bid9PJKXfXj/ysh4/TnpOK4f4f7PtgMdRxPcNdHrCvqqSEu9C2BpEOEXH5qe+ ixmvSqmZcpVtiazDXB7X1trTU8LMZ2dOj4bPUKTpvUi6gedUOQwBkcHORMe7AfUr55Yf mhrlo15sgQmDDJG35HMPCy9wL20ls4NsYOrko/sA7PYK6a4KcNscJV8fSeVm1tp1YTb3 ypyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9HScBZzsgsZiNgPWKczwDVxmiNeW3c94TJANnZN9vJM=; b=ThgPDmosQnKuCjaIkJfUb6/s/pVinrtM9/3GMNQ91RvFDHQIkJXk/TPAlo9wZjk4DR qKOqPTdzjRiB3Z2Nkg+a8tTONCOJMP+Bt4AazOGa2ZQD87BIpw9LIg/zWJvG/49LuxAH GxtQkGM5HqjfQbpT60m3OztIoLjq3MKjLIGHCqd6TnH54ss951yxMLZk76T8NZ/LfOAc Tm6m8Gv0hFHg0NZQexWFibQyFKRfpFQA6lkxQkYNA9980YHS1QJNIbrjZlzG2lgR6nEm MBDvDgDM6tZsK2Of3w3vSDicoy2XZhYKGJlpROEsm/RUgEtRb5als5UPd+9YzjnoO56V 9psA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rWqcXuFw; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.26; Thu, 27 Jun 2019 03:28:26 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rWqcXuFw; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726480AbfF0K2Z (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:25 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:40313 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726623AbfF0K2Y (ORCPT ); Thu, 27 Jun 2019 06:28:24 -0400 Received: by mail-wr1-f65.google.com with SMTP id p11so1911428wre.7 for ; Thu, 27 Jun 2019 03:28:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9HScBZzsgsZiNgPWKczwDVxmiNeW3c94TJANnZN9vJM=; b=rWqcXuFw4fCIceVQFWbO3SfX9Y3RP0E6o8/bTP9HvpQrcZf1B+tRSOC/4UhLvagx6Z hunVgmeLNbjaYM50xZ56N7d8Rq/sUIKP3iYnIKEJuCNNtc0dhKSGt8rpe9roa6g/0R0J SmHS61Xyba6vy6oIGW4ipfm1URyKji2NA5hiwUEwr4zmHUh4nvDrpMWf7B3ManuaRw0b QMEf5lz44Vbjli03f98JwdPvYHMDFf8AAUteWposhxak7hM2OemExeUtxH8t1igM38bC yjLk2KdBUpK/I/3RYq0x8miQixYXV7v5112V7hbyT601LCmY/NOdrebzsmcYR87VsOKv qWDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9HScBZzsgsZiNgPWKczwDVxmiNeW3c94TJANnZN9vJM=; b=U0EJ82q3YPzjzhGzs51zO0hz5aqv6SU03Zmx1IGgU/DiD9PFNf1UzMBknKGm2tu2v1 FQ8uEp6XHN2um1VV5XRXwtI6TlV+6D+RNPlNjmUmNO/DjNZ553tN4S6Zm0+wuAQOtWwM oiwPc2V6Od1pXu62L1zGUYdtmtsQtcDImMvLEs6/lYlQmMKNC+Ut7r/klU9CGdxxBMPk xvmk0FNR5njlijzpkQtRT3zlBV2b2uU5vNstY6MTHqhGtNTzASdL+T0V5F+tZBkdz1pD JyQ3hUrOIFU7/RuXYrpyekrIQSZOGsrPzgRIAFpBmZ6jNjrN7voXkO9mKBNVt4XJ6OA6 iqjg== X-Gm-Message-State: APjAAAXYfqG1XbQ9KUINuT43e9uxBbQGdb9yGiFPlzDSgc+tr33SIsdj brlXo/Fxq1ZBcEP1vYVqFrJFeskOztE= X-Received: by 2002:a5d:400f:: with SMTP id n15mr2738803wrp.312.1561631301667; Thu, 27 Jun 2019 03:28:21 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.20 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:20 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 29/32] crypto: arm64/aes-neon - switch to shared AES Sboxes Date: Thu, 27 Jun 2019 12:26:44 +0200 Message-Id: <20190627102647.2992-30-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/aes-neon.S | 74 +------------------- 1 file changed, 3 insertions(+), 71 deletions(-) -- 2.20.1 diff --git a/arch/arm64/crypto/aes-neon.S b/arch/arm64/crypto/aes-neon.S index 29100f692e8a..169e86d8ae36 100644 --- a/arch/arm64/crypto/aes-neon.S +++ b/arch/arm64/crypto/aes-neon.S @@ -50,7 +50,7 @@ /* do preload for encryption */ .macro enc_prepare, ignore0, ignore1, temp - prepare .LForward_Sbox, .LForward_ShiftRows, \temp + prepare crypto_aes_sbox, .LForward_ShiftRows, \temp .endm .macro enc_switch_key, ignore0, ignore1, temp @@ -59,7 +59,7 @@ /* do preload for decryption */ .macro dec_prepare, ignore0, ignore1, temp - prepare .LReverse_Sbox, .LReverse_ShiftRows, \temp + prepare crypto_aes_inv_sbox, .LReverse_ShiftRows, \temp .endm /* apply SubBytes transformation using the the preloaded Sbox */ @@ -279,75 +279,7 @@ #include "aes-modes.S" .section ".rodata", "a" - .align 6 -.LForward_Sbox: - .byte 0x63, 0x7c, 0x77, 0x7b, 0xf2, 0x6b, 0x6f, 0xc5 - .byte 0x30, 0x01, 0x67, 0x2b, 0xfe, 0xd7, 0xab, 0x76 - .byte 0xca, 0x82, 0xc9, 0x7d, 0xfa, 0x59, 0x47, 0xf0 - .byte 0xad, 0xd4, 0xa2, 0xaf, 0x9c, 0xa4, 0x72, 0xc0 - .byte 0xb7, 0xfd, 0x93, 0x26, 0x36, 0x3f, 0xf7, 0xcc - .byte 0x34, 0xa5, 0xe5, 0xf1, 0x71, 0xd8, 0x31, 0x15 - .byte 0x04, 0xc7, 0x23, 0xc3, 0x18, 0x96, 0x05, 0x9a - .byte 0x07, 0x12, 0x80, 0xe2, 0xeb, 0x27, 0xb2, 0x75 - .byte 0x09, 0x83, 0x2c, 0x1a, 0x1b, 0x6e, 0x5a, 0xa0 - .byte 0x52, 0x3b, 0xd6, 0xb3, 0x29, 0xe3, 0x2f, 0x84 - .byte 0x53, 0xd1, 0x00, 0xed, 0x20, 0xfc, 0xb1, 0x5b - .byte 0x6a, 0xcb, 0xbe, 0x39, 0x4a, 0x4c, 0x58, 0xcf - .byte 0xd0, 0xef, 0xaa, 0xfb, 0x43, 0x4d, 0x33, 0x85 - .byte 0x45, 0xf9, 0x02, 0x7f, 0x50, 0x3c, 0x9f, 0xa8 - .byte 0x51, 0xa3, 0x40, 0x8f, 0x92, 0x9d, 0x38, 0xf5 - .byte 0xbc, 0xb6, 0xda, 0x21, 0x10, 0xff, 0xf3, 0xd2 - .byte 0xcd, 0x0c, 0x13, 0xec, 0x5f, 0x97, 0x44, 0x17 - .byte 0xc4, 0xa7, 0x7e, 0x3d, 0x64, 0x5d, 0x19, 0x73 - .byte 0x60, 0x81, 0x4f, 0xdc, 0x22, 0x2a, 0x90, 0x88 - .byte 0x46, 0xee, 0xb8, 0x14, 0xde, 0x5e, 0x0b, 0xdb - .byte 0xe0, 0x32, 0x3a, 0x0a, 0x49, 0x06, 0x24, 0x5c - .byte 0xc2, 0xd3, 0xac, 0x62, 0x91, 0x95, 0xe4, 0x79 - .byte 0xe7, 0xc8, 0x37, 0x6d, 0x8d, 0xd5, 0x4e, 0xa9 - .byte 0x6c, 0x56, 0xf4, 0xea, 0x65, 0x7a, 0xae, 0x08 - .byte 0xba, 0x78, 0x25, 0x2e, 0x1c, 0xa6, 0xb4, 0xc6 - .byte 0xe8, 0xdd, 0x74, 0x1f, 0x4b, 0xbd, 0x8b, 0x8a - .byte 0x70, 0x3e, 0xb5, 0x66, 0x48, 0x03, 0xf6, 0x0e - .byte 0x61, 0x35, 0x57, 0xb9, 0x86, 0xc1, 0x1d, 0x9e - .byte 0xe1, 0xf8, 0x98, 0x11, 0x69, 0xd9, 0x8e, 0x94 - .byte 0x9b, 0x1e, 0x87, 0xe9, 0xce, 0x55, 0x28, 0xdf - .byte 0x8c, 0xa1, 0x89, 0x0d, 0xbf, 0xe6, 0x42, 0x68 - .byte 0x41, 0x99, 0x2d, 0x0f, 0xb0, 0x54, 0xbb, 0x16 - -.LReverse_Sbox: - .byte 0x52, 0x09, 0x6a, 0xd5, 0x30, 0x36, 0xa5, 0x38 - .byte 0xbf, 0x40, 0xa3, 0x9e, 0x81, 0xf3, 0xd7, 0xfb - .byte 0x7c, 0xe3, 0x39, 0x82, 0x9b, 0x2f, 0xff, 0x87 - .byte 0x34, 0x8e, 0x43, 0x44, 0xc4, 0xde, 0xe9, 0xcb - .byte 0x54, 0x7b, 0x94, 0x32, 0xa6, 0xc2, 0x23, 0x3d - .byte 0xee, 0x4c, 0x95, 0x0b, 0x42, 0xfa, 0xc3, 0x4e - .byte 0x08, 0x2e, 0xa1, 0x66, 0x28, 0xd9, 0x24, 0xb2 - .byte 0x76, 0x5b, 0xa2, 0x49, 0x6d, 0x8b, 0xd1, 0x25 - .byte 0x72, 0xf8, 0xf6, 0x64, 0x86, 0x68, 0x98, 0x16 - .byte 0xd4, 0xa4, 0x5c, 0xcc, 0x5d, 0x65, 0xb6, 0x92 - .byte 0x6c, 0x70, 0x48, 0x50, 0xfd, 0xed, 0xb9, 0xda - .byte 0x5e, 0x15, 0x46, 0x57, 0xa7, 0x8d, 0x9d, 0x84 - .byte 0x90, 0xd8, 0xab, 0x00, 0x8c, 0xbc, 0xd3, 0x0a - .byte 0xf7, 0xe4, 0x58, 0x05, 0xb8, 0xb3, 0x45, 0x06 - .byte 0xd0, 0x2c, 0x1e, 0x8f, 0xca, 0x3f, 0x0f, 0x02 - .byte 0xc1, 0xaf, 0xbd, 0x03, 0x01, 0x13, 0x8a, 0x6b - .byte 0x3a, 0x91, 0x11, 0x41, 0x4f, 0x67, 0xdc, 0xea - .byte 0x97, 0xf2, 0xcf, 0xce, 0xf0, 0xb4, 0xe6, 0x73 - .byte 0x96, 0xac, 0x74, 0x22, 0xe7, 0xad, 0x35, 0x85 - .byte 0xe2, 0xf9, 0x37, 0xe8, 0x1c, 0x75, 0xdf, 0x6e - .byte 0x47, 0xf1, 0x1a, 0x71, 0x1d, 0x29, 0xc5, 0x89 - .byte 0x6f, 0xb7, 0x62, 0x0e, 0xaa, 0x18, 0xbe, 0x1b - .byte 0xfc, 0x56, 0x3e, 0x4b, 0xc6, 0xd2, 0x79, 0x20 - .byte 0x9a, 0xdb, 0xc0, 0xfe, 0x78, 0xcd, 0x5a, 0xf4 - .byte 0x1f, 0xdd, 0xa8, 0x33, 0x88, 0x07, 0xc7, 0x31 - .byte 0xb1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xec, 0x5f - .byte 0x60, 0x51, 0x7f, 0xa9, 0x19, 0xb5, 0x4a, 0x0d - .byte 0x2d, 0xe5, 0x7a, 0x9f, 0x93, 0xc9, 0x9c, 0xef - .byte 0xa0, 0xe0, 0x3b, 0x4d, 0xae, 0x2a, 0xf5, 0xb0 - .byte 0xc8, 0xeb, 0xbb, 0x3c, 0x83, 0x53, 0x99, 0x61 - .byte 0x17, 0x2b, 0x04, 0x7e, 0xba, 0x77, 0xd6, 0x26 - .byte 0xe1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0c, 0x7d - + .align 4 .LForward_ShiftRows: .octa 0x0b06010c07020d08030e09040f0a0500 From patchwork Thu Jun 27 10:26:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167926 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2148983ilk; Thu, 27 Jun 2019 03:28:26 -0700 (PDT) X-Google-Smtp-Source: APXvYqwrk9XfPKD+LrVdGCkCcBAqPEevYygX9OnrOODEUUwV6o7R1j42p9g4B9wsm7yqO/vmzNoT X-Received: by 2002:a17:902:205:: with SMTP id 5mr3554637plc.165.1561631306598; Thu, 27 Jun 2019 03:28:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631306; cv=none; d=google.com; s=arc-20160816; b=aTSHcajEV992X8GG55ix4FlmYkzKKLu0ePHvSHRvVHV+OcUJw/kMxNn2idi9ccM58X Xt8yC3pnSSkREqX1IkHgSq9jMMit6IJ5/IrcEFhkiPUbKX1awiEbh2uGmffk9WAnJnGg 8o09hzVcY3Pe2ijyRPJdQByUWIaO5BuSS7Jnp9Ug12PvFXEoHRGLMOhH5iDKC/R8/0gZ ENHDqfJsz43kJjANpJxm3UtwgHeHEfvhtIehdOVYUNqJglavvU8G3Q4vSGBdUx3ZiY1C 9aAV7tkMchJbK3p22SzkQSawJoN9pkYA00Z+ecvv+/euIWDn8pD2UW9tMPJGh6nqH2Uw 0WeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=e0cmL8XP70YBaJ5yxi1SfUhq2Tu3KlK+P3hfuATsmNM=; b=HUww/kDBYRdiC0hAxFAa7y0Q2ctqVAUWgT+PUD7twroqcdgEFVHOommJ07N3RaxA3c b+dbDpQrw6q9UCA0CwWOujD4BCmPTxOpsRHodhBt//S5GOCY7+iVbsSKWipLBcp1+BER Jm9t3zz2dTo+xebfgXQLENHJoLWwfviyOSF8GUJB+18WF+OTqw9yTwAJ2OSyr+Qu6Zfi sGMxexAoFdhFPoKp7lCyh0HWFror7hPEPXimSpYVA/6TiMDciX0OCGoRhRQwLJPs1CY4 BP8EmmEwNrdTutUvhVqT2UEESd8+3LZj7aScIZmqNvDScDC2VTqcU7Qa8bqsd0jI3wHd ti/g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QrJ5irI4; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.26; Thu, 27 Jun 2019 03:28:26 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QrJ5irI4; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726623AbfF0K2Z (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:25 -0400 Received: from mail-wr1-f44.google.com ([209.85.221.44]:45358 "EHLO mail-wr1-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726508AbfF0K2Z (ORCPT ); Thu, 27 Jun 2019 06:28:25 -0400 Received: by mail-wr1-f44.google.com with SMTP id f9so1886663wre.12 for ; Thu, 27 Jun 2019 03:28:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=e0cmL8XP70YBaJ5yxi1SfUhq2Tu3KlK+P3hfuATsmNM=; b=QrJ5irI4oz0I/xDIZtqLkW1aKRg1lfIsJMZybbfy/vvFq8fctxMpdXr5Jf4/44JtYw ldbQd8JYPLEaSWG8zJcBC6Btbf0b0s+etoO5lnroxL+lKWmkz3Cp/jMg+RlQVEUPVja+ C4cEteFjXSl+3WTfYbkWPgt2apAfJSZX7e4h7wLKdZ1NMzomLoMNXWAxiG+S2u8b9kRG gBYKhRjq65nz3wJS2rxt5wGpk6J9WnC9YdmgiwVE7MZ0ksPVAzVeAIjrUlgT4t/oasTA 20coZ8sgB4ZAidKYwEQEgKYiDhMmb9eUv/VeqVA37/To7T+ho5nr+jcPDsXKzkwLD2Mf EREA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e0cmL8XP70YBaJ5yxi1SfUhq2Tu3KlK+P3hfuATsmNM=; b=dydc8DaBcGPis21dyfTrOJZl69IOG2bSrRURkHNFse5ObiLHV62NGsUJLRl+CarFAa SbZcviXK9JIgmJN3Udy9GpQf3+v11dX6h0mDAqKv6oUDQkvl0pTLmy+SkgjluaJ4JLpI LgTTBLVGfDq8tUrl23hI88Ifl2YU2K198AcN+aOdPVU8m8DAwlCN0U6MmQ6HPKSFsaNM 9QFEfYGd6A9JH0xZB/E74M62Glf8g42HKvguhldJNp8E8o7dw4Az62D74Mfs6Z0DcbFX Umt3wdshmxUZ8M1MMNuvQaEbdwFbPVyAmRLzu3DSvNqlBbxoJV/i4saZgHy6HAGO+2GV ZmPg== X-Gm-Message-State: APjAAAXqDfmieP5c4Ouuy/4AOGR3g6dg+B+jaLN/d0PfEgkUi5GOeE7e 2AasHfLOOmY0NkvP8bhctOnpGC9JXws= X-Received: by 2002:a5d:6583:: with SMTP id q3mr2869601wru.184.1561631302730; Thu, 27 Jun 2019 03:28:22 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.21 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:22 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 30/32] crypto: arm/aes-cipher - switch to shared AES inverse Sbox Date: Thu, 27 Jun 2019 12:26:45 +0200 Message-Id: <20190627102647.2992-31-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/aes-cipher-core.S | 40 +------------------- 1 file changed, 1 insertion(+), 39 deletions(-) -- 2.20.1 diff --git a/arch/arm/crypto/aes-cipher-core.S b/arch/arm/crypto/aes-cipher-core.S index f2d67c095e59..180d8555a09c 100644 --- a/arch/arm/crypto/aes-cipher-core.S +++ b/arch/arm/crypto/aes-cipher-core.S @@ -222,43 +222,5 @@ ENDPROC(__aes_arm_encrypt) .align 5 ENTRY(__aes_arm_decrypt) - do_crypt iround, crypto_it_tab, __aes_arm_inverse_sbox, 0 + do_crypt iround, crypto_it_tab, crypto_aes_inv_sbox, 0 ENDPROC(__aes_arm_decrypt) - - .section ".rodata", "a" - .align L1_CACHE_SHIFT - .type __aes_arm_inverse_sbox, %object -__aes_arm_inverse_sbox: - .byte 0x52, 0x09, 0x6a, 0xd5, 0x30, 0x36, 0xa5, 0x38 - .byte 0xbf, 0x40, 0xa3, 0x9e, 0x81, 0xf3, 0xd7, 0xfb - .byte 0x7c, 0xe3, 0x39, 0x82, 0x9b, 0x2f, 0xff, 0x87 - .byte 0x34, 0x8e, 0x43, 0x44, 0xc4, 0xde, 0xe9, 0xcb - .byte 0x54, 0x7b, 0x94, 0x32, 0xa6, 0xc2, 0x23, 0x3d - .byte 0xee, 0x4c, 0x95, 0x0b, 0x42, 0xfa, 0xc3, 0x4e - .byte 0x08, 0x2e, 0xa1, 0x66, 0x28, 0xd9, 0x24, 0xb2 - .byte 0x76, 0x5b, 0xa2, 0x49, 0x6d, 0x8b, 0xd1, 0x25 - .byte 0x72, 0xf8, 0xf6, 0x64, 0x86, 0x68, 0x98, 0x16 - .byte 0xd4, 0xa4, 0x5c, 0xcc, 0x5d, 0x65, 0xb6, 0x92 - .byte 0x6c, 0x70, 0x48, 0x50, 0xfd, 0xed, 0xb9, 0xda - .byte 0x5e, 0x15, 0x46, 0x57, 0xa7, 0x8d, 0x9d, 0x84 - .byte 0x90, 0xd8, 0xab, 0x00, 0x8c, 0xbc, 0xd3, 0x0a - .byte 0xf7, 0xe4, 0x58, 0x05, 0xb8, 0xb3, 0x45, 0x06 - .byte 0xd0, 0x2c, 0x1e, 0x8f, 0xca, 0x3f, 0x0f, 0x02 - .byte 0xc1, 0xaf, 0xbd, 0x03, 0x01, 0x13, 0x8a, 0x6b - .byte 0x3a, 0x91, 0x11, 0x41, 0x4f, 0x67, 0xdc, 0xea - .byte 0x97, 0xf2, 0xcf, 0xce, 0xf0, 0xb4, 0xe6, 0x73 - .byte 0x96, 0xac, 0x74, 0x22, 0xe7, 0xad, 0x35, 0x85 - .byte 0xe2, 0xf9, 0x37, 0xe8, 0x1c, 0x75, 0xdf, 0x6e - .byte 0x47, 0xf1, 0x1a, 0x71, 0x1d, 0x29, 0xc5, 0x89 - .byte 0x6f, 0xb7, 0x62, 0x0e, 0xaa, 0x18, 0xbe, 0x1b - .byte 0xfc, 0x56, 0x3e, 0x4b, 0xc6, 0xd2, 0x79, 0x20 - .byte 0x9a, 0xdb, 0xc0, 0xfe, 0x78, 0xcd, 0x5a, 0xf4 - .byte 0x1f, 0xdd, 0xa8, 0x33, 0x88, 0x07, 0xc7, 0x31 - .byte 0xb1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xec, 0x5f - .byte 0x60, 0x51, 0x7f, 0xa9, 0x19, 0xb5, 0x4a, 0x0d - .byte 0x2d, 0xe5, 0x7a, 0x9f, 0x93, 0xc9, 0x9c, 0xef - .byte 0xa0, 0xe0, 0x3b, 0x4d, 0xae, 0x2a, 0xf5, 0xb0 - .byte 0xc8, 0xeb, 0xbb, 0x3c, 0x83, 0x53, 0x99, 0x61 - .byte 0x17, 0x2b, 0x04, 0x7e, 0xba, 0x77, 0xd6, 0x26 - .byte 0xe1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0c, 0x7d - .size __aes_arm_inverse_sbox, . - __aes_arm_inverse_sbox From patchwork Thu Jun 27 10:26:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167928 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2149014ilk; Thu, 27 Jun 2019 03:28:28 -0700 (PDT) X-Google-Smtp-Source: APXvYqxW0bERU5+bfcZyrC3lDL9Ic6jbg4qdgYaccCxCxhPIZA2+0YUtm6nzGh/lb2/MaCt623oD X-Received: by 2002:a63:e506:: with SMTP id r6mr3118157pgh.324.1561631307989; Thu, 27 Jun 2019 03:28:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631307; cv=none; d=google.com; s=arc-20160816; b=QYEuhmU8+7Dz5IXKil1AdoqxamLj6cOTuctkqW7AVfBey5uEF6mToJbFV+H9hUfNyt mdSzL2bHXS7XB843mF+Y86N38kVhFo2G8lxxAqnK7unFrcquBD8HQXw7YvAeBTUQvl+x tHrtuRaxTjhH4vwjXnIMwExxBK8zoy1VvLkuueYtq9rtLRz/MG2AciSTNvHcmJmnJpfo oYViXqp2gQW/YhK/UVxg0O3r65wTncOXSV+Fc/xQvFt506ZdEuhOtHMw6IR+g4e93tib 58bR9MGiRTOs69MyJGCP/KXPg1VHzJWMM2b84ztby5b2IKo5rkuHNmtbkcQdFl8I81AT V8YQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=LgOxz3M+88gVu7JIGRNWNEXUFNxqJnrZXlO07MCiXP4=; b=slTPI+XpgJkpLb5cBstLPL7BQu8OPBdI7VOkEUjGS/tW9b1sMcBUPA1isAqP1CXhcO 6t2k4xeSGIp/Lwzl+VQg9BfCh2xu6elxsaLPQtl8aYMBl32b6HseiWLXKER+hiMAiDrt /TKjOWPKsODuMgdawdlTzOq34GGEfzof75zET6wicwG+JfIn5T+0+OKhQtRNAgvDNC+3 aoylL52L85eH1GP1wxV5hle7M9ryWl9LDismUlZ3PphZ8DHOwZ9xaaZuZb0auAGK+QGG GyXWlZfj6cCOrAx+VAAj/vBp2BfeX0vecWb3hScta13x/ACz8qlUichYDnGYIRZo+8r6 hUjg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ffLR9+B7; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.27; Thu, 27 Jun 2019 03:28:27 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ffLR9+B7; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726487AbfF0K21 (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:27 -0400 Received: from mail-wr1-f42.google.com ([209.85.221.42]:36610 "EHLO mail-wr1-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726678AbfF0K20 (ORCPT ); Thu, 27 Jun 2019 06:28:26 -0400 Received: by mail-wr1-f42.google.com with SMTP id n4so1935754wrs.3 for ; Thu, 27 Jun 2019 03:28:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LgOxz3M+88gVu7JIGRNWNEXUFNxqJnrZXlO07MCiXP4=; b=ffLR9+B7GcB8IqxtWJlDizIKlinnoboJrNnyqRd1IMlSqNUg5dBUrDR2j51obEqx+P idgeEGkeN8ZtAIhat3fhEB03qmD2i/0ziqm14X1FFTB7EerGh5VDPMnT9iIvkrxtYqEd oXS3LcUthszwe1PC1s4VALHoWHHM6fXYdL8Zc4w7aFzNcUD/5vdVKEB64BFOhkl7RNcp DbmV7k8RPLWvDJQfzdlQLVIQuFY5ls2x+02nPo4oXPCe7X+nzyyQ04vhYOjuV9v0FdD/ 4NImUWNnhLUpLG2q2fyO9FkenF6OaZolSSIxPnH3L4bGpRYmhCEWo1TbgL4oAw/+gHx8 EzAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LgOxz3M+88gVu7JIGRNWNEXUFNxqJnrZXlO07MCiXP4=; b=Y+s30LpkPQcOMN11lcHVIfOrzd70BSyHM0N46uRpmaOcqN77EXudoqkVGtHo6xFwmW A5gx/8dBC3v8mUqdC8GdXGEvgVtvVmNnbpAB1P+pn7PZvQZZYwbKDqPBf8wc/3J35yHG u5kyXsEtZr7dgRDoOSmI578ZmOh3Qcw5NTTfoCI/WcsgHnd9HHNLM4ggGjucXdlxF508 y74ijOIuSP1soHqmwmoZh0d2z1KRR2tvNo9V2ZzhVLJwMA88TiKqVdlH38muattLiM8d zI/fSwGrYCYNrz8FzujxZ/tJDNf/LLSB3hYo5fMrzZPtzk25Xu/UXsjfCnu3/KEiS2Ek 6PaQ== X-Gm-Message-State: APjAAAX6xDWooGgho2jyb9vMe+A3eL9b5CE5MTO2dn+Ft9WUWHUU7Uqn hauIFxitpNR8EVSbu1lg7C+wwlAW0Kg= X-Received: by 2002:adf:e806:: with SMTP id o6mr485789wrm.269.1561631303799; Thu, 27 Jun 2019 03:28:23 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.22 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:23 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 31/32] crypto: arm64/aes-cipher - switch to shared AES inverse Sbox Date: Thu, 27 Jun 2019 12:26:46 +0200 Message-Id: <20190627102647.2992-32-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/aes-cipher-core.S | 40 +------------------- 1 file changed, 1 insertion(+), 39 deletions(-) -- 2.20.1 diff --git a/arch/arm64/crypto/aes-cipher-core.S b/arch/arm64/crypto/aes-cipher-core.S index 3a44eada2347..27dac259b359 100644 --- a/arch/arm64/crypto/aes-cipher-core.S +++ b/arch/arm64/crypto/aes-cipher-core.S @@ -131,43 +131,5 @@ ENDPROC(__aes_arm64_encrypt) .align 5 ENTRY(__aes_arm64_decrypt) - do_crypt iround, crypto_it_tab, __aes_arm64_inverse_sbox, 0 + do_crypt iround, crypto_it_tab, crypto_aes_inv_sbox, 0 ENDPROC(__aes_arm64_decrypt) - - .section ".rodata", "a" - .align L1_CACHE_SHIFT - .type __aes_arm64_inverse_sbox, %object -__aes_arm64_inverse_sbox: - .byte 0x52, 0x09, 0x6a, 0xd5, 0x30, 0x36, 0xa5, 0x38 - .byte 0xbf, 0x40, 0xa3, 0x9e, 0x81, 0xf3, 0xd7, 0xfb - .byte 0x7c, 0xe3, 0x39, 0x82, 0x9b, 0x2f, 0xff, 0x87 - .byte 0x34, 0x8e, 0x43, 0x44, 0xc4, 0xde, 0xe9, 0xcb - .byte 0x54, 0x7b, 0x94, 0x32, 0xa6, 0xc2, 0x23, 0x3d - .byte 0xee, 0x4c, 0x95, 0x0b, 0x42, 0xfa, 0xc3, 0x4e - .byte 0x08, 0x2e, 0xa1, 0x66, 0x28, 0xd9, 0x24, 0xb2 - .byte 0x76, 0x5b, 0xa2, 0x49, 0x6d, 0x8b, 0xd1, 0x25 - .byte 0x72, 0xf8, 0xf6, 0x64, 0x86, 0x68, 0x98, 0x16 - .byte 0xd4, 0xa4, 0x5c, 0xcc, 0x5d, 0x65, 0xb6, 0x92 - .byte 0x6c, 0x70, 0x48, 0x50, 0xfd, 0xed, 0xb9, 0xda - .byte 0x5e, 0x15, 0x46, 0x57, 0xa7, 0x8d, 0x9d, 0x84 - .byte 0x90, 0xd8, 0xab, 0x00, 0x8c, 0xbc, 0xd3, 0x0a - .byte 0xf7, 0xe4, 0x58, 0x05, 0xb8, 0xb3, 0x45, 0x06 - .byte 0xd0, 0x2c, 0x1e, 0x8f, 0xca, 0x3f, 0x0f, 0x02 - .byte 0xc1, 0xaf, 0xbd, 0x03, 0x01, 0x13, 0x8a, 0x6b - .byte 0x3a, 0x91, 0x11, 0x41, 0x4f, 0x67, 0xdc, 0xea - .byte 0x97, 0xf2, 0xcf, 0xce, 0xf0, 0xb4, 0xe6, 0x73 - .byte 0x96, 0xac, 0x74, 0x22, 0xe7, 0xad, 0x35, 0x85 - .byte 0xe2, 0xf9, 0x37, 0xe8, 0x1c, 0x75, 0xdf, 0x6e - .byte 0x47, 0xf1, 0x1a, 0x71, 0x1d, 0x29, 0xc5, 0x89 - .byte 0x6f, 0xb7, 0x62, 0x0e, 0xaa, 0x18, 0xbe, 0x1b - .byte 0xfc, 0x56, 0x3e, 0x4b, 0xc6, 0xd2, 0x79, 0x20 - .byte 0x9a, 0xdb, 0xc0, 0xfe, 0x78, 0xcd, 0x5a, 0xf4 - .byte 0x1f, 0xdd, 0xa8, 0x33, 0x88, 0x07, 0xc7, 0x31 - .byte 0xb1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xec, 0x5f - .byte 0x60, 0x51, 0x7f, 0xa9, 0x19, 0xb5, 0x4a, 0x0d - .byte 0x2d, 0xe5, 0x7a, 0x9f, 0x93, 0xc9, 0x9c, 0xef - .byte 0xa0, 0xe0, 0x3b, 0x4d, 0xae, 0x2a, 0xf5, 0xb0 - .byte 0xc8, 0xeb, 0xbb, 0x3c, 0x83, 0x53, 0x99, 0x61 - .byte 0x17, 0x2b, 0x04, 0x7e, 0xba, 0x77, 0xd6, 0x26 - .byte 0xe1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0c, 0x7d - .size __aes_arm64_inverse_sbox, . - __aes_arm64_inverse_sbox From patchwork Thu Jun 27 10:26:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167929 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp2149018ilk; Thu, 27 Jun 2019 03:28:28 -0700 (PDT) X-Google-Smtp-Source: APXvYqz/nXdknUPTrLqE+owlzYw7F8udCvfDVet4jbbqQnqPlnH8LRcpGcGPIYu5MzrJySlme395 X-Received: by 2002:a17:90a:9382:: with SMTP id q2mr5154727pjo.131.1561631308328; Thu, 27 Jun 2019 03:28:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561631308; cv=none; d=google.com; s=arc-20160816; b=qdxZj43jYu+h+MopyddrnXfNsUn88UMeCY28L/DxbfXKMhP4VUyZaWcksyYYWOmcAD DnyCqL37/V96rJIX/GeJVMUbtXn6Do5B0XtLqXutg8R3BIdhiTJ9epBjuLrXAc3+n8x1 78mf9k7KxSz/09P5xiagD1GMpXLA9kSVJvKmXmEJt2IwDUHP3yeEV+SX5qenCs2jmN0a VGDs5LoRDYU+dZ4YKc52MhLI02DRIjGli1tFouj8In79btGV9KWlPhqoW99dlWSM2sjh x2M/V7MbPGt2cKwH1dD4J60YLOZtYB+D4B8/nN+9sOYKIZdPgJXwJwm/fgXbLfhLuZcL +unQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=4lc0G+siZ09AfrMnEtgKbgVUS7Kj+FyziWNUdbDRXLk=; b=LccKb7TJ49C3yUzBR/42Odm1OFX/SDZ1S6Y6l8KmvfWYPO8cx4Y45dzSGT/VhsvYCR 7xSNuuWZm1f6snZ4GE/7qVhJQNvdnhWEpn+I2KHJ2kS2xnyFp+8Y8ngCvyP/vhj0ZG7p M3QL86dZ3IxyIWZvfF0rHdHJg5ql4ugfnP3Huk8FRkorx1q4rf34sn/XCOpKrQOCw/dq ChAERC6z6ZYuynEQ9BmlesAkcO1QIkHISmDDQ+iE/yI8U+sI0KtLh7p714zvtIsslf1+ GCQha2kGaVJVU9OiB9tNtLlk6wrjiLG9yUqp7ryCIv8hJCUKrncJ6oUSizMukf5O61fV +VyQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fc+jtIn6; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f32si4829522pjg.42.2019.06.27.03.28.28; Thu, 27 Jun 2019 03:28:28 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fc+jtIn6; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726678AbfF0K21 (ORCPT + 3 others); Thu, 27 Jun 2019 06:28:27 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:43811 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726508AbfF0K21 (ORCPT ); Thu, 27 Jun 2019 06:28:27 -0400 Received: by mail-wr1-f65.google.com with SMTP id p13so1901945wru.10 for ; Thu, 27 Jun 2019 03:28:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4lc0G+siZ09AfrMnEtgKbgVUS7Kj+FyziWNUdbDRXLk=; b=fc+jtIn6IYU6NPWid6bB6S68xSHoEFE3Gh4BV2VII3662RVCVZy0onW6zfxIbjm23w Ud2+yAEIDk5130j2ucMOkQJaKqIp4LQwMQJf+MNbAxQLvVZFFCJdCk3ZqtDu1S27qcLo Locc1F4T2UNlXjuftpHv9w2ltSkZtjNLSGBZmHAaETg8AE8PP1y0CiTj4SQ0CjCVpL/b POQZ6fEPC/+INGNa1DNl/ApI+aHvA5bkXQpTQhXRnPc0EdBr934MhhW4BEm5+doYD/3K 1MEn02e7WOZsjg+yfMZWdLRuCtUghw5QLhOkZO4yTbwp5J99wJ6l4RTOHmj+JCx14+/Q uxRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4lc0G+siZ09AfrMnEtgKbgVUS7Kj+FyziWNUdbDRXLk=; b=IkQ93EscIBHyFBf4uSpJMPltV3UN3lzHUo8WGobmQkgyN8+DsjUDmE1SlUxynDAkk5 SR5AycK5jXlZbPo94LgPuAjNOyQgeL9OhS7NxW9Q5QSX9kM1BO0h3EOIJdmog0bdW+46 DL8JHXgoxbB0Zq26xXe4wlKaz9vo6SXyA51Gnt2hGMyC4JZqj4h1tqKpsqxvd468KDvv //X6GEw/PqMTOdmT0VvZCvVHc7BlP0DUbShb4kQo06ZpBZwLRbMCD8+SnBX/ZP5NmmPc OT2HkXNFP8lHkjBupg2K8ds3Zis7Q7vQRtLrBYYPC6LovIKMPERg8VzbH2Xa/KlM7oV2 JjAA== X-Gm-Message-State: APjAAAWbZEd132xLzxmkheW6bxqu6ZbFjone80lHK8FIKN3IX7ZkQI5L lTuRiJxTDiHU9qjkkiKvByCeq1XGPlo= X-Received: by 2002:a05:6000:112:: with SMTP id o18mr2793380wrx.153.1561631304993; Thu, 27 Jun 2019 03:28:24 -0700 (PDT) Received: from localhost.localdomain (aaubervilliers-681-1-8-173.w90-88.abo.wanadoo.fr. [90.88.13.173]) by smtp.gmail.com with ESMTPSA id g2sm5584533wmh.0.2019.06.27.03.28.23 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 27 Jun 2019 03:28:24 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@kernel.org, Ard Biesheuvel Subject: [PATCH v3 32/32] crypto: arm/aes-scalar - unexport en/decryption routines Date: Thu, 27 Jun 2019 12:26:47 +0200 Message-Id: <20190627102647.2992-33-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190627102647.2992-1-ard.biesheuvel@linaro.org> References: <20190627102647.2992-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The scalar table based AES routines are not used by other drivers, so let's keep it that way and unexport the symbols. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/aes-cipher-glue.c | 3 --- 1 file changed, 3 deletions(-) -- 2.20.1 diff --git a/arch/arm/crypto/aes-cipher-glue.c b/arch/arm/crypto/aes-cipher-glue.c index f6c07867b8ff..26a2b81c2c12 100644 --- a/arch/arm/crypto/aes-cipher-glue.c +++ b/arch/arm/crypto/aes-cipher-glue.c @@ -14,10 +14,7 @@ #include asmlinkage void __aes_arm_encrypt(u32 *rk, int rounds, const u8 *in, u8 *out); -EXPORT_SYMBOL(__aes_arm_encrypt); - asmlinkage void __aes_arm_decrypt(u32 *rk, int rounds, const u8 *in, u8 *out); -EXPORT_SYMBOL(__aes_arm_decrypt); static void aes_arm_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) {