From patchwork Tue Nov 5 14:01:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tero Kristo X-Patchwork-Id: 178515 Delivered-To: patch@linaro.org Received: by 2002:ac9:3c86:0:0:0:0:0 with SMTP id w6csp710171ocf; Tue, 5 Nov 2019 06:02:05 -0800 (PST) X-Google-Smtp-Source: APXvYqxCmXbEqvoe+mG1TobM3lCJOlOKcKgmJRIfnjmx5k2Db1OvZnVIeKewtP14Rm2BxU5NuR1O X-Received: by 2002:ac2:5328:: with SMTP id f8mr20961494lfh.128.1572962525401; Tue, 05 Nov 2019 06:02:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1572962525; cv=none; d=google.com; s=arc-20160816; b=gtWtcEU5TC7+3A5L6M0TbcrLpCttQPYQchvCxtY1RsmB6Gz48fzODyI+Xd8BMRl8Hp 8dKncNBELDXdqIvn5kfheA/2vpt9s09U4o+vz5BirtC3Ju41Ia6ABt4aJdzX6+HT0AlE ++lNd2Rskt04qyQz9yWQfJ9EZcPDCQW8eU5DcgYi8HTGpuhiZ0Zy+jiWDW0Rg+trfXA8 ZKSG8ppmHcoCrwt1skLG6j8oETsZHeXrRsaaPv5BloPHYl8FqhTdPiA/9JmNabzON2UP Tv/+rPqEBlc+NKn5DFiWfhxbNHxWwmaA2aanIvKxqYu1J49k7E9s0/+JEs1Nawr7rbZ0 vZYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=t3VIb0F0Yu7RigcFIoRIYL9QulN0D+VeUxrnCGq/nrE=; b=De+9/hJ3ZORxBKNZDRPU4OdHwxrw6F9PebuSSK1Jb81d/Lz+LRlaG9FgqfmbPQJPp5 fGzErriysKebRr+7EB/ScXS0oWYmSY8+NAKgzt/MdgSiTQtPe00vWqDLA6TWA7bIbWJ9 5KdQszVwPhd4RjULoK97k8CfN1NS6iWLgiedZ0Reh/TJHxz7TY2yG5lS96UvNaU1l3ms 3u7/wAi3mjfGsSHhkcJ/SyFvB2aW5f/h4maupZeVUq9MhrPFXUnBHwNXuMw+mTvDzZtP ww42Fv6ysSZIlZkKoWiNjAIZoW5bjwTGmzMav9j4w9R/jER36hXeSic8A5TZDp2syaza oAEA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ti.com header.s=ti-com-17Q1 header.b=bHCWgrJb; spf=pass (google.com: best guess record for domain of linux-omap-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-omap-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=NONE dis=NONE) header.from=ti.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id dk11si13340421ejb.97.2019.11.05.06.02.04; Tue, 05 Nov 2019 06:02:05 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-omap-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@ti.com header.s=ti-com-17Q1 header.b=bHCWgrJb; spf=pass (google.com: best guess record for domain of linux-omap-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-omap-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=NONE dis=NONE) header.from=ti.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2389445AbfKEOCE (ORCPT + 5 others); Tue, 5 Nov 2019 09:02:04 -0500 Received: from lelv0143.ext.ti.com ([198.47.23.248]:35242 "EHLO lelv0143.ext.ti.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2389435AbfKEOCE (ORCPT ); Tue, 5 Nov 2019 09:02:04 -0500 Received: from lelv0265.itg.ti.com ([10.180.67.224]) by lelv0143.ext.ti.com (8.15.2/8.15.2) with ESMTP id xA5E1wBq038266; Tue, 5 Nov 2019 08:01:58 -0600 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=ti-com-17Q1; t=1572962518; bh=t3VIb0F0Yu7RigcFIoRIYL9QulN0D+VeUxrnCGq/nrE=; h=From:To:CC:Subject:Date:In-Reply-To:References; b=bHCWgrJb0xcc5yjLReD/v3pNsJFMYg+JkPEGv7k4a45sn3i4hmq4qaRKUVUKncMN8 pKLlS8tVVHUTOC56PwS40ihG2qMvaVbqIKI61h/nNYTWMiS0krzJaFkkDcdzhA9Qjb VFJWLz/W8I0PDVcbg2+vlSWyEGK6Cqk9QNsj2LV8= Received: from DFLE108.ent.ti.com (dfle108.ent.ti.com [10.64.6.29]) by lelv0265.itg.ti.com (8.15.2/8.15.2) with ESMTPS id xA5E1w36008332 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=FAIL); Tue, 5 Nov 2019 08:01:58 -0600 Received: from DFLE111.ent.ti.com (10.64.6.32) by DFLE108.ent.ti.com (10.64.6.29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1847.3; Tue, 5 Nov 2019 08:01:43 -0600 Received: from fllv0040.itg.ti.com (10.64.41.20) by DFLE111.ent.ti.com (10.64.6.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1713.5 via Frontend Transport; Tue, 5 Nov 2019 08:01:43 -0600 Received: from sokoban.ti.com (ileax41-snat.itg.ti.com [10.172.224.153]) by fllv0040.itg.ti.com (8.15.2/8.15.2) with ESMTP id xA5E1FFw068289; Tue, 5 Nov 2019 08:01:56 -0600 From: Tero Kristo To: , , CC: , Subject: [PATCHv2 19/22] crypto: omap-aes-gcm: convert to use crypto engine Date: Tue, 5 Nov 2019 16:01:08 +0200 Message-ID: <20191105140111.20285-20-t-kristo@ti.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191105140111.20285-1-t-kristo@ti.com> References: <20191105140111.20285-1-t-kristo@ti.com> MIME-Version: 1.0 X-EXCLAIMER-MD-CONFIG: e1e8a2fd-e40a-4ac6-ac9b-f7e9cc9ee180 Sender: linux-omap-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-omap@vger.kernel.org Currently omap-aes-gcm algorithms are using local implementation for crypto request queuing logic. Instead, implement this via usage of crypto engine which is used already for rest of the omap aes algorithms. This avoids some random conflicts / crashes also which can happen if both aes and aes-gcm are attempted to be used simultaneously. Signed-off-by: Tero Kristo --- drivers/crypto/omap-aes-gcm.c | 98 +++++++++++++++++++---------------- drivers/crypto/omap-aes.c | 23 -------- drivers/crypto/omap-aes.h | 2 +- 3 files changed, 55 insertions(+), 68 deletions(-) -- 2.17.1 -- Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki diff --git a/drivers/crypto/omap-aes-gcm.c b/drivers/crypto/omap-aes-gcm.c index e92000846f16..32dc00dc570b 100644 --- a/drivers/crypto/omap-aes-gcm.c +++ b/drivers/crypto/omap-aes-gcm.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -29,11 +30,13 @@ static void omap_aes_gcm_finish_req(struct omap_aes_dev *dd, int ret) { struct aead_request *req = dd->aead_req; - dd->flags &= ~FLAGS_BUSY; dd->in_sg = NULL; dd->out_sg = NULL; - req->base.complete(&req->base, ret); + crypto_finalize_aead_request(dd->engine, req, ret); + + pm_runtime_mark_last_busy(dd->dev); + pm_runtime_put_autosuspend(dd->dev); } static void omap_aes_gcm_done_task(struct omap_aes_dev *dd) @@ -81,7 +84,6 @@ static void omap_aes_gcm_done_task(struct omap_aes_dev *dd) } omap_aes_gcm_finish_req(dd, ret); - omap_aes_gcm_handle_queue(dd, NULL); } static int omap_aes_gcm_copy_buffers(struct omap_aes_dev *dd, @@ -127,6 +129,9 @@ static int omap_aes_gcm_copy_buffers(struct omap_aes_dev *dd, if (cryptlen) { tmp = scatterwalk_ffwd(sg_arr, req->src, req->assoclen); + if (nsg) + sg_unmark_end(dd->in_sgl); + ret = omap_crypto_align_sg(&tmp, cryptlen, AES_BLOCK_SIZE, &dd->in_sgl[nsg], OMAP_CRYPTO_COPY_DATA | @@ -146,7 +151,7 @@ static int omap_aes_gcm_copy_buffers(struct omap_aes_dev *dd, dd->out_sg = req->dst; dd->orig_out = req->dst; - dd->out_sg = scatterwalk_ffwd(sg_arr, req->dst, assoclen); + dd->out_sg = scatterwalk_ffwd(sg_arr, req->dst, req->assoclen); flags = 0; if (req->src == req->dst || dd->out_sg == sg_arr) @@ -202,37 +207,21 @@ void omap_aes_gcm_dma_out_callback(void *data) static int omap_aes_gcm_handle_queue(struct omap_aes_dev *dd, struct aead_request *req) { - struct omap_aes_gcm_ctx *ctx; - struct aead_request *backlog; - struct omap_aes_reqctx *rctx; - unsigned long flags; - int err, ret = 0; - - spin_lock_irqsave(&dd->lock, flags); - if (req) - ret = aead_enqueue_request(&dd->aead_queue, req); - if (dd->flags & FLAGS_BUSY) { - spin_unlock_irqrestore(&dd->lock, flags); - return ret; - } - - backlog = aead_get_backlog(&dd->aead_queue); - req = aead_dequeue_request(&dd->aead_queue); if (req) - dd->flags |= FLAGS_BUSY; - spin_unlock_irqrestore(&dd->lock, flags); - - if (!req) - return ret; + return crypto_transfer_aead_request_to_engine(dd->engine, req); - if (backlog) - backlog->base.complete(&backlog->base, -EINPROGRESS); + return 0; +} - ctx = crypto_aead_ctx(crypto_aead_reqtfm(req)); - rctx = aead_request_ctx(req); +static int omap_aes_gcm_prepare_req(struct crypto_engine *engine, void *areq) +{ + struct aead_request *req = container_of(areq, struct aead_request, + base); + struct omap_aes_reqctx *rctx = aead_request_ctx(req); + struct omap_aes_dev *dd = rctx->dd; + struct omap_aes_gcm_ctx *ctx = crypto_aead_ctx(crypto_aead_reqtfm(req)); + int err; - dd->ctx = &ctx->octx; - rctx->dd = dd; dd->aead_req = req; rctx->mode &= FLAGS_MODE_MASK; @@ -242,20 +231,9 @@ static int omap_aes_gcm_handle_queue(struct omap_aes_dev *dd, if (err) return err; - err = omap_aes_write_ctrl(dd); - if (!err) { - if (dd->in_sg_len) - err = omap_aes_crypt_dma_start(dd); - else - omap_aes_gcm_dma_out_callback(dd); - } - - if (err) { - omap_aes_gcm_finish_req(dd, err); - omap_aes_gcm_handle_queue(dd, NULL); - } + dd->ctx = &ctx->octx; - return ret; + return omap_aes_write_ctrl(dd); } static int omap_aes_gcm_crypt(struct aead_request *req, unsigned long mode) @@ -378,3 +356,35 @@ int omap_aes_4106gcm_setauthsize(struct crypto_aead *parent, { return crypto_rfc4106_check_authsize(authsize); } + +static int omap_aes_gcm_crypt_req(struct crypto_engine *engine, void *areq) +{ + struct aead_request *req = container_of(areq, struct aead_request, + base); + struct omap_aes_reqctx *rctx = aead_request_ctx(req); + struct omap_aes_dev *dd = rctx->dd; + int ret = 0; + + if (!dd) + return -ENODEV; + + if (dd->in_sg_len) + ret = omap_aes_crypt_dma_start(dd); + else + omap_aes_gcm_dma_out_callback(dd); + + return ret; +} + +int omap_aes_gcm_cra_init(struct crypto_aead *tfm) +{ + struct omap_aes_ctx *ctx = crypto_aead_ctx(tfm); + + ctx->enginectx.op.prepare_request = omap_aes_gcm_prepare_req; + ctx->enginectx.op.unprepare_request = NULL; + ctx->enginectx.op.do_one_request = omap_aes_gcm_crypt_req; + + crypto_aead_set_reqsize(tfm, sizeof(struct omap_aes_reqctx)); + + return 0; +} diff --git a/drivers/crypto/omap-aes.c b/drivers/crypto/omap-aes.c index 758c93908fa5..824ddf2a66ff 100644 --- a/drivers/crypto/omap-aes.c +++ b/drivers/crypto/omap-aes.c @@ -657,29 +657,6 @@ static int omap_aes_init_tfm(struct crypto_skcipher *tfm) return 0; } -static int omap_aes_gcm_cra_init(struct crypto_aead *tfm) -{ - struct omap_aes_dev *dd = NULL; - int err; - - /* Find AES device, currently picks the first device */ - spin_lock_bh(&list_lock); - list_for_each_entry(dd, &dev_list, list) { - break; - } - spin_unlock_bh(&list_lock); - - err = pm_runtime_get_sync(dd->dev); - if (err < 0) { - dev_err(dd->dev, "%s: failed to get_sync(%d)\n", - __func__, err); - return err; - } - - tfm->reqsize = sizeof(struct omap_aes_reqctx); - return 0; -} - static void omap_aes_exit_tfm(struct crypto_skcipher *tfm) { struct omap_aes_ctx *ctx = crypto_skcipher_ctx(tfm); diff --git a/drivers/crypto/omap-aes.h b/drivers/crypto/omap-aes.h index b89d2e673699..2d111bf906e1 100644 --- a/drivers/crypto/omap-aes.h +++ b/drivers/crypto/omap-aes.h @@ -80,7 +80,6 @@ #define FLAGS_INIT BIT(5) #define FLAGS_FAST BIT(6) -#define FLAGS_BUSY BIT(7) #define FLAGS_IN_DATA_ST_SHIFT 8 #define FLAGS_OUT_DATA_ST_SHIFT 10 @@ -212,6 +211,7 @@ int omap_aes_4106gcm_encrypt(struct aead_request *req); int omap_aes_4106gcm_decrypt(struct aead_request *req); int omap_aes_4106gcm_setauthsize(struct crypto_aead *parent, unsigned int authsize); +int omap_aes_gcm_cra_init(struct crypto_aead *tfm); int omap_aes_write_ctrl(struct omap_aes_dev *dd); int omap_aes_crypt_dma_start(struct omap_aes_dev *dd); int omap_aes_crypt_dma_stop(struct omap_aes_dev *dd);