From patchwork Fri Mar 14 12:22:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 873720 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E04E1FECD8; Fri, 14 Mar 2025 12:22:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954975; cv=none; b=D4h8Bin0gjqWdt8fm/XltRtWeWEL6BvQOh8pBr2dn23k5+b6UTye/cSQpWKGDbZcV0QehjuqMrks1wlYYVHEIaa1IwAUI8lHctb0t7moI9W0mlsDpZIFhyiaqGGex7gx3uKQVwR/l/Laru2akm8do2yD8QtNPYUP5LijYgeLCD0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954975; c=relaxed/simple; bh=Ytw2m7sDqhddLR3kuWQDrcpgCQy5SNX9aFLD4DY3eNo=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To:Cc; b=O1aMBxJbzczl9ow/NILJNQ0olIjQ93/HiY5r8bDuSet5cIcDQsDCKm0PG4MceDPBLX5mgCPCvbI3NmoZywW/CGIFTr11+z87zzl+Iy4GIWLjuPOxCY1Ay1+4C4XBxVTwoF2QpI8/sFCJsyIeE8WBH9PqK/zq+fnuVo/C+6S7zYY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=gRQlh9ID; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="gRQlh9ID" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=Cc:To:Subject:From:References:In-Reply-To:Message-Id:Date: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=a3l+91BKzsddVZxNXDZKRTkrBuBQtx6ec/a2AoHM33E=; b=gRQlh9ID/YsD2UlOWTIJP1KiyJ N2eGWAG3QMqu7j/lnBMKOGptvG4QBSpUZQmu6BNIUvNqbG0Oi2W6RMZhDIn70rYrTJobU4na3UKqi AlN1mOC1rVuGHzniLIu96RFH8oo9kfu6bPkSl/nJnAeiRI/WqfWKc7wNSbp1aUKIRW+hZruHpW4X2 VsN/2ThFuteg0h7mBraPPCrV8vY0lXp1C2So3/N3DOMrYwJOw3Ht/jhyyhPGgZxF4FlKeR2vgnFed 19YI8q056Lm7rLgAgPbjJL4WDg8POU9YCQaTZlTu1nGiWjOrms4Gwr65UcZrGPahmXQILYm41fzlv /Dp/omlw==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1tt43P-006ZlG-1H; Fri, 14 Mar 2025 20:22:20 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Fri, 14 Mar 2025 20:22:19 +0800 Date: Fri, 14 Mar 2025 20:22:19 +0800 Message-Id: <09715f69801427929354fac5b09929db200c1e92.1741954523.git.herbert@gondor.apana.org.au> In-Reply-To: References: From: Herbert Xu Subject: [v4 PATCH 01/13] crypto: qat - Remove dst_null support To: Linux Crypto Mailing List Cc: Richard Weinberger , Zhihao Cheng , linux-mtd@lists.infradead.org, "Rafael J. Wysocki" , Pavel Machek , linux-pm@vger.kernel.org, Steffen Klassert , netdev@vger.kernel.org Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Remove the unused dst_null support. Signed-off-by: Herbert Xu --- .../intel/qat/qat_common/qat_comp_algs.c | 83 ------------------- 1 file changed, 83 deletions(-) diff --git a/drivers/crypto/intel/qat/qat_common/qat_comp_algs.c b/drivers/crypto/intel/qat/qat_common/qat_comp_algs.c index 2ba4aa22e092..9d5848e28ff8 100644 --- a/drivers/crypto/intel/qat/qat_common/qat_comp_algs.c +++ b/drivers/crypto/intel/qat/qat_common/qat_comp_algs.c @@ -29,11 +29,6 @@ struct qat_compression_ctx { int (*qat_comp_callback)(struct qat_compression_req *qat_req, void *resp); }; -struct qat_dst { - bool is_null; - int resubmitted; -}; - struct qat_compression_req { u8 req[QAT_COMP_REQ_SIZE]; struct qat_compression_ctx *qat_compression_ctx; @@ -42,8 +37,6 @@ struct qat_compression_req { enum direction dir; int actual_dlen; struct qat_alg_req alg_req; - struct work_struct resubmit; - struct qat_dst dst; }; static int qat_alg_send_dc_message(struct qat_compression_req *qat_req, @@ -60,46 +53,6 @@ static int qat_alg_send_dc_message(struct qat_compression_req *qat_req, return qat_alg_send_message(alg_req); } -static void qat_comp_resubmit(struct work_struct *work) -{ - struct qat_compression_req *qat_req = - container_of(work, struct qat_compression_req, resubmit); - struct qat_compression_ctx *ctx = qat_req->qat_compression_ctx; - struct adf_accel_dev *accel_dev = ctx->inst->accel_dev; - struct qat_request_buffs *qat_bufs = &qat_req->buf; - struct qat_compression_instance *inst = ctx->inst; - struct acomp_req *areq = qat_req->acompress_req; - struct crypto_acomp *tfm = crypto_acomp_reqtfm(areq); - unsigned int dlen = CRYPTO_ACOMP_DST_MAX; - u8 *req = qat_req->req; - dma_addr_t dfbuf; - int ret; - - areq->dlen = dlen; - - dev_dbg(&GET_DEV(accel_dev), "[%s][%s] retry NULL dst request - dlen = %d\n", - crypto_tfm_alg_driver_name(crypto_acomp_tfm(tfm)), - qat_req->dir == COMPRESSION ? "comp" : "decomp", dlen); - - ret = qat_bl_realloc_map_new_dst(accel_dev, &areq->dst, dlen, qat_bufs, - qat_algs_alloc_flags(&areq->base)); - if (ret) - goto err; - - qat_req->dst.resubmitted = true; - - dfbuf = qat_req->buf.bloutp; - qat_comp_override_dst(req, dfbuf, dlen); - - ret = qat_alg_send_dc_message(qat_req, inst, &areq->base); - if (ret != -ENOSPC) - return; - -err: - qat_bl_free_bufl(accel_dev, qat_bufs); - acomp_request_complete(areq, ret); -} - static void qat_comp_generic_callback(struct qat_compression_req *qat_req, void *resp) { @@ -131,21 +84,6 @@ static void qat_comp_generic_callback(struct qat_compression_req *qat_req, areq->dlen = 0; - if (qat_req->dir == DECOMPRESSION && qat_req->dst.is_null) { - if (cmp_err == ERR_CODE_OVERFLOW_ERROR) { - if (qat_req->dst.resubmitted) { - dev_dbg(&GET_DEV(accel_dev), - "Output does not fit destination buffer\n"); - res = -EOVERFLOW; - goto end; - } - - INIT_WORK(&qat_req->resubmit, qat_comp_resubmit); - adf_misc_wq_queue_work(&qat_req->resubmit); - return; - } - } - if (unlikely(status != ICP_QAT_FW_COMN_STATUS_FLAG_OK)) goto end; @@ -248,26 +186,6 @@ static int qat_comp_alg_compress_decompress(struct acomp_req *areq, enum directi if (areq->dst && !dlen) return -EINVAL; - qat_req->dst.is_null = false; - - /* Handle acomp requests that require the allocation of a destination - * buffer. The size of the destination buffer is double the source - * buffer (rounded up to the size of a page) to fit the decompressed - * output or an expansion on the data for compression. - */ - if (!areq->dst) { - qat_req->dst.is_null = true; - - dlen = round_up(2 * slen, PAGE_SIZE); - areq->dst = sgl_alloc(dlen, f, NULL); - if (!areq->dst) - return -ENOMEM; - - dlen -= dhdr + dftr; - areq->dlen = dlen; - qat_req->dst.resubmitted = false; - } - if (dir == COMPRESSION) { params.extra_dst_buff = inst->dc_data->ovf_buff_p; ovf_buff_sz = inst->dc_data->ovf_buff_sz; @@ -329,7 +247,6 @@ static struct acomp_alg qat_acomp[] = { { .exit = qat_comp_alg_exit_tfm, .compress = qat_comp_alg_compress, .decompress = qat_comp_alg_decompress, - .dst_free = sgl_free, .reqsize = sizeof(struct qat_compression_req), }}; From patchwork Fri Mar 14 12:22:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 873716 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00FE1200126; Fri, 14 Mar 2025 12:22:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954981; cv=none; b=XmlKrfk+dvoa1OBA7dN0jd/4Bbt0iQH6KPl3PV5GvAsVIlIeCzn4eh9Cu98u93XV2aV9UiuTeyaIhSo0V74KtjKgWUwJ6qvIAYz5gxJ1R19dpkpwd0gcF+BoNd8bDDc6gJUPu/xVjeMJoZr1mEHae5cZrYRSFVzOLcRDHQPP5/E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954981; c=relaxed/simple; bh=33TF7qV3MJNup+WUJRFG6X8MWWHfFANR2Z4zq3ZTZls=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To:Cc; b=q3Khn+goRgBBNF4do1AoF5+KGjo72Gx4QSe68F0rUvbEio+2aNYpisHapa5vfbfGTrwxCmheqXZHjniXGaSQJ9Fu7/LYGNYFsX9sdflqCmQ7dpHCszE2PxaRLqmKslblIOT0ioz/69LLAXSHnMRKHoHVrDtOZ/5Hv0jqGa/v4zw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=Eadb0vr+; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="Eadb0vr+" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=Cc:To:Subject:From:References:In-Reply-To:Message-Id:Date: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=c5J7R93VbWY8eXPMwB3c29c2QRrhSxoIY0/VZKY8uW4=; b=Eadb0vr+XyYoIWZ2r3FnYPgX9E C+eX67M5h+WDttdAs887y00tXxCpON8uK0RffY+8GZzFFAmWBzMJ2jU4hKtPpOZVWDddTn4oJyaKl 5EmliWy7nyBWp2Ylih3iRGmwanVhLsDUNZCz99h9qnaN5fuJiev13oy/dainbEKgY2OML4lD52s3J rNl0AJQH/SaewrnFd9N0YlxmLLrbeo3ZrX/H9Bo7tYJZOr+KmnCNgJrsquwJTuQSI83pbBDoUa+fQ ln17tewSqOYb8H31AZKmj0pavf+FSYYEwP4w5Ow0tg18qOpaBQcpz45XIjRrMcEHuGQEHPU1WbDdu OYaqKXIA==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1tt43U-006Zld-0D; Fri, 14 Mar 2025 20:22:25 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Fri, 14 Mar 2025 20:22:24 +0800 Date: Fri, 14 Mar 2025 20:22:24 +0800 Message-Id: <11cc12d83023ca382492fc2b2ebf45c84e37acf9.1741954523.git.herbert@gondor.apana.org.au> In-Reply-To: References: From: Herbert Xu Subject: [v4 PATCH 02/13] crypto: iaa - Remove dst_null support To: Linux Crypto Mailing List Cc: Richard Weinberger , Zhihao Cheng , linux-mtd@lists.infradead.org, "Rafael J. Wysocki" , Pavel Machek , linux-pm@vger.kernel.org, Steffen Klassert , netdev@vger.kernel.org Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Remove the unused dst_null support. Signed-off-by: Herbert Xu --- drivers/crypto/intel/iaa/iaa_crypto_main.c | 136 +-------------------- 1 file changed, 6 insertions(+), 130 deletions(-) diff --git a/drivers/crypto/intel/iaa/iaa_crypto_main.c b/drivers/crypto/intel/iaa/iaa_crypto_main.c index 990ea46955bb..50cb100bf1c8 100644 --- a/drivers/crypto/intel/iaa/iaa_crypto_main.c +++ b/drivers/crypto/intel/iaa/iaa_crypto_main.c @@ -1136,8 +1136,7 @@ static int iaa_compress(struct crypto_tfm *tfm, struct acomp_req *req, struct idxd_wq *wq, dma_addr_t src_addr, unsigned int slen, dma_addr_t dst_addr, unsigned int *dlen, - u32 *compression_crc, - bool disable_async) + u32 *compression_crc) { struct iaa_device_compression_mode *active_compression_mode; struct iaa_compression_ctx *ctx = crypto_tfm_ctx(tfm); @@ -1180,7 +1179,7 @@ static int iaa_compress(struct crypto_tfm *tfm, struct acomp_req *req, desc->src2_size = sizeof(struct aecs_comp_table_record); desc->completion_addr = idxd_desc->compl_dma; - if (ctx->use_irq && !disable_async) { + if (ctx->use_irq) { desc->flags |= IDXD_OP_FLAG_RCI; idxd_desc->crypto.req = req; @@ -1193,7 +1192,7 @@ static int iaa_compress(struct crypto_tfm *tfm, struct acomp_req *req, " src_addr %llx, dst_addr %llx\n", __func__, active_compression_mode->name, src_addr, dst_addr); - } else if (ctx->async_mode && !disable_async) + } else if (ctx->async_mode) req->base.data = idxd_desc; dev_dbg(dev, "%s: compression mode %s," @@ -1214,7 +1213,7 @@ static int iaa_compress(struct crypto_tfm *tfm, struct acomp_req *req, update_total_comp_calls(); update_wq_comp_calls(wq); - if (ctx->async_mode && !disable_async) { + if (ctx->async_mode) { ret = -EINPROGRESS; dev_dbg(dev, "%s: returning -EINPROGRESS\n", __func__); goto out; @@ -1234,7 +1233,7 @@ static int iaa_compress(struct crypto_tfm *tfm, struct acomp_req *req, *compression_crc = idxd_desc->iax_completion->crc; - if (!ctx->async_mode || disable_async) + if (!ctx->async_mode) idxd_free_desc(wq, idxd_desc); out: return ret; @@ -1500,13 +1499,11 @@ static int iaa_comp_acompress(struct acomp_req *req) struct iaa_compression_ctx *compression_ctx; struct crypto_tfm *tfm = req->base.tfm; dma_addr_t src_addr, dst_addr; - bool disable_async = false; int nr_sgs, cpu, ret = 0; struct iaa_wq *iaa_wq; u32 compression_crc; struct idxd_wq *wq; struct device *dev; - int order = -1; compression_ctx = crypto_tfm_ctx(tfm); @@ -1536,21 +1533,6 @@ static int iaa_comp_acompress(struct acomp_req *req) iaa_wq = idxd_wq_get_private(wq); - if (!req->dst) { - gfp_t flags = req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP ? GFP_KERNEL : GFP_ATOMIC; - - /* incompressible data will always be < 2 * slen */ - req->dlen = 2 * req->slen; - order = order_base_2(round_up(req->dlen, PAGE_SIZE) / PAGE_SIZE); - req->dst = sgl_alloc_order(req->dlen, order, false, flags, NULL); - if (!req->dst) { - ret = -ENOMEM; - order = -1; - goto out; - } - disable_async = true; - } - dev = &wq->idxd->pdev->dev; nr_sgs = dma_map_sg(dev, req->src, sg_nents(req->src), DMA_TO_DEVICE); @@ -1580,7 +1562,7 @@ static int iaa_comp_acompress(struct acomp_req *req) req->dst, req->dlen, sg_dma_len(req->dst)); ret = iaa_compress(tfm, req, wq, src_addr, req->slen, dst_addr, - &req->dlen, &compression_crc, disable_async); + &req->dlen, &compression_crc); if (ret == -EINPROGRESS) return ret; @@ -1611,100 +1593,6 @@ static int iaa_comp_acompress(struct acomp_req *req) out: iaa_wq_put(wq); - if (order >= 0) - sgl_free_order(req->dst, order); - - return ret; -} - -static int iaa_comp_adecompress_alloc_dest(struct acomp_req *req) -{ - gfp_t flags = req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP ? - GFP_KERNEL : GFP_ATOMIC; - struct crypto_tfm *tfm = req->base.tfm; - dma_addr_t src_addr, dst_addr; - int nr_sgs, cpu, ret = 0; - struct iaa_wq *iaa_wq; - struct device *dev; - struct idxd_wq *wq; - int order = -1; - - cpu = get_cpu(); - wq = wq_table_next_wq(cpu); - put_cpu(); - if (!wq) { - pr_debug("no wq configured for cpu=%d\n", cpu); - return -ENODEV; - } - - ret = iaa_wq_get(wq); - if (ret) { - pr_debug("no wq available for cpu=%d\n", cpu); - return -ENODEV; - } - - iaa_wq = idxd_wq_get_private(wq); - - dev = &wq->idxd->pdev->dev; - - nr_sgs = dma_map_sg(dev, req->src, sg_nents(req->src), DMA_TO_DEVICE); - if (nr_sgs <= 0 || nr_sgs > 1) { - dev_dbg(dev, "couldn't map src sg for iaa device %d," - " wq %d: ret=%d\n", iaa_wq->iaa_device->idxd->id, - iaa_wq->wq->id, ret); - ret = -EIO; - goto out; - } - src_addr = sg_dma_address(req->src); - dev_dbg(dev, "dma_map_sg, src_addr %llx, nr_sgs %d, req->src %p," - " req->slen %d, sg_dma_len(sg) %d\n", src_addr, nr_sgs, - req->src, req->slen, sg_dma_len(req->src)); - - req->dlen = 4 * req->slen; /* start with ~avg comp rato */ -alloc_dest: - order = order_base_2(round_up(req->dlen, PAGE_SIZE) / PAGE_SIZE); - req->dst = sgl_alloc_order(req->dlen, order, false, flags, NULL); - if (!req->dst) { - ret = -ENOMEM; - order = -1; - goto out; - } - - nr_sgs = dma_map_sg(dev, req->dst, sg_nents(req->dst), DMA_FROM_DEVICE); - if (nr_sgs <= 0 || nr_sgs > 1) { - dev_dbg(dev, "couldn't map dst sg for iaa device %d," - " wq %d: ret=%d\n", iaa_wq->iaa_device->idxd->id, - iaa_wq->wq->id, ret); - ret = -EIO; - goto err_map_dst; - } - - dst_addr = sg_dma_address(req->dst); - dev_dbg(dev, "dma_map_sg, dst_addr %llx, nr_sgs %d, req->dst %p," - " req->dlen %d, sg_dma_len(sg) %d\n", dst_addr, nr_sgs, - req->dst, req->dlen, sg_dma_len(req->dst)); - ret = iaa_decompress(tfm, req, wq, src_addr, req->slen, - dst_addr, &req->dlen, true); - if (ret == -EOVERFLOW) { - dma_unmap_sg(dev, req->dst, sg_nents(req->dst), DMA_FROM_DEVICE); - req->dlen *= 2; - if (req->dlen > CRYPTO_ACOMP_DST_MAX) - goto err_map_dst; - goto alloc_dest; - } - - if (ret != 0) - dev_dbg(dev, "asynchronous decompress failed ret=%d\n", ret); - - dma_unmap_sg(dev, req->dst, sg_nents(req->dst), DMA_FROM_DEVICE); -err_map_dst: - dma_unmap_sg(dev, req->src, sg_nents(req->src), DMA_TO_DEVICE); -out: - iaa_wq_put(wq); - - if (order >= 0) - sgl_free_order(req->dst, order); - return ret; } @@ -1727,9 +1615,6 @@ static int iaa_comp_adecompress(struct acomp_req *req) return -EINVAL; } - if (!req->dst) - return iaa_comp_adecompress_alloc_dest(req); - cpu = get_cpu(); wq = wq_table_next_wq(cpu); put_cpu(); @@ -1810,19 +1695,10 @@ static int iaa_comp_init_fixed(struct crypto_acomp *acomp_tfm) return 0; } -static void dst_free(struct scatterlist *sgl) -{ - /* - * Called for req->dst = NULL cases but we free elsewhere - * using sgl_free_order(). - */ -} - static struct acomp_alg iaa_acomp_fixed_deflate = { .init = iaa_comp_init_fixed, .compress = iaa_comp_acompress, .decompress = iaa_comp_adecompress, - .dst_free = dst_free, .base = { .cra_name = "deflate", .cra_driver_name = "deflate-iaa", From patchwork Fri Mar 14 12:22:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 873719 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8F7E1FE463; Fri, 14 Mar 2025 12:22:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954975; cv=none; b=habp2E5dOkGjqyc0AWzjMm0wvU4oLUBrno7kAcfqroOVX+nckfDwAg2RkML/l+qpCZx44anejDaIkU/X8BUKNdM5cjR+1C75qmqcwYfIEFGcjfRtOz4L3/+X3fAGTNW1R7binVp9kOXulIaBrqtUsOxQnH5KK/kdxlXOmghLi4o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954975; c=relaxed/simple; bh=xhJK0q04ma92B5ta0N1sU9Zt3Gp9vAAz/F47G1M6sig=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To:Cc; b=dkkzfLLGId9C5L8vvWK9BLeTqI/TA5tPCN0pDRJERczsNPeYwEIDEztJzqRPf8dHBjtuKapTJGOhfqEm5cCk8rLPlfd1aZ3o8C4i6nnGkT5YvCYG3C7S+wEilGqvTJ0J6ESRKSl2zpLyl13eidl2DOiF/8yxhIwbXi2d4HHlGWo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=VuQtL1wt; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="VuQtL1wt" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=Cc:To:Subject:From:References:In-Reply-To:Message-Id:Date: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=/lhr+zGaM7DIiI+E9GgEBvGtPQKlXrXK4hNUAFbkgRw=; b=VuQtL1wtN14IjlQVQgiT/Qyv7K 59V0FMWNt44dHOGaZYcxctVwPOve9DKE3VpcZS1kKqWG/h1XT+M/LcOeosIIC51GqQRGt+BWgT5G2 MW8EvXd3SMfOzG/ntnQnc/TvVrOaaYwsf1KkT46/fP3OsPvi/aUT5WEGR6fis7/YWtiACrOev0CyW c1n4QntnFBwdL8HjFfG/NGTx/dyEEmlF84/ljJWJsxiywrMjZ2XVZ090TopwYuHOTjXl2gVTqfd/X MborNDj/NvvvXyX5qcE+mf/x2p9mjxp4IoDjTHK9ry3bscZI6qsC93bovySfuNCfZQUzvd5Hi/MlK P78eJlvg==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1tt43h-006Zn3-2u; Fri, 14 Mar 2025 20:22:38 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Fri, 14 Mar 2025 20:22:37 +0800 Date: Fri, 14 Mar 2025 20:22:37 +0800 Message-Id: <730498abde26909d413616ca9f166abd1a254b83.1741954523.git.herbert@gondor.apana.org.au> In-Reply-To: References: From: Herbert Xu Subject: [v4 PATCH 06/13] crypto: acomp - Add ACOMP_REQUEST_ALLOC and acomp_request_alloc_extra To: Linux Crypto Mailing List Cc: Richard Weinberger , Zhihao Cheng , linux-mtd@lists.infradead.org, "Rafael J. Wysocki" , Pavel Machek , linux-pm@vger.kernel.org, Steffen Klassert , netdev@vger.kernel.org Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Add ACOMP_REQUEST_ALLOC which is a wrapper around acomp_request_alloc that falls back to a synchronous stack reqeust if the allocation fails. Also add ACOMP_REQUEST_ON_STACK which stores the request on the stack only. The request should be freed with acomp_request_free. Finally add acomp_request_alloc_extra which gives the user extra memory to use in conjunction with the request. Signed-off-by: Herbert Xu --- crypto/acompress.c | 38 ++++++++++++-- include/crypto/acompress.h | 80 +++++++++++++++++++++++++++-- include/crypto/internal/acompress.h | 6 +++ include/linux/crypto.h | 1 + 4 files changed, 117 insertions(+), 8 deletions(-) diff --git a/crypto/acompress.c b/crypto/acompress.c index 194a4b36f97f..9da033ded193 100644 --- a/crypto/acompress.c +++ b/crypto/acompress.c @@ -60,28 +60,56 @@ static void crypto_acomp_exit_tfm(struct crypto_tfm *tfm) struct crypto_acomp *acomp = __crypto_acomp_tfm(tfm); struct acomp_alg *alg = crypto_acomp_alg(acomp); - alg->exit(acomp); + if (alg->exit) + alg->exit(acomp); + + if (acomp_is_async(acomp)) + crypto_free_acomp(acomp->fb); } static int crypto_acomp_init_tfm(struct crypto_tfm *tfm) { struct crypto_acomp *acomp = __crypto_acomp_tfm(tfm); struct acomp_alg *alg = crypto_acomp_alg(acomp); + struct crypto_acomp *fb = NULL; + int err; + + acomp->fb = acomp; if (tfm->__crt_alg->cra_type != &crypto_acomp_type) return crypto_init_scomp_ops_async(tfm); + if (acomp_is_async(acomp)) { + fb = crypto_alloc_acomp(crypto_acomp_alg_name(acomp), 0, + CRYPTO_ALG_ASYNC); + if (IS_ERR(fb)) + return PTR_ERR(fb); + + err = -EINVAL; + if (crypto_acomp_reqsize(fb) > MAX_SYNC_COMP_REQSIZE) + goto out_free_fb; + + acomp->fb = fb; + } + acomp->compress = alg->compress; acomp->decompress = alg->decompress; acomp->reqsize = alg->reqsize; - if (alg->exit) - acomp->base.exit = crypto_acomp_exit_tfm; + acomp->base.exit = crypto_acomp_exit_tfm; - if (alg->init) - return alg->init(acomp); + if (!alg->init) + return 0; + + err = alg->init(acomp); + if (err) + goto out_free_fb; return 0; + +out_free_fb: + crypto_free_acomp(fb); + return err; } static unsigned int crypto_acomp_extsize(struct crypto_alg *alg) diff --git a/include/crypto/acompress.h b/include/crypto/acompress.h index 53c9e632862b..03cb381c2c54 100644 --- a/include/crypto/acompress.h +++ b/include/crypto/acompress.h @@ -10,9 +10,11 @@ #define _CRYPTO_ACOMP_H #include +#include #include #include #include +#include #include #include #include @@ -32,6 +34,14 @@ #define CRYPTO_ACOMP_DST_MAX 131072 +#define MAX_SYNC_COMP_REQSIZE 0 + +#define ACOMP_REQUEST_ALLOC(name, tfm, gfp) \ + char __##name##_req[sizeof(struct acomp_req) + \ + MAX_SYNC_COMP_REQSIZE] CRYPTO_MINALIGN_ATTR; \ + struct acomp_req *name = acomp_request_on_stack_init( \ + __##name##_req, (tfm), (gfp), false) + struct acomp_req; struct acomp_req_chain { @@ -83,12 +93,14 @@ struct acomp_req { * @compress: Function performs a compress operation * @decompress: Function performs a de-compress operation * @reqsize: Context size for (de)compression requests + * @fb: Synchronous fallback tfm * @base: Common crypto API algorithm data structure */ struct crypto_acomp { int (*compress)(struct acomp_req *req); int (*decompress)(struct acomp_req *req); unsigned int reqsize; + struct crypto_acomp *fb; struct crypto_tfm base; }; @@ -210,24 +222,68 @@ static inline int crypto_has_acomp(const char *alg_name, u32 type, u32 mask) return crypto_has_alg(alg_name, type, mask); } +static inline const char *crypto_acomp_alg_name(struct crypto_acomp *tfm) +{ + return crypto_tfm_alg_name(crypto_acomp_tfm(tfm)); +} + +static inline const char *crypto_acomp_driver_name(struct crypto_acomp *tfm) +{ + return crypto_tfm_alg_driver_name(crypto_acomp_tfm(tfm)); +} + /** * acomp_request_alloc() -- allocates asynchronous (de)compression request * * @tfm: ACOMPRESS tfm handle allocated with crypto_alloc_acomp() + * @gfp: gfp to pass to kzalloc (defaults to GFP_KERNEL) * * Return: allocated handle in case of success or NULL in case of an error */ -static inline struct acomp_req *acomp_request_alloc_noprof(struct crypto_acomp *tfm) +static inline struct acomp_req *acomp_request_alloc_extra_noprof( + struct crypto_acomp *tfm, size_t extra, gfp_t gfp) { struct acomp_req *req; + size_t len; - req = kzalloc_noprof(sizeof(*req) + crypto_acomp_reqsize(tfm), GFP_KERNEL); + len = ALIGN(sizeof(*req) + crypto_acomp_reqsize(tfm), CRYPTO_MINALIGN); + if (check_add_overflow(len, extra, &len)) + return NULL; + + req = kzalloc_noprof(len, gfp); if (likely(req)) acomp_request_set_tfm(req, tfm); return req; } +#define acomp_request_alloc_noprof(tfm, ...) \ + CONCATENATE(acomp_request_alloc_noprof_, COUNT_ARGS(__VA_ARGS__))( \ + tfm, ##__VA_ARGS__) +#define acomp_request_alloc_noprof_0(tfm) \ + acomp_request_alloc_noprof_1(tfm, GFP_KERNEL) +#define acomp_request_alloc_noprof_1(tfm, gfp) \ + acomp_request_alloc_extra_noprof(tfm, 0, gfp) #define acomp_request_alloc(...) alloc_hooks(acomp_request_alloc_noprof(__VA_ARGS__)) +/** + * acomp_request_alloc_extra() -- allocate acomp request with extra memory + * + * @tfm: ACOMPRESS tfm handle allocated with crypto_alloc_acomp() + * @extra: amount of extra memory + * @gfp: gfp to pass to kzalloc + * + * Return: allocated handle in case of success or NULL in case of an error + */ +#define acomp_request_alloc_extra(...) alloc_hooks(acomp_request_alloc_extra_noprof(__VA_ARGS__)) + +static inline void *acomp_request_extra(struct acomp_req *req) +{ + struct crypto_acomp *tfm = crypto_acomp_reqtfm(req); + size_t len; + + len = ALIGN(sizeof(*req) + crypto_acomp_reqsize(tfm), CRYPTO_MINALIGN); + return (void *)((char *)req + len); +} + /** * acomp_request_free() -- zeroize and free asynchronous (de)compression * request as well as the output buffer if allocated @@ -237,6 +293,8 @@ static inline struct acomp_req *acomp_request_alloc_noprof(struct crypto_acomp * */ static inline void acomp_request_free(struct acomp_req *req) { + if (!req || (req->base.flags & CRYPTO_TFM_REQ_ON_STACK)) + return; kfree_sensitive(req); } @@ -257,7 +315,8 @@ static inline void acomp_request_set_callback(struct acomp_req *req, void *data) { u32 keep = CRYPTO_ACOMP_REQ_SRC_VIRT | CRYPTO_ACOMP_REQ_SRC_NONDMA | - CRYPTO_ACOMP_REQ_DST_VIRT | CRYPTO_ACOMP_REQ_DST_NONDMA; + CRYPTO_ACOMP_REQ_DST_VIRT | CRYPTO_ACOMP_REQ_DST_NONDMA | + CRYPTO_TFM_REQ_ON_STACK; req->base.complete = cmpl; req->base.data = data; @@ -446,4 +505,19 @@ int crypto_acomp_compress(struct acomp_req *req); */ int crypto_acomp_decompress(struct acomp_req *req); +static inline struct acomp_req *acomp_request_on_stack_init( + char *buf, struct crypto_acomp *tfm, gfp_t gfp, bool stackonly) +{ + struct acomp_req *req; + + if (!stackonly && (req = acomp_request_alloc(tfm, gfp))) + return req; + + req = (void *)buf; + acomp_request_set_tfm(req, tfm->fb); + req->base.flags = CRYPTO_TFM_REQ_ON_STACK; + + return req; +} + #endif diff --git a/include/crypto/internal/acompress.h b/include/crypto/internal/acompress.h index 575dbcbc0df4..c1ed55a0e3bf 100644 --- a/include/crypto/internal/acompress.h +++ b/include/crypto/internal/acompress.h @@ -12,6 +12,12 @@ #include #include +#define ACOMP_REQUEST_ON_STACK(name, tfm) \ + char __##name##_req[sizeof(struct acomp_req) + \ + MAX_SYNC_COMP_REQSIZE] CRYPTO_MINALIGN_ATTR; \ + struct acomp_req *name = acomp_request_on_stack_init( \ + __##name##_req, (tfm), 0, true) + /** * struct acomp_alg - asynchronous compression algorithm * diff --git a/include/linux/crypto.h b/include/linux/crypto.h index 61ac11226638..ea3b95bdbde3 100644 --- a/include/linux/crypto.h +++ b/include/linux/crypto.h @@ -138,6 +138,7 @@ #define CRYPTO_TFM_REQ_FORBID_WEAK_KEYS 0x00000100 #define CRYPTO_TFM_REQ_MAY_SLEEP 0x00000200 #define CRYPTO_TFM_REQ_MAY_BACKLOG 0x00000400 +#define CRYPTO_TFM_REQ_ON_STACK 0x00000800 /* * Miscellaneous stuff. From patchwork Fri Mar 14 12:22:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 873718 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B7571FF1CE; Fri, 14 Mar 2025 12:22:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954977; cv=none; b=obG2h+hN7Mkp5NVQTXSNFhDIQeS17bLa5S9sU/R43IK2zVVGtiFWuJmnxbwV/KDThJtDjGgWk8UDwntJ0v8W++gaQot0z2K7dCtZYNbSat3SnhcH1XDk2wGIfJg755eoFV00ifo5C9F8VWwQMGPq9jPqcPLB4ogwpC1+l95q2N0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954977; c=relaxed/simple; bh=+uuxJaOeVr3jZTRaC2FAbtdc5dJBjxropLllTH5FOcQ=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To:Cc; b=gSi5KclHO0b3D9CnabQ9DU1fEMQKduxyyFek9W6ZQ4WqT2HR3L+Av+v2YceHVb/QVlkop4kbh+rjGhde8qgz+HOOc0+b2pTeNxmFRFGliTOMbk4oGquUUz0g6sTU9af77WTrx1t5JHdnCRoiPJsWaUUI/MgPnpXGdaMMSLwrGxw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=UIesLuOk; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="UIesLuOk" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=Cc:To:Subject:From:References:In-Reply-To:Message-Id:Date: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=zIb0wSG71Ygj2H3jMgCe2Okz2AI9eV57QCfyyCpN7fc=; b=UIesLuOkOKkXi6wOEUTzd8PLUa xc77yNRZ84G492gM0yD5b2fCNAUd92yIdOAEjzDcY9qJfxGqIaI/GQMoh+NAEwN68XTFkTDyFe/9V Ec49HFWezk6LFLaJAyGFYive0u7KYruVUi0Z/OlOzJV1gyJVAKuzDthWxNoiaesG6dp7er+IQrGwH FZ2QKx8L+hxO47Bv9cVhwXGp3+S9KV+a1FIQxkqD2IB3kT0cPAG0yKqeJLov66/sADzg8Pj0Wgopu uCxFTE+YtZc4+tXRG93Yh/Sv8Vy54O8CorJ01niKH4j/VbhhfcaBtQC9qm1ti8l9QBlNZOS1nqyA8 Gfg+9dXA==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1tt43k-006ZnL-0c; Fri, 14 Mar 2025 20:22:41 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Fri, 14 Mar 2025 20:22:40 +0800 Date: Fri, 14 Mar 2025 20:22:40 +0800 Message-Id: <96c3973e2ef9cd6762071a6dc5d6b34c660f9fa7.1741954523.git.herbert@gondor.apana.org.au> In-Reply-To: References: From: Herbert Xu Subject: [v4 PATCH 07/13] crypto: iaa - Use acomp stack fallback To: Linux Crypto Mailing List Cc: Richard Weinberger , Zhihao Cheng , linux-mtd@lists.infradead.org, "Rafael J. Wysocki" , Pavel Machek , linux-pm@vger.kernel.org, Steffen Klassert , netdev@vger.kernel.org Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Use ACOMP_REQUEST_ON_STACK instead of allocating legacy fallback compression transform. Signed-off-by: Herbert Xu --- drivers/crypto/intel/iaa/iaa_crypto_main.c | 28 +++++----------------- 1 file changed, 6 insertions(+), 22 deletions(-) diff --git a/drivers/crypto/intel/iaa/iaa_crypto_main.c b/drivers/crypto/intel/iaa/iaa_crypto_main.c index 50cb100bf1c8..09d9589f2d68 100644 --- a/drivers/crypto/intel/iaa/iaa_crypto_main.c +++ b/drivers/crypto/intel/iaa/iaa_crypto_main.c @@ -33,8 +33,6 @@ static unsigned int nr_cpus_per_node; /* Number of physical cpus sharing each iaa instance */ static unsigned int cpus_per_iaa; -static struct crypto_comp *deflate_generic_tfm; - /* Per-cpu lookup table for balanced wqs */ static struct wq_table_entry __percpu *wq_table; @@ -1001,17 +999,14 @@ static inline int check_completion(struct device *dev, static int deflate_generic_decompress(struct acomp_req *req) { - void *src, *dst; + ACOMP_REQUEST_ON_STACK(fbreq, crypto_acomp_reqtfm(req)); int ret; - src = kmap_local_page(sg_page(req->src)) + req->src->offset; - dst = kmap_local_page(sg_page(req->dst)) + req->dst->offset; - - ret = crypto_comp_decompress(deflate_generic_tfm, - src, req->slen, dst, &req->dlen); - - kunmap_local(src); - kunmap_local(dst); + acomp_request_set_callback(fbreq, 0, NULL, NULL); + acomp_request_set_params(fbreq, req->src, req->dst, req->slen, + req->dlen); + ret = crypto_acomp_decompress(fbreq); + req->dlen = fbreq->dlen; update_total_sw_decomp_calls(); @@ -1898,15 +1893,6 @@ static int __init iaa_crypto_init_module(void) } nr_cpus_per_node = nr_cpus / nr_nodes; - if (crypto_has_comp("deflate-generic", 0, 0)) - deflate_generic_tfm = crypto_alloc_comp("deflate-generic", 0, 0); - - if (IS_ERR_OR_NULL(deflate_generic_tfm)) { - pr_err("IAA could not alloc %s tfm: errcode = %ld\n", - "deflate-generic", PTR_ERR(deflate_generic_tfm)); - return -ENOMEM; - } - ret = iaa_aecs_init_fixed(); if (ret < 0) { pr_debug("IAA fixed compression mode init failed\n"); @@ -1948,7 +1934,6 @@ static int __init iaa_crypto_init_module(void) err_driver_reg: iaa_aecs_cleanup_fixed(); err_aecs_init: - crypto_free_comp(deflate_generic_tfm); goto out; } @@ -1965,7 +1950,6 @@ static void __exit iaa_crypto_cleanup_module(void) &driver_attr_verify_compress); idxd_driver_unregister(&iaa_crypto_driver); iaa_aecs_cleanup_fixed(); - crypto_free_comp(deflate_generic_tfm); pr_debug("cleaned up\n"); } From patchwork Fri Mar 14 12:22:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 873717 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 648871FFC4B; Fri, 14 Mar 2025 12:22:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954979; cv=none; b=qMkeIiX7ApS2K3VJNUV4ezcjCDS8Dc+1DQYxHGjTg48T7L2QYOGA5unXANUGvSDuNfvUzVV/o34GSbx+jAdy8KUKzlo2Tych2kt+4nHfFiYkypmxaPlk9dmD4EBjUfmgyPusNRN7B+PDH/92iyudElw8/fyUXZJ+X4L9p7iuSY0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954979; c=relaxed/simple; bh=GXurj3eAr/J1gshDakXdzCLgyyNpUUNRDhIyf2JgK+4=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To:Cc; b=FB3tPmvyVNpDBAzS+sg2REIqkJb7pcZjt6ES6U21sXuTm6hV5w/r4kil4M0BJkC8lBz8wVINQvA7sh/Naf5PNlZwQmUFr+8Uc5XoffbbGlMORKhclJu7ttA/NUfvou+1ByaOWwFZSnkoEBJ48n4r4leCSeTUSdvfcVDnxXohea8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=jwJ7Fof4; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="jwJ7Fof4" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=Cc:To:Subject:From:References:In-Reply-To:Message-Id:Date: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=l+ZShRAgEc0yiVA0reyVkATRhc0SDpPkUxK/Z/PM0O0=; b=jwJ7Fof4b1Tzltln/IGPTWMw5i kPmCIbqsknsU4xX/ar4lrxeqPLO+en1QzXM6v8LLKpDEwwTJdqpmt7uVByRTR3zAfdb48rWT6kV18 oaTomOWKXoN/NLggIn9klOilJVaFe0zwPIKOin1BvDuac+crTbVjeGeI9yScdCjdk0lMMMr/Powpd DT31k2144ROwP7bYG4odOK3Iok+5IDC09lWgyhTf7HwzeXqT0c4oc+D5cM/SDeI7ejyGlJOYFZSO1 hEXfy5zAQ1B95RtOE0hKu9R/FtD/YZvk5pc6UotsE9xDto7E6KydwvIXqxJQnlVsCYtsCaCHVHnf/ VWMRUo5g==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1tt43m-006Znb-1X; Fri, 14 Mar 2025 20:22:43 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Fri, 14 Mar 2025 20:22:42 +0800 Date: Fri, 14 Mar 2025 20:22:42 +0800 Message-Id: <5498a27375e1f8a7451435195ae7a01fd670fe19.1741954523.git.herbert@gondor.apana.org.au> In-Reply-To: References: From: Herbert Xu Subject: [v4 PATCH 08/13] crypto: acomp - Add async nondma fallback To: Linux Crypto Mailing List Cc: Richard Weinberger , Zhihao Cheng , linux-mtd@lists.infradead.org, "Rafael J. Wysocki" , Pavel Machek , linux-pm@vger.kernel.org, Steffen Klassert , netdev@vger.kernel.org Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Add support for passing non-DMA virtual addresses to async drivers by passing them along to the fallback software algorithm. Signed-off-by: Herbert Xu --- crypto/acompress.c | 69 +++++++++++++++++++++++++++------------------- 1 file changed, 41 insertions(+), 28 deletions(-) diff --git a/crypto/acompress.c b/crypto/acompress.c index 9da033ded193..d54abc27330f 100644 --- a/crypto/acompress.c +++ b/crypto/acompress.c @@ -152,20 +152,6 @@ struct crypto_acomp *crypto_alloc_acomp_node(const char *alg_name, u32 type, } EXPORT_SYMBOL_GPL(crypto_alloc_acomp_node); -static bool acomp_request_has_nondma(struct acomp_req *req) -{ - struct acomp_req *r2; - - if (acomp_request_isnondma(req)) - return true; - - list_for_each_entry(r2, &req->base.list, base.list) - if (acomp_request_isnondma(r2)) - return true; - - return false; -} - static void acomp_save_req(struct acomp_req *req, crypto_completion_t cplt) { struct crypto_acomp *tfm = crypto_acomp_reqtfm(req); @@ -234,6 +220,45 @@ static void acomp_virt_to_sg(struct acomp_req *req) } } +static int acomp_do_nondma(struct acomp_req_chain *state, + struct acomp_req *req) +{ + u32 keep = CRYPTO_ACOMP_REQ_SRC_VIRT | + CRYPTO_ACOMP_REQ_SRC_NONDMA | + CRYPTO_ACOMP_REQ_DST_VIRT | + CRYPTO_ACOMP_REQ_DST_NONDMA; + ACOMP_REQUEST_ON_STACK(fbreq, crypto_acomp_reqtfm(req)); + int err; + + acomp_request_set_callback(fbreq, req->base.flags, NULL, NULL); + fbreq->base.flags &= ~keep; + fbreq->base.flags |= req->base.flags & keep; + fbreq->src = req->src; + fbreq->dst = req->dst; + fbreq->slen = req->slen; + fbreq->dlen = req->dlen; + + if (state->op == crypto_acomp_reqtfm(req)->compress) + err = crypto_acomp_compress(fbreq); + else + err = crypto_acomp_decompress(fbreq); + + req->dlen = fbreq->dlen; + return err; +} + +static int acomp_do_one_req(struct acomp_req_chain *state, + struct acomp_req *req) +{ + state->cur = req; + + if (acomp_request_isnondma(req)) + return acomp_do_nondma(state, req); + + acomp_virt_to_sg(req); + return state->op(req); +} + static int acomp_reqchain_finish(struct acomp_req_chain *state, int err, u32 mask) { @@ -252,10 +277,8 @@ static int acomp_reqchain_finish(struct acomp_req_chain *state, req->base.flags &= mask; req->base.complete = acomp_reqchain_done; req->base.data = state; - state->cur = req; - acomp_virt_to_sg(req); - err = state->op(req); + err = acomp_do_one_req(state, req); if (err == -EINPROGRESS) { if (!list_empty(&state->head)) @@ -308,27 +331,17 @@ static int acomp_do_req_chain(struct acomp_req *req, (!acomp_request_chained(req) && !acomp_request_isvirt(req))) return op(req); - /* - * There are no in-kernel users that do this. If and ever - * such users come into being then we could add a fall-back - * path. - */ - if (acomp_request_has_nondma(req)) - return -EINVAL; - if (acomp_is_async(tfm)) { acomp_save_req(req, acomp_reqchain_done); state = req->base.data; } state->op = op; - state->cur = req; state->src = NULL; INIT_LIST_HEAD(&state->head); list_splice_init(&req->base.list, &state->head); - acomp_virt_to_sg(req); - err = op(req); + err = acomp_do_one_req(state, req); if (err == -EBUSY || err == -EINPROGRESS) return -EBUSY; From patchwork Fri Mar 14 12:22:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 873715 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 58890201013; Fri, 14 Mar 2025 12:23:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954989; cv=none; b=kYby3hn5wx8TZJFJXDB7S8v47z2pBcbdAIHQgR8M8dO47ngFiprs5GgAyu1LbEW5BnkUZvhehiYSKCc4fddLtfK2XEn0DG7UkLbC8QeTa+BSYG5cZZcFPVO8LotIOkvud1fHtA2Xzfrg+9P+8Uvsyyi2pZGYRVhB6AblRndT5M0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954989; c=relaxed/simple; bh=XbQQJ/qOHRDQabW/O+X7JtkYl4B5Sialc7eDChkh2dA=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To:Cc; b=OYh0eLvXTX1Y5JfL88z1S3zEibgfKg79vLoCYhCVvSebnqbYGopwrdXEs5WgkbfN6CbBeZx/uuYbjVcrWj15uK1OqQHPcNiDQ2emrNKL3sbCeoQpvb+Gm7QE22co5M/ZDo4NuQGobiYsrUPGsRp4phVhZvWmn5yTcqCHsD/U0+I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=NfzF02sN; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="NfzF02sN" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=Cc:To:Subject:From:References:In-Reply-To:Message-Id:Date: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=H79hgHg4jw3PQjcoYWfvd84mHhIO9k+ikAcZrZ8jTpM=; b=NfzF02sNvLmDSFYcuX8MqD61YI Vablx9d9egF9FWrGRcYVaxP8/VP/F2jh5vjyHRp8SkZYb9uayforzv7Nx/tkhJF1nE927vrhICU6i hYWAnkVxrONwYFBufisfkYrdCqPvD8siFtlN3nUlMhh/VfmShBp2MLyKeGpj0KWZHquTDDlkBikTe UhTHJyoMT0yzl/e/+37qCP6eh09AQ/yy33xbLY62xQOk7OfNI0AH9IIYT+2sfhOXrRn2m9eehhFGu dY2zCmPV0jgqjWvcvg+17+YqByjPGWgqT2nbZsfohovicNwcRKzPs3MeLbJxrtU4ex//EJH0oPOz2 x1ULebog==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1tt43r-006ZoC-06; Fri, 14 Mar 2025 20:22:48 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Fri, 14 Mar 2025 20:22:47 +0800 Date: Fri, 14 Mar 2025 20:22:47 +0800 Message-Id: <349a78bc53d3620a29cc6105b55985db51aa0a11.1741954523.git.herbert@gondor.apana.org.au> In-Reply-To: References: From: Herbert Xu Subject: [v4 PATCH 10/13] ubifs: Use crypto_acomp interface To: Linux Crypto Mailing List Cc: Richard Weinberger , Zhihao Cheng , linux-mtd@lists.infradead.org, "Rafael J. Wysocki" , Pavel Machek , linux-pm@vger.kernel.org, Steffen Klassert , netdev@vger.kernel.org Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Replace the legacy crypto compression interface with the new acomp interface. Remove the compression mutexes and the overallocation for memory (the offender LZO has been fixed). Signed-off-by: Herbert Xu --- fs/ubifs/compress.c | 116 ++++++++++++++++++++++++++++---------------- fs/ubifs/journal.c | 2 +- fs/ubifs/ubifs.h | 15 +----- 3 files changed, 77 insertions(+), 56 deletions(-) diff --git a/fs/ubifs/compress.c b/fs/ubifs/compress.c index 0b48cbab8a3d..9046e796876d 100644 --- a/fs/ubifs/compress.c +++ b/fs/ubifs/compress.c @@ -15,7 +15,7 @@ * decompression. */ -#include +#include #include "ubifs.h" /* Fake description object for the "none" compressor */ @@ -26,11 +26,8 @@ static struct ubifs_compressor none_compr = { }; #ifdef CONFIG_UBIFS_FS_LZO -static DEFINE_MUTEX(lzo_mutex); - static struct ubifs_compressor lzo_compr = { .compr_type = UBIFS_COMPR_LZO, - .comp_mutex = &lzo_mutex, .name = "lzo", .capi_name = "lzo", }; @@ -42,13 +39,8 @@ static struct ubifs_compressor lzo_compr = { #endif #ifdef CONFIG_UBIFS_FS_ZLIB -static DEFINE_MUTEX(deflate_mutex); -static DEFINE_MUTEX(inflate_mutex); - static struct ubifs_compressor zlib_compr = { .compr_type = UBIFS_COMPR_ZLIB, - .comp_mutex = &deflate_mutex, - .decomp_mutex = &inflate_mutex, .name = "zlib", .capi_name = "deflate", }; @@ -60,13 +52,8 @@ static struct ubifs_compressor zlib_compr = { #endif #ifdef CONFIG_UBIFS_FS_ZSTD -static DEFINE_MUTEX(zstd_enc_mutex); -static DEFINE_MUTEX(zstd_dec_mutex); - static struct ubifs_compressor zstd_compr = { .compr_type = UBIFS_COMPR_ZSTD, - .comp_mutex = &zstd_enc_mutex, - .decomp_mutex = &zstd_dec_mutex, .name = "zstd", .capi_name = "zstd", }; @@ -80,6 +67,40 @@ static struct ubifs_compressor zstd_compr = { /* All UBIFS compressors */ struct ubifs_compressor *ubifs_compressors[UBIFS_COMPR_TYPES_CNT]; +static int ubifs_compress_req(const struct ubifs_info *c, + struct acomp_req *req, + void *out_buf, int *out_len) +{ + struct crypto_wait wait; + int in_len = req->slen; + int err; + + crypto_init_wait(&wait); + acomp_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG, + crypto_req_done, &wait); + acomp_request_set_dst_dma(req, out_buf, *out_len); + err = crypto_acomp_compress(req); + err = crypto_wait_req(err, &wait); + *out_len = req->dlen; + + if (unlikely(err)) { + ubifs_warn(c, "cannot compress %d bytes, compressor %s, error %d, leave data uncompressed", + in_len, + crypto_acomp_alg_name(crypto_acomp_reqtfm(req)), + err); + } else if (in_len - *out_len < UBIFS_MIN_COMPRESS_DIFF) { + /* + * If the data compressed only slightly, it is better + * to leave it uncompressed to improve read speed. + */ + err = -E2BIG; + } + + acomp_request_free(req); + + return err; +} + /** * ubifs_compress - compress data. * @c: UBIFS file-system description object @@ -112,23 +133,14 @@ void ubifs_compress(const struct ubifs_info *c, const void *in_buf, if (in_len < UBIFS_MIN_COMPR_LEN) goto no_compr; - if (compr->comp_mutex) - mutex_lock(compr->comp_mutex); - err = crypto_comp_compress(compr->cc, in_buf, in_len, out_buf, - (unsigned int *)out_len); - if (compr->comp_mutex) - mutex_unlock(compr->comp_mutex); - if (unlikely(err)) { - ubifs_warn(c, "cannot compress %d bytes, compressor %s, error %d, leave data uncompressed", - in_len, compr->name, err); - goto no_compr; + { + ACOMP_REQUEST_ALLOC(req, compr->cc, GFP_NOFS | __GFP_NOWARN); + + acomp_request_set_src_nondma(req, in_buf, in_len); + err = ubifs_compress_req(c, req, out_buf, out_len); } - /* - * If the data compressed only slightly, it is better to leave it - * uncompressed to improve read speed. - */ - if (in_len - *out_len < UBIFS_MIN_COMPRESS_DIFF) + if (err) goto no_compr; return; @@ -139,6 +151,32 @@ void ubifs_compress(const struct ubifs_info *c, const void *in_buf, *compr_type = UBIFS_COMPR_NONE; } +static int ubifs_decompress_req(const struct ubifs_info *c, + struct acomp_req *req, + const void *in_buf, int in_len, int *out_len) +{ + struct crypto_wait wait; + int err; + + crypto_init_wait(&wait); + acomp_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG, + crypto_req_done, &wait); + acomp_request_set_src_dma(req, in_buf, in_len); + err = crypto_acomp_decompress(req); + err = crypto_wait_req(err, &wait); + *out_len = req->dlen; + + if (err) + ubifs_err(c, "cannot decompress %d bytes, compressor %s, error %d", + in_len, + crypto_acomp_alg_name(crypto_acomp_reqtfm(req)), + err); + + acomp_request_free(req); + + return err; +} + /** * ubifs_decompress - decompress data. * @c: UBIFS file-system description object @@ -155,7 +193,6 @@ void ubifs_compress(const struct ubifs_info *c, const void *in_buf, int ubifs_decompress(const struct ubifs_info *c, const void *in_buf, int in_len, void *out_buf, int *out_len, int compr_type) { - int err; struct ubifs_compressor *compr; if (unlikely(compr_type < 0 || compr_type >= UBIFS_COMPR_TYPES_CNT)) { @@ -176,17 +213,12 @@ int ubifs_decompress(const struct ubifs_info *c, const void *in_buf, return 0; } - if (compr->decomp_mutex) - mutex_lock(compr->decomp_mutex); - err = crypto_comp_decompress(compr->cc, in_buf, in_len, out_buf, - (unsigned int *)out_len); - if (compr->decomp_mutex) - mutex_unlock(compr->decomp_mutex); - if (err) - ubifs_err(c, "cannot decompress %d bytes, compressor %s, error %d", - in_len, compr->name, err); + { + ACOMP_REQUEST_ALLOC(req, compr->cc, GFP_NOFS | __GFP_NOWARN); - return err; + acomp_request_set_dst_nondma(req, out_buf, *out_len); + return ubifs_decompress_req(c, req, in_buf, in_len, out_len); + } } /** @@ -199,7 +231,7 @@ int ubifs_decompress(const struct ubifs_info *c, const void *in_buf, static int __init compr_init(struct ubifs_compressor *compr) { if (compr->capi_name) { - compr->cc = crypto_alloc_comp(compr->capi_name, 0, 0); + compr->cc = crypto_alloc_acomp(compr->capi_name, 0, 0); if (IS_ERR(compr->cc)) { pr_err("UBIFS error (pid %d): cannot initialize compressor %s, error %ld", current->pid, compr->name, PTR_ERR(compr->cc)); @@ -218,7 +250,7 @@ static int __init compr_init(struct ubifs_compressor *compr) static void compr_exit(struct ubifs_compressor *compr) { if (compr->capi_name) - crypto_free_comp(compr->cc); + crypto_free_acomp(compr->cc); } /** diff --git a/fs/ubifs/journal.c b/fs/ubifs/journal.c index 36ba79fbd2ff..7629ca9ecfe8 100644 --- a/fs/ubifs/journal.c +++ b/fs/ubifs/journal.c @@ -1625,7 +1625,7 @@ static int truncate_data_node(const struct ubifs_info *c, const struct inode *in int err, dlen, compr_type, out_len, data_size; out_len = le32_to_cpu(dn->size); - buf = kmalloc_array(out_len, WORST_COMPR_FACTOR, GFP_NOFS); + buf = kmalloc(out_len, GFP_NOFS); if (!buf) return -ENOMEM; diff --git a/fs/ubifs/ubifs.h b/fs/ubifs/ubifs.h index 3375bbe0508c..7d0aaf5d2e23 100644 --- a/fs/ubifs/ubifs.h +++ b/fs/ubifs/ubifs.h @@ -124,13 +124,6 @@ #define OLD_ZNODE_AGE 20 #define YOUNG_ZNODE_AGE 5 -/* - * Some compressors, like LZO, may end up with more data then the input buffer. - * So UBIFS always allocates larger output buffer, to be sure the compressor - * will not corrupt memory in case of worst case compression. - */ -#define WORST_COMPR_FACTOR 2 - #ifdef CONFIG_FS_ENCRYPTION #define UBIFS_CIPHER_BLOCK_SIZE FSCRYPT_CONTENTS_ALIGNMENT #else @@ -141,7 +134,7 @@ * How much memory is needed for a buffer where we compress a data node. */ #define COMPRESSED_DATA_NODE_BUF_SZ \ - (UBIFS_DATA_NODE_SZ + UBIFS_BLOCK_SIZE * WORST_COMPR_FACTOR) + (UBIFS_DATA_NODE_SZ + UBIFS_BLOCK_SIZE) /* Maximum expected tree height for use by bottom_up_buf */ #define BOTTOM_UP_HEIGHT 64 @@ -835,16 +828,12 @@ struct ubifs_node_range { * struct ubifs_compressor - UBIFS compressor description structure. * @compr_type: compressor type (%UBIFS_COMPR_LZO, etc) * @cc: cryptoapi compressor handle - * @comp_mutex: mutex used during compression - * @decomp_mutex: mutex used during decompression * @name: compressor name * @capi_name: cryptoapi compressor name */ struct ubifs_compressor { int compr_type; - struct crypto_comp *cc; - struct mutex *comp_mutex; - struct mutex *decomp_mutex; + struct crypto_acomp *cc; const char *name; const char *capi_name; }; From patchwork Fri Mar 14 12:22:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 873714 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4107E1FECAB; Fri, 14 Mar 2025 12:23:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954991; cv=none; b=l5+ckWX2tJalGSYapgDFXKjRFfgAZ1jsd8bbmL+LXQ4hhY/QkTuFtLIRql5mCEaJgpTXW18m1Ghd2btueA+uUPhnbgGisaO2DP8Vmqx0qHX7gcKYJhfjBCP6103Jphejuttg+DjN3QZ5bGpgM8roWerlxWxQjphtIfbIyQUm4rM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954991; c=relaxed/simple; bh=IOqcbcSyIMPkljBgvP6atjQQPggxoxOzt9hxbvb7/uU=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To:Cc; b=NPRhq+I2XQq/PYPpkv0b5YlTfprHR/TXJ68fBdv4lRSk3PBbYMpV3wn+rJPL5sWB6Ocm5YZw/fZ542yVTWVLi2It8FloWJWiB5i+A5SBmXsh8F4jyw0HvtBtk9l7W/ID9A5C4L80IVQAe0udLZbiL9cYA5g/8zKFcaxEOr0dXAg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=RzXEf6Qn; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="RzXEf6Qn" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=Cc:To:Subject:From:References:In-Reply-To:Message-Id:Date: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=avLv1kIH8yNek3R4MB5SP5S6/ORyKKw5wrgNAuoZdZU=; b=RzXEf6Qn+9pV5rjrmq+XBvmzn5 9RWfQLwnbKWX/XEJmqNQqNBEAIXUX45TSU764e/PSBWHfr4Orn/WFmu7uh+SwdQ6bXs4liXR6Y7QR hIlBtH3hZceTxzhVQajqzs9EAAGnhGciLXoLh+4pZNIoLrvn6hVv1wn1kmBSjlkcrGYvTYg1CHo+6 CIWR0gm2TxZn5asU5fnXrppSuABsaV7B0WOJMZDpFfrAN3SVV+viNe9jmuvelyqZjmbCGb2wcL959 RFZ4gim2RE4eOJreaSWtvV9AUHrl3t4gFFXrIVrQWHoG9IfAOtse3X3b1y0TbN0Lfed3jT3EINlN9 +61CRThQ==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1tt43v-006Zou-2J; Fri, 14 Mar 2025 20:22:52 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Fri, 14 Mar 2025 20:22:51 +0800 Date: Fri, 14 Mar 2025 20:22:51 +0800 Message-Id: <785c7858e03ad03a56ffaee0e413c72e0a307a63.1741954523.git.herbert@gondor.apana.org.au> In-Reply-To: References: From: Herbert Xu Subject: [v4 PATCH 12/13] PM: hibernate: Use crypto_acomp interface To: Linux Crypto Mailing List Cc: Richard Weinberger , Zhihao Cheng , linux-mtd@lists.infradead.org, "Rafael J. Wysocki" , Pavel Machek , linux-pm@vger.kernel.org, Steffen Klassert , netdev@vger.kernel.org Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Replace the legacy crypto compression interface with the new acomp interface. Signed-off-by: Herbert Xu --- kernel/power/swap.c | 58 ++++++++++++++++++++++++++++++--------------- 1 file changed, 39 insertions(+), 19 deletions(-) diff --git a/kernel/power/swap.c b/kernel/power/swap.c index 82b884b67152..80ff5f933a62 100644 --- a/kernel/power/swap.c +++ b/kernel/power/swap.c @@ -12,6 +12,7 @@ #define pr_fmt(fmt) "PM: " fmt +#include #include #include #include @@ -635,7 +636,8 @@ static int crc32_threadfn(void *data) */ struct cmp_data { struct task_struct *thr; /* thread */ - struct crypto_comp *cc; /* crypto compressor stream */ + struct crypto_acomp *cc; /* crypto compressor */ + struct acomp_req *cr; /* crypto request */ atomic_t ready; /* ready to start flag */ atomic_t stop; /* ready to stop flag */ int ret; /* return code */ @@ -656,7 +658,6 @@ static atomic_t compressed_size = ATOMIC_INIT(0); static int compress_threadfn(void *data) { struct cmp_data *d = data; - unsigned int cmp_len = 0; while (1) { wait_event(d->go, atomic_read_acquire(&d->ready) || @@ -670,11 +671,13 @@ static int compress_threadfn(void *data) } atomic_set(&d->ready, 0); - cmp_len = CMP_SIZE - CMP_HEADER; - d->ret = crypto_comp_compress(d->cc, d->unc, d->unc_len, - d->cmp + CMP_HEADER, - &cmp_len); - d->cmp_len = cmp_len; + acomp_request_set_callback(d->cr, CRYPTO_TFM_REQ_MAY_SLEEP, + NULL, NULL); + acomp_request_set_src_nondma(d->cr, d->unc, d->unc_len); + acomp_request_set_dst_nondma(d->cr, d->cmp + CMP_HEADER, + CMP_SIZE - CMP_HEADER); + d->ret = crypto_acomp_compress(d->cr); + d->cmp_len = d->cr->dlen; atomic_set(&compressed_size, atomic_read(&compressed_size) + d->cmp_len); atomic_set_release(&d->stop, 1); @@ -745,13 +748,20 @@ static int save_compressed_image(struct swap_map_handle *handle, init_waitqueue_head(&data[thr].go); init_waitqueue_head(&data[thr].done); - data[thr].cc = crypto_alloc_comp(hib_comp_algo, 0, 0); + data[thr].cc = crypto_alloc_acomp(hib_comp_algo, 0, CRYPTO_ALG_ASYNC); if (IS_ERR_OR_NULL(data[thr].cc)) { pr_err("Could not allocate comp stream %ld\n", PTR_ERR(data[thr].cc)); ret = -EFAULT; goto out_clean; } + data[thr].cr = acomp_request_alloc(data[thr].cc); + if (!data[thr].cr) { + pr_err("Could not allocate comp request\n"); + ret = -ENOMEM; + goto out_clean; + } + data[thr].thr = kthread_run(compress_threadfn, &data[thr], "image_compress/%u", thr); @@ -899,8 +909,8 @@ static int save_compressed_image(struct swap_map_handle *handle, for (thr = 0; thr < nr_threads; thr++) { if (data[thr].thr) kthread_stop(data[thr].thr); - if (data[thr].cc) - crypto_free_comp(data[thr].cc); + acomp_request_free(data[thr].cr); + crypto_free_acomp(data[thr].cc); } vfree(data); } @@ -1142,7 +1152,8 @@ static int load_image(struct swap_map_handle *handle, */ struct dec_data { struct task_struct *thr; /* thread */ - struct crypto_comp *cc; /* crypto compressor stream */ + struct crypto_acomp *cc; /* crypto compressor */ + struct acomp_req *cr; /* crypto request */ atomic_t ready; /* ready to start flag */ atomic_t stop; /* ready to stop flag */ int ret; /* return code */ @@ -1160,7 +1171,6 @@ struct dec_data { static int decompress_threadfn(void *data) { struct dec_data *d = data; - unsigned int unc_len = 0; while (1) { wait_event(d->go, atomic_read_acquire(&d->ready) || @@ -1174,10 +1184,13 @@ static int decompress_threadfn(void *data) } atomic_set(&d->ready, 0); - unc_len = UNC_SIZE; - d->ret = crypto_comp_decompress(d->cc, d->cmp + CMP_HEADER, d->cmp_len, - d->unc, &unc_len); - d->unc_len = unc_len; + acomp_request_set_callback(d->cr, CRYPTO_TFM_REQ_MAY_SLEEP, + NULL, NULL); + acomp_request_set_src_nondma(d->cr, d->cmp + CMP_HEADER, + d->cmp_len); + acomp_request_set_dst_nondma(d->cr, d->unc, UNC_SIZE); + d->ret = crypto_acomp_decompress(d->cr); + d->unc_len = d->cr->dlen; if (clean_pages_on_decompress) flush_icache_range((unsigned long)d->unc, @@ -1254,13 +1267,20 @@ static int load_compressed_image(struct swap_map_handle *handle, init_waitqueue_head(&data[thr].go); init_waitqueue_head(&data[thr].done); - data[thr].cc = crypto_alloc_comp(hib_comp_algo, 0, 0); + data[thr].cc = crypto_alloc_acomp(hib_comp_algo, 0, CRYPTO_ALG_ASYNC); if (IS_ERR_OR_NULL(data[thr].cc)) { pr_err("Could not allocate comp stream %ld\n", PTR_ERR(data[thr].cc)); ret = -EFAULT; goto out_clean; } + data[thr].cr = acomp_request_alloc(data[thr].cc); + if (!data[thr].cr) { + pr_err("Could not allocate comp request\n"); + ret = -ENOMEM; + goto out_clean; + } + data[thr].thr = kthread_run(decompress_threadfn, &data[thr], "image_decompress/%u", thr); @@ -1507,8 +1527,8 @@ static int load_compressed_image(struct swap_map_handle *handle, for (thr = 0; thr < nr_threads; thr++) { if (data[thr].thr) kthread_stop(data[thr].thr); - if (data[thr].cc) - crypto_free_comp(data[thr].cc); + acomp_request_free(data[thr].cr); + crypto_free_acomp(data[thr].cc); } vfree(data); } From patchwork Fri Mar 14 12:22:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 873713 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E5B71FECBD; Fri, 14 Mar 2025 12:23:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954996; cv=none; b=sgUfmFEJztP4J1OXgbs7U+sGe3IiV/xuANzXTK7fdIf9UzjuGoW3ucIcexvRFU0mRNyuteN5TyaLz04Bxg+GhlugtuAykXK9GXi3Z6bQqVgBvI1ikzUuCTl4Pc412/vbaDE98Wp3tDmu3W0uQp6x5/YRR+84iRqRFdToUwvF1ZI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741954996; c=relaxed/simple; bh=VM4Q6pAZsZPDq0qh2C6B1k3SAA371Gum6X84HPVg+8Y=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To:Cc; b=jLMfOCg1eJvpoZlHX7yHpdifAvtw6gIBq184voD695IaaaQWtea3X1doMdcyC7a/PbjFPkyGCuwKtR96wyADazMJnMs2c/hMq8kWPi/pMgOBUvpFxbbFuwaBzL1PTPLasX4w/XyISgqmJMYhMtqFS6p47Pfayg1crvyy2yNPVL4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=e/jH/N3n; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="e/jH/N3n" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=Cc:To:Subject:From:References:In-Reply-To:Message-Id:Date: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=2X3eDxot5tjnnXYocKH3npnxgHjqvCSOBPQWfrr1UJg=; b=e/jH/N3nuMDhQJZd196wpDE7Ai l8gkrES2rUsPwT/o1wlszmk3wVEalPEF7gHeYwds9e+RsFI6m121+mhVQqu991yvJqsZ/vjQ1R8V+ K+j1nKA6G2F1Y9/Ix86mYYEvvdvP74Is5xWbtppdPPE3fQ79m8C5DGeem4F+ey4dpNu93VcBuhSs4 dsrvpwgL8+6WJHvRNcnOo0i1yNfmc79fLLFuKd/uhYHVVhpSGyppss4vqf71rppd+WVbAXHu+2yJv UhnsEcvJtSkLbvsCZUXoMu6Sbsb/WhzA/SQn7VfUxLjju0ffN81Z5OlxzlyEa2kDByeAF9m23c9X9 xQIYDlcg==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1tt43y-006ZpD-04; Fri, 14 Mar 2025 20:22:55 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Fri, 14 Mar 2025 20:22:54 +0800 Date: Fri, 14 Mar 2025 20:22:54 +0800 Message-Id: In-Reply-To: References: From: Herbert Xu Subject: [v4 PATCH 13/13] xfrm: ipcomp: Use crypto_acomp interface To: Linux Crypto Mailing List Cc: Richard Weinberger , Zhihao Cheng , linux-mtd@lists.infradead.org, "Rafael J. Wysocki" , Pavel Machek , linux-pm@vger.kernel.org, Steffen Klassert , netdev@vger.kernel.org Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Replace the legacy comperssion interface with the new acomp interface. This is the first user to make full user of the asynchronous nature of acomp by plugging into the existing xfrm resume interface. As a result of SG support by acomp, the linear scratch buffer in ipcomp can be removed. Signed-off-by: Herbert Xu --- include/net/ipcomp.h | 13 +- net/xfrm/xfrm_ipcomp.c | 477 ++++++++++++++++++++--------------------- 2 files changed, 233 insertions(+), 257 deletions(-) diff --git a/include/net/ipcomp.h b/include/net/ipcomp.h index 8660a2a6d1fc..51401f01e2a5 100644 --- a/include/net/ipcomp.h +++ b/include/net/ipcomp.h @@ -3,20 +3,9 @@ #define _NET_IPCOMP_H #include -#include - -#define IPCOMP_SCRATCH_SIZE 65400 - -struct crypto_comp; -struct ip_comp_hdr; - -struct ipcomp_data { - u16 threshold; - struct crypto_comp * __percpu *tfms; -}; struct ip_comp_hdr; -struct sk_buff; +struct netlink_ext_ack; struct xfrm_state; int ipcomp_input(struct xfrm_state *x, struct sk_buff *skb); diff --git a/net/xfrm/xfrm_ipcomp.c b/net/xfrm/xfrm_ipcomp.c index 43eae94e4b0e..a5246227951f 100644 --- a/net/xfrm/xfrm_ipcomp.c +++ b/net/xfrm/xfrm_ipcomp.c @@ -3,7 +3,7 @@ * IP Payload Compression Protocol (IPComp) - RFC3173. * * Copyright (c) 2003 James Morris - * Copyright (c) 2003-2008 Herbert Xu + * Copyright (c) 2003-2025 Herbert Xu * * Todo: * - Tunable compression parameters. @@ -11,169 +11,240 @@ * - Adaptive compression. */ -#include +#include #include -#include #include -#include -#include +#include #include -#include -#include -#include #include #include -struct ipcomp_tfms { - struct list_head list; - struct crypto_comp * __percpu *tfms; - int users; +#define IPCOMP_SCRATCH_SIZE 65400 + +struct ipcomp_skb_cb { + struct xfrm_skb_cb xfrm; + struct acomp_req *req; }; -static DEFINE_MUTEX(ipcomp_resource_mutex); -static void * __percpu *ipcomp_scratches; -static int ipcomp_scratch_users; -static LIST_HEAD(ipcomp_tfms_list); +struct ipcomp_data { + u16 threshold; + struct crypto_acomp *tfm; +}; -static int ipcomp_decompress(struct xfrm_state *x, struct sk_buff *skb) +struct ipcomp_req_extra { + struct xfrm_state *x; + struct scatterlist sg[]; +}; + +static inline struct ipcomp_skb_cb *ipcomp_cb(struct sk_buff *skb) { - struct ipcomp_data *ipcd = x->data; - const int plen = skb->len; - int dlen = IPCOMP_SCRATCH_SIZE; - const u8 *start = skb->data; - u8 *scratch = *this_cpu_ptr(ipcomp_scratches); - struct crypto_comp *tfm = *this_cpu_ptr(ipcd->tfms); - int err = crypto_comp_decompress(tfm, start, plen, scratch, &dlen); - int len; + struct ipcomp_skb_cb *cb = (void *)skb->cb; - if (err) - return err; + BUILD_BUG_ON(sizeof(*cb) > sizeof(skb->cb)); + return cb; +} - if (dlen < (plen + sizeof(struct ip_comp_hdr))) - return -EINVAL; +static int ipcomp_post_acomp(struct sk_buff *skb, int err, int hlen) +{ + struct acomp_req *req = ipcomp_cb(skb)->req; + struct ipcomp_req_extra *extra; + const int plen = skb->data_len; + struct scatterlist *dsg; + int len, dlen; - len = dlen - plen; - if (len > skb_tailroom(skb)) - len = skb_tailroom(skb); + if (unlikely(err)) + goto out_free_req; - __skb_put(skb, len); + extra = acomp_request_extra(req); + dsg = extra->sg; + dlen = req->dlen; - len += plen; - skb_copy_to_linear_data(skb, scratch, len); + pskb_trim_unique(skb, 0); + __skb_put(skb, hlen); - while ((scratch += len, dlen -= len) > 0) { + /* Only update truesize on input. */ + if (!hlen) + skb->truesize += dlen - plen; + skb->data_len = dlen; + skb->len += dlen; + + do { skb_frag_t *frag; struct page *page; - if (WARN_ON(skb_shinfo(skb)->nr_frags >= MAX_SKB_FRAGS)) - return -EMSGSIZE; - frag = skb_shinfo(skb)->frags + skb_shinfo(skb)->nr_frags; - page = alloc_page(GFP_ATOMIC); - - if (!page) - return -ENOMEM; + page = sg_page(dsg); + dsg = sg_next(dsg); len = PAGE_SIZE; if (dlen < len) len = dlen; skb_frag_fill_page_desc(frag, page, 0, len); - memcpy(skb_frag_address(frag), scratch, len); - - skb->truesize += len; - skb->data_len += len; - skb->len += len; skb_shinfo(skb)->nr_frags++; - } + } while ((dlen -= len)); - return 0; + for (; dsg; dsg = sg_next(dsg)) + __free_page(sg_page(dsg)); + +out_free_req: + acomp_request_free(req); + return err; +} + +static int ipcomp_input_done2(struct sk_buff *skb, int err) +{ + struct ip_comp_hdr *ipch = ip_comp_hdr(skb); + const int plen = skb->len; + + skb_reset_transport_header(skb); + + return ipcomp_post_acomp(skb, err, 0) ?: + skb->len < (plen + sizeof(ip_comp_hdr)) ? -EINVAL : + ipch->nexthdr; +} + +static void ipcomp_input_done(void *data, int err) +{ + struct sk_buff *skb = data; + + xfrm_input_resume(skb, ipcomp_input_done2(skb, err)); +} + +static struct acomp_req *ipcomp_setup_req(struct xfrm_state *x, + struct sk_buff *skb, int minhead, + int dlen) +{ + const int dnfrags = min(MAX_SKB_FRAGS, 16); + struct ipcomp_data *ipcd = x->data; + struct ipcomp_req_extra *extra; + struct scatterlist *sg, *dsg; + const int plen = skb->len; + struct crypto_acomp *tfm; + struct acomp_req *req; + int nfrags; + int total; + int err; + int i; + + ipcomp_cb(skb)->req = NULL; + + do { + struct sk_buff *trailer; + + if (skb->len > PAGE_SIZE) { + if (skb_linearize_cow(skb)) + return ERR_PTR(-ENOMEM); + nfrags = 1; + break; + } + + if (!skb_cloned(skb) && skb_headlen(skb) >= minhead) { + if (!skb_is_nonlinear(skb)) { + nfrags = 1; + break; + } else if (!skb_has_frag_list(skb)) { + nfrags = skb_shinfo(skb)->nr_frags; + nfrags++; + break; + } + } + + nfrags = skb_cow_data(skb, skb_headlen(skb) < minhead ? + minhead - skb_headlen(skb) : 0, + &trailer); + if (nfrags < 0) + return ERR_PTR(nfrags); + } while (0); + + tfm = ipcd->tfm; + req = acomp_request_alloc_extra( + tfm, sizeof(*extra) + sizeof(*sg) * (nfrags + dnfrags), + GFP_ATOMIC); + ipcomp_cb(skb)->req = req; + if (!req) + return ERR_PTR(-ENOMEM); + + extra = acomp_request_extra(req); + extra->x = x; + + dsg = extra->sg; + sg = dsg + dnfrags; + sg_init_table(sg, nfrags); + err = skb_to_sgvec(skb, sg, 0, plen); + if (unlikely(err < 0)) + return ERR_PTR(err); + + sg_init_table(dsg, dnfrags); + total = 0; + for (i = 0; i < dnfrags && total < dlen; i++) { + struct page *page; + + page = alloc_page(GFP_ATOMIC); + if (!page) + break; + sg_set_page(dsg + i, page, PAGE_SIZE, 0); + total += PAGE_SIZE; + } + if (!i) + return ERR_PTR(-ENOMEM); + sg_mark_end(dsg + i - 1); + + acomp_request_set_params(req, sg, dsg, plen, dlen); + + return req; +} + +static int ipcomp_decompress(struct xfrm_state *x, struct sk_buff *skb) +{ + struct acomp_req *req; + int err; + + req = ipcomp_setup_req(x, skb, 0, IPCOMP_SCRATCH_SIZE); + err = PTR_ERR(req); + if (IS_ERR(req)) + goto out; + + acomp_request_set_callback(req, 0, ipcomp_input_done, skb); + err = crypto_acomp_decompress(req); + if (err == -EINPROGRESS) + return err; + +out: + return ipcomp_input_done2(skb, err); } int ipcomp_input(struct xfrm_state *x, struct sk_buff *skb) { - int nexthdr; - int err = -ENOMEM; - struct ip_comp_hdr *ipch; + struct ip_comp_hdr *ipch __maybe_unused; if (!pskb_may_pull(skb, sizeof(*ipch))) return -EINVAL; - if (skb_linearize_cow(skb)) - goto out; - skb->ip_summed = CHECKSUM_NONE; /* Remove ipcomp header and decompress original payload */ - ipch = (void *)skb->data; - nexthdr = ipch->nexthdr; - - skb->transport_header = skb->network_header + sizeof(*ipch); __skb_pull(skb, sizeof(*ipch)); - err = ipcomp_decompress(x, skb); - if (err) - goto out; - err = nexthdr; - -out: - return err; + return ipcomp_decompress(x, skb); } EXPORT_SYMBOL_GPL(ipcomp_input); -static int ipcomp_compress(struct xfrm_state *x, struct sk_buff *skb) +static int ipcomp_output_push(struct sk_buff *skb) { - struct ipcomp_data *ipcd = x->data; - const int plen = skb->len; - int dlen = IPCOMP_SCRATCH_SIZE; - u8 *start = skb->data; - struct crypto_comp *tfm; - u8 *scratch; - int err; - - local_bh_disable(); - scratch = *this_cpu_ptr(ipcomp_scratches); - tfm = *this_cpu_ptr(ipcd->tfms); - err = crypto_comp_compress(tfm, start, plen, scratch, &dlen); - if (err) - goto out; - - if ((dlen + sizeof(struct ip_comp_hdr)) >= plen) { - err = -EMSGSIZE; - goto out; - } - - memcpy(start + sizeof(struct ip_comp_hdr), scratch, dlen); - local_bh_enable(); - - pskb_trim(skb, dlen + sizeof(struct ip_comp_hdr)); + skb_push(skb, -skb_network_offset(skb)); return 0; - -out: - local_bh_enable(); - return err; } -int ipcomp_output(struct xfrm_state *x, struct sk_buff *skb) +static int ipcomp_output_done2(struct xfrm_state *x, struct sk_buff *skb, + int err) { - int err; struct ip_comp_hdr *ipch; - struct ipcomp_data *ipcd = x->data; - if (skb->len < ipcd->threshold) { - /* Don't bother compressing */ + err = ipcomp_post_acomp(skb, err, sizeof(*ipch)); + if (err) goto out_ok; - } - - if (skb_linearize_cow(skb)) - goto out_ok; - - err = ipcomp_compress(x, skb); - - if (err) { - goto out_ok; - } /* Install ipcomp header, convert into ipcomp datagram. */ ipch = ip_comp_hdr(skb); @@ -182,135 +253,59 @@ int ipcomp_output(struct xfrm_state *x, struct sk_buff *skb) ipch->cpi = htons((u16 )ntohl(x->id.spi)); *skb_mac_header(skb) = IPPROTO_COMP; out_ok: - skb_push(skb, -skb_network_offset(skb)); - return 0; + return ipcomp_output_push(skb); +} + +static void ipcomp_output_done(void *data, int err) +{ + struct ipcomp_req_extra *extra; + struct sk_buff *skb = data; + struct acomp_req *req; + + req = ipcomp_cb(skb)->req; + extra = acomp_request_extra(req); + + xfrm_output_resume(skb_to_full_sk(skb), skb, + ipcomp_output_done2(extra->x, skb, err)); +} + +static int ipcomp_compress(struct xfrm_state *x, struct sk_buff *skb) +{ + struct ip_comp_hdr *ipch __maybe_unused; + struct acomp_req *req; + int err; + + req = ipcomp_setup_req(x, skb, sizeof(*ipch), + skb->len - sizeof(*ipch)); + err = PTR_ERR(req); + if (IS_ERR(req)) + goto out; + + acomp_request_set_callback(req, 0, ipcomp_output_done, skb); + err = crypto_acomp_compress(req); + if (err == -EINPROGRESS) + return err; + +out: + return ipcomp_output_done2(x, skb, err); +} + +int ipcomp_output(struct xfrm_state *x, struct sk_buff *skb) +{ + struct ipcomp_data *ipcd = x->data; + + if (skb->len < ipcd->threshold) { + /* Don't bother compressing */ + return ipcomp_output_push(skb); + } + + return ipcomp_compress(x, skb); } EXPORT_SYMBOL_GPL(ipcomp_output); -static void ipcomp_free_scratches(void) -{ - int i; - void * __percpu *scratches; - - if (--ipcomp_scratch_users) - return; - - scratches = ipcomp_scratches; - if (!scratches) - return; - - for_each_possible_cpu(i) - vfree(*per_cpu_ptr(scratches, i)); - - free_percpu(scratches); - ipcomp_scratches = NULL; -} - -static void * __percpu *ipcomp_alloc_scratches(void) -{ - void * __percpu *scratches; - int i; - - if (ipcomp_scratch_users++) - return ipcomp_scratches; - - scratches = alloc_percpu(void *); - if (!scratches) - return NULL; - - ipcomp_scratches = scratches; - - for_each_possible_cpu(i) { - void *scratch; - - scratch = vmalloc_node(IPCOMP_SCRATCH_SIZE, cpu_to_node(i)); - if (!scratch) - return NULL; - *per_cpu_ptr(scratches, i) = scratch; - } - - return scratches; -} - -static void ipcomp_free_tfms(struct crypto_comp * __percpu *tfms) -{ - struct ipcomp_tfms *pos; - int cpu; - - list_for_each_entry(pos, &ipcomp_tfms_list, list) { - if (pos->tfms == tfms) - break; - } - - WARN_ON(list_entry_is_head(pos, &ipcomp_tfms_list, list)); - - if (--pos->users) - return; - - list_del(&pos->list); - kfree(pos); - - if (!tfms) - return; - - for_each_possible_cpu(cpu) { - struct crypto_comp *tfm = *per_cpu_ptr(tfms, cpu); - crypto_free_comp(tfm); - } - free_percpu(tfms); -} - -static struct crypto_comp * __percpu *ipcomp_alloc_tfms(const char *alg_name) -{ - struct ipcomp_tfms *pos; - struct crypto_comp * __percpu *tfms; - int cpu; - - - list_for_each_entry(pos, &ipcomp_tfms_list, list) { - struct crypto_comp *tfm; - - /* This can be any valid CPU ID so we don't need locking. */ - tfm = this_cpu_read(*pos->tfms); - - if (!strcmp(crypto_comp_name(tfm), alg_name)) { - pos->users++; - return pos->tfms; - } - } - - pos = kmalloc(sizeof(*pos), GFP_KERNEL); - if (!pos) - return NULL; - - pos->users = 1; - INIT_LIST_HEAD(&pos->list); - list_add(&pos->list, &ipcomp_tfms_list); - - pos->tfms = tfms = alloc_percpu(struct crypto_comp *); - if (!tfms) - goto error; - - for_each_possible_cpu(cpu) { - struct crypto_comp *tfm = crypto_alloc_comp(alg_name, 0, - CRYPTO_ALG_ASYNC); - if (IS_ERR(tfm)) - goto error; - *per_cpu_ptr(tfms, cpu) = tfm; - } - - return tfms; - -error: - ipcomp_free_tfms(tfms); - return NULL; -} - static void ipcomp_free_data(struct ipcomp_data *ipcd) { - if (ipcd->tfms) - ipcomp_free_tfms(ipcd->tfms); - ipcomp_free_scratches(); + crypto_free_acomp(ipcd->tfm); } void ipcomp_destroy(struct xfrm_state *x) @@ -319,9 +314,7 @@ void ipcomp_destroy(struct xfrm_state *x) if (!ipcd) return; xfrm_state_delete_tunnel(x); - mutex_lock(&ipcomp_resource_mutex); ipcomp_free_data(ipcd); - mutex_unlock(&ipcomp_resource_mutex); kfree(ipcd); } EXPORT_SYMBOL_GPL(ipcomp_destroy); @@ -348,15 +341,10 @@ int ipcomp_init_state(struct xfrm_state *x, struct netlink_ext_ack *extack) if (!ipcd) goto out; - mutex_lock(&ipcomp_resource_mutex); - if (!ipcomp_alloc_scratches()) + ipcd->tfm = crypto_alloc_acomp(x->calg->alg_name, 0, 0); + if (IS_ERR(ipcd->tfm)) goto error; - ipcd->tfms = ipcomp_alloc_tfms(x->calg->alg_name); - if (!ipcd->tfms) - goto error; - mutex_unlock(&ipcomp_resource_mutex); - calg_desc = xfrm_calg_get_byname(x->calg->alg_name, 0); BUG_ON(!calg_desc); ipcd->threshold = calg_desc->uinfo.comp.threshold; @@ -367,7 +355,6 @@ int ipcomp_init_state(struct xfrm_state *x, struct netlink_ext_ack *extack) error: ipcomp_free_data(ipcd); - mutex_unlock(&ipcomp_resource_mutex); kfree(ipcd); goto out; }