From patchwork Thu Feb 27 10:14:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 869078 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1A2C227E95 for ; Thu, 27 Feb 2025 10:14:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740651302; cv=none; b=iLEhO3KPa5IqPQja0Uh0Pf2nYTj7W393Gb8LRUlK+aSBm8FNpGGgvd+lkkmRL/aG6/EJUrp+shdioyNP66U+JecUoUV1SQIXb3Us2SYzE3S94lK4q81Yii3otoe3uVNcPQwbueKWj+Z7Hryv3K2tRL0Bu0pES9OwV83MDt2Yxp0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740651302; c=relaxed/simple; bh=K7VNYRDV1yfJVKgFsqppm610jih+a7OPEQ+mKbsc+N0=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To:Cc; b=JAVycvcyfmyVtATRykQVhBmjD4Un9OdZiOAkFxSwDoU2B1GNZvnQweqHwrp1IiwzixClJmLxnweXPnuWIGQqGqquIB14Z5MvnGiJZ8MrFhoJFF7WiI7om1Aj2oqxHnasOsuSfCjelohRgrdIkcG3hNWyc5asZCmfH1rZynIHjak= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=dTTXozfv; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="dTTXozfv" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=Cc:To:Subject:From:References:In-Reply-To:Message-Id:Date: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=B0mE1hvDwybCOi8Ma7SncwsCTkexIE9OZoKz2lEpXNE=; b=dTTXozfvVo/hJt1WOaRBEC3ouX lqowE/dGvjahetzBhkorXNlmG//rfv/fUaxwOgjcg8clQPZBqs+/m+5/i1PCqQYnvPUGtL4oPi1e3 JQJF+4/Z5RfUzxOoN6ihJZrfe3cppEbpToVSRGkFvPU4jLvtdQYvWa0Uz75B9YH/Eic0wVqA9lt1r 2FanPtFLMIK6B2e12+dtkq4ah3sK4phQfNrZvGghq/QbpuRHguDdsA3fGPDxZL5Ey7Ja/c/CsAY83 a17cAj0AS4spz8/cIssGRzF/mOWpfwXYjk/ngmwWTgMpOd2SJOBgvoDBzM4/QsjEOZh2E0OZ+uhX+ jxRUidLA==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1tnaut-002Dqc-2J; Thu, 27 Feb 2025 18:14:56 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Thu, 27 Feb 2025 18:14:55 +0800 Date: Thu, 27 Feb 2025 18:14:55 +0800 Message-Id: In-Reply-To: References: From: Herbert Xu Subject: [PATCH 1/7] crypto: iaa - Test the correct request flag To: Linux Crypto Mailing List Cc: linux-mm@kvack.org, Yosry Ahmed Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Test the correct flags for the MAY_SLEEP bit. Fixes: 2ec6761df889 ("crypto: iaa - Add support for deflate-iaa compression algorithm") Signed-off-by: Herbert Xu --- drivers/crypto/intel/iaa/iaa_crypto_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/crypto/intel/iaa/iaa_crypto_main.c b/drivers/crypto/intel/iaa/iaa_crypto_main.c index c3776b0de51d..990ea46955bb 100644 --- a/drivers/crypto/intel/iaa/iaa_crypto_main.c +++ b/drivers/crypto/intel/iaa/iaa_crypto_main.c @@ -1537,7 +1537,7 @@ static int iaa_comp_acompress(struct acomp_req *req) iaa_wq = idxd_wq_get_private(wq); if (!req->dst) { - gfp_t flags = req->flags & CRYPTO_TFM_REQ_MAY_SLEEP ? GFP_KERNEL : GFP_ATOMIC; + gfp_t flags = req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP ? GFP_KERNEL : GFP_ATOMIC; /* incompressible data will always be < 2 * slen */ req->dlen = 2 * req->slen; @@ -1619,7 +1619,7 @@ static int iaa_comp_acompress(struct acomp_req *req) static int iaa_comp_adecompress_alloc_dest(struct acomp_req *req) { - gfp_t flags = req->flags & CRYPTO_TFM_REQ_MAY_SLEEP ? + gfp_t flags = req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP ? GFP_KERNEL : GFP_ATOMIC; struct crypto_tfm *tfm = req->base.tfm; dma_addr_t src_addr, dst_addr; From patchwork Thu Feb 27 10:15:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 869077 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C6CC22A4E0 for ; Thu, 27 Feb 2025 10:15:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740651306; cv=none; b=nmiHD2ZWM7EfeEi5c7w8Xxe8YONgmVn2kKEAZXTW8MusHrtX3+V+9MXUTq+4lNI5eQc0KjtfMkJFPdlW68yB6UQ4Xhm16PCJQBMcqBMZRDi9peFn1AP+8QYC5rcrOmuFmBg7juaDaI8klYs9+TyaSKvrGM5lCYCpk3ITKLNql68= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740651306; c=relaxed/simple; bh=983/R3Wrh+r4T8gJqoNIDGV/NxBoLkMuaxzLgQnbS04=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To:Cc; b=Qj2LzURV90n5daY0crT2LuVIolXnxIhfFVcnzMvZVYKMnBHRyxrS7QauzXerHPZ88rak+eFwek63o+0w58b1UV3HLz2dKy+RN21GN+QcduEs/FQEDPxJwi87YU/e+a5ZYVvLpbLdNnUvxns1EChgSJoCzd06hHJq3rMt3T11ygI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=b0jjzhxD; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="b0jjzhxD" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=Cc:To:Subject:From:References:In-Reply-To:Message-Id:Date: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=5bLm0Q94gEJgQpTepUh8jBt4BX7iBQgckVIz1+HDSw8=; b=b0jjzhxDEm4w22vBdiS01nujcH tOwm/kRJycBH+iY8c3ynLYLldXGucQyNXLUH1C5HN8sVPf89f22M0/gB0y6c0Ww4UJVJgVmgITy1l mE3ukpTSVLy9hs7EBC8WzHQuT8inkTgul5MSFDk+X1nImEQ8/geww9KXM+i3mjD8B3lGT20jwhYF2 3qqjBxTAbZ1Hv2hxsQO1OYQh7RwCF47ccJyRUmIdYB6n1i7HUfZ5q63fzpfUAZ8fg6guKiG6VBivL QXVCaFTGPzO+WvRWzpdVhFK/myDTZPnu10hxF7axIZiVVfryaOihfYxYxUaVj6YDOOglRRIyrmmve VWK91Fbg==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1tnauy-002Dqy-10; Thu, 27 Feb 2025 18:15:01 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Thu, 27 Feb 2025 18:15:00 +0800 Date: Thu, 27 Feb 2025 18:15:00 +0800 Message-Id: In-Reply-To: References: From: Herbert Xu Subject: [PATCH 3/7] crypto: acomp - Add request chaining and virtual addresses To: Linux Crypto Mailing List Cc: linux-mm@kvack.org, Yosry Ahmed Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: This adds request chaining and virtual address support to the acomp interface. It is identical to the ahash interface, except that a new flag CRYPTO_ACOMP_REQ_NONDMA has been added to indicate that the virtual addresses are not suitable for DMA. This is because all existing and potential acomp users can provide memory that is suitable for DMA so there is no need for a fall-back copy path. Signed-off-by: Herbert Xu --- crypto/acompress.c | 201 ++++++++++++++++++++++++++++ include/crypto/acompress.h | 89 ++++++++++-- include/crypto/internal/acompress.h | 22 +++ 3 files changed, 299 insertions(+), 13 deletions(-) diff --git a/crypto/acompress.c b/crypto/acompress.c index 30176316140a..d2103d4e42cc 100644 --- a/crypto/acompress.c +++ b/crypto/acompress.c @@ -23,6 +23,8 @@ struct crypto_scomp; static const struct crypto_type crypto_acomp_type; +static void acomp_reqchain_done(void *data, int err); + static inline struct acomp_alg *__crypto_acomp_alg(struct crypto_alg *alg) { return container_of(alg, struct acomp_alg, calg.base); @@ -153,6 +155,205 @@ void acomp_request_free(struct acomp_req *req) } EXPORT_SYMBOL_GPL(acomp_request_free); +static bool acomp_request_has_nondma(struct acomp_req *req) +{ + struct acomp_req *r2; + + if (acomp_request_isnondma(req)) + return true; + + list_for_each_entry(r2, &req->base.list, base.list) + if (acomp_request_isnondma(r2)) + return true; + + return false; +} + +static void acomp_save_req(struct acomp_req *req, crypto_completion_t cplt) +{ + struct crypto_acomp *tfm = crypto_acomp_reqtfm(req); + struct acomp_req_chain *state = &req->chain; + + if (!acomp_is_async(tfm)) + return; + + state->compl = req->base.complete; + state->data = req->base.data; + req->base.complete = cplt; + req->base.data = state; + state->req0 = req; +} + +static void acomp_restore_req(struct acomp_req_chain *state) +{ + struct acomp_req *req = state->req0; + struct crypto_acomp *tfm; + + tfm = crypto_acomp_reqtfm(req); + if (!acomp_is_async(tfm)) + return; + + req->base.complete = state->compl; + req->base.data = state->data; +} + +static void acomp_reqchain_virt(struct acomp_req_chain *state, int err) +{ + struct acomp_req *req = state->cur; + unsigned int slen = req->slen; + unsigned int dlen = req->dlen; + + req->base.err = err; + if (!state->src) + return; + + acomp_request_set_virt(req, state->src, state->dst, slen, dlen); + state->src = NULL; +} + +static int acomp_reqchain_finish(struct acomp_req_chain *state, + int err, u32 mask) +{ + struct acomp_req *req0 = state->req0; + struct acomp_req *req = state->cur; + struct acomp_req *n; + + acomp_reqchain_virt(state, err); + + if (req != req0) + list_add_tail(&req->base.list, &req0->base.list); + + list_for_each_entry_safe(req, n, &state->head, base.list) { + list_del_init(&req->base.list); + + req->base.flags &= mask; + req->base.complete = acomp_reqchain_done; + req->base.data = state; + state->cur = req; + + if (acomp_request_isvirt(req)) { + unsigned int slen = req->slen; + unsigned int dlen = req->dlen; + const u8 *svirt = req->svirt; + u8 *dvirt = req->dvirt; + + state->src = svirt; + state->dst = dvirt; + + sg_init_one(&state->ssg, svirt, slen); + sg_init_one(&state->dsg, dvirt, dlen); + + acomp_request_set_params(req, &state->ssg, &state->dsg, + slen, dlen); + } + + err = state->op(req); + + if (err == -EINPROGRESS) { + if (!list_empty(&state->head)) + err = -EBUSY; + goto out; + } + + if (err == -EBUSY) + goto out; + + acomp_reqchain_virt(state, err); + list_add_tail(&req->base.list, &req0->base.list); + } + + acomp_restore_req(state); + +out: + return err; +} + +static void acomp_reqchain_done(void *data, int err) +{ + struct acomp_req_chain *state = data; + crypto_completion_t compl = state->compl; + + data = state->data; + + if (err == -EINPROGRESS) { + if (!list_empty(&state->head)) + return; + goto notify; + } + + err = acomp_reqchain_finish(state, err, CRYPTO_TFM_REQ_MAY_BACKLOG); + if (err == -EBUSY) + return; + +notify: + compl(data, err); +} + +static int acomp_do_req_chain(struct acomp_req *req, + int (*op)(struct acomp_req *req)) +{ + struct crypto_acomp *tfm = crypto_acomp_reqtfm(req); + struct acomp_req_chain *state = &req->chain; + int err; + + if (crypto_acomp_req_chain(tfm) || + (!acomp_request_chained(req) && !acomp_request_isvirt(req))) + return op(req); + + /* + * There are no in-kernel users that do this. If and ever + * such users come into being then we could add a fall-back + * path. + */ + if (acomp_request_has_nondma(req)) + return -EINVAL; + + if (acomp_is_async(tfm)) { + acomp_save_req(req, acomp_reqchain_done); + state = req->base.data; + } + + state->op = op; + state->cur = req; + state->src = NULL; + INIT_LIST_HEAD(&state->head); + list_splice_init(&req->base.list, &state->head); + + if (acomp_request_isvirt(req)) { + unsigned int slen = req->slen; + unsigned int dlen = req->dlen; + const u8 *svirt = req->svirt; + u8 *dvirt = req->dvirt; + + state->src = svirt; + state->dst = dvirt; + + sg_init_one(&state->ssg, svirt, slen); + sg_init_one(&state->dsg, dvirt, dlen); + + acomp_request_set_params(req, &state->ssg, &state->dsg, + slen, dlen); + } + + err = op(req); + if (err == -EBUSY || err == -EINPROGRESS) + return -EBUSY; + + return acomp_reqchain_finish(state, err, ~0); +} + +int crypto_acomp_compress(struct acomp_req *req) +{ + return acomp_do_req_chain(req, crypto_acomp_reqtfm(req)->compress); +} +EXPORT_SYMBOL_GPL(crypto_acomp_compress); + +int crypto_acomp_decompress(struct acomp_req *req) +{ + return acomp_do_req_chain(req, crypto_acomp_reqtfm(req)->decompress); +} +EXPORT_SYMBOL_GPL(crypto_acomp_decompress); + void comp_prepare_alg(struct comp_alg_common *alg) { struct crypto_alg *base = &alg->base; diff --git a/include/crypto/acompress.h b/include/crypto/acompress.h index b6d5136e689d..15bb13e47f8b 100644 --- a/include/crypto/acompress.h +++ b/include/crypto/acompress.h @@ -12,10 +12,34 @@ #include #include #include +#include +#include #define CRYPTO_ACOMP_ALLOC_OUTPUT 0x00000001 + +/* Set this bit for virtual address instead of SG list. */ +#define CRYPTO_ACOMP_REQ_VIRT 0x00000002 + +/* Set this bit for if virtual address buffer cannot be used for DMA. */ +#define CRYPTO_ACOMP_REQ_NONDMA 0x00000004 + #define CRYPTO_ACOMP_DST_MAX 131072 +struct acomp_req; + +struct acomp_req_chain { + struct list_head head; + struct acomp_req *req0; + struct acomp_req *cur; + int (*op)(struct acomp_req *req); + crypto_completion_t compl; + void *data; + struct scatterlist ssg; + struct scatterlist dsg; + const u8 *src; + u8 *dst; +}; + /** * struct acomp_req - asynchronous (de)compression request * @@ -24,14 +48,24 @@ * @dst: Destination data * @slen: Size of the input buffer * @dlen: Size of the output buffer and number of bytes produced + * @chain: Private API code data, do not use * @__ctx: Start of private context data */ struct acomp_req { struct crypto_async_request base; - struct scatterlist *src; - struct scatterlist *dst; + union { + struct scatterlist *src; + const u8 *svirt; + }; + union { + struct scatterlist *dst; + u8 *dvirt; + }; unsigned int slen; unsigned int dlen; + + struct acomp_req_chain chain; + void *__ctx[] CRYPTO_MINALIGN_ATTR; }; @@ -200,10 +234,14 @@ static inline void acomp_request_set_callback(struct acomp_req *req, crypto_completion_t cmpl, void *data) { + u32 keep = CRYPTO_ACOMP_ALLOC_OUTPUT | CRYPTO_ACOMP_REQ_VIRT; + req->base.complete = cmpl; req->base.data = data; - req->base.flags &= CRYPTO_ACOMP_ALLOC_OUTPUT; - req->base.flags |= flgs & ~CRYPTO_ACOMP_ALLOC_OUTPUT; + req->base.flags &= keep; + req->base.flags |= flgs & ~keep; + + crypto_reqchain_init(&req->base); } /** @@ -230,11 +268,42 @@ static inline void acomp_request_set_params(struct acomp_req *req, req->slen = slen; req->dlen = dlen; - req->base.flags &= ~CRYPTO_ACOMP_ALLOC_OUTPUT; + req->base.flags &= ~(CRYPTO_ACOMP_ALLOC_OUTPUT | CRYPTO_ACOMP_REQ_VIRT); if (!req->dst) req->base.flags |= CRYPTO_ACOMP_ALLOC_OUTPUT; } +/** + * acomp_request_set_virt() -- Sets virtual address request parameters + * + * Sets virtual address parameters required by an acomp operation + * + * @req: asynchronous compress request + * @src: virtual address pointer to input buffer + * @dst: virtual address pointer to output buffer. + * @slen: size of the input buffer + * @dlen: size of the output buffer. + */ +static inline void acomp_request_set_virt(struct acomp_req *req, + const u8 *src, u8 *dst, + unsigned int slen, + unsigned int dlen) +{ + req->svirt = src; + req->dvirt = dst; + req->slen = slen; + req->dlen = dlen; + + req->base.flags &= ~CRYPTO_ACOMP_ALLOC_OUTPUT; + req->base.flags |= CRYPTO_ACOMP_REQ_VIRT; +} + +static inline void acomp_request_chain(struct acomp_req *req, + struct acomp_req *head) +{ + crypto_request_chain(&req->base, &head->base); +} + /** * crypto_acomp_compress() -- Invoke asynchronous compress operation * @@ -244,10 +313,7 @@ static inline void acomp_request_set_params(struct acomp_req *req, * * Return: zero on success; error code in case of error */ -static inline int crypto_acomp_compress(struct acomp_req *req) -{ - return crypto_acomp_reqtfm(req)->compress(req); -} +int crypto_acomp_compress(struct acomp_req *req); /** * crypto_acomp_decompress() -- Invoke asynchronous decompress operation @@ -258,9 +324,6 @@ static inline int crypto_acomp_compress(struct acomp_req *req) * * Return: zero on success; error code in case of error */ -static inline int crypto_acomp_decompress(struct acomp_req *req) -{ - return crypto_acomp_reqtfm(req)->decompress(req); -} +int crypto_acomp_decompress(struct acomp_req *req); #endif diff --git a/include/crypto/internal/acompress.h b/include/crypto/internal/acompress.h index 8831edaafc05..b3b48dea7f2f 100644 --- a/include/crypto/internal/acompress.h +++ b/include/crypto/internal/acompress.h @@ -109,4 +109,26 @@ void crypto_unregister_acomp(struct acomp_alg *alg); int crypto_register_acomps(struct acomp_alg *algs, int count); void crypto_unregister_acomps(struct acomp_alg *algs, int count); +static inline bool acomp_request_chained(struct acomp_req *req) +{ + return crypto_request_chained(&req->base); +} + +static inline bool acomp_request_isvirt(struct acomp_req *req) +{ + return req->base.flags & CRYPTO_ACOMP_REQ_VIRT; +} + +static inline bool acomp_request_isnondma(struct acomp_req *req) +{ + return (req->base.flags & + (CRYPTO_ACOMP_REQ_NONDMA | CRYPTO_ACOMP_REQ_VIRT)) == + (CRYPTO_ACOMP_REQ_NONDMA | CRYPTO_ACOMP_REQ_VIRT); +} + +static inline bool crypto_acomp_req_chain(struct crypto_acomp *tfm) +{ + return crypto_tfm_req_chain(&tfm->base); +} + #endif From patchwork Thu Feb 27 10:15:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 869076 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DF3C6229B0D for ; Thu, 27 Feb 2025 10:15:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740651311; cv=none; b=QthGKbjIxIcaNCVikwsg9ObvFVk4QuXO/1QNjZVISWRJxCBjDKH3yYaILLMOrR4f5SrcxZyIbQwwvR25suyqLvdoOJ5/0QG1TpAhLUqWHvYzl/p6D2CaPgYPPQjfd8IXOAEm8572EeNwEyQvVW9XhG35M8MCYWyPnBlgeYKxZCQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740651311; c=relaxed/simple; bh=LvQVjH0+P2+mWaNaLQeuHJHr7Ix5DGyTIaSVmO37Ufg=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To:Cc; b=L7+qQ4fHLmUkIaEqmlNciLJXcrcBLuNOzc+lCgBPadCO1ooHA0uyKs4fB2gsOyb8qKI5UpYCp4LKG1zv3j8BM1vthkNJdQsZilOWXANaBN9H6XyfJS5XonHYyQOy4dB/pI2DI8mm/3ICvppv7fOqgGpXpRnHePGhmkp4NKGkUOc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=c95stfs5; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="c95stfs5" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=Cc:To:Subject:From:References:In-Reply-To:Message-Id:Date: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=eSOj7vOfSpGgQ5Kj0GYCHDDmdqiWsm4T+x7O3x8fxBs=; b=c95stfs5HmYKPE0pSQB1u/9+9m eNW8XoG3ZufiFA7486HXkW4q8y/NUBHBIHlLfx7JgpoMia3q17JBW0B8zduZl6+gqJwBSrbjk1JXr gtSlDfbKeWvMGaBR8n7C4AXW/vWGALoo1blWXp3//n25MgO4sO8gFqzdYrCH+Iet1+5hXEUX1jM3V v8yaRWJkBCuQXqAtY6D8w24/K758HJt5Lg14VaVLcQUNFFne2OuUcU3GMw3zd9Eiaem2pYtz1f79Z kpWVvYJv739ZaF/rvfdg98nhdMNB+N9kPZGpqOU5z4TPgFjm95b0m1jEAEkqPhz4+148LySIH+4s3 NYkRMvFw==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1tnav2-002DrQ-2q; Thu, 27 Feb 2025 18:15:05 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Thu, 27 Feb 2025 18:15:04 +0800 Date: Thu, 27 Feb 2025 18:15:04 +0800 Message-Id: <30f87ea9a3738c57287f9099a9e638781fa212c5.1740651138.git.herbert@gondor.apana.org.au> In-Reply-To: References: From: Herbert Xu Subject: [PATCH 5/7] crypto: scomp - Remove support for non-trivial SG lists To: Linux Crypto Mailing List Cc: linux-mm@kvack.org, Yosry Ahmed Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: As the only user of scomp uses a trivial single-page SG list, remove support for everything else in preprataion for a shift to virtual addresses. Signed-off-by: Herbert Xu --- crypto/acompress.c | 6 -- crypto/scompress.c | 139 +++++----------------------- include/crypto/acompress.h | 12 +-- include/crypto/internal/scompress.h | 2 - 4 files changed, 27 insertions(+), 132 deletions(-) diff --git a/crypto/acompress.c b/crypto/acompress.c index d2103d4e42cc..8914d0c4cc75 100644 --- a/crypto/acompress.c +++ b/crypto/acompress.c @@ -73,7 +73,6 @@ static int crypto_acomp_init_tfm(struct crypto_tfm *tfm) acomp->compress = alg->compress; acomp->decompress = alg->decompress; - acomp->dst_free = alg->dst_free; acomp->reqsize = alg->reqsize; if (alg->exit) @@ -146,11 +145,6 @@ void acomp_request_free(struct acomp_req *req) if (tfm->__crt_alg->cra_type != &crypto_acomp_type) crypto_acomp_scomp_free_ctx(req); - if (req->base.flags & CRYPTO_ACOMP_ALLOC_OUTPUT) { - acomp->dst_free(req->dst); - req->dst = NULL; - } - __acomp_request_free(req); } EXPORT_SYMBOL_GPL(acomp_request_free); diff --git a/crypto/scompress.c b/crypto/scompress.c index 1cef6bb06a81..ffc8e7f4862c 100644 --- a/crypto/scompress.c +++ b/crypto/scompress.c @@ -18,24 +18,11 @@ #include #include #include -#include #include #include "compress.h" -struct scomp_scratch { - spinlock_t lock; - void *src; - void *dst; -}; - -static DEFINE_PER_CPU(struct scomp_scratch, scomp_scratch) = { - .lock = __SPIN_LOCK_UNLOCKED(scomp_scratch.lock), -}; - static const struct crypto_type crypto_scomp_type; -static int scomp_scratch_users; -static DEFINE_MUTEX(scomp_lock); static int __maybe_unused crypto_scomp_report( struct sk_buff *skb, struct crypto_alg *alg) @@ -58,127 +45,57 @@ static void crypto_scomp_show(struct seq_file *m, struct crypto_alg *alg) seq_puts(m, "type : scomp\n"); } -static void crypto_scomp_free_scratches(void) -{ - struct scomp_scratch *scratch; - int i; - - for_each_possible_cpu(i) { - scratch = per_cpu_ptr(&scomp_scratch, i); - - vfree(scratch->src); - vfree(scratch->dst); - scratch->src = NULL; - scratch->dst = NULL; - } -} - -static int crypto_scomp_alloc_scratches(void) -{ - struct scomp_scratch *scratch; - int i; - - for_each_possible_cpu(i) { - void *mem; - - scratch = per_cpu_ptr(&scomp_scratch, i); - - mem = vmalloc_node(SCOMP_SCRATCH_SIZE, cpu_to_node(i)); - if (!mem) - goto error; - scratch->src = mem; - mem = vmalloc_node(SCOMP_SCRATCH_SIZE, cpu_to_node(i)); - if (!mem) - goto error; - scratch->dst = mem; - } - return 0; -error: - crypto_scomp_free_scratches(); - return -ENOMEM; -} - static int crypto_scomp_init_tfm(struct crypto_tfm *tfm) { - int ret = 0; - - mutex_lock(&scomp_lock); - if (!scomp_scratch_users++) - ret = crypto_scomp_alloc_scratches(); - mutex_unlock(&scomp_lock); - - return ret; + return 0; } static int scomp_acomp_comp_decomp(struct acomp_req *req, int dir) { struct crypto_acomp *tfm = crypto_acomp_reqtfm(req); - void **tfm_ctx = acomp_tfm_ctx(tfm); + struct crypto_scomp **tfm_ctx = acomp_tfm_ctx(tfm); struct crypto_scomp *scomp = *tfm_ctx; void **ctx = acomp_request_ctx(req); - struct scomp_scratch *scratch; + unsigned int slen = req->slen; + unsigned int dlen = req->dlen; void *src, *dst; - unsigned int dlen; int ret; - if (!req->src || !req->slen || req->slen > SCOMP_SCRATCH_SIZE) + if (!req->src || !slen) return -EINVAL; - if (req->dst && !req->dlen) + if (req->dst && !dlen) return -EINVAL; - if (!req->dlen || req->dlen > SCOMP_SCRATCH_SIZE) - req->dlen = SCOMP_SCRATCH_SIZE; + if (!req->dst || !dlen) + return -ENOSYS; - dlen = req->dlen; + if (sg_nents(req->src) > 1 || req->src->offset + slen > PAGE_SIZE) + return -ENOSYS; - scratch = raw_cpu_ptr(&scomp_scratch); - spin_lock(&scratch->lock); + if (sg_nents(req->dst) > 1) + return -ENOSYS; - if (sg_nents(req->src) == 1 && !PageHighMem(sg_page(req->src))) { - src = page_to_virt(sg_page(req->src)) + req->src->offset; - } else { - scatterwalk_map_and_copy(scratch->src, req->src, 0, - req->slen, 0); - src = scratch->src; - } + if (req->dst->offset >= PAGE_SIZE) + return -ENOSYS; - if (req->dst && sg_nents(req->dst) == 1 && !PageHighMem(sg_page(req->dst))) - dst = page_to_virt(sg_page(req->dst)) + req->dst->offset; - else - dst = scratch->dst; + if (req->dst->offset + dlen > PAGE_SIZE) + dlen = PAGE_SIZE - req->dst->offset; + + src = kmap_local_page(sg_page(req->src)) + req->src->offset; + dst = kmap_local_page(sg_page(req->dst)) + req->dst->offset; if (dir) - ret = crypto_scomp_compress(scomp, src, req->slen, + ret = crypto_scomp_compress(scomp, src, slen, dst, &req->dlen, *ctx); else - ret = crypto_scomp_decompress(scomp, src, req->slen, + ret = crypto_scomp_decompress(scomp, src, slen, dst, &req->dlen, *ctx); - if (!ret) { - if (!req->dst) { - req->dst = sgl_alloc(req->dlen, GFP_ATOMIC, NULL); - if (!req->dst) { - ret = -ENOMEM; - goto out; - } - } else if (req->dlen > dlen) { - ret = -ENOSPC; - goto out; - } - if (dst == scratch->dst) { - scatterwalk_map_and_copy(scratch->dst, req->dst, 0, - req->dlen, 1); - } else { - int nr_pages = DIV_ROUND_UP(req->dst->offset + req->dlen, PAGE_SIZE); - int i; - struct page *dst_page = sg_page(req->dst); - for (i = 0; i < nr_pages; i++) - flush_dcache_page(dst_page + i); - } - } -out: - spin_unlock(&scratch->lock); + kunmap_local(src); + kunmap_local(dst); + flush_dcache_page(sg_page(req->dst)); + return ret; } @@ -197,11 +114,6 @@ static void crypto_exit_scomp_ops_async(struct crypto_tfm *tfm) struct crypto_scomp **ctx = crypto_tfm_ctx(tfm); crypto_free_scomp(*ctx); - - mutex_lock(&scomp_lock); - if (!--scomp_scratch_users) - crypto_scomp_free_scratches(); - mutex_unlock(&scomp_lock); } int crypto_init_scomp_ops_async(struct crypto_tfm *tfm) @@ -225,7 +137,6 @@ int crypto_init_scomp_ops_async(struct crypto_tfm *tfm) crt->compress = scomp_acomp_compress; crt->decompress = scomp_acomp_decompress; - crt->dst_free = sgl_free; crt->reqsize = sizeof(void *); return 0; diff --git a/include/crypto/acompress.h b/include/crypto/acompress.h index 15bb13e47f8b..25e193b0b8b4 100644 --- a/include/crypto/acompress.h +++ b/include/crypto/acompress.h @@ -15,8 +15,6 @@ #include #include -#define CRYPTO_ACOMP_ALLOC_OUTPUT 0x00000001 - /* Set this bit for virtual address instead of SG list. */ #define CRYPTO_ACOMP_REQ_VIRT 0x00000002 @@ -75,15 +73,12 @@ struct acomp_req { * * @compress: Function performs a compress operation * @decompress: Function performs a de-compress operation - * @dst_free: Frees destination buffer if allocated inside the - * algorithm * @reqsize: Context size for (de)compression requests * @base: Common crypto API algorithm data structure */ struct crypto_acomp { int (*compress)(struct acomp_req *req); int (*decompress)(struct acomp_req *req); - void (*dst_free)(struct scatterlist *dst); unsigned int reqsize; struct crypto_tfm base; }; @@ -234,7 +229,7 @@ static inline void acomp_request_set_callback(struct acomp_req *req, crypto_completion_t cmpl, void *data) { - u32 keep = CRYPTO_ACOMP_ALLOC_OUTPUT | CRYPTO_ACOMP_REQ_VIRT; + u32 keep = CRYPTO_ACOMP_REQ_VIRT; req->base.complete = cmpl; req->base.data = data; @@ -268,9 +263,7 @@ static inline void acomp_request_set_params(struct acomp_req *req, req->slen = slen; req->dlen = dlen; - req->base.flags &= ~(CRYPTO_ACOMP_ALLOC_OUTPUT | CRYPTO_ACOMP_REQ_VIRT); - if (!req->dst) - req->base.flags |= CRYPTO_ACOMP_ALLOC_OUTPUT; + req->base.flags &= ~CRYPTO_ACOMP_REQ_VIRT; } /** @@ -294,7 +287,6 @@ static inline void acomp_request_set_virt(struct acomp_req *req, req->slen = slen; req->dlen = dlen; - req->base.flags &= ~CRYPTO_ACOMP_ALLOC_OUTPUT; req->base.flags |= CRYPTO_ACOMP_REQ_VIRT; } diff --git a/include/crypto/internal/scompress.h b/include/crypto/internal/scompress.h index 07a10fd2d321..25c5f8126618 100644 --- a/include/crypto/internal/scompress.h +++ b/include/crypto/internal/scompress.h @@ -12,8 +12,6 @@ #include #include -#define SCOMP_SCRATCH_SIZE 131072 - struct acomp_req; struct crypto_scomp { From patchwork Thu Feb 27 10:15:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 869075 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A489E1E1E01 for ; Thu, 27 Feb 2025 10:15:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740651315; cv=none; b=Y3br+hoeNYPG+T9KWjfFaiLsj7GNndLOKsZHLrqPC7zeqVrSPyzC6RqjgJ/3IB5Q7Rn+31SqjQJiDfQkW7YuGR62V2zQDB6cvpJpA5Mrs2k8yyvm3VXdgM4NLr+V7OLQuSu8h6bK3h51EQ/wmUgeS2hPcwz68ZuKTiK55MhVrpk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740651315; c=relaxed/simple; bh=3Hi/AbFrNA5O6i2dQMMc/ck14Tw+KZLqX3ywTJA6zk4=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To:Cc; b=sV8eW7FOS9izero3VDUESa1cAEfqXbwwmS5WKRt4DxVp1lrFvExQTA00N7i5pX4NL0+XHrNQtTZUgE2EgWXy7xaEWepP/OWeDaMMsovHyZa27EtUGZW2VpGmeDF2lHLqQBAtSbNhpGAioeM+Xjinpi8wJpw2AJfk4cUVDUZoYbI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=Z7sMw0fg; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="Z7sMw0fg" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=Cc:To:Subject:From:References:In-Reply-To:Message-Id:Date: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=X/COYPSBnxqacFq0opJtS5fN0VNbCBV4FqPa/0g+uW0=; b=Z7sMw0fg2Or5wgGuT/y40ULroe f60K2ja10VqUBJI2JgbgCgl+Q9xr7sYFCxC6GRDXcgjsqIgW8fvzIFHAIDsGhxLsj6hYJ/R1fLgrD hJkSqj2zDvxgZqs6c9Zw5Y8a2S0pEzMpy3+vOUu7mWHUx0fyadmf27RPg2x/XnFrb5twOq9CYPLxO l2Uoh2nG93hM2JulS2hbFYw9dFxJf3o4WDAxsThLOV7PHAuFplNHm01Ci06fgc3OrEKpIBj+9U4xH EWAg9SZ7cYAAustIzKjGxTXvUgITXSjt0ryklqVuh/wnk784rD/xJeEWM+NVdWswOK4oosMNgF8UU LocAv05A==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1tnav7-002Drs-1X; Thu, 27 Feb 2025 18:15:10 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Thu, 27 Feb 2025 18:15:09 +0800 Date: Thu, 27 Feb 2025 18:15:09 +0800 Message-Id: <153c340a52090f2ff82f8f066203186a932d3f99.1740651138.git.herbert@gondor.apana.org.au> In-Reply-To: References: From: Herbert Xu Subject: [RFC PATCH 7/7] mm: zswap: Use acomp virtual address interface To: Linux Crypto Mailing List Cc: linux-mm@kvack.org, Yosry Ahmed Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Use the acomp virtual address interface. Signed-off-by: Herbert Xu --- mm/zswap.c | 23 ++++++++--------------- 1 file changed, 8 insertions(+), 15 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 6504174fbc6a..2b5a2398a9be 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -925,27 +925,20 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry, struct zswap_pool *pool) { struct crypto_acomp_ctx *acomp_ctx; - struct scatterlist input, output; int comp_ret = 0, alloc_ret = 0; unsigned int dlen = PAGE_SIZE; unsigned long handle; struct zpool *zpool; + const u8 *src; char *buf; gfp_t gfp; u8 *dst; acomp_ctx = acomp_ctx_get_cpu_lock(pool); + src = kmap_local_page(page); dst = acomp_ctx->buffer; - sg_init_table(&input, 1); - sg_set_page(&input, page, PAGE_SIZE, 0); - /* - * We need PAGE_SIZE * 2 here since there maybe over-compression case, - * and hardware-accelerators may won't check the dst buffer size, so - * giving the dst buffer with enough length to avoid buffer overflow. - */ - sg_init_one(&output, dst, PAGE_SIZE * 2); - acomp_request_set_params(acomp_ctx->req, &input, &output, PAGE_SIZE, dlen); + acomp_request_set_virt(acomp_ctx->req, src, dst, PAGE_SIZE, dlen); /* * it maybe looks a little bit silly that we send an asynchronous request, @@ -960,6 +953,7 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry, * acomp instance, so multiple threads can do (de)compression in parallel. */ comp_ret = crypto_wait_req(crypto_acomp_compress(acomp_ctx->req), &acomp_ctx->wait); + kunmap_local(src); dlen = acomp_ctx->req->dlen; if (comp_ret) goto unlock; @@ -994,9 +988,9 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry, static void zswap_decompress(struct zswap_entry *entry, struct folio *folio) { struct zpool *zpool = entry->pool->zpool; - struct scatterlist input, output; struct crypto_acomp_ctx *acomp_ctx; u8 *src; + u8 *dst; acomp_ctx = acomp_ctx_get_cpu_lock(entry->pool); src = zpool_map_handle(zpool, entry->handle, ZPOOL_MM_RO); @@ -1016,11 +1010,10 @@ static void zswap_decompress(struct zswap_entry *entry, struct folio *folio) zpool_unmap_handle(zpool, entry->handle); } - sg_init_one(&input, src, entry->length); - sg_init_table(&output, 1); - sg_set_folio(&output, folio, PAGE_SIZE, 0); - acomp_request_set_params(acomp_ctx->req, &input, &output, entry->length, PAGE_SIZE); + dst = kmap_local_folio(folio, 0); + acomp_request_set_virt(acomp_ctx->req, src, dst, entry->length, PAGE_SIZE); BUG_ON(crypto_wait_req(crypto_acomp_decompress(acomp_ctx->req), &acomp_ctx->wait)); + kunmap_local(dst); BUG_ON(acomp_ctx->req->dlen != PAGE_SIZE); if (src != acomp_ctx->buffer)