From patchwork Wed Dec 6 10:02:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 750986 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="OoT/jEVE" Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7384AD4D for ; Wed, 6 Dec 2023 02:10:55 -0800 (PST) Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20231206101053epoutp02f110db6b03bdfc6be7b7e31377e0322a~eNlZMayug2240722407epoutp02B for ; Wed, 6 Dec 2023 10:10:53 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20231206101053epoutp02f110db6b03bdfc6be7b7e31377e0322a~eNlZMayug2240722407epoutp02B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857453; bh=Ht/Do8cTuJq3u2yJt3wmG4hHCRAbWvK0/GxdjTa0bnE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OoT/jEVES07IzsqBprqRYW4rnbTfG+Rec80k5makyHcuCWU/kNE+Lswx5CZ5W6S8p iEg3LitIwj6qXGbDl+RHyzzy8sXDvRsQjUS9KAj/62faUujzAeziGrbP/GKHsUyJH6 /RIMzV1VgjMjJHEcyV1ubELKTxsdgj7+yoepFCUE= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20231206101052epcas5p1f74433f38c2c0a95c3448556d538f621~eNlYlyY1U0449604496epcas5p1D; Wed, 6 Dec 2023 10:10:52 +0000 (GMT) Received: from epsmgec5p1-new.samsung.com (unknown [182.195.38.179]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4SlY7v0xjCz4x9Px; Wed, 6 Dec 2023 10:10:51 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmgec5p1-new.samsung.com (Symantec Messaging Gateway) with SMTP id 2D.58.19369.AA840756; Wed, 6 Dec 2023 19:10:50 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20231206101050epcas5p2c8233030bbf74cef0166c7dfc0f41be7~eNlWbwK8W3161231612epcas5p2p; Wed, 6 Dec 2023 10:10:50 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20231206101050epsmtrp11528effcadae6126b68eea06fd58e0ac~eNlWagLGu1050310503epsmtrp1J; Wed, 6 Dec 2023 10:10:50 +0000 (GMT) X-AuditID: b6c32a50-c99ff70000004ba9-26-657048aacece Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id B8.7B.08755.AA840756; Wed, 6 Dec 2023 19:10:50 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101046epsmtip29bbd9da58b30c5b273ec0cbf08b87c0b~eNlSljNQa1163811638epsmtip2B; Wed, 6 Dec 2023 10:10:46 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 02/12] Add infrastructure for copy offload in block and request layer. Date: Wed, 6 Dec 2023 15:32:34 +0530 Message-Id: <20231206100253.13100-3-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02Te1BUVRzH59x7ubsQ0G0xPG4ksEGGBu7Gw4PxDEbvRM3g1PQgGNxZ7iwb y+6yDytjRmJzA+TlMhm7xCwaSkCJvB+CQ7shaSUoIoIjjbooBOSAihIi7bJb+d/n9zvf3/md 7+/Mj41zdCwuWyJTM0qZUMoj3YgOS9CW4AZawfCXjuxETefP4mjhwQqB8stXcdR4vYxEs5ZF gKz9XwFUc6yaQOP93RiqbxzAkN58BaCpUSOG+ia2oaO6WgL19p0j0EjPtyQynZhioUNjXSSq G3yCoavlUwAdLhjFUJf1C4A6Vkw4Ojl7l0C/TLyAhlYHXeI20d3G6yx6aLKZoEd+19AtDYUk 3Vp7gJ5uNQD69HgeSX9XWuFCl2j/IumFqQmCvntmlKRL2xoAfa9lM91inceSPVOyojIZYQaj 9GNkInmGRCaO5iW9k56QHh7BFwQLItEOnp9MmM1E8xLfSg7eJZHaJsHz2yeUamypZKFKxdse E6WUa9SMX6ZcpY7mMYoMqSJMEaISZqs0MnGIjFHvFPD5r4XbhHuzMn9urWIpmqI+XVi7gOWB ZX4RcGVDKgyWVlZjduZQvQBqa1OKgJuNFwH82zRCOIIlADtOzNgC9nrF7MABR74PwMmvzzhF 9wDU/9ZG2kUkFQSHKzT2WzdQP+Cwu1lg1+DUYwzq75hd7AdeVCr89VAry84EFQh/HKxfZw8K QeOVdtLxPF9ouPRwPe9KRcKKO8dwh+Y5eM5gJeyM2zTa9irc3gBSja5wrqsYdxQnQkttHcvB XvDPwTYnc+FMmc7JInjJcAFzsBre6v3JybHw4Pky3G4Gt5lp6tnu6OUJS1asmGMQHrBAx3Go /eGkfsrFwRvhjcpaJ9OwpvAb53yKAXxgLQblwNf4lAXjUxaM/3erAXgD4DIKVbaYEYUrBMEy 5pP/PlYkz24B6wuxNbkLNJ5aDTEDjA3MALJx3gYP6ZCc4XhkCD/bzyjl6UqNlFGZQbhtyIdx 7vMiuW2jZOp0QVgkPywiIiIsMjRCwNvoMXuwOoNDiYVqJothFIzy3zqM7crNw+L3sjgvJ/3R 7aZ/RuSXcnJpaTihxV807XVR2jZjmE+t21OQX2JejX+zWrrWzV3stbzvHeBTyNwam/g8f9l9 AeceFy27C1DU7eKj2Tr/7989ez/OLY3X81D70fzwq6H6t3dcCw36YKDoSXxOJdYem7w6/ciU 2wbTJK8wJo0k16fZNw16zpkSxQEmoWugf+IjdYK65n7h5qX63hwdy+z+xuVpZtP+L8cvx5zy h7vXHse+tIttCfw4r31hUTsU0/nhs3FjRZKh/NSkG4bjaKIz8aI2P1ff7G1pfy+2R3xk99w+ Ive2+5bXq19MuNrp7XOz6vS2hj09AdjAzfQc32v9fQFRfTxClSkUbMWVKuE/TyWjmJkEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrFIsWRmVeSWpSXmKPExsWy7bCSvO4qj4JUg8uXRCzWnzrGbPHx628W i6YJf5ktVt/tZ7N4ffgTo8WTA+2MFgsWzWWxuHlgJ5PFytVHmSwmHbrGaPH06iwmi723tC0W ti1hsdiz9ySLxeVdc9gs5i97ym7RfX0Hm8Xy4/+YLG5MeMpoMbHjKpPFjieNjBbbfs9ntlj3 +j2LxYlb0hbn/x5ndZD02DnrLrvH+XsbWTwuny312LSqk81j85J6jxebZzJ67L7ZwOaxuG8y q0dv8zs2j49Pb7F4vN93lc2jb8sqRo/Pm+Q8Nj15yxTAF8Vlk5Kak1mWWqRvl8CVcWTzbPaC 9TYVH/+fY2pg/GnQxcjBISFgIvH6aH0XIxeHkMBuRokrD7azdTFyAsXFJZqv/WCHsIUlVv57 zg5R9JFRYuWSZnaQZjYBTYkLk0tB4iICO5glfq5tZgJxmAW6mCUm/f8L1i0sECXxoXUlI4jN IqAqsfb4SrA4r4CFxKxrW6G2yUvMvPQdLM4pYCkx+fkiZhBbCKhmX+N0Foh6QYmTM5+A2cxA 9c1bZzNPYBSYhSQ1C0lqASPTKkbJ1ILi3PTcYsMCw7zUcr3ixNzi0rx0veT83E2M4OjW0tzB uH3VB71DjEwcjIcYJTiYlUR4c87npwrxpiRWVqUW5ccXleakFh9ilOZgURLnFX/RmyIkkJ5Y kpqdmlqQWgSTZeLglGpgWnBb5h7rruOn1N+e+FfScvDaq1SrOqG8A1OlPwbtS10/l7FEgmOK xROnGZ8EJdPP3eOedtuLV+CpcumDOZ0ze+YmZig+kmHLT3ibJrsy4myZ5/Ndrod1Hp+Pbpz0 5/EToyv9G/oEbsX0i1YXhZ3T//ny463jX3Lnbnq3ZVVf3vHlZ39qeYh++jaJ7WLz7Lse+osN zBUDVsfI8pfYPRdrTzZgcExqSonRZGp8nu/btL6ib//RsBAZ8ZDJEXP2p01aemEWTyGTopUd 11aGKY2bLhi8YGZV2vpTjTP0Zlrm04/TZ3PNPmDRv39rRMCrlNl5YUuX7PaPm//HmJUr+dCv TuuTXBarzmw+G8Odv2/rRCWW4oxEQy3mouJEAHH4sytdAwAA X-CMS-MailID: 20231206101050epcas5p2c8233030bbf74cef0166c7dfc0f41be7 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101050epcas5p2c8233030bbf74cef0166c7dfc0f41be7 References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty We add two new opcode REQ_OP_COPY_SRC, REQ_OP_COPY_DST. Since copy is a composite operation involving src and dst sectors/lba, each needs to be represented by a separate bio to make it compatible with device mapper. We expect caller to take a plug and send bio with source information, followed by bio with destination information. Once the src bio arrives we form a request and wait for destination bio. Upon arrival of destination we merge these two bio's and send corresponding request down to device driver. Merging non copy offload bio is avoided by checking for copy specific opcodes in merge function. Signed-off-by: Nitesh Shetty Signed-off-by: Anuj Gupta --- block/blk-core.c | 7 +++++++ block/blk-merge.c | 41 +++++++++++++++++++++++++++++++++++++++ block/blk.h | 16 +++++++++++++++ block/elevator.h | 1 + include/linux/bio.h | 6 +----- include/linux/blk_types.h | 10 ++++++++++ 6 files changed, 76 insertions(+), 5 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 2eca76ccf4ee..51c6cc3022f4 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -121,6 +121,8 @@ static const char *const blk_op_name[] = { REQ_OP_NAME(ZONE_FINISH), REQ_OP_NAME(ZONE_APPEND), REQ_OP_NAME(WRITE_ZEROES), + REQ_OP_NAME(COPY_SRC), + REQ_OP_NAME(COPY_DST), REQ_OP_NAME(DRV_IN), REQ_OP_NAME(DRV_OUT), }; @@ -800,6 +802,11 @@ void submit_bio_noacct(struct bio *bio) if (!q->limits.max_write_zeroes_sectors) goto not_supported; break; + case REQ_OP_COPY_SRC: + case REQ_OP_COPY_DST: + if (!q->limits.max_copy_sectors) + goto not_supported; + break; default: break; } diff --git a/block/blk-merge.c b/block/blk-merge.c index 65e75efa9bd3..bcb55ba48107 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -158,6 +158,20 @@ static struct bio *bio_split_write_zeroes(struct bio *bio, return bio_split(bio, lim->max_write_zeroes_sectors, GFP_NOIO, bs); } +static struct bio *bio_split_copy(struct bio *bio, + const struct queue_limits *lim, + unsigned int *nsegs) +{ + *nsegs = 1; + if (bio_sectors(bio) <= lim->max_copy_sectors) + return NULL; + /* + * We don't support splitting for a copy bio. End it with EIO if + * splitting is required and return an error pointer. + */ + return ERR_PTR(-EIO); +} + /* * Return the maximum number of sectors from the start of a bio that may be * submitted as a single request to a block device. If enough sectors remain, @@ -366,6 +380,12 @@ struct bio *__bio_split_to_limits(struct bio *bio, case REQ_OP_WRITE_ZEROES: split = bio_split_write_zeroes(bio, lim, nr_segs, bs); break; + case REQ_OP_COPY_SRC: + case REQ_OP_COPY_DST: + split = bio_split_copy(bio, lim, nr_segs); + if (IS_ERR(split)) + return NULL; + break; default: split = bio_split_rw(bio, lim, nr_segs, bs, get_max_io_size(bio, lim) << SECTOR_SHIFT); @@ -922,6 +942,9 @@ bool blk_rq_merge_ok(struct request *rq, struct bio *bio) if (!rq_mergeable(rq) || !bio_mergeable(bio)) return false; + if (blk_copy_offload_mergable(rq, bio)) + return true; + if (req_op(rq) != bio_op(bio)) return false; @@ -951,6 +974,8 @@ enum elv_merge blk_try_merge(struct request *rq, struct bio *bio) { if (blk_discard_mergable(rq)) return ELEVATOR_DISCARD_MERGE; + else if (blk_copy_offload_mergable(rq, bio)) + return ELEVATOR_COPY_OFFLOAD_MERGE; else if (blk_rq_pos(rq) + blk_rq_sectors(rq) == bio->bi_iter.bi_sector) return ELEVATOR_BACK_MERGE; else if (blk_rq_pos(rq) - bio_sectors(bio) == bio->bi_iter.bi_sector) @@ -1053,6 +1078,20 @@ static enum bio_merge_status bio_attempt_discard_merge(struct request_queue *q, return BIO_MERGE_FAILED; } +static enum bio_merge_status bio_attempt_copy_offload_merge(struct request *req, + struct bio *bio) +{ + if (req->__data_len != bio->bi_iter.bi_size) + return BIO_MERGE_FAILED; + + req->biotail->bi_next = bio; + req->biotail = bio; + req->nr_phys_segments++; + req->__data_len += bio->bi_iter.bi_size; + + return BIO_MERGE_OK; +} + static enum bio_merge_status blk_attempt_bio_merge(struct request_queue *q, struct request *rq, struct bio *bio, @@ -1073,6 +1112,8 @@ static enum bio_merge_status blk_attempt_bio_merge(struct request_queue *q, break; case ELEVATOR_DISCARD_MERGE: return bio_attempt_discard_merge(q, rq, bio); + case ELEVATOR_COPY_OFFLOAD_MERGE: + return bio_attempt_copy_offload_merge(rq, bio); default: return BIO_MERGE_NONE; } diff --git a/block/blk.h b/block/blk.h index 08a358bc0919..b0c17ad635a5 100644 --- a/block/blk.h +++ b/block/blk.h @@ -159,6 +159,20 @@ static inline bool blk_discard_mergable(struct request *req) return false; } +/* + * Copy offload sends a pair of bio with REQ_OP_COPY_SRC and REQ_OP_COPY_DST + * operation by taking a plug. + * Initially SRC bio is sent which forms a request and + * waits for DST bio to arrive. Once DST bio arrives + * we merge it and send request down to driver. + */ +static inline bool blk_copy_offload_mergable(struct request *req, + struct bio *bio) +{ + return (req_op(req) == REQ_OP_COPY_SRC && + bio_op(bio) == REQ_OP_COPY_DST); +} + static inline unsigned int blk_rq_get_max_segments(struct request *rq) { if (req_op(rq) == REQ_OP_DISCARD) @@ -300,6 +314,8 @@ static inline bool bio_may_exceed_limits(struct bio *bio, case REQ_OP_DISCARD: case REQ_OP_SECURE_ERASE: case REQ_OP_WRITE_ZEROES: + case REQ_OP_COPY_SRC: + case REQ_OP_COPY_DST: return true; /* non-trivial splitting decisions */ default: break; diff --git a/block/elevator.h b/block/elevator.h index 7ca3d7b6ed82..eec442bbf384 100644 --- a/block/elevator.h +++ b/block/elevator.h @@ -18,6 +18,7 @@ enum elv_merge { ELEVATOR_FRONT_MERGE = 1, ELEVATOR_BACK_MERGE = 2, ELEVATOR_DISCARD_MERGE = 3, + ELEVATOR_COPY_OFFLOAD_MERGE = 4, }; struct blk_mq_alloc_data; diff --git a/include/linux/bio.h b/include/linux/bio.h index ec4db73e5f4e..5816e41588af 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -53,11 +53,7 @@ static inline unsigned int bio_max_segs(unsigned int nr_segs) */ static inline bool bio_has_data(struct bio *bio) { - if (bio && - bio->bi_iter.bi_size && - bio_op(bio) != REQ_OP_DISCARD && - bio_op(bio) != REQ_OP_SECURE_ERASE && - bio_op(bio) != REQ_OP_WRITE_ZEROES) + if (bio && (bio_op(bio) == REQ_OP_READ || bio_op(bio) == REQ_OP_WRITE)) return true; return false; diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 7c2316c91cbd..bd821eaa7a02 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -393,6 +393,10 @@ enum req_op { /* reset all the zone present on the device */ REQ_OP_ZONE_RESET_ALL = (__force blk_opf_t)17, + /* copy offload dst and src operation */ + REQ_OP_COPY_SRC = (__force blk_opf_t)19, + REQ_OP_COPY_DST = (__force blk_opf_t)21, + /* Driver private requests */ REQ_OP_DRV_IN = (__force blk_opf_t)34, REQ_OP_DRV_OUT = (__force blk_opf_t)35, @@ -481,6 +485,12 @@ static inline bool op_is_write(blk_opf_t op) return !!(op & (__force blk_opf_t)1); } +static inline bool op_is_copy(blk_opf_t op) +{ + return ((op & REQ_OP_MASK) == REQ_OP_COPY_SRC || + (op & REQ_OP_MASK) == REQ_OP_COPY_DST); +} + /* * Check if the bio or request is one that needs special treatment in the * flush state machine. From patchwork Wed Dec 6 10:02:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 750985 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="Zvsgxf9B" Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1D4AD6D for ; Wed, 6 Dec 2023 02:11:22 -0800 (PST) Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20231206101121epoutp02ea4ee37802ddb1975eaacc7dc4f19c8d~eNlzDRe9t2244322443epoutp02L for ; Wed, 6 Dec 2023 10:11:21 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20231206101121epoutp02ea4ee37802ddb1975eaacc7dc4f19c8d~eNlzDRe9t2244322443epoutp02L DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857481; bh=/6OhQ9ODX//umK/uU/u7BcJDEaHDVpwjz5uay8Kefew=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Zvsgxf9BFlB3by/cG0LjnxSw9PuIrmn+Ve6wLU+4nkYVKhCVyUYAV49SvJFxqXE2a FhUTk0u0yqd621sjpzzUXVqKKMTaaU+632OV7znew28DJLjcrqtZoIXt5ltX0G9DcT 1kamBpJgHgrhwn7TugpqnIZ17VLgwnjtYLYWMStw= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20231206101120epcas5p2f9ffcb1fb083bd23d3b4fb98147d54d9~eNlyWpE5K2940529405epcas5p27; Wed, 6 Dec 2023 10:11:20 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.174]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4SlY8Q5kFBz4x9Pv; Wed, 6 Dec 2023 10:11:18 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id F5.C6.10009.6C840756; Wed, 6 Dec 2023 19:11:18 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20231206101118epcas5p1cc77b49dbd8bc1601423d02527b03122~eNlwWIZO30449604496epcas5p1v; Wed, 6 Dec 2023 10:11:18 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20231206101118epsmtrp2882cd222b4c8fcca5aaafc6113b65840~eNlwVBeI-0924709247epsmtrp2d; Wed, 6 Dec 2023 10:11:18 +0000 (GMT) X-AuditID: b6c32a4a-261fd70000002719-2d-657048c6a55e Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id B1.52.08817.6C840756; Wed, 6 Dec 2023 19:11:18 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101113epsmtip278f834a99ee8a24a8447ab449801a2ff~eNlrq_aCz3173831738epsmtip2Z; Wed, 6 Dec 2023 10:11:13 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Vincent Fu , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 04/12] block: add emulation for copy Date: Wed, 6 Dec 2023 15:32:36 +0530 Message-Id: <20231206100253.13100-5-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01TbUxTVxjeufdyWyB1bdVxVhOBOtwE+ShruwMDNGLcjWMZC/90k93QO0BK 2/UDYdkyKoJYUPmQAVU+hM4PMDAKQ0C6ACqgC2AkdYMENqSdczAU2TBaYGtp3fz3vM/7PHnf 5z05bJxfzBKw0xVaRq2g5ULSh+i8vuOt0EFKxUQsVryJWm8P4mjxbweBjpas4qh56jSJ5q4/ AcjWdxyg+oYaAk30dWPocvNNDJUN3APIbjViyDIZgs4XmAjUa7lFoPGecySqu2BnoaKfukh0 cWgNQz+X2AEqLbRiqMumB6jTUYejlrlHBBqe3IJmiwoBGlsd8totoLqNUyxqbLqNoMZHdJS5 6QRJtZu+pn5vrwbUtYlckmo8Ve5FncxbIKlF+yRBPfrBSlKnOpoAtWTeSpltf2KJGw5kxKQx tIxRBzCKFKUsXZEaK3w/KTk+WSKNEIWKotA7wgAFncnECvcmJIbuS5c7zyEMyKLlOieVSGs0 wvC4GLVSp2UC0pQabayQUcnkKrEqTENnanSK1DAFo40WRURESpzCTzPS6hqrMdXanmxL/n1W LjBKDcCbDbliOL/8i5cB+LD53GsALhvPeIonABZYbbi7WAYw76aeeGEpMOtJd8MCYGmlwWNZ ArCrfcRpYbNJ7g54p1znMmziXsFhd5vIpcG5jTi0TNeRrsZGLoJPHzSsY4IbBEcfm4ELc5z8 1IkVzD3NH1bffcpyYW9uFCx/0IC7NTx4q9q2vhHu1OR9f3Z9Vcht9oa1fVbcbd4Ln/e3km68 Ef4x1MFyYwFcWrB4+BR4t3rUM0wLZ3v7PXgXzL99ej0M7gzT2hPunrUBnnTYMBcNuRxYWMB3 qwPhdJndy4394EyVyYMpWLFQ6bliMYDHzzjIEuBvfCmC8aUIxv+n1QO8CbzOqDSZqYxGoopU MEf+e9kUZaYZrH+L4P1dYObXx2EDAGODAQDZuHATRz6mZPgcGZ3zBaNWJqt1ckYzACTOG5fi gs0pSue/UmiTReKoCLFUKhVHvS0VCf04c/k1Mj43ldYyGQyjYtQvfBjbW5CLHY7P9x+TJSx8 80zTtnKBlEhHR2iJrmN5WF+/1h7MYz7x0dtO768N510y7Ml6pb9eEDqX/bmpNyZ2vr32K8eq 32vfPjRE51Z999lC/BX7fBnZ+WGsMr21SPXxoUFfwrRdygvPqpq+kR1/x5EjzNiWdMN3URx9 eXem0bYlsOX+q/T0wbOVB3lf6g/HV/GPgn1bI7OuXqo4BoNUVQdsjp6HV01HmH92jdeY69Y4 YY0zm5fj5tFHgiC6z3+4x94zqh191i+dNIRd3B4Xfqhv24xS917WwM4JVsK5D3LLBn8MTmxr M4WcX13xKy66R/NUwbOBLbbfQnz7n79h/QvtfDfmWFKOkNCk0aJgXK2h/wVQ6RsynwQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA02Ra0hTYRjHe885nh1H2mmWvnnJGhSpNZtEvF3JCDsFUlFGFJRLD2ptc+xo N4000VLzHtROhdes1tVlMq/pLM2oVt4qaRW5sUwst252meUcQd9+PL//8/w/PBQuMhK+VIIy iVUrZXIxKSTq2sWBizoYFbvYZpmKbj7swJHt6y8CHS904OiqqYBEw+12gMytJwAqq7hAoJet 9Ri6cvU+hooN/QBZ+ngMNQ+EoPKsKgI1NXcRqKfhPIlKqy0ClPtcT6JLneMYelFoAajoZB+G 9OZ0gOp+leLoxvAnAj0Y8EODuScBMjo63db4MvW8ScAYX9cQTM/jZEanzSaZ21XHmPe3NYBp fJlGMpX5JW5MXsZHkrFZBgjmU0sfyeTXagHzWTeb0ZlHsM2eO4UrY1l5wgFWHbo6WhhfWqnB VONrDzVnvhOkAX5pDnCnIL0EZunSyRwgpER0I4D8rRuES/jAjP4xgYu94JVxq8AVsgE4ducD ngMoiqSD4NOSZOd8Bq3H4Y/rGZhzAadrcNjdMdngRSP43VpBOpmg58EnozrgZI+JuSn7N+Yq CISa7u+TZe70MlhircCdLJrItKSfIVz56bBLYyZc9wNhxp1zeCGg+f8U/58qA5gWzGJVnCJO wUlVYUr2oISTKbhkZZwkJlGhA5M/Dw7WgybtqMQAMAoYAKRw8QwPuTGRFXnEyg4fYdWJe9TJ cpYzAD+KEPt4fBvOixXRcbIkdj/Lqlj1P4tR7r5pmK+3/4lDl8FyxR/vftHCqI2nPmyTvA2r Dh9ZtY4vmGKd/7AoZWSBnbPSx66fXv5z9OC13AsJbdPW1wfE7mjcG3W47Uv7G+ub2tQAPmlm zM8QB2OXrM8acjQmxUVuN68TzpVL+reeDY2I8equHHmWPX50S4GxIf5Mb2ZDud27q/yden6R dMXrojFoEM/pexXR+nhtVdfgzUf+X03f2jR8AdguLUv1CQ5x643Y5Zmiq0HS81HVOdy2WcYN vSWiVLtJHxSWN9QTXdEb6B8Ejea9LcW7Pod3rrzorcxuytwXWuu+ZIi+Fxm+W5tG2Ui30Rr4 xdOUUnx36uLaOsfg4KbIu9FigouXSYNxNSf7C2PAyuliAwAA X-CMS-MailID: 20231206101118epcas5p1cc77b49dbd8bc1601423d02527b03122 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101118epcas5p1cc77b49dbd8bc1601423d02527b03122 References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty For the devices which does not support copy, copy emulation is added. It is required for in-kernel users like fabrics, where file descriptor is not available and hence they can't use copy_file_range. Copy-emulation is implemented by reading from source into memory and writing to the corresponding destination. At present in kernel user of emulation is fabrics. Signed-off-by: Nitesh Shetty Signed-off-by: Vincent Fu Signed-off-by: Anuj Gupta --- block/blk-lib.c | 223 +++++++++++++++++++++++++++++++++++++++++ include/linux/blkdev.h | 4 + 2 files changed, 227 insertions(+) diff --git a/block/blk-lib.c b/block/blk-lib.c index 05dbe7fa5354..e32e00e4a1eb 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -26,6 +26,20 @@ struct blkdev_copy_offload_io { loff_t offset; }; +/* Keeps track of single outstanding copy emulation IO */ +struct blkdev_copy_emulation_io { + struct blkdev_copy_io *cio; + struct work_struct emulation_work; + void *buf; + ssize_t buf_len; + loff_t pos_in; + loff_t pos_out; + ssize_t len; + struct block_device *bdev_in; + struct block_device *bdev_out; + gfp_t gfp; +}; + static sector_t bio_discard_limit(struct block_device *bdev, sector_t sector) { unsigned int discard_granularity = bdev_discard_granularity(bdev); @@ -319,6 +333,215 @@ ssize_t blkdev_copy_offload(struct block_device *bdev, loff_t pos_in, } EXPORT_SYMBOL_GPL(blkdev_copy_offload); +static void *blkdev_copy_alloc_buf(ssize_t req_size, ssize_t *alloc_size, + gfp_t gfp) +{ + int min_size = PAGE_SIZE; + char *buf; + + while (req_size >= min_size) { + buf = kvmalloc(req_size, gfp); + if (buf) { + *alloc_size = req_size; + return buf; + } + req_size >>= 1; + } + + return NULL; +} + +static struct bio *bio_map_buf(void *data, unsigned int len, gfp_t gfp) +{ + unsigned long kaddr = (unsigned long)data; + unsigned long end = (kaddr + len + PAGE_SIZE - 1) >> PAGE_SHIFT; + unsigned long start = kaddr >> PAGE_SHIFT; + const int nr_pages = end - start; + bool is_vmalloc = is_vmalloc_addr(data); + struct page *page; + int offset, i; + struct bio *bio; + + bio = bio_kmalloc(nr_pages, gfp); + if (!bio) + return ERR_PTR(-ENOMEM); + bio_init(bio, NULL, bio->bi_inline_vecs, nr_pages, 0); + + if (is_vmalloc) { + flush_kernel_vmap_range(data, len); + bio->bi_private = data; + } + + offset = offset_in_page(kaddr); + for (i = 0; i < nr_pages; i++) { + unsigned int bytes = PAGE_SIZE - offset; + + if (len <= 0) + break; + + if (bytes > len) + bytes = len; + + if (!is_vmalloc) + page = virt_to_page(data); + else + page = vmalloc_to_page(data); + if (bio_add_page(bio, page, bytes, offset) < bytes) { + /* we don't support partial mappings */ + bio_uninit(bio); + kfree(bio); + return ERR_PTR(-EINVAL); + } + + data += bytes; + len -= bytes; + offset = 0; + } + + return bio; +} + +static void blkdev_copy_emulation_work(struct work_struct *work) +{ + struct blkdev_copy_emulation_io *emulation_io = container_of(work, + struct blkdev_copy_emulation_io, emulation_work); + struct blkdev_copy_io *cio = emulation_io->cio; + struct bio *read_bio, *write_bio; + loff_t pos_in = emulation_io->pos_in, pos_out = emulation_io->pos_out; + ssize_t rem, chunk; + int ret = 0; + + for (rem = emulation_io->len; rem > 0; rem -= chunk) { + chunk = min_t(int, emulation_io->buf_len, rem); + + read_bio = bio_map_buf(emulation_io->buf, + emulation_io->buf_len, + emulation_io->gfp); + if (IS_ERR(read_bio)) { + ret = PTR_ERR(read_bio); + break; + } + read_bio->bi_opf = REQ_OP_READ | REQ_SYNC; + bio_set_dev(read_bio, emulation_io->bdev_in); + read_bio->bi_iter.bi_sector = pos_in >> SECTOR_SHIFT; + read_bio->bi_iter.bi_size = chunk; + ret = submit_bio_wait(read_bio); + kfree(read_bio); + if (ret) + break; + + write_bio = bio_map_buf(emulation_io->buf, + emulation_io->buf_len, + emulation_io->gfp); + if (IS_ERR(write_bio)) { + ret = PTR_ERR(write_bio); + break; + } + write_bio->bi_opf = REQ_OP_WRITE | REQ_SYNC; + bio_set_dev(write_bio, emulation_io->bdev_out); + write_bio->bi_iter.bi_sector = pos_out >> SECTOR_SHIFT; + write_bio->bi_iter.bi_size = chunk; + ret = submit_bio_wait(write_bio); + kfree(write_bio); + if (ret) + break; + + pos_in += chunk; + pos_out += chunk; + } + cio->status = ret; + kvfree(emulation_io->buf); + kfree(emulation_io); + blkdev_copy_endio(cio); +} + +static inline ssize_t queue_max_hw_bytes(struct request_queue *q) +{ + return min_t(ssize_t, queue_max_hw_sectors(q) << SECTOR_SHIFT, + queue_max_segments(q) << PAGE_SHIFT); +} +/* + * @bdev_in: source block device + * @pos_in: source offset + * @bdev_out: destination block device + * @pos_out: destination offset + * @len: length in bytes to be copied + * @endio: endio function to be called on completion of copy operation, + * for synchronous operation this should be NULL + * @private: endio function will be called with this private data, + * for synchronous operation this should be NULL + * @gfp_mask: memory allocation flags (for bio_alloc) + * + * For synchronous operation returns the length of bytes copied or error + * For asynchronous operation returns -EIOCBQUEUED or error + * + * Description: + * If native copy offload feature is absent, caller can use this function + * to perform copy. + * We store information required to perform the copy along with temporary + * buffer allocation. We async punt copy emulation to a worker. And worker + * performs copy in 2 steps. + * 1. Read data from source to temporary buffer + * 2. Write data to destination from temporary buffer + */ +ssize_t blkdev_copy_emulation(struct block_device *bdev_in, loff_t pos_in, + struct block_device *bdev_out, loff_t pos_out, + size_t len, void (*endio)(void *, int, ssize_t), + void *private, gfp_t gfp) +{ + struct request_queue *in = bdev_get_queue(bdev_in); + struct request_queue *out = bdev_get_queue(bdev_out); + struct blkdev_copy_emulation_io *emulation_io; + struct blkdev_copy_io *cio; + ssize_t ret; + size_t max_hw_bytes = min(queue_max_hw_bytes(in), + queue_max_hw_bytes(out)); + + ret = blkdev_copy_sanity_check(bdev_in, pos_in, bdev_out, pos_out, len); + if (ret) + return ret; + + cio = kzalloc(sizeof(*cio), gfp); + if (!cio) + return -ENOMEM; + + cio->waiter = current; + cio->copied = len; + cio->endio = endio; + cio->private = private; + + emulation_io = kzalloc(sizeof(*emulation_io), gfp); + if (!emulation_io) + goto err_free_cio; + emulation_io->cio = cio; + INIT_WORK(&emulation_io->emulation_work, blkdev_copy_emulation_work); + emulation_io->pos_in = pos_in; + emulation_io->pos_out = pos_out; + emulation_io->len = len; + emulation_io->bdev_in = bdev_in; + emulation_io->bdev_out = bdev_out; + emulation_io->gfp = gfp; + + emulation_io->buf = blkdev_copy_alloc_buf(min(max_hw_bytes, len), + &emulation_io->buf_len, gfp); + if (!emulation_io->buf) + goto err_free_emulation_io; + + schedule_work(&emulation_io->emulation_work); + + if (cio->endio) + return -EIOCBQUEUED; + + return blkdev_copy_wait_for_completion_io(cio); + +err_free_emulation_io: + kfree(emulation_io); +err_free_cio: + kfree(cio); + return -ENOMEM; +} +EXPORT_SYMBOL_GPL(blkdev_copy_emulation); + static int __blkdev_issue_write_zeroes(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop, unsigned flags) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index e8582a38adb7..9fa1ad68beb5 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1046,6 +1046,10 @@ ssize_t blkdev_copy_offload(struct block_device *bdev, loff_t pos_in, loff_t pos_out, size_t len, void (*endio)(void *, int, ssize_t), void *private, gfp_t gfp_mask); +ssize_t blkdev_copy_emulation(struct block_device *bdev_in, loff_t pos_in, + struct block_device *bdev_out, loff_t pos_out, + size_t len, void (*endio)(void *, int, ssize_t), + void *private, gfp_t gfp); #define BLKDEV_ZERO_NOUNMAP (1 << 0) /* do not free blocks */ #define BLKDEV_ZERO_NOFALLBACK (1 << 1) /* don't write explicit zeroes */ From patchwork Wed Dec 6 10:02:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 750984 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="cd1wParl" Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02E4AD62 for ; Wed, 6 Dec 2023 02:11:53 -0800 (PST) Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20231206101152epoutp022e8216c145a9492f61bc00d774660b0d~eNmP_8_sy2310223102epoutp02I for ; Wed, 6 Dec 2023 10:11:52 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20231206101152epoutp022e8216c145a9492f61bc00d774660b0d~eNmP_8_sy2310223102epoutp02I DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857512; bh=AOfFIjYI2wLqNYIL53RS+cLCGUBaMkDlBy8uIvV9EEU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cd1wParlIHUxsUbfkdHucKN5S4B108SkOdn2UE2QFoT5CT9a09J5xJIjTgbgMYLNr GB9uAp0SLhK6JPqS7QZJLg/pb3xUUY2nBJoUOm/6KxXmJT5Sa3lfEHj4R+WXWGWJoI 0X04DP+lwYEmhwwR1JG2g98PfxOo6mJC0L7NNfjE= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20231206101151epcas5p13faeb3c54d5e6e95672f50b1041fd65d~eNmPQdroJ1468514685epcas5p1B; Wed, 6 Dec 2023 10:11:51 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.176]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4SlY915zhlz4x9Pv; Wed, 6 Dec 2023 10:11:49 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 80.D6.10009.5E840756; Wed, 6 Dec 2023 19:11:49 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20231206101149epcas5p429e2a328af02819c71fa48ad01308a89~eNmNLRqTM2528425284epcas5p40; Wed, 6 Dec 2023 10:11:49 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20231206101149epsmtrp14047bbe569c539379849c452b0a67208~eNmNKMzPF1112611126epsmtrp1H; Wed, 6 Dec 2023 10:11:49 +0000 (GMT) X-AuditID: b6c32a4a-261fd70000002719-67-657048e52afe Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 89.52.08817.5E840756; Wed, 6 Dec 2023 19:11:49 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101141epsmtip2611ca7ae3531b6917df9b8dbc3213075~eNmF_pEh-1181011810epsmtip2G; Wed, 6 Dec 2023 10:11:41 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 06/12] fs, block: copy_file_range for def_blk_ops for direct block device Date: Wed, 6 Dec 2023 15:32:38 +0530 Message-Id: <20231206100253.13100-7-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Tf1CTdRzu+76v2zCWbwOPr+sHMEODDtgWG18UqDu13iuvSKs7oZrvsbdB G9vaD81+EKQCWvw+L5jyI0AtJAgGArIJQoBQAsGhAuEBskAoSIS4JKWNzfK/5/P9PM899zzf +3BwXhabz4lXGxidmlYJWOuJ823+WwNtlJYR/n3GH1V1d+Do9tIKgb7Iuoejc6OZLDTbtgDQ ZEsqQMUlBQQaamnEkKUkB0PfnWvHUE7rVYBsgyYMWYefQ9+klBHIYu0i0MCFUyxUdMbGRl9e a2Chs533MXQ9ywZQdtoghhomkwE6v1KEo8rZeQJdHn4C9d7rXPcin2o0jbKp3hvVBDVwxUjV lB9jUeayz6lpcz6gmoaSWFRpRu46Kv3wHIu6bRsmqPmLgywqo7YcUOafPqHu1DxN1Uz+gUVt iFaGxzG0nNH5MOpYjTxerYgQvLpXtkMmkQpFgaIwFCrwUdMJTIRg5+6owJfiVfZKBD4HaJXR /hRF6/WC4MhwncZoYHziNHpDhIDRylXaEG2Qnk7QG9WKIDVj2CYSCsUSO3G/Mu5E9i+YdmbD RxOtA3gSmHc/Dtw4kAyBtSbbuuNgPYdHNgGY8lsT4RwWAFy4aHFt/gKw8u4S+4GkLs/Kdi6s ACb3VGDO4Q6A1Uea7QOHwyL9YV+u0SHwJCtw2FgtcnBwshCH5olOzLHwIGVwdXUCd2CC9IO1 q8uEA3NJBK/MNRNON2+Y37+85uxGhsHcqRLcyXkcduVPrnFwO+dw3UncYQDJejdY3lXlEu+E fc0dmBN7wJnOWlcEPryVmeLCsbA/v8fFMcCblksu/AI82p2JO8Lg9jBVF4KdXo/B9JXJtYyQ 5MK0FJ6T7Qtv5Dh6dGAvOJ5X5sIUnM8vcPXzFYDT31eys4C36aEIpocimP53KwZ4OdjEaPUJ CkYv0YrVzMH/fjZWk1AD1k4j4JUGMD72Z1ArwDigFUAOLvDkqno1DI8rpw99zOg0Mp1Rxehb gcTecTbO3xirsd+W2iAThYQJQ6RSaUjY81KRwIs7e7RAziMVtIFRMoyW0T3QYRw3fhJWtN1/ KS3GN6eTM0ZbEy1vmJ6t2pLD8p3K2CTcU6kkxJHbgtzfrS+b4DZ5FtIJKlm0R30/VTJWGtb2 T9ojQk/3LYtVZtmJ7mVr3Y91qQt9978NZe6+6f7UgbmDqUXk67vxr6//sPlUu7/8mmKEV2Ns U1amh/56cl7djgqpIaVkxA/xn9nlfnof8U6A+GVpxYpO3Le93k/U5XVV5F382mdwbLjIMrK8 lLeYnOitvTSbXNo9xZh10++VLz5ZHzmzeqwjr+X9/s3my6P7RwvfCh435CXSn34IDv188+zv og/woF1bib2SWzGn3x40Rdn24JkNw6mPKsKj96lKeqQbxdNHdsQICH0cLQrAdXr6X6DehwWj BAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrMIsWRmVeSWpSXmKPExsWy7bCSvO5Tj4JUg92XuC3WnzrGbPHx628W i6YJf5ktVt/tZ7N4ffgTo8WTA+2MFgsWzWWxuHlgJ5PFnkWTmCxWrj7KZDHp0DVGi6dXZzFZ 7L2lbbGwbQmLxZ69J1ksLu+aw2Yxf9lTdovu6zvYLJYf/8dkcWPCU0aLiR1XmSx2PGlktNj2 ez6zxbrX71ksTtyStjj/9zirg5THzll32T3O39vI4nH5bKnHplWdbB6bl9R7vNg8k9Fj980G No/FfZNZPXqb37F5fHx6i8Xj/b6rbB59W1Yxemw+Xe3xeZOcx6Ynb5kC+KO4bFJSczLLUov0 7RK4MqZOvMhU8Iq/4tGhy8wNjO95uhg5OSQETCS2ztjL3sXIxSEksJtRon3CIjaIhLhE87Uf 7BC2sMTKf8+hij4ySvzc2cHYxcjBwSagKXFhcilIXERgB7PEz7XNTCANzAJrmCW2zOUFsYUF YiVO9r1mBbFZBFQltvz/zgJi8wpYSJx9t58FYoG8xMxL38GWcQpYSkx+vogZxBYCqtnXOB2q XlDi5MwnLBDz5SWat85mnsAoMAtJahaS1AJGplWMkqkFxbnpucWGBUZ5qeV6xYm5xaV56XrJ +bmbGMHxrqW1g3HPqg96hxiZOBgPMUpwMCuJ8Oacz08V4k1JrKxKLcqPLyrNSS0+xCjNwaIk zvvtdW+KkEB6YklqdmpqQWoRTJaJg1OqgclTvuZhtd+36MK4Yx08wR//CWft7HOoLxMyuXZ0 n63VJG2hzbLm6hPiVIsSKmx/cM3omPOFiSE29HqO3/KIecJskSu7X1jyCTQ65vT1nXhxeUZZ 6OaDO9TcYlvyy5Ztaz6YtvtbkMuq/T3Mtt9e2DfYLtaUNFu6RXHP7oDUORscJ8xev4XLOfzI Kcdw97z4hrxvD7O3CIXMr+lM9Hfclaj05I5Qy7YPAkV2v1YyLn17JVo6tiLndspbtitdjg8e 9LzTsy26GbN3q9Klme+t405e9VrpNut4FlOE9iuxmWJpjVsZXoacWKEifLu83WPR72TL0PB5 y5PzJ53837N63b/Fb660NpuxzDp8tc9cV4mlOCPRUIu5qDgRAAqih8RmAwAA X-CMS-MailID: 20231206101149epcas5p429e2a328af02819c71fa48ad01308a89 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101149epcas5p429e2a328af02819c71fa48ad01308a89 References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty For direct block device opened with O_DIRECT, use copy_file_range to issue device copy offload, or use generic_copy_file_range in case device copy offload capability is absent or the device files are not open with O_DIRECT. Reviewed-by: Hannes Reinecke Signed-off-by: Anuj Gupta Signed-off-by: Nitesh Shetty --- block/fops.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/block/fops.c b/block/fops.c index 0bdad1e8d514..08b652788e55 100644 --- a/block/fops.c +++ b/block/fops.c @@ -747,6 +747,30 @@ static ssize_t blkdev_read_iter(struct kiocb *iocb, struct iov_iter *to) return ret; } +static ssize_t blkdev_copy_file_range(struct file *file_in, loff_t pos_in, + struct file *file_out, loff_t pos_out, + size_t len, unsigned int flags) +{ + struct block_device *in_bdev = I_BDEV(bdev_file_inode(file_in)); + struct block_device *out_bdev = I_BDEV(bdev_file_inode(file_out)); + ssize_t copied = 0; + + if ((in_bdev == out_bdev) && bdev_max_copy_sectors(in_bdev) && + (file_in->f_iocb_flags & IOCB_DIRECT) && + (file_out->f_iocb_flags & IOCB_DIRECT)) { + copied = blkdev_copy_offload(in_bdev, pos_in, pos_out, len, + NULL, NULL, GFP_KERNEL); + if (copied < 0) + copied = 0; + } else { + copied = generic_copy_file_range(file_in, pos_in + copied, + file_out, pos_out + copied, + len - copied, flags); + } + + return copied; +} + #define BLKDEV_FALLOC_FL_SUPPORTED \ (FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE | \ FALLOC_FL_ZERO_RANGE | FALLOC_FL_NO_HIDE_STALE) @@ -851,6 +875,7 @@ const struct file_operations def_blk_fops = { .splice_read = filemap_splice_read, .splice_write = iter_file_splice_write, .fallocate = blkdev_fallocate, + .copy_file_range = blkdev_copy_file_range, }; static __init int blkdev_init(void) From patchwork Wed Dec 6 10:02:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 750983 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="L6bKngid" Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 156F7D68 for ; Wed, 6 Dec 2023 02:12:23 -0800 (PST) Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20231206101221epoutp020054b829b23150d9029c318b9f1df8f3~eNmrPKT_C2244322443epoutp023 for ; Wed, 6 Dec 2023 10:12:21 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20231206101221epoutp020054b829b23150d9029c318b9f1df8f3~eNmrPKT_C2244322443epoutp023 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857541; bh=uOAKePf/EC3LoZnFEX09yOYQgAaLwiBeLuJB0GWSSoA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=L6bKngidO1Tuve/sM0yGzW4aHSvJ4V8zR65igPFyX04th1v/DVpinxheb80zFGx1q +LLk5iRa+j2rigQFDStB1DggeOGZAxb6LtK64w2XM+zcsHq2GIc3lCmb2B4lyuTDnI pfm9Y0ynPIF1LRf/a3jXIHYchL9PLVnfzb0l5iwc= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20231206101220epcas5p35bbaf46727278f8d48a5fe4d6be0f5c4~eNmqZcRLT0789207892epcas5p3j; Wed, 6 Dec 2023 10:12:20 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.177]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4SlY9b22qSz4x9Pw; Wed, 6 Dec 2023 10:12:19 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 7E.D6.10009.30940756; Wed, 6 Dec 2023 19:12:19 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20231206101218epcas5p3bd5005a84adf67a80d394c3e05796bd7~eNmomUm130791407914epcas5p3g; Wed, 6 Dec 2023 10:12:18 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20231206101218epsmtrp217a2e5b0a9cb015fb653b55a7c6ddaed~eNmolF_vN1007210072epsmtrp2O; Wed, 6 Dec 2023 10:12:18 +0000 (GMT) X-AuditID: b6c32a4a-ff1ff70000002719-aa-6570490359be Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 80.62.08817.20940756; Wed, 6 Dec 2023 19:12:18 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101214epsmtip248d44b17f67df20f2c2e9a1abb6e15b2~eNmkNOL7S1181011810epsmtip2L; Wed, 6 Dec 2023 10:12:13 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 08/12] nvmet: add copy command support for bdev and file ns Date: Wed, 6 Dec 2023 15:32:40 +0530 Message-Id: <20231206100253.13100-9-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02TeVBTVxTG57738pLQiU2DHW/jTAmvwzBgWWIhvSh0UcY+u4HTzbbTgUje AJKtWYq2MqKMIBjZUh1Jla1oKahUAkyQTaHsBSyILC1M0aRQEAFRawWkCYmt//3OPd83Z75z 53BwQRZbyIlX6hiNUiqnSDeipsXH2w/fqWYCr8+xUEVXG44W7i8R6HD2Co7Kx7JINNNyFyDr lTSACovPEGjkSi2G6otzMfRjeSuGcptvAGQbNGGoYXQTKkotIVB9QyeBBi6fJlHBORsbHRuy kOiH9scYGs62AZRzdBBDFushgGqWCnB0cWaOQB2jG1HfSjvrDSFdaxpj033jlwh6oEdPV5al k7S55CA9Zc4DdN1IMkl/n2lk0cdT7pD0gm2UoOcaB0k6s6oM0Obub+jFyhfpSussFvnsZwmh cYxUxmhEjDJGJYtXxoZR73wQtT0qWBIo9hOHoFcpkVKqYMKo8Hcj/XbEy+0roURfSeV6+1Ok VKulAl4L1aj0OkYUp9LqwihGLZOrg9T+WqlCq1fG+isZ3RZxYODmYLswOiFu6oSJpV55a1/3 xWZWMhjZmgG4HMgPgv+cbmNnADeOgF8H4NDtacJZ3AXw0MlO0lk8APDX9A78iWUy/RzubDQA +KjjKHAWiwC2ZUzaLRwOyfeB14x6h2E9/zwOay+JHRqcn49D8812zNFw5++C5ZnGNSb4XnBo Nm1tAo+P4NipXtc0D5jX/zfbwVx+CDROFrs0z8HOPCvhYNyuSan+zqU3c6HFFu3kcNhbXc92 sjucbq9ysRAu3mkgnRwD+/N6MSfr4K36qy5+HR7pysIdWXB7lorLAc5R6+DxJSvmeIZ8Hjya KnCqPeF4ro3l5A1w4lSJi2m4kFvi2pUBwIdNtXg28DA9lcD0VALT/9MKAV4GXmDUWkUsow1W b1Yyif99bIxKUQnWLsP3bQuY+GPevxlgHNAMIAen1vPkfSpGwJNJ93/NaFRRGr2c0TaDYPuK c3Dh8zEq+2kpdVHioJDAIIlEEhTyikRMbeDNHDkjE/BjpTomgWHUjOaJD+NwhckY+3NbOHe5 2hIqyxhpOtByU8UpldvCuCM+6WRFkpsgvWhf4liKovUjr9/H0wL0ewr0XfNJ7z3w33r4ZGPh 2Y2d+lRvQ+Evnn37/bv2nk8ZBDI3fdJP5ZI9J7wOrOK2VTKfKBpdKfRsXfx5Sw2W3ZdIf/Ib DB/YPvzho2DR3r/4UztFEfcUuzzyL4BYWjz+6b0bMyaN+ury/Lbll67tPthIVmWLjvlGZO0W eVPRPfc1mjr3WWvPt13W5QvPrGuiAtyn/+xcNrxf2i8c8lr98vrZiNtk/sOZ9oQJcwFza/7l KKOl0WZo25HzhXEuQrXNQL8ZU5on6daG+RQYHidu+piXOEwR2jip2BfXaKX/AvwJ/J6iBAAA X-Brightmail-Tracker: H4sIAAAAAAAAA02Ra0hTYRjHe895d3ZmnTytsjdXSetCTfJG0RtdIcMTREqRRAS58jStTdeW VlZkjm4zTVtRrtRaZWZWOC9Z04jZbZqaiZepC5ItV2Jq0Q2dlY6gbz/+v//zPB8emhQ3Q386 PmE/r0mQK6WUD6yokQYsItar+ZBqx2z8oPYFiQe/DUGcluUh8V3HOQr31nwB2Pn0FMDXTLkQ 258+InCV6TyB79x9TuDz1laAXS1GAld3BOLrJ29CXFVtg7j58VUK5xe4hDi9rZLCt1+OELg9 ywVw9ukWAlc6jwNcMZRP4vu9/RC/6pDgRs9LwRp/7pHRIeQa35VArrk+iTMXnaG40pvHOHdp DuAs9lSKu5FpEHAZus8UN+jqgFz/kxaKyywrAlxp3WHuq3kWZ3b2EVG+23xWxPLK+GReE7wq xifOfdEoUHsiDtbdtwpSgX25HohoxC5GPWcKSD3wocWsBaDu4WHgFdOQrvWn0MuT0Z2RHqG3 NAiQ3lgL9YCmKXYhemNIGs2nsJUk+nVPR4wOkGwxicpymVGezEaivlsl5ChDdh5q6zs1xgyL keNyA+k9EIBy3v4YOyZilyFDj2ksF//tPDl+CXr7k5Atxwm9+wOQrvwKmQVY43/K+J+6Bogi MJ1Xa1UKlTZUHZbAHwjSylXapARF0K5ElRmM/V0mqwRVRQNBVkDQwAoQTUqnMMrGRF7MxMoP pfCaxB2aJCWvtQIJDaXTmO+9GbFiViHfz+/leTWv+WcJWuSfSqgF4rpXaPxKOniiIvWL343n TaLVSt2m4hhT4+uS3p65Z9MeZttTVq3rCmeEfsPPGGk9nufosxwoZiw1uzdKLLnEjKjuH7kb DB/7yzPTrOajwU6bUJU3VDgQUZZyjlo5J1LmiW3VJrc3bU0u6LKta+iw7YjrHNpOLI9ksvvj 00LCfX3XaNySPV0lnR8UBk+0M15XvoCE/nlnjcMZM/XQLXEfUiqiVm/eNS4mLzoqzDVVtv3W 6++7j+wrLgs/8T5wrshlbvpUG1gxNfDDQL3kmH2prLMzfUEYSeJvDTFBS9q5nW9Me+enTJ/Q nY7rA7pCfx9Zq3OtHWm7EPH49pZCKdTGyUNlpEYr/wMbdITmZgMAAA== X-CMS-MailID: 20231206101218epcas5p3bd5005a84adf67a80d394c3e05796bd7 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101218epcas5p3bd5005a84adf67a80d394c3e05796bd7 References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty Add support for handling nvme_cmd_copy command on target. For bdev-ns if backing device supports copy offload we call device copy offload (blkdev_copy_offload). In case of absence of device copy offload capability, we use copy emulation (blkdev_copy_emulation) For file-ns we call vfs_copy_file_range to service our request. Currently target always shows copy capability by setting NVME_CTRL_ONCS_COPY in controller ONCS. loop target has copy support, which can be used to test copy offload. trace event support for nvme_cmd_copy. Reviewed-by: Hannes Reinecke Signed-off-by: Nitesh Shetty Signed-off-by: Anuj Gupta --- drivers/nvme/target/admin-cmd.c | 9 +++- drivers/nvme/target/io-cmd-bdev.c | 71 +++++++++++++++++++++++++++++++ drivers/nvme/target/io-cmd-file.c | 50 ++++++++++++++++++++++ drivers/nvme/target/nvmet.h | 1 + drivers/nvme/target/trace.c | 19 +++++++++ 5 files changed, 148 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c index 39cb570f833d..4e1a6ca09937 100644 --- a/drivers/nvme/target/admin-cmd.c +++ b/drivers/nvme/target/admin-cmd.c @@ -433,8 +433,7 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req) id->nn = cpu_to_le32(NVMET_MAX_NAMESPACES); id->mnan = cpu_to_le32(NVMET_MAX_NAMESPACES); id->oncs = cpu_to_le16(NVME_CTRL_ONCS_DSM | - NVME_CTRL_ONCS_WRITE_ZEROES); - + NVME_CTRL_ONCS_WRITE_ZEROES | NVME_CTRL_ONCS_COPY); /* XXX: don't report vwc if the underlying device is write through */ id->vwc = NVME_CTRL_VWC_PRESENT; @@ -536,6 +535,12 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req) if (req->ns->bdev) nvmet_bdev_set_limits(req->ns->bdev, id); + else { + id->msrc = (__force u8)to0based(BIO_MAX_VECS - 1); + id->mssrl = cpu_to_le16(BIO_MAX_VECS << + (PAGE_SHIFT - SECTOR_SHIFT)); + id->mcl = cpu_to_le32(le16_to_cpu(id->mssrl)); + } /* * We just provide a single LBA format that matches what the diff --git a/drivers/nvme/target/io-cmd-bdev.c b/drivers/nvme/target/io-cmd-bdev.c index f11400a908f2..f974858ae5a0 100644 --- a/drivers/nvme/target/io-cmd-bdev.c +++ b/drivers/nvme/target/io-cmd-bdev.c @@ -46,6 +46,18 @@ void nvmet_bdev_set_limits(struct block_device *bdev, struct nvme_id_ns *id) id->npda = id->npdg; /* NOWS = Namespace Optimal Write Size */ id->nows = to0based(bdev_io_opt(bdev) / bdev_logical_block_size(bdev)); + + if (bdev_max_copy_sectors(bdev)) { + id->msrc = id->msrc; + id->mssrl = cpu_to_le16((bdev_max_copy_sectors(bdev) << + SECTOR_SHIFT) / bdev_logical_block_size(bdev)); + id->mcl = cpu_to_le32((__force u32)id->mssrl); + } else { + id->msrc = (__force u8)to0based(BIO_MAX_VECS - 1); + id->mssrl = cpu_to_le16((BIO_MAX_VECS << PAGE_SHIFT) / + bdev_logical_block_size(bdev)); + id->mcl = cpu_to_le32((__force u32)id->mssrl); + } } void nvmet_bdev_ns_disable(struct nvmet_ns *ns) @@ -451,6 +463,61 @@ static void nvmet_bdev_execute_write_zeroes(struct nvmet_req *req) } } +static void nvmet_bdev_copy_endio(void *private, int status, + ssize_t copied) +{ + struct nvmet_req *rq = (struct nvmet_req *)private; + u16 nvme_status; + + if (copied == rq->copy_len) + rq->cqe->result.u32 = cpu_to_le32(1); + else + rq->cqe->result.u32 = cpu_to_le32(0); + + nvme_status = errno_to_nvme_status(rq, status); + nvmet_req_complete(rq, nvme_status); +} + +/* + * At present we handle only one range entry, since copy offload is aligned with + * copy_file_range, only one entry is passed from block layer. + */ +static void nvmet_bdev_execute_copy(struct nvmet_req *rq) +{ + struct nvme_copy_range range; + struct nvme_command *cmd = rq->cmd; + ssize_t ret; + off_t dst, src; + + u16 status; + + status = nvmet_copy_from_sgl(rq, 0, &range, sizeof(range)); + if (status) + goto err_rq_complete; + + dst = le64_to_cpu(cmd->copy.sdlba) << rq->ns->blksize_shift; + src = le64_to_cpu(range.slba) << rq->ns->blksize_shift; + rq->copy_len = ((__force size_t)range.nlb + 1) << rq->ns->blksize_shift; + + if (bdev_max_copy_sectors(rq->ns->bdev)) { + ret = blkdev_copy_offload(rq->ns->bdev, dst, src, rq->copy_len, + nvmet_bdev_copy_endio, + (void *)rq, GFP_KERNEL); + } else { + ret = blkdev_copy_emulation(rq->ns->bdev, dst, + rq->ns->bdev, src, rq->copy_len, + nvmet_bdev_copy_endio, + (void *)rq, GFP_KERNEL); + } + if (ret == -EIOCBQUEUED) + return; + + rq->cqe->result.u32 = cpu_to_le32(0); + status = errno_to_nvme_status(rq, ret); +err_rq_complete: + nvmet_req_complete(rq, status); +} + u16 nvmet_bdev_parse_io_cmd(struct nvmet_req *req) { switch (req->cmd->common.opcode) { @@ -469,6 +536,10 @@ u16 nvmet_bdev_parse_io_cmd(struct nvmet_req *req) case nvme_cmd_write_zeroes: req->execute = nvmet_bdev_execute_write_zeroes; return 0; + case nvme_cmd_copy: + req->execute = nvmet_bdev_execute_copy; + return 0; + default: return nvmet_report_invalid_opcode(req); } diff --git a/drivers/nvme/target/io-cmd-file.c b/drivers/nvme/target/io-cmd-file.c index 2d068439b129..0a8337596f0c 100644 --- a/drivers/nvme/target/io-cmd-file.c +++ b/drivers/nvme/target/io-cmd-file.c @@ -322,6 +322,47 @@ static void nvmet_file_dsm_work(struct work_struct *w) } } +static void nvmet_file_copy_work(struct work_struct *w) +{ + struct nvmet_req *req = container_of(w, struct nvmet_req, f.work); + int nr_range = req->cmd->copy.nr_range + 1; + u16 status = 0; + int src, id; + ssize_t len, ret; + loff_t pos; + + pos = le64_to_cpu(req->cmd->copy.sdlba) << req->ns->blksize_shift; + if (unlikely(pos + req->transfer_len > req->ns->size)) { + nvmet_req_complete(req, errno_to_nvme_status(req, -ENOSPC)); + return; + } + + for (id = 0 ; id < nr_range; id++) { + struct nvme_copy_range range; + + status = nvmet_copy_from_sgl(req, id * sizeof(range), &range, + sizeof(range)); + if (status) + break; + + src = (le64_to_cpu(range.slba) << (req->ns->blksize_shift)); + len = (le16_to_cpu(range.nlb) + 1) << (req->ns->blksize_shift); + ret = vfs_copy_file_range(req->ns->file, src, req->ns->file, + pos, len, COPY_FILE_SPLICE); + pos += ret; + if (ret != len) { + req->cqe->result.u32 = cpu_to_le32(id); + if (ret < 0) + status = errno_to_nvme_status(req, ret); + else + status = errno_to_nvme_status(req, -EIO); + break; + } + } + + nvmet_req_complete(req, status); +} + static void nvmet_file_execute_dsm(struct nvmet_req *req) { if (!nvmet_check_data_len_lte(req, nvmet_dsm_len(req))) @@ -330,6 +371,12 @@ static void nvmet_file_execute_dsm(struct nvmet_req *req) queue_work(nvmet_wq, &req->f.work); } +static void nvmet_file_execute_copy(struct nvmet_req *req) +{ + INIT_WORK(&req->f.work, nvmet_file_copy_work); + queue_work(nvmet_wq, &req->f.work); +} + static void nvmet_file_write_zeroes_work(struct work_struct *w) { struct nvmet_req *req = container_of(w, struct nvmet_req, f.work); @@ -376,6 +423,9 @@ u16 nvmet_file_parse_io_cmd(struct nvmet_req *req) case nvme_cmd_write_zeroes: req->execute = nvmet_file_execute_write_zeroes; return 0; + case nvme_cmd_copy: + req->execute = nvmet_file_execute_copy; + return 0; default: return nvmet_report_invalid_opcode(req); } diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h index 6c8acebe1a1a..b648baeb52cf 100644 --- a/drivers/nvme/target/nvmet.h +++ b/drivers/nvme/target/nvmet.h @@ -405,6 +405,7 @@ struct nvmet_req { struct device *p2p_client; u16 error_loc; u64 error_slba; + size_t copy_len; }; #define NVMET_MAX_MPOOL_BVEC 16 diff --git a/drivers/nvme/target/trace.c b/drivers/nvme/target/trace.c index bff454d46255..551fdf029381 100644 --- a/drivers/nvme/target/trace.c +++ b/drivers/nvme/target/trace.c @@ -92,6 +92,23 @@ static const char *nvmet_trace_dsm(struct trace_seq *p, u8 *cdw10) return ret; } +static const char *nvmet_trace_copy(struct trace_seq *p, u8 *cdw10) +{ + const char *ret = trace_seq_buffer_ptr(p); + u64 sdlba = get_unaligned_le64(cdw10); + u8 nr_range = get_unaligned_le16(cdw10 + 8); + u16 control = get_unaligned_le16(cdw10 + 10); + u32 dsmgmt = get_unaligned_le32(cdw10 + 12); + u32 reftag = get_unaligned_le32(cdw10 + 16); + + trace_seq_printf(p, + "sdlba=%llu, nr_range=%u, ctrl=1x%x, dsmgmt=%u, reftag=%u", + sdlba, nr_range, control, dsmgmt, reftag); + trace_seq_putc(p, 0); + + return ret; +} + static const char *nvmet_trace_common(struct trace_seq *p, u8 *cdw10) { const char *ret = trace_seq_buffer_ptr(p); @@ -129,6 +146,8 @@ const char *nvmet_trace_parse_nvm_cmd(struct trace_seq *p, return nvmet_trace_read_write(p, cdw10); case nvme_cmd_dsm: return nvmet_trace_dsm(p, cdw10); + case nvme_cmd_copy: + return nvmet_trace_copy(p, cdw10); default: return nvmet_trace_common(p, cdw10); } From patchwork Wed Dec 6 10:02:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 750982 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="F90pFI3S" Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66BDAD6D for ; Wed, 6 Dec 2023 02:12:49 -0800 (PST) Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20231206101247epoutp0453d4a5f07e142dc04804494a819c1bb9~eNnDztogb1716717167epoutp04K for ; Wed, 6 Dec 2023 10:12:47 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20231206101247epoutp0453d4a5f07e142dc04804494a819c1bb9~eNnDztogb1716717167epoutp04K DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857567; bh=til+KvRoG+1Iq7dmMib83hgZfmKc66+HX7+qqGC9x84=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F90pFI3S3E8AN0HKk4FzmhMPO2pnz6KMJAfes8vFUt5UUpjnvfEw8vqGJ0OjyfVGj Z6dTzZJ3Uijz3LJIkpG94H+vlLwojwa+mqCPclRMn+p+ETok1c7si8akUnYiqV47Ve wnX/jVWQdaZmSz5EWcxv2KJjTL1qodnf3y5Yu308= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20231206101246epcas5p34ee90169766a41f9474ed71bea400f35~eNnC6IJGp0789207892epcas5p3S; Wed, 6 Dec 2023 10:12:46 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.178]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4SlYB50Nk1z4x9Q5; Wed, 6 Dec 2023 10:12:45 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 9A.22.09672.C1940756; Wed, 6 Dec 2023 19:12:44 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20231206101244epcas5p38c9d5f1cb01158321f59dbfb4f957470~eNnAdnHs90698906989epcas5p3Z; Wed, 6 Dec 2023 10:12:44 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20231206101244epsmtrp1c90fe9cdb177c4af9e602b20f796b0ae~eNnAcgHiz1112611126epsmtrp1x; Wed, 6 Dec 2023 10:12:44 +0000 (GMT) X-AuditID: b6c32a4b-60bfd700000025c8-98-6570491c2584 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 46.62.08817.C1940756; Wed, 6 Dec 2023 19:12:44 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101240epsmtip26a0b39642efe1b16d765e62dd823455a~eNm8nTf371205412054epsmtip2Q; Wed, 6 Dec 2023 10:12:39 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 10/12] dm: Enable copy offload for dm-linear target Date: Wed, 6 Dec 2023 15:32:42 +0530 Message-Id: <20231206100253.13100-11-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Ta0xTZxjOd87pBRb0rDL5wAWwcUuEgC203YcBNzc3jmOLZGKWjGzQweHa m70okywyuQyZgFDRgRKkK3KRyCiIlNugXrgsCuGi0gWmox1lDBDUqWGOtbRz/nve932ePN/z fnnZOKeA5cNOlalppUws4TLdibar2wOCXt+roHlzReGoaegGjpYfrxLo4lQxE81fXQHI0vst QOd1lQSa7DViqEtXiqH6i9cxVGq6DZB1ogJD3eZAVJ2nJ1BX9yCBxjrOMVHVBSsLfXennYlq +//B0N2TVoBK8icw1G75BqC21SocXZpfItCAeQsaft7PeMebMlZMsajh6WaCGrupoQwNx5lU i/4oZWspB1TnZBaT+qFIy6AKsxeZ1LLVTFBLPRNMqqi1AVAtP2dSDw2+lMGygEVv/Cw9PIUW J9JKf1qWIE9MlSVHcKP2x70XJxTx+EH8MPQW118mltIR3D0fRQd9kCqxb4Prf0gs0dhb0WKV irtjV7hSrlHT/ilylTqCSysSJQqBIlgllqo0suRgGa3eyefxQoR2Ynx6yuT0FFBMMTJqTvVg WcBGFAA3NiQF0DxyASsA7mwO2Qlg56M1wlmsAFhoaXYVfwE4O5b3QlJ/pRw4B90A1twyulgP ARybsbEKAJvNJLfDEa3GIfAkG3FobOY7ODj5DIPZdQPAMdhERsKnl/9mOTBBvgHrsrIwB/Yg w2DfUKvLzQ+Wjz5Z57jZ+9pZHe7kvAoHyy3rHNzOyb58FncYQLLJDV6qa2U4xXugrbWD5cSb 4B/9rS7sA+eK81w4AY6W38KcWA1nuvpc+G2YO1SMO8Lg9jBNHTucXhtg4aoFc7Qh6QHz8zhO 9lY4XWp1uXrB+9/rXZiCLabf15/JIU8A2Fuz/yTwq3gpQcVLCSr+NzsP8AbgTStU0mRaJVSE yujDLz42QS41gPWjCIhqB7/dexBsAhgbmABk41xPD8mwnOZ4JIq/OkIr5XFKjYRWmYDQvuIS 3Oe1BLn9qmTqOL4gjCcQiUSCsFARn+vlMZ9bmcghk8VqOp2mFbTyPx3GdvPJwnJqZYvWB59X nQ1IKwVx0qVTbtWfMFK1aeEZezMXr23OHh0v2z0TklFyQphkno4/x4/6YtIQ+UTgk13tdyy+ egP7vlZz+GnswrOZKZk0Q3DTa3jtXuOHvxDalbaD0bFDQv+dP9nWcg6afA9tfk5ONA0Mvs8f PiYJ7Pi0+871kvF9oddsoq7mNK+g3AM13TfMSSMHdO+Oz3Se1tO/9jze90i98eP5K8d1vrpd R3lpXzOq9YE9yXM12iGTNDKpvzL/rmCLIfzNnIzMM2XbPLsKte7TsVtnf+xUe0+E/Nm3wHpl MabodmZj/XIMmK/TrxnntKZtxiO5Ft4ZU8puXlnMeO2XpCeXUKWI+QG4UiX+F+4VIRudBAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrPIsWRmVeSWpSXmKPExsWy7bCSvK6MZ0Gqwbu/fBbrTx1jtvj49TeL xeq7/WwWrw9/YrR4cqCd0WLBorksFjcP7GSy2LNoEpPFytVHmSwmHbrGaPH06iwmi723tC0W ti1hsdiz9ySLxeVdc9gs5i97ym7RfX0Hm8Xy4/+YLG5MeMpoMbHjKpPFjieNjBbbfs9ntlj3 +j2LxYlb0hbn/x5ndZD02DnrLrvH+XsbWTwuny312LSqk81j85J6jxebZzJ67L7ZwOaxuG8y q0dv8zs2j49Pb7F4vN93lc2jb8sqRo/Np6s9Pm+S89j05C1TAH8Ul01Kak5mWWqRvl0CV8bN e3cZC+6yViydso+pgfEFSxcjJ4eEgInEyu0zGbsYuTiEBHYzSpy9fZwJIiEu0XztBzuELSyx 8t9zdoiij4wS0/uuAxVxcLAJaEpcmFwKEhcR2MEs8XNtMxOIwyzQzixx+tVrsEnCAu4SP7b+ AZvEIqAqsaKhASzOK2ApcfDUFqgz5CVmXvoOVsMJFJ/8fBEziC0kYCGxr3E6C0S9oMTJmU/A bGag+uats5knMArMQpKahSS1gJFpFaNkakFxbnpusWGBUV5quV5xYm5xaV66XnJ+7iZGcJxr ae1g3LPqg94hRiYOxkOMEhzMSiK8OefzU4V4UxIrq1KL8uOLSnNSiw8xSnOwKInzfnvdmyIk kJ5YkpqdmlqQWgSTZeLglGpgkrzp4N/ssnTl1ZksvfcP5OkeODfPYqvm6ux7sYvzlkw4Lba6 vMXc85oy7/PCiBNvl1wtdjy2/MsvTg0Rqw8XyjL8dsquUtZVWqSUPfu72l6uNfFVed/vG896 veWbWEjCIqXebWkt9gEmyy+fndMh6n3nwYn96lWdqse0vFmLJ6pscZ7U9CSLiW2ujonw7mkH pudufF63+2XbVLeyUP+PM8Lfp24Tc9y5kHl9nL8R29wQ/WmXLq+T7v4vwzY7PO1Qd+Gt3tWr lDbYBz3iWnkkfXLty0OCNdxzpr0sySqUn881/eDPj4UJ5Ur69r7XdZVYozM+/Kq3UvugobRs /fJf+eENNh8XfI3RimXh61ozRYmlOCPRUIu5qDgRABxSzHRiAwAA X-CMS-MailID: 20231206101244epcas5p38c9d5f1cb01158321f59dbfb4f957470 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101244epcas5p38c9d5f1cb01158321f59dbfb4f957470 References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty Setting copy_offload_supported flag to enable offload. Reviewed-by: Hannes Reinecke Signed-off-by: Nitesh Shetty --- drivers/md/dm-linear.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/md/dm-linear.c b/drivers/md/dm-linear.c index 2d3e186ca87e..cfec2fac28e1 100644 --- a/drivers/md/dm-linear.c +++ b/drivers/md/dm-linear.c @@ -62,6 +62,7 @@ static int linear_ctr(struct dm_target *ti, unsigned int argc, char **argv) ti->num_discard_bios = 1; ti->num_secure_erase_bios = 1; ti->num_write_zeroes_bios = 1; + ti->copy_offload_supported = 1; ti->private = lc; return 0; From patchwork Wed Dec 6 10:02:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 750981 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="RONvNQbf" Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC71C1989 for ; Wed, 6 Dec 2023 02:13:15 -0800 (PST) Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20231206101313epoutp023ed449afab2d10af5c3be90689f4551b~eNnbrzrYc2240722407epoutp02G for ; Wed, 6 Dec 2023 10:13:13 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20231206101313epoutp023ed449afab2d10af5c3be90689f4551b~eNnbrzrYc2240722407epoutp02G DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857593; bh=xTrfyi/yJidoNrl3TPzzopI+GnPn+ZUhLrlBrAJpZmk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RONvNQbfKPP7JVvjs7/Cza28A52wzlIx90cwYe3YvAdVCJEITxSXUxRQRkVuU4Hrs EXjltQKrrtIHLDJNzBZc0Si7sXIvm09t8hUWceIR4z3qQVGCJjLsRAkPxiyMnS4Ifn sKZNZuXoukFqHOHKNCzHaWtMMOTo3WvokxoY+HMs= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20231206101312epcas5p27ba89066024e85afe3dcbfa68f2ad196~eNna9UUf43251632516epcas5p20; Wed, 6 Dec 2023 10:13:12 +0000 (GMT) Received: from epsmgec5p1-new.samsung.com (unknown [182.195.38.175]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4SlYBb3qKRz4x9Pw; Wed, 6 Dec 2023 10:13:11 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmgec5p1-new.samsung.com (Symantec Messaging Gateway) with SMTP id 74.A8.19369.73940756; Wed, 6 Dec 2023 19:13:11 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20231206101310epcas5p43f06695868958526fed828762e728e7b~eNnZC_V9U2295022950epcas5p46; Wed, 6 Dec 2023 10:13:10 +0000 (GMT) Received: from epsmgmcp1.samsung.com (unknown [182.195.42.82]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20231206101310epsmtrp24b03b0bf079829d4f931fce53db76085~eNnZBqF9B1007210072epsmtrp21; Wed, 6 Dec 2023 10:13:10 +0000 (GMT) X-AuditID: b6c32a50-c99ff70000004ba9-4a-657049372387 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgmcp1.samsung.com (Symantec Messaging Gateway) with SMTP id 6A.85.18939.63940756; Wed, 6 Dec 2023 19:13:10 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101305epsmtip2045c6dd8db41916fbde2ae92d7cd27d5~eNnUSGSf50087600876epsmtip2e; Wed, 6 Dec 2023 10:13:05 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Damien Le Moal , Anuj Gupta , Vincent Fu , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 12/12] null_blk: add support for copy offload Date: Wed, 6 Dec 2023 15:32:44 +0530 Message-Id: <20231206100253.13100-13-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02Te0xbdRTH97v3trTL2K5lZL+wZbBGEoEUWuXxAwq6bBlXRqSJURPRQC03 hVDa2seGmmUdWB7teIxOM2CjvNwDEOQpRVgmxCE4Vjbs1pKAOlo0MkFgAycPbS2b++9zTr4n 33zPyWHhnFqfAFa2XEOr5GIZl7mT6B0OCeHFvK6k+Vd1ELWP3cTR0uN1AuVXbOKoZbqcieaH lwFy3igCaH3ciqO6hksEctywYGigoRJD11q+w1Dl0D2AXLZqDA1OhaH6wiYCDQyOEmiy/yIT mS+7fJDxfh8TXRnZwpC9wgXQuWIbhvqcZwDqXTfjqG1+kUDfT+1Hs8ZigKybI4zXDlCW6mkf yjrTQVCT41qqs7mESXU1naZ+66oC1DcOHZNqLDMxqNKCBSZl0f/EoJZcUwS1eN3GpMq6mwHV 9cMn1ErnQarT+QcmIt/NEWbR4kxaFUTLJYrMbLk0gXv8zfQj6VHRfAFPEItiuEFycS6dwD2a IuIdy5a5N8QNOiGWad0tkVit5kYkClUKrYYOylKoNQlcWpkpU0Yqw9XiXLVWLg2X05o4AZ// cpRbmJGTVblhYCh1SXl9owoduBdnAGwWJCPhF2sjTAPYyeKQAwAWuK5j3mIZQN2VVeaz4tP2 HuzpyNn8y7iHOaQFwGIz6RWtAPijfY5hACwWkwyBEyatR7OXbMWhpUPg0eDkXzhsHPH4sVl+ 5GHomGoCHibIYPjAXsrwsC8ZCwcetTK9ZoGw6u6aj4fZ7r7p1wbcq3kBjlY5CQ/jbk1BTw3u MYDkMBtOzzkY3uGjcO7rEuBlP/j7SLePlwPgysLgtoEE3q26vZ1MA2cHvt3mV6F+rBz3hMHd Ydr7I7xeu2HpuhPztCHpC4sLOV71IThT6dp23Qd/udC0zRRsKRwnvPs5C+DMmJlRAQKrn4tQ /VyE6v/d6gDeDAJopTpXSkuilAKenD757LASRW4n+O9RQkV9oOWrzfAhgLHAEIAsnLvXV2ZV 0BzfTPFHH9MqRbpKK6PVQyDKveRzeIC/ROH+NLkmXRAZy4+Mjo6OjH0lWsDd5zuvv5TJIaVi DZ1D00pa9XQOY7EDdJhRa5d9UGtvTbJFSZMNh4vUX9ruhE4K+/qPv3jnVINT/49vPB4u+RCr HytvPBXSSpaJ8Pbyh0OzQRt80UL8g5vx72vyKvP0Vtd+aczJ3vsm09buNHNYkZ82P0N45G+j xS+wUGgvnre9ETbNS1UR4wfNu2rXUopqatK3DMNntOLzvHrhbWE9uWcpyXWiZtV84MKxxccR s8Fs04Sxvz0Ye+JI1qMWxi3jjl1pC4aS+rbEsoDOlw619cSTDVlPEvf8PGidTDE4Coyp7yRn 2FZXrxUkmicu/il9+FZ4U+6y/60i7XshaY/eFkn9ZdTnNhiXlLrx2TRMqxtpO301t7tjB5dQ Z4kFobhKLf4Xr0p7nLEEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrLIsWRmVeSWpSXmKPExsWy7bCSvK6ZZ0Gqwbb1KhbrTx1jtvj49TeL RdOEv8wWq+/2s1m8PvyJ0eLJgXZGi99nzzNbLFg0l8Xi5oGdTBZ7Fk1isli5+iiTxaRD1xgt nl6dxWSx95a2xcK2JSwWe/aeZLG4vGsOm8X8ZU/ZLbqv72CzWH78H5PFjQlPGS0mdlxlstjx pJHRYtvv+cwW616/Z7E4cUva4nF3B6PF+b/HWR1kPHbOusvucf7eRhaPy2dLPTat6mTz2Lyk 3uPF5pmMHrtvNrB5LO6bzOrR2/yOzWNn631Wj49Pb7F4vN93lc2jb8sqRo/Np6s9Pm+S89j0 5C1TgEAUl01Kak5mWWqRvl0CV8akP12sBQ3uFTtO5jcwXrPqYuTkkBAwkehpWsYMYgsJbGeU mPFXHiIuLtF87Qc7hC0ssfLfcyCbC6jmI6PE3487WLoYOTjYBDQlLkwuBYmLCOxglvi5tpkJ xGEW6GCRuLBtL1i3sICjxM1bSxhBbBYBVYlHN3pZQWxeAUuJPV/WsEFskJeYeek7WD0nUHzy 80VQF1lI7GuczgJRLyhxcuYTMJsZqL5562zmCYwCs5CkZiFJLWBkWsUomlpQnJuem1xgqFec mFtcmpeul5yfu4kRnAK0gnYwLlv/V+8QIxMH4yFGCQ5mJRHenPP5qUK8KYmVValF+fFFpTmp xYcYpTlYlMR5lXM6U4QE0hNLUrNTUwtSi2CyTBycUg1M3iXFdZdzDiSpbVI1MvnQbW25/k4/ k5I99w2unWuuPjvNkWF2933znmezbBM69h9tXLix7gDH+W9zZeJMdp17fbdOdFZL3pUPBv9C XknlvPgaIzN9DZvj1YS3y5J9cqI3PlthfKTY7+PRGWuvBrzJOtQQaH9cmkdYZH7+Jpddk9rN 5jwTfuloJR8cwB9zacMayQ1ClqXhP0xu8Cp7LZh3syr5cdPm3BdG7XN+pb+z38nDcJfhu1CE Y/TboJut/gvylXSWi89Me8KzvuZsSsjKKaG3O1r6FMTzZ6YVb6sJXfiryfviGfOzLqqPMz59 Mp1829o4OOP08y61fvYjoT+3PVjQeC6Z8ROXrPPUOdd3KrEUZyQaajEXFScCAPx2ZblwAwAA X-CMS-MailID: 20231206101310epcas5p43f06695868958526fed828762e728e7b X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101310epcas5p43f06695868958526fed828762e728e7b References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty Implementation is based on existing read and write infrastructure. copy_max_bytes: A new configfs and module parameter is introduced, which can be used to set hardware/driver supported maximum copy limit. Only request based queue mode will support for copy offload. Added tracefs support to copy IO tracing. Reviewed-by: Hannes Reinecke Suggested-by: Damien Le Moal Signed-off-by: Anuj Gupta Signed-off-by: Nitesh Shetty Signed-off-by: Vincent Fu --- Documentation/block/null_blk.rst | 5 ++ drivers/block/null_blk/main.c | 97 ++++++++++++++++++++++++++++++- drivers/block/null_blk/null_blk.h | 1 + drivers/block/null_blk/trace.h | 23 ++++++++ 4 files changed, 123 insertions(+), 3 deletions(-) diff --git a/Documentation/block/null_blk.rst b/Documentation/block/null_blk.rst index 4dd78f24d10a..6153e02fcf13 100644 --- a/Documentation/block/null_blk.rst +++ b/Documentation/block/null_blk.rst @@ -149,3 +149,8 @@ zone_size=[MB]: Default: 256 zone_nr_conv=[nr_conv]: Default: 0 The number of conventional zones to create when block device is zoned. If zone_nr_conv >= nr_zones, it will be reduced to nr_zones - 1. + +copy_max_bytes=[size in bytes]: Default: COPY_MAX_BYTES + A module and configfs parameter which can be used to set hardware/driver + supported maximum copy offload limit. + COPY_MAX_BYTES(=128MB at present) is defined in fs.h diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c index 1b40c674f62b..5b1b401a68fb 100644 --- a/drivers/block/null_blk/main.c +++ b/drivers/block/null_blk/main.c @@ -160,6 +160,10 @@ static int g_max_sectors; module_param_named(max_sectors, g_max_sectors, int, 0444); MODULE_PARM_DESC(max_sectors, "Maximum size of a command (in 512B sectors)"); +static unsigned long g_copy_max_bytes = BLK_COPY_MAX_BYTES; +module_param_named(copy_max_bytes, g_copy_max_bytes, ulong, 0444); +MODULE_PARM_DESC(copy_max_bytes, "Maximum size of a copy command (in bytes)"); + static unsigned int nr_devices = 1; module_param(nr_devices, uint, 0444); MODULE_PARM_DESC(nr_devices, "Number of devices to register"); @@ -412,6 +416,7 @@ NULLB_DEVICE_ATTR(home_node, uint, NULL); NULLB_DEVICE_ATTR(queue_mode, uint, NULL); NULLB_DEVICE_ATTR(blocksize, uint, NULL); NULLB_DEVICE_ATTR(max_sectors, uint, NULL); +NULLB_DEVICE_ATTR(copy_max_bytes, uint, NULL); NULLB_DEVICE_ATTR(irqmode, uint, NULL); NULLB_DEVICE_ATTR(hw_queue_depth, uint, NULL); NULLB_DEVICE_ATTR(index, uint, NULL); @@ -553,6 +558,7 @@ static struct configfs_attribute *nullb_device_attrs[] = { &nullb_device_attr_queue_mode, &nullb_device_attr_blocksize, &nullb_device_attr_max_sectors, + &nullb_device_attr_copy_max_bytes, &nullb_device_attr_irqmode, &nullb_device_attr_hw_queue_depth, &nullb_device_attr_index, @@ -659,7 +665,8 @@ static ssize_t memb_group_features_show(struct config_item *item, char *page) "poll_queues,power,queue_mode,shared_tag_bitmap,size," "submit_queues,use_per_node_hctx,virt_boundary,zoned," "zone_capacity,zone_max_active,zone_max_open," - "zone_nr_conv,zone_offline,zone_readonly,zone_size\n"); + "zone_nr_conv,zone_offline,zone_readonly,zone_size," + "copy_max_bytes\n"); } CONFIGFS_ATTR_RO(memb_group_, features); @@ -725,6 +732,7 @@ static struct nullb_device *null_alloc_dev(void) dev->queue_mode = g_queue_mode; dev->blocksize = g_bs; dev->max_sectors = g_max_sectors; + dev->copy_max_bytes = g_copy_max_bytes; dev->irqmode = g_irqmode; dev->hw_queue_depth = g_hw_queue_depth; dev->blocking = g_blocking; @@ -1274,6 +1282,81 @@ static int null_transfer(struct nullb *nullb, struct page *page, return err; } +static inline int nullb_setup_copy(struct nullb *nullb, struct request *req, + bool is_fua) +{ + sector_t sector_in = 0, sector_out = 0; + loff_t offset_in, offset_out; + void *in, *out; + ssize_t chunk, rem = 0; + struct bio *bio; + struct nullb_page *t_page_in, *t_page_out; + u16 seg = 1; + int status = -EIO; + + if (blk_rq_nr_phys_segments(req) != BLK_COPY_MAX_SEGMENTS) + return status; + + /* + * First bio contains information about source and last bio contains + * information about destination. + */ + __rq_for_each_bio(bio, req) { + if (seg == blk_rq_nr_phys_segments(req)) { + sector_out = bio->bi_iter.bi_sector; + if (rem != bio->bi_iter.bi_size) + return status; + } else { + sector_in = bio->bi_iter.bi_sector; + rem = bio->bi_iter.bi_size; + } + seg++; + } + + trace_nullb_copy_op(req, sector_out << SECTOR_SHIFT, + sector_in << SECTOR_SHIFT, rem); + + spin_lock_irq(&nullb->lock); + while (rem > 0) { + chunk = min_t(size_t, nullb->dev->blocksize, rem); + offset_in = (sector_in & SECTOR_MASK) << SECTOR_SHIFT; + offset_out = (sector_out & SECTOR_MASK) << SECTOR_SHIFT; + + if (null_cache_active(nullb) && !is_fua) + null_make_cache_space(nullb, PAGE_SIZE); + + t_page_in = null_lookup_page(nullb, sector_in, false, + !null_cache_active(nullb)); + if (!t_page_in) + goto err; + t_page_out = null_insert_page(nullb, sector_out, + !null_cache_active(nullb) || + is_fua); + if (!t_page_out) + goto err; + + in = kmap_local_page(t_page_in->page); + out = kmap_local_page(t_page_out->page); + + memcpy(out + offset_out, in + offset_in, chunk); + kunmap_local(out); + kunmap_local(in); + __set_bit(sector_out & SECTOR_MASK, t_page_out->bitmap); + + if (is_fua) + null_free_sector(nullb, sector_out, true); + + rem -= chunk; + sector_in += chunk >> SECTOR_SHIFT; + sector_out += chunk >> SECTOR_SHIFT; + } + + status = 0; +err: + spin_unlock_irq(&nullb->lock); + return status; +} + static int null_handle_rq(struct nullb_cmd *cmd) { struct request *rq = cmd->rq; @@ -1283,13 +1366,16 @@ static int null_handle_rq(struct nullb_cmd *cmd) sector_t sector = blk_rq_pos(rq); struct req_iterator iter; struct bio_vec bvec; + bool fua = rq->cmd_flags & REQ_FUA; + + if (op_is_copy(req_op(rq))) + return nullb_setup_copy(nullb, rq, fua); spin_lock_irq(&nullb->lock); rq_for_each_segment(bvec, rq, iter) { len = bvec.bv_len; err = null_transfer(nullb, bvec.bv_page, len, bvec.bv_offset, - op_is_write(req_op(rq)), sector, - rq->cmd_flags & REQ_FUA); + op_is_write(req_op(rq)), sector, fua); if (err) { spin_unlock_irq(&nullb->lock); return err; @@ -2074,6 +2160,9 @@ static int null_validate_conf(struct nullb_device *dev) return -EINVAL; } + if (dev->queue_mode == NULL_Q_BIO) + dev->copy_max_bytes = 0; + return 0; } @@ -2193,6 +2282,8 @@ static int null_add_dev(struct nullb_device *dev) dev->max_sectors = queue_max_hw_sectors(nullb->q); dev->max_sectors = min(dev->max_sectors, BLK_DEF_MAX_SECTORS); blk_queue_max_hw_sectors(nullb->q, dev->max_sectors); + blk_queue_max_copy_hw_sectors(nullb->q, + dev->copy_max_bytes >> SECTOR_SHIFT); if (dev->virt_boundary) blk_queue_virt_boundary(nullb->q, PAGE_SIZE - 1); diff --git a/drivers/block/null_blk/null_blk.h b/drivers/block/null_blk/null_blk.h index 929f659dd255..e82e53a2e2df 100644 --- a/drivers/block/null_blk/null_blk.h +++ b/drivers/block/null_blk/null_blk.h @@ -107,6 +107,7 @@ struct nullb_device { unsigned int queue_mode; /* block interface */ unsigned int blocksize; /* block size */ unsigned int max_sectors; /* Max sectors per command */ + unsigned long copy_max_bytes; /* Max copy offload length in bytes */ unsigned int irqmode; /* IRQ completion handler */ unsigned int hw_queue_depth; /* queue depth */ unsigned int index; /* index of the disk, only valid with a disk */ diff --git a/drivers/block/null_blk/trace.h b/drivers/block/null_blk/trace.h index 91446c34eac2..2f2c1d1c2b48 100644 --- a/drivers/block/null_blk/trace.h +++ b/drivers/block/null_blk/trace.h @@ -70,6 +70,29 @@ TRACE_EVENT(nullb_report_zones, ); #endif /* CONFIG_BLK_DEV_ZONED */ +TRACE_EVENT(nullb_copy_op, + TP_PROTO(struct request *req, + sector_t dst, sector_t src, size_t len), + TP_ARGS(req, dst, src, len), + TP_STRUCT__entry( + __array(char, disk, DISK_NAME_LEN) + __field(enum req_op, op) + __field(sector_t, dst) + __field(sector_t, src) + __field(size_t, len) + ), + TP_fast_assign( + __entry->op = req_op(req); + __assign_disk_name(__entry->disk, req->q->disk); + __entry->dst = dst; + __entry->src = src; + __entry->len = len; + ), + TP_printk("%s req=%-15s: dst=%llu, src=%llu, len=%lu", + __print_disk_name(__entry->disk), + blk_op_str(__entry->op), + __entry->dst, __entry->src, __entry->len) +); #endif /* _TRACE_NULLB_H */ #undef TRACE_INCLUDE_PATH