From patchwork Tue Aug 17 10:14:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SelvaKumar S X-Patchwork-Id: 498451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B017C4338F for ; Tue, 17 Aug 2021 11:28:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 315EC60F38 for ; Tue, 17 Aug 2021 11:28:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236541AbhHQL3W (ORCPT ); Tue, 17 Aug 2021 07:29:22 -0400 Received: from mailout3.samsung.com ([203.254.224.33]:27507 "EHLO mailout3.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236705AbhHQL3U (ORCPT ); Tue, 17 Aug 2021 07:29:20 -0400 Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20210817112846epoutp03117b682a0992ebbce75a43791d68454e~cFIT79KiG2956529565epoutp03B for ; Tue, 17 Aug 2021 11:28:46 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20210817112846epoutp03117b682a0992ebbce75a43791d68454e~cFIT79KiG2956529565epoutp03B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1629199726; bh=JaOd9JjOYxrnNIdPMICXxFxsRx83cGVgqCM0mT/jHME=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AlXnInoy1Kv4PPwyKYUQEqeghURA6YA5QB4F/t41nrdMTLahYpei2cW6OaeHCAPdo QyU82jqOsoSS9wdgbnYjQYuh4sk3fi8MAHrDLzWeeI196zjnBerU+5XbxdtycO+/DN HXbwdFcnBV0gfUMLDjd6ziwHU4kB/zHJpkX3+oJA= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20210817112845epcas5p32d1c089554c65f3e91025beaf33a6e7a~cFITDu5-e3045330453epcas5p3g; Tue, 17 Aug 2021 11:28:45 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.183]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4Gppgr4VQ5z4x9Pv; Tue, 17 Aug 2021 11:28:40 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 1D.6B.09595.86D9B116; Tue, 17 Aug 2021 20:28:40 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20210817101747epcas5p1242e63ec29b127b03b6f9f5f1b57f86e~cEKVUwePg2631326313epcas5p1u; Tue, 17 Aug 2021 10:17:47 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20210817101747epsmtrp2b441c8dd1f0d6b49880929fbd90a78c8~cEKVTZycv2759327593epsmtrp2e; Tue, 17 Aug 2021 10:17:47 +0000 (GMT) X-AuditID: b6c32a4a-ed5ff7000000257b-db-611b9d682598 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id CF.4E.08394.BCC8B116; Tue, 17 Aug 2021 19:17:47 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20210817101743epsmtip26ea1dba895d2587d2d1253811ee1d451~cEKRjxdLs0079500795epsmtip2g; Tue, 17 Aug 2021 10:17:43 +0000 (GMT) From: SelvaKumar S To: linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: linux-api@vger.kernel.org, linux-scsi@vger.kernel.org, linux-fsdevel@vger.kernel.org, dm-devel@redhat.com, kbusch@kernel.org, axboe@kernel.dk, damien.lemoal@wdc.com, asml.silence@gmail.com, johannes.thumshirn@wdc.com, hch@lst.de, willy@infradead.org, kch@kernel.org, martin.petersen@oracle.com, mpatocka@redhat.com, bvanassche@acm.org, djwong@kernel.org, snitzer@redhat.com, agk@redhat.com, selvajove@gmail.com, joshiiitr@gmail.com, nj.shetty@samsung.com, nitheshshetty@gmail.com, joshi.k@samsung.com, javier.gonz@samsung.com, SelvaKumar S Subject: [PATCH 1/7] block: make bio_map_kern() non static Date: Tue, 17 Aug 2021 15:44:17 +0530 Message-Id: <20210817101423.12367-2-selvakuma.s1@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210817101423.12367-1-selvakuma.s1@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01TbUxTVxj23FtuC6HmWjAeq9tYE2CFAK2j3QEBMaK5EXAky8ZwOLjATcso bdMPcOoGuA8QpoKWr1oGMQwHJDiLVD4NAcTxFZgoiBsGNtCIU0CmYABZS3Hz33Oe933eJ+9z 8nJwXiGbz0lW6hiNklYICCeWpVP4no+8bActmnjIR5d7u3FkqrEAVDt+lkBFcy9x9F32CwwN T21GbU8vOKDq2hsY+uuPBTZazb2PoRtrTwg0+KQTQ+c6RgBqNRtYqH4xh0Bt97xRa1sPC5VX TbNR3mgjgS7dfIWhgpw7GGqcygLIslyOo9uGiziamB1lo8dLPQT6/spzgJaXTEToTmr4djjV ZBxnU/U/e1HDA3rKXHOKoOorM6iWsUyCmp++x6Jmr98hqDNXawC1YH6bym7Pw6KcD6cEyRk6 idG4McpEVVKyUhYsCP8obl+cRCoS+4gD0AcCNyWdygQLwiKifA4kK6xhCNzSaIXeSkXRWq3A LyRIo9LrGDe5SqsLFjDqJIXaX+2rpVO1eqXMV8noAsUi0S6JtTE+Rd7UOeSgHnM+Ovh8lMgE TU65wJEDSX/4e95P7FzgxOGRLQCe7zJg9sczAPset29UXgBYdf0H7LWkpapuo9AG4LmKk7it wCMXABxpkNkwQfrAkUozy4ZdyT2w+mWNg02Ak70sWDx7kW0ruJCBsNGY5WDDLNIdDixPr/Nc MhgaJ06y7W7vwNJbi1bM4TiSIbD5mbu9ZQvsKZ1an49bW75puIDb5kOy1hEWWOaBXRsGS1qK cTt2gTM3r27M5MOFp22EHafDB6eKNjbLBPDMbLod74G/ta5iNl+cFMLLzX52+i1Y2FuH2X03 w9PLUxtSLmz8cWq9HZIesPea1E7vhHOdzRtOFCxpX2XZcyuwpjt6hZUP3IxvrGN8Yx3j/84V AK8B2xm1NlXGaCXqXUom/b9PTlSlmsH6fXgdbASTE3O+HQDjgA4AObjAlevJ4dM8bhL95TFG o4rT6BWMtgNIrHEX4PytiSrrgSl1cWL/AJG/VCr1D3hfKhZs48ZG7KB5pIzWMSkMo2Y0r3UY x5Gfie2nV/w8jsSuxJi7nTkN40dn9xd6C6fXJksf9vRFhwRGhxH94Nfu7n3KfEPiNql3/PLe /rSMlQcfk1tE1b/kx3oc2h6SzDN+EhTo07UKDot55si9f6vr3CVxbM/o8BOfOifohZt80+ay dIZQGUzoKopNE7p8fTyKGPDs2eR57JrTu0EJuQdOdPDlJV94vTqbE3P6q/GK0EsztJPFlKk4 YmzpPx6vC2GI+Xy982yGUXBfmvfI1Rtnhszhn3Fj6ky9I6MJJsehwdWysRjToicZOaAM7Lu7 dbfhz6JHH2avlVfKb90t+3wfv2B3sVBycPAflxnivBmPWJKOfTvZ7hppCROwtHJa7IVrtPS/ ClP5S6gEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrDIsWRmVeSWpSXmKPExsWy7bCSvO7pHulEg++HRSzWnzrGbDFn1TZG i9V3+9kspn34yWzR2v6NyeLyEz6Lve9ms1qsXH2UyeLxnc/sFn+77jFZHP3/ls3i/NvDTBaT Dl1jtNizaQqLxebvHWwWe29pW+zZe5LFYv6yp+wW3dd3sFksP/6PyWJix1Umix1PGhkttv2e z2xxZcoiZosH76+zW7z+cZLNom3jV0aL3z/msDnIeFy+4u2xc9Zddo/NK7Q8Lp8t9di0qpPN Y/OSeo/dNxvYPD4+vcXi8X7fVTaPvi2rGD0+b5LzaD/QzRTAE8Vlk5Kak1mWWqRvl8CVsfPw BdaCmzwV579eZ2tg3MnVxcjJISFgIrF72Tp2EFtIYDejxIoOeYi4jMTau51sELawxMp/z4Fq uIBqPjJKLOu8xwSSYBPQlbi2ZBMLiC0i4Chx+tM2JpAiZoFXLBL37u5mBkkIC1hJ7JjVyApi swioSpz9/RRsG6+ArcSsB03sEBvkJWZe+g5kc3BwCthJ7PqkCnGQrcT+PVOhygUlTs58AraL Gai8eets5gmMArOQpGYhSS1gZFrFKJlaUJybnltsWGCYl1quV5yYW1yal66XnJ+7iREc4Vqa Oxi3r/qgd4iRiYPxEKMEB7OSCK86h1SiEG9KYmVValF+fFFpTmrxIUZpDhYlcd4LXSfjhQTS E0tSs1NTC1KLYLJMHJxSDUxVLwud3lq2fFOscdqW2XPEjUdUI8bw3+elt1tX8QrIMLEeu57/ fq9V8cbPwdYGxr+srV/27dU+YXW69WnZ7EeJZ0qdJHN+evEdjXdke/Xbd1r7mbMT1jJUWbB5 PCm8++tr9O/uZ9ufpR6PrLFb/J0vZ2Or/fR/fz90rcuc0PjQ+012lIDKcVWNRW7KYTPZPNfY 77GQnmhyyT14D0Osf/CRR2ra9iX+uro7N68WfN96xL5/thy356K0qqXX9q42Nr2gV3hZW7Go sorjqOCfs4WrS2Kf2146aiyTfnzluV6Ly8tYymOnFq4NYhfnEP3te0ViXUNXG2enzo4P5xf+ 1epx+8ppmMq6YkPBg8rnPUosxRmJhlrMRcWJAJ0CkOFfAwAA X-CMS-MailID: 20210817101747epcas5p1242e63ec29b127b03b6f9f5f1b57f86e X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20210817101747epcas5p1242e63ec29b127b03b6f9f5f1b57f86e References: <20210817101423.12367-1-selvakuma.s1@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Make bio_map_kern() non static, so that copy offload/emulation can use it to add vmalloced memory to bio. Signed-off-by: SelvaKumar S Signed-off-by: Nitesh Shetty --- block/blk-map.c | 2 +- include/linux/blkdev.h | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/block/blk-map.c b/block/blk-map.c index d1448aaad980..1b97b9b503f4 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -336,7 +336,7 @@ static void bio_map_kern_endio(struct bio *bio) * Map the kernel address into a bio suitable for io to a block * device. Returns an error pointer in case of error. */ -static struct bio *bio_map_kern(struct request_queue *q, void *data, +struct bio *bio_map_kern(struct request_queue *q, void *data, unsigned int len, gfp_t gfp_mask) { unsigned long kaddr = (unsigned long)data; diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index df404c1fb087..28a193225cf2 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -903,6 +903,8 @@ extern int blk_rq_map_user(struct request_queue *, struct request *, struct rq_map_data *, void __user *, unsigned long, gfp_t); extern int blk_rq_unmap_user(struct bio *); +struct bio *bio_map_kern(struct request_queue *q, void *data, unsigned int len, + gfp_t gfp_mask); extern int blk_rq_map_kern(struct request_queue *, struct request *, void *, unsigned int, gfp_t); extern int blk_rq_map_user_iov(struct request_queue *, struct request *, struct rq_map_data *, const struct iov_iter *, From patchwork Tue Aug 17 10:14:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SelvaKumar S X-Patchwork-Id: 499350 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB698C4320A for ; Tue, 17 Aug 2021 11:29:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B219860F38 for ; Tue, 17 Aug 2021 11:29:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239726AbhHQL3e (ORCPT ); Tue, 17 Aug 2021 07:29:34 -0400 Received: from mailout4.samsung.com ([203.254.224.34]:23389 "EHLO mailout4.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236541AbhHQL3d (ORCPT ); Tue, 17 Aug 2021 07:29:33 -0400 Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20210817112859epoutp04671cf0e69f0ee30ff6f4670ff39eb692~cFIgDbLuI1039610396epoutp04k for ; Tue, 17 Aug 2021 11:28:59 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20210817112859epoutp04671cf0e69f0ee30ff6f4670ff39eb692~cFIgDbLuI1039610396epoutp04k DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1629199739; bh=x5K+sgPjgv7IAM8fIByO+RHh1QxZ8lh9PFslWCdabrQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lnucnK/WD+UxYyd+UnmujXGy4dut6Y+9JrHxQRnrPogteddz/bx4gPKOujGxlAagG +t30mez1ovx6KgJSZhyColfhan7Wr+CveAfbqyoULowaUFgDDNKAflDI2WAtk9WoUH Cc3TwA0yYpgE2uiBNtH6hXFsTbSleqX/LfDT0fzM= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20210817112858epcas5p22783cb3d20c2c36d7fcc5ff299ae74da~cFIe23gd31218412184epcas5p20; Tue, 17 Aug 2021 11:28:58 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.180]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4Gppgz1k5tz4x9Ps; Tue, 17 Aug 2021 11:28:47 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 41.AD.41701.F6D9B116; Tue, 17 Aug 2021 20:28:47 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20210817101753epcas5p4f4257f8edda27e184ecbb273b700ccbc~cEKavAPR62768027680epcas5p4G; Tue, 17 Aug 2021 10:17:53 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20210817101753epsmtrp2a9f778c4fe57b2289c6f982dd4085289~cEKat0o112821228212epsmtrp2C; Tue, 17 Aug 2021 10:17:53 +0000 (GMT) X-AuditID: b6c32a4b-0abff7000001a2e5-ca-611b9d6f9869 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 92.5E.08394.0DC8B116; Tue, 17 Aug 2021 19:17:53 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20210817101749epsmtip2865161915607c3347d306ff9267e2714~cEKW6oO2d0132601326epsmtip2Z; Tue, 17 Aug 2021 10:17:49 +0000 (GMT) From: SelvaKumar S To: linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: linux-api@vger.kernel.org, linux-scsi@vger.kernel.org, linux-fsdevel@vger.kernel.org, dm-devel@redhat.com, kbusch@kernel.org, axboe@kernel.dk, damien.lemoal@wdc.com, asml.silence@gmail.com, johannes.thumshirn@wdc.com, hch@lst.de, willy@infradead.org, kch@kernel.org, martin.petersen@oracle.com, mpatocka@redhat.com, bvanassche@acm.org, djwong@kernel.org, snitzer@redhat.com, agk@redhat.com, selvajove@gmail.com, joshiiitr@gmail.com, nj.shetty@samsung.com, nitheshshetty@gmail.com, joshi.k@samsung.com, javier.gonz@samsung.com, SelvaKumar S Subject: [PATCH 2/7] block: Introduce queue limits for copy-offload support Date: Tue, 17 Aug 2021 15:44:18 +0530 Message-Id: <20210817101423.12367-3-selvakuma.s1@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210817101423.12367-1-selvakuma.s1@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02Te1BUVRzH59y73N210CuSndYX7oQ8ElhyWQ4FqAPYncCGommUwfAGd4CA 3W0fYVEJJYlrYBCCLBAJhIImw7LgikCxCMiCMbIKI0QxvOIxvANRIttlsfzv8/ud7/d85/zO /Di4zXk2jxMjVjAyMR3HJzawapqcHF0kBdtogWGQiyoMLTjKL68B6Er/OQJlzz7CUcrpJQwZ hzei+uk8K1R2pRlDQ78tsNGq6ncMNT+ZIlDnVBOGMvXdANVpslio6mEqgep7X0F19W0sVFg6 wkZne3QEutT6D4YyUu9jSDecDFDNSiGO7mUV4WhgpoeNJpfbCPR15SJAK8v5xIHtlPFeIHVD 3c+mqi47U8Y7SkpTfoagqkpOUjcfJBHU3Egvi5ppuE9Q6dpyQC1odlKnfzmLBT8fGusdzdCR jMyOEUdIImPEUT78wJBwv3APkcDdxd0LefLtxHQ848P3Dwp2ORQTZxoG3+5jOk5pagXTcjnf zddbJlEqGLtoiVzhw2ekkXFSodRVTsfLleIoVzGjeM1dIHjVwyQ8HhudU/oAl646nxhR12BJ oPFlFeByICmEfbmdQAU2cGzImwCm3FnELMU8gAWnBwmzyoZcMp1M+D515Ksb1kX1AJYlTbIs xQKApzIm1xwE6QK7SzQsM9uS+2HZo3IrswgnDSyYM1PENh9sIQPhr1f7MTOzSHvYVzwAzGxN +sDVrMdWlrhdMLfroUnP4XBJX1g7b2+RbIZtucNr9+MmyVfVebj5fkhWcKFqcGHd6w//NvYA C2+BE61atoV5cGG6nrBwAhw9k41ZOAnA9JkEC++Hd+tWMXMuTjrBilo3S3sHPG+4hllyN8K0 leF1qzXUfT+8JofkHmi4LrK0t8PZptr1JAo2zqevjzoDwMelxVbfAjv1M89RP/Mc9f/JPwC8 HLzESOXxUYzcQ7pPzCT898kRkngNWNsP50AdGByYddUDjAP0AHJwvq21A4dH21hH0p98ysgk 4TJlHCPXAw/TuDNw3gsREtOCiRXh7kIvgVAkEgm99onc+S9ahwVto23IKFrBxDKMlJE99WEc Li8JU7b2poskgrfhuKJOtdch5Y34Wbbjc5UXl+q2bgod6Hr/u7mjo+fGG/RzM2EGbcGPiR07 HBzDDmcnF+0Jmes7rkrEPxRnfuP02ea3srryWphWp3HRl7v4Q/S7vddDrjnaO+S7njzyp6ZR PeZ428H3i45TCStl7wWoIz5vzzzWMlH6Tpqx5/KRi7rCg3jL0amf3lRtSiweGhjL6bDrnBbr V4S83Luhbtrm/LEL7W3VCv+d1VJugm5RXxLA1/jZD1V6lx2Q+C1zZ7O0HdzDly5oPcm0RuOJ IOcyz/gnqcc+GE5me/7x827V3kMRt27l2KZEjXpT4MbW9m6971+34z5qfX331QAXPkseTbs7 4zI5/S/ElVd9qAQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA02Ra0xSYRzGe885HY5O3BGdvaFpw6xFw8ty6y3TtPXhbNXyQ31xbUV4RpYi QWKxmjZnGpplkSVZdnFesKZhESV2wbSoDEu0tEhdOl03pfJSDEu0tr49+/9+e54PfwrntRN8 KlW2n1XIxGkC0pswtghCRS+LgsRRxQMYqn/ahqNyvRGgOscJEpWO/cRRXv4EhjoHfVHz1/Pz UW1dK4Y+vPvOQW7Newy1/v5CItuXFgydsnQDZDZoCdQ4WUCi5t4VyNxsJVBF1RAHFb42kaj6 8TSGSgq6MGQaPAKQ0VWBI7v2Co76R19z0KcpK4mO3hgHyDVVTiYEM532jcwdnYPDNNYImc72 TMagP0YyjZXZTFNPDsk4h3oJZvReF8kU39QD5rshhMl/UIgl+SR7r01h01JVrCIyfqf37rNV PbjcLTwwpDNiOeDhEg3woiAdA8t19zAN8KZ4dBOAfafN8+dAMLzuOEbOZX9YOz3MmZOcAI7m OTgeQNIi2F1pIDw5gE6Ez74ZZ5tw+iMB3zuacA/wpzfCF9ccmCcTdDh8e7UfeDKXjoNu7a+/ a6Gw7NXkTClFedHx8O63cM+ZN6PcN5/hzOl+0Fo2OLuFz+i5t87jJwGt+w/p/kOXAKYHC1m5 Ml2aroyWR8vYrAilOF2ZKZNGSDLSDWD248LlJnBbPxZhARgFLABSuCCAu4zii3ncFPFBNavI 2KHITGOVFhBEEYIF3A6NdQePlor3s3tZVs4q/lGM8uLnYAkjU4UuEX+A/mx9vvJCrCgv9+YD y+P+krstiuitt2qimpU250BbRbzGbghVNa7JnCd89Wtd0XbR5oWfQ2SL9paeq+28/qNvsaRh 9YYXRTH8uOqwa7V+a22l7jTXrrDlleY7yeolZd1T82JNv/ttkUEq7ctzdIp2U+BOqSog92Li gkdhJndf1/qJLdn+9tREv0PVS+tH9nRIWiXq47acJ4rcWN99+zomOc4s53BCwciN43zoMyFK inpanDVsjH2iFmzreRZJ1j86XM9tSPwIzkjtvW1r3sRZJF+rerOniwSjrnWqwJiQVUnB8sBy 38qjLlcDqV76dprHjB+4cvqyoURAKHeLo4W4Qin+A10p3OdgAwAA X-CMS-MailID: 20210817101753epcas5p4f4257f8edda27e184ecbb273b700ccbc X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20210817101753epcas5p4f4257f8edda27e184ecbb273b700ccbc References: <20210817101423.12367-1-selvakuma.s1@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Nitesh Shetty Add device limits as sysfs entries, - copy_offload (READ_WRITE) - max_copy_sectors (READ_ONLY) - max_copy_ranges_sectors (READ_ONLY) - max_copy_nr_ranges (READ_ONLY) copy_offload(= 0), is disabled by default. This needs to be enabled if copy-offload needs to be used. max_copy_sectors = 0, indicates the device doesn't support native copy. Signed-off-by: Nitesh Shetty Signed-off-by: SelvaKumar S Signed-off-by: Kanchan Joshi --- block/blk-settings.c | 4 ++++ block/blk-sysfs.c | 51 ++++++++++++++++++++++++++++++++++++++++++ include/linux/blkdev.h | 6 +++++ 3 files changed, 61 insertions(+) diff --git a/block/blk-settings.c b/block/blk-settings.c index 3613d2cc0688..ac59922e0cde 100644 --- a/block/blk-settings.c +++ b/block/blk-settings.c @@ -57,6 +57,10 @@ void blk_set_default_limits(struct queue_limits *lim) lim->misaligned = 0; lim->zoned = BLK_ZONED_NONE; lim->zone_write_granularity = 0; + lim->copy_offload = 0; + lim->max_copy_sectors = 0; + lim->max_copy_nr_ranges = 0; + lim->max_copy_range_sectors = 0; } EXPORT_SYMBOL(blk_set_default_limits); diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 1832587dce3a..aabab65f10ab 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -169,6 +169,48 @@ static ssize_t queue_discard_granularity_show(struct request_queue *q, char *pag return queue_var_show(q->limits.discard_granularity, page); } +static ssize_t queue_copy_offload_show(struct request_queue *q, char *page) +{ + return queue_var_show(q->limits.copy_offload, page); +} + +static ssize_t queue_copy_offload_store(struct request_queue *q, + const char *page, size_t count) +{ + unsigned long copy_offload; + ssize_t ret = queue_var_store(©_offload, page, count); + + if (ret < 0) + return ret; + + if (copy_offload && q->limits.max_copy_sectors == 0) + return -EINVAL; + + if (copy_offload) + q->limits.copy_offload = BLK_COPY_OFFLOAD_SCC; + else + q->limits.copy_offload = 0; + + return ret; +} + +static ssize_t queue_max_copy_sectors_show(struct request_queue *q, char *page) +{ + return queue_var_show(q->limits.max_copy_sectors, page); +} + +static ssize_t queue_max_copy_range_sectors_show(struct request_queue *q, + char *page) +{ + return queue_var_show(q->limits.max_copy_range_sectors, page); +} + +static ssize_t queue_max_copy_nr_ranges_show(struct request_queue *q, + char *page) +{ + return queue_var_show(q->limits.max_copy_nr_ranges, page); +} + static ssize_t queue_discard_max_hw_show(struct request_queue *q, char *page) { @@ -611,6 +653,11 @@ QUEUE_RO_ENTRY(queue_nr_zones, "nr_zones"); QUEUE_RO_ENTRY(queue_max_open_zones, "max_open_zones"); QUEUE_RO_ENTRY(queue_max_active_zones, "max_active_zones"); +QUEUE_RW_ENTRY(queue_copy_offload, "copy_offload"); +QUEUE_RO_ENTRY(queue_max_copy_sectors, "max_copy_sectors"); +QUEUE_RO_ENTRY(queue_max_copy_range_sectors, "max_copy_range_sectors"); +QUEUE_RO_ENTRY(queue_max_copy_nr_ranges, "max_copy_nr_ranges"); + QUEUE_RW_ENTRY(queue_nomerges, "nomerges"); QUEUE_RW_ENTRY(queue_rq_affinity, "rq_affinity"); QUEUE_RW_ENTRY(queue_poll, "io_poll"); @@ -657,6 +704,10 @@ static struct attribute *queue_attrs[] = { &queue_discard_max_entry.attr, &queue_discard_max_hw_entry.attr, &queue_discard_zeroes_data_entry.attr, + &queue_copy_offload_entry.attr, + &queue_max_copy_sectors_entry.attr, + &queue_max_copy_range_sectors_entry.attr, + &queue_max_copy_nr_ranges_entry.attr, &queue_write_same_max_entry.attr, &queue_write_zeroes_max_entry.attr, &queue_zone_append_max_entry.attr, diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 28a193225cf2..fd4cfaadda5b 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -321,10 +321,14 @@ struct queue_limits { unsigned int discard_granularity; unsigned int discard_alignment; unsigned int zone_write_granularity; + unsigned int copy_offload; + unsigned int max_copy_sectors; unsigned short max_segments; unsigned short max_integrity_segments; unsigned short max_discard_segments; + unsigned short max_copy_range_sectors; + unsigned short max_copy_nr_ranges; unsigned char misaligned; unsigned char discard_misaligned; @@ -600,6 +604,7 @@ struct request_queue { #define QUEUE_FLAG_RQ_ALLOC_TIME 27 /* record rq->alloc_time_ns */ #define QUEUE_FLAG_HCTX_ACTIVE 28 /* at least one blk-mq hctx is active */ #define QUEUE_FLAG_NOWAIT 29 /* device supports NOWAIT */ +#define QUEUE_FLAG_SIMPLE_COPY 30 /* supports simple copy */ #define QUEUE_FLAG_MQ_DEFAULT ((1 << QUEUE_FLAG_IO_STAT) | \ (1 << QUEUE_FLAG_SAME_COMP) | \ @@ -622,6 +627,7 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q); #define blk_queue_io_stat(q) test_bit(QUEUE_FLAG_IO_STAT, &(q)->queue_flags) #define blk_queue_add_random(q) test_bit(QUEUE_FLAG_ADD_RANDOM, &(q)->queue_flags) #define blk_queue_discard(q) test_bit(QUEUE_FLAG_DISCARD, &(q)->queue_flags) +#define blk_queue_copy(q) test_bit(QUEUE_FLAG_SIMPLE_COPY, &(q)->queue_flags) #define blk_queue_zone_resetall(q) \ test_bit(QUEUE_FLAG_ZONE_RESETALL, &(q)->queue_flags) #define blk_queue_secure_erase(q) \ From patchwork Tue Aug 17 10:14:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SelvaKumar S X-Patchwork-Id: 498450 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C9A2C4320A for ; Tue, 17 Aug 2021 11:29:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 33E2B60FC3 for ; Tue, 17 Aug 2021 11:29:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239751AbhHQL3n (ORCPT ); Tue, 17 Aug 2021 07:29:43 -0400 Received: from mailout4.samsung.com ([203.254.224.34]:23485 "EHLO mailout4.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236683AbhHQL3l (ORCPT ); Tue, 17 Aug 2021 07:29:41 -0400 Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20210817112907epoutp04cd619a73989aaa25f8c2c877b6e3dcf7~cFIne7deh0974109741epoutp04E for ; Tue, 17 Aug 2021 11:29:07 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20210817112907epoutp04cd619a73989aaa25f8c2c877b6e3dcf7~cFIne7deh0974109741epoutp04E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1629199747; bh=bJCtbBa6L0JIXNd1NmoxdOdob35u4fPJVTOVrtL8lKw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LLGmUsztrjZH9oTXXVa0czwhxbbU2VsOqT8yo6fHSwl5XWjYP4IywljFRyekVPdaD cde4XeVoP0P5SMKyHVA6qj3NyKbf22cEGbJrSPXjMFxkPF4yZPcVQqSTvJNc/H4Qrp ooicdRtv9w39DCrewfzfQObcUsTvpyJu102DvVNs= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20210817112906epcas5p1a9aafa18c5d6e13a6ce843903ffa7d0f~cFIm3BqwY0478504785epcas5p18; Tue, 17 Aug 2021 11:29:06 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.176]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4GpphF42j2z4x9Pt; Tue, 17 Aug 2021 11:29:01 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id F5.7B.09595.D7D9B116; Tue, 17 Aug 2021 20:29:01 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20210817101758epcas5p1ec353b3838d64654e69488229256d9eb~cEKflp_Yi1790617906epcas5p1I; Tue, 17 Aug 2021 10:17:58 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20210817101758epsmtrp27b65d2a71ea6a8093a18f110e2c8b214~cEKfkTALz2821228212epsmtrp2F; Tue, 17 Aug 2021 10:17:58 +0000 (GMT) X-AuditID: b6c32a4a-eebff7000000257b-fe-611b9d7df523 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id F4.5E.08394.6DC8B116; Tue, 17 Aug 2021 19:17:58 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20210817101754epsmtip2241477dc9632667b7a94aa0f84b4d014~cEKbzz4q70132301323epsmtip2e; Tue, 17 Aug 2021 10:17:54 +0000 (GMT) From: SelvaKumar S To: linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: linux-api@vger.kernel.org, linux-scsi@vger.kernel.org, linux-fsdevel@vger.kernel.org, dm-devel@redhat.com, kbusch@kernel.org, axboe@kernel.dk, damien.lemoal@wdc.com, asml.silence@gmail.com, johannes.thumshirn@wdc.com, hch@lst.de, willy@infradead.org, kch@kernel.org, martin.petersen@oracle.com, mpatocka@redhat.com, bvanassche@acm.org, djwong@kernel.org, snitzer@redhat.com, agk@redhat.com, selvajove@gmail.com, joshiiitr@gmail.com, nj.shetty@samsung.com, nitheshshetty@gmail.com, joshi.k@samsung.com, javier.gonz@samsung.com, SelvaKumar S Subject: [PATCH 3/7] block: copy offload support infrastructure Date: Tue, 17 Aug 2021 15:44:19 +0530 Message-Id: <20210817101423.12367-4-selvakuma.s1@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210817101423.12367-1-selvakuma.s1@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Te0xTZxTnu/dye3FWSmXyDd1k3RgCFiiP8qE85sTlBowhI5Fg1HoD10Io bdMW2NRNDHEqTBClEF7DMB0K23gKCJQRWOWhogPBYGDTUDYfvBmgQ+ZaLm7+9zvn/M75fef3 5VC4MIfnSMUrdaxGyShE5BqivsPVTfxl8UbGq3lpA6rsuYGjovJ6gCpGskiUO/0CRydPLWCo 37QOGSYLrdHVCiOGRofneGg5/TcMGV9NkOjORAeGzrcPAtRSk0Og2sXTJDI8cEcthm4ClXw/ xkMZ9xtJVNb5D4ayTw9gqNF0AqD6pRIc3cspxdHDqfs89Ox5N4m+rp4HaOl5EfnxJrr/Xjh9 vWCER9decaP7byfRNeVnSLr20nG6eSiVpGfGHhD0VOsASWfWlQN6ruY9+lRbBhaxdl9CYBzL xLIaJ1YZo4qNV8qDROGRsp0yP6mXRCwJQP4iJyWTyAaJQndHiD+NV5jNEDklM4okcyqC0WpF nsGBGlWSjnWKU2l1QSJWHatQ+6o9tEyiNkkp91Cyum0SLy9vPzPxUELclOECqS46/nnByCKR Cm4eTgcUBQW+8GybTTpYQwkFzQBmnb/L44JZAMdf9pBcsADgcJoevO4Yuq7i8gYAq29P4Fww B2Bx67R1OrChSIEYDl6qISzYXhACr74ot7aQcEEPAfOmSnmWwnpz4emIAbNgQuAM/+r9ZQXz BUGwbyZtBUPBZpjft8izKNsIgmHTrDNHsYPd+aaV+biZknatcOURUFBpA/X6lyTXGwobFp+t zlkPn3bW8TjsCOcmDaucFPjHmdxVTiqAmVMpHA6Bv7YsYxZdXOAKK5s8ufS7UN/zE8bproNn l0yrrXzY+K0J4wz6CPY0SLn0Jjjd0bSqRMOBugFrzqtsAI11k8Q54FTwxjoFb6xT8L/yRYCX g3dYtTZRzmr91N5KNuW/P45RJdaAlfNwC2sEjx5Oe7QDjALtAFK4yJ7vQjkyQn4s88URVqOS aZIUrLYd+JntzsYd345Rme9LqZNJfAO8fKVSqW+Aj1QicuDv372REQrkjI5NYFk1q3ndh1E2 jqmYd3x+8odbf5+fsQqc6HJocwmU8yuyHvOx7Xrr6Br95s+uVSffaarIay6sij7pT+CVIT8e 1flFRR7bH83e2prctNBqvOsQOBuj6S2+uK/s8BPj8qEfrBpyt1yevmnb3DSxbXy571h78Ac+ tRFHhg4YtdJw6Dva8+pAXhUdlSn7aodi1zdd/e//vFdsq1gQdb51qzbqxMHw3jSXivrM4Sjc J2yPj+fBR0JZ2YXLYmfXPsruRtuWYd2elFH5YyD8O84uQwbHvLTFjiWmqvnIZfs/TW3+iifi Pv+cnbapYXuFVq5r3UujVdXb9evGJgmq1fqT0isxkzs2DFq5d30nDB2P3HXURURo4xiJG67R Mv8CfPz2C6cEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA02RfUgTcRzG+d1dd+dycU3LSyVtJaGmaS/4q1YriLgMoijslfKq0yQ319bM 7G0hla4XTV21OVRCLGcvtOnybRXTLF8qUZutWkUplmXTrKymllOC/vvwPB94vvAlUUEz5ksm SA9wcimbKMR5mLlWGBBmO+vHRuR0ecFbjfUo1BvMAJY6MnF4se8XCk+e/oHAts5J0PIlbwIs KX2AwPevBgg4rH6NwAd/enH4tLcWgdlWG4A1xlwMmgbTcWh5EQprLA0YLCjuIuCZjgocXn04 gsAL6c8QWNF5AkCzqwCF7blXUPjW2UHATz8bcHjq9ncAXT/1+HJ/pq19DVOpcxCM6VoI0/ZY yRgNGThjKjrOVNtVONPf9QJjnHef4cz5MgNgBozTmdP3zyDrPLfyRHu4xIRkTj53WSxvr9OS g8v0x1N0jkFMBZri1IAkaWoBba9MUgMeKaCqAX2yZ4hQA4/R3J++4cjAx9mLLhnpJsalfkBf ttaj7gKnwmhbkRFzsze1gm76akbcEkr1YPRrR/WY5EWJ6R6HBXEzRgXR357UjTGfWkq39qch 4wsBtLZ1kHBf5EEto6u+Brljwahyr0ZDjOuT6QZt59gWOqqnleehWYDS/Vfp/qsKAWIA0ziZ QhIvUUTKIqXcwXAFK1EopfHhu5MkRjD28JDgCnDH0BduBQgJrIAmUaE3fzbpywr4e9hDqZw8 aadcmcgprMCPxIQ+/BZ1w04BFc8e4PZxnIyT/2sR0sNXhbBqp3aosmll2vNVw9aIOKxJMuNy VnlN9EirUrjY73CH7V3gzVTdLGVjVPMSr1MJiWXbFpTnVc2vO1e19ocrdSToY8zvAhHJt9d/ 1AeXdNvDtWIXP1aOFZdlF52YED0xf/3w6kWH0eyg0ptvgpcKNkw17fIMcRbCHR5ldmHOfIFJ /HK4N9bnUstQ+sre8nlSVcH+mUjjwkhz8zGNMaawdY3/Et5G1YXAz/lYnGLwaXC3Zwv/d7LI 9c4mz/igfxQqC9gSetC7Ej3ah96o2zTrCJiYabtTGiVarxXGRr8amLZITKTszxBpNJaI61Pm cOJV5zbbi73rBMX52wM/a9Q8XIgp9rKRIahcwf4FnFbyz18DAAA= X-CMS-MailID: 20210817101758epcas5p1ec353b3838d64654e69488229256d9eb X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20210817101758epcas5p1ec353b3838d64654e69488229256d9eb References: <20210817101423.12367-1-selvakuma.s1@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Nitesh Shetty Introduce REQ_OP_COPY, a no-merge copy offload operation. Create bio with control information as payload and submit to the device. Larger copy operation may be divided if necessary by looking at device limits. REQ_OP_COPY(19) is a write op and takes zone_write_lock when submitted to zoned device. Native copy offload is not supported for stacked devices. Signed-off-by: Nitesh Shetty Signed-off-by: SelvaKumar S --- block/blk-core.c | 84 ++++++++++++- block/blk-lib.c | 252 ++++++++++++++++++++++++++++++++++++++ block/blk-zoned.c | 1 + block/bounce.c | 1 + include/linux/bio.h | 1 + include/linux/blk_types.h | 20 +++ include/linux/blkdev.h | 13 ++ include/uapi/linux/fs.h | 12 ++ 8 files changed, 378 insertions(+), 6 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index d2722ecd4d9b..541b1561b4af 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -704,6 +704,17 @@ static noinline int should_fail_bio(struct bio *bio) } ALLOW_ERROR_INJECTION(should_fail_bio, ERRNO); +static inline int bio_check_copy_eod(struct bio *bio, sector_t start, + sector_t nr_sectors, sector_t max_sect) +{ + if (nr_sectors && max_sect && + (nr_sectors > max_sect || start > max_sect - nr_sectors)) { + handle_bad_sector(bio, max_sect); + return -EIO; + } + return 0; +} + /* * Check whether this bio extends beyond the end of the device or partition. * This may well happen - the kernel calls bread() without checking the size of @@ -723,6 +734,61 @@ static inline int bio_check_eod(struct bio *bio) return 0; } +/* + * check for eod limits and remap ranges if needed + */ +static int blk_check_copy(struct bio *bio) +{ + struct blk_copy_payload *payload = bio_data(bio); + sector_t dst_max_sect, dst_start_sect, copy_size = 0; + sector_t src_max_sect, src_start_sect; + struct block_device *bd_part; + int i, ret = -EIO; + + rcu_read_lock(); + + bd_part = bio->bi_bdev; + if (unlikely(!bd_part)) + goto err; + + dst_max_sect = bdev_nr_sectors(bd_part); + dst_start_sect = bd_part->bd_start_sect; + + src_max_sect = bdev_nr_sectors(payload->src_bdev); + src_start_sect = payload->src_bdev->bd_start_sect; + + if (unlikely(should_fail_request(bd_part, bio->bi_iter.bi_size))) + goto err; + + if (unlikely(bio_check_ro(bio))) + goto err; + + rcu_read_unlock(); + + for (i = 0; i < payload->copy_nr_ranges; i++) { + ret = bio_check_copy_eod(bio, payload->range[i].src, + payload->range[i].len, src_max_sect); + if (unlikely(ret)) + goto out; + + payload->range[i].src += src_start_sect; + copy_size += payload->range[i].len; + } + + /* check if copy length crosses eod */ + ret = bio_check_copy_eod(bio, bio->bi_iter.bi_sector, + copy_size, dst_max_sect); + if (unlikely(ret)) + goto out; + + bio->bi_iter.bi_sector += dst_start_sect; + return 0; +err: + rcu_read_unlock(); +out: + return ret; +} + /* * Remap block n of partition p to block n+start(p) of the disk. */ @@ -799,13 +865,15 @@ static noinline_for_stack bool submit_bio_checks(struct bio *bio) if (should_fail_bio(bio)) goto end_io; - if (unlikely(bio_check_ro(bio))) - goto end_io; - if (!bio_flagged(bio, BIO_REMAPPED)) { - if (unlikely(bio_check_eod(bio))) - goto end_io; - if (bdev->bd_partno && unlikely(blk_partition_remap(bio))) + if (likely(!op_is_copy(bio->bi_opf))) { + if (unlikely(bio_check_ro(bio))) goto end_io; + if (!bio_flagged(bio, BIO_REMAPPED)) { + if (unlikely(bio_check_eod(bio))) + goto end_io; + if (bdev->bd_partno && unlikely(blk_partition_remap(bio))) + goto end_io; + } } /* @@ -829,6 +897,10 @@ static noinline_for_stack bool submit_bio_checks(struct bio *bio) if (!blk_queue_discard(q)) goto not_supported; break; + case REQ_OP_COPY: + if (unlikely(blk_check_copy(bio))) + goto end_io; + break; case REQ_OP_SECURE_ERASE: if (!blk_queue_secure_erase(q)) goto not_supported; diff --git a/block/blk-lib.c b/block/blk-lib.c index 9f09beadcbe3..7fee0ae95c44 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -151,6 +151,258 @@ int blkdev_issue_discard(struct block_device *bdev, sector_t sector, } EXPORT_SYMBOL(blkdev_issue_discard); +/* + * Wait on and process all in-flight BIOs. This must only be called once + * all bios have been issued so that the refcount can only decrease. + * This just waits for all bios to make it through cio_bio_end_io. IO + * errors are propagated through cio->io_error. + */ +static int cio_await_completion(struct cio *cio) +{ + int ret = 0; + + while (atomic_read(&cio->refcount)) { + cio->waiter = current; + __set_current_state(TASK_UNINTERRUPTIBLE); + blk_io_schedule(); + /* wake up sets us TASK_RUNNING */ + cio->waiter = NULL; + ret = cio->io_err; + } + kvfree(cio); + + return ret; +} + +/* + * The BIO completion handler simply decrements refcount. + * Also wake up process, if this is the last bio to be completed. + * + * During I/O bi_private points at the cio. + */ +static void cio_bio_end_io(struct bio *bio) +{ + struct cio *cio = bio->bi_private; + + if (bio->bi_status) + cio->io_err = bio->bi_status; + kvfree(page_address(bio_first_bvec_all(bio)->bv_page) + + bio_first_bvec_all(bio)->bv_offset); + bio_put(bio); + + if (atomic_dec_and_test(&cio->refcount) && cio->waiter) + wake_up_process(cio->waiter); +} + +int blk_copy_offload_submit_bio(struct block_device *bdev, + struct blk_copy_payload *payload, int payload_size, + struct cio *cio, gfp_t gfp_mask) +{ + struct request_queue *q = bdev_get_queue(bdev); + struct bio *bio; + + bio = bio_map_kern(q, payload, payload_size, gfp_mask); + if (IS_ERR(bio)) + return PTR_ERR(bio); + + bio_set_dev(bio, bdev); + bio->bi_opf = REQ_OP_COPY | REQ_NOMERGE; + bio->bi_iter.bi_sector = payload->dest; + bio->bi_end_io = cio_bio_end_io; + bio->bi_private = cio; + atomic_inc(&cio->refcount); + submit_bio(bio); + + return 0; +} + +/* Go through all the enrties inside user provided payload, and determine the + * maximum number of entries in a payload, based on device's scc-limits. + */ +static inline int blk_max_payload_entries(int nr_srcs, struct range_entry *rlist, + int max_nr_srcs, sector_t max_copy_range_sectors, sector_t max_copy_len) +{ + sector_t range_len, copy_len = 0, remaining = 0; + int ri = 0, pi = 1, max_pi = 0; + + for (ri = 0; ri < nr_srcs; ri++) { + for (remaining = rlist[ri].len; remaining > 0; remaining -= range_len) { + range_len = min3(remaining, max_copy_range_sectors, + max_copy_len - copy_len); + pi++; + copy_len += range_len; + + if ((pi == max_nr_srcs) || (copy_len == max_copy_len)) { + max_pi = max(max_pi, pi); + pi = 1; + copy_len = 0; + } + } + } + + return max(max_pi, pi); +} + +/* + * blk_copy_offload_scc - Use device's native copy offload feature + * Go through user provide payload, prepare new payload based on device's copy offload limits. + */ +int blk_copy_offload_scc(struct block_device *src_bdev, int nr_srcs, + struct range_entry *rlist, struct block_device *dest_bdev, + sector_t dest, gfp_t gfp_mask) +{ + struct request_queue *q = bdev_get_queue(dest_bdev); + struct cio *cio = NULL; + struct blk_copy_payload *payload; + sector_t range_len, copy_len = 0, remaining = 0; + sector_t src_blk, cdest = dest; + sector_t max_copy_range_sectors, max_copy_len; + int ri = 0, pi = 0, ret = 0, payload_size, max_pi, max_nr_srcs; + + cio = kzalloc(sizeof(struct cio), GFP_KERNEL); + if (!cio) + return -ENOMEM; + atomic_set(&cio->refcount, 0); + + max_nr_srcs = q->limits.max_copy_nr_ranges; + max_copy_range_sectors = q->limits.max_copy_range_sectors; + max_copy_len = q->limits.max_copy_sectors; + + max_pi = blk_max_payload_entries(nr_srcs, rlist, max_nr_srcs, + max_copy_range_sectors, max_copy_len); + payload_size = struct_size(payload, range, max_pi); + + payload = kvmalloc(payload_size, gfp_mask); + if (!payload) { + ret = -ENOMEM; + goto free_cio; + } + payload->src_bdev = src_bdev; + + for (ri = 0; ri < nr_srcs; ri++) { + for (remaining = rlist[ri].len, src_blk = rlist[ri].src; remaining > 0; + remaining -= range_len, src_blk += range_len) { + + range_len = min3(remaining, max_copy_range_sectors, + max_copy_len - copy_len); + payload->range[pi].len = range_len; + payload->range[pi].src = src_blk; + pi++; + copy_len += range_len; + + /* Submit current payload, if crossing device copy limits */ + if ((pi == max_nr_srcs) || (copy_len == max_copy_len)) { + payload->dest = cdest; + payload->copy_nr_ranges = pi; + ret = blk_copy_offload_submit_bio(dest_bdev, payload, + payload_size, cio, gfp_mask); + if (ret) + goto free_payload; + + /* reset index, length and allocate new payload */ + pi = 0; + cdest += copy_len; + copy_len = 0; + payload = kvmalloc(payload_size, gfp_mask); + if (!payload) { + ret = -ENOMEM; + goto free_cio; + } + payload->src_bdev = src_bdev; + } + } + } + + if (pi) { + payload->dest = cdest; + payload->copy_nr_ranges = pi; + ret = blk_copy_offload_submit_bio(dest_bdev, payload, payload_size, cio, gfp_mask); + if (ret) + goto free_payload; + } + + /* Wait for completion of all IO's*/ + ret = cio_await_completion(cio); + + return ret; + +free_payload: + kvfree(payload); +free_cio: + cio_await_completion(cio); + return ret; +} + +static inline sector_t blk_copy_len(struct range_entry *rlist, int nr_srcs) +{ + int i; + sector_t len = 0; + + for (i = 0; i < nr_srcs; i++) { + if (rlist[i].len) + len += rlist[i].len; + else + return 0; + } + + return len; +} + +static inline bool blk_check_offload_scc(struct request_queue *src_q, + struct request_queue *dest_q) +{ + if (src_q == dest_q && src_q->limits.copy_offload == BLK_COPY_OFFLOAD_SCC) + return true; + + return false; +} + +/* + * blkdev_issue_copy - queue a copy + * @src_bdev: source block device + * @nr_srcs: number of source ranges to copy + * @src_rlist: array of source ranges + * @dest_bdev: destination block device + * @dest: destination in sector + * @gfp_mask: memory allocation flags (for bio_alloc) + * @flags: BLKDEV_COPY_* flags to control behaviour + * + * Description: + * Copy source ranges from source block device to destination block device. + * length of a source range cannot be zero. + */ +int blkdev_issue_copy(struct block_device *src_bdev, int nr_srcs, + struct range_entry *src_rlist, struct block_device *dest_bdev, + sector_t dest, gfp_t gfp_mask, int flags) +{ + struct request_queue *src_q = bdev_get_queue(src_bdev); + struct request_queue *dest_q = bdev_get_queue(dest_bdev); + sector_t copy_len; + int ret = -EINVAL; + + if (!src_q || !dest_q) + return -ENXIO; + + if (!nr_srcs) + return -EINVAL; + + if (nr_srcs >= MAX_COPY_NR_RANGE) + return -EINVAL; + + copy_len = blk_copy_len(src_rlist, nr_srcs); + if (!copy_len && copy_len >= MAX_COPY_TOTAL_LENGTH) + return -EINVAL; + + if (bdev_read_only(dest_bdev)) + return -EPERM; + + if (blk_check_offload_scc(src_q, dest_q)) + ret = blk_copy_offload_scc(src_bdev, nr_srcs, src_rlist, dest_bdev, dest, gfp_mask); + + return ret; +} +EXPORT_SYMBOL(blkdev_issue_copy); + /** * __blkdev_issue_write_same - generate number of bios with same page * @bdev: target blockdev diff --git a/block/blk-zoned.c b/block/blk-zoned.c index 86fce751bb17..7643fc868521 100644 --- a/block/blk-zoned.c +++ b/block/blk-zoned.c @@ -67,6 +67,7 @@ bool blk_req_needs_zone_write_lock(struct request *rq) case REQ_OP_WRITE_ZEROES: case REQ_OP_WRITE_SAME: case REQ_OP_WRITE: + case REQ_OP_COPY: return blk_rq_zone_is_seq(rq); default: return false; diff --git a/block/bounce.c b/block/bounce.c index 05fc7148489d..d9b05aaf6e56 100644 --- a/block/bounce.c +++ b/block/bounce.c @@ -176,6 +176,7 @@ static struct bio *bounce_clone_bio(struct bio *bio_src) bio->bi_iter.bi_size = bio_src->bi_iter.bi_size; switch (bio_op(bio)) { + case REQ_OP_COPY: case REQ_OP_DISCARD: case REQ_OP_SECURE_ERASE: case REQ_OP_WRITE_ZEROES: diff --git a/include/linux/bio.h b/include/linux/bio.h index 3d67d0fbc868..068fa2e8896a 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -73,6 +73,7 @@ static inline bool bio_has_data(struct bio *bio) static inline bool bio_no_advance_iter(const struct bio *bio) { return bio_op(bio) == REQ_OP_DISCARD || + bio_op(bio) == REQ_OP_COPY || bio_op(bio) == REQ_OP_SECURE_ERASE || bio_op(bio) == REQ_OP_WRITE_SAME || bio_op(bio) == REQ_OP_WRITE_ZEROES; diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 9e392daa1d7f..1ab77176cb46 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -347,6 +347,8 @@ enum req_opf { REQ_OP_ZONE_RESET = 15, /* reset all the zone present on the device */ REQ_OP_ZONE_RESET_ALL = 17, + /* copy ranges within device */ + REQ_OP_COPY = 19, /* Driver private requests */ REQ_OP_DRV_IN = 34, @@ -470,6 +472,11 @@ static inline bool op_is_discard(unsigned int op) return (op & REQ_OP_MASK) == REQ_OP_DISCARD; } +static inline bool op_is_copy(unsigned int op) +{ + return (op & REQ_OP_MASK) == REQ_OP_COPY; +} + /* * Check if a bio or request operation is a zone management operation, with * the exception of REQ_OP_ZONE_RESET_ALL which is treated as a special case @@ -529,4 +536,17 @@ struct blk_rq_stat { u64 batch; }; +struct cio { + atomic_t refcount; + blk_status_t io_err; + struct task_struct *waiter; /* waiting task (NULL if none) */ +}; + +struct blk_copy_payload { + struct block_device *src_bdev; + sector_t dest; + int copy_nr_ranges; + struct range_entry range[]; +}; + #endif /* __LINUX_BLK_TYPES_H */ diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index fd4cfaadda5b..38369dff6a36 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -52,6 +52,12 @@ struct blk_keyslot_manager; /* Doing classic polling */ #define BLK_MQ_POLL_CLASSIC -1 +/* Define copy offload options */ +enum blk_copy { + BLK_COPY_OFFLOAD_EMULATE = 0, + BLK_COPY_OFFLOAD_SCC, +}; + /* * Maximum number of blkcg policies allowed to be registered concurrently. * Defined here to simplify include dependency. @@ -1051,6 +1057,9 @@ static inline unsigned int blk_queue_get_max_sectors(struct request_queue *q, return min(q->limits.max_discard_sectors, UINT_MAX >> SECTOR_SHIFT); + if (unlikely(op == REQ_OP_COPY)) + return q->limits.max_copy_sectors; + if (unlikely(op == REQ_OP_WRITE_SAME)) return q->limits.max_write_same_sectors; @@ -1326,6 +1335,10 @@ extern int __blkdev_issue_discard(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, int flags, struct bio **biop); +int blkdev_issue_copy(struct block_device *src_bdev, int nr_srcs, + struct range_entry *src_rlist, struct block_device *dest_bdev, + sector_t dest, gfp_t gfp_mask, int flags); + #define BLKDEV_ZERO_NOUNMAP (1 << 0) /* do not free blocks */ #define BLKDEV_ZERO_NOFALLBACK (1 << 1) /* don't write explicit zeroes */ diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index bdf7b404b3e7..7a97b588d892 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -64,6 +64,18 @@ struct fstrim_range { __u64 minlen; }; +/* Maximum no of entries supported */ +#define MAX_COPY_NR_RANGE (1 << 12) + +/* maximum total copy length */ +#define MAX_COPY_TOTAL_LENGTH (1 << 21) + +/* Source range entry for copy */ +struct range_entry { + __u64 src; + __u64 len; +}; + /* extent-same (dedupe) ioctls; these MUST match the btrfs ioctl definitions */ #define FILE_DEDUPE_RANGE_SAME 0 #define FILE_DEDUPE_RANGE_DIFFERS 1 From patchwork Tue Aug 17 10:14:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SelvaKumar S X-Patchwork-Id: 499349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE535C432BE for ; Tue, 17 Aug 2021 11:29:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C7BC760EBD for ; Tue, 17 Aug 2021 11:29:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239723AbhHQL35 (ORCPT ); Tue, 17 Aug 2021 07:29:57 -0400 Received: from mailout1.samsung.com ([203.254.224.24]:41948 "EHLO mailout1.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239775AbhHQL3w (ORCPT ); Tue, 17 Aug 2021 07:29:52 -0400 Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20210817112918epoutp014fdcda28964444dcc1a076cae0d4f211~cFIxOjiKn2795827958epoutp01l for ; Tue, 17 Aug 2021 11:29:18 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20210817112918epoutp014fdcda28964444dcc1a076cae0d4f211~cFIxOjiKn2795827958epoutp01l DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1629199758; bh=8k4v8Wc22T8Nl4fGkl2T3OtUvUqtjnmyhgGSjr7Uuqs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UXALpCP40dUOXWiE9lkcxp2jMrUYb0j8TfVy6eK0ti6NKD7ST/PWccaB7mOrH4Aev fYwj5EVop2J2XX8L0gHkV1d+MVYkaoruIwNb6VZQ6OIz6I9pdGNk2wong9H0f+NIh1 NvxoUEwHQcc9CwJVwaD2E9GEW8kTzLSuwE80IRdo= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20210817112917epcas5p17eb00a14b2416924d18537aae51021fe~cFIwQ4GbS0075100751epcas5p1V; Tue, 17 Aug 2021 11:29:17 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.175]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4GpphP5ngcz4x9Pt; Tue, 17 Aug 2021 11:29:09 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 7F.AD.41701.58D9B116; Tue, 17 Aug 2021 20:29:09 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20210817101803epcas5p10cda1d52f8a8f1172e34b1f9cf8eef3b~cEKkbCg8f0725807258epcas5p1E; Tue, 17 Aug 2021 10:18:03 +0000 (GMT) Received: from epsmgms1p2.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20210817101803epsmtrp21b0f42eda6731a61723b78dc17c2ae6c~cEKkZtkPw2821228212epsmtrp2R; Tue, 17 Aug 2021 10:18:03 +0000 (GMT) X-AuditID: b6c32a4b-0c1ff7000001a2e5-09-611b9d85ad8f Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2.samsung.com (Symantec Messaging Gateway) with SMTP id 13.19.32548.BDC8B116; Tue, 17 Aug 2021 19:18:03 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20210817101759epsmtip29d696043a21492e64b762aeb5d3a4451~cEKgr5htn3169331693epsmtip2i; Tue, 17 Aug 2021 10:17:59 +0000 (GMT) From: SelvaKumar S To: linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: linux-api@vger.kernel.org, linux-scsi@vger.kernel.org, linux-fsdevel@vger.kernel.org, dm-devel@redhat.com, kbusch@kernel.org, axboe@kernel.dk, damien.lemoal@wdc.com, asml.silence@gmail.com, johannes.thumshirn@wdc.com, hch@lst.de, willy@infradead.org, kch@kernel.org, martin.petersen@oracle.com, mpatocka@redhat.com, bvanassche@acm.org, djwong@kernel.org, snitzer@redhat.com, agk@redhat.com, selvajove@gmail.com, joshiiitr@gmail.com, nj.shetty@samsung.com, nitheshshetty@gmail.com, joshi.k@samsung.com, javier.gonz@samsung.com, SelvaKumar S Subject: [PATCH 4/7] block: Introduce a new ioctl for simple copy Date: Tue, 17 Aug 2021 15:44:20 +0530 Message-Id: <20210817101423.12367-5-selvakuma.s1@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210817101423.12367-1-selvakuma.s1@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02Te1BUVRzHO/cuu4sK3gDhuGBuO6IB8VhYlkMjoAPT3AGmoXFyDBzpCjdg YB/uI6ImoxzUhdpEBRMfKGrlLiEtD9flEfIStihey2vUrGDHRAHZVDZAW7hY/vc5v/P9nu/8 fmd+XNztGIfHzZKqaIWUyhGwV7Hq2/y2BBac8aZCFtpfR1fMnTg6rasHSH/rKzYqnbHjqODQ YwwNjLuipqlTTuiyvgNDf960cdBi4W0MdTx7wEa/PmjD0NHWIYAaDcdZqObJYTZqGgtAjU3d LFT+zQQHFQ0b2ejbG08xVHzYgiHj+GcA1c+X42jweAWO7kwPc9DkXDcbHfzhEUDzc6fZ23zI gcEE8lrZLQ5Z850/OdCjJg06DZusufgp2TCazyYfToyxyOlmC5vU1uoAaTO8Qh5qKcKS1iRn b82kqXRawaelabL0LGlGlCBhR2psarg4RBgojEQRAr6UktBRgrjEpMA3s3IcwxDwP6By1I5S EqVUCoKjtypkahXNz5QpVVECWp6eIxfJg5SURKmWZgRJadUbwpCQ0HCH8L3sTNuUiSPv8Pnw 7plqp3xQ7FUInLmQEMHK6xbOErsRDQBWmCSFYJWDZwGc7KljMQcbgFMXHmLPHfnWpxzmwgSg ZrEIMHaHqndw/xKziUA4dNHAWmIPIgZetuuclgw4YWbBE9MVy3nuxHb4+8IwvsQswhdebSlY TnAhouCF9p8Bk7YRnux/4tBzuc5ENDTN+jKSl2H3yfHl93GH5EDdKZyR651h82gew3HwftVB FsPu8N6NWg7DPGibamIznAutmtKVxvIB1E7nMhwD+xoXsaVYnPCDV0zBTHkDLDFXYUysK/xy fnzF6gKNZ8eX5ZDYDM1XxUzZB860mVaSSDjS0o8zYysGcHD2C+wI4Je90E3ZC92U/Z98DuA6 sJ6WKyUZtDJcHialc//74jSZxACWt8M/wQj+uDMT1AowLmgFkIsLPFy2cHmUm0s6lfcRrZCl KtQ5tLIVhDumXYzz1qXJHOslVaUKRZEhIrFYLIoMEwsFXi67E70pNyKDUtHZNC2nFc99GNeZ l481GakMPq/sbLyXa1sVzzfDMz5lz8i+ZHrPP276r0tDU1bXqC32Ttv5dzbsOvKbltpxLu7R q31rxtJ3H622l+xUUx/fuy28Vnle27PToEm2b/cLnay5L+o8ILZtI3y6PAMiYsP+TvNXlpd4 yzb5di3UJZaG9tb2D82IgtcWpLzU9Ut9Xexe97clDTr32nU/ip07qJH3975F/iT6vCq7MTUw D67v9oyJTzjht+lxdIFGUhit6ewdiN8VtnHf/k7tXau1SJQZ0Kf9C7ZsG5gyV+jnI4hI62vv BrpYJi9ZNhvXzo16CFzbrPrvmy/d/GR1EXZdntZ5jENWTdgtlc/a+Qnm9moBS5lJCf1xhZL6 F1yxgWOmBAAA X-Brightmail-Tracker: H4sIAAAAAAAAA02RfyyUcRzH932ep+eeu3KeLuYJy7qxdBdhqW+/LP+0p+nX+itaXVeesNyx O6e0rGtFRz+Jyo/KHOGMdMLFifyonZC4Y5WzcMoyxFWYH+Worf/e+7xe773/+BAorw1zJiKk MYxMKo7k4xysspHv5vXphovYx9DKh09bXqMwW1MJYLH5Ng7vfZ9BYcK1XwjssnBh7VjWClhU 3IzAwV4rC84n9yGw+fcoDt+NNiIwtaEbQL02DYPlUyoc1n4UQn2tAYOPnwyx4PUeHQ4L3iwg MEVlQqDOchnAytnHKDSm5aLw83gPC45MG3CY+OwngLPT2fgeV7rLGES/yDSz6PJCAd3VpqC1 miScLs+7RNd8UOL0xNBHjB5/acLpW881gLZq19HX6q8jh1eFcHaFMpERsYxsc8BJTrh1rJoV 3ex6fvhh2QolSHFKBmyCIrdQyi8LrGTAIXikDlCTuVNgGbhSJeYkfDmvoYoWvv6VJgD142U6 YgM46UV152kxW3YgA6m3k5WITULJbxjVZ65BbWDNIuif61nKGOlBVdUnLJXtyN2Uuqn175ob ldE5tbhAEGwygKqe9LCdeYtKnT6dtayvpgwZlqUtdFG/UpGF3gFk5n8o8z+UAxANWMtEyyVh ErlvtJ+UOectF0vkCmmY9+koiRYsfVwg0AG95rt3A0AI0AAoAuU72G0gnMU8u1Bx3AVGFiWS KSIZeQNwITC+k11HskHEI8PEMcxZholmZP8oQrCdlUhpun9xwaib28X5uqMOG/26MwNFE575 MSp7i3GAc27I4dinXZbJ9e5hg68KB3rt4+mSVH2OsJRfeFPUJG1U55g2HRreGhxy4lRTZ1nv Nv8MlSKg6nPsfqFz7MrQB62qkCLli5mgyPA0/nvTprl8r+BuoT7uZ6Dr9v7dNzz3cZEfs6H3 z2QNbE1lP5TknRG8iZi3+qUlBhvDSbL9+E5RQszdqiOJpS4+/lfbd16agf7WHT31e4bi+6QF 2mBh4oGkEOtmNbd6YEFXpx7X1HuPO6kddfa93O26oL7L1R3smvuPUlrHcke4a81Zporpg3vj vFrm1F8Ud9son7NRjsaCQXc+Jg8X+wpQmVz8B1BWO4dgAwAA X-CMS-MailID: 20210817101803epcas5p10cda1d52f8a8f1172e34b1f9cf8eef3b X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20210817101803epcas5p10cda1d52f8a8f1172e34b1f9cf8eef3b References: <20210817101423.12367-1-selvakuma.s1@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Nitesh Shetty Add new BLKCOPY ioctl that offloads copying of one or more sources ranges to a destination in the device. COPY ioctl accepts a 'copy_range' structure that contains destination (in sectors), no of sources and pointer to the array of source ranges. Each source range is represented by 'range_entry' that contains start and length of source ranges (in sectors) MAX_COPY_NR_RANGE, limits the number of entries for the IOCTL and MAX_COPY_TOTAL_LENGTH limits the total copy length, IOCTL can handle. Example code, to issue BLKCOPY: /* Sample example to copy three source-ranges [0, 8] [16, 8] [32,8] to * [64,24], on the same device */ int main(void) { int ret, fd; struct range_entry source_range[] = {{.src = 0, .len = 8}, {.src = 16, .len = 8}, {.src = 32, .len = 8},}; struct copy_range cr; cr.dest = 64; cr.nr_range = 3; cr.range_list = (__u64)&source_range; fd = open("/dev/nvme0n1", O_RDWR); if (fd < 0) return 1; ret = ioctl(fd, BLKCOPY, &cr); if (ret < 0) printf("copy failure\n"); close(fd); return ret; } Signed-off-by: Nitesh Shetty Signed-off-by: SelvaKumar S Signed-off-by: Kanchan Joshi --- block/ioctl.c | 33 +++++++++++++++++++++++++++++++++ include/uapi/linux/fs.h | 8 ++++++++ 2 files changed, 41 insertions(+) diff --git a/block/ioctl.c b/block/ioctl.c index eb0491e90b9a..2af56d01e9fe 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -143,6 +143,37 @@ static int blk_ioctl_discard(struct block_device *bdev, fmode_t mode, GFP_KERNEL, flags); } +static int blk_ioctl_copy(struct block_device *bdev, fmode_t mode, + unsigned long arg) +{ + struct copy_range crange; + struct range_entry *rlist; + int ret; + + if (!(mode & FMODE_WRITE)) + return -EBADF; + + if (copy_from_user(&crange, (void __user *)arg, sizeof(crange))) + return -EFAULT; + + rlist = kmalloc_array(crange.nr_range, sizeof(*rlist), + GFP_KERNEL); + if (!rlist) + return -ENOMEM; + + if (copy_from_user(rlist, (void __user *)crange.range_list, + sizeof(*rlist) * crange.nr_range)) { + ret = -EFAULT; + goto out; + } + + ret = blkdev_issue_copy(bdev, crange.nr_range, rlist, bdev, crange.dest, + GFP_KERNEL, 0); +out: + kfree(rlist); + return ret; +} + static int blk_ioctl_zeroout(struct block_device *bdev, fmode_t mode, unsigned long arg) { @@ -468,6 +499,8 @@ static int blkdev_common_ioctl(struct block_device *bdev, fmode_t mode, case BLKSECDISCARD: return blk_ioctl_discard(bdev, mode, arg, BLKDEV_DISCARD_SECURE); + case BLKCOPY: + return blk_ioctl_copy(bdev, mode, arg); case BLKZEROOUT: return blk_ioctl_zeroout(bdev, mode, arg); case BLKGETDISKSEQ: diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 7a97b588d892..4183688ff398 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -76,6 +76,13 @@ struct range_entry { __u64 len; }; +struct copy_range { + __u64 dest; + __u64 nr_range; + __u64 range_list; + __u64 rsvd; +}; + /* extent-same (dedupe) ioctls; these MUST match the btrfs ioctl definitions */ #define FILE_DEDUPE_RANGE_SAME 0 #define FILE_DEDUPE_RANGE_DIFFERS 1 @@ -197,6 +204,7 @@ struct fsxattr { #define BLKROTATIONAL _IO(0x12,126) #define BLKZEROOUT _IO(0x12,127) #define BLKGETDISKSEQ _IOR(0x12,128,__u64) +#define BLKCOPY _IOWR(0x12, 129, struct copy_range) /* * A jump here: 130-136 are reserved for zoned block devices * (see uapi/linux/blkzoned.h) From patchwork Tue Aug 17 10:14:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SelvaKumar S X-Patchwork-Id: 498449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12DF1C432BE for ; Tue, 17 Aug 2021 11:29:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EDA0360F38 for ; Tue, 17 Aug 2021 11:29:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239775AbhHQLaB (ORCPT ); Tue, 17 Aug 2021 07:30:01 -0400 Received: from mailout3.samsung.com ([203.254.224.33]:27897 "EHLO mailout3.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239743AbhHQL37 (ORCPT ); Tue, 17 Aug 2021 07:29:59 -0400 Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20210817112925epoutp031dfbd7a113e017363f63de6c939e09d1~cFI3_ZFN-2784027840epoutp03C for ; Tue, 17 Aug 2021 11:29:25 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20210817112925epoutp031dfbd7a113e017363f63de6c939e09d1~cFI3_ZFN-2784027840epoutp03C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1629199765; bh=wc+WvxlrxFo6cfYgE96D3st/XNkIhsyXUXNjiBbfxn8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZedZ/tENruX7+OWaROCLdRskFtf7AvaHUPKhTlj+Qq+OpWzSOHsRsoTRLs9nyOG8T hrjDl8eLivDlJ6ryhH+61fh5zkyxG2IwbixMPLjjxyT2fEMw4tDcs+RL7jXT2vL/ch O3iPIR7TUzxh70VB+eH4zUWyrPp7fH7CZgxSTNLw= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20210817112924epcas5p34748629b4eefb2a5e4447803ad88bac6~cFI3VRqJF0728107281epcas5p34; Tue, 17 Aug 2021 11:29:24 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.180]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4GpphX07Jgz4x9Q3; Tue, 17 Aug 2021 11:29:16 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id D4.3C.40257.B8D9B116; Tue, 17 Aug 2021 20:29:15 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20210817101809epcas5p39eed3531ed82f5f08127eb3dba1fc50f~cEKpkFYrp2834328343epcas5p35; Tue, 17 Aug 2021 10:18:09 +0000 (GMT) Received: from epsmgms1p2.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20210817101809epsmtrp12dea27ad71f675f5748808fd117e1e45~cEKpiyntr2042320423epsmtrp1G; Tue, 17 Aug 2021 10:18:09 +0000 (GMT) X-AuditID: b6c32a49-ed1ff70000019d41-55-611b9d8bb635 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2.samsung.com (Symantec Messaging Gateway) with SMTP id 58.19.32548.0EC8B116; Tue, 17 Aug 2021 19:18:09 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20210817101805epsmtip21f597ab79bcdfc0f01aa56cbc0e830ab~cEKl0OOkM0079500795epsmtip2j; Tue, 17 Aug 2021 10:18:05 +0000 (GMT) From: SelvaKumar S To: linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: linux-api@vger.kernel.org, linux-scsi@vger.kernel.org, linux-fsdevel@vger.kernel.org, dm-devel@redhat.com, kbusch@kernel.org, axboe@kernel.dk, damien.lemoal@wdc.com, asml.silence@gmail.com, johannes.thumshirn@wdc.com, hch@lst.de, willy@infradead.org, kch@kernel.org, martin.petersen@oracle.com, mpatocka@redhat.com, bvanassche@acm.org, djwong@kernel.org, snitzer@redhat.com, agk@redhat.com, selvajove@gmail.com, joshiiitr@gmail.com, nj.shetty@samsung.com, nitheshshetty@gmail.com, joshi.k@samsung.com, javier.gonz@samsung.com, SelvaKumar S Subject: [PATCH 5/7] block: add emulation for simple copy Date: Tue, 17 Aug 2021 15:44:21 +0530 Message-Id: <20210817101423.12367-6-selvakuma.s1@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210817101423.12367-1-selvakuma.s1@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Tf1CTZRy/533Hu+Ft+AZ6PAyq3cAMCtiKrQcDzKB8O72OhM6TTHwP3gOO sa1tiFhdmMoPSQRElIGIQCijIgYRDLBugPLLwwQUEQJ0qFnIlOPXAdHG0Pzv8/1+P5/nc9/P c18O7pjP5nPi5BpGJadlQmIdq77V83XvzHOutOhErgRVd13BUZGuHqCqkZMEyjcv4OhY2iyG +kwOqOVxoR2qrGrH0L3haTZaPv4nhtpXJgnUO9mKoVzjTYCa9XksVDuXTqCWoTdQc0snC52v mGCjzFsNBLp49V8M5aQPYKjBdBig+sXzOOrPK8XR2NQtNvp7vpNAqTUzAC3OFxHvuVF9/Tuo Ru0Im6q95EX1XUuk9LoMgqot/4Zqup1CUE8mhljU1OUBgsqq0wFqWv8KlfZ7JhbKjYgPiGXo aEYlYORRiug4eUygcEdYZHCkRCoSe4v90TtCgZxOYAKFITtDvT+Mk1nCEAoO0LJESyuUVquF vkEBKkWihhHEKtSaQCGjjJYp/ZQ+ajpBnSiP8ZEzmi1ikegtiYW4Pz62+/SCnXLA/eBUzwqe Au64HQf2HEj6we9SevDjYB3HkWwC8Lf2x5iteApgdlE921ZMA/jAcJF4JintvQZsAwOAVRl9 2HNWUWoHbmURpDe8Wa5nWfEGciusXNDZWUk42cWCZ6ZK2daBE+kPCxbSVjGL3AT1lYOYFfPI QHhhdBbY7F6FBTfmLBwOx54Mgoanm2yUl2BngWn1fdxCOfJL4eoSkKyyh+dmr7Bs2hCYdXd4 DTvBR1fr2DbMh3+dTF3DSfB+Rj5mwykAZk0l2fBW+EfzMmb1xUlPWG3wtbVfhqe7fsJsvg7w xKJpTcqDDcWmVTokX4Ndv0ptbTdobjWsBUfBxiHdWlY5AKbmNdplA4H2hXW0L6yj/d+5BOA6 4MIo1QkxjFqiFMuZpOe/HKVI0IPVA/H6qAGMjJl9jADjACOAHFy4gbeZw6cdedF08iFGpYhU JcoYtRFILHHn4PyNUQrLhck1kWI/f5GfVCr1839bKhY68/budKUdyRhaw8QzjJJRPdNhHHt+ CobjvPSyCOf07NHSu77jDyu5W+47R/Uuj3evP1rGlObMlBxxmPsRLAUdaBpuq4n9oCIkuEfy 5Q/HuHanZnd10CtLzPKhzOptnl8Xpx/s2NwzH+bxQNSIRpcMex4+0h09O+jAKenYbW5y2buL K8iZ3njB8UxBzRy1Pc4/XObzpnuGa+i3pMAzkAsDfk7aHryv3rjbYSL0sDvmerbjsxuF3W39 16+vJL//PSfI42MWGTlTNxL+7n7uV7PdWcZP9rhFXMIuf95ToY9qK6/YV2NeDNaO3wvTJoed emL2+EKeq6z1ME06FWdDYQI/6074+n+6zIOezXOMCz83Yhv56VhZUd7tlhEhSx1Li71wlZr+ D8IJLompBAAA X-Brightmail-Tracker: H4sIAAAAAAAAA02Rf1CLcRzHfZ/n8explMfK+aouZ84wK7ULX786P89zuMud487PGh4V22RT 5JxGZVkoKncl0vxs5dfKrrTIQla6Omskll/DifSTVTvS2rnz3/s+r9fd648PhfPqCV8qRr6P VcglUj7JJQzV/ImB70/4SYK/tGHoVu0THOXpDAAV2dJJdLazH0cp6l8Ysti9UOWPcyNRYdFj DH1808NBvzWtGHo82E6ihvZqDJ0xvQDIqM8iUIkjlUSVLTOQsdJMoPyrnzgo7WUZia7V/MHQ 6VQrhsrsRwAyOPNx1JSlxdG7jpcc9K3PTKJjd34C5OzLIxf5M5amVUx5ro3DlFwXMpb6OEav O04yJZcTmYpXKpLp+tRCMB33rSRzqlQHmB59AKOuSsPWjN7IXbCDlcbEs4qZYZHc6Lrs/pGx 1skHOp4N4irw2l8DPChIh0JtQz3QAC7Fo8sAzDmahbmBP7xhO066tzcs/POF45a6APygeUe4 AEkHwheX9cPbh14M67oNmEvC6TYCttoqcBfwpufCnH41x7UJegrUFzYPFzzphbDg7S/gLkyE Oc8dQw5FedBh8F73FNeZN6Q8MGZz3PpYaM6xD7fwIT3p7jk8A9C5/6Hc/9BFgOnABDZWKYuS KUNixXJ2f5BSIlPGyaOCtu+R6cHwx4XCMmDUdQaZAEYBE4AUzvfxnEr5SnieOyQJB1nFnghF nJRVmoAfRfDHezZqzBE8Okqyj93NsrGs4h/FKA9fFbZ06+z7AyJvsfqeI1TnWBs/vWnO7oH0 n1Z6TevNTON4ca0ps2vc4fDvquDe2SFekX3ra9fx+2puG1cLekIvbRP1zpPfujJLZNauPP9U rCm2LCnyHqgrn7QsISOi4EKHx5b+4u+azTGqk7LGjRh1qUKjDfsxzW9/aX57t2VqzIhEYzhF PHm2IiF9L7NTOmA7OL8qKWnTibYVqaHXA5rFvLERR5cGWkssBb2iKu7DdLZ5gXCMwGfXzRpn 4DH7V/9xdmVncoDK0dIQLCg+XMxrBKNup43OW36oXBBlEPp2//6cLIrOrp7klwrVzkyv8I9k ijRkA1/rFByJrC8d8wjG54v5hDJaEiLEFUrJX9ryORVgAwAA X-CMS-MailID: 20210817101809epcas5p39eed3531ed82f5f08127eb3dba1fc50f X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20210817101809epcas5p39eed3531ed82f5f08127eb3dba1fc50f References: <20210817101423.12367-1-selvakuma.s1@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Nitesh Shetty For the devices which does not support simple copy, copy emulation is added. Also for stacked devices, copy is performed via emulation. Copy-emulation is implemented by allocating maximum possible memory less than or equal to total copy size. The source ranges are read into memory by chaining bio for each source ranges and submitting them async and the last bio waits for completion. After data is read, it is written to the destination and the process is repeated till no source ranges left. bio_map_kern() is used to allocate bio and add pages of copy buffer to bio. As bio->bi_private and bio->bi_end_io are needed for chaining the bio and gets over-written, invalidate_kernel_vmap_range() for read is called in the caller. Signed-off-by: Nitesh Shetty Signed-off-by: SelvaKumar S Reported-by: kernel test robot --- block/blk-lib.c | 100 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/block/blk-lib.c b/block/blk-lib.c index 7fee0ae95c44..d29c52b90dcf 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -333,6 +333,64 @@ int blk_copy_offload_scc(struct block_device *src_bdev, int nr_srcs, return ret; } +int blk_submit_rw_buf(struct block_device *bdev, void *buf, sector_t buf_len, + sector_t sector, unsigned int op, gfp_t gfp_mask) +{ + struct request_queue *q = bdev_get_queue(bdev); + struct bio *bio, *parent = NULL; + sector_t max_hw_len = min_t(unsigned int, queue_max_hw_sectors(q), + queue_max_segments(q) << (PAGE_SHIFT - SECTOR_SHIFT)); + sector_t len, remaining; + int ret; + + for (remaining = buf_len; remaining > 0; remaining -= len) { + len = min_t(int, max_hw_len, remaining); +retry: + bio = bio_map_kern(q, buf, len << SECTOR_SHIFT, gfp_mask); + if (IS_ERR(bio)) { + len >>= 1; + if (len) + goto retry; + return PTR_ERR(bio); + } + + bio->bi_iter.bi_sector = sector; + bio->bi_opf = op; + bio_set_dev(bio, bdev); + bio->bi_end_io = NULL; + bio->bi_private = NULL; + + if (parent) { + bio_chain(parent, bio); + submit_bio(parent); + } + parent = bio; + sector += len; + } + ret = submit_bio_wait(bio); + bio_put(bio); + + return ret; +} + +static void *blk_alloc_buf(sector_t req_size, sector_t *alloc_size, gfp_t gfp_mask) +{ + int min_size = PAGE_SIZE; + void *buf; + + while (req_size >= min_size) { + buf = kvmalloc(req_size, gfp_mask); + if (buf) { + *alloc_size = (req_size >> SECTOR_SHIFT); + return buf; + } + /* retry half the requested size */ + req_size >>= 1; + } + + return NULL; +} + static inline sector_t blk_copy_len(struct range_entry *rlist, int nr_srcs) { int i; @@ -348,6 +406,46 @@ static inline sector_t blk_copy_len(struct range_entry *rlist, int nr_srcs) return len; } +/* + * If native copy offload feature is absent, this function tries to emulate, + * by copying data from source to a temporary buffer and from buffer to + * destination device. + */ +static int blk_copy_emulate(struct block_device *src_bdev, int nr_srcs, + struct range_entry *rlist, struct block_device *dest_bdev, + sector_t dest, gfp_t gfp_mask) +{ + void *buf = NULL; + int ret, nr_i = 0; + sector_t src_blk, copy_len, buf_len, read_len, copied_len, remaining = 0; + + copy_len = blk_copy_len(rlist, nr_srcs); + buf = blk_alloc_buf(copy_len << SECTOR_SHIFT, &buf_len, gfp_mask); + if (!buf) + return -ENOMEM; + + for (copied_len = 0; copied_len < copy_len; copied_len += read_len) { + if (!remaining) { + src_blk = rlist[nr_i].src; + remaining = rlist[nr_i++].len; + } + + read_len = min_t(sector_t, remaining, buf_len); + ret = blk_submit_rw_buf(src_bdev, buf, read_len, src_blk, REQ_OP_READ, gfp_mask); + if (ret) + goto out; + src_blk += read_len; + remaining -= read_len; + ret = blk_submit_rw_buf(dest_bdev, buf, read_len, dest + copied_len, REQ_OP_WRITE, + gfp_mask); + if (ret) + goto out; + } +out: + kvfree(buf); + return ret; +} + static inline bool blk_check_offload_scc(struct request_queue *src_q, struct request_queue *dest_q) { @@ -398,6 +496,8 @@ int blkdev_issue_copy(struct block_device *src_bdev, int nr_srcs, if (blk_check_offload_scc(src_q, dest_q)) ret = blk_copy_offload_scc(src_bdev, nr_srcs, src_rlist, dest_bdev, dest, gfp_mask); + else + ret = blk_copy_emulate(src_bdev, nr_srcs, src_rlist, dest_bdev, dest, gfp_mask); return ret; } From patchwork Tue Aug 17 10:14:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: SelvaKumar S X-Patchwork-Id: 499348 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58773C4338F for ; Tue, 17 Aug 2021 11:29:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 41D7960FBF for ; Tue, 17 Aug 2021 11:29:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239805AbhHQLaE (ORCPT ); Tue, 17 Aug 2021 07:30:04 -0400 Received: from mailout3.samsung.com ([203.254.224.33]:27951 "EHLO mailout3.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239754AbhHQLaC (ORCPT ); Tue, 17 Aug 2021 07:30:02 -0400 Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20210817112928epoutp033dc925ce8c750ad33fb4a4c268edf642~cFI6zNF_C2784227842epoutp034 for ; Tue, 17 Aug 2021 11:29:28 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20210817112928epoutp033dc925ce8c750ad33fb4a4c268edf642~cFI6zNF_C2784227842epoutp034 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1629199768; bh=2NSwAJ4QSL4cIweNj4RQY2Ku8GUcNJplrAoR+sS/NWs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NivmeookvcnDZOz1ecvGDbNNhNdB7agfdaF4mN0+cEf3ZG0BjVI5inODF1AdyBfGh 0ruH/Y2nG4sIDTLpdxImisijBre9clOv2d5hzR0cmWB2dheEDsOWrKZuN4XmwEl4/a 72tZzQifOrxcbpdf9HGZwW/1BURoYbz0v1OhgXzM= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20210817112927epcas5p2bb2fb797e40e073a30aec8e0b6a8fcc5~cFI6FKHC-1778517785epcas5p22; Tue, 17 Aug 2021 11:29:27 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.178]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4Gpphg4DFzz4x9Px; Tue, 17 Aug 2021 11:29:23 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id FA.8B.09595.39D9B116; Tue, 17 Aug 2021 20:29:23 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20210817101814epcas5p41db3d7269f5139efcaf2ca685cd04a16~cEKusWySs0955209552epcas5p4P; Tue, 17 Aug 2021 10:18:14 +0000 (GMT) Received: from epsmgms1p2.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20210817101814epsmtrp1e53a42834f8a0bb773c66dfd1c0ac19a~cEKurLd0y2073620736epsmtrp1c; Tue, 17 Aug 2021 10:18:14 +0000 (GMT) X-AuditID: b6c32a4a-eebff7000000257b-4a-611b9d93e418 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2.samsung.com (Symantec Messaging Gateway) with SMTP id AA.19.32548.6EC8B116; Tue, 17 Aug 2021 19:18:14 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20210817101810epsmtip206bab5a6d4d72e2cd99e1ec61c9525e6~cEKq6bEIJ0079500795epsmtip2k; Tue, 17 Aug 2021 10:18:10 +0000 (GMT) From: SelvaKumar S To: linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: linux-api@vger.kernel.org, linux-scsi@vger.kernel.org, linux-fsdevel@vger.kernel.org, dm-devel@redhat.com, kbusch@kernel.org, axboe@kernel.dk, damien.lemoal@wdc.com, asml.silence@gmail.com, johannes.thumshirn@wdc.com, hch@lst.de, willy@infradead.org, kch@kernel.org, martin.petersen@oracle.com, mpatocka@redhat.com, bvanassche@acm.org, djwong@kernel.org, snitzer@redhat.com, agk@redhat.com, selvajove@gmail.com, joshiiitr@gmail.com, nj.shetty@samsung.com, nitheshshetty@gmail.com, joshi.k@samsung.com, javier.gonz@samsung.com, SelvaKumar S Subject: [PATCH 6/7] nvme: add simple copy support Date: Tue, 17 Aug 2021 15:44:22 +0530 Message-Id: <20210817101423.12367-7-selvakuma.s1@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210817101423.12367-1-selvakuma.s1@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Ta1AbZRR1d8MSGMElvD4DamahD6hAAgQ/aqFOH3ZHoGV02qow4g6swBCS mISiVUq0tQE6QHmFQqEogzyChRYiUhpAkikdYBxKoVTb8tIgg52SFlqhUooJSbX/zpzvnHvu vd9cNsYpteeyU8UKRiamRSTuyOow+G0JKKn2ovljWk/YOtiPwSpNBwKbJwpxqL7/GIPfqP5G 4ajRGXYvnLWDTc1XUPjHnSV7uJY3icIr6/dwOHzPgMJi/TgCdW2lLNi+nIPD7lvboK57gAVr 6mft4ambnThsuPoUhUU5N1DYafwKgR2rNRgcK63F4LTppj28uzKAw5MXHyFwdaUKf8ubGh2L oi5VTthT7Y3+1OgvGVSbJhen2uuyqcu/KXHqwewtFmXquYFTBVoNQi21vUqpfj6Fxr74YdqO FIZOYmQ8RpwoSUoVJ0eQUe8l7E4QhvEFAYJw+AbJE9PpTAS5Jzo24O1UkXkZJO8ILcowU7G0 XE4GRe6QSTIUDC9FIldEkIw0SSQNlQbK6XR5hjg5UMwotgv4/GChWfhxWsrxvgJMqt73Wa2u H1MiCzAPcWADIhSoSrrwPMSRzSEuI6DfoLOzPHCIRQRcmHS3PiwhQG0y2D9zaB5W2xxdCFA9 0bKsDrNKVUJZME4EgPG6tg3ejdgJmh5r7CwGjBhkgXJT7UYlVyIYlC9UYhbMIjaBoeYpxIKd iAhwvuwfzJr2Gqi4vmzWs9kORCToWtxklbiAgQrjRn3MLDn+41nMUh8QBgdQOdKAW717wKjm tq1rV/DXVa0Nc8F84UkbzgR/5qpRK1YioMCUacU7wYhuDbXkYoQfaO0KstKvgLLBFtSa6wzy V402qxPoPGfckANiMxj8KcxKe4P7hi5bNxTIne9ErXsrQsBQ6wLrNMKrfG6cyufGqfw/+VsE 0yAvM1J5ejIjF0qDxUzmf3+cKElvQzbOw/+dTmRm+n6gHkHZiB4BbIx0c9rC5tIcpyT686OM TJIgyxAxcj0iNK+7COO6J0rM9yVWJAhCw/mhYWFhoeEhYQLS0yk+2ovmEMm0gkljGCkje+ZD 2Q5cJep7bC1ruXM2psa5z3dAETnaIJzb9YDX11hgfLN5pSf74Hdsnwv5WlXOfo9PXmL6HuWX a3ILd2/OWyy/tDVGPDQed90lML7x2BHhiQVptLvLfFPJnI/f6xAdixE1zDO64nVGnlU/vG3u wNILu+7O7P9APxTb46o8NxwRr3QeMf16eC/5Ud2JfYKvod2TdUXkUV+/lTPxnvkqDjd6b28L r98td8I3ruzgl1nXqlhbf59WD38/PARmjKTX+eJ6/aGotOzq6L6pse3lI1Nq+PRQUJxjSG+Q 84Fr1RVfTHl7MbrJlpDe97UeJaDIlX/nBx+tx0Ppu+vLfrfPfLqaeVFxWCJkB5AseQot8Mdk cvpfaDbvJqcEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrMIsWRmVeSWpSXmKPExsWy7bCSvO6zHulEg3lH5C3WnzrGbDFn1TZG i9V3+9kspn34yWzR2v6NyeLyEz6Lve9ms1qsXH2UyeLxnc/sFn+77jFZHP3/ls3i/NvDTBaT Dl1jtNizaQqLxebvHWwWe29pW+zZe5LFYv6yp+wW3dd3sFksP/6PyWJix1Umix1PGhkttv2e z2xxZcoiZosH76+zW7z+cZLNom3jV0aL3z/msDnIeFy+4u2xc9Zddo/NK7Q8Lp8t9di0qpPN Y/OSeo/dNxvYPD4+vcXi8X7fVTaPvi2rGD0+b5LzaD/QzRTAE8Vlk5Kak1mWWqRvl8CV0Xyw j7lgmnvFoj3HmBsY31l0MXJySAiYSKz6Mpeti5GLQ0hgB6PEz9UzWCASMhJr73ayQdjCEiv/ PWeHKPrIKHFkSQcrSIJNQFfi2pJNYA0iAo4Spz9tYwIpYhZ4xSJx7+5uZpCEsICRxPR3s8Bs FgFVidOr7zOC2LwCthJrp/5ihtggLzHz0negDRwcnAJ2Ers+qYKEhYBK9u+Zyg5RLihxcuYT FpASZgF1ifXzhEDCzECdzVtnM09gFJyFpGoWQtUsJFULGJlXMUqmFhTnpucWGxYY5aWW6xUn 5haX5qXrJefnbmIEpwMtrR2Me1Z90DvEyMTBeIhRgoNZSYRXnUMqUYg3JbGyKrUoP76oNCe1 +BCjNAeLkjjvha6T8UIC6YklqdmpqQWpRTBZJg5OqQamHS8yH3zmirnTP9PXuXfxlIl3uMSk N921z27g6jJ5469aeiXi36K6C/b/fUP07my2u8fG1vKfpfrseRN3/xuFn9V+mlXu6p6y8LBe 6Gr1ddzKBeG9T/47bM7cwFZxbDrTytrA/q++h4N2CZmKVoYu4XzyPidn00TPqw9UVrqJ7XbS Zk8WFXzzhV907qZp4v8japMsHLPEb08qvK90euadhz335v7Zq2582YWp7YLe7nMpi60W9nnp zVf4W3tAjd+rjo2v7P0hNkcjlg0zrfQ/Zvlauy+PzN8benVKR1/G43iDFpn1ZduP6mTXXmnk y519SpTpdF2XwibfDZX2lzlEfm58u5bx3AX9n4FLXugrsRRnJBpqMRcVJwIAwmWwxXYDAAA= X-CMS-MailID: 20210817101814epcas5p41db3d7269f5139efcaf2ca685cd04a16 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20210817101814epcas5p41db3d7269f5139efcaf2ca685cd04a16 References: <20210817101423.12367-1-selvakuma.s1@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add support for TP 4065a ("Simple Copy Command"), v2020.05.04 ("Ratified") For device supporting native simple copy, this implementation accepts the payload passed from the block layer and convert payload to form simple copy command and submit to the device. Set the device copy limits to queue limits. By default copy_offload is disabled. End-to-end protection is done by setting both PRINFOR and PRINFOW to 0. Signed-off-by: SelvaKumar S Signed-off-by: Kanchan Joshi Signed-off-by: Nitesh Shetty Signed-off-by: Javier González --- drivers/nvme/host/core.c | 83 +++++++++++++++++++++++++++++++++++++++ drivers/nvme/host/trace.c | 19 +++++++++ include/linux/nvme.h | 43 ++++++++++++++++++-- 3 files changed, 142 insertions(+), 3 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index f5aabbb4a00f..41f02ee8c0d2 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -825,6 +825,59 @@ static inline void nvme_setup_flush(struct nvme_ns *ns, cmnd->common.nsid = cpu_to_le32(ns->head->ns_id); } +static inline blk_status_t nvme_setup_copy(struct nvme_ns *ns, + struct request *req, struct nvme_command *cmnd) +{ + struct nvme_ctrl *ctrl = ns->ctrl; + struct nvme_copy_range *range = NULL; + struct blk_copy_payload *payload; + unsigned short nr_range = 0; + u16 control = 0, ssrl; + u32 dsmgmt = 0; + u64 slba; + int i; + + payload = bio_data(req->bio); + nr_range = payload->copy_nr_ranges; + + if (req->cmd_flags & REQ_FUA) + control |= NVME_RW_FUA; + + if (req->cmd_flags & REQ_FAILFAST_DEV) + control |= NVME_RW_LR; + + cmnd->copy.opcode = nvme_cmd_copy; + cmnd->copy.nsid = cpu_to_le32(ns->head->ns_id); + cmnd->copy.sdlba = cpu_to_le64(blk_rq_pos(req) >> (ns->lba_shift - 9)); + + range = kmalloc_array(nr_range, sizeof(*range), + GFP_ATOMIC | __GFP_NOWARN); + if (!range) + return BLK_STS_RESOURCE; + + for (i = 0; i < nr_range; i++) { + slba = (payload->range[i].src) >> (ns->lba_shift - 9); + ssrl = (payload->range[i].len) >> (ns->lba_shift - 9); + range[i].slba = cpu_to_le64(slba); + range[i].nlb = cpu_to_le16(ssrl - 1); + } + + cmnd->copy.nr_range = nr_range - 1; + + req->special_vec.bv_page = virt_to_page(range); + req->special_vec.bv_offset = offset_in_page(range); + req->special_vec.bv_len = sizeof(*range) * nr_range; + req->rq_flags |= RQF_SPECIAL_PAYLOAD; + + if (ctrl->nr_streams) + nvme_assign_write_stream(ctrl, req, &control, &dsmgmt); + + cmnd->rw.control = cpu_to_le16(control); + cmnd->rw.dsmgmt = cpu_to_le32(dsmgmt); + + return BLK_STS_OK; +} + static blk_status_t nvme_setup_discard(struct nvme_ns *ns, struct request *req, struct nvme_command *cmnd) { @@ -1012,6 +1065,9 @@ blk_status_t nvme_setup_cmd(struct nvme_ns *ns, struct request *req) case REQ_OP_DISCARD: ret = nvme_setup_discard(ns, req, cmd); break; + case REQ_OP_COPY: + ret = nvme_setup_copy(ns, req, cmd); + break; case REQ_OP_READ: ret = nvme_setup_rw(ns, req, cmd, nvme_cmd_read); break; @@ -1640,6 +1696,31 @@ static void nvme_config_discard(struct gendisk *disk, struct nvme_ns *ns) blk_queue_max_write_zeroes_sectors(queue, UINT_MAX); } +static void nvme_config_copy(struct gendisk *disk, struct nvme_ns *ns, + struct nvme_id_ns *id) +{ + struct nvme_ctrl *ctrl = ns->ctrl; + struct request_queue *queue = disk->queue; + + if (!(ctrl->oncs & NVME_CTRL_ONCS_COPY)) { + queue->limits.copy_offload = 0; + queue->limits.max_copy_sectors = 0; + queue->limits.max_copy_range_sectors = 0; + queue->limits.max_copy_nr_ranges = 0; + blk_queue_flag_clear(QUEUE_FLAG_SIMPLE_COPY, queue); + return; + } + + /* setting copy limits */ + blk_queue_flag_test_and_set(QUEUE_FLAG_SIMPLE_COPY, queue); + queue->limits.copy_offload = 0; + queue->limits.max_copy_sectors = le64_to_cpu(id->mcl) * + (1 << (ns->lba_shift - 9)); + queue->limits.max_copy_range_sectors = le32_to_cpu(id->mssrl) * + (1 << (ns->lba_shift - 9)); + queue->limits.max_copy_nr_ranges = id->msrc + 1; +} + static bool nvme_ns_ids_valid(struct nvme_ns_ids *ids) { return !uuid_is_null(&ids->uuid) || @@ -1811,6 +1892,7 @@ static void nvme_update_disk_info(struct gendisk *disk, nvme_config_discard(disk, ns); blk_queue_max_write_zeroes_sectors(disk->queue, ns->ctrl->max_zeroes_sectors); + nvme_config_copy(disk, ns, id); set_disk_ro(disk, (id->nsattr & NVME_NS_ATTR_RO) || test_bit(NVME_NS_FORCE_RO, &ns->flags)); @@ -4564,6 +4646,7 @@ static inline void _nvme_check_size(void) BUILD_BUG_ON(sizeof(struct nvme_download_firmware) != 64); BUILD_BUG_ON(sizeof(struct nvme_format_cmd) != 64); BUILD_BUG_ON(sizeof(struct nvme_dsm_cmd) != 64); + BUILD_BUG_ON(sizeof(struct nvme_copy_command) != 64); BUILD_BUG_ON(sizeof(struct nvme_write_zeroes_cmd) != 64); BUILD_BUG_ON(sizeof(struct nvme_abort_cmd) != 64); BUILD_BUG_ON(sizeof(struct nvme_get_log_page_command) != 64); diff --git a/drivers/nvme/host/trace.c b/drivers/nvme/host/trace.c index 6543015b6121..0e7db7ad5ed3 100644 --- a/drivers/nvme/host/trace.c +++ b/drivers/nvme/host/trace.c @@ -136,6 +136,23 @@ static const char *nvme_trace_read_write(struct trace_seq *p, u8 *cdw10) return ret; } +static const char *nvme_trace_copy(struct trace_seq *p, u8 *cdw10) +{ + const char *ret = trace_seq_buffer_ptr(p); + u64 slba = get_unaligned_le64(cdw10); + u8 nr_range = get_unaligned_le16(cdw10 + 8); + u16 control = get_unaligned_le16(cdw10 + 10); + u32 dsmgmt = get_unaligned_le32(cdw10 + 12); + u32 reftag = get_unaligned_le32(cdw10 + 16); + + trace_seq_printf(p, + "slba=%llu, nr_range=%u, ctrl=0x%x, dsmgmt=%u, reftag=%u", + slba, nr_range, control, dsmgmt, reftag); + trace_seq_putc(p, 0); + + return ret; +} + static const char *nvme_trace_dsm(struct trace_seq *p, u8 *cdw10) { const char *ret = trace_seq_buffer_ptr(p); @@ -227,6 +244,8 @@ const char *nvme_trace_parse_nvm_cmd(struct trace_seq *p, return nvme_trace_zone_mgmt_send(p, cdw10); case nvme_cmd_zone_mgmt_recv: return nvme_trace_zone_mgmt_recv(p, cdw10); + case nvme_cmd_copy: + return nvme_trace_copy(p, cdw10); default: return nvme_trace_common(p, cdw10); } diff --git a/include/linux/nvme.h b/include/linux/nvme.h index b7c4c4130b65..4f79f223c9eb 100644 --- a/include/linux/nvme.h +++ b/include/linux/nvme.h @@ -295,7 +295,7 @@ struct nvme_id_ctrl { __u8 nvscc; __u8 nwpc; __le16 acwu; - __u8 rsvd534[2]; + __le16 ocfs; __le32 sgls; __le32 mnan; __u8 rsvd544[224]; @@ -320,6 +320,7 @@ enum { NVME_CTRL_ONCS_WRITE_ZEROES = 1 << 3, NVME_CTRL_ONCS_RESERVATIONS = 1 << 5, NVME_CTRL_ONCS_TIMESTAMP = 1 << 6, + NVME_CTRL_ONCS_COPY = 1 << 8, NVME_CTRL_VWC_PRESENT = 1 << 0, NVME_CTRL_OACS_SEC_SUPP = 1 << 0, NVME_CTRL_OACS_DIRECTIVES = 1 << 5, @@ -368,7 +369,10 @@ struct nvme_id_ns { __le16 npdg; __le16 npda; __le16 nows; - __u8 rsvd74[18]; + __le16 mssrl; + __le32 mcl; + __u8 msrc; + __u8 rsvd91[11]; __le32 anagrpid; __u8 rsvd96[3]; __u8 nsattr; @@ -689,6 +693,7 @@ enum nvme_opcode { nvme_cmd_resv_report = 0x0e, nvme_cmd_resv_acquire = 0x11, nvme_cmd_resv_release = 0x15, + nvme_cmd_copy = 0x19, nvme_cmd_zone_mgmt_send = 0x79, nvme_cmd_zone_mgmt_recv = 0x7a, nvme_cmd_zone_append = 0x7d, @@ -710,7 +715,8 @@ enum nvme_opcode { nvme_opcode_name(nvme_cmd_resv_release), \ nvme_opcode_name(nvme_cmd_zone_mgmt_send), \ nvme_opcode_name(nvme_cmd_zone_mgmt_recv), \ - nvme_opcode_name(nvme_cmd_zone_append)) + nvme_opcode_name(nvme_cmd_zone_append), \ + nvme_opcode_name(nvme_cmd_copy)) @@ -883,6 +889,36 @@ struct nvme_dsm_range { __le64 slba; }; +struct nvme_copy_command { + __u8 opcode; + __u8 flags; + __u16 command_id; + __le32 nsid; + __u64 rsvd2; + __le64 metadata; + union nvme_data_ptr dptr; + __le64 sdlba; + __u8 nr_range; + __u8 rsvd12; + __le16 control; + __le16 rsvd13; + __le16 dspec; + __le32 ilbrt; + __le16 lbat; + __le16 lbatm; +}; + +struct nvme_copy_range { + __le64 rsvd0; + __le64 slba; + __le16 nlb; + __le16 rsvd18; + __le32 rsvd20; + __le32 eilbrt; + __le16 elbat; + __le16 elbatm; +}; + struct nvme_write_zeroes_cmd { __u8 opcode; __u8 flags; @@ -1427,6 +1463,7 @@ struct nvme_command { struct nvme_download_firmware dlfw; struct nvme_format_cmd format; struct nvme_dsm_cmd dsm; + struct nvme_copy_command copy; struct nvme_write_zeroes_cmd write_zeroes; struct nvme_zone_mgmt_send_cmd zms; struct nvme_zone_mgmt_recv_cmd zmr; From patchwork Tue Aug 17 10:14:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SelvaKumar S X-Patchwork-Id: 498448 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 179DDC4320A for ; Tue, 17 Aug 2021 11:29:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0085060EBD for ; Tue, 17 Aug 2021 11:29:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239809AbhHQLaN (ORCPT ); Tue, 17 Aug 2021 07:30:13 -0400 Received: from mailout2.samsung.com ([203.254.224.25]:27564 "EHLO mailout2.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239754AbhHQLaL (ORCPT ); Tue, 17 Aug 2021 07:30:11 -0400 Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20210817112936epoutp029c9410f5a8df585bea815aa6e9fd1e6f~cFJC4L-mI0850008500epoutp02N for ; Tue, 17 Aug 2021 11:29:36 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20210817112936epoutp029c9410f5a8df585bea815aa6e9fd1e6f~cFJC4L-mI0850008500epoutp02N DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1629199777; bh=muF+tmBf18oq+jqxS8Klsw9Cot4xGLU3sYZJeI5PbUg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FtlI5eYOYCGUG/61qGsJgWinqIwb8hf4mv1CnWOvuBgxRLncXf0L2InO514rIwSxu QvfzVwvhxtvTnV8IAio/Nw0cIxPeOc1NBezsQFMiJgKJ/ThpnKyg8jxeji4ebhNnuA rgQwIEQY01lNOARa9GxWnFnZNwUJ2qqqCglraCdo= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20210817112935epcas5p1c85a4d338de4388002d2ed46ef2c3f39~cFJB1cQDN0060000600epcas5p15; Tue, 17 Aug 2021 11:29:35 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.183]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4Gpphq5p9Mz4x9Px; Tue, 17 Aug 2021 11:29:31 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 13.9B.09595.B9D9B116; Tue, 17 Aug 2021 20:29:31 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20210817101822epcas5p470644cf681d5e8db5367dc7998305c65~cEK1x04pr2768027680epcas5p47; Tue, 17 Aug 2021 10:18:22 +0000 (GMT) Received: from epsmgms1p2.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20210817101822epsmtrp138474bfe16a311ca3898be176791ab29~cEK1wcEAH2073620736epsmtrp1f; Tue, 17 Aug 2021 10:18:22 +0000 (GMT) X-AuditID: b6c32a4a-ed5ff7000000257b-5d-611b9d9b47ff Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2.samsung.com (Symantec Messaging Gateway) with SMTP id 1D.19.32548.EEC8B116; Tue, 17 Aug 2021 19:18:22 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20210817101818epsmtip24573d2010be5372cecb5a4286a23f483~cEKyBa81A3274532745epsmtip2H; Tue, 17 Aug 2021 10:18:18 +0000 (GMT) From: SelvaKumar S To: linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: linux-api@vger.kernel.org, linux-scsi@vger.kernel.org, linux-fsdevel@vger.kernel.org, dm-devel@redhat.com, kbusch@kernel.org, axboe@kernel.dk, damien.lemoal@wdc.com, asml.silence@gmail.com, johannes.thumshirn@wdc.com, hch@lst.de, willy@infradead.org, kch@kernel.org, martin.petersen@oracle.com, mpatocka@redhat.com, bvanassche@acm.org, djwong@kernel.org, snitzer@redhat.com, agk@redhat.com, selvajove@gmail.com, joshiiitr@gmail.com, nj.shetty@samsung.com, nitheshshetty@gmail.com, joshi.k@samsung.com, javier.gonz@samsung.com, SelvaKumar S Subject: [PATCH 7/7] dm kcopyd: add simple copy offload support Date: Tue, 17 Aug 2021 15:44:23 +0530 Message-Id: <20210817101423.12367-8-selvakuma.s1@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210817101423.12367-1-selvakuma.s1@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Te0xTZxTfd29pC1lNVyB+VhzsJsjAAa0W9rGBksjMFR/DkcVA2OAOLs++ 0hZRMxyMN8jLIWABUac4WwVX5CEW40DX0IVtDijDiIGtzIyHIJ0DB4yVFp3//c7v/M73O+d8 OWycV83is1OkKlohpcQE04nR3uvt7VvXsJkSrGZtQy3G73FUr2kHSDtazkTVc89xlFfwN4YG zBtQ95M6B3RFew9Dvz+0sNBK8SMM3VudYaKfZnoxdKrHBJBeV8VArQuFTNT9YBvSd/cxUGPT BAuVDHcy0WXDvxiqLBzCUKc5G6D2pUYcDVZdwNHY7DALTS32MVH+t88AWlqsZ4a6kQOD+8ib 6lEW2fqNDznQn07qNEVMsvXiF+StkSwm+XTiAYOcvT3EJMtuaABp0b1JFtwpwSJej04LTqap BFrhQUvjZQkp0qQQYl9k7O7YgECB0FcYhN4lPKSUhA4hwvZH+O5JEVuXQXgcocTpViqCUioJ /53BClm6ivZIlilVIQQtTxDLRXI/JSVRpkuT/KS06j2hQLA9wCqMS0uuH61jyc3uR//RX8az QAW/GDiyIVcET/1w1aEYOLF53FsA1pXWrAfzAOY1NOP2wAJgx+kq/EVJg/4OZk90AWgYtLBe qh62VTisqZhcX2i6qGOsYRfuLnjlucb2Ls41MmDN7AXWWsLZmpj6tRZbwwyuJzyfu2LjOdwQ OHP3Z8xu5w7P/LJg5dlsR+5O2DXvaZe8AfvOmG3v41ZJTludrVXI1TrCq6Yapr02DBrPLTHs 2BlOGm6w7JgP/yzPX8cZ8I+i6nWvLADLZjPseBe8r1/B1nxxrjds6fK301vgaWMzZvfdAEuX zOulHNh51myTQ+5WaOwItNNucK63a70bEpbf1KwvrhLA/uXvHCqAh/qVcdSvjKP+3/kcwDVg Ey1XSpJoZYB8u5TOePnL8TKJDtgOxCe8E4yPzfn1AIwNegBk44QLx4vNp3icBOrYcVohi1Wk i2llDwiwrrsS57vGy6wXJlXFCkVBAlFgYKAoaEegkNjIidm/meJxkygVnUbTclrxog5jO/Kz sIyDezmGty5df9o2b5Is0FMRi65PTkamVro9DqvtatmY+Ki90TPq7tixyEuvVfDzS6gopfNk U3/4llGBU8ymI7UZ13cI+9SlH33qN97T3Hot80BTYWhRaN6Yi0K7mtM8J6nOrdXTBF9R5hjf K5teqJiUE+Wp4zMKQ67UU9BwOLrAf4/4aFJL99vPdNPvSJarGv2/DjfK3v+wT3P4eKElZaID jP01WJPo7jXMM2VOp33lTjuvfpki+fx21CfL2Sd0lsaRzEPXTi4fqMx2fXy/2nhQDX6MIeJG ks4rcqI5H2hxQ6TWxButkB/6eCpRJVrx+2x36m8ntkaKw4K9huKqcs82EAxlMiX0wRVK6j8p fxJIqQQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA02Ra0hTYRjHe885nZ2tVscp9M4uxsCiVUtL6i3TiiAOlFYQCGmsoQctb2PT 7AJpmNXULppWusW6SOq026ylc6thas2SpGle6pTpLMk0tcyszHQa9O3P//d7nufDQ+GiesKT 2heXwKriFDESUkCYHku8lvdlzlX4ZJkodLuuFkc6gwmgEu4siS70j+Ao7eR3DDmcs5C1Tzsd FZfUYKjzzVceGk1/i6GasV4Sveh9jKHsqlcAWYw5BCobPkUia9tSZLHaCaS/0cVDGc3lJCp8 8gdDWaeaMFTuPAaQ6ZceR40513DU/qWZh3p+2El04u4QQL9+6MiN8xhH41amIp/jMWVFUsZR n8gYDRqSKStIZipbU0hmoKuNYL48bCKZM/cMgPlqXMCctGVgO2buFqyPYGP2HWBVKwL3CqJ0 nJandHod/GkpxFPAOc90wKcg7QcvW2xYOhBQIrocwOvt+dgkmAdvchpyMrvD4j8feZPSAIBP 02wuiaSXw1cFRmIie9Cb4LNBk2sTTn8i4FuuEp8A7vQG2NNyyTVA0N7w6vFR3kQW0gGwt7ph 6poXzHs5PN5TFJ8OhOZB74laNK48suRO6W7Qnud03cLH9dT7WvwcoPP/Q/n/oSsAMwAxq1TH RsaqfZUr49gkmVoRq06Mi5SFx8cagevjUmk5sBj6ZVUAo0AVgBQu8RAupjwVImGE4tBhVhUv VyXGsOoqMJciJHOEDel2uYiOVCSw0SyrZFX/KEbxPVOwgG9B3OHtIeeviqfNqo/w6WI+67Cm LfKwwfbQ7FG3m2eEI1CUevTIWf26/lKrPeS2d7S5o9FjJ/8IeVqckCxoq/AL7LPqzZl7CuR1 3dJFqm65YBk/zSa+HzYQ9KYmdX5owIoM7evkhpYUaXBBy/7eqMyR4Lo9d2yJ1avMtcPf6LwP +ncHHQH6zbIikf+22Yb4leE+O8wznpOyav8x60W3z0ty39+FODcjZq27mxLThtInutfoFvol FZk7s7lnFTe6dLc0d3ZlPekplRc+iHaeLikN35n0tGZklZ7r0OQU+QYGi98Rn8I0rZWW32Ot 694PLQhuXg2Hhhz+eTB7aa2EUEcpfKW4Sq34CwqI2DlgAwAA X-CMS-MailID: 20210817101822epcas5p470644cf681d5e8db5367dc7998305c65 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20210817101822epcas5p470644cf681d5e8db5367dc7998305c65 References: <20210817101423.12367-1-selvakuma.s1@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Introduce copy_jobs to use copy-offload, if supported by underlying devices otherwise fall back to existing method. run_copy_jobs() calls block layer copy offload API, if both source and destination request queue are same and support copy offload. On successful completion, destination regions copied count is made zero, failed regions are processed via existing method. Signed-off-by: SelvaKumar S Signed-off-by: Nitesh Shetty --- drivers/md/dm-kcopyd.c | 56 +++++++++++++++++++++++++++++++++++++----- 1 file changed, 50 insertions(+), 6 deletions(-) diff --git a/drivers/md/dm-kcopyd.c b/drivers/md/dm-kcopyd.c index 37b03ab7e5c9..d9ee105a6127 100644 --- a/drivers/md/dm-kcopyd.c +++ b/drivers/md/dm-kcopyd.c @@ -74,18 +74,20 @@ struct dm_kcopyd_client { atomic_t nr_jobs; /* - * We maintain four lists of jobs: + * We maintain five lists of jobs: * - * i) jobs waiting for pages - * ii) jobs that have pages, and are waiting for the io to be issued. - * iii) jobs that don't need to do any IO and just run a callback - * iv) jobs that have completed. + * i) jobs waiting to try copy offload + * ii) jobs waiting for pages + * iii) jobs that have pages, and are waiting for the io to be issued. + * iv) jobs that don't need to do any IO and just run a callback + * v) jobs that have completed. * - * All four of these are protected by job_lock. + * All five of these are protected by job_lock. */ spinlock_t job_lock; struct list_head callback_jobs; struct list_head complete_jobs; + struct list_head copy_jobs; struct list_head io_jobs; struct list_head pages_jobs; }; @@ -579,6 +581,43 @@ static int run_io_job(struct kcopyd_job *job) return r; } +static int run_copy_job(struct kcopyd_job *job) +{ + int r, i, count = 0; + unsigned long flags = 0; + struct range_entry srange; + + struct request_queue *src_q, *dest_q; + + for (i = 0; i < job->num_dests; i++) { + srange.src = job->source.sector; + srange.len = job->source.count; + + src_q = bdev_get_queue(job->source.bdev); + dest_q = bdev_get_queue(job->dests[i].bdev); + + if (src_q != dest_q && !src_q->limits.copy_offload) + break; + + r = blkdev_issue_copy(job->source.bdev, 1, &srange, + job->dests[i].bdev, job->dests[i].sector, GFP_KERNEL, flags); + if (r) + break; + + job->dests[i].count = 0; + count++; + } + + if (count == job->num_dests) { + push(&job->kc->complete_jobs, job); + } else { + push(&job->kc->pages_jobs, job); + r = 0; + } + + return r; +} + static int run_pages_job(struct kcopyd_job *job) { int r; @@ -659,6 +698,7 @@ static void do_work(struct work_struct *work) spin_unlock_irq(&kc->job_lock); blk_start_plug(&plug); + process_jobs(&kc->copy_jobs, kc, run_copy_job); process_jobs(&kc->complete_jobs, kc, run_complete_job); process_jobs(&kc->pages_jobs, kc, run_pages_job); process_jobs(&kc->io_jobs, kc, run_io_job); @@ -676,6 +716,8 @@ static void dispatch_job(struct kcopyd_job *job) atomic_inc(&kc->nr_jobs); if (unlikely(!job->source.count)) push(&kc->callback_jobs, job); + else if (job->source.bdev->bd_disk == job->dests[0].bdev->bd_disk) + push(&kc->copy_jobs, job); else if (job->pages == &zero_page_list) push(&kc->io_jobs, job); else @@ -916,6 +958,7 @@ struct dm_kcopyd_client *dm_kcopyd_client_create(struct dm_kcopyd_throttle *thro spin_lock_init(&kc->job_lock); INIT_LIST_HEAD(&kc->callback_jobs); INIT_LIST_HEAD(&kc->complete_jobs); + INIT_LIST_HEAD(&kc->copy_jobs); INIT_LIST_HEAD(&kc->io_jobs); INIT_LIST_HEAD(&kc->pages_jobs); kc->throttle = throttle; @@ -971,6 +1014,7 @@ void dm_kcopyd_client_destroy(struct dm_kcopyd_client *kc) BUG_ON(!list_empty(&kc->callback_jobs)); BUG_ON(!list_empty(&kc->complete_jobs)); + WARN_ON(!list_empty(&kc->copy_jobs)); BUG_ON(!list_empty(&kc->io_jobs)); BUG_ON(!list_empty(&kc->pages_jobs)); destroy_workqueue(kc->kcopyd_wq);