From patchwork Wed May 18 15:11:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 573903 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC1F9C433FE for ; Wed, 18 May 2022 15:11:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239317AbiERPLf (ORCPT ); Wed, 18 May 2022 11:11:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41726 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239210AbiERPLS (ORCPT ); Wed, 18 May 2022 11:11:18 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E20B5F41; Wed, 18 May 2022 08:11:17 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B6E33B82161; Wed, 18 May 2022 15:11:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DEDD1C385A9; Wed, 18 May 2022 15:11:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1652886674; bh=O39hNoKdp2kdUukiFkmASShHZ2jD97pa/tqJ99wtPvA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=m/VcLNQsRtsPalLKYbuQx3y3jHaEVX8oM3Slu5wrNOuB5eQ/01eQJKLThdRgn+1Qv gFvp4Fg7k9vbwCpzM4+ZG2K0SBG7QrmDRpJg022ICG1ZXXlniuu+8mRNdIFWt5OIL1 fWVSUZwcrmgrwBWwQD20/IxUARp5k0InyItQS6M1iLv/PwG8X6KgMADOcTzztmtojQ Fn66r3j8SMBf74F0hBw8Pe2pIkTJKsedi9q5o+ZVBAjX5T6WPF4UTTlBrfczr1SxrX 2V/0fWZa4FzIgGNWe02EbSXT+4Q8kOn8Iy1f0dobOCowGhDCMjtmYYiV/ThjfqXAO9 C/7dOQeQhKLtg== From: Jeff Layton To: dhowells@redhat.com Cc: linux-fsdevel@vger.kernel.org, ceph-devel@vger.kernel.org, xiubli@redhat.com, idryomov@gmail.com Subject: [PATCH 1/4] netfs: fix sense of DIO test on short read Date: Wed, 18 May 2022 11:11:08 -0400 Message-Id: <20220518151111.79735-2-jlayton@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220518151111.79735-1-jlayton@kernel.org> References: <20220518151111.79735-1-jlayton@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org The sense of this test is reversed. There's nothing that prevents userland from requesting a DIO read that is longer than the available data. Conversely, we don't expect a buffered read to be short unless it hits the EOF. Suggested-by: David Howells Signed-off-by: Jeff Layton --- fs/netfs/io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) David, feel free to fold this into the patch that adds the condition so we can avoid the regression. diff --git a/fs/netfs/io.c b/fs/netfs/io.c index e5a15a924fc7..8188d43e8044 100644 --- a/fs/netfs/io.c +++ b/fs/netfs/io.c @@ -728,7 +728,7 @@ ssize_t netfs_begin_read(struct netfs_io_request *rreq, bool sync) ret = rreq->error; if (ret == 0 && rreq->submitted < rreq->len && - rreq->origin == NETFS_DIO_READ) { + rreq->origin != NETFS_DIO_READ) { trace_netfs_failure(rreq, NULL, ret, netfs_fail_short_read); ret = -EIO; } From patchwork Wed May 18 15:11:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 573904 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EDCCC433EF for ; Wed, 18 May 2022 15:11:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239278AbiERPLZ (ORCPT ); Wed, 18 May 2022 11:11:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41688 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239206AbiERPLS (ORCPT ); Wed, 18 May 2022 11:11:18 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41CFEB1C1; Wed, 18 May 2022 08:11:16 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D1699619A0; Wed, 18 May 2022 15:11:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AAE5AC34100; Wed, 18 May 2022 15:11:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1652886675; bh=8Xbp9VGRkRu9DADovZIgmzELCmC4Crhgf7Qr4utrpxE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qTuQJVBHPPrsnt2ojg1qp83eN3IzRVte7f6jq7jELH2AxitcvL3Vn7LBg564ilICO R1mWg4xYnIgLEPblkIXhXuoG0D7WtKavP9QOFkC+nD5LWYjSW0rzrMe4IwsAAN8SBk z34umzE8JHSsrCD8Cye/fRIsB4POxdiLFGIbBXFfGajEOjxnqTukRly/UyMZIc+52D 1Q0rlcJyYhc8+3jdDzLIUPNm0sMNQCaj5UlyfE6oTrizt9UiThNdquODUaxeH/D5Pk ByVwz7e1xwElB0nvCkESF+l3bFjirqEGN1kMCB/Eeqapxcfy3VqJfvlXtUigh2OgIQ YNyT6e1JnFEJw== From: Jeff Layton To: dhowells@redhat.com Cc: linux-fsdevel@vger.kernel.org, ceph-devel@vger.kernel.org, xiubli@redhat.com, idryomov@gmail.com Subject: [PATCH 2/4] ceph: Use the provided iterator in ceph_netfs_issue_op() Date: Wed, 18 May 2022 11:11:09 -0400 Message-Id: <20220518151111.79735-3-jlayton@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220518151111.79735-1-jlayton@kernel.org> References: <20220518151111.79735-1-jlayton@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org From: David Howells The netfs_read_subrequest struct now contains a persistent iterator representing the destination buffer for a read that the network filesystem should use. Make ceph use this. Signed-off-by: David Howells Signed-off-by: Jeff Layton --- fs/ceph/addr.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c index 8e6a931f3a0f..d14a9378d120 100644 --- a/fs/ceph/addr.c +++ b/fs/ceph/addr.c @@ -233,7 +233,6 @@ static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq) struct ceph_mds_request *req; struct ceph_mds_client *mdsc = ceph_sb_to_mdsc(inode->i_sb); struct ceph_inode_info *ci = ceph_inode(inode); - struct iov_iter iter; ssize_t err = 0; size_t len; int mode; @@ -268,8 +267,7 @@ static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq) } len = min_t(size_t, iinfo->inline_len - subreq->start, subreq->len); - iov_iter_xarray(&iter, READ, &rreq->mapping->i_pages, subreq->start, len); - err = copy_to_iter(iinfo->inline_data + subreq->start, len, &iter); + err = copy_to_iter(iinfo->inline_data + subreq->start, len, &subreq->iter); if (err == 0) err = -EFAULT; @@ -287,7 +285,6 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq) struct ceph_fs_client *fsc = ceph_inode_to_client(inode); struct ceph_osd_request *req; struct ceph_vino vino = ceph_vino(inode); - struct iov_iter iter; struct page **pages; size_t page_off; int err = 0; @@ -308,15 +305,14 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq) } dout("%s: pos=%llu orig_len=%zu len=%llu\n", __func__, subreq->start, subreq->len, len); - iov_iter_xarray(&iter, READ, &rreq->mapping->i_pages, subreq->start, len); - err = iov_iter_get_pages_alloc(&iter, &pages, len, &page_off); + + err = iov_iter_get_pages_alloc(&subreq->iter, &pages, len, &page_off); if (err < 0) { dout("%s: iov_ter_get_pages_alloc returned %d\n", __func__, err); goto out; } - /* should always give us a page-aligned read */ - WARN_ON_ONCE(page_off); + /* FIXME: adjust the len in req downward if necessary */ len = err; osd_req_op_extent_osd_data_pages(req, 0, pages, len, 0, false, false); From patchwork Wed May 18 15:11:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 575445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9985C433F5 for ; Wed, 18 May 2022 15:11:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239294AbiERPL1 (ORCPT ); Wed, 18 May 2022 11:11:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239209AbiERPLS (ORCPT ); Wed, 18 May 2022 11:11:18 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 031EE26C9; Wed, 18 May 2022 08:11:17 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 935C26198B; Wed, 18 May 2022 15:11:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 771A2C385A5; Wed, 18 May 2022 15:11:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1652886676; bh=dkf7GEao8AI86nB58euIIl4y2hwL/HMgAhrL7Jqi2PU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=koja2v5EfPv5p8uYwa2nV3BWflBJjkS3mdnxieoyw3NN3tW3uy/W/wmasPSj3G30s ajmU8C/CZamVATA9MmWousMT9I3DBTagpsTaq3Qi8LENMhxOT+me4G3pxDOlpTH5SQ w4GbT+Yu2yoo124FKiTK9laWV1sqrgXhpaapV8lpCwx1NpwGySKF/PHvAhU0Jq7fPC RxOxKp1po61jGXUNGkCYXrEPRihckbDfxs/XnV1wmnkrOKWKozQFnr7IVWc8ttrLiJ Aew1xHDRpq5908e46d3N1yyxyaaVVXGftrU73bTQhaaZdfraSiF34+vrkLDqIfGZj1 6QIHVolLCZp+g== From: Jeff Layton To: dhowells@redhat.com Cc: linux-fsdevel@vger.kernel.org, ceph-devel@vger.kernel.org, xiubli@redhat.com, idryomov@gmail.com Subject: [PATCH 3/4] ceph: enhance dout messages in issue_read codepaths Date: Wed, 18 May 2022 11:11:10 -0400 Message-Id: <20220518151111.79735-4-jlayton@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220518151111.79735-1-jlayton@kernel.org> References: <20220518151111.79735-1-jlayton@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org Signed-off-by: Jeff Layton --- fs/ceph/addr.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c index d14a9378d120..475df4efd2c7 100644 --- a/fs/ceph/addr.c +++ b/fs/ceph/addr.c @@ -190,6 +190,8 @@ static bool ceph_netfs_clamp_length(struct netfs_io_subrequest *subreq) /* Truncate the extent at the end of the current block */ ceph_calc_file_object_mapping(&ci->i_layout, subreq->start, subreq->len, &objno, &objoff, &xlen); + dout("%s: subreq->len=0x%zx xlen=0x%x rsize=0x%x", + __func__, subreq->len, xlen, fsc->mount_options->rsize); subreq->len = min(xlen, fsc->mount_options->rsize); return true; } @@ -304,7 +306,9 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq) goto out; } - dout("%s: pos=%llu orig_len=%zu len=%llu\n", __func__, subreq->start, subreq->len, len); + dout("%s: pos=%llu orig_len=%zu len=%llu debug_id=%x debug_idx=%hx iter->count=%zx\n", + __func__, subreq->start, subreq->len, len, rreq->debug_id, + subreq->debug_index, iov_iter_count(&subreq->iter)); err = iov_iter_get_pages_alloc(&subreq->iter, &pages, len, &page_off); if (err < 0) { From patchwork Wed May 18 15:11:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 575444 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 441C3C433F5 for ; Wed, 18 May 2022 15:11:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239320AbiERPLi (ORCPT ); Wed, 18 May 2022 11:11:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239230AbiERPLV (ORCPT ); Wed, 18 May 2022 11:11:21 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64F05B875; Wed, 18 May 2022 08:11:19 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 10318B81BD9; Wed, 18 May 2022 15:11:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 439A5C385A9; Wed, 18 May 2022 15:11:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1652886676; bh=9W71JbdZqORON3ZtMJgsQNIwd/tY1dgG9++fufgZKwA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qpS827J0FpXk4qMRKfP+Hq5MO//kSLfG/2gJaudQpOvEBGjoeUR4ao0EJWYcvFdXS kqjNEYG/zzwGx7BsV2TYGmAvdOwkw5Ike7tK6BdbJRrLxPhtr3B9aEZ+FONWrvSmij kj64goLQtUrRivIM4iuEfo8q1IagOIYR19gukMfaWmGgkkn8Dz/O4TkkQvbIKxrfJp beMzi4EZLz4/jGALPVUVLx/AIYoWTBPzVbSNSmeQzzst8TNmcNX7y5oaWJZ/HMQ2jV 1F7kGvwLQzjzaco50YOKEZX5MH2/YVYvwVReOLjq4sFnR8X+tbR+7j5ABBpF3D5i66 WQxbpkd6lmLRg== From: Jeff Layton To: dhowells@redhat.com Cc: linux-fsdevel@vger.kernel.org, ceph-devel@vger.kernel.org, xiubli@redhat.com, idryomov@gmail.com Subject: [PATCH 4/4] ceph: switch to netfs_direct_read_iter Date: Wed, 18 May 2022 11:11:11 -0400 Message-Id: <20220518151111.79735-5-jlayton@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220518151111.79735-1-jlayton@kernel.org> References: <20220518151111.79735-1-jlayton@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org Signed-off-by: Jeff Layton --- fs/ceph/addr.c | 41 +++++++++++++++++++++++++++++------------ fs/ceph/file.c | 3 +-- 2 files changed, 30 insertions(+), 14 deletions(-) diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c index 475df4efd2c7..938679a7a1e3 100644 --- a/fs/ceph/addr.c +++ b/fs/ceph/addr.c @@ -201,7 +201,6 @@ static void finish_netfs_read(struct ceph_osd_request *req) struct ceph_fs_client *fsc = ceph_inode_to_client(req->r_inode); struct ceph_osd_data *osd_data = osd_req_op_extent_osd_data(req, 0); struct netfs_io_subrequest *subreq = req->r_priv; - int num_pages; int err = req->r_result; ceph_update_read_metrics(&fsc->mdsc->metric, req->r_start_latency, @@ -216,13 +215,18 @@ static void finish_netfs_read(struct ceph_osd_request *req) else if (err == -EBLOCKLISTED) fsc->blocklisted = true; - if (err >= 0 && err < subreq->len) - __set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags); + if (err >= 0) { + if (err < subreq->len) + __set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags); + iov_iter_advance(&subreq->iter, err); + } + if (!iov_iter_is_bvec(&subreq->iter)) + ceph_put_page_vector(osd_data->pages, + calc_pages_for(osd_data->alignment, + osd_data->length), + false); netfs_subreq_terminated(subreq, err, true); - - num_pages = calc_pages_for(osd_data->alignment, osd_data->length); - ceph_put_page_vector(osd_data->pages, num_pages, false); iput(req->r_inode); } @@ -287,6 +291,7 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq) struct ceph_fs_client *fsc = ceph_inode_to_client(inode); struct ceph_osd_request *req; struct ceph_vino vino = ceph_vino(inode); + struct iov_iter *iter = &subreq->iter; struct page **pages; size_t page_off; int err = 0; @@ -310,16 +315,28 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq) __func__, subreq->start, subreq->len, len, rreq->debug_id, subreq->debug_index, iov_iter_count(&subreq->iter)); - err = iov_iter_get_pages_alloc(&subreq->iter, &pages, len, &page_off); - if (err < 0) { - dout("%s: iov_ter_get_pages_alloc returned %d\n", __func__, err); - goto out; + if (iov_iter_is_bvec(iter)) { + /* + * FIXME: remove force cast, ideally by plumbing an IOV_ITER osd_data + * variant. + */ + osd_req_op_extent_osd_data_bvecs(req, 0, (__force struct bio_vec *)iter->bvec, + iter->nr_segs, len); + goto submit; } - /* FIXME: adjust the len in req downward if necessary */ - len = err; + err = iov_iter_get_pages_alloc(&subreq->iter, &pages, len, &page_off); + if (err < len) { + if (err < 0) { + dout("%s: iov_ter_get_pages_alloc returned %d\n", __func__, err); + goto out; + } + len = err; + req->r_ops[0].extent.length = err; + } osd_req_op_extent_osd_data_pages(req, 0, pages, len, 0, false, false); +submit: req->r_callback = finish_netfs_read; req->r_priv = subreq; req->r_inode = inode; diff --git a/fs/ceph/file.c b/fs/ceph/file.c index 8c8226c0feac..81ce6753fa67 100644 --- a/fs/ceph/file.c +++ b/fs/ceph/file.c @@ -1634,8 +1634,7 @@ static ssize_t ceph_read_iter(struct kiocb *iocb, struct iov_iter *to) if (ci->i_inline_version == CEPH_INLINE_NONE) { if (!retry_op && (iocb->ki_flags & IOCB_DIRECT)) { - ret = ceph_direct_read_write(iocb, to, - NULL, NULL); + ret = netfs_direct_read_iter(iocb, to); if (ret >= 0 && ret < len) retry_op = CHECK_EOF; } else {