diff mbox series

ceph: try to choose the auth MDS if possible for getattr

Message ID 20220421042028.92787-1-xiubli@redhat.com
State New
Headers show
Series ceph: try to choose the auth MDS if possible for getattr | expand

Commit Message

Xiubo Li April 21, 2022, 4:20 a.m. UTC
If any 'x' caps is issued we can just choose the auth MDS instead
of the random replica MDSes. Because only when the Locker is in
LOCK_EXEC state will the loner client could get the 'x' caps. And
if we send the getattr requests to any replica MDS it must auth pin
and tries to rdlock from the auth MDS, and then the auth MDS need
to do the Locker state transition to LOCK_SYNC. And after that the
lock state will change back.

This cost much when doing the Locker state transition and usually
will need to revoke caps from clients.

URL: https://tracker.ceph.com/issues/55240
Signed-off-by: Xiubo Li <xiubli@redhat.com>
---
 fs/ceph/inode.c | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

Comments

Xiubo Li April 21, 2022, 7:04 a.m. UTC | #1
Will send a V2 to fix the same issue in ceph_netfs_issue_op_inline().

-- Xiubo


On 4/21/22 12:20 PM, Xiubo Li wrote:
> If any 'x' caps is issued we can just choose the auth MDS instead
> of the random replica MDSes. Because only when the Locker is in
> LOCK_EXEC state will the loner client could get the 'x' caps. And
> if we send the getattr requests to any replica MDS it must auth pin
> and tries to rdlock from the auth MDS, and then the auth MDS need
> to do the Locker state transition to LOCK_SYNC. And after that the
> lock state will change back.
>
> This cost much when doing the Locker state transition and usually
> will need to revoke caps from clients.
>
> URL: https://tracker.ceph.com/issues/55240
> Signed-off-by: Xiubo Li <xiubli@redhat.com>
> ---
>   fs/ceph/inode.c | 20 +++++++++++++++++++-
>   1 file changed, 19 insertions(+), 1 deletion(-)
>
> diff --git a/fs/ceph/inode.c b/fs/ceph/inode.c
> index b45f321910af..2a5023b1272d 100644
> --- a/fs/ceph/inode.c
> +++ b/fs/ceph/inode.c
> @@ -2270,6 +2270,7 @@ int __ceph_do_getattr(struct inode *inode, struct page *locked_page,
>   	struct ceph_fs_client *fsc = ceph_sb_to_client(inode->i_sb);
>   	struct ceph_mds_client *mdsc = fsc->mdsc;
>   	struct ceph_mds_request *req;
> +	int issued = ceph_caps_issued(ceph_inode(inode));
>   	int mode;
>   	int err;
>   
> @@ -2283,7 +2284,24 @@ int __ceph_do_getattr(struct inode *inode, struct page *locked_page,
>   	if (!force && ceph_caps_issued_mask_metric(ceph_inode(inode), mask, 1))
>   			return 0;
>   
> -	mode = (mask & CEPH_STAT_RSTAT) ? USE_AUTH_MDS : USE_ANY_MDS;
> +	/*
> +	 * If any 'x' caps is issued we can just choose the auth MDS
> +	 * instead of the random replica MDSes. Because only when the
> +	 * Locker is in LOCK_EXEC state will the exclusive client could
> +	 * get the 'x' caps. And if we send the getattr requests to any
> +	 * replica MDS it must auth pin and tries to rdlock from the auth
> +	 * MDS, and then the auth MDS need to do the Locker state transition
> +	 * to LOCK_SYNC. And after that the lock state will change back.
> +	 *
> +	 * This cost much when doing the Locker state transition and
> +	 * usually will need to revoke caps from clients.
> +	 */
> +	if (((mask & CEPH_CAP_ANY_SHARED) && (issued & CEPH_CAP_ANY_EXCL))
> +	    || (mask & CEPH_STAT_RSTAT))
> +		mode = USE_AUTH_MDS;
> +	else
> +		mode = USE_ANY_MDS;
> +
>   	req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_GETATTR, mode);
>   	if (IS_ERR(req))
>   		return PTR_ERR(req);
diff mbox series

Patch

diff --git a/fs/ceph/inode.c b/fs/ceph/inode.c
index b45f321910af..2a5023b1272d 100644
--- a/fs/ceph/inode.c
+++ b/fs/ceph/inode.c
@@ -2270,6 +2270,7 @@  int __ceph_do_getattr(struct inode *inode, struct page *locked_page,
 	struct ceph_fs_client *fsc = ceph_sb_to_client(inode->i_sb);
 	struct ceph_mds_client *mdsc = fsc->mdsc;
 	struct ceph_mds_request *req;
+	int issued = ceph_caps_issued(ceph_inode(inode));
 	int mode;
 	int err;
 
@@ -2283,7 +2284,24 @@  int __ceph_do_getattr(struct inode *inode, struct page *locked_page,
 	if (!force && ceph_caps_issued_mask_metric(ceph_inode(inode), mask, 1))
 			return 0;
 
-	mode = (mask & CEPH_STAT_RSTAT) ? USE_AUTH_MDS : USE_ANY_MDS;
+	/*
+	 * If any 'x' caps is issued we can just choose the auth MDS
+	 * instead of the random replica MDSes. Because only when the
+	 * Locker is in LOCK_EXEC state will the exclusive client could
+	 * get the 'x' caps. And if we send the getattr requests to any
+	 * replica MDS it must auth pin and tries to rdlock from the auth
+	 * MDS, and then the auth MDS need to do the Locker state transition
+	 * to LOCK_SYNC. And after that the lock state will change back.
+	 *
+	 * This cost much when doing the Locker state transition and
+	 * usually will need to revoke caps from clients.
+	 */
+	if (((mask & CEPH_CAP_ANY_SHARED) && (issued & CEPH_CAP_ANY_EXCL))
+	    || (mask & CEPH_STAT_RSTAT))
+		mode = USE_AUTH_MDS;
+	else
+		mode = USE_ANY_MDS;
+
 	req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_GETATTR, mode);
 	if (IS_ERR(req))
 		return PTR_ERR(req);