From patchwork Mon Jun 13 11:31:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Luis Henriques X-Patchwork-Id: 581430 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6018C433EF for ; Mon, 13 Jun 2022 13:41:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378335AbiFMNlF (ORCPT ); Mon, 13 Jun 2022 09:41:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59054 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1379262AbiFMNkJ (ORCPT ); Mon, 13 Jun 2022 09:40:09 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1143EDEC9; Mon, 13 Jun 2022 04:31:02 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id C3FD51FA74; Mon, 13 Jun 2022 11:31:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1655119860; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5ug4pTmLE6Ljo/diY5BUB/450JYo9SA6nq2eET7/wAU=; b=F2MOaYco0NXpy1cS2EbcaUqibjAgxytQIGAK3sEgdQYJ2DU6pwSsYLDPsy+XUoAGVxxM+j sTEjgcmCDVwyg02MzezLa73bPJY1F8jDNUbeYc30D+6MCtzw/Rdewj+aHqQO16HtuyjtGN HZUqC3eyikQkg9GEvn0D1qyAAOBXno4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1655119860; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5ug4pTmLE6Ljo/diY5BUB/450JYo9SA6nq2eET7/wAU=; b=q9YTR1K6ixcNd6Xo7UwAfmN9QY/HdW6UxmqWmdhMGlRDw1W+Gq9OMN1/nJzfFuvNVz3z+x C4h8g/OoKmgXAgBg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 37116134CF; Mon, 13 Jun 2022 11:31:00 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id WBTUCvQfp2ICeQAAMHmgww (envelope-from ); Mon, 13 Jun 2022 11:31:00 +0000 Received: from localhost (brahms.olymp [local]) by brahms.olymp (OpenSMTPD) with ESMTPA id 752b8581; Mon, 13 Jun 2022 11:31:43 +0000 (UTC) From: =?utf-8?q?Lu=C3=ADs_Henriques?= To: fstests@vger.kernel.org Cc: David Disseldorp , Zorro Lang , Dave Chinner , "Darrick J. Wong" , Jeff Layton , Xiubo Li , ceph-devel@vger.kernel.org, =?utf-8?q?Lu=C3=ADs_Henriques?= Subject: [PATCH v3 1/2] generic/020: adjust max_attrval_size for ceph Date: Mon, 13 Jun 2022 12:31:41 +0100 Message-Id: <20220613113142.4338-2-lhenriques@suse.de> In-Reply-To: <20220613113142.4338-1-lhenriques@suse.de> References: <20220613113142.4338-1-lhenriques@suse.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org CephFS doesn't have a maximum xattr size. Instead, it imposes a maximum size for the full set of xattrs names+values, which by default is 64K. This patch fixes the max_attrval_size for ceph so that it is takes into account any already existing attrs in the file. Signed-off-by: Luís Henriques Reviewed-by: David Disseldorp --- tests/generic/020 | 33 +++++++++++++++++++++++++-------- 1 file changed, 25 insertions(+), 8 deletions(-) diff --git a/tests/generic/020 b/tests/generic/020 index d8648e96286e..b91bca34dcd4 100755 --- a/tests/generic/020 +++ b/tests/generic/020 @@ -51,13 +51,9 @@ _attr_list() fi } -# set fs-specific max_attrs and max_attrval_size values. The parameter -# @max_attrval_namelen is required for filesystems which take into account attr -# name lengths (including namespace prefix) when determining limits. +# set fs-specific max_attrs _attr_get_max() { - local max_attrval_namelen="$1" - # set maximum total attr space based on fs type case "$FSTYP" in xfs|udf|pvfs2|9p|ceph|nfs) @@ -112,6 +108,16 @@ _attr_get_max() # overhead let max_attrs=$BLOCK_SIZE/40 esac +} + +# set fs-specific max_attrval_size values. The parameter @max_attrval_namelen is +# required for filesystems which take into account attr name lengths (including +# namespace prefix) when determining limits; parameter @filename is required for +# filesystems that need to take into account already existing attrs. +_attr_get_maxval_size() +{ + local max_attrval_namelen="$1" + local filename="$2" # Set max attr value size in bytes based on fs type case "$FSTYP" in @@ -128,7 +134,7 @@ _attr_get_max() pvfs2) max_attrval_size=8192 ;; - xfs|udf|9p|ceph) + xfs|udf|9p) max_attrval_size=65536 ;; bcachefs) @@ -139,6 +145,15 @@ _attr_get_max() # the underlying filesystem, so just use the lowest value above. max_attrval_size=1024 ;; + ceph) + # CephFS does not have a maximum value for attributes. Instead, + # it imposes a maximum size for the full set of xattrs + # names+values, which by default is 64K. Compute the maximum + # taking into account the already existing attributes + max_attrval_size=$(getfattr --dump -e hex $filename 2>/dev/null | \ + awk -F "=0x" '/^user/ {len += length($1) + length($2) / 2} END {print len}') + max_attrval_size=$((65536 - $max_attrval_size - $max_attrval_namelen)) + ;; *) # Assume max ~1 block of attrs BLOCK_SIZE=`_get_block_size $TEST_DIR` @@ -181,8 +196,7 @@ echo "*** remove attribute" _attr -r fish $testfile _attr_list $testfile -max_attrval_name="long_attr" # add 5 for "user." prefix -_attr_get_max "$(( 5 + ${#max_attrval_name} ))" +_attr_get_max echo "*** add lots of attributes" v=0 @@ -226,6 +240,9 @@ done _attr_list $testfile echo "*** really long value" +max_attrval_name="long_attr" # add 5 for "user." prefix +_attr_get_maxval_size "$(( 5 + ${#max_attrval_name} ))" "$testfile" + dd if=/dev/zero bs=1 count=$max_attrval_size 2>/dev/null \ | _attr -s "$max_attrval_name" $testfile >/dev/null From patchwork Mon Jun 13 11:31:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Luis Henriques X-Patchwork-Id: 583007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34DB1CCA47B for ; Mon, 13 Jun 2022 13:41:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378399AbiFMNlS (ORCPT ); Mon, 13 Jun 2022 09:41:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1379266AbiFMNkP (ORCPT ); Mon, 13 Jun 2022 09:40:15 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BAF49DF99; Mon, 13 Jun 2022 04:31:02 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 739EC21D97; Mon, 13 Jun 2022 11:31:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1655119861; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T/Ex511XuC26n5wfZYvopT0bcTC6AXzGxUIymSLcgms=; b=FQiqw9j9N44s/RaZVxh2N5XOl0AHJn/c84EiVZ+SylLAca4ImXEHoqSEbOfcDXTkBEhD7d 3w5c5a8NXEOVvyWwhOX4OOQRWbjkq7YIHrB3iSbd26RdR3PY0mVEin0gphnGg1pULU5kZT 8f2OFYMSXPUy9Ieoqa3MxOtjb4PW9GY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1655119861; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T/Ex511XuC26n5wfZYvopT0bcTC6AXzGxUIymSLcgms=; b=cWzveilIrWdo7cULXTFleIOEpe0hv5bK9iM4p7CDnqa17Hc2zeN+B+d7jO991dMUyvA79v 9dS6yEuAMyiQdWBQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D4543134CF; Mon, 13 Jun 2022 11:31:00 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id qP4iMfQfp2ICeQAAMHmgww (envelope-from ); Mon, 13 Jun 2022 11:31:00 +0000 Received: from localhost (brahms.olymp [local]) by brahms.olymp (OpenSMTPD) with ESMTPA id 96a7b3fd; Mon, 13 Jun 2022 11:31:43 +0000 (UTC) From: =?utf-8?q?Lu=C3=ADs_Henriques?= To: fstests@vger.kernel.org Cc: David Disseldorp , Zorro Lang , Dave Chinner , "Darrick J. Wong" , Jeff Layton , Xiubo Li , ceph-devel@vger.kernel.org, =?utf-8?q?Lu=C3=ADs_Henriques?= Subject: [PATCH v3 2/2] generic/486: adjust the max xattr size Date: Mon, 13 Jun 2022 12:31:42 +0100 Message-Id: <20220613113142.4338-3-lhenriques@suse.de> In-Reply-To: <20220613113142.4338-1-lhenriques@suse.de> References: <20220613113142.4338-1-lhenriques@suse.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org CephFS doesn't have a maximum xattr size. Instead, it imposes a maximum size for the full set of xattrs names+values, which by default is 64K. And since it reports 4M as the blocksize (the default ceph object size), generic/486 will fail in ceph because the XATTR_SIZE_MAX value can't be used in attr_replace_test. The fix is to add a new argument to the test so that the max size can be passed in instead of trying to auto-probe a value for it. Signed-off-by: Luís Henriques --- src/attr_replace_test.c | 30 ++++++++++++++++++++++++++---- tests/generic/486 | 11 ++++++++++- 2 files changed, 36 insertions(+), 5 deletions(-) diff --git a/src/attr_replace_test.c b/src/attr_replace_test.c index cca8dcf8ff60..1218e7264c8f 100644 --- a/src/attr_replace_test.c +++ b/src/attr_replace_test.c @@ -20,19 +20,41 @@ exit(1); } while (0) fprintf(stderr, __VA_ARGS__); exit (1); \ } while (0) +void usage(char *progname) +{ + fail("usage: %s [-m max_attr_size] \n", progname); +} + int main(int argc, char *argv[]) { int ret; int fd; + int c; char *path; char *name = "user.world"; char *value; struct stat sbuf; size_t size = sizeof(value); + size_t maxsize = XATTR_SIZE_MAX; + + while ((c = getopt(argc, argv, "m:")) != -1) { + char *endp; + + switch (c) { + case 'm': + maxsize = strtoul(optarg, &endp, 0); + if (*endp || (maxsize > XATTR_SIZE_MAX)) + fail("Invalid 'max_attr_size' value\n"); + break; + default: + usage(argv[0]); + } + } - if (argc != 2) - fail("Usage: %s \n", argv[0]); - path = argv[1]; + if (optind == argc - 1) + path = argv[optind]; + else + usage(argv[0]); fd = open(path, O_RDWR | O_CREAT, S_IRUSR | S_IWUSR); if (fd < 0) die(); @@ -46,7 +68,7 @@ int main(int argc, char *argv[]) size = sbuf.st_blksize * 3 / 4; if (!size) fail("Invalid st_blksize(%ld)\n", sbuf.st_blksize); - size = MIN(size, XATTR_SIZE_MAX); + size = MIN(size, maxsize); value = malloc(size); if (!value) fail("Failed to allocate memory\n"); diff --git a/tests/generic/486 b/tests/generic/486 index 7de198f93a71..7dbfcb9835d9 100755 --- a/tests/generic/486 +++ b/tests/generic/486 @@ -41,7 +41,16 @@ filter_attr_output() { sed -e 's/has a [0-9]* byte value/has a NNNN byte value/g' } -$here/src/attr_replace_test $SCRATCH_MNT/hello +max_attr_size=65536 + +# attr_replace_test can't easily auto-probe the attr size for ceph because: +# - ceph imposes a maximum value for the total xattr names+values, and +# - ceph reports the 'object size' in the block size, which is, by default, much +# larger than XATTR_SIZE_MAX (4M > 64k) +# Hence, we need to provide it with a maximum size. +[ "$FSTYP" = "ceph" ] && max_attr_size=65000 + +$here/src/attr_replace_test -m $max_attr_size $SCRATCH_MNT/hello $ATTR_PROG -l $SCRATCH_MNT/hello >>$seqres.full 2>&1 $ATTR_PROG -l $SCRATCH_MNT/hello | filter_attr_output